Wednesday, December 23, 2015

What a Tangled Web We Weave

One of the key debates in education is whether there is a causal relationship between education funding and student outcomes.  As I've discussed in previous posts, this is something that you can't really "prove" using statistics.  In this post, I want to expand on that by illustrating the issue of multicollinearity.

Multicollinearity, according to Wikipedia, is "a phenomenon in which two or more predictor variables in a multiple regression model are highly correlated, meaning that one can be linearly predicted from the others with a substantial degree of accuracy... the issue of multicollinearity arises when there is an approximate linear relationship among two or more independent variables."

So, aside from the limitations in the statistics themselves, which are only able to say whether two variables tend to move together (correlation) or whether movement in one can predict movement in another (regression), our ability to identify patterns is further complicated when the things we are using to predict are highly related to one another.

The following table shows the list of independent (predictor) variables we are currently working with, followed by a count of how many other independent variables each one is correlated with (statistically significant at .05 or less based on available data), along with which how many dependent variables and variables overall it is correlated.


The variables in the table are sorted based on how many of the dependent variables (student outcomes) with which each one is correlated.  As you can see, the spending variables rank pretty highly out of those which are at least in part under the control of the education system (highlighted in yellow).  

This means that education spending tends to move with student outcomes more consistently than some things like the percent of children in poverty.  (To see more details on the actual correlation coefficients, click here.)  

However, you will see that these funding variables are also significantly correlated with many of the other independent (predictor) variables.  For example, the table shows that The Current Spending Per Pupil RPP (state-cost-of-living adjusted) variable is significantly correlated with 15 of the independent variables.  They are:
  • Current spending per pupil RPP (of course)
  • Period (Year)
  • ACT percent of graduates tested*
  • Current spending per pupil
  • Median household income
  • Percent of 25 year olds and older with at least a:
    • High school diploma
    • Bachelor's degree
    • Graduate degree or higher
  • Percent of children in poverty*
  • Percent of public school students who are white
  • Percent of Students 3-21 served under IDEA
  • Percent of students eligible for free or reduced-price lunch*
  • Percent of students in English language learner programs*
  • Percent of the population under the poverty level within the past 12 months*
  • Population per square mile
  • SAT percent of graduates tested
  • Spending on instruction
  • Spending on instruction RPP
  • Student district ratio*
  • Student school ratio*
  • Student staff ratio*
  • Total revenue per pupil
  • Total revenue per pupil RPP
* Indicates a negative correlation, where an increase in one variable is accompanied by a decrease in the other. 

In fact, there were only six independent variables with which Current spending per pupil RPP was not correlated, and these were census population distribution variables that were only available for one year.  This means that, even though we call them "independent" variables, they are anything but.  We have to look at them in the context of all the other factors that impact student outcomes.  

Nonetheless, note that there is very little on the list that we can impact directly.  The only variables that we can directly change are those related to school funding, and those related to district, school, and classroom size.  KASB has been asserting that more funding is related to better outcomes.  Others have suggested that too much money is being spent on administration, and that larger districts are the answer.  The correlations support the former theory, but actually show that larger districts are associated with lower student outcomes.  

The KASB Research Department will continue to examine this data and provide analysis and perspectives to inform the ongoing debate.  

Thursday, December 17, 2015

New KPI poll shows Kansans have more confidence in local government than in the state

The following post presents research or analyses from outside KASB and is presented for information purposes.  KASB neither endorses nor refutes the conclusions or recommendations contained herein.
A new poll released by KPI this month shows that 25% of respondents agreed with the statement “Kansas state government operates pretty efficiently and makes effective use of my tax dollars,” whereas 45% agreed with the statement “Local government operates pretty efficiently and makes effective use of my tax dollars.”  


The poll contained 15 questions, many of which were aimed to see how much knowledge the respondents had about school funding, asking such questions as “How much funding per pupil do you think Kansas school districts currently receive from ALL taxpayer sources per year, including State, Federal and Local taxpayers?” and “Over the last 5 years, how much do you think total per-pupil funding has changed?”


Other questions were targeted very specifically, such as two that began with “Hypothetically speaking, if total taxpayer support of Kansas public schools were more than $13,000 per-pupil and school districts had also used more than 350 million of state and local taxes to increase cash reserves, how much would you agree or disagree with this statement…”  


Based on the premise above, the two statements given were “Funding for other state agencies should be reduced in order to give more money to local school districts” and “I would be willing to pay higher taxes in order to give more money to local school districts.”  Interestingly, 48% agreed with the first statement and 41% agreed with the second; compared to 39% and 50% that disagreed with each.  


Still others were based on data or premises that could be called into question, such as “Spending on out-of-the-classroom expenses - administration, building operations, transport, and food service - varies among Kansas school districts, up to as much as $8,000 per pupil” and “If an individual school district wants to spend more than is necessary to provide the same function or service, or add extras like retiree health care…”  

For more specific results from the poll, visit SurveyUSA’s site here.  

Thursday, December 3, 2015

Time to Regroup

This past summer, KASB revealed a method for identifying states to use for comparisons.  These included Peer States, Aspiration States, and Higher Impact States.  Since then, more recent data has become available, and KASB has had time to see how well our original calculations and comparisons have worked.  As a result, we have a) updated our original calculations based on the new data and b) modified the calculations and groups themselves based on experience.

Higher Impact States

The first notable change is that (for the time being at least) we are no longer using the "Higher Impact" comparisons.  The theory behind these was to look at states that have better student outcomes than would be expected based on their student demographics and population characteristics.  The problems with this grouping were:
  1. It was very difficult to explain to most audiences,
  2. It was difficult to calculate, making ongoing updates fairly time-consuming, and
  3. The states identified were for the most part those that had much lower student outcomes than Kansas and therefore it did not seem beneficial to look to them for ideas.  

Aspiration States

The second change relates to the Aspiration States.  The calculations for this group did not change, but there were updates to the following outcomes data:
  1. The percent of 18-24 year olds with at least a high school diploma
  2. Average freshman graduation rate
  3. NAEP
  4. ACT
  5. SAT
Based on these updates, the list of Aspiration States went from this:
  • New Hampshire
  • New Jersey
  • Massachusetts
  • Vermont
  • Minnesota
To this:
  • New Hampshire
  • New Jersey
  • Massachusetts
  • Vermont
  • Indiana
  • Iowa
  • Nebraska
So, Minnesota fell off the list, and Indiana, Iowa, and Nebraska were added.  This means that overall Kansas went from having five states with better student outcomes on at least 8 of the 14 measures used to having seven states with better student outcomes on a majority of these measures.

Peer States

The peer states have changed both in terms of updated data and modified calculations.  Originally the list was based on those states that were within half a standard deviation +/- of Kansas' value on a majority of the 10 student demographic and population characteristic variables used.  Of these, the following data was updated:
  1. Percent of Children in Poverty
  2. Percent of students who are white
  3. Adults (25 and up) with at least a high school diploma, bachelor's degree, or graduate degree
In addition, we identified seven new variables to use:
  1. Population per square mile in 
    • urbanized areas (50,000 or more people), 
    • urban clusters (at least 2,500 and less than 50,000 people), and 
    • urban areas (urbanized areas + urban clusters)
  2. Percent of the population in
    • urbanized areas
    • urban clusters
    • urban areas
  3. Percent of the population below poverty in the past 12 months
Finally, based on the addition of new variables, KASB decide to split the Peer States group into four types of peers:
  1. Student Peers - states with values within +/- 1/2 standard deviation of Kansas on at least 3/5 student demographic variables:
    • Percent of children at 100% poverty
    • Percent of students eligible for free or reduced-price lunch (at-risk)
    • Percent of students served under IDEA (special education)
    • Percent of students participating in English Language Learners program (bilingual)
    • Percent of student who are white
  2. Adult Peers - states with values within +/- 1/2 standard deviation of Kansas on at least 3/5 adult  demographic variables:
    • Median household income
    • Percent of 25 year olds and older with at least a:
      • High school diploma
      • Bachelor's degree
      • Graduate degree
    • Percent of the population with income below the poverty level in the past 12 months
  3. Distribution Peers - states with values within +/- 1/2 standard deviation of Kansas on at least 4/7 population distribution variables:
    • Population per square mile
    • Population per square mile in urbanized areas, urban clusters, and urban areas
    • Percent of the population in urbanized areas, urban clusters, and urban areas
  4. Overall Peers - states with values within +/- 1/2 standard deviation of Kansas on at least 9/17 of the Student, Adult, and Distribution variables.
The original list of Peer States was:
  • Oregon
  • Washington
  • Michigan
  • Nebraska
  • Pennsylvania
  • Wisconsin
  • Illinois

Overall Peers

The new list of Overall Peers is very similar; but Illinois was replaced by five other states:
  • Oregon
  • Washington
  • Michigan
  • Nebraska
  • Pennsylvania
  • Wisconsin
  • Alaska
  • Idaho
  • Iowa
  • Missouri
  • South Dakota

Student Peers

The Student Peers, or states with student populations very similar to Kansas, are:
  • Illinois
  • Michigan
  • Missouri
  • Rhode Island
  • Washington
  • Arkansas
  • Oregon
  • Virginia 
  • Wisconsin

Adult Peers

The Adult Peers, or states with adult populations very similar to Kansas, are:
  • Alaska
  • Illinois
  • Iowa
  • Michigan
  • Missouri
  • Nebraska
  • Oregon
  • Pennsylvania
  • South Dakota
  • Utah
  • Washington
  • Vermont
  • Wisconsin

Distribution Peers

The Distribution Peers, or states with similar urban/rural population distributions to Kansas, are:
  • Alaska
  • Idaho
  • Indiana
  • Iowa
  • Minnesota
  • New Mexico
  • North Dakota
  • Oklahoma
  • Missouri
  • South Dakota
  • Wisconsin
These different peer groups should allow us to look at the question "what do states similar to Kansas do?" in different ways.
Below is a map KASB prepared showing the new state groups, along with the difference between what each spends (in terms of total revenue per pupil) and what Kansas spends on education to share with the legislature and other interested parties.  Starting next week you will be seeing new information from KASB organized around these new state groupings.  Stay tuned!