Monday, September 29, 2014

Poverty in Kansas - How Are We Doing?

A colleague of mine shared this report from Oklahoma related to poverty and household income, and asked me if there was anything similar available for Kansas.  I told her that aside from the report KASB did recently about at-risk funding and poverty, there wasn't much to be found out there, so I decided to do some brief analysis myself.

Based on data obtained from the US Census Bureau, I found that, thought the percent of the Kansas population at or below the poverty level is fairly consistent over time, the percent of students enrolled in grades K-12 is increasing, as shown in the following graph:


This means that the poverty rate for the students in our public schools is higher than the overall poverty rate.  Between 2008 and 2010, and again from 2011 to 2012, the student poverty rate increased at a higher rate than did the overall rate, and also decreased at a higher rate from 2010 to 2011 then again from 2012 to 2013.  Breaking this out further, here is how the numbers look by grade ranges:


It would appear the largest growth in the percent of the school population below poverty would be in the 1st through 4th grade group, and that the largest increase in poverty in recent years happened between the 2008 and 2009 calendar years.  

So, the number of school-aged children in poverty has been increasing.  What about family income in general?  How are families in Kansas doing?  Again going to the Census Bureau, I looked at Household income for families, and found the following:



  That doesn't look too bad, right?  Family incomes overall have been seeing their median income increasing since 2009 or so, with the possible exception of female householders with no husband present; which have remained fairly level aside from a bit of a dip in 2009-2010 and male householders with no wife present; which have seen a less consistent pattern but remain level overall.

However, when looking at dollar amounts over time, it is always important to consider inflation.  Looking at these same amounts reported in 2013 inflation-adjusted dollars, we see this:



Based on this data, inflation-adjusted median family incomes decreased in Kansas between 2007 and 2012, then started to show some increases between 2012 and 2013.

There are a lot of discussions and debates going on right now about the overall prosperity of Kansans and more specifically in the education world about the future for Kansas students.  This data suggests the road ahead will be more difficult rather than easier, and indicates we need to work harder to find good solutions in the light of more students coming to our schools who don't have the financial stability that their predecessors had.

KASB will work on further analysis to look at these trends over longer periods of time and also to look at factors such as parent level of education, urbanicity, and various demographic characteristics and their impact on poverty and income.


Tuesday, September 2, 2014

2014 ACT results and the impact of spending

Note:  This is a longer version of an article originally appearing here.


2014 Results


ACT released its annual “Condition of College and Career Readiness” report last week, providing information on average scores and percent meeting benchmarks for college and career readiness in Reading, Mathematics, English, Science, and across all four measures.  For Kansas, the percentage of students meeting all college ready benchmarks increased for the third straight year.


The following table shows the percent of Kansas students meeting all four benchmarks as compared to the national percent.  Starting in 2006, Kansas students performed notably above the national average.  




It is important to note there is a statistically significant negative relationship between the percent of high school graduates taking the ACT exam and that state’s percent of students meeting all four benchmarks. (Statistically speaking, linear regression with percent of graduating seniors participating as independent and percent of students meeting all four benchmarks yielded a significance of p < .001.  Correlation between the two variables is -.58.)  This means that as the percent of students taking the exam increases, the percent of students meeting the benchmarks decreases. In 2014, 75 percent of high school graduates in Kansas took the ACT, compared to 57 percent nationally. Therefore, based on the relationship described, the Kansas percents would be expected to be lower than those for the U.S.


In order to control for percent participation, KASB looked at the actual observed percent meeting all four benchmarks compared to the expected percent, as shown in the following graph:




This indicates that when the percent participating is controlled for, Kansas outperforms the national average, and in fact Kansas has consistently exceeded expected results based on participation. Up until 2013, the national average lagged behind expected results based on participation.


KASB also looked at performance of Kansas and peer states identified as described here, showing the following results with no adjustments for percent participation:




This data suggests that Kansas is continuing to see an upward trend similar to its peers, and that the state falls approximately in the middle of the states in terms of performance.  


When adjusted for percent participation, the graph looks like this:




According to this data, Kansas, Minnesota, Nebraska, and Iowa all performed better than predicted based on percent participation across all years; whereas all the remaining peer states performed under the predicted amount across part of or all years.  Kansas continues to fall approximately in the middle of its peer states in this analysis.  


These trends are similar when considering the four College Ready Benchmarks (English, Math, Reading and Science) individually; both with and without adjustment for percent participation.  However, the trends for individual subjects show more clearly the impact of the re-calculation of the benchmark score values that happened in 2013; the first time they had been changed since they were first established in 2005.  


Kansas, again consistently above the national average, shows a decline in percent meeting the benchmark for English, but not as noticeable a decline as that seen nationally; particularly between 2012 and 2013.  This decline decreases in magnitude when adjusting for percent participation.




Reading increased slightly through 2012 for Kansas and the nation, with Kansas above the national average.  In 2013 Kansas percents declined, but to a lesser degree than nationally:




The percent meeting the mathematics benchmark increased dramatically for both Kansas and the nation from 2006 to 2007, and then showed a sharp decline between 2012 and 2013.  Kansas mirrors the national trend at a higher percent across the period.  




In science, again Kansas is well above the national average, but both show sharp increases from 2012 to 2013.  




In summary, it appears that, both with and without adjustment for the influence of percent participation on overall results, Kansas follows similar trends to those seen for the country as a whole in the ACT performance data in terms of percent of students meeting the benchmarks in each subject and overall, and that the percent of students meeting all four benchmarks in Kansas is consistently higher than the percent of students nationwide. State and national data shows a very slight increase in overall performance across subjects from 2013 to 2014, while at the same time showing virtually no change for Reading and English, noticeable decreases in Mathematics, and fairly dramatic increases in Science.  


Spending and Performance - Total Revenue, Percent Spent on Instruction, and the ACT Scores


Linear regressions run using Total revenue or spending on instruction as a percent of total revenue to predict ACT scores and the differences between actual and predicted ACT scores show that funding is, in almost all cases, a statistically significant predictor of student performance, as shown in the following table:


Independent
Dependent
Correlation
P
Revenue Per Pupil
Met All 4 Benchmarks
0.58
0.000
Revenue Per Pupil
Met English Benchmark
0.44
0.000
Revenue Per Pupil
Met Reading Benchmark
0.47
0.000
Revenue Per Pupil
Met Math Benchmark
0.51
0.000
Revenue Per Pupil
Met Science Benchmark
0.55
0.000
Revenue Per Pupil
Difference All 4 Benchmarks
0.41
0.000
Revenue Per Pupil
Difference English Benchmark
0.26
0.000
Revenue Per Pupil
Difference Reading Benchmark
0.30
0.000
Revenue Per Pupil
Difference Math Benchmark
0.29
0.000
Revenue Per Pupil
Difference Science Benchmark
0.40
0.000
Instruction as a % of Total Revenue
Met All 4 Benchmarks
0.21
0.000
Instruction as a % of Total Revenue
Met English Benchmark
0.25
0.000
Instruction as a % of Total Revenue
Met Reading Benchmark
0.15
0.034
Instruction as a % of Total Revenue
Met Math Benchmark
0.19
0.002
Instruction as a % of Total Revenue
Met Science Benchmark
0.23
0.000
Instruction as a % of Total Revenue
Difference All 4 Benchmarks
0.23
0.000
Instruction as a % of Total Revenue
Difference English Benchmark
0.27
0.000
Instruction as a % of Total Revenue
Difference Reading Benchmark
0.16
0.009
Instruction as a % of Total Revenue
Difference Math Benchmark
0.20
0.001
Instruction as a % of Total Revenue
Difference Science Benchmark
0.24
0.000


The column labeled “P” contains the measure of statistical significance; indicating the percent chance that the relationship between the two variables is due to chance.  In general, any p value of .05 or less is considered statistically significant.  Therefore all of the comparisons in the table above can be interpreted to indicate that both total revenue per pupil and the percent of total revenue going directly to instruction are significant predictors of the percent of students meeting the ACT benchmarks; with an increase in funding or percent to instruction predictive of an increase in percent meeting the benchmarks.


The correlations column gives an indication of the strength of the relationship between the variables.  In general the table shows that the total amount of revenue per pupil is more closely related to ACT benchmark attainment than is the percent of total revenue going to instruction.  


The following scatterplot shows the 2012 data for the states in terms of total revenue per pupil and the percent meeting all four benchmarks on the ACT:




Though there is a general upward trend; with ACT scores increasing as the amount of total revenue per pupil increases, there is wide variation around the line; suggesting a strong influence of other factors.  The correlation between these variables is .58.  Kansas, represented by the maroon square, performs slightly above what would be expected based on total revenue per pupil.  Some peer states, such as Minnesota and Idaho, performed noticeably higher than expected based on total revenue, whereas Arizona performed much lower than would be expected.  


The following scatterplot shows the 2012 data for the states in terms of spending on instruction as a percent of total revenue per pupil and the percent meeting all four benchmarks on the ACT:




The scatter of the states is much wider when looking at percent spent on instruction.  The correlation between these variables is .21.  Kansas again does better than would be predicted, but more of the peer states do worse than would be predicted.  


The following scatterplot shows the 2012 data for the states in terms of total revenue per pupil and the observed difference between expected performance based on participation for all four benchmarks on the ACT:




The correlation between these variables is .41; slightly below the correlation between total revenue and the actual percent of students meeting all four ACT benchmarks.  Kansas again exceeds expectations based on total revenue per pupil, as do the majority of the peer states.


The following scatterplot shows the 2012 data for the states in terms of instruction as a percent of total revenue per pupil and the observed difference between expected performance based on participation for all four benchmarks on the ACT:


Though the regression analysis showed the percent of total revenue spent on instruction to be a significant predictor of the difference observed on the percent of students meeting all 4 ACT benchmarks, this scatterplot shows that, with a correlation of .23, there are obviously other things impacting student performance.  Nonetheless as with all three other comparisons, Kansas does better than would be predicted.  There are more peer states below the predicted performance for this measure.  

In summary, as with most statistics, there are several ways to compare funding and ACT performance.  The ones above are just a few possible ways.  Nonetheless, across approaches the data consistently suggests that a) there is a relationship between funding and student outcomes, and b) Kansas consistently performs above what would be expected based on this relationship.  To put it another way, this data again says that money matters, and Kansas manages to get “more bang for the buck” than other states.