Wednesday, July 22, 2015

Knowing Your Peers Revisited


Last July, we attempted to identify “peer” states for Kansas, as described in this blog post.  We used four variables, Z scores, and standard deviations in order to try and determine which states are most like Kansas. Here is the resulting list of states:


  • Arizona
  • Arkansas
  • Idaho
  • Iowa
  • Minnesota
  • Nebraska
  • Oklahoma
  • Oregon


The four variables we used were:


  • Population per Square Mile
  • Percent of Students Eligible for Free or Reduced-Price Lunch
  • Percent of Students Participating in Programs for English language Learners
  • Percent of Students Served under IDEA (Special Education)


We selected these variables because they are commonly cited as factors influencing the nature of education at the state level.


Since July, we have spent a lot of time with this state-level data, and even undertook a large multi-year multiple regression analysis trying to determine what factors, in what combinations, have the largest impact on student outcomes. The results of the analysis suggested a much larger list of variables had a noticeable impact on student outcomes than the ones we had used to identify peer states.


In January, KASB released our first “Kansas Educational Achievement Report Card,” in which we discuss Kansas’ ranking on a variety of funding, outcome, and demographic variables to determine how our state compares with others.  


Based on the work we’ve done since last July, we decided it was time to revisit the notion of peer states and see if we can better identify those states that are similar to us. Taking the multiple regression analysis and the report card data, we identified 34 measures to be included.  For each of these variables, we pulled the most recent year’s data.  From here we could have calculated some form of overall rank or some kind of average of ranks to attempt to identify states similar to Kansas.  


But when we say we want to compare ourselves to states similar to ourselves, what are we really going for?  And further, is it reasonable to expect that we would want to compare ourselves to the same states each time regardless of the kinds of questions we are asking?  Thinking about this, and looking at how we are going to organize the next version of the Kansas Education Achievement Report Card, we put the 34 variables into six categories:


  • Student Attainment
    • Freshman Graduation Rate
    • Cohort Graduation Rate (4 measures)
      • All Students
      • Economically Disadvantaged Students
      • Limited English Proficiency Students
      • Students with Disabilities
    • Percent of 18-25 year olds with a high school diploma
  • Student Achievement
    • Percent performing at or above “Basic” on the NAEP assessment (3 measures)
      • All Students
      • Students Eligible for the National School Lunch Program
      • Students Not Eligible for the National School Lunch Program
    • Percent performing at or above “Proficient” on the NAEP assessment (3 measures)
      • All Students
      • Students Eligible for the National School Lunch Program
      • Students Not Eligible for the National School Lunch Program
    • Percent meeting all four ACT benchmarks (adjusted for percent participation)
    • Average composite SAT score (adjusted for percent participation)
  • Money (School Spending)
    • Total Revenue Per Pupil (2 measures)
      • Actual Dollars
      • Dollars Adjusted for Regional Cost of Living (RPP)
    • Current Spending Per Pupil (2 measures)
      • Actual Dollars
      • Dollars Adjusted for Regional Cost of Living (RPP)
    • Spending on Instruction Per Pupil (2 measures)
      • Actual Dollars
      • Dollars Adjusted for Regional Cost of Living (RPP)
  • Organization Size
    • Ratio of Students per District
    • Ratio of Students per School
    • Ratio of Students per Staff Member
  • Student Demographics
    • Percent of Children at 100 percent Poverty
    • Percent Eligible for Free or Reduced-Price Lunch
    • Percent Served under IDEA (Special Education)
    • Percent in English Language Learner Programs
    • Percent of Students who are Non-White
  • Population
    • Median Household Income
    • Population per Square Mile
    • Percent of 25 Year Olds and Older with a High School Diploma
    • Percent of 25 Year Olds and Older with a Bachelor’s Degree
    • Percent of 25 Year Olds and Older with a Graduate Degree


By grouping the variables this way, we can ask more specific questions, such as “How do other states who are similar to Kansas in terms of school spending do on the ACT exam?” Because we are usually interested in comparing ourselves in terms of something specific, these kinds of groupings might help us more than the generic groupings we’ve been discussing.


Let’s assume, for a moment, that when we say “peer states,” we are talking about states that have similar populations.  We’d want to compare Kansas to other states based on the measures in the “Population” category, for sure.  But more specifically, we would want to look at other states that have similar public school student populations, so we’d also want to compare on the measures in the “Student Demographics” category.  


Combining these two categories, we have 10 demographic measures.  We took these measures, and marked any state within plus or minus one-half of a standard deviation of Kansas’s value as “similar to Kansas.”   Using this approach, we found the following number of “similar” states for each measure:


  • Student Demographics
    • Percent of Children at 100 percent Poverty - 14
    • Percent Eligible for Free or Reduced-Price Lunch - 14
    • Percent Served under IDEA (Special Education) - 21
    • Percent in English Language Learner Programs - 11
    • Percent of Students who are Non-White - 15
  • Population
    • Median Household Income - 18
    • Population per Square Mile - 31
    • Percent of 25 Year Olds and Older with a High School Diploma - 16
    • Percent of 25 Year Olds and Older with a Bachelor’s Degree - 15
    • Percent of 25 Year Olds and Older with a Graduate Degree - 14


The following shows the states that were similar to Kansas based on the criteria above on at least 6 of the 10 demographic measures, along with information on how they compare to Kansas on spending, student outcomes, and student ratios:


Peer States (similar (+/-.5 Std Dev) to KS on at least 6 of 10 demographic measures)
  • Oregon (7)
    • Spends more on 0/6 funding measures
    • Better on 1/14 outcome measure
    • Has more students per district, school, and staff
  • Washington (7)
    • Spends more on 0/6 funding measures
    • Better on 4/4 outcome measures
    • Has more students per district, school, and staff
  • Illinois (6)
    • Spends more on 6/6 funding measures
    • Better on 2/14 outcome measures
    • Has more students per district, school, and staff
  • Michigan (6)
    • Spends more on 6/6 funding measures
    • Better on 2/14 outcome measures
    • Has more students per district, school, and staff
  • Nebraska (6)
    • Spends more on 6/6 funding measures
    • Better on 5/14 outcome measures
    • Has fewer students per district, school, and staff
  • Pennsylvania (6)
    • Spends more on 6/6 funding measures
    • Better on 4/14 outcome measures
    • Has more students per district, school, and staff
  • Wisconsin (6)
    • Spends more on 6/6 funding measures
    • Better on 5/14 outcome measures
    • Has more students per school and staff, but fewer students per district



OR
WA
IL
MI
NE
PA
WI
Pct of Children in Poverty (100%)
Pct of students eligible for FRL
Pct of students (3-21) Served Under IDEA
Pct of students in ELL programs
Pct of Public School Students, Non-White
Median Household Income - Actual Dollars
Population Per Square Mile
Pct of 25-year-olds and > - HS completion or >
Pct of 25-year-olds and > - Bachelors or >
Pct of 25-year-olds and > - Graduate degree or >


↑ = Value Higher than Kansas
↕ = Value Similar to Kansas
↓ = Value Lower than Kansas


So, what does this tell us?  


First of all, of the 7 states similar to us, 5 spend more than Kansas per pupil based on all 6 funding measures.  That is 70% of our peer states.


Second, of the 7 states similar to us, no state has better student outcomes on a majority of achievement and attainment measures.  Peer states perform better on Kansas on anywhere from 1 to 5 of the measures each; meaning Kansas outperforms each of these peer states on anywhere from 65% to 90% of the achievement and attainment measures.


Third, Kansas has fewer students per district than 5/7 peer states, fewer students per school than 6/7 peer states, and fewer students per staff than 6/7 peer states.   It is unclear specifically how these ratios impact student outcomes, but it is notable that Kansas has more districts, schools, and staff members per student than most of its peers.


Fourth, it is important to note that many of our neighbor states; including Colorado, Iowa, Missouri, and Oklahoma, are not on this list of peer states.  Kansas is frequently compared to our neighbor states, despite the fact that we are notably different from them in terms of student demographics and population characteristics.


Next time, we’ll look at states that outperform Kansas on a majority of outcome measures and discuss what we might be able to learn from them.  

To examine the data more closely, use the following links:



3 comments:

  1. your blog helpful for me. your blog is very nice for me please keep it up.I like it
    Visit :- School Achievement Cards

    ReplyDelete
  2. We found this weblog very great and we wanna thank you for that. We hope you keep up the great work!
    School Achievement Cards

    ReplyDelete