Testing… What Are We Doing?
Standardized testing has overrun public education. When I have visited schools, talked with administrators and teachers, my faith in the American Dream has come under attack. No longer has passion and engagement been the underlying themes to student growth. Instead, passion and engagement have been replaced by standardization and data mining.
Please know this. I love numbers. Absolutely adore numbers. If I could, I would help Stephen Hawking figure out how to explain life with one simple equation. Unfortunately, though, my love of numbers has also initiated my feeling of disgust when watching others misuse them and equate fantasy football methodology to that of a 3rd graders state test score.
In an effort to take a closer look at the “state” of testing, this blog’s theme will focus on my home state, Tennessee. I love Tennessee – the opportunities, the mountains, the people, the climate, and the lifestyle. I only wish I could say the same of Tennessee’s use of testing.
Tennessee is not the best state when it comes to funding education. As I looked for the most recent numbers I could find, I did find a fairly recent state-by-state ranking of spending per student. In 2012 Tennessee was 7th from the bottom on the list of 50 states. In 2013 Tennessee actually moved further down, ranking 4th from the bottom. In general, comparably to other states, let’s just say that Tennessee is not a state that prioritizes education when it comes to funding.
***Side note… for more detailed information on state spending, you can find items up through 2013 at Governing Data (fyi… I will post all the source sites for italicized terms at the bottom of the blog).
There is an outlier when it comes to educational funding in Tennessee though. The Brown Center in Education Policy completed a 2012 study on state spending for K-12 assessment systems. Tennessee was actually 25th out of 50. Oops. Maybe that is not the best outlier. Its probably not the best thing to brag how Tennessee is a legitimate spender on testing, just not everything else like teachers, curriculum, technology, etc..
Looking at the Brown Center study from 2012, we can approximate the spending per student in Tennessee on testing materials as $26 per student. Unfortunately, I could not find an accurate number of students in Tennessee for 2012, but using the Tennessee Department of Education website, I did find in 2013-2014 the number of students in public education in Tennessee to be 993,814. I know the years are off, but for the purpose of getting a general number, we can estimate the cost of test materials to be the product of $26 times 993,814. That is $25,839,164. Of course, it is now 2015, so that dollar value is certainly a low threshold. But it gives us a starting point in terms of basic test material cost.
There is one inherent problem with this number of $25,000,000, however; it does not give us an idea of time cost. This cost only represents booklets and score reports. What about the cost of time?
There are two costs people do not consider when it comes to testing. The first is the time cost associated with administering the test, which disrupts teaching and learning time throughout a building. The second is the time cost for preparing for the tests.
In order to better understand this value, we first must understand the value of a teacher. In 2014-2015 the Teacher Portal to Teaching Salary Data by State posted the average teacher salary for each state. As you may have guessed, Tennessee was not high on the list. Tennessee did manage to stay out of the bottom ten, it was 11th from the bottom with an average yearly salary of $47,563.
Let’s begin with considering the first cost of time – actual testing time. In general, an elementary and middle school shutdown for about two weeks in order to administer the state tests. The dirty secret about testing is it does not just disrupt the students who are testing, but it disrupts an entire school. At a high school the situation is actually somewhat laughable. Having worked as a traditional high school principal for 10 years, I have a difficult time remembering a time in the month of May where we didn’t test everyday. We even used to have a saying, “there’s a test every day the month of May.”
In order to err on the side of extremely conservative, let’s say state testing takes only one week from a school year. We’ll also be conservative again and say this only impacted 70% of the teachers. 30% managed to somehow avoid any school testing issues at all and were able to do everything they normally do with students throughout this testing week.
Here is some quick math. Teachers are employed for 200 instructional days in Tennessee. Each instructional day is worth 1/200th of the average teacher’s salary ($47,563), or $238 a day. In 2013-2014 there were 64,112 teachers in Tennessee. Thus, if we take 70% of the number of teachers in Tennessee and multiply that by $238 per day for 5 days, we should arrive at the monetary value for one week of testing time. The math then is as such: 64,112 times 0.70 times 238 times 5 = $53,405,296.
The second cost of time to consider is preparation time. Before I provide my crude stats here, I would like to make a disclaimer that I wish someone would do an actual study of the preparation time. I am going to be VERY conservative on my estimates here. These numbers are very low, but at least they will provide some schema for non-teachers. Here are my LOW ESTIMATES based on what I have seen at district, school, and classroom levels.
–Percent of teachers who have a single state test or multiple state tests – 75%
–Number of days a teacher spends in district trainings to prepare for state tests – 1 day
–Number of days a teacher spends working at the school level in school based testing preparation training, planning, data analysis, team meetings, etc. – 3 days
–Number of consecutive days spent on state test prep review with students immediately prior to the testing window – 5 days
–Percent of time on average a teacher spends throughout the year where the teacher is having students practice state test questions – 20% (equates to 1 day a week, or 30 days across a 30 week time period)
Cost of preparation time is then calculated as such: 75% of the number of teachers in Tennessee multiplied by $238 per day times the number of days spent in test preparation. The math then is as such: 64,112 times 0.75 times 238 times (1+3+5+30) = $446,315,688.
To quickly summarize, the cost of testing in Tennessee for one year is roughly:
–Test administration time: $53,405,296
–Test preparation time: $446,315,688
Keep in mind that none of this includes how testing impacts the time of a school administrator (principal, assistant principal), school counselor (the 21st century testing coordinator), special education staff or any other support personnel.
In one year in Tennessee, we roughly spend over half a billion dollars worth of time on testing. Let’s now talk testing data.
In Tennessee we have a “non-partisan” education advocacy group called SCORE. SCORE is an acronym for State Collaborative on Reforming Education. SCORE does a great job in putting together data for Tennessee and recently published the 2014-2015 State of Education in Tennessee. All the data below, besides a study through Brookings on NAEP scores, can be found in this SCORE publication.
The best place to begin in the SCORE publication is with state assessments for grades 3-8. Below are the reported gains from 2010-2011 through 2013-2014 on the state tests. (In parentheses are the listed percentages for proficient and advanced students for 10-11 and 13-14)
–Reading: Up 2% (47.5 to 49.5)
–Math: Up 10.3% (41.0 to 51.3)
–Science: Up 8.7% (54.9 to 63.6)
Along with this state data, there is of course Tennessee’s pride of 2013, the NAEP scores. In the SCORE report you will find the following rankings:
Tennessee NAEP 2011 State Rankings (out of 50):
–Math: 4th grade – 46th, 8th grade – 45th
–Reading: 4th grade – 41st, 8th grade – 41st
Tennessee NAEP 2013 State Rankings (out of 50):
–Math: 4th grade – 37th, 8th grade – 43rd
–Reading: 4th grade – 31st, 8th grade – 34th
At first look, all of this data looks amazing. Tennessee has fabulous growth in state testing results. Tennessee also improved their ranking in every NAEP tested area and nationally has been outspoken about how we showed the most growth of any state in the country. Tennessee rocks!
All looks great, right? Well, lets hold off a minute on the state data and look more closely at the NAEP data. The problem with the NAEP is we don’t really know what gains were made. We only know that Tennessee’s overall score moved up a few spots when compared to other poor performing states. What happens if we compare the actual test score gains with gains made by other states? Well, lucky for us, Brookings, a nonprofit public policy organization, did just that and performed a statistical study on actual test gains. Tennessee is still clearly the best, right? The results of comparing gains on the each test are below.
- 4th Grade Math: TN had gains greater than 42 states. Not significantly greater than 8 states.
- 8th Grade Math: Greater than 15 states. Not significantly greater than 35 states.
- 4th Grade Reading: Greater than 22 states. Not significantly greater than 28 states.
- 8th Grade Reading: Greater than 34 states. Not significantly greater than 16 states.
So on two of the tests TN students improved more than the average state. And on the other two tests, well, we did beat some other states. It’s probably why the article that shared these results was named “Be Wary of Ranking NAEP Gains.”
Now, I know I poked small holes in the testing data above, but let’s just assume for one minute that this data is just amazing and there are no holes. Let’s just accept that we should be happy with ourselves in Tennessee and based on this data, we are doing great.
Unfortunately, someone might come along and ask us about our ACT data. What do you mean? You want to know about the ACT data in Tennessee? I guess if we are going to talk about state testing and brag on our successes for tests that no one ever uses to determine post high school placement, I guess we can look at the one test that actually does matter in post k-12 opportunities… the ACT.
The data for the ACT in Tennessee, which can be found almost in the very back of the SCORE report, is as follows. ACT comparisons from 2009-2010 to 2013-2014 (five year period):
- 2009-10: 19.6 composite score average
- 2013-14: 19.8 composite score average (up a whopping 0.2)
- Side note… it’s a good thing we didn’t use 2012-13: 19.5 composite score average (it actually went down 0.1)
- Another side note… the national average is 21.0.
Now, for the naysayers out there, you may be thinking, you cannot compare 4th and 8th grade test scores with the ACT. Those kids may or may not have yet taken the ACT. Valid point. So how about we look at another set of state tests highlighted for their successes in the SCORE report. In fact, we will look at all five high school state tests highlighted in the report and see what trend there is. Below is the reported growth over a four-year period for each of the following state assessments.
- Algebra I: Up 15.4% (47.0 proficient/advanced students to 62.4 proficient/advanced students)
- Algebra II: Up 17.1% (30.8 to 47.9)
- English I: Up 5.1% (66.3 to 71.4)
- English II: Up 5.3% (58.1 to 63.4)
- Biology: Up 11.5% (52.0 to 63.5)
Do these gains equate to student success on the ACT? No. In fact, these gains are actually meaningless when compared to student scores on the most used college entrance exam in the world.
Let’s get back to the money piece.
Let’s suppose TN moves the ACT score up 0.2 every five years just like we have the past five years. We’ll ignore the fact that the score actually went down if we use the year prior and that 0.2 is a real gain. Using the costs we described earlier, lets look out 35 years from now to see our future.
In 35 years, Tennessee will have an ACT average of 21.2, finally topping the nation’s average by a whopping 0.2 (assuming no other state makes any improvement).
The cost of the state tests (assuming they are provided at the 2012 price for the next 35 years and the number of teachers is exactly the same) will be $25,839,164 times 35 years, or $904,370,740. Remember, this cost does not include paying for the ACT suite of tests (EXPLORE, PLAN, and ACT) that are also administered each year, nor does it include the TNReady assessments that are newly created and have increased both testing costs as well as testing dates.
So, we have the 35-year cost of just the tests: just under a billion dollars. We now add to that the cost of the time for actual testing. We calculated one of the current years to be $53,405,296 and times that by 35, we get $1,869,185,360. And, lets add in test prep time. 35 times $446,315,688, which equals $15,621,049,080 (15 billion dollars!!!).
Is the testing accountability model actually working? Who would invest this kind of time and money into this return on investment?
Testing accountability systems are not the answer to helping students become productive and meaningful citizens. I have many ideas on how we need to spend our money and our time. But this article is not intended for me to tell you my ideas for what to do. This article is intended for you to ask:
What are we doing and what should we be doing?
Brown Center in Education Policy – 2012 study on state spending for K-12 Assessment Systems
TN Dept of Ed website
Teacher Portal to Teaching Salary Data by State
2014-2015 State of Education in Tennessee
Tennessee: SCORE – State Collaborative on Reforming Education