George Bernard Shaw once said: “Live in contact with dreams and you will get something of their charm: live in contact with facts and you get something of their brutality”. When it comes to the current preoccupation with standardized testing in the United States (and, more particularly, in New York State), my fear is that we are replacing dreams and compassion and thoughtful analysis with a quick but brutal search for data and statistics. And it is wrong to do this. We ought not to acquiesce to that which diminishes the quality of life for our children, our teachers, our schools and our communities.

I have to admit to a deep ambivalence about standardized testing. A certain amount of testing is necessary in schools (probably more so for secondary students than for elementary students). Valid data that is appropriately gleaned from real performance is necessary for assessing — and enhancing — the quality of what we do. It is just that I do not see that happening very much these days. We seem to love “facts” more than people and learning, and we have left common sense behind.

I can use my own community to illustrate this point, but the argument does begin in a rather meandering way. In late August, 2011, our community was devastated by Hurricane Irene. Two of the main towns in our school district were almost completely destroyed. Throughout our district, we lost roads, bridges, homes, businesses, and churches. We were without electricity and phone service for an extended period of time. We had many families who were living in tents and temporary housing. More than a few of our children escaped from the flooding with only the clothes on their backs. The most brutal fact for us then (and now) is that it will at best take us years to recover, and perhaps we never will.

For many of our children, the school was the only “normal” in their lives at the beginning of September. But our school could not open on time despite our best efforts — and therein lies our tale. Our whole area had been declared a disaster area, and roads in one of the counties we serve were officially closed and we could not drive our school buses on those roads. We were forced to delay our opening. This, of course, meant that, in addition to all the other problems we would have to face this year, we would also have trouble meeting the state mandated 180 day school year. Since the state would not allow any deviations from the 180 day school year (even for an area struck by a natural disaster), the school was forced to take two days of Christmas vacation away from our students. Yes, facts are facts and 180 means 180. The harsh reality was that our students (and their families) as well as our teachers had to accept that, within the 180 day school year, each day is precious and sacrosanct, despite their having endured an autumn full of incredible loss and hardship and stress. Facts are fact.

And now for the connection to standardized testing. In the real life terms of a 180 day school year, how important is the instructional time we do have? How does it all play out in context? In New York State, we have just finished two weeks of standardized testing. The first week was devoted to English tests that were administered on three successive days. Each test was 90 minutes long. The second week was devoted to mathematics. Again, three successive days of testing. Each test again 90 minutes long. In reality, all New York State students in grades 3 through 8 spent a total of 540 minutes (9 hours) in testing over the past two weeks. And they are not done. They will have more tests at the end of the school term. At a time when schools are being subject to severe budget limitations and unfunded mandates, one aspect of our school operation did increase: the amount of time we spend testing students. Last year, the mandated state tests in English took 150 minutes and we managed to increase the testing time by 120 minutes this year. Last year, mathematics testing took 100 minutes and we managed to increase that by 170 minutes this year as well. The pertinent question is why the increase? Are the children learning that much more that has to be measured?

In New York State, the answer is partly that the state has signed a contract with a private company to develop tests for student assessment. There was a significant amount of field testing of items on this year’s tests that had nothing at all to do with the students taking the tests. In fact, some of those test items that students had to grapple with were not even a part of their curriculum this year. The test is being developed first and then the curriculum for the test will be developed (by April, 2013, we are told) and then it will be taught to students. Personally, I question the content validity of the tests being given to our students, and I will continue to feel that way until a curriculum has been published that aligns with the tests we give. Where is the common sense in all of this? Were these tests a “fair” measure of student performance? Were these tests a prudent use of instructional time? If we can afford to spend this much time on field testing, was it unreasonable for a district in a natural disaster to look to the state for a variance on the 180 day school year?

And it is not just students who are being assessed by these tests. Under our state’s new teacher evaluation system, student test results will become a significant percentage of a teacher’s annual professional performance evaluation. There does not seem to be a great deal of research to support the contention that this is a fair way to evaluate teachers. Since the tests were designed for a different purpose, I wonder about the construct validity of those tests as a measure of teaching ability. It might be argued perhaps that good teachers ought to be able to look at a test and figure out how to teach for it. I do not quite agree with a “teach to the test” philosophy, but, even if I did, the current tests are being kept “secret”. Teachers are not allowed to discuss them, and no one is allowed to keep copies of the test for public scrutiny because the intent is to use the test items again and again in New York State and elsewhere.

In the meantime, student scores will be collated and used in teacher evaluations. Those scores will be fed into a formula from which we will derive a numerical rating for teachers that will reportedly show how good that teacher really is. While the tests themselves will be kept “secret”, teacher evaluation scores (reported as a two digit number) might well be released to the public. For the most part, the data will be assessed without reference to the contexts surrounding those performances.

The decision to withhold these tests from public scrutiny is itself the subject of a public debate. Just because something can be reduced to a number does not make that number an infallible indicator of quality. There were obvious problems with the design of this particular set of tests that have been reported extensively in NY papers over the past two weeks (see the Related Articles at the end of this blog). An entire reading passage on the 8th grade test had to be eliminated because it made no sense. On one of the eighth grade tests, there was an item with no correct answer. On one of the fourth grade tests, there was an item with two correct answers. My personal favorite was reported in the New York Times and involved a 5th grade math “problem [that] asked students to find the perimeter of a trapezoid that could not exist within the bounds of mathematics” ( http://www.nytimes.com/schoolbook/2012/05/02/state-officials-throw-out-another-pearson-test-question/ ). Facts are indeed facts, but how fair is all of this to the students taking the tests or to the teachers whose professional performance evaluations will be indexed to those student test scores?

Yes, we all need to be concerned with the learning in our schools. My feeling, however, is that we ought to begin the process by learning how to use common sense to inform our efforts. There are some situations where standardized testing is appropriate and relevant. And there are other situations in which standardized testing is simply brutal and unjust. I am not at all convinced that political decisions over the use of standardized tests (made at either the state or federal levels) will do much to advance the cause of learning.

Related Articles

“Pearson and how 2012 standardized tests were designed” by Valerie Strauss < http://www.washingtonpost.com/blogs/answer-sheet/post/pearson-and-how-2012-standardized-tests-were-designed/2012/04/ >

“Is it right that your 8-year-old is an exam guinea pig?” by Juan Gonzalez< http://articles.nydailynews.com/2012-04-18/news/31357009_1_state-tests-standardized-tests-field-questions >

“Public Applies More Scrutiny to Exams That Are Supposed to Be Secret” by Anna M. Phillips < http://www.nytimes.com/schoolbook/2012/04/25/public-applies-more-scrutiny-to-exams-that-are-supposed-to-be-secret/ >

“A Very Pricey Pineapple” by Gail Collins < http://www.nytimes.com/2012/04/28/opinion/collins-a-very-pricey-pineapple.html?_r=2 >

“Principals Criticize Increased Costs of Test Scoring” by Beth Fertig < http://www.nytimes.com/schoolbook/2012/04/24/principals-criticize-increased-costs-of-test-scoring/ >

“State Officials Throw Out Another Pearson Test Question” by Anna M. Phillips < http://www.nytimes.com/schoolbook/2012/05/02/state-officials-throw-out-another-pearson-test-question/&gt;