What Tests Can – and Cannot – Tell Us
A teacher’s view
Campbell’s Law – The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption measures, and the more apt it will be to distort and corrupt the social processes it is meant to monitor.
Tests are a part of every teacher’s life – an important part. Regardless of how well you think you are teaching, if your students are not putting that knowledge into long-term memory you are wasting a lot of time, both theirs and yours. As the student-teacher ratio spirals upward, which it is and will continue to do for the foreseeable future, the test remains a key indicator revealing learning. Simply put tests can reveal the most information for the largest number of students in the least time.
What teachers understand, and non-teachers do not, is the grade reveals very little of the necessary information. I will speak to this from a math perspective – because I taught math – but it is true in all subjects. I always preached the importance of showing your work in math class. Let’s look at an example: Suppose I gave a test where the answers were in multiple choice format; this format has the advantage of being very easy to grade and the disadvantage of revealing little to nothing about where and how the student is missing the knowledge being tested. With this in mind space is provided to show how your answer was determined. Two students both score 30% on the test. The first student showed work and a study of the work reveals that the student learned a particular step involved in the process incorrectly, and virtually every time this step was used (about 70%) the answer was wrong. The second student showed no work, but resorted to a time-honored strategy of marking ‘C’ for every problem, which was the correct answer 30% of the time. The lesson here is if the only piece of information gained from the test is the grade, these two students are equally ill-prepared by the teaching they received. If the grade is the only information provided to the teacher – while understanding that the lesson wasn’t received – there is no clue how to address that failure.
Significantly in Standardized Tests, the ONLY information a teacher receives is the final grade. Two important conclusions: Standardized Tests have NOTHING to do with improving learning; and a more striking example of Campbell’s Law would be difficult to find.
So why do we have standardized tests? Second point very apparent to teachers and invisible to non-teachers is Standardized Test results ARE NOT FOR THE TEACHERS! Not only are teachers not allowed to know which type of questions any particular student got wrong, they are not allowed to see the questions! Tests are not to help the students OR the teachers to improve learning, but to allow politicians to make judgment on a system they do not understand; as evidenced by the wacky requirements imposed by No Child Left Behind, since replaced by ESSA which was actually little better.
Two stories illustrate both the illegitimacy and unfairness of such judgment. The first is about a first grade teacher in inner city Baltimore whose performance was evaluated based on standardized tests given at the end of the year. She pointed out that the turnover rate in her class was 100%! Meaning not a single student in the class at the start of the year took the test! A number of them had only joined the class in the last few weeks before the test, yet they were tested and those results were attributed to her teaching.
The second involved a Geometry class I taught in which a Senior student who had failed the class needed to pass the Standardized Test to qualify for graduation (graduation rate being a very important metric on which schools are judged). I explained that the reason she failed the course, and in my opinion would not pass the test, was because she had missed over half the classes: More than 75% during the second semester. She didn’t know any Geometry because she had been there only half the time, and done no work when she was there. I was told that I must attempt to get her through the test regardless. I said I would work with her after school – and was told she could not come after school and it would be necessary to help during my free period, meaning the absolutely essential work I did for the rest of my students would have to be done at home – or not at all. With my reluctant agreement we scheduled the help. She didn’t bother to show. Every teacher knows examples where incredible amounts of teacher’s time are spent on a few, very un-motivated, students – for The Test.
These two stories highlight the idiocy of Standardized Testing – it doesn’t help students OR teachers and it paints education as a one-way street with zero accountability by anyone OTHER than the teacher.
Does this mean testing is innately bad? Of course not! One of the values of testing – which makes it very attractive to those who fail to understand how it should be used – is the necessary feedback it gives on student achievement. The question is to whom is this feedback directed, and to what purpose. Since the feedback of Standardized Testing is NOT directed to either the teachers OR the students it is not intended to improve student performance.
Now we can make sense of Campbell’s Law: These tests are designed as a quantitative social indicator and the purpose is social decision-making. It is virtually impossible to overstate the sway test scores currently hold over every educational decision made. If you are marketing educational materials (a VERY significant growth industry) claiming increased test scores is essential. This speaks directly to the ‘more subject it will be to corruption measures’ conclusion of Campbell. In 2009 there was a well-publicized case in Atlanta where a superintendent hired for the primary purpose of raising Test scores (who at the time the story broke was enjoying a Hawaii trip on the bonus received) had applied such pressure to achieve higher scores that many teachers had resorted to helping students cheat – to keep their jobs. Of course the teachers were sanctioned with loss of job, and although the superintendent was also fired, she kept the money paid for fraudulent results. Corruption with test scores continues and worsens as their interpretation explains whatever point you wish to make about the state of our educational system, or whatever ‘cure’ you are espousing. THIS is the ‘distortion and corruption of the social processes it is meant to monitor’ which Campbell’s Law asserts.
The teacher’s view: Standardized Tests have become a purpose unto themselves to a degree that the test is more important than the learning it is supposed to measure. The solution is to divorce the scores from the social decision-making and instead use tests as they SHOULD be used, to identify those students who are not learning – and use the money saved on all the accoutrements of high-stakes testing and scoring systems to actually TEACH the students what the test says they don’t know. Teachers are more than capable of using tests this way – it IS the job! Until this is done the teacher shortage – and student learning – will continue to worsen, as the teachers live the Engineer’s Dilemma:
“It’s not my place to run the train
The whistle I can’t blow.
It’s not my place to say how far
The train’s allowed to go.
It’s not my place to shoot off steam
Nor even clang the bell.
But let the damn thing jump the track
And see who catches hell!”
https://en.wikipedia.org/wiki/Campbell%27s_law
https://en.wikipedia.org/wiki/Atlanta_Public_Schools_cheating_scandal