By Joshua Benton
Staff Writer
Page 1A
Starting next spring, Texas schools will have to record whom students sit next to during the TAKS test, according to a set of anti-cheating reforms announced Monday.
The Texas Education Agency also will send inspectors unannounced to schools on test day, track which adults administer the tests to students and create an honor code for test takers.
The moves come one week after a Dallas Morning News investigation found more than 50,000 students with extremely unusual answer patterns on the 2005 and 2006 TAKS test. Experts say those patterns strongly suggest cheating by students or school personnel.
“The findings were definitely troubling and certainly raised suspicions,” TEA spokeswoman Debbie Graves Ratcliffe said Monday.
But the agency will not take the step researchers say would be most effective at deterring cheating: scrambling the order of test questions so students can’t copy off each other.
Among the reforms announced in the agency’s 14-point plan:
*School districts will have to keep more records about how they administer the TAKS test. Districts will have to record where students sit on test day and which adults proctor the exams. Without that information, it can be difficult for investigators to confirm suspicions of cheating.
*Campuses suspected of cheating problems will be assigned independent monitors to oversee testing. State officials will make unannounced visits to schools on test day. And districts found to have problems could have their state ratings lowered.
*The TEA will create a “transparent method” to look for statistical anomalies that suggest cheating has taken place. Ms. Ratcliffe said that would include something “pretty similar if not exact” to the answer-copying analysis The News performed.
But the TEA won’t create multiple versions of each TAKS test, featuring the same questions in slightly different order.
Several researchers who study cheating have said that such a move would eliminate the vast majority of answer copying, because the answer sheet of a student’s neighbor would be aligned with a different set of questions. Scrambled item order is often used on college final exams and other tests to prevent cheating.
“It’s the single most important thing you can do,” said Robert Frary, a professor emeritus of educational measurement at Virginia Tech who has studied cheating for more than 30 years. “With one change, you can get rid of 90 percent plus of cheating.”
In Monday’s news release, the TEA noted that the plan includes “using multiple versions of tests,” and agency officials initially said that some questions that count toward a student’s score would appear on the test in scrambled order.
But the agency later clarified to say that none of those questions would be in scrambled order. Only questions that don’t count – ones being tried out for use on future exams – would be different on each student’s test. That’s no change from the way the TAKS has been given since its inception.
Those field test questions appear at an identical fixed position on each test, but are different from test booklet to test booklet.
Gregory Cizek, who has been a TEA consultant on cheating issues since 2005, said scrambling regular test items is a bad idea. He said it would be “outrageously expensive,” costing millions of dollars. It also would increase the risk of a grading error because it would require multiple answer keys and make the grading process more complex.
“Most testing specialists recommend that if you can avoid introducing a potential source of error, then you should definitely do so,” said Dr. Cizek, a professor of educational measurement at the University of North Carolina.
He also said that even the same questions can, when put in a different order, have different levels of difficulty. That could lead to a situation in which two scrambled versions of the same test could have different cut scores – leading to confusion among school officials and parents.
“Let’s say my kid took a test and needed to get 30 questions right to pass,” Dr. Cizek said. “And your kid took the same test, in a different order, and needed 31. How would people react to that?”
Other researchers disagreed with Dr. Cizek. Dr. Frary argued that the chance of a change in cut score resulting from scrambled questions would be “trivial.”
“And if it happens, you just deal with it,” Dr. Frary said. “Compared to cheating, it’s a very small issue.”
Dr. Frary said that the cost of scrambling would not be on the scale Dr. Cizek said and that his worries about grading error were not big enough to counteract the score-warping power of cheating.
“They’ve taken their head from one hole in the sand and put it in another one,” said David Harpp, a professor at McGill University who researches cheating.
Ms. Ratcliffe acknowledged that maintaining the status quo on scrambling would not help prevent cheating.
“That particular point isn’t aimed at prevention – it’s aimed at detection,” she said. The agency will begin analyzing how student answer patterns change around the field test portion of the exam. A student who answers “real” questions perfectly but gives boneheaded answers to field test items could be copying answers from a neighbor who has a different version of the test.
In addition to the other announced changes, the agency will require more extensive training for teachers who administer the test and better reporting of cheating incidents by school officials.
A number of the plan’s 14 points actually have been standing agency policy for several years. For example, TEA has imposed test monitors on school district a number of times before – most recently this spring in North Forest ISD in Houston. TEA also has reduced a school district’s rating based on a cheating investigation – such as in the Wilmer-Hutchins school district two years ago.
Ms. Ratcliffe said most of the changes should be in place for testing next spring.