By Joshua Benton
Staff Writer
Page 1A
Last of two parts
Education researchers are clear: The vast majority of teachers are honest people and wouldn’t think of doctoring their students’ results on a standardized test.
But unfortunately, “the vast majority” doesn’t include everybody. In a high-pressure, high-stakes environment, some teachers are going to cross an ethical line.
Some experts say the Texas Education Agency isn’t doing enough to track them down. In some cases, the agency ignores information that could tip the agency off to improper behavior.
“It’s something we don’t want to admit,” said Tom Haladyna, a professor at Arizona State University who studies cheating. “Teachers are the most respected, most admired profession. But we badger them to get high test scores. And some feel the pressure to get test scores at any expense.”
A Dallas Morning News analysis of test scores found suspicious gaps and unlikely one-year swings in the performance of more than 200 Texas public schools.
Some of the schools may have legitimate explanations for their unusual score patterns. But in some cases, the gaps are so wide that experts say it’s difficult to imagine a cause other than cheating. One Houston school, Sanderson Elementary, finished in the bottom 2 percent of the state at fourth grade but had the best scores in Texas at fifth grade – with the majority of students suddenly getting perfect scores.
As a result of The News’ analysis, the TEA is now investigating allegations of cheating in three Texas school districts, including Dallas and Houston.
The newspaper’s analysis was performed entirely with publicly available test scores obtained from the TEA, using basic statistical techniques. The agency could perform a similar analysis on its own, but its leaders choose not to.
“Typically, school districts police themselves,” said Lisa Chandler, the TEA’s director of assessment. “We trust educators to educate our kids.”
TEA policy is to investigate allegations of cheating only when a district requests it or when it receives credible eyewitness evidence of cheating. Since most cheating occurs behind the closed doors of a testing classroom, that sort of firsthand evidence can be hard to find.
“Circumstantial evidence just doesn’t cut it,” TEA spokesperson Debbie Graves Ratcliffe said.
Rapid improvement
Researchers differ on how common it is for teachers to improperly help students on standardized tests – either during the test or by altering test documents after the fact. Estimates range from 1 to 10 percent of classrooms.
Many researchers say that radical one-year turnarounds in test scores are very difficult to achieve. Reforms take time, particularly with children who have already had several years of sub-par school experiences.
“A kid can learn a large amount in a year in the right situation,” said John Fremer, former top test developer of the SAT and one of the most respected men in the testing industry. “But it’s hard for a whole class or a school to make as large a jump as an individual. When they make large jumps, you really have to ask yourself what’s going on.”
Ms. Chandler said she refuses to react cynically when TAKS scores improve by leaps and bounds.
“It may be an optimistic viewpoint, but it’s also a necessary viewpoint,” she said. “We can’t afford to lose five years in a child’s educational career. They have to have improvement. Can we expect quick turnarounds? We have to. We can’t let those kids not be successful.”
The Texas accountability system rewards schools and districts for high test scores. Principals and teachers often see their careers advance if their students score well. That can leave few incentives for educators to be vigorous about pursuing cheaters.
When The News informed Houston officials about the suspect scores at Sanderson, the district asked the TEA to begin a cheating investigation within 48 hours.
But Sanderson’s extraordinary fifth-grade math performance was not a secret. The school’s suspect scores are posted on the Houston ISD’s own Web site.
According to data on the site, not one of Sanderson’s fourth-graders got a perfect or near-perfect score on that grade’s math test. In contrast, 92 percent of the school’s fifth-graders did – the highest number in the state.
Looking at erasures
If the TEA wants to find cheaters, the agency already has a powerful tool available: the results of its annual erasure analysis.
Test-grading companies have equipment that can identify student answer sheets with unusually high numbers of answers erased and replaced. Researchers have developed formulas to determine when the number of erasures becomes so high that cheating is suspected. The equipment also can determine whether an unusual number of the erasures turned wrong answers into correct ones – another sign of potential cheating.
Texas performs an erasure analysis on every TAKS answer sheet in the state. But, under TEA policy, it does nothing with the information it produces.
If an analysis finds alarmingly high levels of erasures on a school’s answer sheets, the TEA doesn’t start an investigation – or even ask the school for an explanation.
Erasure information is only used if the agency receives a separate complaint alleging cheating, Ms. Chandler said. Then the agency may check the results of the erasure analysis to see if they support the allegations.
“It doesn’t necessarily show us the complete picture,” Ms. Chandler said when asked why the agency doesn’t examine the erasure data. She said that more conclusive evidence would be necessary to be certain cheating has occurred.
The TEA has used erasure analysis proactively only once before. In 1999, the agency found 33 schools in 11 districts that had high numbers of erasures for three consecutive years. The TEA demanded that each school district launch an internal cheating investigation.
One of the targets was Harrell Budd Elementary in Dallas, where the subsequent investigation found a sixth-grade teacher was helping students correct their answers during the test.
Another was Alta Mesa Elementary in the Wilmer-Hutchins district. When TEA officials sent monitors to oversee the testing process the next year, the school’s passing rate on the TAAS test dropped 40 percentage points.
Both Harrell Budd and Alta Mesa were found to have highly suspicious test scores in The News’ analysis. As a result, both are now back under state investigation.
Taking action
Ms. Chandler said the agency is willing to consider toughening its stance. It is considering setting erasure thresholds for schools that, if exceeded, would trigger investigations.
TEA officials also said that the agency is considering adding a check similar to The News’ analysis: searching for schools with wide swings in average scale scores.
“Stakes have gotten drastically higher in recent years,” Ms. Ratcliffe said. “Testing has gotten more complicated. The potential for more problems to arise is there.”
It’s unclear whether the agency has the manpower that would be necessary to be more aggressive in investigating cheating. The TEA’s test-security department has only three employees. Ms. Chandler said that the staffing level is sufficient to investigate violations reported to the agency but that proactively looking for cheaters would “certainly take more personnel.”
Ms. Chandler said the agency has investigated about 1,700 complaints about testing procedure in the last two years. But that total includes a large number of procedural complaints about testing irregularities that do not affect scores, such as an educator mistakenly breaking the seal on a test booklet five minutes too soon. She said the district has launched on-site investigations in a “handful” of districts over that span.
Standard practice?
Experts say many states take an approach similar to Texas’ – investigating cheating only when specific evidence is reported. But as stakes continue to rise – both via the state ratings system and the federal No Child Left Behind system – that may change.
“You have to check to see the data is reliable,” said Dr. Fremer, who last year founded Caveon, a company that sells test-security services to states. “I believe that five years from now, you won’t find a state where these checks aren’t part of their standard practice. That will change.”
In Tennessee, for example, red flags go up when a school shows massive, unexpected improvement in scores. Huge swings can lead to an inquiry with the school district about possible improprieties.
“A huge gain by a teacher – that would attract attention,” said Ben Brown, the state’s executive director of evaluation and assessment.
But he acknowledged it is very difficult to turn circumstantial evidence – like score anomalies and erasure analysis – into something that can stand up in court.
“Cheating is a very severe charge,” he said. “We do everything we can to certainly protect and guarantee the rights of the professionals in the schools. But at the same time, it is our professional responsibility to make sure the assessment produces accurate results.”
This story is the second of two parts. For part one of this series, go online to DallasNews.com or read it in Sunday’s Dallas Morning News.