By Joshua Benton
Staff Writer
Page 1A
Nearly 600 Texas public schools have been cleared of suspicions of cheating, state officials said Thursday, leaving 105 other schools still under investigation.
Texas Education Agency officials cited the clearing of 592 schools as evidence of the integrity of the state’s influential testing system.
“It is imperative that Texans trust our test results and have confidence that they are valid and reliable,” Education Commissioner Shirley Neeley said in a prepared statement.
But some question the thoroughness of the agency’s investigation, which relied heavily on self-reported questionnaires filled out by school officials a year and a half after the 2005 tests in question.
“I don’t know how accurate a set of responses you’re going to get from sending people a questionnaire,” said Jason Stephens, an assistant professor at the University of Connecticut who studies cheating. “That might be expedient, but if there is something going on, nobody’s going to go out and admit that.”
The investigation stems from a report produced in May by Caveon, a test-security firm. It analyzed schools’ scores on the 2005 Texas Assessment of Knowledge and Skills and tried to determine which schools had unusual patterns that could suggest cheating.
The report flagged 700 schools for a variety of reasons, including scores that jumped too quickly, answer sheets with too many erasures and students whose answer patterns suggested they might have copied off a classmate.
Differing approaches
After some deliberation, state officials decided this fall to investigate the schools. But the schools received different levels of scrutiny. Sixty-five received on-site visits by teams from the agency, in which investigators interviewed educators and other staff about test security.
Instead, the remaining 635 schools were asked to complete a questionnaire asking about a variety of test-security matters. Topics included school policies on cellphone use, the training provided to test monitors, security measures taken to protect test documents, and the straightforward “Did anything out of the ordinary occur that has not previously been reported?”
A 15-person panel of educators met in Austin about a month ago to go over the questionnaires. They didn’t have access to any other information about the schools – such as why Caveon considered them suspicious, how many students had suspect scores, or how extreme the statistical anomalies were. Instead, panel members were asked to evaluate how completely the questionnaires were filled out and to look for any suspect answers.
Unanswered questions
State officials did not disclose what the schools still under investigation had done to earn that status. But it appears that not answering a portion of the state’s security questionnaire may have played a role.
In Dallas, for example, at least three schools – Carter High, Blair Elementary and Holmes Middle – did not answer all or part of the questionnaire because they had switched principals since the 2005 TAKS tests. All three remain under investigation.
That’s despite the fact that statistical evidence against those schools is not as strong as that against other Dallas schools that were cleared.
At South Oak Cliff High, for example, more than 230 TAKS answer sheets were unusually similar to others, which suggests answer copying, according to a News analysis. That’s more than 10 percent of all tests taken at South Oak Cliff that year and far above the state average.
By contrast, the analysis flagged only about 6 percent of Carter High’s 2005 TAKS answer sheets. But Carter remains under investigation, and South Oak Cliff has been cleared.
Schools that received an on-site investigation did not appear to fare any worse than their peers in the state’s analysis. Of the 65 visited, only two remain under investigation: Theresa B. Lee Academy in Fort Worth and Winona High in East Texas.
It’s also unclear how schools were selected for visits. No schools in the state’s two largest districts, Houston and Dallas, were visited, although they had by far the state’s largest number of flagged schools. But 16 schools in Plano and Rockwall received on-site visits; all were cleared.
For schools that remain under investigation, there were many unanswered questions.
“We don’t know anything you don’t know,” said Tam Jones, an assistant superintendent in Crowley ISD, which has two schools still under investigation. “We sent off a stack of paper two inches thick awhile back, and we’ve just been waiting to hear what was going to happen.”
Both Crowley schools were flagged for large gains in test scores, the type of anomaly cited by Caveon that has drawn the most scrutiny.
Brad Lancaster, assistant superintendent in Allen, said he wasn’t surprised that his district’s three schools on the list were cleared.
“We knew we could explain the gains our students had made,” he said.
Mac Bernd, Arlington’s superintendent, said he believed that the questionnaires could be a good way to detect wrongdoing.
“People are not going to perjure themselves when they fill out a government document,” he said. “The penalties for that get really severe.”
All Arlington schools were cleared by the TEA on Thursday.
“My problem with the whole thing is that it seemed our schools were being penalized for doing too well” on the TAKS, Dr. Bernd said. “The explanations we got from the schools were very credible.”
Dr. Bernd and Allen ISD’s Dr. Lancaster said the TEA should have some way to police schools for cheating, but they agreed that Caveon’s methodology was too broad.
“TEA is between a rock and a hard place,” Dr. Lancaster said. “There’s clearly some stuff going on that shouldn’t be going on. But it’s hard to catch those without a net so big that it catches other schools too.”
Some of the 105 remaining schools will receive on-site visits from investigators after the holidays, TEA spokeswoman Suzanne Marchman said. The agency hopes to conclude all investigations by the end of January.
Investigations drag
The calendar has perhaps been the investigations’ biggest roadblock. The tests in question were administered in spring 2005. Caveon was hired in June 2005 and promised it could turn out results in six weeks.
However, for reasons TEA has not made clear, Caveon’s findings were not announced until May 2006. It took several more months for the agency to determine how it would deal with Caveon’s findings.
The result is that more than 16 months passed between the test’s administration and the TEA’s first investigations.
“Most people can’t remember who they sat next to yesterday, much less who they sat next to in April 2005,” Ms. Marchman said. “It makes it difficult when you want to have documentation or witnesses, or hopefully both. People can’t remember that far back.”
The state’s test-security task force is finalizing a series of recommendations to Dr. Neeley, the state commissioner, about test security and how best to look for cheaters – if at all. Neither Caveon nor the agency has analyzed the 2006 test scores, now 8 months old, and it is unclear whether they will ever be analyzed.
But Ms. Marchman said it is unlikely there will be a wait like this year’s again.
“It really concerned the task force, the amount of time that has passed,” she said. “They think that’s not acceptable.”