By Joshua Benton
Staff Writer
Page 1A
Does Forest Brook High School have a TAKS cheating problem? It depends whom you believe. But new evidence points to yes.
Despite highly suspicious test scores, a February report by the Texas Education Agency declared the Houston school cheating-free – largely because school officials, when asked, said they were unaware of any wrongdoing on their campus.
But last month, a Dallas Morning News statistical analysis found that Forest Brook had one of the worst cheating problems in Texas. Looking at two years of scores, the analysis found more than 350 TAKS answer sheets had answer patterns that were suspiciously similar – in some cases identical – to those of at least one classmate.
Now, newly released test scores give further support to the idea. This spring, the state required outside monitors to oversee TAKS testing at Forest Brook. They watched over every stage of the testing process in an attempt to prevent any potential misdeeds.
The result? Under outside scrutiny, the school’s scores collapsed.
On the 11th-grade test – the one students must pass to graduate – Forest Brook’s math passing rate dropped from 80 percent in 2006 to 44 percent this year.
In science, the tumble was from 89 percent to 39 percent.
And in social studies – traditionally the easiest of the TAKS tests – Forest Brook dropped from a perfect 100 percent to 72 percent.
Despite the evidence, TEA officials defend their investigation, which looked at scores from 2005. The February report said the school should be “cleared of any wrongdoing” in its administration of the test.
“We did our investigation and we did not find the evidence we would have needed” to prove cheating, said Jim Lyde, TEA’s deputy inspector general, who investigated Forest Brook and other schools in the North Forest Independent School District.
Despite numerous attempts over the last week, North Forest officials did not respond to requests for comment.
History of problems
North Forest, a low-income district, has a long history of difficulties, both academic and financial. In many ways, its schools are to Houston what the since-closed Wilmer-Hutchins schools were to Dallas: the ones that were always in trouble.
Enrollment has declined more than 25 percent since 1997, when it was 13,758. The district has had six superintendents since 2000. The most recent to go, James Simpson, was fired in March for, among other reasons, failing to notify the board about a Lifetime Channel movie on one of the district’s high schools.
When Dr. Simpson appealed his dismissal to the board, several of its members walked out of a meeting – and then declared the meeting over because they had walked out. (A state hearing examiner later ordered Dr. Simpson reinstated.)
Last month, the district’s interim police chief filed a whistle-blower lawsuit, alleging he was suspended from his job because he was investigating whether four school board members were responsible for $23 million in missing funds.
The district’s management problems have been echoed in its academic performance. Only three of its schools earned a rating of acceptable from the state last year. The eight others that were rated were declared unacceptable. That included Forest Brook, whose scores on other TAKS tests were low. North Forest’s performance was, by a wide margin, the worst of any major Texas school district.
One school, Oak Village Middle, has earned an unacceptable rating in each of the last three years. It’s one of only five Texas schools with such a string of low performance.
It was Oak Village that led TEA to require monitors to oversee TAKS testing this spring. Under Texas law, a fourth year of unacceptable scores gives the state education commissioner the authority to order a school closed.
In March, then-Commissioner Shirley Neeley sent a letter to North Forest officials saying she wanted to make sure the scores reported this year were completely accurate. That meant, she wrote, the district had to bring in outside monitors to oversee testing and ensure no security rules were violated.
As it turns out, the monitors appear not to have had a significant impact on Oak Village’s scores, which increased slightly from 2006. But scores at Forest Brook High took a remarkable nosedive under outside supervision.
Independent scrutiny
Over the last two years, two independent analyses have found signs of substantial cheating at Forest Brook. The first was performed by the test-security firm Caveon, which TEA hired in 2005 to analyze that year’s TAKS scores at all Texas schools.
Caveon “flagged” 700 schools whose scores it considered suspicious in some way. Schools with extremely suspect scores were flagged multiple times – indicating they were suspicious in a number of ways.
Most of the 700 schools Caveon identified received only one or two flags. Forest Brook received 51. No other school in the state received more than 30.
Then, the News analysis of scores from 2005 and 2006 again found extremely unlikely answer patterns at Forest Brook. Many students had answer sheets that were identical or nearly identical to their fellow students’ – the sort of similarity that experts say cannot be the result of chance.
The Caveon analysis triggered Mr. Lyde’s investigation. But that investigation cleared Forest Brook entirely, finding “no evidence of purposive impropriety” by school or district officials.
To explain the astounding number of flags that Caveon assigned their school, North Forest officials advanced two theories.
The first was that Forest Brook teachers had worked very hard – that they had improved instruction in many ways and that better student learning could have triggered Caveon’s suspicions.
The second was creative: that Forest Brook had been flagged so many times because of the way it boxed its answer sheets.
Officials said that, when they shipped student answer sheets off to be graded, they divided them up by the classroom in which the tests were taken: Miss Smith’s students in one group, Miss Jones’ in another, and so on.
They argued that this boxing could have somehow garnered Caveon’s attention and suspicions because several similar answer sheets could have wound up together in a small batch. In a school that packed an entire grade’s worth of answer sheets in one batch, they argued, a few suspicious students would be less likely to stand out.
“The district testing coordinator gave me an explanation and I found evidence to support it,” Mr. Lyde said in a recent interview.
That evidence consisted of asking three individuals whether the explanation could be true. The first was the testing coordinator in another school district Mr. Lyde was investigating for possible cheating. That person said she considered North Forest’s explanation “a possibility,” according to records.
The second person was an employee of Pearson, the company that produces the TAKS, who said the North Forest theory “could be possible.” The third person was a TEA testing official who said the theory was “plausible.”
Based on that evidence, Mr. Lyde said he accepted the North Forest explanation and cleared Forest Brook of any wrongdoing.
“I have a protocol and a process that I’m supposed to follow,” he said. “In this case, I had sufficient reason given to me to explain the flags.”
‘Not even remotely possible’
But experts say – and easily accessible records show – that North Forest’s explanation does not reflect reality.
“This is not even remotely possible,” said Robert Frary, a professor emeritus of educational measurement at Virginia Tech, who has studied cheating detection for more than 30 years and is familiar with Caveon’s methods. “This is not even a 1-in-billions chance.”
First, Forest Brook is by no means alone in batching its testing documents by classroom. Hundreds, if not thousands, of Texas schools package their documents that way. None of them had scores anywhere near as suspicious as Forest Brook’s.
Second, Caveon’s own report to the state shows it does not evaluate schools in the manner that would be required for North Forest’s explanation to be correct. It is actually substantially harder for a small batch of students – like Forest Brook’s – to earn a Caveon flag, not easier.
Caveon’s analysis gives small groups the benefit of the doubt. In other words, Caveon considers a four-student classroom with two suspicious answer sheets less worthy of note than a 40-student classroom with 20 suspicious answer sheets.
Third, the district’s explanation would not account for why Forest Brook was also flagged nine times in a part of Caveon’s analysis that looked at answer patterns for the whole school, not individual classrooms. In that analysis, Caveon ignored distinctions among the various batches of answer sheets sent by schools and examined schools as a whole. Forest Brook’s nine school-wide flags were the second-highest total in the state.
North Forest’s other explanation for its Caveon flags – that its teachers worked very hard to boost scores – also has little connection to Caveon’s findings. What Caveon found most suspicious in Forest Brook was that its students had long strings of incorrect answers in common.
Decades of research on cheating have shown those long, identical runs of wrong answers are common when students or adults are copying answers from one answer sheet to another. Excellent instruction would not increase the number of identical wrong answers.
But despite all the statistical evidence and the sudden performance drop this year with monitors, TEA officials are standing by their decision to clear Forest Brook.
“I think what we may have, and that includes Forest Brook, is kids copying off of other kids,” said Mr. Lyde, who did not interview any students as part of his investigation. “That’s not something we were specifically geared to locate. We’re looking for educators who are cheating.”
TEA spokeswoman Debbie Graves Ratcliffe suggested another explanation: that the decline in scores this spring might be attributable to the district’s change in superintendents, which could have created a difficult atmosphere for students.
But other North Forest schools – despite experiencing the same administrative chaos – did not experience score declines anywhere near the scale Forest Brook did.
The state’s clearing of Forest Brook had one other effect on the school. It freed up its access to the $165,000 Governor’s Educator Excellence Grant the school had qualified for. It was awarded the money because of its rapidly rising test scores in 2006.