News

Project examines report card data

Posted November 11, 2007 10:00 pm

Pete Sherman

Each year, Harvey Smith sifts through an enormous stack of Illinois school data at his office at Northern Illinois University's Interactive Illinois Report Card project.

Included are reading, math and science state assessment results from thousands of schools and from seven different grades ranging from third to 11th, school and school district financial and demographic information, and data about teacher and administrative salaries and experience levels.

"We get so busy processing the data files that, for about the first week or so, I don't even look at the results," said Smith, who directs the IIRC.

If the report card project has a single mission, it's to challenge the one temptation to which most people - and reporters - succumb when they look at the data.

"The biggest single mistake," Smith said, "is to rank schools by their scores."

Where most newspapers prefer to rank schools from year to year based on annual test results (the way the data are provided by the Illinois State Board of Education), the IIRC will show how a fourth-grade class at a school compares with how the students performed the previous year in third grade.

"It is really hard to evaluate school performance. The (IIRC) report is an effort to design a set of standard measurements. From that point of view, it's a useful project," Smith said. "It's all compounded a little bit by the state and federal reporting mandates, which are well-intentioned but do not necessarily focus on students and learning."

Part of the problem is that the data are gathered based on the requirements of the 2001 federal No Child Left Behind Act, which primarily relies on yearly results versus progress over time.

As far as NCLB is concerned, it's more important for a school to show that this year's third- graders did better than last year's third-graders. Whether last year's third-graders improved in fourth grade isn't as significant.

Regardless, state education departments, charged with implementing NCLB, assemble the data as they're told, handing the results over to the public with little reorganization.

People like Smith are attempting to realign the numbers so apples and oranges are separated, putting what can be very helpful information into a clearer context.

"It is usually pretty accurate," said Chris Stahly, principal of Feitshans Academy, about the state's test score data. "The labeling is inaccurate - listing 'failing schools' really bothers me. Many schools might have students who are really struggling."

That has been the case at Feitshans, which, this year, lifted itself off the state's warning list of schools not measuring up to NCLB's academic benchmarks. Feitshans, where 98 percent of the students are from low-income families and 90 percent are black, has been on the list since NCLB took effect.

The IIRC's Web site, which allows visitors to look up school and district information in a variety of ways, shows that Feitshans has been making steady progress over the past six years. This year, Feitshans' fourth-graders were particularly impressive in math. More than 90 percent of them met grade-level standards, well above district and state averages.

Unfortunately, number-crunching doesn't take into account what a score means to a particular school.

"We don't have added-value measures," Smith said. "We can't show what schools contribute to learning, how culturally enriched households contribute, how exposure to books, activities, museums, blah, blah, blah contribute.

"Finding out one school scored 990 out of a 1,000 and another scored 700 out of 1,000 doesn't really mean the one school is better or the other is not as good. That's why I finding rankings troublesome."

The state board of education, which provides funding to the report card project, comes out with its own versions of state, district and school report cards.

"The report cards are a snapshot in time," said ISBE spokesman Matt Vanover. "It provides a wealth of information to parents, to school administrators, to taxpayers. It gives you information about how an individual school is doing in that one snapshot (when state tests are taken in the spring). But it is only one measure."

If there has been any shift in how the data are being used, it doesn't involve figuring out how to improve rankings. It's how to use the information to adjust teaching methods.

The state board only recently has started to break down data in a way that helps schools evaluate student weaknesses and help students do better.

"We've only been able to use (the test data) like this for a couple years," Stahly said. "Now we can tell, item by item, how children are doing."

It's what Stahly and others define as a shift in the use of the report card data from "of learning" to "for learning."

The shift also has the Springfield School District rethinking how it uses tests to measure academic achievement.

"We use the report card data all the time - for presentations, board discussions, and people use them when they move into the district," said Kathy Sanders, the district's director of assessment and secondary education.

But she said maybe the district is assessing students too much.

"We're taking an inventory of all our assessments," she said. "Are they really giving us the info we want?"

Meanwhile, however, the temptation to judge schools based on test scores remains strong.

Stahly recommends a different approach.

"Looking at the atmosphere of a school is the only way to assess it, and you can only do that with visits," Stahly said. "Visit classrooms. See what's going on. Feel that atmosphere - the atmosphere the students are experiencing every day."

Pete Sherman can be reached at (217) 788-1539 or pete.sherman@sj-r.com.

On the Web

nhttp://iirc.niu.edu/ - Web site for the Interactive Illinois Report Card based at Northern Illinois University.

nhttp://webprod.isbe.net/ereportcard/publicsite/getSearchCriteria.aspx - The Illinois State Board of Education's Web site providing its long-form version of its yearly report cards.

nhttp://webprod.isbe.net/ereportcard/publicsite/getProfileSearchCriteria.aspx - Where ISBE provides an "easy to understand" version.

Article Categories
News