The Idaho teachers who are trying out a new reading test generally like what they see, so far.

But the Boise State University researchers who surveyed the teachers aren’t jumping to any conclusions. They say it is simply too soon to grade the new test — a possible replacement for the old Idaho Reading Indicator.

That leaves lawmakers nervous about putting more money into the pilot. Two key legislators are even questioning the value of a statewide reading test in the first place.

The debate over the reading test is playing out largely behind the scenes at the Statehouse. The outcome will affect how Idaho identifies at-risk readers, and how the state hands out millions of taxpayer dollars for extra reading help.

How we got here

For nearly 20 years, the state has used some variation of the IRI to screen kindergarten through third-grade students. It’s designed to be a quick screening test that identifies students who aren’t reading at grade level — and are at risk of falling behind their classmates.

The scores have been sobering. Every fall, some 35,000 K-3 students show up for school not reading at grade level. Last year, kindergarten IRI scores reached an 11-year low. Fearing that at-risk readers will fall hopelessly behind in school, the state launched an ambitious literacy initiative.

And with that, the IRI has taken on added importance. The state uses IRI scores to distribute more than $11 million a year — putting money in the hands of districts and charters with higher numbers of at-risk readers.

Some educators have lamented the IRI’s limitations. The test measures reading speed, not comprehension, and doesn’t diagnose the cause of reading problems.

In late 2016, state superintendent Sherri Ybarra agreed to a contract with Istation, a Dallas-based vendor, to develop an IRI replacement. This is potentially a five-year contract, and the work began last fall, as 58 schools tested out Istation’s online reading assessment.

What did the survey say?

The Legislature’s budget-writing Joint Finance-Appropriations Committee asked Boise State’s Idaho Policy Institute to evaluate the pilot.

The institute asked teachers to offer their first impressions of the Istation pilot. By most metrics, the pilot test fared well:

• Seventy percent of teachers said they preferred the content of the new test. Only 13 percent said they favored the content of the old IRI. Other respondents said they had no preference, or declined to answer.
• Teachers said the new test seemed to do a good job of measuring reading skills. Eighty-one percent of teachers said they were confident in the Istation test results, while 18 percent said they were not confident in the results.
• There was at least one downside; 57 percent of teachers said it takes more time to administer the Istation test.

Boise State researchers were quick to put their numbers into perspective. They were able to survey only 133 teachers, and ask only about one round of testing. “A full-scale evaluation of the Istation IRI pilot program is not possible until fall 2018.”

Researchers also asked teachers for open-ended comments, which ran the gamut from high praise to deep skepticism.

A few recurring criticisms emerged.

Some teachers said the test results were unreliable or inconsistent.

Time management was a concern. As one first-grade teacher put it, “I watched several of my students simply lose interest and just start clicking without even thinking about what or why they were clicking. They just wanted to be done.”

And several kindergarten teachers said an online test measures computer skills, not reading skills. “For little people who don’t have experience with a computer or iPad it is very hard and intimidating at the beginning of the year,” said one teacher. “I had students in tears and just plain quit taking the assessment.”

Reactions

In a prepared statement, Istation said the Boise State survey was “largely positive,” but the company criticized the methodology. For one thing, the company criticized Boise State for surveying only teachers, and not the reading specialists who administer the IRI.

“We would welcome a consistent approach to research that applies to all education vendors who are funded by the state,” Istation said in its statement. “Other education technology companies, who receive hundreds of thousands or even millions of dollars from the state of Idaho, currently receive funding without any corresponding research efforts or data analysis requirements.”

Ybarra still wants to get rid of the old IRI, and she has proposed to take the Istation test statewide in 2018-19. She was pleased by the survey results. “It’s clear that the new test provides a more complete view of young students’ reading readiness and reading skills,” she said.

Two prominent JFAC members had a different read of the report.

The survey results raise “important questions” about continued Istation funding, Sen. Dean Mortimer and Rep. Wendy Horman said in a Feb. 28 memo to JFAC colleagues. The two Idaho Falls Republicans have considerable sway over K-12 policy and spending; Mortimer chairs the Senate Education Committee and Horman plays a central role in writing the committee’s education budgets.

What happens next?

Two days earlier, JFAC grounded the Istation pilot, removing $100,000 from Ybarra’s budget. Voting unanimously, the budget-writers also sent Ybarra a clear message. They attached language to her budget forbidding her from spending any other money to “redesign or replace” the IRI.

Since then, the debate has gone into the background. Last week, a large group met to discuss next steps. Among those at the table: Mortimer, Horman, House Education Committee Chair Julie VanOrden; Sen. Steven Thayn, vice chairman of Senate Education; aides to Ybarra; Marilyn Whitney, an aide to Gov. Butch Otter; State Board of Education President Linda Clark; and lobbyists for several prominent education groups.

It will likely take legislation to resolve the issue, and so far, no bill has surfaced. “I still fully support replacing the legacy IRI, but talks about time frame and what that will look like are still ongoing,” Ybarra said Wednesday,

VanOrden also wants to replace the IRI, and she has for years. The Pingree Republican served on a state work group that studied literacy issues, an offshoot of Otter’s 2013 K-12 task force. The group recommended replacing the IRI.

VanOrden said she believes the Istation test gives teachers better data about their students’ reading skills — and in that regard, she said, almost any test would be an improvement over the IRI. Looking at the Boise State survey, she said she was most interested in the comments, and the mixed reviews teachers gave the Istation pilot.

And now, VanOrden and Horman both say they are skeptical about a statewide approach. They like the idea of letting local schools decide how to test their students.

“They’re the ones that know how to do it better than we do,” VanOrden said this week.

More coverage: Link to “Reading Revisited,” an Idaho Education News-Idaho Public Television series on the state’s literacy initiative.