(UPDATED, 7:49 p.m. Thursday, with comment from Preston School District.)

Reading scores improved in the first year of Idaho’s $11.25 million “literacy initiative,” but most scores fell short of the goals set by local educators.

More than two-thirds of spring 2017 test scores failed to meet the benchmark goals school leaders set in the fall of 2016, according to an Idaho Education News analysis of test scores and local reading plans.

Schools aren’t penalized for falling short on their benchmarks — nor are they rewarded for meeting them. But the benchmarks provide the state one more yardstick as it embarks on a multiyear effort to provide extra help for at-risk readers.

However, some districts and charters didn’t set benchmarks. Several districts didn’t turn in reading plans at all — leaving their share of 2017-18 literacy money in limbo.

While some schools set ambitious improvement goals, a handful set goals that were actually lower than their preceding years’ reading scores. And the State Board of Education has no authority to challenge these goals.

The literacy initiative, so far

The 2016 Legislature addressed one of Idaho’s pervasive education challenges. Each fall, some 35,000 kindergartners through third-graders show up for school unable to read at grade level. Political and education leaders agree on one point; if students aren’t reading at grade level at the end of third grade, they are likely to struggle throughout the rest of their school years.

So lawmakers infused new money into the literacy initiative, to pay for 30 to 60 hours of extra help for young at-risk readers.

After the initiative became law, two things began  happening — on parallel tracks.

First, the state carved up the $11.25 million. The state based the payments on historic fall scores on the Idaho Reading Indicator. If districts and charters tend to have more at-risk readers, they would receive a bigger slice of the money. (Click here to download a spreadsheet breaking down 2016-17 payments.)

Meanwhile, districts and charters were supposed to submit literacy plans to the State Board of Education. The plans required a heavy accounting component. The state wanted schools to explain how they would spend their share of literacy money and provide the required 30 to 60 hours of reading help. The state also wanted schools to set benchmark goals.

Literacy dollars were never tied to literacy plans. Schools received their money even if they turned in an incomplete plan — or even if they turned in nothing.

To gauge the results of the first year of the reading initiative, Idaho Education News conducted an in-depth analysis of spring IRI scores and school literacy plans. Idaho Education News could not draw comparisons for every district and charter, at every grade level, but compared data points where available.

The key findings:

• Schools gained ground, compared to spring 2016 test scores. Kindergarten, second- and third-grade scores showed improvement. First-grade scores tended to drop from 2016, however. (Click here to download spreadsheets comparing 2016 and 2017 scores.)
• The 2017 scores did not measure up to benchmarks, at any grade level. Kindergarten scores came closest to meeting local goals. In first through third grade, barely a fourth of test scores met or exceeded benchmarks. (Click here to download spreadsheets comparing 2017 scores with benchmarks.)

Two thumbnails — and two different outcomes

In the Pocatello-Chubbuck School District, the spring IRI scores reflected a clean sweep. The district improved at every grade level and met each benchmark. Curriculum director Chuck Orr was quick to point to another milestone. Pocatello-Chubbuck also beat statewide averages across the board.

Pocatello-Chubbuck received $474,949 in literacy money. With this new money, the district focused on providing small-group instruction for struggling readers, and bolstered staffing by hiring paraprofessionals.

Not surprisingly, Pocatello-Chubbuck’s plan for 2017-18 calls for more of the same. Teachers are sold on the small-group approach. Now, it’s a matter of refining the strategy and training.

“We’ve got a good thing going here,” curriculum coordinator Kathy Luras said.

Nearly 500 miles away, the Coeur d’Alene School District received some sobering  numbers. Spring scores still topped the state averages. But Coeur d’Alene didn’t meet its benchmarks, and in every grade but kindergarten, scores dropped from 2016.

Coeur d’Alene’s literacy payment came to $321,307, and the district focused on teacher training.

Coeur d’Alene’s results aren’t necessarily a surprise. The district’s new in-house reading strategy emphasizes comprehension. Teachers don’t just want their kids reading quickly, elementary education director Kate Orozco said; they want kids reading “carefully.” However, that strategy runs at odds with the current IRI, which emphasizes speed.

Orozco hopes a new IRI will measure comprehension as well as speed. This year, the new assessment will get a pilot test in 57 schools, including three in Coeur d’Alene.

“When scores drop, it’s always a concern,” she said. “(But) we need to rely on an assessment that accurately assesses our kids’ reading ability.”

Changing plans, and missing data

As Idaho’s third largest district — with a relatively high number of at-risk readers — Nampa is a large beneficiary of the reading initiative. The district received $670,049 in literacy money in 2016-17.

Nampa did not set clear reading benchmarks in its initial reading plan, saying it would look to compare fall and spring IRI scores “as a measure of effectiveness.” In an updated plan submitted to the State Board, Nampa said it was aiming for 5 percent improvement.

Nampa hit this benchmark in kindergarten and second grade.

But several districts and charters never set a goal, even on the fly. Some said they would use IRI scores as their benchmark, but did not spell out a goal. Other reading plans failed to address benchmarks at all.

The State Board worked with districts and charters to make sure their documents met state law. “Overall, districts were making a sincere effort to be compliant,” State Board spokesman Blake Youde said.

Eighty-nine districts and charters submitted plans that met the law, or needed minor tweaking, he said.

However, eight districts never submitted a plan. They missed a final deadline in July, more than nine months after their original Oct. 1 deadline. Now, these districts won’t receive their 2017-18 literacy money until they submit a plan that meets legal muster, Youde said.

This list includes three of Idaho’s tiniest and most remote school districts — Avery, Pleasant Valley and Prairie — and rural districts in Butte County, Grace, Midvale, Orofino and Preston. All told, these districts received more than $185,000 in literacy money; Preston alone received $101,818.

Preston superintendent Marc Gee said the district turned in a plan, but he conceded it was “very late.”

“Part of our concern was the turnaround time on creating a literacy plan that would be an actual enhancement of what we were doing, not just throwing money at the problem,” Gee said in an email.

Modest goals

Most of the schools that set benchmarks looked to compare spring 2016 and spring 2017 test scores. This aligns with the state, which tracks year-to-year trends in statewide scores.

One exception is the Vallivue School District. Students in the Canyon County district are highly mobile, so Superintendent Pat Charlton says he sees little value in spring-to-spring comparisons. Instead, Vallivue used a shorter window, and sought 5 percent improvements from the fall 2016 IRI.

In a classroom of 20 students, a 5 percent improvement translates to one student advancing to grade level during the course of the year.

And even before 2016-17, and the infusion of $321,307 in literacy money, Vallivue generally exceeded 5 percent improvements in 2014-15 and 2015-16. The district’s 2016-17 results were similar; once again, Vallivue had at least 5 percent improvement in every grade except first grade.

In some schools, however, their goals don’t call for any growth.

Bannock County’s Marsh Valley School District said it would use its $42,678 to try to get 60 percent of students to grade level. But the district has met this threshold every year, in every grade, since at least 2012-13. (Marsh Valley Superintendent Marvin Hansen did not respond to a request for comment.)

Nampa’s Victory Charter School, which received $4,573 in literacy money, also set 2016-17 benchmarks below its 2015-16 scores. Gayle O’Donahue, the school’s federal programs and community relations coordinator, blamed the glitch on last year’s rapid rollout of the literacy initiative. Future goals, she said, will be “more in line with our overall educational objectives and outcomes.”

Administrator Sue Smith makes no apologies for Upper Carmen Public Charter School’s 80 percent benchmark — even though 100 percent of her students read at grade level the preceding two years.

“I’m not always going to have 100 percent. That’s not possible,” Smith said. “Why would 80 percent be too low of a goal?”

Upper Carmen received $1,829 in literacy money in 2016-17 — based on the handful of students who scored below grade level in fall IRI tests. The small Lemhi County school hit its 80 percent benchmarks.

Idaho law doesn’t give the State Board the power to second-guess benchmarks — or tell schools they are aiming too low. Besides, said Youde, local administrators have a better handle on local demographics, and are best equipped to set realistic goals.

“They know their students better than we do,” he said.

What’s next for the state

In 2016-17, the State Board concentrated on logistics: doublechecking literacy plans and making sure the schools spent their share of the $11.25 million in accordance with the law. Starting in 2017-18, the board hopes to put data to use.

State Board staff will compare 2016-17 IRI scores against scores that predate the literacy initiative. Staff will also compare scores against benchmarks.

And now that it has “a library of plans,” Youde said the State Board should also be able to suggest successful strategies to struggling schools.

The State Board will collect 2017-18 literacy plans that could look a lot like the 2016-17 plans. Districts and charters will again need to submit a budget, an intervention plan for at-risk readers and benchmarks.

This comes after the 2017 Legislature haggled over school reporting requirements. The Senate Education Committee voted to get rid of the literacy plan requirement entirely, calling the paperwork burdensome. The House Education Committee didn’t buy off, and the reporting requirements remained intact.

What’s next for schools

In Idaho Falls, 87 percent of kindergartners hit grade level on the spring IRI — not quite the district’s 90 percent goal, but well above the 46 percent of students that started 2016-17 at grade level.

“It gives us a starting point,” said Kelly Coughenour, the district’s director of elementary education.

Now, Idaho Falls will start taking the long view. Some of its $463,364 in literacy money went to voluntary summer programs — in hopes of keeping students at grade level at the start of the 2017-18 school year.

With another $11.25 million on its way for 2017-18 — and hopes that the literacy program will remain an ongoing priority — educators are looking past last year’s startup.

The West Ada School District met its 2017 benchmarks only in kindergarten, although grades were up from 2016 in every grade but first grade. But in essence, West Ada also treated 2016-17 as a pilot year. Since then, the state’s largest district spent the summer digging into local test scores, trying to figure out where it got its best returns from its $907,217 in literacy money.

That will inform West Ada’s 2017-18 strategy.

“We’ve got much better ideas of where we want to focus our resources,” said Joe Kelly, West Ada’s director of assessment and accountability.

For 2017-18, the Bonneville School District has added extended kindergarten programs for at-risk readers. That wasn’t an option in 2016-17. District officials knew they had a new $457,877 on the way, but they didn’t have time to tweak bus routes for extended kindergarten sessions. The district implemented what it could in a limited amount of time.

“That’s not the right way to roll things out … if you want to have any meaningful impact in the first year,” said assistant superintendent Scott Woolstenhulme.

Idaho Education News data analyst Randy Schrader contributed to this report.

More reading 

• 2017 reading scores hint at an urban-rural achievement gap.
• Check out “Reading Revisited,” an Idaho Education News-Idaho Public Television series on the literacy initiative.