(EDITOR’S NOTE: I recently attended the Education Writers Association’s annual national seminar in Boston. I wrote about a panel discussion on teacher evaluations for the EWA’s blog. Here’s the story.)

The not-so-old teacher evaluation model was based on a “30-minute drive-by,” according to education policy expert Thomas Toch of Georgetown University.

In a white paper presented during the Education Writers Association’s national seminar in May, Toch laid out the standard operating procedure. Principals would show up with a cursory checklist, looking for “clean classrooms and quiet students.” They didn’t spend much time critiquing the teaching and learning process. Nearly every teacher received a satisfactory rating, and few were fired for poor job performance.

“Because public school teachers have traditionally been hired, paid, and promoted strictly on the basis of their college credentials and their years in the classroom, there were few incentives for school systems to thoughtfully compare teacher performance,” wrote Toch.

However, evaluations have changed rapidly — affecting pay and benefits for 3.1 million teachers and reshaping the way states and school districts distribute some half a trillion dollars of taxpayer money annually. Since 2009, 46 states have adopted more stringent evaluations systems, according to Toch. Nearly two dozen states use a combination of experience and performance as the basis for salary and tenure decisions; in some states, student performance accounts for up to 50 percent of an evaluation, Toch said.

The process is often complicated and contentious. And it’s still evolving.

The change has been “tumultuous,” Toch, a research fellow at Georgetown’s McCourt School of Public Policy, told the EWA audience during a panel on teacher evaluation. New systems are costly and prone to technical glitches, and state and district infrastructure has failed to keep pace with the changes. And that says nothing about the politics of the switch — and the effects on teacher morale.

For all the obstacles, Toch offered a generally optimistic assessment of the evaluations overhaul. Among early adopters — such as Tennessee, Cincinnati and Washington, D.C. — overall, student performance appears to be improving. (Last fall Washington was recognized as the nation’s fastest-improving urban school district based on student test scores.) Teacher morale also appears to be on an upswing, Toch said, since a more robust evaluation system creates the kind of professional community teachers crave.

By contrast, Elizabeth Davis of the Washington Teachers Union depicted teacher evaluation reform as a symptom of a reform agenda that has not yielded results in the District of Columbia.

The evaluation system has had a “devastating” effect on low-income schools, she said during the EWA panel, since the district’s achievement gap has widened.

The exact effects on teacher turnover are a matter of debate.

Davis estimated that 75 percent of teachers have turned over in some fashion since the city’s IMPACT evaluation system went into effect. As a result, she said, “The bad teacher narrative cannot possibly work in D.C. any longer.”

(Michelle Lerner, a spokeswoman for the District of Columbia Public Schools, later said the district retains 90 percent of teachers who are deemed “highly effective” or “effective” through their evaluations.)

The grading process

Framing of the issue has caused some of the problems, said Heather Peske, an associate commissioner at the Massachusetts Department of Elementary and Secondary Education. Supporters pitched the evaluations change as a way of weeding out poor teachers, she said, which is “such a small percentage” of what needs to be done.

In Massachusetts, the state worked with stakeholders, including the Massachusetts Teachers Association, on a plan first rolled out in the 2012-13 academic year. Administrators collect information on student performance — called a “student impact rating” — but this has no set weight in evaluations.

The grading process is another obstacle. Principals and administrators are trying to grade teachers objectively, but the learning curve is steep, according to Nathan Jones, an assistant professor at Boston University. Administrators need training to learn to conduct a rigorous but unbiased evaluation, he said. Observational scores will vary, depending on who conducts the observations.

Still, Jones said, administrators will be able to master this task — as long as they receive adequate support.

And what does the future hold?

That could well depend on how states implement the Every Student Succeeds Act — the far-reaching federal education measure signed into law by President Obama in December.

Here, Davis is hopeful. She said teachers want a hand in developing evaluation systems that help them improve their craft, and ESSA will give states more control over evaluations.

ESSA could also result in a shift in political momentum, emboldening opponents who have never grown comfortable with scoring systems that are based, in part, on student achievement and growth in test scores.

“You would expect to see challenges,” Peske said.

‘We just got here’

Even as state superintendents and local administrators start to examine ESSA and figure out what it allows, some officials are already making changes. One example is the District of Columbia, which has scaled back the percentage of evaluation scores that are based on student performance.

But the challenges to evaluation systems — and the changes to the frameworks — come at something of an odd time. After all, the rewritten systems have been on the books for only a few years.

As a nod to that, Jones of Boston University kicked off his PowerPoint presentation with a tongue-in-cheek title: “But wait, we just got here.”

Principals are learning to get a handle on the change, and they aren’t alone. Meanwhile, researchers are just beginning to assess the evaluations movement.

“There’s so much that remains needed to be answered,” Jones said.