Do Tarrant County hospitals make the grade in safety?
Scores have safety in mind; critics say they're based on flawed data
People pass through hospital doors and liken the experience to getting off an airplane in a foreign country. Uncertainty can give a sense of unease, as patients, surrounded by strangers who use words they may not understand, worry about risks they can't anticipate.
Authorities say more than 400 Americans die every day because of preventable accidents, errors and infections suffered in hospitals. And despite various efforts to make hospital operations more transparent, those who seek healing have a hard time judging their chances of being hurt.
To try to make that easier, a consortium representing major employers has issued A through F safety grades for hospitals nationwide.
The grading system, developed by nine healthcare quality experts for The Leapfrog Group, gives consumers a glimpse of what is going on behind hospital doors before they have to go through those doors themselves, Leapfrog CEO Leah Binder said.
"We looked at all the information that is publically available on patient safety -- all the things that you wished never happened at hospitals but frequently do," Binder said.
Some information was culled from Leapfrog's own hospital surveys, and some was extracted from federal Centers for Medicare and Medicaid Services data. Reviewers looked not only at hospital-acquired infection rates, for example, but also at nursing staffing and hand hygiene.
The ABC scores
An A grade went to hospitals that well exceeded national averages, Binder said. Hospitals across the service spectrum received top marks, including some that serve vulnerable and impoverished populations, such as Bellevue Hospital Center in New York and Detroit Receiving Hospital. Five hospitals in Tarrant County got A's.
C's went to those scoring below national averages on 26 evidence-based measures, Leapfrog says. Getting C's were some of the nation's most renowned hospitals, as well as five Tarrant County hospitals.
Some hospitals in Tarrant County, such as Medical Center of Arlington, lost points because of shortcomings identified in the Leapfrog survey on safe practice measures. MCA scored 28 points out of a possible 40 points on teamwork training and skill building, a category Leapfrog says is important because care can become fragmented if communication fails. North Hills Hospital fell short on some safe practice measures, such as having an adequate staff of registered nurses.
Some Tarrant hospitals have had a rate of falls and other trauma that topped national averages, and others reported patients who suffered severe bed sores or urinary tract infections from catheters. Some hospitals, such as JPS Health Network, received a low score on incidences of blood clots after surgery, according to Leapfrog. That could mean that doctors aren't taking enough preventive steps, Leapfrog says.
Data, method criticized
As has been typical with other hospital rating and ranking projects, critics say Leapfrog's effort is imperfect at best.
"Many of the measures Leapfrog uses to grade hospitals are flawed," said Steve Love, president and CEO of the Dallas-Fort Worth Hospital Council. "Leapfrog uses a group of 26 measures, and of those 11 are scores that are derived from a survey. Some hospitals do not participate. Instead, some subjectivity goes into the measure. There are a lot of patient safety measures that should be included that Leapfrog does not take into account."
Many healthcare administrators in the Tarrant County area agreed, saying that rating hospitals is complex, and that some of the conclusions were subjective, incomplete and failed to recognize progress made after the analysis was completed.
Mark Lester, a physician appointed interim chief quality officer for Texas Health Resources, noted that the data used for the scoring was anywhere from six months to several years old. He said that even physicians have difficulty keeping up with everything going on at a particular hospital.
The time lag with the data creates unintended consequences, said Don Kennerly, a physician and associate chief quality officer for patient safety in the Baylor Health Care System. Sometimes the problem is fixed before the data is released, and information about a how a hospital has solved a problem can take many more months to reach the consumer, he said.
But such criticisms should not be taken as complaints, Lester said.
"The Leapfrog survey gives us a snapshot in time," he said. "It gives us an idea on how to improve. Quality improvement is a complex undertaking with many, many facets. The more information we have, the better opportunity we have to make things better."
Other hospital officials also emphasized the complexity of gauging safety.
The grades issued by Leapfrog are best used as a way to begin a conversation about quality improvement, but such rating systems can be confusing, said Steve Newton, president of Baylor All Saints Medical Center in Fort Worth.
The hospital received a C on the Leapfrog safety assessment. Points were deducted in safe practice areas such as leadership, which Leapfrog experts say can cause inconsistent execution.
Newton pointed out, though, that All Saints received awards this year from HealthGrades, an independent healthcare ratings company, for patient safety and patient experience.
Too complex for grades
A single tool that will give consumers a simple, easily accessible way to compare hospitals is not yet available. "Hopefully we will get there in the next three to five years," Newton said.
Robert Earley, CEO of JPS, sees the simplicity of the grades as a flaw.
"Unfortunately, there is no one indisputable measure or grade for patient safety or hospital performance," Earley said in an e-mail. "Healthcare monitoring is complex and is often different from organization to organization and state to state."
Leapfrog members, frustrated with the slow progress in improving patient safety measurements, have heard many of the reasons physicians and hospital administrators are critical about efforts to compare hospitals, Binder said. But she said no system is perfect.
"I have a 5-year-old and a 12-year-old, and I consult the movie ratings," Binder said. "I don't want to see all the movies they may want to see beforehand so I can make a judgment. I know the movie rating system is not perfect, but I will not let my preteens go and see a PG-rated movie without me."
Kennerly would tell her that the job of calculating movie ratings is easier. "What's being asked by Leapfrog is the overall judgment concerning whether this is a good hospital. And that's a little harder," he said.
The Hospital Safety Score will be reissued using updated data in November, with an annual score to follow in 2013 and beyond, The Leapfrog Group said.
Mitch Mitchell, 817-390-7752