Federal hospital safety measure fails to accurately assess emergency stroke care

A new UCLA study reveals that a widely used federal hospital safety metric is fundamentally flawed when applied to emergency stroke care, potentially creating incentives that may discourage hospitals from performing life-saving procedures on the sickest patients.

The research, published in Journal of NeuroInterventional Surgery, examined the Patient Safety Index 04 (PSI 04), a “failure to rescue” measure developed by the US Agency for Healthcare Research and Quality (AHRQ) to monitor deaths following treatable complications in surgical patients. The study analyzed data from the Nationwide Inpatient Sample covering 73,580 stroke thrombectomy procedures between 2016-2019, along with detailed consecutive case reviews at UCLA.

While it states that the measurement is appropriate for elective procedures performed on relatively healthy patients, the study found that the measurement is inappropriate for endovascular thrombectomy, an emergency procedure to remove blood clots in stroke patients who are already critically ill on admission.

“This metric was designed to identify preventable deaths, but when applied to emergency stroke care, it highlights unavoidable complications of severe strokes rather than problems with the procedure itself,” said Dr. Melissa Marie Reider-Demer, the study’s first author and UCLA Health DNP. “The unintended consequence is that hospitals that provide excellent care to the sickest patients may appear to have poor safety records.”

PSI 04 is activated when patients develop any of five complications after an operation (pneumonia, blood clots, sepsis, shock/cardiac arrest or gastrointestinal bleeding) and then die in hospital. The metric is used nationally for public reporting, hospital quality assessments, and pay-for-performance programs by Medicare and major organizations such as the Leapfrog Group.

The UCLA team analyzed both national data and detailed case reviews to assess the measurement’s suitability for stroke care. Their findings included:

• PSI 04 occurred in 20.5% of stroke thrombectomy patients nationally, which is one to three orders of magnitude higher than all other 17 patient safety indices (median: 0.10%)
• The rate for stroke operations was much higher than the 14.3% rate for all surgeries combined
• Among the 18 federal patient safety indicators, PSI 04 for all procedures had by far the highest rate of events, suggesting that the measure may be fundamentally flawed

At UCLA’s Comprehensive Stroke Center, researchers reviewed every case of thrombectomy flagged by PSI 04 between 2016-2018. A team of specialist neurointerventionists and neurologists reviewed each case and found:

• All patient deaths were related to complications of the major stroke and not to the thrombectomy procedure
• EVT procedures accounted for 7.2% of PSI 04 neurosurgical flags despite accounting for only 1.5% of neurosurgical procedures
• Neither case represented a real safety concern that could have been avoided

The study authors found the measurement to be flawed for two main reasons when applied to stroke thrombectomy:

1. The complications it tracks are often consequences of the severe strokes themselves, not the procedure. Patients who arrive with massive strokes are at high risk for pneumonia, blood clots, and other complications regardless of treatment.
2. Stroke patients are already in critical condition before the procedure, unlike patients undergoing elective surgeries. Even when complications arise, these critically ill patients have much less stamina to survive compared to relatively healthy surgical patients.

“We’re essentially penalizing hospitals for trying to save patients who are already dying of stroke,” Dr. Ryder-Demer said. “These procedures give severely affected patients their only chance of survival or functional recovery, but current metrics show that hospitals are providing poor care.”

Unexpected consequences

Researchers warn that inappropriate security measures can create perverse incentives. Previous research has shown that public reporting of surgical death rates has led some heart surgeons to select healthier patients to protect their performance ratings, limiting access for the sickest patients who need the most care.

“There is real concern that hospitals may be discouraged from performing thrombectomy on severe stroke patients, or that stroke centers with high volumes of critically ill patients could be unfairly penalized in quality assessments and reimbursement,” said Dr. UCLA Health.

This issue has become more pressing as recent clinical trials have extended thrombectomy to patients with even larger strokes, who have high mortality rates even with intervention, although still lower than without it.

A path forward

The Centers for Medicare & Medicaid Services proposed revising PSI 04 to exclude patients with acute conditions such as stroke coded as the primary reason for admission, with implementation planned for FY 2027.

In the opinion of Dr. Saver, the revision faces significant shortcomings.

“This revision makes clinical sense,” said Dr. Shaver. “Current measurement does not identify preventable events in stroke care and has the potential to mislead the public about hospital quality while creating incentives that could harm the sickest patients.”