From NPR:
COVID-19 Hospital Data System That Bypasses CDC Plagued By Delays, Inaccuracies
Earlier this month, when the Trump administration told hospitals to send crucial data about coronavirus cases and intensive care capacity to a new online system, it promised the change would be worth it. The data would be more complete and transparent and an improvement over the old platform run by the Centers for Disease Control and Prevention, administration officials said.
Instead, the public data hub created under the new system is updated erratically and is rife with inconsistencies and errors, data analysts say.
Lawmakers planned to grill members of the White House Coronavirus Task Force about the reporting change and what is being done to ensure the data remains public and reliable, in a Friday morning hearing of the House Select Subcommittee on the Coronavirus. Several House subcommittees have already launched an investigation into the data change.
The delays and problems with data on the availability of beds, ventilators and safety equipment could have profound consequences as infections and deaths soar throughout most of the country, public health experts say.
"If the information is not accurate, it could cost time — and lives," says Lisa M. Lee, formerly the chief science officer for public health surveillance at CDC. She is now at Virginia Tech. For instance, knowing which hospitals have the capacity to take on new patients is critical, she explains. "If all the ICU beds are taken up, emergency medical personnel need to take [new patients] to the next town over or to the next county."
The Department of Health and Human Services is standing by its decision. While some hospital associations and states have had "difficulty," HHS has worked to quickly resolve errors, and they "are pleased with the progress we have made during this transition and the actionable data it is providing," a spokesperson wrote to NPR in a statement.
Hospitals report to an untested contractor
When infection numbers start to rise in a state, public health officials and residents worry about whether there are enough hospital beds for seriously ill patients. An area's hospital capacity is an estimate, fed by the data that individual hospitals report. Public health officials rely on the estimates to see whether hospitals are at risk of being overwhelmed.
Hospitals are supposed to report daily to the federal government how many beds they have, the number occupied and the availability of intensive care beds. Under the new system, the Department of Health and Human Services aggregates the information at a state level and shares a daily spreadsheet of the information that has been reported — gaps and all.
But the old CDC approach interpreted the data a step further. CDC posted estimates derived from the data to show an approximation of the actual availability of ICU beds, accounting for the lags and gaps in reporting. These estimates — promised on the HHS website — have not been updated in over a week.
By contrast, the CDC estimates was updated three times a week. And while the data sent to CDC was vetted for accuracy before being posted publicly, the data sent to the new platform appears to be posted as it is received and contains multiple anomalies, analysts note.
Since the beginning of the pandemic, hospitals have been reporting bed capacity and other key data to CDC through its National Healthcare Safety Network COVID-19 module. No hospital staff interviewed by NPR described problems with this system. They said that it had been used by hospitals for 15 years to monitor the impact of other illnesses and that staff are comfortable with it.
The established system was disrupted by a memo dated July 10, issued to hospitals by HHS. In the memo, HHS took the unusual step of instructing hospitals to stop reporting the capacity data to CDC and to instead use a reporting platform developed recently by private contractor TeleTracking. As NPR has reported, the details of how the contract was awarded to TeleTracking are unclear.
Hospitals received only a few days notice of the change and scrambled to adapt.
The sudden reporting switch to HHS, bypassing CDC, prompted an immediate outcry. Public health groups objected to the change as unnecessary and burdensome for hospitals in the midst of a pandemic. Open government organizations demanded that HHS rescind its guidance and return control of the data to CDC. On Tuesday, 22 state attorneys general added to the chorus of those demanding that HHS reverse course.
Michael Caputo, HHS assistant secretary for public affairs, framed the switch at the time in a statement to NPR as a technology upgrade from "the CDC's old data gathering operation," which "just cannot keep up with this pandemic."
"The new faster and complete data system is what our nation needs to defeat the coronavirus," Caputo wrote.
The promise of better data, unfulfilled
The data now available to the public appears to be neither faster nor more complete.
When HHS took over the collection and reporting of this hospital capacity data, it promised to update "multiple times each day." Later, the agency walked that back to say it would be updated daily.
Those daily updates have yet to materialize. On Thursday, an HHS spokesperson told NPR via email, "We will be updating the site to make it clear that the estimates are only updated weekly."
The HHS Protect Public Data Hub, the public-facing website set up by HHS, offers three items as a "Hospital Utilization Snapshot," all of which have data that is over a week old.
A "Downloadable Dataset" estimating how many hospital beds are occupied by state — last updated on July 21.
A table tallying the total number of hospital beds occupied across the country, which has not been updated since July 23.
A map showing the percent of hospital beds occupied by state, which has not been updated since July 23.
Still, hospitals are required to report this data daily to TeleTracking, either directly or via their state health department. Failure to do so could affect the state's access to remdesivir, one of the few drugs that has shown promise in treating severe COVID-19 cases, according to the HHS guidance to hospitals and the American Hospital Association.
Ryan Panchadsaram, co-founder of the tracking website CovidExitStrategy.org, expressed frustration over the delays in a tweet: "2 weeks since the switch from @HHSGov to @CDCgov...Daily updates never happened... 29 states see increased hospitalizations & this critical indicator is still missing."
Moving the goal posts?
The only information about hospital capacity that appears to be updated regularly on the HHS Protect site is the percentage of hospitals that have submitted data in the past seven days.
HHS's Caputo wrote in a statement to NPR in mid-July that CDC's voluntary reporting system "provided data from only 85 percent of hospitals; the President's COVID response requires 100 percent to report."
The new dataset does not show 100% compliance. According to the dataset posted on July 30, 92% of hospitals have reported some information to HHS in the past week, ranging from 75% in Delaware to 100% in eight states, including Nevada, Iowa, and Minnesota.
Asked about compliance, an HHS spokesperson told NPR Thursday, "The consistency is improving, and we are proud of the progress that has been made."
But the higher level of compliance is not the whole picture, according to a CDC official familiar with the former reporting system. The tallies do not include certain categories of hospitals, including rehabilitation or veterans' hospitals, which have suffered COVID-19 outbreaks. These rehabilitation and veterans' hospitals had previously been included in the data reported by CDC, says the official, who spoke to NPR on background because they were not authorized to speak on the record.
"It's not unusual to split out categories of hospitals and only focus on the subset of acute care hospitals," says Thomas Tsai, an assistant professor of health policy and management at Harvard University. Tsai pointed out that general hospitals care for the bulk of COVID-19 cases, and patients in alternate systems like military and veterans hospitals access COVID-19 care differently than the broader population.
Still, omitting these hospitals from the count means better compliance with the data mandate can be attained with fewer hospitals reporting.
"These hospitals are not on our list of priority hospitals that we are seeking daily information from because they treat very few, if any, COVID-19 patients," an HHS spokesperson explained in response to questions about the omitted facilities.
Anomalies and inconsistencies
After the data reporting switch, unusual numbers started cropping up in data that show how many hospital beds are filled in a given state, data analysts say. In some states, the bed occupancy rates soared, even though the number of hospitalized COVID-19 patients dropped or only increased modestly.
Take, for example, Arizona. Under the old system, in data last collected by CDC on July 14, an estimated 3,205 COVID-19 patients in Arizona occupied 24% of the state's inpatient hospital beds. After the switch to the new HHS reporting system, an analogous dataset posted by HHS showed 82 fewer COVID-19 patients hospitalized, but the bed occupancy rate had jumped to 42%. It's unclear how fewer patients could be occupying more hospital capacity.
There are similar anomalies in the data for other states, including Georgia and New Mexico.
In Colorado, the hospitalization data maintained by HHS conflicts with the state's data posted to a daily dashboard. As of July 30, the state dashboard lists 341 patients hospitalized in Colorado with confirmed or suspected COVID-19 cases. A dataset maintained by the HHS, updated on July 30, lists 491 patients in Colorado.
"There's either a difference in how we're accounting or there's a difference in how we're reporting," Dr. Stephen Cantrill, an emergency physician at Denver Health, a health care system in Colorado. "Is this a systemic issue or is this just a one-state issue?"
Cantrill says he has been hearing about data inconsistencies from colleagues in other states.
COVID-19 tracking websites that had relied on the data collected and reported by CDC have also found anomalies. In a post, members of The COVID Tracking Project from The Atlantic describe the hospital capacity data as being "highly erratic in recent weeks," and noted that data has been missing or incomplete from many states, including California, Texas, South Carolina, Idaho, Missouri and Wyoming, because of complications related to switching reporting systems.
An agency spokesperson acknowledged that some facilities have had trouble making that switch, suggesting that this is the cause of the data anomalies. "When HHS identifies errors in the data submissions, we work directly with the state or hospital association to quickly resolve them," an agency spokesperson wrote to NPR. "The goal of full transparency is to acknowledge when we find discrepancies in the data and correct them."
The organizers of the tracking website COVID Exit Strategy initially found the data provided by HHS Protect to be unusable. "It had some states like Rhode Island having an inpatient bed utilization of above 100%," says site co-founder Ryan Panchadsaram. "And Rhode Island is a state where hospitalizations are quite low for COVID."
The site stopped pulling hospital capacity data from HHS Protect for several days but resumed the reporting on July 30, using a dataset from HealthData.gov with a "user beware" warning that reads in part: "For the time being, we recommend cross-checking these numbers with the ones reported on a state's official website to get an accurate picture."
"Everyone can see where the inconsistencies are," says Panchadsaram. "The public can see it. The states can see it. The federal government can see it."
"We're sort of stuck in this moment where it's like a lot of debugging and a lot of aligning of data is happening," he says. He's hoping, though, that the issues will be resolved soon.