Developing and testing models for COVID-19 health outcomes
In the Netherlands, we conducted 3 focus groups and 4 individual interviews with providers (n = 6) and with people who had had COVID-19 and their surrogates (n = 9) between May and July 2021. Clinicians expressed concern about data accuracy and validity as well as a belief that patients may interpre...
| Main Author: | |
|---|---|
| Corporate Author: | |
| Format: | eBook |
| Language: | English |
| Published: |
[Washington, D.C.]
Patient-Centered Outcomes Research Institute (PCORI)
2023, [2023 Mar]
|
| Series: | Final research report
|
| Online Access: | |
| Collection: | National Center for Biotechnology Information - Collection details see MPG.ReNa |
| Summary: | In the Netherlands, we conducted 3 focus groups and 4 individual interviews with providers (n = 6) and with people who had had COVID-19 and their surrogates (n = 9) between May and July 2021. Clinicians expressed concern about data accuracy and validity as well as a belief that patients may interpret the data as absolutes rather than estimates. Health care professionals described CPMs as being useful for resource allocation, triaging, education, and research. Most people who had had COVID-19 and their surrogates were not given prognostic estimates but believed that this information would have supported and influenced their decision-making. CONCLUSIONS: The NOCOS performed moderately worse than the COPE on second-wave data, both at temporal and geographic validation, likely reflecting unique aspects of the early pandemic in NYC. Frequent updating of prognostic models is required to warrant transportability over time and space, particularly during a dynamic pandemic. Although discrimination was adequate when NOCOS mortality predictions were evaluated against second-wave data in NYC (AUC, 0.76), NOCOS systematically overestimated the mortality risk (E statistic, 5.1%). Discrimination in the second-wave Dutch data was good (AUC, 0.81) but with overprediction of risk, particularly in lower-risk patients (E statistic, 4.0%). Recalibration of COPE and NOCOS led to negligible net benefit improvement in Dutch data but substantial net benefit improvement in NYC data. Predictions of the need for MV or ICU admission had a pattern of performance similar to mortality predictions, but models predicting outcomes among the subset of patients admitted to the ICU performed poorly. In the United States, we conducted 4 online focus groups with health care professionals (n = 9) and with people who had had COVID-19 and their surrogates (n = 12) between January 2021 and July 2021. Online focus groups and interviews were conducted among health care professionals, people who had had COVID-19, and surrogates in the United States and the Netherlands. Semistructured questions were used to explore experiences about clinical decision-making in COVID-19 care as well as facilitators and barriers for implementing CPMs. RESULTS: Twenty-eight-day mortality was considerably higher in the NYC first-wave data (19.8%) than the second-wave (9.6%) and Dutch data (first wave, 10.8%; second wave, 10.0%). The COPE mortality predictions discriminated well at temporal validation (area under the curve [AUC], 0.82), with excellent calibration (E statistic, 0.8%). At geographic validation restricted to the second wave, COPE mortality predictions had satisfactory discrimination (AUC, 0.77) but with moderate overprediction of mortality risk, particularly in higher-risk patients (E statistic, 2.9%). BACKGROUND: Supporting decisions that patients who present at the emergency department with COVID-19 make requires accurate prognostication, but a highly dynamic pandemic poses special challenges for predicting patient outcomes. OBJECTIVES: We aimed to develop clinical prediction models (CPMs) to support shared decision-making for COVID-19 care. We also aimed to evaluate geographic transportability by assessing model performance across different data sets (from different countries) as well as temporal transportability by assessing model performance within the same data set across time periods. We convened focus groups with COVID-19 care providers, survivors, and surrogates to elicit feedback about care-related decision-making during the COVID-19 pandemic. METHODS: Clinical prediction models to predict the probability of (1) mortality, (2) whether a patient will require mechanical ventilation (MV) or intensive care unit (ICU) admission, (3) mortality if a patient is placed on MV, and (4) length of stay in the ICU were developed in each of 2 hospital systems using easily obtainable variables at the time of admission to the emergency department. The Northwell COVID-19 Survival (NOCOS) models were developed based on New York City (NYC) data, and the COVID-19 Outcome Prediction in the Emergency Department (COPE) models were developed based on data from the Netherlands, both using data from the first wave (March 2020 through August 2020). For temporal validation, all models were tested on second-wave data (September 2020 through December 2020) within the same set of hospitals as well as on both first- and second-wave data from the alternative country. Many health care professionals had reservations about using CPMs for people with COVID-19 because of concerns about data accuracy and patient-level data interpretation. Several people who had had COVID-19 and their surrogates, however, indicated that they would have found this information useful for decision-making. LIMITATIONS: The performance of these models, as measured in second-wave data, may not apply currently because the pandemic has continued to evolve. In particular, the widespread dissemination of vaccines may well affect clinical presentation, patient risk, and predictor effects |
|---|---|
| Physical Description: | 1 PDF file (88 pages) illustrations |