Statistical methods for reducing bias in comparative effectiveness research when using patient data from doctor visits
LIMITATIONS: While our work developed extensive theory to assess the effects of outcome-dependent visits, many of our results were based on simulation studies in which we introduced known degrees of bias. As with any simulation-based approach, the results cannot be known to apply in full generality....
| Main Authors: | , |
|---|---|
| Corporate Author: | |
| Format: | eBook |
| Language: | English |
| Published: |
Washington (DC)
Patient-Centered Outcomes Research Institute
[2019], 2019
|
| Series: | Final research report
|
| Online Access: | |
| Collection: | National Center for Biotechnology Information - Collection details see MPG.ReNa |
| Summary: | LIMITATIONS: While our work developed extensive theory to assess the effects of outcome-dependent visits, many of our results were based on simulation studies in which we introduced known degrees of bias. As with any simulation-based approach, the results cannot be known to apply in full generality. However, the breadth of our simulations, the approximate calculations, and sensitivity analyses all support the primary conclusions BACKGROUND: Clinical databases, such as those collected in electronic health records systems, are increasingly being used for patient-centered outcomes research. Because the number and timing of visits in such data are often driven by patient characteristics and may be related to the outcomes being measured (which we call an outcome-dependent visit process), the danger is that this will result in biased and misleading conclusions compared with analyses from designed, prospective studies. Most, if not all, of the extant statistical methodology relies on unrealistic models for the outcome-dependent visit process. We also developed methods to diagnose the outcome dependence and showed that these diagnostic methods have high power to detect outcome-dependent visit processes. Furthermore, high power was achieved using these methods before maximum likelihood-based statistical analysis methods exhibited significant bias. CONCLUSIONS: The results of our research give practical guidance in the validity of inferences from outcome-dependent visit processes. When data are subject to outcome dependence, bias is restricted to a subset of the covariates (those with associated random effects in the outcome model). Standard, maximum likelihood-based methods such as mixed-model regression often exhibited little bias, even for parameters with associated random effects. Generalized estimating equations methods, especially those based on an independence working correlation, were more susceptible to bias. The diagnostic methods we developed have high power to detect outcome-dependent visit processes. OBJECTIVES: Our goals were to (1) develop realistic models for outcome-dependent visit time models, (2) use theoretical calculations and simulation models to assess bias and efficiency in longitudinal statistical analyses applied to outcome-dependent visit databases, (3) provide guidance as to which types of statistical inferences are accurate and exhibit little bias when using databases collected with outcome-dependent visit times vs those that are likely to be inaccurate, and (4) make recommendations of how to deal with outcome-dependent visit processes in clinical research. METHODS: We used semistructured interviews with clinician-scientists and analysis of their visit pattern data to develop realistic models for the connections between the likelihood of a visit and the outcome process. Using these realistic models, we assessed the performance of standard statistical methods, such as mixed-model regression, as well as analysis methods that purported to deal with outcome dependence. We used theory and simulations to evaluate the bias as well as the impact of including a small number of regularly scheduled visits. We used a wide variety of sensitivity analyses to determine the generality of the results. Using theory and simulation we also developed and evaluated the performance of methods to diagnose the outcome dependence. RESULTS: Analysis approaches designed to deal with outcome dependence fared no better and, for some methods, worse than ignoring the outcome dependence and using standard, mixed-model analysis methods. The bias using standard methods under outcome-dependent visit processes is mostly confined to covariates that have associated random effects. A wide variety of sensitivity analyses confirm the generality of the results. To address this bias, we showed that inclusion of a few, non-outcome dependent observations can significantly reduce the bias when using maximum likelihood fitting methods. |
|---|---|
| Physical Description: | 1 PDF file (123 pages) illustrations |