Methods/Statistics
Considerations for synthetic control generation in difference-in-differences via Gaussian process: A use-case Hayden Smith* Hayden Smith UnityPoint Health – Des Moines
Background: Emergency Department (ED) patients that left without being seen (LWBS) present a clinical challenge. Evaluation of interventions to reduce LWBS rates requires methods to account for temporal trends. Conventional difference-in-differences (DID) approaches may be biased when pre-intervention trends are not parallel. Objective: To present an example of using a Gaussian process-based synthetic control DID (GPSDID) model and describe limitations.
Methods: A GPSDID framework was used to examine LWBS rates after the implementation of an ED staffing change. One-year of data was collected before and after the change from three EDs (i.e., study center and two control centers) in the same Midwestern health system. The GP was trained on pre-intervention data to capture nonlinear temporal trends and to generate a synthetic post-period counterfactual. Differences between the post-intervention LWBS rates and synthetic control were used to compute the average treatment effect on treated (ATT), with 95% credible intervals (Crl) derived from GP posteriors.
Results: There were 88,000 ED encounters across the reviewed centers. The GP-based synthetic control had nominal coverage of the pre-intervention observations (Figure). Post-intervention data failed to reveal an effect relative to synthetic control with ATT being 1.97% (95% CrI: -1.89%, 2.05%). Modeling considerations included the smoothness of trends, interpretability of latent functions, sparse controls, covariate adjustment, complexities for covariance in uncertainty measures, and extrapolation concerns – these topics will be expanded upon at the conference.
Conclusions: GPSDID provides a flexible and interpretable framework for assessing interventions by capturing nonlinear trends and providing uncertainty quantification via Bayesian posteriors. The approach may enable robust causal inference in complex settings and can be possibly applied to outcomes with dynamic trends. Of note, GP may not be without limitations.
