LATEBREAKER
Methods/Statistics
Difference-in-Differences with Unpoolable Data (UN-DiD) Nichole Austin* Nichole Austin Erin Strumpf Sunny Karim Matthew Webb
Background: Difference-in-differences (DiD) is an indispensable tool to estimate the effects of health policies and interventions. However, data sources are sometimes siloed by jurisdiction and are therefore considered unpoolable. This is a significant barrier to DiD estimation; no methods currently exist for conducting DiD analyses in these scenarios.
Methods: We propose an innovative approach to estimate DiD with unpoolable data. UN-DiD uses a weighted combination of siloed effect estimates and standard errors to recover pooled estimates. We begin with a simple two group two period case with no covariates, and then extend our analysis to include a single time-invariant covariate, and a single time varying covariate. We extend the UN-DiD model to more complex/realistic settings with multiple treated and control groups, multiple time periods, and various data generating processes. We use Monte Carlo simulations to compare UN-DID to conventional DiD before applying our estimator to an empirical example.
Results: Assuming a sufficiently large sample, we show the mathematical equivalence of our estimator with the conventional pooled estimator in varying data settings reflecting common real-world conditions. The UN-DiD estimate of the ATT is equivalent to the conventional estimate in the simple case, with no covariates and with time-invariant covariates. The estimates are no longer exactly equivalent in the presence of time varying covariates, but the estimates of the two methods (and the associated standard errors) converge as sample size increases.
Discussion: Our UN-DiD method can be used to estimate treatment effects when data are siloed by treatment and control units. This will facilitate policy evaluation when data cannot be combined across countries or other jurisdictional boundaries.