Methods/Statistics
Adjusting for unmeasured confounders with validation data using disease risk score calibration: a simulation study Xiaojuan Liu* Xiaojuan Liu Liu Liu Brigham and Women’s Hospital
Background: Leveraging a validation sub-cohort containing confounders unmeasured in the main cohort is increasingly used to reduce bias in observational studies. Existing approaches, such as propensity score (PS) calibration and two-stage calibration, rely on modeling the exposure mechanism and require strong assumptions. We propose an alternative calibration framework using a gold-standard disease risk score (DRS) from the validation cohort.
Methods: We consider a main cohort containing exposure, outcome, and measured confounders X, and a nested validation sub-cohort with an additional confounder U. In the sub-cohort, we defined a gold-standard DRS as the linear predictor of the outcome regression model conditional on X and U. An error-prone DRS constructed from X alone is available in both cohorts. A calibration function mapping the error-prone DRS to the gold-standard DRS is estimated using validation data and applied to the main cohort to obtain predicted values. The exposure effect is then estimated by adjusting for the calibrated DRS. We compared the performance of DRS, PS, and two-stage calibration.
Results: Across 1,000 Monte Carlo replications (main cohort n=10,000; validation n=1,000) under a null scenario with moderate unmeasured confounding, naïve adjustment yielded substantial bias (median odds ratio [OR]=1.61). DRS calibration substantially reduced bias to near null (median OR=0.99) with lower mean squared error (MSE = 0.03) compared with PS calibration (median OR=1.40, MSE = 0.11). Two-stage calibration achieved the lowest bias (median OR=1.00) and MSE (0.01).
Conclusions: PS calibration showed instability and residual bias when strong assumptions required for PS calibration were not met. DRS calibration relaxes some assumptions and provides a more robust alternative, substantially reducing bias, though with greater variability than two-stage calibration. DRS calibration may complement existing methods when assumptions for PS calibration are not met.