Methods/Statistics
Robustness of effect estimates to varying model specifications: an application to statin use and risk of dementia using data from a large healthcare plan Erin Ferguson* Erin Ferguson Scott C Zimmerman Chen Jiang Minhyuk Choi Travis J Meyers Thomas J Hoffmann Paola Gilsanz Jingxuan Wang Akinyemi Oni-Orisan Rachel A Whitmer Neil Risch Ronald M Krauss Chirag Patel Catherine A Schaefer M Maria Glymour
Background: Model specification includes decisions such as sample eligibility; confounder selection; and operationalization of exposure and outcome measures. Effect estimates that are robust to a wide range of reasonable model specifications are most convincing. We assessed variation in the estimated association between statin initiation and risk of Alzheimer’s disease and related dementias (ADRD) across many model specifications.
Methods: For 1.5 million Kaiser Permanente Northern California members who initiated statins between 2000-2015, linked electronic health records from 2000-2020 were used to identify ICD diagnoses of ADRD. We compared estimated effect of statin initiation on ADRD incidence from Cox models across 8,200 distinct covariate combinations, considering demographics and clinical factors. We next compared estimates across 1,253 datasets holding the covariate set constant but varying the restriction of multiple enrollments, burn-in and re-enrollment periods, excluding participants with mild cognitive impairment, and changing trial follow-up duration.
Results: Across alternative covariate adjustment sets, the estimated HR for statin initiation ranged from 0.91 to 1.23. Adjusting for diabetes history increased the variance of point estimates (Fig 1A) while age adjustment increased point estimates (1B). Adjustment for other covariates (e.g. healthcare utilization (1C) and low-density lipoprotein cholesterol (1D)) had smaller impacts. Across all datasets using the most adjusted covariate set, the point estimate for statin initiation ranged from 0.94-1.39 (average HR 1.07; SD=0.05), and 61% of these estimates were statistically significant. The variance of estimates across all specifications (variance of ln(HR)=.004) was nearly 6 times larger than the variance of any specific estimate within a single specification (average variance of the ln(HR)=.0007).
Conclusions: The association between statin initiation and ADRD varied based on model specification.
