Methods/Statistics
Trends in statistical results presentation of benefits and harms in abstracts of published randomized controlled trials with a focus on asymmetric framing Andreas Stang* Andreas Stang Jennifer Rauscher
Background: Asymmetric framing means that results on the benefits and harms of a new therapy are not presented in the same level of detail. In most cases, the benefits are presented in more detail than the harms. The aim of this study was to investigate how often results of randomized controlled trials (RCTs), phase 3, are presented asymmetrically in abstracts of peer-reviewed publications.
Methods: All 1225 abstracts of RCTs, phase 3, published in PubMed in 2000, 2010 and 2020 were identified. After excluding noneligible abstracts, 698 abstracts remained in the analysis. Two independent raters extracted features (presence of frequency and effect measures, confidence intervals, p-values and significance terminology for benefit and harm) of the abstracts.
Results: In the context of benefit, the presentation of effect measures (2000: 18.2%, 2010: 36.4%, 2020: 63.3%) and p-values (2000: 56.4%, 2010: 69.1%, 2020: 70.4%) increased over time; confidence intervals were reported in 80%, 63.2% and 56.7% in 2000, 2010, and 2020, respectively. The use of significance terminology decreased over time (2000: 54.6%, 2010: 50.6%, 2020: 47.2%). In the context of harm, the use of significance terminology has decreased (2000: 25.5%, 2010: 14.6%, 2020: 5.4%) and the reporting of the frequency of harm per study arm has increased (2000: 36.4%, 2010: 39.3%, 2020: 56.0%). A total of 89% of all abstracts provided benefit information (effect measures, confidence intervals, p-values, significance terminology) without corresponding harm information. Journals with higher impact factors tended to report more statistical features of both benefits and harms.
Conclusion: Even though the reporting of benefits and harms has improved somewhat, there is still a very strong asymmetry in the reporting of benefits and harms in abstracts of RCTs, phase III. Obviously, recommendations of the CONSORT statements are not sufficiently implemented.