Infectious Disease
In-season evaluation of influenza forecasts during the 2023-2024 US influenza season Sarabeth M. Mathis* Sarabeth Mathis Rebecca K. Borchering Alexander E. Webber Matthew Biggerstaff FluSight Influenza Forecasting Consortium
Influenza-related hospitalizations are a major contributor to the disease burden in the United States with CDC estimating 100,000 to 710,000 hospitalizations annually between 2010 and 2022. Accurate predictions of influenza hospital admissions can help support public health planning and interventions. CDC coordinated a collaborative forecasting challenge for the 2023-24 influenza season beginning October 11, 2023, through April 30, 2024. Forecasting teams were asked to provide national and jurisdiction-specific probabilistic predictions for weekly laboratory-confirmed influenza hospital admissions during the 2023-24 influenza season.
Similar to previous years, FluSight prepared an ensemble and baseline model each week. The ensemble calculates the unweighted median of each quantile among eligible forecasts, and the baseline estimates the most recent observed weekly admission count as the median for all forecast horizons. A distribution of historical change around the median is used to determine prediction intervals.
Forecast skill was evaluated on absolute weighted interval score (WIS), a proper score that generates interval scores for probabilistic forecasts provided in the quantile format, WIS relative to the baseline model, and coverage, a metric that assess how often the prediction interval contained the actual observed value.
As of January 3, 2024, an average of 24 modeling teams submitted 25-30 models each week. Eight models outperformed the FluSight ensemble in terms of WIS and 27 models outperformed the FluSight baseline including the FluSight ensemble. 95% coverage was over 90% for five teams. Forecast overage and accuracy during the initial period influenza hospitalizations were increasing improved compared to results from the 2022-23 season when influenza activity increased atypically early in the season.
Accurate forecasting can help public health officials anticipate trends throughout the influenza season.