Tabular Forecasters with Boosting Surrogates

The Workhorse of Epidemic Forecasting: Tabular Forecasters with Boosting Surrogates

Your Reliable Short-Term Crystal Ball for Disease Prediction

📊 Introduction: Sometimes Simple is Superb

In the flashy world of deep learning and neural networks, it’s easy to overlook the quiet powerhouse of machine learning: gradient boosting on tabular data. While attention-grabbing models like transformers and graph neural networks dominate headlines, boosting algorithms like XGBoost, LightGBM, and CatBoost consistently deliver state-of-the-art performance on real-world forecasting tasks—including epidemic prediction.

The Tabular Forecaster with Boosting Surrogate treats epidemic forecasting as a classic supervised learning problem: given a table of features (past cases, seasonal patterns, covariates), predict future case counts. Unlike complex deep learning models that require massive data and computational resources, tabular forecasters work reliably with modest datasets, provide interpretable results, and often outperform more sophisticated approaches on short-term epidemic forecasting tasks [1-2].

From predicting weekly influenza cases to forecasting daily COVID-19 hospitalizations, these models have become the workhorse of operational epidemic forecasting systems worldwide [3-4]. Their secret? Additive modeling with weak learners—combining hundreds of simple decision trees to capture complex, nonlinear patterns in epidemic time series.

🧮 Model Description: The Mathematics of Additive Boosting

Tabular forecasters frame epidemic prediction as learning a function f(·) that maps input features to predicted case counts:

ŷₜ₊ₕ = f(xₜ)

Where ŷₜ₊ₕ is the predicted incidence h days ahead, and xₜ is a feature vector constructed from historical data up to time t.

Core Boosting Framework

Following your reference, the model uses stagewise additive modeling:

f(xₜ) = ∑₍ₘ₌₁₎ᴹ ν · hₘ(xₜ)

Where:

  • hₘ(xₜ): Weak learner (typically a shallow decision tree) at boosting step m
  • ν (nu): Learning rate (shrinkage parameter) controlling step size
  • M: Total number of boosting iterations (trees)
  • ŷₜ: Final prediction for time t

Each weak learner is trained to correct the residuals (errors) of the previous ensemble:

hₘ = argminₕ ∑ L(yᵢ, fₘ₋₁(xᵢ) + h(xᵢ)) + Ω(h)

Where L(·) is the loss function and Ω(h) is a regularization term.

Feature Engineering for Epidemic Forecasting

The feature vector xₜ typically includes:

xₜ = [Iₜ, Iₜ₋₁, …, Iₜ₋L, MAₜ, …, seasonₜ, trendₜ, covariatesₜ]

Where:

  • Iₜ₋ℓ: Lagged incidence values (ℓ = 0, 1, …, L)
  • MAₜ: Moving averages (e.g., 7-day average of recent cases)
  • seasonₜ: Seasonal features (sin(2πt/52), cos(2πt/52) for weekly data)
  • trendₜ: Long-term trend indicators (polynomial terms or cumulative sums)
  • covariatesₜ: External factors (temperature, mobility, interventions)

Regularized Loss Function

Modern boosting algorithms use regularized objective functions:

Loss = ∑₍i=1₎ᵀ L(yᵢ, ŷᵢ) + λ · ∑₍m=1₎ᴹ Ω(hₘ)

Where the regularization term Ω(hₘ) = γ · Tₘ + ½ · λ · ||wₘ||² includes:

  • Tₘ: Number of leaves in tree m
  • wₘ: Leaf weights (prediction values)
  • γ (gamma): Complexity penalty for additional leaves
  • λ (lambda): L₂ regularization on leaf weights

This prevents overfitting and improves generalization to unseen data.

📊 Key Parameter Definitions and Typical Values

Understanding these parameters helps you configure and interpret tabular forecasters effectively.

HForecast horizon1 – 14 daysShorter H = more accurate predictions
LNumber of lags7 – 52 time pointsLonger L = more historical context
TTraining period100 – 1000 daysLonger T = better pattern learning
νLearning rate0.01 – 0.3Smaller ν = more robust, needs larger M
MBoosting steps100 – 1000More trees = better fit (risk of overfitting)
λL₂ regularization0.1 – 10Larger λ = smoother predictions
γTree complexity penalty0 – 10Larger γ = simpler trees
ϕSeasonal amplitude0.1 – 2.0Controls strength of seasonal features

Typical Feature Combinations

Short-term forecasting (H = 1–3 days):

  • Lags: Iₜ, Iₜ₋₁, Iₜ₋₂
  • Moving averages: 3-day, 7-day MA
  • No seasonal terms (too short-term)

Medium-term forecasting (H = 4–14 days):

  • Lags: Iₜ, …, Iₜ₋₁₄
  • Moving averages: 7-day, 14-day MA
  • Seasonal terms: sin(2πt/52), cos(2πt/52)
  • Trend: Linear or quadratic time terms

Loss Functions for Count Data

Since epidemic data consists of counts, appropriate loss functions include:

  • Poisson loss: L(y, ŷ) = ŷ − y · log(ŷ)
  • Negative binomial loss: Accounts for overdispersion
  • Mean absolute error (MAE): Robust to outliers
  • Mean squared error (MSE): Standard for continuous approximations

⚠️ Assumptions and Applicability: When Tabular Forecasters Shine

Tabular forecasters with boosting are powerful but work best under specific conditions.

✅ Ideal Applications

  • Short-to-medium term forecasting: 1–14 day horizons where patterns are most reliable
  • Moderate to high case counts: Sufficient signal for learning
  • Stable surveillance systems: Consistent case definitions and reporting
  • Multiple seasons of data: To capture annual or multi-annual cycles
  • Operational settings: When interpretability and reliability matter more than theoretical elegance

❌ Limitations and Challenges

  • Very long horizons: Beyond 2–3 weeks, compounding errors degrade performance
  • Structural breaks: Major changes in disease behavior or surveillance
  • Rare diseases: Low signal-to-noise ratio makes pattern learning difficult
  • Spatial forecasting: Requires separate models for each location (no spatial sharing)
  • Real-time adaptation: Retraining needed when new patterns emerge

💡 Pro Tip: Always use time series cross-validation—train on past data, validate on immediate future, then roll forward. Random splits violate temporal dependencies and give overly optimistic results [5].

🚀 Model Extensions and Variants: Advanced Tabular Forecasting

The basic boosting framework has inspired numerous sophisticated extensions for epidemiological challenges.

1. Quantile Boosting

Generate prediction intervals instead of point estimates:

ŷₜ(τ) = ∑ ν · hₘ,τ(xₜ)

Where each quantile τ (e.g., 0.1, 0.5, 0.9) has its own ensemble trained with pinball loss [6].

2. Multitask Boosting

Predict multiple horizons simultaneously:

ŷₜ = [ŷₜ₊₁, ŷₜ₊₂, …, ŷₜ₊ₕ] = f(xₜ)

Using shared features with horizon-specific output layers, improving data efficiency [7].

3. Causal Boosting

Estimate heterogeneous treatment effects for interventions:

τ(xₜ) = E[Y(1) − Y(0) | xₜ]

Using orthogonalized boosting to separate treatment effects from confounding [8].

4. Online Boosting

Continuously update models as new data arrives:

fₜ(x) = fₜ₋₁(x) + ν · hₜ(x)

With incremental learning algorithms that adapt to changing epidemic dynamics [9].

5. Ensemble Boosting

Combine multiple boosting algorithms:

ŷₜ = w₁ · XGBoostₜ + w₂ · LightGBMₜ + w₃ · CatBoostₜ

Often outperforming individual models through diversity [10].

6. Hybrid Statistical-Boosting Models

Combine boosting with traditional epidemic models:

ŷₜ = f_boost(xₜ) + f_statistical(t)

Where the boosting component captures complex patterns and the statistical component ensures long-term consistency [11].

🎯 Conclusion: The Enduring Power of Practical Machine Learning

Tabular forecasters with boosting surrogates represent the perfect balance of sophistication and practicality in epidemic forecasting. By treating disease prediction as a well-structured machine learning problem with carefully engineered features, these models consistently deliver reliable, interpretable, and actionable short-term forecasts.

What makes this approach particularly valuable is its robustness and accessibility. Unlike deep learning models that require specialized hardware and expertise, boosting algorithms work reliably on standard computers with modest datasets. They provide feature importance scores that help epidemiologists understand which factors drive predictions, and they handle missing data, outliers, and mixed data types gracefully.

In an era where operational epidemic forecasting must balance accuracy, reliability, and interpretability, tabular forecasters provide your ML Epidemics Toolbox with a proven, battle-tested foundation. Whether you’re predicting next week’s flu cases, forecasting hospital capacity needs, or monitoring emerging outbreaks, these models ensure that your predictions are grounded in real patterns rather than theoretical assumptions.

The next time you see an epidemic forecast that “just works,” chances are it was powered by gradient boosting—the unsung hero of practical machine learning that proves sometimes the most effective solutions are also the most straightforward.

📚 References

[1] Chen, T., & Guestrin, C. (2016). XGBoost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 785–794. https://doi.org/10.1145/2939672.2939785

[2] Ke, G., Meng, Q., Finley, T., Wang, T., Chen, W., Ma, W., … & Liu, T. Y. (2017). LightGBM: A highly efficient gradient boosting decision tree. Advances in Neural Information Processing Systems, 30, 3146–3154. https://proceedings.neurips.cc/paper/2017/file/6449f44a102fde848669bdd9eb6b76fa-Paper.pdf

[3] Ray, E. L., Wattanachit, N., Niemi, J., Kanji, A. H., House, K., Cramer, E. Y., … & Reich, N. G. (2022). Ensemble forecasts of coronavirus disease 2019 (COVID-19) in the US. Harvard Data Science Review, 4(1).

[4] Adhikari, S. R., Cao, J., Liu, Y., & Wang, L. (2021). Deep learning for influenza forecasting: A systematic review. Infectious Disease Modelling, 6, 1032–1045. https://doi.org/10.1016/j.idm.2021.08.007

[5] Hyndman, R. J., & Athanasopoulos, G. (2021). Forecasting: Principles and Practice (3rd ed.). OTexts. https://otexts.com/fpp3/

[6] Meinshausen, N. (2006). Quantile regression forests. Journal of Machine Learning Research, 7, 983–999. https://www.jmlr.org/papers/volume7/meinshausen06a/meinshausen06a.pdf

[7] Taieb, S. B., & Hyndman, R. J. (2014). A gradient boosting approach to the Kaggle load forecasting competition. International Journal of Forecasting, 30(2), 382–394. https://doi.org/10.1016/j.ijforecast.2013.07.002

[8] Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., & Robins, J. (2018). Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1), C1–C68. https://doi.org/10.1111/ectj.12097

[9] Gomes, H. M., Bifet, A., Read, J., & Pfahringer, B. (2017). Adaptive random forests for evolving data stream classification. Machine Learning, 106(9), 1469–1495.

[10] Januschowski, T., Gasthaus, J., Wang, Y., Salinas, D., Mukherjee, S., & Turkmen, A. (2020). Criteria for classifying forecasting methods. International Journal of Forecasting, 36(1), 167–177. https://doi.org/10.1016/j.ijforecast.2019.05.004

[11] Kamarthi, H., Shah, S., & Rodriguez, A. (2020). Deep learning for multivariate time series forecasting in epidemiology. Proceedings of the AAAI Conference on Artificial Intelligence, 34(01), 542–549. https://doi.org/10.1609/aaai.v34i01.5396