Time-Series Forecasting: Classical Methods vs. Neural Approaches

There's a persistent assumption that neural networks are simply the more advanced choice for forecasting, and that classical statistical methods are a stepping stone you graduate past. In practice, on a single well-behaved time series with limited history, a properly tuned classical model still frequently beats a neural network, and understanding why tells you a lot about how to actually choose between them.

What classical models assume, and why that's often a feature

ARIMA (AutoRegressive Integrated Moving Average) models a series as a function of its own past values and past forecast errors, after differencing to remove trend. Exponential smoothing methods (like Holt-Winters) model level, trend, and seasonality as explicitly separate, smoothly-updating components. Both encode strong, explicit assumptions about how time series behave — and when those assumptions roughly hold, that structure is exactly what lets the model fit well from a small number of observations, because it isn't trying to learn the assumption from data, it starts with it.

from statsmodels.tsa.statespace.sarimax import SARIMAX

model = SARIMAX(
    series,
    order=(1, 1, 1),
    seasonal_order=(1, 1, 1, 12),
)
fit = model.fit()
forecast = fit.forecast(steps=12)

This is why a single retail store's monthly sales — one series, a few years of history, clear seasonality — is often best served by SARIMA or Holt-Winters rather than a neural model: there simply isn't enough data for a neural network to learn seasonality from scratch that the statistical model gets for free from its structural assumptions.

Where neural approaches earn their complexity

Neural forecasting methods — from sequence models like LSTMs to more recent architectures purpose-built for forecasting like N-BEATS, DeepAR, or Temporal Fusion Transformer — become genuinely worth the added complexity in a specific situation: when you have many related series to forecast simultaneously, not just one. A neural model can learn shared patterns across thousands of related series — thousands of SKUs, thousands of sensor streams — and transfer that learned structure to series with limited individual history, something a classical model fit independently to each series structurally cannot do.

# Conceptual shape of global model training across many series
# Each row: one (series_id, time_window) example sharing model weights
for series_id, window in dataloader:
    covariates = build_covariates(series_id, window)  # holidays, price, category
    pred = model(window.history, covariates)
    loss = quantile_loss(pred, window.target)

DeepAR-style models also naturally produce probabilistic forecasts — full predictive distributions rather than point estimates — which matters when downstream decisions (inventory, staffing) need to reason about the range of plausible outcomes, not just a single expected value.

The comparison that actually matters

A practical middle ground: gradient boosting on lagged features

Before reaching for a neural architecture, it's worth trying gradient boosted trees on an engineered feature set — lagged values, rolling statistics, calendar features, and (critically) the series identifier itself as a categorical feature. This gets you most of the "learn shared structure across many series" benefit of a neural model with far less engineering and training overhead, and it's frequently the best-performing approach in forecasting competitions on structured business data with many related series and rich covariates.

The decision that actually drives the outcome

The single biggest lever isn't the model family — it's whether you're forecasting one series in isolation or many related series that can share statistical strength. Get that framing right first, because it determines whether the extra complexity of a neural or boosted global model has anything to actually learn from, or whether you're just adding degrees of freedom to a problem that was already well-served by a much simpler, much cheaper structural model.