Paper accepted at Twenty-Sixth IEEE International Conference on Data Mining (ICDM-26)

Accuracy Is Not Service: A Decision-Aware Benchmark for Intermittent-Demand Forecasting

It is well known that better forecasts do not always lead to better decisions. Our paper asks a more specific question: how serious is this problem for intermittent demand, where demand is sparse and inventory decisions are especially sensitive to forecast bias?

In our IEEE ICDM 2026 paper with Joo Ern Chin and Aldy Gunawan, we conduct a comprehensive benchmark of 38 forecasting methods, from moving averages and intermittent-demand specialists to Chronos and TimesFM, on a live spare-parts contract. We then replay 20,330 real customer orders under the same inventory policy and compare the resulting service performance.

The result is surprising: methods that ranked best on forecast accuracy tended to rank worse on order-fill performance. Some of the most accurate models produced the most stockouts.

For intermittent demand, small forecast biases can act like implicit safety stock, so a forecast that looks worse on error metrics can still support better inventory decisions.

The takeaway: If a forecast feeds a decision, evaluate it on the decision. Better yet, optimize the decision objective when formulating the forecasting problem; but that will be another paper.

• Paper
• Code and data