A Check on Forecast Automation with the Attention Index

The Smart Forecaster

Pursuing best practices in demand planning,

forecasting and inventory optimization

A new metric we call the “Attention Index” will help forecasters identify situations where “data behaving badly” can distort automatic statistical forecasts (see adjacent poem). It quickly identifies those items most likely to require forecast overrides—providing a more efficient way to put business experience and other human intelligence to work maximizing the accuracy of forecasts. How does it work?

Classical forecasting methods, such as the various flavors of exponential smoothing and moving averages, insist on a leap of faith. They require that we trust present conditions to persist into the future. If present conditions do persist, then it is sensible to use these extrapolative methods—methods which quantify the current level, trend, seasonality and “noise” of a time series and project them into the future.

But if they do not persist, extrapolative methods can get us into trouble. What had been going up might suddenly be going down. What used to be centered around one level might suddenly jump to another. Or something really odd might happen that is entirely out of pattern. In these surprising circumstances, forecast accuracy deteriorates, inventory calculations go wrong and general unhappiness ensues.

One way to cope with this problem is to rely on more complex forecasting models that account for external factors that drive the variable being forecasted. For instance, sales promotions attempt to disrupt buying patterns and move them in a positive direction, so including promotion activity in the forecasting process can improve sales forecasting. Sometimes macroeconomic indicators, such as housing starts or inflation rates, can be used to improve forecast accuracy. But more complex models require more data and more expertise, and they may not be useful for some problems—such as managing parts or subsystems, rather than finished goods.

If one is stuck using simple extrapolative methods, it is useful to have a way to flag items that will be difficult to forecast. This is the Attention Index. As the name suggests, items to be forecast with a high Attention Index require special handling—at least a review, and usually some sort of forecast adjustment.



The Attention Index detects three types of problems:

An outlier in the demand history of an item.
An abrupt change in the level of an item.
An abrupt change in the trend of an item.
Using software like SmartForecasts™, the forecaster can deal with an outlier by replacing it with a more typical value.

An abrupt change in level or trend can be dealt with by omitting, from the forecasting calculations, all data from before the “rupture” in the demand pattern—assuming that the item has switched into a new regime that renders the older data irrelevant.

While no index is perfect, the Attention Index does a good job of focusing attention on the most problematic demand histories. This is demonstrated in the two figures below, which were produced with data from the M3 Competition, well known in the forecasting world. Figure 1 shows the 20 items (out of the contest’s 3,003) with the highest Attention Index scores; all of these have grotesque outliers and ruptures. Figure 2 shows the 20 items with the lowest Attention Index scores; most (but not all) of the items with low scores have relatively benign patterns.

If you have thousands of items to forecast, the new Attention Index will be very useful for focusing your attention on those items most likely to be problematic.

Thomas Willemain, PhD, co-founded Smart Software and currently serves as Senior Vice President for Research. Dr. Willemain also serves as Professor Emeritus of Industrial and Systems Engineering at Rensselaer Polytechnic Institute and as a member of the research staff at the Center for Computing Sciences, Institute for Defense Analyses.

Leave a Comment

Related Posts

Probabilistic vs. Deterministic Order Planning

Probabilistic vs. Deterministic Order Planning

Consider the problem of replenishing inventory. To be specific, suppose the inventory item in question is a spare part. Both you and your supplier will want some sense of how much you will be ordering and when. And your ERP system may be insisting that you let it in on the secret too.

Four Useful Ways to Measure Forecast Error

Four Useful Ways to Measure Forecast Error

In this video, Dr. Thomas Willemain, co-Founder and SVP Research, talks about improving Forecast Accuracy by measuring Forecast Error. We begin by overviewing the various types of Error Metrics: Scale-dependent error, Percentage error, Relative error, and Scale-free error Metrics. While some error is inevitable, there are ways to reduce it, and forecast metrics are necessary aids for monitoring and improving forecast accuracy. Then we will explain the special problem of intermittent demand and divide-by-zero problems. Tom concludes by explaining how to assess forecasts of multiple items and how it often makes sense to use weighted averages, weighting items differently by volume or revenue.

Improve Forecast Accuracy by Managing Error

Improve Forecast Accuracy by Managing Error

In this video, Dr. Thomas Willemain, co-Founder and SVP Research, talks about improving Forecast Accuracy by Managing Error. This video is the first in our series on effective methods to Improve Forecast Accuracy.  We begin by looking at how forecast error causes pain and the consequential cost related to it. Then we will explain the three most common mistakes to avoid that can help us increase revenue and prevent excess inventory.

Recent Posts

  • Smart Software CEO to present at Epicor Insights 2022Smart Software to Present at Epicor Insights 2022
    Smart Software CEO will present at this year's Epicor Insights event in Nashville. If you plan to attend this year, please join us at booth #9 and #705, and learn more about Epicor Smart Inventory Planning and Optimization. […]
  • Smart Software and Arizona Public Service to Present at WERC 2022
    Smart Software CEO and APS Inventory Manager to present WERC 2022 Studio Session on implementing Smart IP&O in 90 Days and achieve significant savings by optimizing reorder points and order quantities for over 250,000 spare parts. […]