Every field, including forecasting, accumulates folk wisdom that eventually starts masquerading as “best practices.” These best practices are often wise, at least in part, but they often lack context and may not be appropriate for certain customers, industries, or business situations. There is often a catch, a “Yes, but”. This note is about six usually true forecasting precepts that nevertheless do have their caveats.
- Organize your company around a one-number forecast. This sounds sensible: it’s good to have a shared vision. But each part of the company will have its own idea about which number is the number. Finance may want quarterly revenue, Marketing may want web site visits, Sales may want churn, Maintenance may want mean time to failure. For that matter, each unit probably has a handful of key metrics. You don’t need a slogan – you need to get your job done.
- Incorporate business knowledge into a collaborative forecasting process. This is a good general rule, but if your collaborative process is flawed, messing with a statistical forecast via management overrides can decrease accuracy. You don’t need a slogan – you need to measure and compare the accuracy of any and all methods and go with the winners.
- Forecast using causal modeling. Extrapolative forecasting methods take no account of the underlying forces driving your sales, they just work with the results. Causal modeling takes you deeper into the fundamental drivers and can improve both accuracy and insight. However, causal models (implemented through regression analysis) can be less accurate, especially when they require forecasts of the drivers (“predictions of the predictors”) rather than simply plugging in recorded values of lagged predictor variables. You don’t need a slogan: You need a head-to-head comparison.
- Forecast demand instead of shipments. Demand is what you really want, but “composing a demand signal” can be tricky: what do you do with internal transfers? One-off’s? Lost sales? Furthermore, demand data can be manipulated. For example, if customers intentionally don’t place orders or try to game their orders by ordering too far in advance, then order history won’t be better than shipment history. At least with shipment history, it’s accurate: You know what you shipped. Forecasts of shipments are not forecasts of “demand”, but they are a solid starting point.
- Use Machine Learning methods. First, “Machine learning” is an elastic concept that includes an ever-growing set of alternatives. Under the hood of many ML advertised models is just an auto-pick an extrapolative forecast method (i.e., best fit) which while great at forecasting normal demand, has been around since the 1980’s (Smart Software was the first company to release an auto-pick method for the PC). ML models are data hogs that require larger data sets than you may have available. Properly choosing then training an ML model requires a level of statistical expertise that is uncommon in many manufacturing and distribution businesses. You might want to find somebody to hold your hand before you start playing this game.
- Removing outliers creates better forecasts. While it is true that very unusual spikes or drops in demand will mask underlying demand patterns such as trend or seasonality, it isn’t always true that you should remove the spikes. Often these demand surges reflect the uncertainty that can randomly interfere with your business and thus need to be accounted for. Removing this type of data from your demand forecast model might make the data more predictable on paper but will leave you surprised when it happens again. So, be careful about removing outliers, especially en masse.
