“Regardless of how much effort is poured into training forecasters and developing elaborate forecast support systems, decision-makers will either modify or discard the predictions if they do not trust them.”  — Dilek Onkal, International Journal of Forecasting 38:3 (July-September 2022), p.802.

The words quoted above grabbed my attention and prompted this post. Those of a geekly persuasion, like your blogger, are inclined to think of forecasting as a statistical problem. While that is obviously true, those of a certain age, like your blogger, understand that forecasting is also a social activity and therefore has a large human component.

Who Do You Trust?

Trust is always a two-way street, but let’s stay on the demand forecaster’s side. What characteristics of and actions by forecasters and demand planners build trust in their work? The above quoted Professor Onkal reviewed academic research on this topic going back to 2006. She summarized results from practitioner surveys that identified key trust factors related to forecaster characteristics, forecasting process, and forecasting communication.

Forecaster characteristics

Key to building trust among the users of forecasts are perceptions of forecaster and demand planner competence and objectivity. Competence has a mathematical component, but many managers confuse computer skills with analytic skills, so users of forecasting software can usually clear this hurdle. However, since the two are not the same, it pays dividends to absorb your vendor’s training and learn not just the math but the lingo of your forecasting software. In my observation, trust can also be increased by showing knowledge of the company’s business.

Objectivity is also a key to trustworthiness. It may be uncomfortable for the forecaster to be put in the middle of occasional departmental squabbles, but those will come up and must be handled with tact. Squabbles? Well, silos exist and tilt in different directions. Sales departments favor higher demand forecasts that drive production increases, so that they never have to say “Sorry, we are fresh out of that.” Inventory managers are wary of high demand forecasts, because “excess enthusiasm” can leave them holding the bag, sitting on bloated inventory.

Sometimes the forecaster becomes a de facto referee, and in this role must display overt signs of objectivity. That can mean first recognizing that every management decision involves tradeoffs of good things against other good things, e.g., product availability versus lean operations, and then helping the parties strike a painful but tolerable balance by surfacing the links between operational decisions and the key performance metrics that matter to folks like Chief Financial Officers.

The Forecasting process

The forecasting process can be thought of as having three phases: data inputs, calculations, and outputs. Actions can be taken to increase trust in each phase.

Regarding inputs:

Trust can be increased if obviously relevant inputs are at least acknowledged if not directly used in calculations. Thus, factors like social media sentiment and regional sales managers’ gut instincts can be legitimate parts of a forecast consensus process. However, objectivity requires that these putative predictors of profit be tested objectively. For instance, a professional-grade forecasting process may well include subjective adjustment to statistical forecasts but must then also assess whether the adjustments actually end up improving accuracy, not just making some people feel listened to.

Regarding the second phase, calculations:

The forecaster will be trusted to the extent that they are able to deploy more than one way to calculate forecasts and then articulate a good reason why they chose the method eventually used. In addition, the forecaster should be able to explain in accessible language how even complicated techniques do their job. It is difficult to put trust in a “black box” method that is so opaque as to be inscrutable. The importance of explainability is amplified by the fact of life that the forecaster’s superior must themselves in turn be able to justify the choice of technique to their supervisor.

For instance, exponential smoothing uses this equation: S(t) = αX(t)+(1-α)S(t-1). Many forecasters are familiar with this equation, but many forecast users are not. There is a story that explains the equation in terms of averaging irrelevant “noise” in an item’s demand history and the need to strike a balance between smoothing out noise and being able to react to sudden shifts in the level of demand. The forecaster who can tell that story will be more credible. (My own version of that story uses phrases from sports, i.e., “head fakes” and “jukes”. Finding folksy analogs appropriate to your specific audience always pays dividends.)

A final point: best practice demands that any forecast be accompanied by an honest assessment of its uncertainty. A forecaster who tries to build trust by being overly specific (“Sales next quarter will be 12,184 units”) will always fail. A forecaster who says “Sales next quarter will have a 90% chance of falling between 12,000 and 12,300 units” will be both correct more often and  also more helpful to decision makers. After all, forecasting is essentially a job of risk management, so the decision maker is best served by knowing the risks.

Forecasting communication:

Finally, consider the third phase, communication of forecast results. Research suggests that continual communication with forecast users builds trust. It avoids those horrible, deflating moments when a nicely formatted report is shot down because of some fatal flaw that could have been foreseen: “This is no good because you didn’t take account of X, Y or Z” or “We really wanted you to present results rolled up to the top of the product hierarchies (or by sales region or by product line or…)”.

Even when everybody is aligned as to what is expected, trust is enhanced by presenting results using well-crafted graphics, with massive numerical tables provided for backup but not as the main way of communicating results. My experience has been that, just as a meeting-control device, a graph is usually much better than a large numerical table. With a graph, everybody’s attention is focused on the same thing and many aspects of the analysis are immediately (and literally) visible. With a table of results, the table of participants often splinters into side conversations in which each voice is focused on different pieces of the table.

Onkal summarizes the research this way: “Take-aways for those who make forecasts and those who use them converge around clarity of communication as well as perceptions of competence and integrity.”

What Do You Trust?

There is a related dimension of trust: not who do you trust but what do you trust? By this I mean both data and software….  Read the 2nd part of this Blog “What do you Trust” here  https://smartcorp.com/forecasting/the-role-of-trust-in-the-demand-forecasting-process-part-2-what/

How long should it take for a demand forecast to be computed using statistical methods?  This question is often asked by customers and prospects.  The answer truly depends.  Forecast results for a single item can be computed in the blink of an eye, in as little as a few hundredths of a second, but sometimes they may require as much as five seconds.  To understand the differences, it’s important to understand that there is more involved than grinding through the forecast arithmetic itself.   Here are six factors that influence the speed of your forecast engine.

1) Forecasting method.  Traditional time-series extrapolative techniques (such as exponential smoothing and moving average methods), when cleverly coded, are lighting fast.  For example, the Smart Forecast automatic forecasting engine that leverages these techniques and powers our demand planning and inventory optimization software can crank out statistical forecasts on 1,000 items in 1 second!  Extrapolative methods produce an expected forecast and a summary measure of forecast uncertainty. However, more complex models in our platform that generate probabilistic demand scenarios take much longer given the same computing resources.  This is partly because they create a much larger volume of output, usually thousands of plausible future demand sequences. More time, yes, but not time wasted, since these results are much more complete and form the basis for downstream optimization of inventory control parameters.

2) Computing resources.  The more resources you throw at the computation, the faster it will be.  However, resources cost money and it may not be economical to invest in these resources.  For example, to make certain types of machine learning-based forecasts work, the system will need to multi-thread computations across multiple servers to deliver results quickly.  So, make sure you understand the assumed compute resources and associated costs. Our computations happen on the Amazon Web Services cloud, so it is possible to pay for a great deal of parallel computation if desired.

3) Number of time-series.  Do you have to forecast only a few hundred items in a single location or many thousands of items across dozens of locations?  The greater the number of SKU x Location combinations, the greater the time required.  However, it is possible to trim the time to get demand forecasts by better demand classification.  For example, it is not important to forecast every single SKU x Location combination. Modern Demand Planning Software can first subset the data based on volume/frequency classifications before running the forecast engine.  We’ve observed situations where over one million SKU x Location combinations existed, but only ten percent had demand in the preceding twelve months.

4) Historical Bucketing.  Are you forecasting using daily, weekly, or monthly time buckets?  The more granular the bucketing, the more time it is going to take to compute statistical forecasts.  Many companies will wonder, “Why would anyone want to forecast on a daily basis?” However, state-of-the-art demand forecasting software can leverage daily data to detect simultaneous day-of-week and week-of-month patterns that would otherwise be obscured with traditional monthly demand buckets. And the speed of business continues to accelerate, threatening the competitive viability of the traditional monthly planning tempo.

5) Amount of History.  Are you limiting the model by only feeding it the most recent demand history, or are you feeding all available history to the demand forecasting software? The more history you feed the model, the more data must be analyzed and the longer it is going to take.

6) Additional analytical processing.  So far, we’ve imagined feeding items’ demand history in and getting forecasts out. But the process can also involve additional analytical steps that can improve results. Examples include:

a) Outlier detection and removal to minimize the distortion caused by one-off events like storm damage.

b) Machine learning that decides how much history should be used for each item by detecting regime change.

c) Causal modeling that identifies how changes in demand drivers (such as price, interest rate, customer sentiment, etc.) impact future demand.

d) Exception reporting that uses data analytics to identify unusual situations that merit further management review.

The Rest of the Story. It’s also critical to understand that the time to get an answer involves more than the speed of forecasting computations per se.  Data must be loaded into memory before computing can begin. Once the forecasts are computed, your browser must load the results so that they may be rendered on screen for you to interact with.  If you re-forecast a product, you may choose to save the results.  If you are working with product hierarchies (aggregating item forecasts up to product families, families up to product lines, etc.), the new forecast is going to impact the hierarchy, and everything must be reconciled.   All of this takes time.

Fast Enough for You? When you are evaluating software to see whether your need for speed will be satisfied, all of this can be tested as part of a proof of concept or trial offered by demand planning software solution providers.  Test it out, and make sure that the compute, load, and save times are acceptable given the volume of data and forecasting methods you want to use to support your process.

 What is the wiggle effect? 

It’s when your statistical forecast incorrectly predicts the ups and downs observed in your demand history when there really isn’t a pattern.  It’s important to make sure your forecasts don’t wiggle unless there is a real pattern.

Here is a transcript from a recent customer where this issue was discussed:

Customer: “The forecast isn’t picking up on the patterns I see in the history.  Why not?” 

Smart:  “If you look closely, the ups and downs you see aren’t patterns.  It’s really noise.”  

Customer:  “But if we don’t predict the highs, we’ll stock out.”

Smart: “If the forecast were to ‘wiggle’ it would be much less accurate.  The system will forecast whatever pattern is evident, in this case a very slight uptrend.  We’ll buffer against the noise with safety stocks. The wiggles are used to set the safety stocks.”

Customer: “Ok. Makes sense now.” 

The wiggle looks reassuring but, in this case, it is resulting in an incorrect demand forecast. The ups and downs aren’t really occurring at the same times each month.  A better statistical forecast is shown in light green.

A statistical forecast of zero can cause lots of confusion for forecasters, especially when the historical demand is non-zero.  Sure, it’s obvious that demand is trending downward, but should it trend to zero?  When the older demand is much greater than the more recent demand and the more recent demand is very low volume (i.e., 1,2,3 units demanded), the answer is, statistically speaking, yes.  However, this might not jive with the planner’s business knowledge and expected minimum level of demand.  So, what should a forecaster do to correct this? Here are three suggestions:

1. Limit the historical data fed to the model. In a down trending situation, the older data is often much greater than the recent data.   When the older much higher volume demand is ignored, the down trend won’t be nearly as significant.  You’ll still forecast a down trend, but results are more likely to be line with business expectations.

2. Try trend dampening. Smart Demand Planner has a feature called “trend hedging” that enables users to define how a trend should phase out over time. The higher the percentage trend hedge (0-100%), the more pronounced the trend dampening.  This means that a forecasted trend will not continue through the whole forecast horizon.  This means the demand forecast will start to flatten before it hits zero on a downtrend.

3. Change the forecast model. Switch from a trending method like Double Exponential Smoothing or Linear Moving Average to a non-trending method such as Single Exponential Smoothing or Simple Moving Average. You won’t forecast a downtrend, but at least your forecast won’t be zero and thus more likely to be accepted by the business.

5 Steps to Consensus Demand Planning

The whole point of demand forecasting is to establish the best possible view of future demand.  This requires that we draw upon the best data and inputs we can get, leverage statistics to capture underlying patterns, put our heads together to apply overrides based on business knowledge, and agree on a consensus demand plan that serves as cornerstone to the company’s overall demand plan.

Step 1: Develop an accurate demand signal.   What constitutes demand?  Consider how  your organization defines demand – say, confirmed sales orders net of cancellations or shipment data adjusted to remove the impact of historical stockouts  – and use this consistently.  This is your measure of what the market is requesting you to deliver.  Don’t confuse this with your ability to deliver – that should be reflected in the revenue plan.

Step 2: Generate a statistical forecast.  Plan for thousands of items, using a proven forecasting application that automatically pulls in your data and reliably produces accurate forecasts for all of your items.  Review the first pass of your forecast, then make adjustments.  A strike or train wreck may have interrupted shipping last month – don’t let that wag your forecast.  Adjust for these and reforecast.  Do the best you can, then invite others to weigh in.

Step 3: Bring on the experts.  Product line managers, sales leaders, key distribution partners know their markets.  Share your forecast with them.  Smart uses the concept of a “Snapshot” to share a facsimile of your forecast – at any level, for any product line – with people who may know better.  There could be an enormous order that hasn’t hit the pipeline, or a channel partner is about to run their annual promotion.  Give them an easy way to take their portion of the forecast and change it.  Drag this month up, that one down …

Step 4:  Measure Accuracy and Forecast Value Add.  Some of your contributors may be right on the money, other tend to be biased high or low.  Use forecast vs. actuals reporting and measure forecast value add analysis to measure forecast errors and whether changes to the forecast are hurting or helping.  By informing the process with this information, your company will improve it’s ability to forecast more accurately.

Step 5: Agree on the Consensus Forecast.  You can do this one product line or geography at a time, or business by  business.  Convene the team, graphically stack up their inputs, review past accuracy performance, discuss their reasons for increasing or reducing the forecast, and agree on whose inputs to use.  This becomes your consensus plan.  Finalize the plan and send it off – upload forecasts to MRP, send to finance and manufacturing.  You have just kicked off your Sales, Inventory and Operational Planning process.

You can do this.  And we can help.  If you have any questions about collaborative demand planning please reply to this blog, we will follow up.