Improve Forecast Accuracy by Managing Error

The Smart Forecaster

 Pursuing best practices in demand planning,

forecasting and inventory optimization

Improve Forecast Accuracy, Eliminate Excess Inventory, & Maximize Service Levels

In this video, Dr. Thomas Willemain, co-Founder and SVP Research, talks about improving Forecast Accuracy by Managing Error. This video is the first in our series on effective methods to Improve Forecast Accuracy.  We begin by looking at how forecast error causes pain and the consequential cost related to it. Then we will explain the three most common mistakes to avoid that can help us increase revenue and prevent excess inventory. Tom concludes by reviewing the methods to improve Forecast Accuracy, the importance of measuring forecast error, and the technological opportunities to improve it.

 

Forecast error can be consequential

Consider one item of many

  • Product X costs $100 to make and nets $50 profit per unit.
  • Sales of Product X will turn out to be 1,000/month over the next 12 months.
  • Consider one item of many

What is the cost of forecast error?

  • If the forecast is 10% high, end the year with $120,000 of excess inventory.
  • 100 extra/month x 12 months x $100/unit
  • If the forecast is 10% low, miss out on $60,000 of profit.
  • 100 too few/month x 12 months x $50/unit

 

Three mistakes to avoid

1. Ignoring error.

  • Unprofessional, dereliction of duty.
  • Wishing will not make it so.
  • Treat accuracy assessment as data science, not a blame game.

2. Tolerating more error than necessary.

  • Statistical forecasting methods can improve accuracy at scale.
  • Improving data inputs can help.
  • Collecting and analyzing forecast error metrics can identify weak spots.

3. Wasting time and money going too far trying to eliminate error.

  • Some product/market combinations are inherently more difficult to forecast. After a point, let them be (but be alert for new specialized forecasting methods).
  • Sometimes steps meant to reduce error can backfire (e.g., adjustment).
Leave a Comment

RECENT POSTS

Maximize Machine Uptime with Probabilistic Modeling

Maximize Machine Uptime with Probabilistic Modeling

If you both make and sell things, you own two inventory problems. Companies that sell things must focus relentlessly on having enough product inventory to meet customer demand. Manufacturers and asset intensive industries such as power generation, public transportation, mining, and refining, have an additional inventory concern: having enough spare parts to keep their machines running.
This technical brief reviews the basics of two probabilistic models of machine breakdown. It also relates machine uptime to the adequacy of spare parts inventory.

Want to Optimize Inventory? Follow These 4 Steps

Want to Optimize Inventory? Follow These 4 Steps

Service Level Driven Planning (SLDP) is an approach to inventory planning based on exposing the tradeoffs between SKU availability and inventory cost that are at the root of all wise inventory decisions. When organizations understand these tradeoffs, they can make better decisions and have greater variability into the risk of stockouts. SLDP unfolds in four steps: Benchmark, Collaborate, Plan, and Track.

Four Ways to Optimize Inventory

Four Ways to Optimize Inventory

Inventory optimization has become an even higher priority in recent months for many of our customers.  Some are finding their products in vastly greater demand; more have the opposite problem. In either case, events like the Covid19 pandemic are forcing a reexamination of standard operating conditions, such as choices of reorder points and order quantities.

Recent Posts

  • Ship and Supply Chain Blockage in Suez CanalRedefine Exceptions and Fine Tune Planning to Address Uncertainty
    In a perfect world, Just in Time (JIT) would be the appropriate solution for inventory management. But as the saying goes “everyone has a plan until they get punched in the mouth.” One enormous punch in the mouth for the global supply chain was Suez Canal Blockage that held up $9.6B in trade costing an estimated $6.7M per minute. […]
  • Inventory Planning Processes Challenges OpportunitiesInventory Planning Processes: Challenges and Opportunities
    Smart Software is pleased to introduce our new series of educational webinars, offered exclusively for Epicor Users. Greg Hartunian, CEO at Smart Software, will lead a 45-minute webinar focusing on specific approaches to demand forecasting and inventory planning. […]

    A Primer on Probabilistic Forecasting

    The Smart Forecaster

     Pursuing best practices in demand planning,

    forecasting and inventory optimization

    If you keep up with the news about supply chain analytics, you are more frequently encountering the phrase “probabilistic forecasting.” If this phrase is puzzling, read on.

    You probably already know what “forecasting” means. And you probably also know that there seem to be lots of different ways to do it. And you’ve probably heard pungent little phrases like “every forecast is wrong.” So you know that some kind of mathemagic might calculate that “the forecast is you will sell 100 units next month”, and then you might sell 110 units, in which case you have a 10% forecast error.

    You may not know that what I just described is a particular kind of forecast called a “point forecast.” A point forecast is so named because it consists of just a single number (i.e., one point on the number line, if you recall the number line from your youth).

    Point forecasts have one virtue: They are simple. They also have a flaw: They give rise to snarky statements like “every forecast is wrong.” That is, in most realistic cases, it is unlikely that the actual value will exactly equal the forecast. (Which isn’t such a big deal if the forecast is close enough.)

    This gets us to “probabilistic forecasting.” This approach is a step up, because instead of producing a single-number (point) forecast, it yields a probability distribution for the forecast. And unlike traditional extrapolative models that rely purely on the historical data, probabilistic forecasts have the ability to simulate future values that aren’t anchored to the past.

    “Probability distribution” is a forbidding phrase, evoking some arcane math that you may have heard of but never studied. Luckily, most adults have enough life experience to have an intuitive grasp of the concept.  When broken down, it’s quite straightforward to understand.

    Imagine the simple act of flipping two coins. You might call this harmless fun, but I call it a “probabilistic experiment.” The total number of heads that turn up on the two coins will be either zero, one or two. Flipping two coins is a “random experiment.” The resulting number of heads is a “random variable.” It has a “probability distribution”, which is nothing more than a table of how likely it is that the random variable will turn out to have any of its possible values. The probability of getting two heads when the coins are fair works out to be ¼, as is the probability of no heads. The chance of one head is ½.

    The same approach can describe a more interesting random variable, like the daily demand for a spare part.  Figure 2 shows such a probability distribution. It was computed by compiling three years of daily demand data on a certain part used in a scientific instrument sold to hospitals.

     

    Probabilistic demand forecast 1

    Figure 1: The probability distribution of daily demand for a certain spare part

     

    The distribution in Figure 1 can be thought of as a probabilistic forecast of demand in a single day. For this particular part, we see that the forecast is very likely to be zero (97% chance), but sometimes will be for a handful of units, and once in three years will be twenty units. Even though the most likely forecast is zero, you would want to keep a few on hand if this part were critical (“…for want of a nail…”)

    Now let’s use this information to make a more complicated probabilistic forecast. Suppose you have three units on hand. How many days will it take for you to have none? There are many possible answers, ranging from a single day (if you immediately get a demand for three or more) up to a very large number (since 97% of days see no demand).  The analysis of this question is a bit complicated because of all the many ways this situation can play out, but the final answer that is most informative will be a probability distribution. It turns out that the number of days until there are no units left in stock has the distribution shown in Figure 2.

    Probabilistic demand forecast 2

    Figure 2: Distribution of the number of days until all three units are gone

     

    The average number of days is 74, which would be a point forecast, but there is a lot of variation around the average. From the perspective of inventory management, it is notable that there is a 25% chance that all the units will be gone after 32 days. So if you decided to order more when you were down to only three on the shelf, it would be good to have the supplier get them to you before a month has passed. If they couldn’t, you’d have a 75% chance of stocking out – not good for a critical part.

    The analysis behind Figure 2 involved making some assumptions that were convenient but not necessary if they were not true. The results came from a method called “Monte Carlo simulation”, in which we start with three units, pick a random demand from the distribution in Figure 1, subtract it from the current stock, and continue until the stock is gone, recording how many days went by before you ran out. Repeating this process 100,000 times produced Figure 2.

    Applications of Monte Carlo simulation extend to problems of even larger scope than the “when do we run out” example above. Especially important are Monte Carlo forecasts of future demand. While the usual forecasting result is a set of point forecasts (e.g., expected unit demand over the next twelve months), we know that there are any number of ways that the actual demand could play out. Simulation could be used to produce, say, one thousand possible sets of 365 daily demand demands.

    This set of demand scenarios would more fully expose the range of possible situations with which an inventory system would have to cope. This use of simulation is called “stress testing”, because it exposes a system to a range of varied but realistic scenarios, including some nasty ones. Those scenarios are then input to mathematical models of the system to see how well it will cope, as reflected in key performance indicators (KPI’s). For instance, in those thousand simulated years of operation, how many stockouts are there in the worst year? the average year? the best year? In fact, what is the full probability distribution of the number of stockouts in a year, and what is the distribution of their size?

    Figures 3 and 4 illustrate probabilistic modeling of an inventory control system that converts stockouts to backorders. The system simulated uses a Min/Max control policy with Min = 10 units and Max = 20 units.

    Figure 3 shows one simulated year of daily operations in four plots. The first plot shows a particular pattern of random daily demand in which average demand increases steadily from Monday to Friday but disappears on weekends. The second plot shows the number of units on hand each day. Note that there are a dozen times during this simulated year when inventory goes negative, indicating stockouts. The third plot shows the size and timing of replenishment orders. The fourth plot shows the size and timing of backorders.  The information in these plots can be translated into estimates of inventory investment, average units on hand, holding costs, ordering costs and shortage costs.

    Probabilistic demand forecast 3

    Figure 3: One simulated year of inventory system operation

     

    Figure 3 shows one of one thousand simulated years. Each year will have different daily demands, resulting in different values of metrics like units on hand and the various components of operating cost. Figure 4 plots the distribution of 1,000 simulated values of four KPI’s. Simulating 1,000 years of imagined operation exposes the range of possible results so that planners can account not just for average results but also see best-case and worst-case values.

    Probabilistic demand forecast 4

    Figure 4: Distributions of four KPI’s based on 1,000 simulations

     

    Monte Carlo simulation is a low-math/high-results approach to probabilistic forecasting: very practical and easy to explain. Advanced probabilistic forecasting methods employed by Smart Software expand upon standard Monte Carlo simulation, yielding extremely accurate estimates of required inventory levels.

     

    Leave a Comment

    Related Posts

    Four Ways to Optimize Inventory

    Four Ways to Optimize Inventory

    Inventory optimization has become an even higher priority in recent months for many of our customers.  Some are finding their products in vastly greater demand; more have the opposite problem. In either case, events like the Covid19 pandemic are forcing a reexamination of standard operating conditions, such as choices of reorder points and order quantities.

    Top 3 Most Common Inventory Control Policies

    Top 3 Most Common Inventory Control Policies

    To make the right decision, you’ll need to know how demand forecasting supports inventory management, choice of which policy to use, and calculation of the inputs that drive these policies.The process of ordering replenishment stock is sufficiently expensive and cumbersome that you also want to minimize the number of purchase orders you must generate.

    How to Choose a Target Service Level to Optimize Inventory

    How to Choose a Target Service Level to Optimize Inventory

    When setting a target service level, make sure to take into account factors like current service levels, replenishment lead times, cost constraints, the pain inflicted by shortages on you and your customers, and your competitive position.

    Recent Posts

    • Ship and Supply Chain Blockage in Suez CanalRedefine Exceptions and Fine Tune Planning to Address Uncertainty
      In a perfect world, Just in Time (JIT) would be the appropriate solution for inventory management. But as the saying goes “everyone has a plan until they get punched in the mouth.” One enormous punch in the mouth for the global supply chain was Suez Canal Blockage that held up $9.6B in trade costing an estimated $6.7M per minute. […]
    • Inventory Planning Processes Challenges OpportunitiesInventory Planning Processes: Challenges and Opportunities
      Smart Software is pleased to introduce our new series of educational webinars, offered exclusively for Epicor Users. Greg Hartunian, CEO at Smart Software, will lead a 45-minute webinar focusing on specific approaches to demand forecasting and inventory planning. […]

      Automatic Forecasting for Time Series Demand Projections

      The Smart Forecaster

       Pursuing best practices in demand planning,

      forecasting and inventory optimization

      Improve Forecast Accuracy, Eliminate Excess Inventory, & Maximize Service Levels

      In this video tutorial Dr. Thomas Willemain, co–Founder and SVP Research at Smart Software, presents Automatic Forecasting for Time Series Demand Projections, a specialized algorithmic tournament to determine an appropriate time series model and estimate the parameters to compute the best forecasts methods. Automatic forecasts of large numbers of time series are frequently used in business, some have trend either up or down, and some have seasonality so they are cyclic, and each of those specific patterns requires a suitable technical approach, and an appropriate statistical forecasting method.  Tom explains how the tournament computes the best forecasts methods and works through a practical example.

      AUTOMATIC FORECASTING COMPLETE-VIDEO-2
      Leave a Comment

      RECENT POSTS

      Maximize Machine Uptime with Probabilistic Modeling

      Maximize Machine Uptime with Probabilistic Modeling

      If you both make and sell things, you own two inventory problems. Companies that sell things must focus relentlessly on having enough product inventory to meet customer demand. Manufacturers and asset intensive industries such as power generation, public transportation, mining, and refining, have an additional inventory concern: having enough spare parts to keep their machines running.
      This technical brief reviews the basics of two probabilistic models of machine breakdown. It also relates machine uptime to the adequacy of spare parts inventory.

      Want to Optimize Inventory? Follow These 4 Steps

      Want to Optimize Inventory? Follow These 4 Steps

      Service Level Driven Planning (SLDP) is an approach to inventory planning based on exposing the tradeoffs between SKU availability and inventory cost that are at the root of all wise inventory decisions. When organizations understand these tradeoffs, they can make better decisions and have greater variability into the risk of stockouts. SLDP unfolds in four steps: Benchmark, Collaborate, Plan, and Track.

      Four Ways to Optimize Inventory

      Four Ways to Optimize Inventory

      Inventory optimization has become an even higher priority in recent months for many of our customers.  Some are finding their products in vastly greater demand; more have the opposite problem. In either case, events like the Covid19 pandemic are forcing a reexamination of standard operating conditions, such as choices of reorder points and order quantities.

      Recent Posts

      • Ship and Supply Chain Blockage in Suez CanalRedefine Exceptions and Fine Tune Planning to Address Uncertainty
        In a perfect world, Just in Time (JIT) would be the appropriate solution for inventory management. But as the saying goes “everyone has a plan until they get punched in the mouth.” One enormous punch in the mouth for the global supply chain was Suez Canal Blockage that held up $9.6B in trade costing an estimated $6.7M per minute. […]
      • Inventory Planning Processes Challenges OpportunitiesInventory Planning Processes: Challenges and Opportunities
        Smart Software is pleased to introduce our new series of educational webinars, offered exclusively for Epicor Users. Greg Hartunian, CEO at Smart Software, will lead a 45-minute webinar focusing on specific approaches to demand forecasting and inventory planning. […]

        Forecast Using Leading Indicators – Regression Analysis:

        The Smart Forecaster

         Pursuing best practices in demand planning,

        forecasting and inventory optimization

        Improve Forecast Accuracy, Eliminate Excess Inventory, & Maximize Service Levels

        In this video tutorial Dr. Thomas Willemain, co–Founder and SVP Research at Smart Software, presents Regression Analysis, a specialized statistical modeling technique to identify and harness leading indicators to achieve more accurate forecasts.  Regression analysis is a statistical procedure to estimate the relationship between a response variable and one or more predictor variables. Housing starts, for example, might be a good leading indicator of vinyl siding demand.  Tom explains how and when to use regression analysis and works through a practical example.

        Forecasting Techniques for a more profitable business
        Leave a Comment

        RECENT POSTS

        Maximize Machine Uptime with Probabilistic Modeling

        Maximize Machine Uptime with Probabilistic Modeling

        If you both make and sell things, you own two inventory problems. Companies that sell things must focus relentlessly on having enough product inventory to meet customer demand. Manufacturers and asset intensive industries such as power generation, public transportation, mining, and refining, have an additional inventory concern: having enough spare parts to keep their machines running.
        This technical brief reviews the basics of two probabilistic models of machine breakdown. It also relates machine uptime to the adequacy of spare parts inventory.

        Want to Optimize Inventory? Follow These 4 Steps

        Want to Optimize Inventory? Follow These 4 Steps

        Service Level Driven Planning (SLDP) is an approach to inventory planning based on exposing the tradeoffs between SKU availability and inventory cost that are at the root of all wise inventory decisions. When organizations understand these tradeoffs, they can make better decisions and have greater variability into the risk of stockouts. SLDP unfolds in four steps: Benchmark, Collaborate, Plan, and Track.

        Four Ways to Optimize Inventory

        Four Ways to Optimize Inventory

        Inventory optimization has become an even higher priority in recent months for many of our customers.  Some are finding their products in vastly greater demand; more have the opposite problem. In either case, events like the Covid19 pandemic are forcing a reexamination of standard operating conditions, such as choices of reorder points and order quantities.

        Recent Posts

        • Ship and Supply Chain Blockage in Suez CanalRedefine Exceptions and Fine Tune Planning to Address Uncertainty
          In a perfect world, Just in Time (JIT) would be the appropriate solution for inventory management. But as the saying goes “everyone has a plan until they get punched in the mouth.” One enormous punch in the mouth for the global supply chain was Suez Canal Blockage that held up $9.6B in trade costing an estimated $6.7M per minute. […]
        • Inventory Planning Processes Challenges OpportunitiesInventory Planning Processes: Challenges and Opportunities
          Smart Software is pleased to introduce our new series of educational webinars, offered exclusively for Epicor Users. Greg Hartunian, CEO at Smart Software, will lead a 45-minute webinar focusing on specific approaches to demand forecasting and inventory planning. […]

          5 Demand Planning Tips for Calculating Forecast Uncertainty

          The Smart Forecaster

           Pursuing best practices in demand planning,

          forecasting and inventory optimization

          Those who produce forecasts owe it to those who consume forecasts, and to themselves, to be aware of the uncertainty in their forecasts. This note is about how to estimate forecast uncertainty and use the estimates in your demand planning process. We focus on forecasts made in support of demand planning as well as forecasts inherent in optimizing inventory policies involving reorder points, safety stocks, and min/max levels.

          Reading this, you will learn about:

          -Criteria for assessing forecasts
          -Sources of forecast error
          -Calculating forecast error
          -Converting forecast error into prediction intervals
          -The relationship between demand forecasting and inventory optimization.
          -Actions you can take to use these concepts to improve your company’s processes.

          Criteria for Assessing Forecasts

          Forecast error alone is not reason enough to reject forecasting as a management tool. To twist a famous aphorism by George Box, “All forecasts are wrong, but some are useful.” Of course, business professionals will always search for ways to make forecasts more useful. This usually involves work to reduce forecast error. But while forecast accuracy is the most obvious criterion by which to judge forecasts, but it is not the only one. Here’s a list of criteria for evaluating forecasts:

          Accuracy: Forecasts of future values should, in retrospect, be very close to the actual values that eventually reveal themselves. But there may be diminishing returns to squeezing another half percent of accuracy out of forecasts otherwise good enough to use in decision making.

          Timeliness: Fighter pilots refer to the OODA Loop (Observe, Orient, Decide, and Act) and the “need to get inside the enemy’s OODA loop” so they can shoot first. Businesses too have decision cycles. Delivering a perfectly accurate forecast the day after it was needed is not helpful. Better is a good forecast that arrives in time to be useful.

          Cost: Forecasting data, models, processes and people all cost money.  A less expensive forecast might be fueled by data that are readily available; more expensive would be a forecast that runs on data that have to be collected in a special process outside the scope of a firm’s information infrastructure.  A classic, off-the-shelf forecasting technique will be less costly to acquire, feed and exploit than a complex, custom, consultant-supplied method. Forecasts could be mass-produced by software overseen by a single analyst, or they might emerge from a collaborative process requiring time and effort from large groups of people, such as district sales managers, production teams, and others. Technically advanced forecasting techniques often require hiring staff with specialized technical expertise, such as a master’s degree in statistics, who tend to cost more than staff with less advanced training.

          Credibility: Ultimately, some executive has to accept and act on each forecast. Executives have a tendency to distrust or ignore recommendations that they can neither understand nor explain to the next person above them in the hierarchy. For many, believing in a “black box” is too severe a test of faith, and they reject the black box’s forecasts in favor of something more transparent.

          All that said, we will focus now on forecast accuracy and its evil twin, forecast error.

          Sources of Forecast Error

          Those seeking to reduce error can look in three places to find trouble:
          1. The data that goes into a forecasting model
          2. The model itself
          3. The context of the forecasting exercise

          There are several ways in which data problems can lead to forecast error.

          Gross errors: Wrong data produce wrong forecasts. We have seen an instance in which computer records of product demand were wrong by a factor of two! Those involved spotted that problem immediately, but a less egregious situation can easily slip through to poison the forecasting process. In fact, just organizing, acquiring and checking data is often the largest source of delay in the implementation of forecasting software. Many data problems seem to derive from the data having been unimportant until a forecasting project made them important.

          Anomalies: Even with perfectly curated forecasting databases, there are often “needle in a haystack” type data problems. In these cases, it is not data errors but demand anomalies that contribute to forecast error. In a set of, say, 50,000 products, some number of items are likely to have odd details that can distort forecasts.

          Holdout analysis is a simple but powerful method of analysis. To see how well a method forecasts, use it with older known data to forecast newer data, then see how it would have turned out! For instance, suppose you have 36 months of demand data and need to forecast 3 months ahead. You can simulate the forecasting process by holding out (i.e., hiding) the most recent 3 months of data, forecasting using only data from months 1 to 33, then comparing the forecasts for months 34-36 against the actual values in months 34-36. Sliding simulation merely repeats the holdout analysis, sliding along the demand history. The example above used the first 33 months of data to get 3 estimates of forecast error. Suppose we start the process by using the first 12 months to forecast the next 3. Then we slide forward and use the first 13 months to forecast the next 3. We continue until finally we use the first 35 months to forecast the last month, giving us one more estimate of the error we make when forecasting one month ahead. Summarizing all the 1-step ahead, 2-step ahead and 3-step ahead forecast errors provides a way to calculate prediction intervals.

          Calculating Prediction Intervals

          The final step in calculating prediction intervals is to convert the estimates of average absolute error into the upper and lower limits of the prediction interval. The prediction interval at any future time is computed as

          Prediction interval = Forecast ± Multiplier x Average absolute error.

          The final step is the choice of the multiplier. The typical approach is to imagine some probability distribution of error around the forecast, then estimate the ends of the prediction interval using appropriate percentiles of that distribution. Usually, the assumed distribution of error is the Normal distribution, also called the Gaussian distribution or the “bell-shaped curve”.

          Use of Prediction Intervals
          The most immediate, informal use of prediction intervals is to convey a sense of how “squishy” a forecast is. Prediction intervals that are wide compared to the size of the forecasts indicate high uncertainty.

          There are two more formal uses in demand forecasting: Hedging your bets about future demand and guiding forecast adjustment.

          Hedging your bets: The forecast values themselves approximate the most likely values of future demand. A more ominous way to say the same thing is that there is about a 50% chance that the actual value will be above (or below) the forecast. If the forecast is being used to plan future production (or raw materials purchase or hiring), you might want to build in a cushion to keep from being caught short if demand spikes (assuming that under-building is worse than over-building). If the forecast is converted from units to dollars for revenue projections, you might want to use a value below the forecast to be conservative in projecting cash flow. In either case, you first have to choose the coverage of the prediction interval. A 90% prediction interval is a range of values that covers 90% of the possibilities. This implies that there is a 5% chance of a value falling above the upper limit of the 90% prediction interval. In other words, the upper limit of a 90% prediction interval marks the 95th percentile of the distribution of predicted demand at that time period. Similarly, there is a 5% chance of falling below the lower limit, which marks the 5th percentile of the demand distribution.

          Guiding forecast adjustment: It is quite common for statistical forecasts to be revised by some sort of collaborative process. These adjustments are based on information not recorded in an item’s demand history, such as intelligence about competitor actions. Sometimes they are based on a more vaporous source, such as sales force optimism. When the adjustments are made on-screen for all to see, the prediction intervals provide a useful reference: If someone wants to move the forecasts outside the prediction intervals, they are crossing a fact-based line and should have a good story to justify their argument that things will be really different in the future.

          Prediction Intervals and Inventory Optimization

          Finally, the concept behind prediction intervals play an essential role in a problem related to demand forecasting: Inventory Optimization.
          The core analytic task in setting reorders points (also called Mins) is to forecast total demand over a replenishment lead time. This total is called the lead time demand. When on-hand inventory falls down to or below the reorder point, a replenishment order is triggered. If the reorder point is high enough, there will be an acceptably small risk of a stockout, i.e., of lead time demand driving inventory below zero and creating either lost sales or backorders.

          SDP_Screenshot new statistical methods planning

          New statistical methods, and we can start planning more effectively.

          The forecasting task is to determine all the possible values of cumulative demand over the lead time and their associated probabilities of occurring. In other words, the basic task is to determine a prediction interval for some future random variable. Suppose you have computed a 90% prediction interval for lead time demand. Then the upper end of the interval represents the 95th percentile of the distribution. Setting the reorder point at this level will accommodate 95% of the possible lead time demand values, meaning there will be only a 5% chance of stocking out before replenishment arrives to re-stock the shelves. Thus there is an intimate relationship between prediction intervals in demand forecasting and calculation of reorder points in inventory optimization.

           

          5 Recommendations for Practice

          1. Set expectations about error: Sometimes  managers have unreasonable expectations about reducing forecast error to zero. You can point out that error is only one of the dimensions on which a forecasting process must be judged; you may be doing fine on both timeliness and cost. Also point out that zero error is no more realistic a goal than 100% conversion of prospects into customers, perfect supplier performance, or zero stock price volatility.

          2. Track down sources of error: Double check the accuracy of demand histories. Use statistical methods to identify outliers in demand histories and react appropriately, replacing verified anomalies with more typical values and omitting data from before major changes in the character of the demand. If you use a collaborative forecasting process, compare its accuracy against a purely statistical approach to identify items for which collaboration does not reduce error.

          3. Evaluate the error of alternative statistical methods: There may be off-the-shelf techniques that do better than your current methods, or do better for some subsets of your items. The key is to be empirical, using the idea of holdout analysis. Gather your data and do a “bake off” between different methods to see which work better for you. If you are not already using statistical forecasting methods, compare them against whoever’s “golden gut” is your current standard. Use the naïve forecast as a benchmark in the comparisons.

          4. Investigate the use of new data sources: Especially if you have items that are heavily promoted, test out statistical methods that incorporate promotional data into the forecasting process. Also check whether information from outside your company can be exploited; for instance, see whether macroeconomic indicators for your sector can be combined with company data to improve forecast accuracy (this is usually done using a method called multiple regression analysis).

          5. Use prediction intervals: Plots of prediction intervals can improve your feel for the uncertainty in your forecasts, helping you select items for additional scrutiny. While it’s true that what you don’t know can hurt you, it’s also true that knowing what you don’t know can help you.

          Leave a Comment

          Related Posts

          Maximize Machine Uptime with Probabilistic Modeling

          Maximize Machine Uptime with Probabilistic Modeling

          If you both make and sell things, you own two inventory problems. Companies that sell things must focus relentlessly on having enough product inventory to meet customer demand. Manufacturers and asset intensive industries such as power generation, public transportation, mining, and refining, have an additional inventory concern: having enough spare parts to keep their machines running.
          This technical brief reviews the basics of two probabilistic models of machine breakdown. It also relates machine uptime to the adequacy of spare parts inventory.

          Want to Optimize Inventory? Follow These 4 Steps

          Want to Optimize Inventory? Follow These 4 Steps

          Service Level Driven Planning (SLDP) is an approach to inventory planning based on exposing the tradeoffs between SKU availability and inventory cost that are at the root of all wise inventory decisions. When organizations understand these tradeoffs, they can make better decisions and have greater variability into the risk of stockouts. SLDP unfolds in four steps: Benchmark, Collaborate, Plan, and Track.

          Four Ways to Optimize Inventory

          Four Ways to Optimize Inventory

          Inventory optimization has become an even higher priority in recent months for many of our customers.  Some are finding their products in vastly greater demand; more have the opposite problem. In either case, events like the Covid19 pandemic are forcing a reexamination of standard operating conditions, such as choices of reorder points and order quantities.

          Recent Posts

          • Ship and Supply Chain Blockage in Suez CanalRedefine Exceptions and Fine Tune Planning to Address Uncertainty
            In a perfect world, Just in Time (JIT) would be the appropriate solution for inventory management. But as the saying goes “everyone has a plan until they get punched in the mouth.” One enormous punch in the mouth for the global supply chain was Suez Canal Blockage that held up $9.6B in trade costing an estimated $6.7M per minute. […]
          • Inventory Planning Processes Challenges OpportunitiesInventory Planning Processes: Challenges and Opportunities
            Smart Software is pleased to introduce our new series of educational webinars, offered exclusively for Epicor Users. Greg Hartunian, CEO at Smart Software, will lead a 45-minute webinar focusing on specific approaches to demand forecasting and inventory planning. […]

            Cloud computing companies with unique server and hardware parts, e-commerce, online retailers, home and office supply companies, onsite furniture, power utilities, intensive assets maintenance or warehousing for water supply companies have increased their activity during the pandemic. Garages selling car parts and truck parts, pharmaceuticals, healthcare or medical supply manufacturers and safety product suppliers are dealing with increasing demand. Delivery service companies, cleaning services, liquor stores and canned or jarred goods warehouses, home improvement stores, gardening suppliers, yard care companies, hardware, kitchen and baking supplies stores, home furniture suppliers with high demand are facing stockouts, long lead times, inventory shortage costs, higher operating costs and ordering costs.

            Reveal Your Real Inventory Planning and Forecasting Policy by Answering These 10 Questions

            The Smart Forecaster

             Pursuing best practices in demand planning,

            forecasting and inventory optimization

            In another blog we posed the question:  How can you be sure that you really have a policy for inventory planning and demand forecasting? We explained how an organization’s lack of understanding on the basics (how a forecast is created, how safety stock buffers are determined, and how/why these values are adjusted) contributes to poor forecast accuracy, misallocated inventory, and lack of trust in the whole process.

            In this blog, we review 10 specific questions you can ask to uncover what’s really happening at your company. We detail the typical answers provided when a forecasting/inventory planning policy doesn’t really exist, explain how to interpret these answers, and offer some clear advice on what to do about it.

            Always start with a simple hypothetical example. Focusing on a specific problem you just experienced is bound to provoke defensive answers that hide the full story. The goal is to uncover the actual approach used to plan inventory and forecasts that has been baked into the mental math or spreadsheets.   Here is an example:

            Suppose you have 100 units on hand, the lead time to replenish is 3 months, and the average monthly demand is 20 units?   When should you order more?  How much would you order? How will your answer change if expected receipts of 10 per month were scheduled to arrive?  How will your answer change if the item is the item is an A, B, or C item, the cost of the item is high or low, lead time of the item is long or short?  Simply put, when you schedule a production job or place a new order with a supplier, why did you do it? What triggered the decision to get more?  What planning inputs were considered?

            When getting answers to the above question, focus on uncovering answers to the following questions:

            1. What is the underlying replenishment approach? This will typically be one of Min/Max, forecast/safety stock, Reorder Point/Order Quantity, Periodic Review/Order Up To or even some odd combination

            2. How are the planning parameters, such as demand forecasts, reorder points, or Min/Max, actually calculated? It’s not enough to know that you use Min/Max.  You have to know exactly how these values are calculated. Answers such as “We use history” or “We use an average” are not specific enough.   You’ll need answers that clearly outline how history is used.  For example, “We take an average of the last 6 months, divide that by 30 to get a daily average, and then multiply that by the lead time in days.  For ‘A’ items we then multiply the lead time average by 2 and for ‘B’ items we use a multiplier of 1.5.” (While that is not an especially good technical approach, at least it has a clear logic.)

            Once you have a policy well-defined, you can identify its weaknesses in order to improve it.  But if the answer provided doesn’t get much further past “We use history”, then you don’t have a policy to start with.   Answers will often reveal that different planners use history in different ways.  Some may only consider the most recent demand, others might stock according to the average of the highest demand periods, etc.  In other words, you may find that you actually have multiple ill-conceived “policies”.

            3. Are forecasts used to drive replenishment planning and if so, how? Many companies will say they forecast, but their forecasts are calculated and used differently. Is the forecast used to predict what on hand inventory will be in the future, resulting in an order being triggered?  Or is it used to derive a reorder point but not to predict when to order (i.e. I predict we’ll sell 10 a week so to help protect against stock out, I’ll order more when on hand gets to 15)? Is it used as a guide for the planner to help subjectively determine when they should order more?  Is it used to set up blanket orders with suppliers?  Some use it to drive MRP. You’ll need to know these specifics.  A thorough answer to this question might look like this: “My forecast is 10 per week and my lead time is 3 weeks so I make my reorder point a multiple of that forecast, typically 2 x lead time demand or 60 unit for important items and I use a smaller multiple for less important items.  (Again, not a great technical approach, but clear.)

            4.  What technique is actually used to generate the forecast? Is it an average, a trending model such as double exponential smoothing, a seasonal model? Does the choice of technique change depend on the type of demand data or when new demand data is available? (Spare parts and high-volume items have very different demand patterns.) How do you go about selecting the forecast model? Is this process automated?  How often is the choice of model reconsidered?  How often are the model parameters recomputed? What is the process used to reconsider your approach?  The answer here documents how the baseline forecasts are produced.  Once determined, you can conduct an analysis to identify whether other forecasting methods would improve forecast accuracy.  If you aren’t documenting forecast accuracy and conducting “forecast value add” analysis then you aren’t in a position to properly assess whether the forecasts being produced are the best that they can be.  You’ll miss out on opportunities to improve the process, increase forecast accuracy, and educate the business on what type of forecast error is normal and should be expected.

            5. How do you use safety stock? Notice the question was not “Do you use safety stock?” In this context, and to keep it simple, the term “safety stock” means stock used to buffer inventory against supply and demand variability.  All companies use buffering approaches in some way.  There are some exceptions though.  Maybe you are a job shop manufacturer that procures all parts to order and your customers are completely fine waiting weeks or months for you to source material, manufacture, QA, and ship.  Or maybe you are high-volume manufacturer with tons of buying power so your suppliers set up local warehouses that are stocked full and ready to provide inventory to you almost immediately.  If these descriptions don’t describe your company, you will definitely have some sort of buffer to protect against demand and supply variability.  You may not use the “safety stock” field in your ERP but you are definitely buffering.

            Answers might be provided such as “We don’t use safety stock because we forecast.”  Unfortunately, a good forecast will have a 50/50 chance of being over/under the actual demand.  This means you’ll incur a stock out 50% of the time without a safety stock buffer added to the forecast.  Forecasts are only perfect when there is no randomness. Since there is always randomness, you’ll need to buffer if you don’t want to have abysmal service levels.

            If the answer isn’t revealed, you can probe a bit more into how the varying replenishment levers are used to add possible buffers which leads to questions 6 & 7.

            6. Do you ever increase the lead time or order earlier than you truly need to?
            In our hypothetical example, your supplier typically takes 4 weeks to deliver and is pretty consistent. But to protect against stockouts your buyer routinely orders 6 weeks out instead of 4 weeks.  The safety stock field in your ERP system might be set to zero because “we don’t use safety stock”, but in reality, the buyer’s ordering approach just added 2 weeks of buffer stock.

            7. Do you pad the demand forecast?
            In our example, the planner expects to consume 10 units per month but “just in case” enters a forecast of 20 per month.  The safety stock field in the MRP system is left blank but the now disguised buffer stock has been smuggled into the demand forecast.  This is a mistake that introduces “forecast bias.”  Not only will your forecasts be less accurate but if the bias isn’t accounted for and safety stock is added by other departments, you will overstock.

            The ad-hoc nature of the above approaches compounds the problems by not considering the actual demand or supply variability of the item. For example, the planner might simply make a rule of thumb that doubles the lead time forecast for important items.  One-size doesn’t fit all when it comes to inventory management.  This approach will substantially overstock the predictable items while substantially understocking the intermittently demanded items. You can read “Beware of Simple Rules of Thumb for Managing Inventory” to learn more about why this type of approach is so costly.

            The ad-hoc nature of the approaches also ignores what happens the company is faced with a huge overstock or stock out. When trying to understand what happened, the stated policies will be examined. In the case of an overstock, the system will show zero safety stock.  The business leaders will assume they aren’t carrying any safety stock, scratch their heads, and eventually just blame the forecast, declare “Our business can’t be forecasted” and stumble on. They may even blame the supplier for shipping too early and making them hold more than needed. In the case of a stock out, they will think they aren’t carrying enough and arbitrarily add more stock across many items not realizing there is in fact lots of extra safety stock baked into process.  This makes it more likely inventory will need to be written off in the future.

            8. What is the exact inventory terminology used? Define what you mean by safety stock, Min, reorder point, EOQ, etc.  While there are standard technical definitions it’s possible that something differs, and miscommunication here will be problematic.  For example, some companies refer to Min as the amount of inventory needed to satisfy lead time demand while some may define Min as inclusive of both lead time demand and safety stock to buffer against demand variability. Others may mean the minimum order quantity.

            9. Is on hand inventory consistent with the policy? When your detective work is done and everything is documented, open your spreadsheet or ERP system and look at the on-hand quantity. It should be more or less in line with your planning parameters (i.e. if Min/Max is 20/40 and typical lead time demand is 10, then you should have roughly 10 to 40 units on hand at any given point in time.  Surprisingly, for many companies there is often a huge inconsistency. We have observed situations where the Min/Max setting is 20/40 but the on-hand inventory is 300+.  This indicates that whatever policy has been prescribed just isn’t being followed.   That’s a bigger problem.

            10. What are you going to do next?

            Demand forecasting and inventory stocking policy need to be well-defined processes that are understood and accepted by everybody involved.  There should be zero mystery.

            To do this right, the demand and supply variability must be analyzed and used to compute the proper levels of safety stock.   Adding buffers without an implicit understanding of what each additional unit of buffer stock is buying you in terms of service is like arbitrarily throwing a handful of ingredients into a cake recipe.  A small change in ingredients can have a huge impact on what comes out of the oven – one bite too sweet but the next too sour.  It is the same with inventory management.  A little extra here, a little less there, and pretty soon you find yourself with costly excess inventory in some areas, painful shortages in others, no idea how you got there, and with little guidance on how to make things better.

            Modern inventory optimization and demand planning software with its advanced analytics and strong basis in forecast analysis can help a good deal with this problem. But even the best software won’t help if it is used inconsistently.

            Leave a Comment

            Related Posts

            Maximize Machine Uptime with Probabilistic Modeling

            Maximize Machine Uptime with Probabilistic Modeling

            If you both make and sell things, you own two inventory problems. Companies that sell things must focus relentlessly on having enough product inventory to meet customer demand. Manufacturers and asset intensive industries such as power generation, public transportation, mining, and refining, have an additional inventory concern: having enough spare parts to keep their machines running.
            This technical brief reviews the basics of two probabilistic models of machine breakdown. It also relates machine uptime to the adequacy of spare parts inventory.

            Want to Optimize Inventory? Follow These 4 Steps

            Want to Optimize Inventory? Follow These 4 Steps

            Service Level Driven Planning (SLDP) is an approach to inventory planning based on exposing the tradeoffs between SKU availability and inventory cost that are at the root of all wise inventory decisions. When organizations understand these tradeoffs, they can make better decisions and have greater variability into the risk of stockouts. SLDP unfolds in four steps: Benchmark, Collaborate, Plan, and Track.

            Four Ways to Optimize Inventory

            Four Ways to Optimize Inventory

            Inventory optimization has become an even higher priority in recent months for many of our customers.  Some are finding their products in vastly greater demand; more have the opposite problem. In either case, events like the Covid19 pandemic are forcing a reexamination of standard operating conditions, such as choices of reorder points and order quantities.

            Recent Posts

            • Ship and Supply Chain Blockage in Suez CanalRedefine Exceptions and Fine Tune Planning to Address Uncertainty
              In a perfect world, Just in Time (JIT) would be the appropriate solution for inventory management. But as the saying goes “everyone has a plan until they get punched in the mouth.” One enormous punch in the mouth for the global supply chain was Suez Canal Blockage that held up $9.6B in trade costing an estimated $6.7M per minute. […]
            • Inventory Planning Processes Challenges OpportunitiesInventory Planning Processes: Challenges and Opportunities
              Smart Software is pleased to introduce our new series of educational webinars, offered exclusively for Epicor Users. Greg Hartunian, CEO at Smart Software, will lead a 45-minute webinar focusing on specific approaches to demand forecasting and inventory planning. […]

              The average monthly demand is 20 unitsand the lead time is 90 days When should you order more? Cloud computing companies with unique server and hardware parts, e-commerce, online retailers, home and office supply companies, onsite furniture, power utilities, intensive assets maintenance or warehousing for water supply companies have increased their activity during the pandemic. Garages selling car parts and truck parts, pharmaceuticals, healthcare or medical supply manufacturers and safety product suppliers are dealing with increasing demand. Delivery service companies, cleaning services, liquor stores and canned or jarred goods warehouses, home improvement stores, gardening suppliers, yard care companies, hardware, kitchen and baking supplies stores, home furniture suppliers with high demand are facing stockouts, long lead times, inventory shortage costs, higher operating costs and ordering costs.