Improve Forecast Accuracy by Managing Error

The Smart Forecaster

 Pursuing best practices in demand planning,

forecasting and inventory optimization

Improve Forecast Accuracy, Eliminate Excess Inventory, & Maximize Service Levels

In this video, Dr. Thomas Willemain, co-Founder and SVP Research, talks about improving Forecast Accuracy by Managing Error. This video is the first in our series on effective methods to Improve Forecast Accuracy.  We begin by looking at how forecast error causes pain and the consequential cost related to it. Then we will explain the three most common mistakes to avoid that can help us increase revenue and prevent excess inventory. Tom concludes by reviewing the methods to improve Forecast Accuracy, the importance of measuring forecast error, and the technological opportunities to improve it.

Download the Presentation

 

Forecast error can be consequential

Consider one item of many

  • Product X costs $100 to make and nets $50 profit per unit.
  • Sales of Product X will turn out to be 1,000/month over the next 12 months.
  • Consider one item of many

What is the cost of forecast error?

  • If the forecast is 10% high, end the year with $120,000 of excess inventory.
  • 100 extra/month x 12 months x $100/unit
  • If the forecast is 10% low, miss out on $60,000 of profit.
  • 100 too few/month x 12 months x $50/unit

 

Three mistakes to avoid

1. Ignoring error.

  • Unprofessional, dereliction of duty.
  • Wishing will not make it so.
  • Treat accuracy assessment as data science, not a blame game.

2. Tolerating more error than necessary.

  • Statistical forecasting methods can improve accuracy at scale.
  • Improving data inputs can help.
  • Collecting and analyzing forecast error metrics can identify weak spots.

3. Wasting time and money going too far trying to eliminate error.

  • Some product/market combinations are inherently more difficult to forecast. After a point, let them be (but be alert for new specialized forecasting methods).
  • Sometimes steps meant to reduce error can backfire (e.g., adjustment).
Leave a Comment

RECENT POSTS

The Supply Chain Blame Game: Top 3 Excuses for Inventory Shortage and Excess

The Supply Chain Blame Game: Top 3 Excuses for Inventory Shortage and Excess

The supply chain has become the blame game for almost any industrial or retail problem. Shortages on lead time variability, bad forecasts, and problems with bad data are facts of life, yet inventory-carrying organizations are often caught by surprise when any of these difficulties arise. So, again, who is to blame for the supply chain chaos? Keep reading this blog and we will try to show you how to prevent product shortages and overstocking.

Recent Posts

  • Epicor Prophet 21 with Forecasting Inventory PlanningExtend Epicor Prophet 21 with Smart IP&O’s Forecasting & Dynamic Reorder Point Planning
    Smart Inventory Planning & Optimization (Smart IP&O) can help with inventory ordering functionality in Epicor P21, reduce inventory, minimize stockouts and restore your organization’s trust by providing robust predictive analytics, consensus-based forecasting, and what-if scenario planning. […]
  • Supply Chain Math large-scale decision-making analyticsSupply Chain Math: Don’t Bring a Knife to a Gunfight
    Math and the supply chain go hand and hand. As supply chains grow, increasing complexity will drive companies to look for ways to manage large-scale decision-making. Math is a fact of life for anyone in inventory management and demand forecasting who is hoping to remain competitive in the modern world. Read our article to learn more. […]
  • Mature bearded mechanic in uniform examining the machine and repairing it in factoryService Parts Planning: Planning for consumable parts vs. Repairable Parts
    When deciding on the right stocking parameters for spare and replacement parts, it is important to distinguish between consumable and repairable servoce parts. These differences are often overlooked by inventory planning software and can result in incorrect estimates of what to stock. Different approaches are required when planning for consumables vs. repairable service parts. […]
  • Four Common Mistakes when Planning Replenishment TargetsFour Common Mistakes when Planning Replenishment Targets
    How often do you recalibrate your stocking policies? Why? Learn how to avoid key mistakes when planning replenishment targets by automating the process, recalibrating parts, using targeting forecasting methods, and reviewing exceptions. […]
  • Smart Software is pleased to introduce our series of webinars, offered exclusively for Epicor Users.Extend Epicor Kinetic’s Forecasting & Min/Max Planning with Smart IP&O
    Epicor Kinetic can manage replenishment by suggesting what to order and when via reorder point-based inventory policies. The problem is that the ERP system requires that the user either manually specify these reorder points, or use a rudimentary “rule of thumb” approach based on daily averages. In this article, we will review the inventory ordering functionality in Epicor Kinetic, explain its limitations, and summarize how to reduce inventory, and minimize stockouts by providing the robust predictive functionality that is missing in Epicor. […]

    Inventory Optimization for Manufacturers, Distributors, and MRO

    • Blanket Orders Smart Software Demand and Inventory Planning HDBlanket Orders
      Our customers are great teachers who have always helped us bridge the gap between textbook theory and practical application. A prime example happened over twenty years ago, when we were introduced to the phenomenon of intermittent demand, which is common among spare parts but rare among the finished goods managed by our original customers working in sales and marketing. This revelation soon led to our preeminent position as vendors of software for managing inventories of spare parts. Our latest bit of schooling concerns “blanket orders.” […]
    • Hand placing pieces to build an arrowProbabilistic Forecasting for Intermittent Demand
      The New Forecasting Technology derives from Probabilistic Forecasting, a statistical method that accurately forecasts both average product demand per period and customer service level inventory requirements. […]
    • Engineering to Order at Kratos Space – Making Parts Availability a Strategic Advantage
      The Kratos Space group within National Security technology innovator Kratos Defense & Security Solutions, Inc., produces COTS s software and component products for space communications - Making Parts Availability a Strategic Advantage […]
    • wooden-figures-of-people-and-a-magnet-team-management-warehouse inventoryManaging the Inventory of Promoted Items
      In a previous post, I discussed one of the thornier problems demand planners sometimes face: working with product demand data characterized by what statisticians call skewness—a situation that can necessitate costly inventory investments. This sort of problematic data is found in several different scenarios. In at least one, the combination of intermittent demand and very effective sales promotions, the problem lends itself to an effective solution. […]

        Four Useful Ways to Measure Forecast Error

        The Smart Forecaster

         Pursuing best practices in demand planning,

        forecasting and inventory optimization

        Improve Forecast Accuracy, Eliminate Excess Inventory, & Maximize Service Levels

        In this video, Dr. Thomas Willemain, co-Founder and SVP Research, talks about improving forecast accuracy by measuring forecast error. We begin by overviewing the various types of error metrics: scale-dependent error, percentage error, relative error, and scale-free error metrics. While some error is inevitable, there are ways to reduce it, and forecast metrics are necessary aids for monitoring and improving forecast accuracy. Then we will explain the special problem of intermittent demand and divide-by-zero problems. Tom concludes by explaining how to assess forecasts of multiple items and how it often makes sense to use weighted averages, weighting items differently by volume or revenue.

        Download the Presentation

         

        Four general types of error metrics 

        1. Scale-dependent error
        2. Percentage error
        3. Relative error
        4 .Scale-free error

        Remark: Scale-dependent metrics are expressed in the units of the forecasted variable. The other three are expresses as percentages.

         

        1. Scale-dependent error metrics

        • Mean Absolute Error (MAE) aka Mean Absolute Deviation (MAD)
        • Median Absolute Error (MdAE)
        • Root Mean Square Error (RMSE)
        • These metrics express the error in the original units of the data.
          • Ex: units, cases, barrels, kilograms, dollars, liters, etc.
        • Since forecasts can be too high or too low, the signs of the errors will be either positive or negative, allowing for unwanted cancellations.
          • Ex: You don’t want errors of +50 and -50 to cancel and show “no error”.
        • To deal with the cancellation problem, these metrics take away negative signs by either squaring or using absolute value.

         

        2. Percentage error metric

        • Mean Absolute Percentage Error (MAPE)
        • This metric expresses the size of the error as a percentage of the actual value of the forecasted variable.
        • The advantage of this approach is that it immediately makes clear whether the error is a big deal or not.
        • Ex: Suppose the MAE is 100 units. Is a typical error of 100 units horrible? ok? great?
        • The answer depends on the size of the variable being forecasted. If the actual value is 100, then a MAE = 100 is as big as the thing being forecasted. But if the actual value is 10,000, then a MAE = 100 shows great accuracy, since the MAPE is only 1% of the actual.

         

        3. Relative error metric

        • Median Relative Absolute Error (MdRAE)
        • Relative to what? To a benchmark forecast.
        • What benchmark? Usually, the “naïve” forecast.
        • What is the naïve forecast? Next forecast value = last actual value.
        • Why use the naïve forecast? Because if you can’t beat that, you are in tough shape.

         

        4. Scale-Free error metric

        • Median Relative Scaled Error (MdRSE)
        • This metric expresses the absolute forecast error as a percentage of the natural level of randomness (volatility) in the data.
        • The volatility is measured by the average size of the change in the forecasted variable from one time period to the next.
          • (This is the same as the error made by the naïve forecast.)
        • How does this metric differ from the MdRAE above?
          • They do both use the naïve forecast, but this metric uses errors in forecasting the demand history, while the MdRAE uses errors in forecasting future values.
          • This matters because there are usually many more history values than there are forecasts.
          • In turn, that matters because this metric would “blow up” if all the data were zero, which is less likely when using the demand history.

         

        Intermittent Demand Planning and Parts Forecasting

         

        The special problem of intermittent demand

        • “Intermittent” demand has many zero demands mixed in with random non-zero demands.
        • MAPE gets ruined when errors are divided by zero.
        • MdRAE can also get ruined.
        • MdSAE is less likely to get ruined.

         

        Recap and remarks

        • Forecast metrics are necessary aids for monitoring and improving forecast accuracy.
        • There are two major classes of metrics: absolute and relative.
        • Absolute measures (MAE, MdAE, RMSE) are natural choices when assessing forecasts of one item.
        • Relative measures (MAPE, MdRAE, MdSAE) are useful when comparing accuracy across items or between alternative forecasts of the same item or assessing accuracy relative to the natural variability of an item.
        • Intermittent demand presents divide-by-zero problems which favor MdSAE over MAPE.
        • When assessing forecasts of multiple items, it often makes sense to use weighted averages, weighting items differently by volume or revenue.
        Leave a Comment

        RECENT POSTS

        The Supply Chain Blame Game: Top 3 Excuses for Inventory Shortage and Excess

        The Supply Chain Blame Game: Top 3 Excuses for Inventory Shortage and Excess

        The supply chain has become the blame game for almost any industrial or retail problem. Shortages on lead time variability, bad forecasts, and problems with bad data are facts of life, yet inventory-carrying organizations are often caught by surprise when any of these difficulties arise. So, again, who is to blame for the supply chain chaos? Keep reading this blog and we will try to show you how to prevent product shortages and overstocking.

        Recent Posts

        • Epicor Prophet 21 with Forecasting Inventory PlanningExtend Epicor Prophet 21 with Smart IP&O’s Forecasting & Dynamic Reorder Point Planning
          Smart Inventory Planning & Optimization (Smart IP&O) can help with inventory ordering functionality in Epicor P21, reduce inventory, minimize stockouts and restore your organization’s trust by providing robust predictive analytics, consensus-based forecasting, and what-if scenario planning. […]
        • Supply Chain Math large-scale decision-making analyticsSupply Chain Math: Don’t Bring a Knife to a Gunfight
          Math and the supply chain go hand and hand. As supply chains grow, increasing complexity will drive companies to look for ways to manage large-scale decision-making. Math is a fact of life for anyone in inventory management and demand forecasting who is hoping to remain competitive in the modern world. Read our article to learn more. […]
        • Mature bearded mechanic in uniform examining the machine and repairing it in factoryService Parts Planning: Planning for consumable parts vs. Repairable Parts
          When deciding on the right stocking parameters for spare and replacement parts, it is important to distinguish between consumable and repairable servoce parts. These differences are often overlooked by inventory planning software and can result in incorrect estimates of what to stock. Different approaches are required when planning for consumables vs. repairable service parts. […]
        • Four Common Mistakes when Planning Replenishment TargetsFour Common Mistakes when Planning Replenishment Targets
          How often do you recalibrate your stocking policies? Why? Learn how to avoid key mistakes when planning replenishment targets by automating the process, recalibrating parts, using targeting forecasting methods, and reviewing exceptions. […]
        • Smart Software is pleased to introduce our series of webinars, offered exclusively for Epicor Users.Extend Epicor Kinetic’s Forecasting & Min/Max Planning with Smart IP&O
          Epicor Kinetic can manage replenishment by suggesting what to order and when via reorder point-based inventory policies. The problem is that the ERP system requires that the user either manually specify these reorder points, or use a rudimentary “rule of thumb” approach based on daily averages. In this article, we will review the inventory ordering functionality in Epicor Kinetic, explain its limitations, and summarize how to reduce inventory, and minimize stockouts by providing the robust predictive functionality that is missing in Epicor. […]

          Inventory Optimization for Manufacturers, Distributors, and MRO

          • Blanket Orders Smart Software Demand and Inventory Planning HDBlanket Orders
            Our customers are great teachers who have always helped us bridge the gap between textbook theory and practical application. A prime example happened over twenty years ago, when we were introduced to the phenomenon of intermittent demand, which is common among spare parts but rare among the finished goods managed by our original customers working in sales and marketing. This revelation soon led to our preeminent position as vendors of software for managing inventories of spare parts. Our latest bit of schooling concerns “blanket orders.” […]
          • Hand placing pieces to build an arrowProbabilistic Forecasting for Intermittent Demand
            The New Forecasting Technology derives from Probabilistic Forecasting, a statistical method that accurately forecasts both average product demand per period and customer service level inventory requirements. […]
          • Engineering to Order at Kratos Space – Making Parts Availability a Strategic Advantage
            The Kratos Space group within National Security technology innovator Kratos Defense & Security Solutions, Inc., produces COTS s software and component products for space communications - Making Parts Availability a Strategic Advantage […]
          • wooden-figures-of-people-and-a-magnet-team-management-warehouse inventoryManaging the Inventory of Promoted Items
            In a previous post, I discussed one of the thornier problems demand planners sometimes face: working with product demand data characterized by what statisticians call skewness—a situation that can necessitate costly inventory investments. This sort of problematic data is found in several different scenarios. In at least one, the combination of intermittent demand and very effective sales promotions, the problem lends itself to an effective solution. […]