1-800-SMART-99
Improve Forecast Accuracy by Managing Error

The Smart Forecaster

 Pursuing best practices in demand planning,

forecasting and inventory optimization

Improve Forecast Accuracy, Eliminate Excess Inventory, & Maximize Service Levels

In this video, Dr. Thomas Willemain, co-Founder and SVP Research, talks about improving Forecast Accuracy by Managing Error. This video is the first in our series on effective methods to Improve Forecast Accuracy.  We begin by looking at how forecast error causes pain and the consequential cost related to it. Then we will explain the three most common mistakes to avoid that can help us increase revenue and prevent excess inventory. Tom concludes by reviewing the methods to improve Forecast Accuracy, the importance of measuring forecast error, and the technological opportunities to improve it.

Download the Presentation

 

Forecast error can be consequential

Consider one item of many

  • Product X costs $100 to make and nets $50 profit per unit.
  • Sales of Product X will turn out to be 1,000/month over the next 12 months.
  • Consider one item of many

What is the cost of forecast error?

  • If the forecast is 10% high, end the year with $120,000 of excess inventory.
  • 100 extra/month x 12 months x $100/unit
  • If the forecast is 10% low, miss out on $60,000 of profit.
  • 100 too few/month x 12 months x $50/unit

 

Three mistakes to avoid

1. Ignoring error.

  • Unprofessional, dereliction of duty.
  • Wishing will not make it so.
  • Treat accuracy assessment as data science, not a blame game.

2. Tolerating more error than necessary.

  • Statistical forecasting methods can improve accuracy at scale.
  • Improving data inputs can help.
  • Collecting and analyzing forecast error metrics can identify weak spots.

3. Wasting time and money going too far trying to eliminate error.

  • Some product/market combinations are inherently more difficult to forecast. After a point, let them be (but be alert for new specialized forecasting methods).
  • Sometimes steps meant to reduce error can backfire (e.g., adjustment).
Leave a Comment

RECENT POSTS

Confused about AI and Machine Learning?

Confused about AI and Machine Learning?

Are you confused about what is AI and what is machine learning? Are you unsure why knowing more will help you with your job in inventory planning? Don’t despair. You’ll be ok, and we’ll show you how some of whatever-it-is can be useful.

Centering Act: Spare Parts Timing, Pricing, and Reliability

Centering Act: Spare Parts Timing, Pricing, and Reliability

In this article, we’ll walk you through the process of crafting a spare parts inventory plan that prioritizes availability metrics such as service levels and fill rates while ensuring cost efficiency. We’ll focus on an approach to inventory planning called Service Level-Driven Inventory Optimization. Next, we’ll discuss how to determine what parts you should include in your inventory and those that might not be necessary. Lastly, we’ll explore ways to enhance your service-level-driven inventory plan consistently.

Recent Posts

  • What is Inventory Control Planning Management Optimization DictionaryWhat is Inventory Planning? A Brief Dictionary of Inventory-Related Terms
    People involved in the supply chain are likely to have questions about various inventory terms and methods used in their jobs. This note may help by explaining these terms and showing how they relate. […]
  • artificial intelligence ai and machine learning inventory managementConfused about AI and Machine Learning?
    Are you confused about what is AI and what is machine learning? Are you unsure why knowing more will help you with your job in inventory planning? Don’t despair. You’ll be ok, and we’ll show you how some of whatever-it-is can be useful. […]
  • Centering Act Spare Parts Timing Pricing and ReliabilityCentering Act: Spare Parts Timing, Pricing, and Reliability
    In this article, we'll walk you through the process of crafting a spare parts inventory plan that prioritizes availability metrics such as service levels and fill rates while ensuring cost efficiency. We'll focus on an approach to inventory planning called Service Level-Driven Inventory Optimization. Next, we'll discuss how to determine what parts you should include in your inventory and those that might not be necessary. Lastly, we'll explore ways to enhance your service-level-driven inventory plan consistently. […]
  • Balance,Concept,With,Chrome,Balls,inventory optimization softwareHow to Forecast Inventory Requirements
    Forecasting inventory requirements is a specialized variant of forecasting that focuses on the high end of the range of possible future demand. Traditional methods often rely on bell-shaped demand curves, but this isn't always accurate. In this article, we delve into the complexities of this practice, especially when dealing with intermittent demand. […]
  • Demand Planning twin brothers holding forecasting toolsSix Demand Planning Best Practices You Should Think Twice About
    Every field, including forecasting, accumulates folk wisdom that eventually starts masquerading as “best practices.” These best practices are often wise, at least in part, but they often lack context and may not be appropriate for certain customers, industries, or business situations. There is often a catch, a “Yes, but”. This note is about six usually true forecasting precepts that nevertheless do have their caveats. […]

    Inventory Optimization for Manufacturers, Distributors, and MRO

    • Centering Act Spare Parts Timing Pricing and ReliabilityCentering Act: Spare Parts Timing, Pricing, and Reliability
      In this article, we'll walk you through the process of crafting a spare parts inventory plan that prioritizes availability metrics such as service levels and fill rates while ensuring cost efficiency. We'll focus on an approach to inventory planning called Service Level-Driven Inventory Optimization. Next, we'll discuss how to determine what parts you should include in your inventory and those that might not be necessary. Lastly, we'll explore ways to enhance your service-level-driven inventory plan consistently. […]
    • 5 Steps to Improve the Financial Impact of Spare Parts Planning5 Steps to Improve the Financial Impact of Spare Parts Planning
      In today’s competitive business landscape, companies are constantly seeking ways to improve their operational efficiency and drive increased revenue. Optimizing service parts management is an often-overlooked aspect that can have a significant financial impact. Companies can improve overall efficiency and generate significant financial returns by effectively managing spare parts inventory. This article will explore the economic implications of optimized service parts management and how investing in Inventory Optimization and Demand Planning Software can provide a competitive advantage. […]
    • Bottom Line strategies for Spare Parts Planning SoftwareBottom Line Strategies for Spare Parts Planning
      Managing spare parts presents numerous challenges, such as unexpected breakdowns, changing schedules, and inconsistent demand patterns. Traditional forecasting methods and manual approaches are ineffective in dealing with these complexities. To overcome these challenges, this blog outlines key strategies that prioritize service levels, utilize probabilistic methods to calculate reorder points, regularly adjust stocking policies, and implement a dedicated planning process to avoid excessive inventory. Explore these strategies to optimize spare parts inventory and improve operational efficiency. […]
    • professional technician engineer planning spare parts in industrial manufacturing factory,Prepare your spare parts planning for unexpected shocks
      In today's unpredictable business climate, we do have to worry about supply chain disruptions, long lead times, rising interest rates, and volatile demand. With all these challenges, it's never been more vital for organizations to accurately forecast parts usage, stocking levels, and to optimize replenishment policies such as reorder points, safety stocks, and order quantities. In this blog, we'll explore how companies can leverage innovative solutions like inventory optimization and parts forecasting software that utilize machine learning algorithms, probabilistic forecasting, and analytics to stay ahead of the curve and protect their supply chains from unexpected shocks. […]

      Four Useful Ways to Measure Forecast Error

      The Smart Forecaster

       Pursuing best practices in demand planning,

      forecasting and inventory optimization

      Improve Forecast Accuracy, Eliminate Excess Inventory, & Maximize Service Levels

      In this video, Dr. Thomas Willemain, co-Founder and SVP Research, talks about improving forecast accuracy by measuring forecast error. We begin by overviewing the various types of error metrics: scale-dependent error, percentage error, relative error, and scale-free error metrics. While some error is inevitable, there are ways to reduce it, and forecast metrics are necessary aids for monitoring and improving forecast accuracy. Then we will explain the special problem of intermittent demand and divide-by-zero problems. Tom concludes by explaining how to assess forecasts of multiple items and how it often makes sense to use weighted averages, weighting items differently by volume or revenue.

      Download the Presentation

       

      Four general types of error metrics 

      1. Scale-dependent error
      2. Percentage error
      3. Relative error
      4 .Scale-free error

      Remark: Scale-dependent metrics are expressed in the units of the forecasted variable. The other three are expresses as percentages.

       

      1. Scale-dependent error metrics

      • Mean Absolute Error (MAE) aka Mean Absolute Deviation (MAD)
      • Median Absolute Error (MdAE)
      • Root Mean Square Error (RMSE)
      • These metrics express the error in the original units of the data.
        • Ex: units, cases, barrels, kilograms, dollars, liters, etc.
      • Since forecasts can be too high or too low, the signs of the errors will be either positive or negative, allowing for unwanted cancellations.
        • Ex: You don’t want errors of +50 and -50 to cancel and show “no error”.
      • To deal with the cancellation problem, these metrics take away negative signs by either squaring or using absolute value.

       

      2. Percentage error metric

      • Mean Absolute Percentage Error (MAPE)
      • This metric expresses the size of the error as a percentage of the actual value of the forecasted variable.
      • The advantage of this approach is that it immediately makes clear whether the error is a big deal or not.
      • Ex: Suppose the MAE is 100 units. Is a typical error of 100 units horrible? ok? great?
      • The answer depends on the size of the variable being forecasted. If the actual value is 100, then a MAE = 100 is as big as the thing being forecasted. But if the actual value is 10,000, then a MAE = 100 shows great accuracy, since the MAPE is only 1% of the actual.

       

      3. Relative error metric

      • Median Relative Absolute Error (MdRAE)
      • Relative to what? To a benchmark forecast.
      • What benchmark? Usually, the “naïve” forecast.
      • What is the naïve forecast? Next forecast value = last actual value.
      • Why use the naïve forecast? Because if you can’t beat that, you are in tough shape.

       

      4. Scale-Free error metric

      • Median Relative Scaled Error (MdRSE)
      • This metric expresses the absolute forecast error as a percentage of the natural level of randomness (volatility) in the data.
      • The volatility is measured by the average size of the change in the forecasted variable from one time period to the next.
        • (This is the same as the error made by the naïve forecast.)
      • How does this metric differ from the MdRAE above?
        • They do both use the naïve forecast, but this metric uses errors in forecasting the demand history, while the MdRAE uses errors in forecasting future values.
        • This matters because there are usually many more history values than there are forecasts.
        • In turn, that matters because this metric would “blow up” if all the data were zero, which is less likely when using the demand history.

       

      Intermittent Demand Planning and Parts Forecasting

       

      The special problem of intermittent demand

      • “Intermittent” demand has many zero demands mixed in with random non-zero demands.
      • MAPE gets ruined when errors are divided by zero.
      • MdRAE can also get ruined.
      • MdSAE is less likely to get ruined.

       

      Recap and remarks

      • Forecast metrics are necessary aids for monitoring and improving forecast accuracy.
      • There are two major classes of metrics: absolute and relative.
      • Absolute measures (MAE, MdAE, RMSE) are natural choices when assessing forecasts of one item.
      • Relative measures (MAPE, MdRAE, MdSAE) are useful when comparing accuracy across items or between alternative forecasts of the same item or assessing accuracy relative to the natural variability of an item.
      • Intermittent demand presents divide-by-zero problems which favor MdSAE over MAPE.
      • When assessing forecasts of multiple items, it often makes sense to use weighted averages, weighting items differently by volume or revenue.
      Leave a Comment

      RECENT POSTS

      Confused about AI and Machine Learning?

      Confused about AI and Machine Learning?

      Are you confused about what is AI and what is machine learning? Are you unsure why knowing more will help you with your job in inventory planning? Don’t despair. You’ll be ok, and we’ll show you how some of whatever-it-is can be useful.

      Centering Act: Spare Parts Timing, Pricing, and Reliability

      Centering Act: Spare Parts Timing, Pricing, and Reliability

      In this article, we’ll walk you through the process of crafting a spare parts inventory plan that prioritizes availability metrics such as service levels and fill rates while ensuring cost efficiency. We’ll focus on an approach to inventory planning called Service Level-Driven Inventory Optimization. Next, we’ll discuss how to determine what parts you should include in your inventory and those that might not be necessary. Lastly, we’ll explore ways to enhance your service-level-driven inventory plan consistently.

      Recent Posts

      • What is Inventory Control Planning Management Optimization DictionaryWhat is Inventory Planning? A Brief Dictionary of Inventory-Related Terms
        People involved in the supply chain are likely to have questions about various inventory terms and methods used in their jobs. This note may help by explaining these terms and showing how they relate. […]
      • artificial intelligence ai and machine learning inventory managementConfused about AI and Machine Learning?
        Are you confused about what is AI and what is machine learning? Are you unsure why knowing more will help you with your job in inventory planning? Don’t despair. You’ll be ok, and we’ll show you how some of whatever-it-is can be useful. […]
      • Centering Act Spare Parts Timing Pricing and ReliabilityCentering Act: Spare Parts Timing, Pricing, and Reliability
        In this article, we'll walk you through the process of crafting a spare parts inventory plan that prioritizes availability metrics such as service levels and fill rates while ensuring cost efficiency. We'll focus on an approach to inventory planning called Service Level-Driven Inventory Optimization. Next, we'll discuss how to determine what parts you should include in your inventory and those that might not be necessary. Lastly, we'll explore ways to enhance your service-level-driven inventory plan consistently. […]
      • Balance,Concept,With,Chrome,Balls,inventory optimization softwareHow to Forecast Inventory Requirements
        Forecasting inventory requirements is a specialized variant of forecasting that focuses on the high end of the range of possible future demand. Traditional methods often rely on bell-shaped demand curves, but this isn't always accurate. In this article, we delve into the complexities of this practice, especially when dealing with intermittent demand. […]
      • Demand Planning twin brothers holding forecasting toolsSix Demand Planning Best Practices You Should Think Twice About
        Every field, including forecasting, accumulates folk wisdom that eventually starts masquerading as “best practices.” These best practices are often wise, at least in part, but they often lack context and may not be appropriate for certain customers, industries, or business situations. There is often a catch, a “Yes, but”. This note is about six usually true forecasting precepts that nevertheless do have their caveats. […]

        Inventory Optimization for Manufacturers, Distributors, and MRO

        • Centering Act Spare Parts Timing Pricing and ReliabilityCentering Act: Spare Parts Timing, Pricing, and Reliability
          In this article, we'll walk you through the process of crafting a spare parts inventory plan that prioritizes availability metrics such as service levels and fill rates while ensuring cost efficiency. We'll focus on an approach to inventory planning called Service Level-Driven Inventory Optimization. Next, we'll discuss how to determine what parts you should include in your inventory and those that might not be necessary. Lastly, we'll explore ways to enhance your service-level-driven inventory plan consistently. […]
        • 5 Steps to Improve the Financial Impact of Spare Parts Planning5 Steps to Improve the Financial Impact of Spare Parts Planning
          In today’s competitive business landscape, companies are constantly seeking ways to improve their operational efficiency and drive increased revenue. Optimizing service parts management is an often-overlooked aspect that can have a significant financial impact. Companies can improve overall efficiency and generate significant financial returns by effectively managing spare parts inventory. This article will explore the economic implications of optimized service parts management and how investing in Inventory Optimization and Demand Planning Software can provide a competitive advantage. […]
        • Bottom Line strategies for Spare Parts Planning SoftwareBottom Line Strategies for Spare Parts Planning
          Managing spare parts presents numerous challenges, such as unexpected breakdowns, changing schedules, and inconsistent demand patterns. Traditional forecasting methods and manual approaches are ineffective in dealing with these complexities. To overcome these challenges, this blog outlines key strategies that prioritize service levels, utilize probabilistic methods to calculate reorder points, regularly adjust stocking policies, and implement a dedicated planning process to avoid excessive inventory. Explore these strategies to optimize spare parts inventory and improve operational efficiency. […]
        • professional technician engineer planning spare parts in industrial manufacturing factory,Prepare your spare parts planning for unexpected shocks
          In today's unpredictable business climate, we do have to worry about supply chain disruptions, long lead times, rising interest rates, and volatile demand. With all these challenges, it's never been more vital for organizations to accurately forecast parts usage, stocking levels, and to optimize replenishment policies such as reorder points, safety stocks, and order quantities. In this blog, we'll explore how companies can leverage innovative solutions like inventory optimization and parts forecasting software that utilize machine learning algorithms, probabilistic forecasting, and analytics to stay ahead of the curve and protect their supply chains from unexpected shocks. […]