Protect your Demand Planning Process from Regime Change

The Smart Forecaster

Pursuing best practices in demand planning, forecasting and inventory optimization

No, not that kind of regime change: Nothing here about cruise missiles and stealth bombers. And no, we’re not talking about the other kind of regime change that hits closer to home: Shuffling the C-Suite at your company.

“Regime change” has a third meaning that is relevant to your profession as a demand planner or inventory manager. To researchers in economics and finance, regime change means sudden shifts in the very character of a time series of random observations. The random time series in question here is the sequence of daily (or weekly or monthly) demand counts for your products and inventory items.

Most forecasting software uses statistical algorithms to process historical demand. It may add additional steps, such as incorporating field intelligence from sales people, but everything starts with the demand history of whatever item you must manage.

The question raised by regime change is, which data do you use? The simple answer is “All of it”, because that leads to the most accurate forecasts — but only if your data world is stable. If your data world is turbulent, then using all the data means you are basing forecasts on bye-gone conditions. In turn, inputting obsolete data into your forecasting algorithms inevitably leads to reduced forecast accuracy.

Note that dealing with regime change is not the same as dealing with outliers. Outliers are usually one-off exceptions caused by transient events, such as a kink in your supply chain caused by a huge blizzard choking off all transit paths. In contrast, regime change persists over a longer period and is therefore capable of doing more damage to your forecasts. Here’s an analogy: Outliers are about weather, and regime change is about climate.

The most drastic forms of regime change are existential. Figure 1 shows an example of an existential change: There was no demand at all for a long time, then suddenly there was demand. If you had no demand for an item because it didn’t exist but you retain zero demand values in your database, and then the item goes live and you do have sales, the transition from nothing to something is an extreme regime change. Including all those zero demand values from before “Day One” is sure to bias statistical forecasts down below where they should be. The same thing happens if you kill off a product but keep recording zero demand: Including all those recent zeros degrades your demand forecasts.

In principle, careful record keeping should eliminate these problems. You should record only meaningful zero values. If you have a new item, start recording when it goes live. If you no longer have any demand for an item and expect none, purge it from your database, or at least forecast zero demand.

Unfortunately, there is a difference between principle and practice. We see many instances in which the data records for both new and dormant items are not properly kept, with “fake zeros” confounded with “real zeros”. This problem is not necessarily the result of incompetence: Usually, it is a byproduct of the scale of the problem, with too few people trying to keep track of too many items.

These existential regime changes are relatively easy to deal with compared to more subtle forms, which appear to afflict more items. Figure 2 shows two examples of regime changes in a pattern of ongoing sales. There are any number of factors that can change the demand for an item: salesforce performance, marketing and advertising efforts, competitor and supplier actions, new customers arising or old customers disappearing, etc. If demand for an item has been chugging along at a steady 1 unit per day but suddenly doubles (or vice versa), that’s a regime change. In the new world order, demand is 2 units/day and forecasts should reflect that. Instead, statistical forecasting algorithms will forecast too little demand if fed all the data, including that from before the regime change.

How do you protect yourself from regime change? The answer is the same for the cruelest dictator or the most innocent demand planner: Intelligence. And because threats are many, the intelligence is best automated. Modern software systems have the capability to screen tens of thousands of items for signs of regime change. Then the software can call your attention to the problematic items and prompt you to designate which recent data to use in calculations. Or the software can automatically detect and correct for regime change, working quickly at a scale that would easily defeat any busy person working “by hand”.

 

Leave a Comment

Related Posts

Protect your Demand Planning Process from Regime Change

Protect your Demand Planning Process from Regime Change

No, not that kind of regime change: Nothing here about cruise missiles and stealth bombers. And no, we’re not talking about the other kind of regime change that hits closer to home: Shuffling the C-Suite at your company. In this blog, we discuss the relevance of regime change on time series data used for demand planning and forecasting.

Don’t Become a Victim of Your Forecast Models

Don’t Become a Victim of Your Forecast Models

Generally, the supply chain field has lagged behind finance in terms of the use of statistical models. My university colleagues and I are chipping away at that, but we have a long way to go. Some supply chains are quite technically sophisticated, but many, perhaps more, are essentially managed as much by gut instinct as by the numbers. Is this avoidance of analytics safer than relying on models?

Reveal Your Real Inventory Planning and Forecasting Policy by Answering These 9 Questions

Reveal Your Real Inventory Planning and Forecasting Policy by Answering These 9 Questions

In this blog, we review 9 specific questions you can ask to uncover what’s really happening with the inventory planning and demand forecasting policy at your company. We detail the typical answers provided when a forecasting/inventory planning policy doesn’t really exist, explain how to interpret these answers, and offer some clear advice on what to do about it.

Don’t Become a Victim of Your Forecast Models

The Smart Forecaster

Pursuing best practices in demand planning, forecasting and inventory optimization

Quants and Financial Meltdowns

I spend much of my time developing new quantitative methods for statistical forecasting, demand forecasting and inventory optimization. For me, this is an engaging way to contribute to society. But I know that the most prudent way to do algorithm development is to stand a little to the side and cast a skeptical eye on my own work.

The need for this skepticism was highlighted for me recently as I read Scott Patterson’s book The Quants: How a New Breed of Math Whizzes Conquered Wall Street and Nearly Destroyed It (Crown Publishing, 2010). This book reviewed the “quants” whose complex financial models were largely responsible for the financial meltdown in 2007. As I read along and thought “What was wrong with these guys?” I began to wonder if we supply chain quants were guilty of some of the same sins.

Models versus Instincts

Generally, the supply chain field has lagged behind finance in terms of the use of statistical models. My university colleagues and I are chipping away at that, but we have a long way to go. Some supply chains are quite technically sophisticated, but many, perhaps more, are essentially managed as much by gut instinct as by the numbers. Is this avoidance of analytics safer than relying on models?

What makes gut instinct dangerous is that it is so amorphous. Everyone who works long in a job develops instincts, but longevity is not the same as wisdom. It is possible to learn all the wrong lessons over a long career. It is also possible to miss the chance to learn the right lessons because certain informative scenarios may never arise in one person’s career. It is also possible to have good days and bad days; even gurus can mess up. Gut instinct is also anti-productive, since all decisions have to pass through that one gut, which becomes an enterprise chokepoint. And Golden Guts eventually reach their Golden Years and take their Golden Watch and go off into a Golden Sunset; at that point, whatever expertise had been present has walked out the door.

In contrast, models have certain advantages. Relative to gut instinct, models are:

  • Explicit: The theory of the supply chain operation is exposed for all to see.
  • Adaptive: Because the theory is visible, it can be reviewed, critiqued, tested against data, and evolved.
  • Consistent: Models may be more or less true, but they are not subject to day-to-day variability.
  • Comprehensive: At least potentially, models can accumulate a wide range of empirical experience, including scenarios never encountered during any one person’s career.
  • Instructive: Models are collections of relationships among variables. If the model’s “guts” are made visible, users can learn about those relationships.

Model Error

Nevertheless, despite all their virtues, models can also be wrong. In fact, that is a given. A constructive way to live with this is encoded in the famous aphorism by Dr. George Box, one of the best modelers of the last half century: “All models are wrong. Some are useful.”

The finance quants’ models were wrong by being oversimplified. They started with a quasi-religious belief in the efficiency of markets and developed statistical models that made certain assumptions that were more likely to be true of the physical world than the financial world. Among these were Normal distributions of changes in asset prices and independence of events across various corners of the market. They also assumed human rationality.

It should be a bit alarming that the Normal distribution and independence assumptions also underlie many of the models in supply chain software. In fact, there are alternative models of supply chain dynamics that do not require these simplifying assumptions, so this is an unnecessary risk being run by many, perhaps most, of the users of supply chain software.

But even with more robust and realistic model assumptions, there is no denying that model error is a constant risk. So, can you be victimized by your models? Of course you can.

Self-Protection: Look at the Data

Every supply chain professional who uses models, then, is subject to model risk. But unlike with decisions based on gut feel, decisions based on model calculations can be exposed and compared to real-world outcomes. Repeated checking is the best way to protect against model error, because it not only tests whether the model is realistic but also signals when it is time to update the model.

As noted above, a model is a set of functional relationships between key variables. Those relationships have parameters that tune the model to the current operating context. For instance, supply chain models often rely, in part, on estimates of demand volatility. Historical demand data are used to calculate numerical values for these parameters. If demand volatility changes, the model becomes obsolete and likely to produce inapt recommendations. Therefore, good practice demands frequent updates to model parameters.

Even when parameter values are current, there may still be trouble due to incorrect functional relationships. For example, consider the relationship between the mean and standard deviation of demand for spare parts. Generally speaking, the greater the average demand, the greater the demand volatility as measured by the standard deviation.

Now consider simplified “old school” models that describe spare part demand as a Poisson process. The Poisson process is widely useful and relatively simple, so it often shows up in Statistics 101 classes. Because of their relative simplicity, Poisson models are the white rats of supply chain analytics for spare parts, i.e., people do computer experiments and theory development based on the behavior of Poisson models of demand. For Poisson models, the standard deviation of demand equals the square root of the mean. However, when we look at our customers’ actual demand data, we discover that the actual relationship between the mean and standard deviation of demand is better described by a more general power-law relationship. Thus, the simple model may use accurate estimates of mean and standard deviation but still not accurately reflect their relationship. This in turn leads to incorrect recommendations about reorder points for spare parts. Checking real data is the best antidote to cavalier assumption-making.

What to Do Next

I do not sense that today’s supply chain models are on the brink of creating the kind of meltdown we saw in the start of the Great Recession. But those of us who are supply chain quants need to show more professional maturity than our financial colleagues. We need to not fall in love with our models, and we need to alert our customers to correct model hygiene.

So, model users, wash your hands frequently as we begin flu season, and wash your models thoroughly through hard data to be sure that the models you rely on are both up-to-date and grounded in reality. Both those steps will protect you from being victimized by your models and let you exploit their advantages over management by gut feel.

Appendix: Technical Tips

Supply chain analytics provide various types of outputs. In the realm of forecasting and demand planning, the obvious empirical check is to compare forecasts against the actual demand values that eventually reveal themselves. This same “forecast then check” approach can also be used in the generation of forecasts.  In the realm of inventory management, the models can build on forecasts to recommend policy choices, such as reorder points and order quantities or Min and Max values. There is a smart way to confirm the accuracy of recommendations of reorder points and Min’s.  See our blog The Right Forecast Accuracy Metric for Inventory Planning

 

Leave a Comment

Related Posts

Protect your Demand Planning Process from Regime Change

Protect your Demand Planning Process from Regime Change

No, not that kind of regime change: Nothing here about cruise missiles and stealth bombers. And no, we’re not talking about the other kind of regime change that hits closer to home: Shuffling the C-Suite at your company. In this blog, we discuss the relevance of regime change on time series data used for demand planning and forecasting.

Don’t Become a Victim of Your Forecast Models

Don’t Become a Victim of Your Forecast Models

Generally, the supply chain field has lagged behind finance in terms of the use of statistical models. My university colleagues and I are chipping away at that, but we have a long way to go. Some supply chains are quite technically sophisticated, but many, perhaps more, are essentially managed as much by gut instinct as by the numbers. Is this avoidance of analytics safer than relying on models?

Reveal Your Real Inventory Planning and Forecasting Policy by Answering These 9 Questions

Reveal Your Real Inventory Planning and Forecasting Policy by Answering These 9 Questions

In this blog, we review 9 specific questions you can ask to uncover what’s really happening with the inventory planning and demand forecasting policy at your company. We detail the typical answers provided when a forecasting/inventory planning policy doesn’t really exist, explain how to interpret these answers, and offer some clear advice on what to do about it.

Reveal Your Real Inventory Planning and Forecasting Policy by Answering These 9 Questions

The Smart Forecaster

Pursuing best practices in demand planning, forecasting and inventory optimization

In our last blog we posed the question:  How can you be sure that you really have a policy for inventory planning and demand forecasting? We explained how an organization’s lack of understanding on the basics (how a forecast is created, how safety stock buffers are determined, and how/why these values are adjusted) contributes to poor forecast accuracy, misallocated inventory, and lack of trust in the whole process.

In this blog, we review 9 specific questions you can ask to uncover what’s really happening at your company. We detail the typical answers provided when a forecasting/inventory planning policy doesn’t really exist, explain how to interpret these answers, and offer some clear advice on what to do about it.

Always start with a simple hypothetical example. Focusing on a specific problem you just experienced is bound to provoke defensive answers that hide the full story. The goal is to uncover the actual approach used to plan inventory and forecasts that has been baked into the mental math or spreadsheets.   Here is an example:

Suppose you have 100 units on hand, the lead time to replenish is 3 months, and the average monthly demand is 20 units?   When should you order more?  How much would you order? How will your answer change if expected receipts of 10 per month were scheduled to arrive?  How will your answer change if the item is the item is an A, B, or C item, the cost of the item is high or low, lead time of the item is long or short?  Simply put, when you schedule a production job or place a new order with a supplier, why did you do it? What triggered the decision to get more?  What planning inputs were considered?

When getting answers to the above question, focus on uncovering answers to the following questions:

1. What is the underlying replenishment approach? This will typically be one of Min/Max, forecast/safety stock, Reorder Point/Order Quantity, Periodic Review/Order Up To or even some odd combination

2. How are the planning parameters, such as demand forecasts, reorder points, or Min/Max, actually calculated? It’s not enough to know that you use Min/Max.  You have to know exactly how these values are calculated. Answers such as “We use history” or “We use an average” are not specific enough.   You’ll need answers that clearly outline how history is used.  For example, “We take an average of the last 6 months, divide that by 30 to get a daily average, and then multiply that by the lead time in days.  For ‘A’ items we then multiply the lead time average by 2 and for ‘B’ items we use a multiplier of 1.5.” (While that is not an especially good technical approach, at least it has a clear logic.)

Once you have a policy well-defined, you can identify its weaknesses in order to improve it.  But if the answer provided doesn’t get much further past “We use history”, then you don’t have a policy to start with.   Answers will often reveal that different planners use history in different ways.  Some may only consider the most recent demand, others might stock according to the average of the highest demand periods, etc.  In other words, you may find that you actually have multiple ill-conceived “policies”.

3. Are forecasts used to drive replenishment planning and if so, how? Many companies will say they forecast, but their forecasts are calculated and used differently. Is the forecast used to predict what on hand inventory will be in the future, resulting in an order being triggered?  Or is it used to derive a reorder point but not to predict when to order (i.e. I predict we’ll sell 10 a week so to help protect against stock out, I’ll order more when on hand gets to 15)? Is it used as a guide for the planner to help subjectively determine when they should order more?  Is it used to set up blanket orders with suppliers?  Some use it to drive MRP. You’ll need to know these specifics.  A thorough answer to this question might look like this: “My forecast is 10 per week and my lead time is 3 weeks so I make my reorder point a multiple of that forecast, typically 2 x lead time demand or 60 unit for important items and I use a smaller multiple for less important items.  (Again, not a great technical approach, but clear.)

4.  What technique is actually used to generate the forecast? Is it an average, a trending model such as double exponential smoothing, a seasonal model? Does the choice of technique change depend on the type of demand data or when new demand data is available? (Spare parts and high-volume items have very different demand patterns.) How do you go about selecting the forecast model? Is this process automated?  How often is the choice of model reconsidered?  How often are the model parameters recomputed? What is the process used to reconsider your approach?  The answer here documents how the baseline forecasts are produced.  Once determined, you can conduct an analysis to identify whether other forecasting methods would improve forecast accuracy.  If you aren’t documenting forecast accuracy and conducting “forecast value add” analysis then you aren’t in a position to properly assess whether the forecasts being produced are the best that they can be.  You’ll miss out on opportunities to improve the process, increase forecast accuracy, and educate the business on what type of forecast error is normal and should be expected.

5. How do you use safety stock? Notice the question was not “Do you use safety stock?” In this context, and to keep it simple, the term “safety stock” means stock used to buffer inventory against supply and demand variability.  All companies use buffering approaches in some way.  There are some exceptions though.  Maybe you are a job shop manufacturer that procures all parts to order and your customers are completely fine waiting weeks or months for you to source material, manufacture, QA, and ship.  Or maybe you are high-volume manufacturer with tons of buying power so your suppliers set up local warehouses that are stocked full and ready to provide inventory to you almost immediately.  If these descriptions don’t describe your company, you will definitely have some sort of buffer to protect against demand and supply variability.  You may not use the “safety stock” field in your ERP but you are definitely buffering.

Answers might be provided such as “We don’t use safety stock because we forecast.”  Unfortunately, a good forecast will have a 50/50 chance of being over/under the actual demand.  This means you’ll incur a stock out 50% of the time without a safety stock buffer added to the forecast.  Forecasts are only perfect when there is no randomness. Since there is always randomness, you’ll need to buffer if you don’t want to have abysmal service levels.

If the answer isn’t revealed, you can probe a bit more into how the varying replenishment levers are used to add possible buffers which leads to questions 6 & 7.

6. Do you ever increase the lead time or order earlier than you truly need to?
In our hypothetical example, your supplier typically takes 4 weeks to deliver and is pretty consistent. But to protect against stockouts your buyer routinely orders 6 weeks out instead of 4 weeks.  The safety stock field in your ERP system might be set to zero because “we don’t use safety stock”, but in reality, the buyer’s ordering approach just added 2 weeks of buffer stock.

7. Do you pad the demand forecast?
In our example, the planner expects to consume 10 units per month but “just in case” enters a forecast of 20 per month.  The safety stock field in the MRP system is left blank but the now disguised buffer stock has been smuggled into the demand forecast.  This is a mistake that introduces “forecast bias.”  Not only will your forecasts be less accurate but if the bias isn’t accounted for and safety stock is added by other departments, you will overstock.

The ad-hoc nature of the above approaches compounds the problems by not considering the actual demand or supply variability of the item. For example, the planner might simply make a rule of thumb that doubles the lead time forecast for important items.  One-size doesn’t fit all when it comes to inventory management.  This approach will substantially overstock the predictable items while substantially understocking the intermittently demanded items. You can read “Beware of Simple Rules of Thumb for Managing Inventory” to learn more about why this type of approach is so costly.

The ad-hoc nature of the approaches also ignores what happens the company is faced with a huge overstock or stock out. When trying to understand what happened, the stated policies will be examined. In the case of an overstock, the system will show zero safety stock.  The business leaders will assume they aren’t carrying any safety stock, scratch their heads, and eventually just blame the forecast, declare “Our business can’t be forecasted” and stumble on. They may even blame the supplier for shipping too early and making them hold more than needed. In the case of a stock out, they will think they aren’t carrying enough and arbitrarily add more stock across many items not realizing there is in fact lots of extra safety stock baked into process.  This makes it more likely inventory will need to be written off in the future.

8. What is the exact inventory terminology used? Define what you mean by safety stock, Min, reorder point, EOQ, etc.  While there are standard technical definitions it’s possible that something differs, and miscommunication here will be problematic.  For example, some companies refer to Min as the amount of inventory needed to satisfy lead time demand while some may define Min as inclusive of both lead time demand and safety stock to buffer against demand variability. Others may mean the minimum order quantity.

9. Is on hand inventory consistent with the policy? When your detective work is done and everything is documented, open your spreadsheet or ERP system and look at the on-hand quantity. It should be more or less in line with your planning parameters (i.e. if Min/Max is 20/40 and typical lead time demand is 10, then you should have roughly 10 to 40 units on hand at any given point in time.  Surprisingly, for many companies there is often a huge inconsistency. We have observed situations where the Min/Max setting is 20/40 but the on-hand inventory is 300+.  This indicates that whatever policy has been prescribed just isn’t being followed.   That’s a bigger problem.

Going Forward

Demand forecasting and inventory stocking policy need to be well-defined processes that are understood and accepted by everybody involved.  There should be zero mystery.

To do this right, the demand and supply variability must be analyzed and used to compute the proper levels of safety stock.   Adding buffers without an implicit understanding of what each additional unit of buffer stock is buying you in terms of service is like arbitrarily throwing a handful of ingredients into a cake recipe.  A small change in ingredients can have a huge impact on what comes out of the oven – one bite too sweet but the next too sour.  It is the same with inventory management.  A little extra here, a little less there, and pretty soon you find yourself with costly excess inventory in some areas, painful shortages in others, no idea how you got there, and with little guidance on how to make things better.

Modern inventory optimization and demand planning software with its advanced analytics and strong basis in forecast analysis can help a good deal with this problem. But even the best software won’t help if it is used inconsistently.

Leave a Comment

Related Posts

Protect your Demand Planning Process from Regime Change

Protect your Demand Planning Process from Regime Change

No, not that kind of regime change: Nothing here about cruise missiles and stealth bombers. And no, we’re not talking about the other kind of regime change that hits closer to home: Shuffling the C-Suite at your company. In this blog, we discuss the relevance of regime change on time series data used for demand planning and forecasting.

Don’t Become a Victim of Your Forecast Models

Don’t Become a Victim of Your Forecast Models

Generally, the supply chain field has lagged behind finance in terms of the use of statistical models. My university colleagues and I are chipping away at that, but we have a long way to go. Some supply chains are quite technically sophisticated, but many, perhaps more, are essentially managed as much by gut instinct as by the numbers. Is this avoidance of analytics safer than relying on models?

Reveal Your Real Inventory Planning and Forecasting Policy by Answering These 9 Questions

Reveal Your Real Inventory Planning and Forecasting Policy by Answering These 9 Questions

In this blog, we review 9 specific questions you can ask to uncover what’s really happening with the inventory planning and demand forecasting policy at your company. We detail the typical answers provided when a forecasting/inventory planning policy doesn’t really exist, explain how to interpret these answers, and offer some clear advice on what to do about it.

Undershoot is Sabotaging your Service Level!

The Smart Forecaster

Pursuing best practices in demand planning, forecasting and inventory optimization

Service level is a key performance indicator for companies that put a premium on satisfying customer demand. Service level is defined as the probability of surviving a replenishment lead time without stocking out.

Inventory management best practice begins with setting service level targets, then calculates reorder points (also called Mins) to achieve those targets. These calculations should account for variability in both demand and replenishment lead time. There are many software systems available for doing these calculations. If everything works out, the achieved service level ends up very close to the target service level. Unfortunately, there is often a painful gap between the two.

One reason for the gap is unrealistic models of demand. In many cases, software for calculating reorder points uses textbook formulas based on mathematical assumptions that make analysis simple at the expense of realism.  Many “Inventory 101” textbooks use formulas that assume demand has a Normal distribution (a.k.a. the “bell-shaped curve”) for finished goods and the Poisson distribution for spare parts. Fortunately, there are now inventory optimization and forecasting systems that process the actual demand histories of the inventory items using probabilistic forecasting.  These solutions calculate an accurate estimate of the distribution – not some idealized version.  To learn more check out this past blog on probabilistic forecasting:

But there is a second source of error in textbooks that operates invisibly in many inventory software package:  “undershoot”.

Calculations of reorder points almost always assume that stockouts arise when the total demand during a replenishment interval exceeds the reorder point. For example, assume that demand averages 1 unit per day. If lead time is 5 days, then on average lead time demand is 5 units. Setting the reorder point at 5 units would yield a laughable service level somewhere in the vicinity of 50%. Adding safety stock to the calculation might result in a reorder point of, say, 11 units, which might correspond to a service level of 95%. Another way to say this is, starting at a reorder point of 11 units, there should be a 95% chance of surviving the 5 day lead time without experiencing cumulative demand of more than 11 units. Theoretically!

What’s missing from this analysis is the undershoot phenomenon. Undershoot means that the lead time begins not at the reorder point but below it. Undershoot happens every time the demand that breached the reorder point took the stock down below (not down to) the reorder point. The figure below shows replenishment cycles with and without undershoot.  Undershoot picks your pocket before you even begin to roll the dice. It deludes the inventory professional into thinking his or her reorder points are sufficient to achieve their targets, whereas actual performance will not make the grade.

There is only one situation in which undershoot is not a worry: when demand is always either zero or one unit. In that case, undershoot is impossible. But in all other cases, undershoot is sure to happen to some extent, and it can seriously undercut the service level actually achieved by a given choice of reorder point. Our analyses show that the conditions most vulnerable to undershoot involve highly intermittent and skewed demand with very short lead times – the very conditions being made most common by market trends.

What can be done to protect yourself from the effect of undershoot on reorder point calculations?  Use inventory optimization and forecasting software that isn’t tied to the old textbook assumptions and instead automatically accounts for undershoot when calculating the service level produced by any choice of reorder point.

To see Smart Software’s Inventory Optimization solution in action, register to see a recorded demo below:

 

Your Name *

Company Name *

Work Email *

Work Phone


 

 

Leave a Comment

Related Posts

Protect your Demand Planning Process from Regime Change

Protect your Demand Planning Process from Regime Change

No, not that kind of regime change: Nothing here about cruise missiles and stealth bombers. And no, we’re not talking about the other kind of regime change that hits closer to home: Shuffling the C-Suite at your company. In this blog, we discuss the relevance of regime change on time series data used for demand planning and forecasting.

Don’t Become a Victim of Your Forecast Models

Don’t Become a Victim of Your Forecast Models

Generally, the supply chain field has lagged behind finance in terms of the use of statistical models. My university colleagues and I are chipping away at that, but we have a long way to go. Some supply chains are quite technically sophisticated, but many, perhaps more, are essentially managed as much by gut instinct as by the numbers. Is this avoidance of analytics safer than relying on models?

Reveal Your Real Inventory Planning and Forecasting Policy by Answering These 9 Questions

Reveal Your Real Inventory Planning and Forecasting Policy by Answering These 9 Questions

In this blog, we review 9 specific questions you can ask to uncover what’s really happening with the inventory planning and demand forecasting policy at your company. We detail the typical answers provided when a forecasting/inventory planning policy doesn’t really exist, explain how to interpret these answers, and offer some clear advice on what to do about it.

How to Choose a Target Service Level

The Smart Forecaster

Pursuing best practices in demand planning, forecasting and inventory optimization

Summary

Setting a target service level or fill rate is a strategic decision about inventory risk management. Choosing service levels can be difficult. Relevant factors include current service levels, replenishment lead times, cost constraints, the pain inflicted by shortages on you and your customers, and your competitive position. Target setting is often best approached as a collaboration among operations, sales and finance. Inventory optimization software is an essential tool in the process.

Service Level Choices

Service level is the probability that no shortages occur between when you order more stock and when it arrives on the shelf. The reasonable range of service levels is from about 70% to 99%. Levels below 70% may signal that you don’t care about or can’t handle your customers. Levels of 100% are almost never appropriate and usually indicate a hugely bloated inventory.

Factors Influencing Choice of Service Level

Several factors influence the choice of service level for an inventory item. Here are some of the more important.

Current service levels:
A reasonable place to start is to find out what your current service levels are for each item and overall. If you are already in good shape, then the job becomes the easier one of tweaking an already-good solution. If you are in bad shape now, then setting service levels can be more difficult. Surprisingly few companies have data on this important metric across their whole fleet of inventory items. What often happens is that reorder points grow willy-nilly from choices made in corporate pre-history and are rarely, sometimes never, systematically reviewed and updated. Since reorder points are a major determinant of service levels, it follows that service levels “just happen”. Inventory optimization software can convert your current reorder points and lead times into solid estimates of your current service levels. This analysis often reveals subset of items with service levels either too high or too low, in which case you have guidance about which items to adjust down or up, respectively.

Replenishment lead times:
Some companies adjust service levels to match replenishment lead times. If it takes a long time to make or buy an item, then it takes a long time to recover from a shortage. Accordingly, they bump up service levels on long-lead-time items and reduce them on items for which backlogs will be brief.

Cost constraints:
Inventory optimization software can find the lowest-cost ways to hit high service level targets, but aggressive targets inevitably imply higher costs. You may find that costs constrain your choice of service level targets. Costs come in various flavors. “Inventory investment” is the dollar value of inventory. “Operating costs” include both holding costs and ordering costs. Constraints on inventory investment are often imposed on inventory executives and always imply ceilings on service level targets; software can make these relationships explicit but not take away the necessity of choice. It is less common to hear of ceilings on operating costs, but they are always at least a secondary factor arguing for lower service levels.

Shortage costs:
Shortage costs depend on whether your shortage policy calls for backorders or lost sales. In either case, shortage costs work counter to inventory investment and operating costs by arguing for higher service levels. These costs may not always be expressed in dollar terms, as in the case of medical/surgical supplies, where shortage costs are denominated in morbidity and mortality.

Competition:
The closer your company is to dominating its market, the more you can ease back on service levels to save money. However, easing back too far carries risks: It encourages potential customers to look elsewhere, and it encourages competitors. Conversely, high product availability can go far to bolstering the position of a minor player.

Collaborative Targeting

Inventory executives may be the ones tasked with setting service level targets, but it may be best to collaborate with other functions when making these calls. Finance can share any “red lines” early in the process, and they should be tasked with estimating holding and ordering costs. Sales can help with estimating shortage costs by explaining likely customer reactions to backlogs or lost sales.

The Role of Inventory Optimization and Planning Software

Without inventory optimization software, setting service level targets is pure guesswork: It is impossible to know how any given target will play out in terms of inventory investment, operating costs, shortage costs. The software can compute the detailed, quantitative tradeoff curves required to make informed choices or even recommend the target service level that results in the lowest overall cost considering holding costs, ordering costs, and stock out costs. However, not all software solutions are created equal. You might enter a user defined 99% service level into your inventory planning system or the system could recommend a target service – but it doesn’t mean you will actually hit that stated service level. In fact, you might not even come close to hitting it and achieve a much lower service level. We’ve observed situations where a targeted service level of 99% actually achieved a service level of just 82%! Any decisions made as a result of the target will result in unintended misallocation of inventory, very costly consequences, and lots of explaining to do. So be sure to check out our next blog article on how to measure the accuracy of your service level forecast so you don’t make this costly mistake.

Leave a Comment

Related Posts

Protect your Demand Planning Process from Regime Change

Protect your Demand Planning Process from Regime Change

No, not that kind of regime change: Nothing here about cruise missiles and stealth bombers. And no, we’re not talking about the other kind of regime change that hits closer to home: Shuffling the C-Suite at your company. In this blog, we discuss the relevance of regime change on time series data used for demand planning and forecasting.

Don’t Become a Victim of Your Forecast Models

Don’t Become a Victim of Your Forecast Models

Generally, the supply chain field has lagged behind finance in terms of the use of statistical models. My university colleagues and I are chipping away at that, but we have a long way to go. Some supply chains are quite technically sophisticated, but many, perhaps more, are essentially managed as much by gut instinct as by the numbers. Is this avoidance of analytics safer than relying on models?

Reveal Your Real Inventory Planning and Forecasting Policy by Answering These 9 Questions

Reveal Your Real Inventory Planning and Forecasting Policy by Answering These 9 Questions

In this blog, we review 9 specific questions you can ask to uncover what’s really happening with the inventory planning and demand forecasting policy at your company. We detail the typical answers provided when a forecasting/inventory planning policy doesn’t really exist, explain how to interpret these answers, and offer some clear advice on what to do about it.

5 Demand Planning Tips for Calculating Forecast Uncertainty

The Smart Forecaster

Pursuing best practices in demand planning, forecasting and inventory optimization

Those who produce forecasts owe it to those who consume forecasts, and to themselves, to be aware of the uncertainty in their forecasts. This note is about how to estimate forecast uncertainty and use the estimates in your demand planning process. We focus on forecasts made in support of demand planning as well as forecasts inherent in optimizing inventory policies involving reorder points, safety stocks, and min/max levels.

Reading this, you will learn about:

-Criteria for assessing forecasts
-Sources of forecast error
-Calculating forecast error
-Converting forecast error into prediction intervals
-The relationship between demand forecasting and inventory optimization.
-Actions you can take to use these concepts to improve your company’s processes.

Criteria for Assessing Forecasts

Forecast error alone is not reason enough to reject forecasting as a management tool. To twist a famous aphorism by George Box, “All forecasts are wrong, but some are useful.” Of course, business professionals will always search for ways to make forecasts more useful. This usually involves work to reduce forecast error. But while forecast accuracy is the most obvious criterion by which to judge forecasts, but it is not the only one. Here’s a list of criteria for evaluating forecasts:

Accuracy: Forecasts of future values should, in retrospect, be very close to the actual values that eventually reveal themselves. But there may be diminishing returns to squeezing another half percent of accuracy out of forecasts otherwise good enough to use in decision making.

Timeliness: Fighter pilots refer to the OODA Loop (Observe, Orient, Decide, and Act) and the “need to get inside the enemy’s OODA loop” so they can shoot first. Businesses too have decision cycles. Delivering a perfectly accurate forecast the day after it was needed is not helpful. Better is a good forecast that arrives in time to be useful.

Cost: Forecasting data, models, processes and people all cost money.  A less expensive forecast might be fueled by data that are readily available; more expensive would be a forecast that runs on data that have to be collected in a special process outside the scope of a firm’s information infrastructure.  A classic, off-the-shelf forecasting technique will be less costly to acquire, feed and exploit than a complex, custom, consultant-supplied method. Forecasts could be mass-produced by software overseen by a single analyst, or they might emerge from a collaborative process requiring time and effort from large groups of people, such as district sales managers, production teams, and others. Technically advanced forecasting techniques often require hiring staff with specialized technical expertise, such as a master’s degree in statistics, who tend to cost more than staff with less advanced training.

Credibility: Ultimately, some executive has to accept and act on each forecast. Executives have a tendency to distrust or ignore recommendations that they can neither understand nor explain to the next person above them in the hierarchy. For many, believing in a “black box” is too severe a test of faith, and they reject the black box’s forecasts in favor of something more transparent.

All that said, we will focus now on forecast accuracy and its evil twin, forecast error.

Sources of Forecast Error

Those seeking to reduce error can look in three places to find trouble:
1. The data that goes into a forecasting model
2. The model itself
3. The context of the forecasting exercise

There are several ways in which data problems can lead to forecast error.

Gross errors: Wrong data produce wrong forecasts. We have seen an instance in which computer records of product demand were wrong by a factor of two! Those involved spotted that problem immediately, but a less egregious situation can easily slip through to poison the forecasting process. In fact, just organizing, acquiring and checking data is often the largest source of delay in the implementation of forecasting software. Many data problems seem to derive from the data having been unimportant until a forecasting project made them important.

Anomalies: Even with perfectly curated forecasting databases, there are often “needle in a haystack” type data problems. In these cases, it is not data errors but demand anomalies that contribute to forecast error. In a set of, say, 50,000 products, some number of items are likely to have odd details that can distort forecasts.

Holdout analysis is a simple but powerful method of analysis. To see how well a method forecasts, use it with older known data to forecast newer data, then see how it would have turned out! For instance, suppose you have 36 months of demand data and need to forecast 3 months ahead. You can simulate the forecasting process by holding out (i.e., hiding) the most recent 3 months of data, forecasting using only data from months 1 to 33, then comparing the forecasts for months 34-36 against the actual values in months 34-36. Sliding simulation merely repeats the holdout analysis, sliding along the demand history. The example above used the first 33 months of data to get 3 estimates of forecast error. Suppose we start the process by using the first 12 months to forecast the next 3. Then we slide forward and use the first 13 months to forecast the next 3. We continue until finally we use the first 35 months to forecast the last month, giving us one more estimate of the error we make when forecasting one month ahead. Summarizing all the 1-step ahead, 2-step ahead and 3-step ahead forecast errors provides a way to calculate prediction intervals.

Calculating Prediction Intervals

The final step in calculating prediction intervals is to convert the estimates of average absolute error into the upper and lower limits of the prediction interval. The prediction interval at any future time is computed as

Prediction interval = Forecast ± Multiplier x Average absolute error.

The final step is the choice of the multiplier. The typical approach is to imagine some probability distribution of error around the forecast, then estimate the ends of the prediction interval using appropriate percentiles of that distribution. Usually, the assumed distribution of error is the Normal distribution, also called the Gaussian distribution or the “bell-shaped curve”.

Use of Prediction Intervals
The most immediate, informal use of prediction intervals is to convey a sense of how “squishy” a forecast is. Prediction intervals that are wide compared to the size of the forecasts indicate high uncertainty.

There are two more formal uses in demand forecasting: Hedging your bets about future demand and guiding forecast adjustment.

Hedging your bets: The forecast values themselves approximate the most likely values of future demand. A more ominous way to say the same thing is that there is about a 50% chance that the actual value will be above (or below) the forecast. If the forecast is being used to plan future production (or raw materials purchase or hiring), you might want to build in a cushion to keep from being caught short if demand spikes (assuming that under-building is worse than over-building). If the forecast is converted from units to dollars for revenue projections, you might want to use a value below the forecast to be conservative in projecting cash flow. In either case, you first have to choose the coverage of the prediction interval. A 90% prediction interval is a range of values that covers 90% of the possibilities. This implies that there is a 5% chance of a value falling above the upper limit of the 90% prediction interval. In other words, the upper limit of a 90% prediction interval marks the 95th percentile of the distribution of predicted demand at that time period. Similarly, there is a 5% chance of falling below the lower limit, which marks the 5th percentile of the demand distribution.

Guiding forecast adjustment: It is quite common for statistical forecasts to be revised by some sort of collaborative process. These adjustments are based on information not recorded in an item’s demand history, such as intelligence about competitor actions. Sometimes they are based on a more vaporous source, such as sales force optimism. When the adjustments are made on-screen for all to see, the prediction intervals provide a useful reference: If someone wants to move the forecasts outside the prediction intervals, they are crossing a fact-based line and should have a good story to justify their argument that things will be really different in the future.

Prediction Intervals and Inventory Optimization

Finally, the concept behind prediction intervals play an essential role in a problem related to demand forecasting: Inventory Optimization.
The core analytic task in setting reorders points (also called Mins) is to forecast total demand over a replenishment lead time. This total is called the lead time demand. When on-hand inventory falls down to or below the reorder point, a replenishment order is triggered. If the reorder point is high enough, there will be an acceptably small risk of a stockout, i.e., of lead time demand driving inventory below zero and creating either lost sales or backorders.

The forecasting task is to determine all the possible values of cumulative demand over the lead time and their associated probabilities of occurring. In other words, the basic task is to determine a prediction interval for some future random variable. Suppose you have computed a 90% prediction interval for lead time demand. Then the upper end of the interval represents the 95th percentile of the distribution. Setting the reorder point at this level will accommodate 95% of the possible lead time demand values, meaning there will be only a 5% chance of stocking out before replenishment arrives to re-stock the shelves. Thus there is an intimate relationship between prediction intervals in demand forecasting and calculation of reorder points in inventory optimization.5

5 Recommendations for Practice

1. Set expectations about error: Sometimes managers have unreasonable expectations about reducing forecast error to zero. You can point out that error is only one of the dimensions on which a forecasting process must be judged; you may be doing fine on both timeliness and cost. Also point out that zero error is no more realistic a goal than 100% conversion of prospects into customers, perfect supplier performance, or zero stock price volatility.

2. Track down sources of error: Double check the accuracy of demand histories. Use statistical methods to identify outliers in demand histories and react appropriately, replacing verified anomalies with more typical values and omitting data from before major changes in the character of the demand. If you use a collaborative forecasting process, compare its accuracy against a purely statistical approach to identify items for which collaboration does not reduce error.

3. Evaluate the error of alternative statistical methods: There may be off-the-shelf techniques that do better than your current methods, or do better for some subsets of your items. The key is to be empirical, using the idea of holdout analysis. Gather your data and do a “bake off” between different methods to see which work better for you. If you are not already using statistical forecasting methods, compare them against whoever’s “golden gut” is your current standard. Use the naïve forecast as a benchmark in the comparisons.

4. Investigate the use of new data sources: Especially if you have items that are heavily promoted, test out statistical methods that incorporate promotional data into the forecasting process. Also check whether information from outside your company can be exploited; for instance, see whether macroeconomic indicators for your sector can be combined with company data to improve forecast accuracy (this is usually done using a method called multiple regression analysis).

5. Use prediction intervals: Plots of prediction intervals can improve your feel for the uncertainty in your forecasts, helping you select items for additional scrutiny. While it’s true that what you don’t know can hurt you, it’s also true that knowing what you don’t know can help you.

Leave a Comment

Related Posts

Protect your Demand Planning Process from Regime Change

Protect your Demand Planning Process from Regime Change

No, not that kind of regime change: Nothing here about cruise missiles and stealth bombers. And no, we’re not talking about the other kind of regime change that hits closer to home: Shuffling the C-Suite at your company. In this blog, we discuss the relevance of regime change on time series data used for demand planning and forecasting.

Don’t Become a Victim of Your Forecast Models

Don’t Become a Victim of Your Forecast Models

Generally, the supply chain field has lagged behind finance in terms of the use of statistical models. My university colleagues and I are chipping away at that, but we have a long way to go. Some supply chains are quite technically sophisticated, but many, perhaps more, are essentially managed as much by gut instinct as by the numbers. Is this avoidance of analytics safer than relying on models?

Reveal Your Real Inventory Planning and Forecasting Policy by Answering These 9 Questions

Reveal Your Real Inventory Planning and Forecasting Policy by Answering These 9 Questions

In this blog, we review 9 specific questions you can ask to uncover what’s really happening with the inventory planning and demand forecasting policy at your company. We detail the typical answers provided when a forecasting/inventory planning policy doesn’t really exist, explain how to interpret these answers, and offer some clear advice on what to do about it.