In another blog we posed the question:  How can you be sure that you really have a policy for inventory planning and demand forecasting? We explained how an organization’s lack of understanding on the basics (how a forecast is created, how safety stock buffers are determined, and how/why these values are adjusted) contributes to poor forecast accuracy, misallocated inventory, and lack of trust in the whole process.

In this blog, we review 10 specific questions you can ask to uncover what’s really happening at your company. We detail the typical answers provided when a forecasting/inventory planning policy doesn’t really exist, explain how to interpret these answers, and offer some clear advice on what to do about it.

Always start with a simple hypothetical example. Focusing on a specific problem you just experienced is bound to provoke defensive answers that hide the full story. The goal is to uncover the actual approach used to plan inventory and forecasts that has been baked into the mental math or spreadsheets.   Here is an example:

Suppose you have 100 units on hand, the lead time to replenish is 3 months, and the average monthly demand is 20 units?   When should you order more?  How much would you order? How will your answer change if expected receipts of 10 per month were scheduled to arrive?  How will your answer change if the item is the item is an A, B, or C item, the cost of the item is high or low, lead time of the item is long or short?  Simply put, when you schedule a production job or place a new order with a supplier, why did you do it? What triggered the decision to get more?  What planning inputs were considered?

When getting answers to the above question, focus on uncovering answers to the following questions:

1. What is the underlying replenishment approach? This will typically be one of Min/Max, forecast/safety stock, Reorder Point/Order Quantity, Periodic Review/Order Up To or even some odd combination

2. How are the planning parameters, such as demand forecasts, reorder points, or Min/Max, actually calculated? It’s not enough to know that you use Min/Max.  You have to know exactly how these values are calculated. Answers such as “We use history” or “We use an average” are not specific enough.   You’ll need answers that clearly outline how history is used.  For example, “We take an average of the last 6 months, divide that by 30 to get a daily average, and then multiply that by the lead time in days.  For ‘A’ items we then multiply the lead time average by 2 and for ‘B’ items we use a multiplier of 1.5.” (While that is not an especially good technical approach, at least it has a clear logic.)

Once you have a policy well-defined, you can identify its weaknesses in order to improve it.  But if the answer provided doesn’t get much further past “We use history”, then you don’t have a policy to start with.   Answers will often reveal that different planners use history in different ways.  Some may only consider the most recent demand, others might stock according to the average of the highest demand periods, etc.  In other words, you may find that you actually have multiple ill-conceived “policies”.

3. Are forecasts used to drive replenishment planning and if so, how? Many companies will say they forecast, but their forecasts are calculated and used differently. Is the forecast used to predict what on hand inventory will be in the future, resulting in an order being triggered?  Or is it used to derive a reorder point but not to predict when to order (i.e. I predict we’ll sell 10 a week so to help protect against stock out, I’ll order more when on hand gets to 15)? Is it used as a guide for the planner to help subjectively determine when they should order more?  Is it used to set up blanket orders with suppliers?  Some use it to drive MRP. You’ll need to know these specifics.  A thorough answer to this question might look like this: “My forecast is 10 per week and my lead time is 3 weeks so I make my reorder point a multiple of that forecast, typically 2 x lead time demand or 60 unit for important items and I use a smaller multiple for less important items.  (Again, not a great technical approach, but clear.)

4.  What technique is actually used to generate the forecast? Is it an average, a trending model such as double exponential smoothing, a seasonal model? Does the choice of technique change depend on the type of demand data or when new demand data is available? (Spare parts and high-volume items have very different demand patterns.) How do you go about selecting the forecast model? Is this process automated?  How often is the choice of model reconsidered?  How often are the model parameters recomputed? What is the process used to reconsider your approach?  The answer here documents how the baseline forecasts are produced.  Once determined, you can conduct an analysis to identify whether other forecasting methods would improve forecast accuracy.  If you aren’t documenting forecast accuracy and conducting “forecast value add” analysis then you aren’t in a position to properly assess whether the forecasts being produced are the best that they can be.  You’ll miss out on opportunities to improve the process, increase forecast accuracy, and educate the business on what type of forecast error is normal and should be expected.

5. How do you use safety stock? Notice the question was not “Do you use safety stock?” In this context, and to keep it simple, the term “safety stock” means stock used to buffer inventory against supply and demand variability.  All companies use buffering approaches in some way.  There are some exceptions though.  Maybe you are a job shop manufacturer that procures all parts to order and your customers are completely fine waiting weeks or months for you to source material, manufacture, QA, and ship.  Or maybe you are high-volume manufacturer with tons of buying power so your suppliers set up local warehouses that are stocked full and ready to provide inventory to you almost immediately.  If these descriptions don’t describe your company, you will definitely have some sort of buffer to protect against demand and supply variability.  You may not use the “safety stock” field in your ERP but you are definitely buffering.

Answers might be provided such as “We don’t use safety stock because we forecast.”  Unfortunately, a good forecast will have a 50/50 chance of being over/under the actual demand.  This means you’ll incur a stock out 50% of the time without a safety stock buffer added to the forecast.  Forecasts are only perfect when there is no randomness. Since there is always randomness, you’ll need to buffer if you don’t want to have abysmal service levels.

If the answer isn’t revealed, you can probe a bit more into how the varying replenishment levers are used to add possible buffers which leads to questions 6 & 7.

6. Do you ever increase the lead time or order earlier than you truly need to?
In our hypothetical example, your supplier typically takes 4 weeks to deliver and is pretty consistent. But to protect against stockouts your buyer routinely orders 6 weeks out instead of 4 weeks.  The safety stock field in your ERP system might be set to zero because “we don’t use safety stock”, but in reality, the buyer’s ordering approach just added 2 weeks of buffer stock.

7. Do you pad the demand forecast?
In our example, the planner expects to consume 10 units per month but “just in case” enters a forecast of 20 per month.  The safety stock field in the MRP system is left blank but the now disguised buffer stock has been smuggled into the demand forecast.  This is a mistake that introduces “forecast bias.”  Not only will your forecasts be less accurate but if the bias isn’t accounted for and safety stock is added by other departments, you will overstock.

The ad-hoc nature of the above approaches compounds the problems by not considering the actual demand or supply variability of the item. For example, the planner might simply make a rule of thumb that doubles the lead time forecast for important items.  One-size doesn’t fit all when it comes to inventory management.  This approach will substantially overstock the predictable items while substantially understocking the intermittently demanded items. You can read “Beware of Simple Rules of Thumb for Managing Inventory” to learn more about why this type of approach is so costly.

The ad-hoc nature of the approaches also ignores what happens the company is faced with a huge overstock or stock out. When trying to understand what happened, the stated policies will be examined. In the case of an overstock, the system will show zero safety stock.  The business leaders will assume they aren’t carrying any safety stock, scratch their heads, and eventually just blame the forecast, declare “Our business can’t be forecasted” and stumble on. They may even blame the supplier for shipping too early and making them hold more than needed. In the case of a stock out, they will think they aren’t carrying enough and arbitrarily add more stock across many items not realizing there is in fact lots of extra safety stock baked into process.  This makes it more likely inventory will need to be written off in the future.

8. What is the exact inventory terminology used? Define what you mean by safety stock, Min, reorder point, EOQ, etc.  While there are standard technical definitions it’s possible that something differs, and miscommunication here will be problematic.  For example, some companies refer to Min as the amount of inventory needed to satisfy lead time demand while some may define Min as inclusive of both lead time demand and safety stock to buffer against demand variability. Others may mean the minimum order quantity.

9. Is on hand inventory consistent with the policy? When your detective work is done and everything is documented, open your spreadsheet or ERP system and look at the on-hand quantity. It should be more or less in line with your planning parameters (i.e. if Min/Max is 20/40 and typical lead time demand is 10, then you should have roughly 10 to 40 units on hand at any given point in time.  Surprisingly, for many companies there is often a huge inconsistency. We have observed situations where the Min/Max setting is 20/40 but the on-hand inventory is 300+.  This indicates that whatever policy has been prescribed just isn’t being followed.   That’s a bigger problem.

10. What are you going to do next?

Demand forecasting and inventory stocking policy need to be well-defined processes that are understood and accepted by everybody involved.  There should be zero mystery.

To do this right, the demand and supply variability must be analyzed and used to compute the proper levels of safety stock.   Adding buffers without an implicit understanding of what each additional unit of buffer stock is buying you in terms of service is like arbitrarily throwing a handful of ingredients into a cake recipe.  A small change in ingredients can have a huge impact on what comes out of the oven – one bite too sweet but the next too sour.  It is the same with inventory management.  A little extra here, a little less there, and pretty soon you find yourself with costly excess inventory in some areas, painful shortages in others, no idea how you got there, and with little guidance on how to make things better.

Modern inventory optimization and demand planning software with its advanced analytics and strong basis in forecast analysis can help a good deal with this problem. But even the best software won’t help if it is used inconsistently.

To test software solutions via a series of empirical competition can be a considerable option. For forecasting / demand planning, a traditional “hold out” test in which 2014-2018 data are provided to software vendors and 2019 is held out for later comparison against forecasts provided by competing vendors. The company then measures forecast error and bias. This approach is advocated nearly universally for assessing forecast accuracy. It’s a good way to assess monthly or weekly forecast accuracy, but it is minimally useful if you have a different objective: Optimizing inventory.

In our last blog, we discussed how to pick a targeted service level. We indicated that just because you set a target (or a system recommends a target) doesn’t mean you’ll actually achieve the target. The right way to measure accuracy if you are interested in optimizing stock levels is to focus on the accuracy of the service level projection. This will account for both lead time demand and safety stock.

Setting a target service level is a strategic decision about inventory risk management. Inventory software does the tactical work by computing reorder points (a.k.a. mins) meant to achieve a user-defined target or that will achieve a system-calculated optimal target. But if the software uses the wrong demand model, the achieved service level will miss the target, sometimes significantly. The result of this error will be either shortages or inventory bloat, depending on the direction of the miss.

Forecasting is a means to an end. The end is to optimize inventory levels. Because demand is uncertain, companies that need to provide even moderate service levels must stock more than the forecast, often much more. But doesn’t low forecast error mean lower safety stock? The better my forecasts, the lower my inventory? Yes, true. But what matters when determining the required inventory are both accurate forecasts of the most likely demand and accurate estimates of the variability around the most likely demand.

Especially with long tailed, intermittent demand, traditional forecast accuracy assessments over a conventional 12 month forecast horizon miss the point three ways.

– First, the relevant time scale for inventory optimization is the replenishment lead time, which is usually much shorter than 12 months. Demand during lead times measured in days or weeks has volatility that gets averaged out over long forecast horizons. This is bad because factoring in the effect of volatility is essential to calculation of optimal reorder points.

– Second, forecast accuracy assessed over a multi-month forecast horizon focuses on the typical error in a typical month within the horizon. In contrast, inventory optimization requires a focus on cumulative demand, not period-by-period demand.

– Third, and most important is that forecast error metrics are focused on the middle of the demand distribution, aiming to estimate the most likely demand. But setting reorder points involves estimating high percentiles of the cumulative demand distribution over a lead time. Estimating the middle a bit better but having no clue about, say, the 95th percentile, is not helpful.

Consider this hypothetical example. If Vendor A forecasts 20 units with 110% error and Vendor B forecasts 22 units with 105% error, then Vendor B has an advantage in the forecasting game. But if you want a high service level and the demand is intermittent, you’ll have to stock a lot more than 20 or 22 units. Let’s assume you select Vendor B’s technology to plan stocking levels. You then notice that when planning reorder points to achieve a 95% service level, you often fall short – way more often that the expected 5% of the time. You come to realize that Vendor B’s approach completely underestimates the safety stock required to achieve the target service target. Focusing on vendors’ forecast error isn’t going to help. You will come to wish that you had verified Vendor A and B’s service level accuracy. Now you are stuck arbitrarily adjusting Vendor B’s service level targets to compensate for the shortfall.

So what’s needed in vendor competitions is assessment of their systems’ abilities to accurately forecast the inventory required to meet a given service level over an item’s replenishment lead time. Narrowly focusing on measuring forecast error is not appropriate if the mission is managing inventory. This is especially true for long tail items with intermittent demand or items that have medium to high volume but don’t have a demand distribution that looks like the classic “bell shaped curve” (Normal distribution).

The remainder of this blog explains how to test the accuracy of software’s service level calculations, so you can monitor the risk of missing your service level targets. We recommend this accuracy test over traditional “forecast versus actuals” tests because it provides much more insight into how reorder point recommendations will influence inventory levels and customer service.

Office staff are analyzing The Right Forecast Accuracy Metric for Inventory Planning

Service Level Defined

Consider a single inventory item. When inventory drops to or below the reorder point, a replenishment order is generated. This starts a period of risk that lasts as long as the replenishment lead time. During the period of risk, there might be enough incoming demands to create backorders or lost sales. The service level is the probability that there are no backorders or stockouts during the replenishment lead time. Critical items might be given very high target service levels, say 99%, whereas other items might have more relaxed targets, such as 75%. Whatever the target service level, it is best to hit that target.

Calculating Service Level

The service level for an individual item can only be estimated by repeated comparison of observed lead time demand against the calculated reorder point. These estimates take a lot of time: at least dozens of lead times. But fleet-wise service level can be estimated using data compiled over a single lead time.

Let’s do an example. Suppose you have demand histories for 1,000 items over 365 days and that (for simplicity) all items have 45-day lead times. For each item, follow these steps to estimate the fleet-wise achieved service level:

Step 1: Step aside (“hold out”) the most recent 45 days of demand (or however many days is closest to your typical lead times). Compute their sum, which is the most recent value of the actual lead time demand. This is the ground truth to be used to estimate the achieved service level.

Step 2: Use the prior 320 days of demand history to forecast the required inventory to hit a range of service level targets, say 90%, 95%, 97%, and 99%.

Step 3: Check whether the observed lead time demand is less than or equal to the reorder point. If it is, count this as a win; otherwise, count it as a loss. For instance, if the reorder point is 15 units but the most recent lead time demand is 10 units, then this is a win, since the reorder point is high enough to cover a lead time demand of 10 without any shortage. However, if the most recent lead time demand is 18 units, there would be a stockout, and 3 units would either be backordered or counted as lost sales.

Step 4: Working across all items, and all service level targets, tally the percentage of tests for each service level target that resulted in a win. This is the achieved service level. If the target was 90% and 853 of the 1,000 units record a win, then the achieved service level is 85.3%.

Example

Consider a real-world example. The data are daily demand histories of 590 medical supply items used in an internationally famous clinic. For simplicity, we assume each item has a lead time of 45 days. We evaluate target service levels of 70%, 90%, 95% and 99%.
We compare two demand models. The “Normal” model assumes that daily demand has a Normal (“bell-shaped”) distribution. This is the classic assumption used in most introductory textbooks on inventory control and in many software products. Classic though it may be, it is often an inappropriate model of demand for spare parts or supplies. The “Probability Forecast” model takes explicit account of the intermittent nature of demand.

Exhibit 1 shows the results. Column J shows the actual demand over the final 45 observations. The computed reorder points for the Advanced Model are shown in columns L-O.  The computed reorder points for the Normal model are not displayed.  Columns Q-T and V-Y hold the results of the tests for whether the reorder points were high enough to handle the lead time demands in column J.

The final results (yellow cells) show a clear difference between the Normal and Probability (Advanced) demand models. Both did a good job of hitting the 70% service level target, but estimating higher service levels is a more delicate calculation, and the Probability model does a much better job. For instance, the Normal model’s supposed 99% service level turned out to be only 94.4%, while the Probability model hit the target with a 98.5% achieved service level.

Implications

Utilizing the more accurate method achieved the targeted service level, while the less accurate method did not. If the less accurate method is used then real and costly business decisions will be made on the false assumption that a higher service level will be achieved. For example, if a Service Level Agreement (SLA) is based on these results and a 99% service level is committed to, the supplier would actually be five times more likely to stock out than planned (service level promised = 99% or 1% stockout risk vs. service level achieved = 94.5% or 5.5% stock out risk)! This means financial penalties will be incurred five times more often than expected.

Suppose that planners knew the target service level would not be met but were stuck using an inaccurate model. They would still need a way to increase inventory and achieve the desired level of service. What might they choose to do? We have observed situations where the planner enters a higher service level target than needed in order to “trick” the system into delivering the required service level. In the above example, the Normal model needed to have a 99.99% service level entered before it could achieve a target service level of 99%. This change resulted in achieving a 99% service but more than doubled the inventory investment when compared to the Advanced model.

Implementing a Service Level Accuracy Test

At Smart Software, we’ve encouraged many of our customers to conduct the test of service level accuracy as a way for them to assess our and other vendors’ claims during the software selection process. Missing the service level target has extremely costly implications resulting in substantial over stocks or under stocks.  So, test service level accuracy before deploying a solution to identify situations when the modeling fails. Don’t assume that you will achieve the service level you decide to target (or that the system recommends). To request an Excel spreadsheet that serves as a template for a service level accuracy test, email your contact information to info@smartcorp.com and enter “Accuracy Template” in the subject line.

You can’t properly manage your inventory levels, let alone optimize them, if you don’t have a handle on exactly how demand forecasts and stocking parameters (such as Min/Max, safety stocks, and reorder points, and order quantities) are determined.

Many organizations cannot specify how policy inputs are calculated or identify situations calling for management overrides to the policy.   For example, many people can say they rely on a particular planning method such as Min/Max, reorder point, or forecast with safety stock, but they can’t say exactly how these planning inputs are calculated.  More fundamentally, they may not understand what would happen to their KPI’s if they were to change Min,Max, or Safety Stock. They may know that the forecast relies on “averages” or “history” or “sales input”, but specific details about how the final forecast is arrived at are unclear.

Often enough, a company’s inventory planning and forecasting logic was developed by a former employee or vanished consultant and entombed in a spreadsheet.  It otherwise may rely on outdated ERP functionality or ERP customization by an IT organization that incorrectly assumed that ERP software can and should do everything. (Read this great and, as they say, “funny because it’s true,” blog by Shaun Snapp about ERP Centric Strategies.)  The policy may not have been properly documented, and no one currently on the job can improve it or use it to best advantage.

This unhappy situation leads to another, in which buyers and inventory planners flat out ignore the output from the ERP system, forcing reliance on Microsoft Excel to determine order schedules.  Ad hoc methods are developed that impede cohesive responses to operational issues and aren’t visible to the rest of the organization (unless you want your CFO to learn the complex and finicky spreadsheet).  These methods often rely on rules of thumb, averaging techniques, or textbook statistics without a full understanding of their shortcomings or applicability.  And even when documented, most companies often discover that actual ordering strays from the documented policy.  One company we consulted for had on hand inventory levels that were routinely 2 x’s the Max quantity!  In other words, there isn’t really a policy at all.

In summary, many current inventory and demand forecast “systems” were developed out of distrust for the previous system’s suggestions but don’t actually improve KPI’s.  They also force the organization to rely on a few employees to manage demand forecasting, daily ordering, and inventory replenishment.

And when there is a problem, it is impossible for the executive team to unwind how you got there, because there are too many moving parts.  For example, was the excess stock the fault of an inaccurate demand forecast that relied on an averaging method that didn’t account for a declining demand?  Or was it due to an outdated lead time setting that was higher than it should’ve been?  Or was it due to a forecast override a planner made to account for an order that just never happened?  And who gave the feedback to make that override?  A customer? Salesperson?

Do you have any of these problems?  If so, you are wasting hundreds of thousands to millions of dollars each year in unnecessary shortage costs, holding costs, and ordering costs.  What would you be able to do with that extra cash?  Imagine the impact that this would have on your business.

This blog details the top 10 questions that you can ask in order to uncover what’s really happening at your company.  We detail the typical answers provided when a forecasting/inventory planning policy doesn’t really exist, explain how to interpret these answers, and offer some clear advice on what to do about it.

Service level is a key performance indicator for companies that put a premium on satisfying customer demand. Service level is defined as the probability of surviving a replenishment lead time without stocking out.

Inventory management best practice begins with setting service level targets, then calculates reorder points (also called Mins) to achieve those targets. These calculations should account for variability in both demand and replenishment lead time. There are many software systems available for doing these calculations. If everything works out, the achieved service level ends up very close to the target service level. Unfortunately, there is often a painful gap between the two.

One reason for the gap is unrealistic models of demand. In many cases, software for calculating reorder points uses textbook formulas based on mathematical assumptions that make analysis simple at the expense of realism.  Many “Inventory 101” textbooks use formulas that assume demand has a Normal distribution (a.k.a. the “bell-shaped curve”) for finished goods and the Poisson distribution for spare parts. Fortunately, there are now inventory optimization and forecasting systems that process the actual demand histories of the inventory items using probabilistic forecasting.  These solutions calculate an accurate estimate of the distribution – not some idealized version.  To learn more check out this past blog on probabilistic forecasting:

But there is a second source of error in textbooks that operates invisibly in many inventory software package:  “undershoot”.

Calculations of reorder points almost always assume that stockouts arise when the total demand during a replenishment interval exceeds the reorder point. For example, assume that demand averages 1 unit per day. If lead time is 5 days, then on average lead time demand is 5 units. Setting the reorder point at 5 units would yield a laughable service level somewhere in the vicinity of 50%. Adding safety stock to the calculation might result in a reorder point of, say, 11 units, which might correspond to a service level of 95%. Another way to say this is, starting at a reorder point of 11 units, there should be a 95% chance of surviving the 5 day lead time without experiencing cumulative demand of more than 11 units. Theoretically!

What’s missing from this analysis is the undershoot phenomenon. Undershoot means that the lead time begins not at the reorder point but below it. Undershoot happens every time the demand that breached the reorder point took the stock down below (not down to) the reorder point. The figure below shows replenishment cycles with and without undershoot.  Undershoot picks your pocket before you even begin to roll the dice. It deludes the inventory professional into thinking his or her reorder points are sufficient to achieve their targets, whereas actual performance will not make the grade.

There is only one situation in which undershoot is not a worry: when demand is always either zero or one unit. In that case, undershoot is impossible. But in all other cases, undershoot is sure to happen to some extent, and it can seriously undercut the service level actually achieved by a given choice of reorder point. Our analyses show that the conditions most vulnerable to undershoot involve highly intermittent and skewed demand with very short lead times – the very conditions being made most common by market trends.

What can be done to protect yourself from the effect of undershoot on reorder point calculations?  Use inventory optimization and forecasting software that isn’t tied to the old textbook assumptions and instead automatically accounts for undershoot when calculating the service level produced by any choice of reorder point.

To see Smart Software’s Inventory Optimization solution in action, register to see a recorded demo below: