Measuring the accuracy of forecasts is an undeniably important part of the demand planning process. This forecasting scorecard could be built based on one of two contrasting viewpoints for computing metrics. The error viewpoint asks, “how far was the forecast from the actual?” The accuracy viewpoint asks, “how close was the forecast to the actual?” Both are valid, but error metrics provide more information.

Accuracy is represented as a percentage between zero and 100, while error percentages start at zero but have no upper limit. Reports of MAPE (mean absolute percent error) or other error metrics can be titled “forecast accuracy” reports, which blurs the distinction.  So, you may want to know how to convert from the error viewpoint to the accuracy viewpoint that your company espouses.  This blog describes how with some examples.

Accuracy metrics are computed such that when the actual equals the forecast then the accuracy is 100% and when the forecast is either double or half of the actual, then accuracy is 0%. Reports that compare the forecast to the actual often include the following:

• The Actual
• The Forecast
• Unit Error = Forecast – Actual
• Absolute Error = Absolute Value of Unit Error
• Absolute % Error = Abs Error / Actual, as a %
• Accuracy % = 100% – Absolute % Error

Look at a couple examples that illustrate the difference in the approaches. Say the Actual = 8 and the forecast is 10.

Unit Error is 10 – 8 = 2

Absolute % Error = 2 / 8, as a % = 0.25 * 100 = 25%

Accuracy = 100% – 25% = 75%.

Now let’s say the actual is 8 and the forecast is 24.

Unit Error is 24– 8 = 16

Absolute % Error = 16 / 8 as a % = 2 * 100 = 200%

Accuracy = 100% – 200% = negative is set to 0%.

In the first example, accuracy measurements provide the same information as error measurements since the forecast and actual are already relatively close. But when the error is more than double the actual, accuracy measurements bottom out at zero. It does correctly indicate the forecast was not at all accurate. But the second example is more accurate than a third, where the actual is 8 and the forecast is 200. That’s a distinction a 0 to 100% range of accuracy doesn’t register. In this final example:

Unit Error is 200 – 8 = 192

Absolute % Error = 192 / 8, as a % = 24 * 100 = 2,400%

Accuracy = 100% – 2,400% = negative is set to 0%.

Error metrics continue to provide information on how far the forecast is from the actual and arguably better represent forecast accuracy.

We encourage adopting the error viewpoint. You simply hope for a small error percentage to indicate the forecast was not far from the actual, instead of hoping for a large accuracy percentage to indicate the forecast was close to the actual.  This shift in mindset offers the same insights while eliminating distortions.

What’s different about inventory planning for Maintenance, Repair, and Operations (MRO) compared to inventory planning in manufacturing and distribution environments? In short, it’s the nature of the demand patterns combined with the lack of actionable business knowledge.

Demand Patterns

Manufacturers and distributors tend to focus on the top sellers that generate the majority of their revenue. These items typically have high demand that is relatively easy to forecast with traditional time series models that capitalize on predictable trend and/or seasonality.  In contrast, MRO planners almost always deal with intermittent demand, which is more sparse, more random, and harder to forecast.  Furthermore, the fundamental quantities of interest are different. MRO planners ultimately care most about the “when” question:  When will something break? Whereas the others focus on the “how much” question of units sold.

Business Knowledge

Manufacturing and distribution planners can often count on gathering customer and sales feedback, which can be combined with statistical methods to improve forecast accuracy. On the other hand, bearings, gears, consumable parts, and repairable parts are rarely willing to share their opinions. With MRO, business knowledge about which parts will be needed and when just isn’t reliable (excepting planned maintenance when higher-volume consumable parts are replaced). So, MRO inventory planning success goes only as far as their probability models’ ability to predict future usage takes them. And since demand is so intermittent, they can’t get past Go with traditional approaches.

Methods for MRO

In practice, it is common for MRO and asset-intensive businesses to manage inventories by resorting to static Min/Max levels based on subjective multiples of average usage, supplemented by occasional manual overrides based on gut feel. The process becomes a bad mixture of static and reactive, with the result that a lot of time and money is wasted on expediting.

There are alternative planning methods based more on math and data, though this style of planning is less common in MRO than in the other domains. There are two leading approaches to modeling part and machine breakdown: models based on reliability theory and “condition-based maintenance” models based on real-time monitoring.

Reliability Models

Reliability models are the simpler of the two and require less data. They assume that all items of the same type, say a certain spare part, are statistically equivalent. Their key component is a “hazard function”, which describes the risk of failure in the next little interval of time. The hazard function can be translated into something better suited for decision making: the “survival function”, which is the probability that the item is still working after X amount of use (where X might be expressed in days, months, miles, uses, etc.). Figure 1 shows a constant hazard function and its corresponding survival function.

Figure 1: Constant hazard function and its survival function

A hazard function that doesn’t change implies that only random accidents will cause a failure. In contrast, a hazard function that increases over time implies that the item is wearing out. And a decreasing hazard function implies that an item is settling in. Figure 2 shows an increasing hazard function and its corresponding survival function.

Figure 2: Increasing hazard function and its survival function

Reliability models are often used for inexpensive parts, such as mechanical fasteners, whose replacement may be neither difficult nor expensive (but still might be essential).

Condition-Based Maintenance

Models based on real-time monitoring are used to support condition-based maintenance (CBM) for expensive items like jet engines. These models use data from sensors embedded in the items themselves. Such data are usually complex and proprietary, as are the probability models supported by the data. The payoff from real-time monitoring is that you can see trouble coming, i.e., the deterioration is made visible, and forecasts can predict when the item will hit its red line and therefore need to be taken off the field of play. This allows individualized, pro-active maintenance or replacement of the item.

Figure 3 illustrates the kind of data used in CBM. Each time the system is used, there is a contribution to its cumulative wear and tear. (However, note that sometimes use can improve the condition of the unit, as when rain helps keep a piece of machinery cool). You can see the general trend upward toward a red line after which the unit will require maintenance. You can extrapolate the cumulative wear to estimate when it will hit the red line and plan accordingly.

Figure 3: Illustrating real-time monitoring for condition-based maintenance

To my knowledge, nobody makes such models of their finished goods customers to predict when and how much they will next order, perhaps because the customers would object to wearing brain monitors all the time. But CBM, with its complex monitoring and modeling, is gaining in popularity for can’t-fail systems like jet engines. Meanwhile, classical reliability models still have a lot of value for managing large fleets of cheaper but still essential items.

Smart’s approach
The above condition-based maintenance and reliability approaches require an excessive data collection and cleansing burden that many MRO companies are unable to manage. For those companies, Smart offers an approach that does not require development of reliability models. Instead, it exploits usage data in a different way. It leverages probability-based models of both usage and supplier lead times to simulate thousands of possible scenarios for replenishment lead times and demand.  The result is an accurate distribution of demand and lead times for each consumable part that can be exploited to determine optimal stocking parameters.   Figure 4 shows a simulation that begins with a scenario for spare part demand (upper plot) then produces a scenario of on-hand supply for particular choices of Min/Max values (lower line). Key Performance Indicators (KPIs) can be estimated by averaging the results of many such simulations.

Figure 4: An example of a simulation of spare part demand and on-hand inventory

You can read about Smart’s approach to forecasting spare parts here: https://smartcorp.com/wp-content/uploads/2019/10/Probabilistic-Forecasting-for-Intermittent-Demand.pdf

Automatic forecasting is the most popular and most used feature of SmartForecasts and Smart Demand Planner. Creating Automatic forecasts is easy. But, the simplicity of Automatic Forecasting masks a powerful interaction of a number of highly effective methods of forecasting. In this blog, we discuss some of the theory behind this core feature. We focus on Automatic forecasting, in part because of its popularity and in part because many other forecasting methods produce similar outputs. Knowledge of Automatic forecasting immediately carries over to Simple Moving Average, Linear Moving Average, Single Exponential Smoothing, Double Exponential Smoothing, Winters’ Exponential Smoothing, and Promo forecasting.

Forecasting tournament

Automatic forecasting works by conducting a tournament among a set of competing methods. Because personal computers and cloud computing are fast, and because we have coded very efficient algorithms into the SmartForecasts’ Automatic forecasting engine, it is practical to take a purely empirical approach to deciding which extrapolative forecasting method to use. This means that you can afford to try out a number of approaches and then retain the one that does best at forecasting the particular data series at hand. SmartForecasts fully automates this process for you by trying the different forecasting methods in a simulated forecasting tournament. The winner of the tournament is the method that comes closest to  predicting new data values from old. Accuracy is measured by average absolute error (that is, the average error, ignoring any minus signs). The average is computed over a set of forecasts, each using a portion of the data, in a process known as sliding simulation.

Sliding simulation

The sliding simulation sweeps repeatedly through ever-longer portions of the historical data, in each case forecasting ahead the desired number of periods in your forecast horizon. Suppose there are 36 historical data values and you need to forecast six periods ahead. Imagine that you want to assess the forecast accuracy of some particular method, say a moving average of four observations, on the data series at hand.

At one point in the sliding simulation, the first 24 points (only) are used to forecast the 25th through 30th historical data values, which we temporarily regard as unknown. We say that points 25-30 are “held out” of the analysis. Computing the absolute values of the differences between the six forecasts and the corresponding actual historical values provides one instance each of a 1-step, 2-step, 3-step, 4-step, 5-step, and 6-step ahead absolute forecast error. Repeating this process using the first 25 points provides more instances of 1-step, 2-step, 3-step ahead errors, and so on. The average over all of the absolute error estimates obtained this way provides a single-number summary of accuracy.

Methods used in Automatic forecasting

Normally, there are six extrapolative forecasting methods competing in the Automatic forecasting tournament:

• Simple moving average
• Linear moving average
• Single exponential smoothing
• Double exponential smoothing
• Additive version of Winters’ exponential smoothing
• Multiplicative version of Winters’ exponential smoothing

The latter two methods are appropriate for seasonal series; however, they are automatically excluded from the tournament if there are fewer than two full seasonal cycles of data (for example, fewer than 24 periods of monthly data or eight periods of quarterly data).

These six classical, smoothing-based methods have proven themselves to be easy to understand, easy to compute and accurate. You can exclude any of these methods from the tournament if you have a preference for some of the competitors and not others.

In today’s competitive business landscape, companies are constantly seeking ways to improve their operational efficiency and drive increased revenue. Optimizing service parts management is an often-overlooked aspect that can have a significant financial impact. Companies can improve overall efficiency and generate significant financial returns by effectively managing spare parts inventory. This article will explore the economic implications of optimized service parts management and how investing in Inventory Optimization and Demand Planning Software can provide a competitive advantage.

The Importance of Optimized Service Parts Planning:

Optimized service parts management plays a vital role in mitigating inventory risks and ensuring critical spare parts availability. While subjective planning may work on a small scale, it becomes insufficient when managing large inventories of intermittently demanded spare parts. Traditional forecasting approaches simply fail to accurately account for the extreme demand variability and frequent periods of zero demand that is so common with spare parts.  This results in large misallocations of stock, higher costs, and poor service levels.

The key to optimized service parts management lies in understanding the trade-off between service and cost. Inventory Optimization and Demand Planning Software powered by probabilistic forecasting and Machine learning Algorithms can help companies better understand the cost vs. benefit of each inventory decision and wield inventory as a competitive asset. By generating accurate demand forecasts and optimal stocking policies such as Min/Max, Safety Stock Levels, and Reorder Points in seconds, companies can know how much is too much and when to add more. By wielding inventory as a competitive asset, companies can drive up service levels and drive down costs.

Improve the Financial Outcome of Spare Parts Planning

1. Accurate forecasting is crucial to optimize inventory planning and meet customer demand effectively. State-of-the-art demand planning software accurately predicts inventory requirements, even for intermittent demand patterns. By automating forecasting, companies can save time, money, and resources while improving accuracy.
2. Meeting customer demand is a critical aspect of service parts management. Companies can enhance customer satisfaction, loyalty, and increase their chances of winning future contracts for the asset-intensive equipment they sell by ensuring the availability of spare parts when needed. Through effective demand planning and inventory optimization, organizations can reduce lead times, minimize stockouts, and maintain service levels, thereby improving the financial impact of all decisions.
3. Financial gains can be achieved through optimized service parts planning, including the reduction of inventory and product costs. Excess storage and obsolete inventory can be significant cost burdens for organizations. By implementing best-of-breed inventory optimization software, companies can identify cost-effective solutions, driving up service levels and reducing costs. This leads to improved inventory turnover, reduced carrying costs, and increased profitability.
4. Procurement planning is another essential aspect of service parts management. Organizations can optimize inventory levels, reduce lead times, and avoid stockouts by aligning procurement and the associated order quantities with accurate demand forecasts. For example, accurate forecasts can be shared with suppliers so that blanket purchase commitments can be made. This provides the supplier revenue certainty and, in exchange, can hold more inventory, thereby reducing lead times.
5. Intermittent demand planning is a particular challenge in spare parts management. Conventional rule-of-thumb approaches fall short in handling demand variability effectively. This is because traditional approaches assume demand is normally distributed when in reality, it is anything but normal. Spare parts demand random bursts of large demand intersperse many period of zero demand.  Smart Software’s solution incorporates advanced statistical models and machine learning algorithms to analyze historical demand patterns, enabling accurate planning for intermittent demand. Companies can significantly reduce stockout costs and improve efficiency by addressing this challenge.

Evidence from Smart Software’s Customers:

Investing in Smart Software’s Inventory Optimization and Demand Planning Software enables companies to unlock cost savings, elevate customer service levels, and enhance operational efficiency. Through accurate demand forecasting, optimized inventory management, and streamlined procurement processes, organizations can achieve financial savings, meet customer demands effectively, and improve overall business performance.

• Metro-North Railroad (MNR) experienced an 8% reduction in parts inventory, reaching a record high customer service level of 98.7%, and reduced inventory growth for new equipment from a projected 10% to only 6%. Smart Software played a crucial role in identifying multi-year service part needs, reducing administrative lead times, formulating stock reduction plans for retiring fleets, and identifying inactive inventory for disposal. MNR saved costs, maximized disposal benefits, improved service levels, and gained accurate insights for informed decision-making, ultimately improving their bottom line and customer satisfaction.
• Seneca Companies, an industry leader in automotive petroleum services, adopted Smart Software to model customer demand, control inventory performance, and drive replenishment. Field service technicians embraced its use, and total inventory investment decreased by more than 25%, from $11 million to $8 million, while maintaining first-time fix rates of 90%+.
• A leading Electric Utility implemented Smart IP&O in just 3 months and then used the software to optimize its reorder points and order quantities for over 250,000 spare parts. During the first phase of the implementation, the platform helped the Utility reduce inventory by $9,000,000 while maintaining service levels. The implementation was part of the company’s strategic supply chain optimization initiative.

Optimizing Service Parts Planning for Competitive Advantage

Optimized service parts management is crucial for companies seeking to improve efficiency, reduce costs, and ensure the availability of necessary spare parts. Organizations can unlock significant value in this field by investing in Smart Software’s Inventory Optimization and Demand Planning Software. Companies can achieve better financial performance and gain a competitive edge in their respective markets through improved data analysis, automation, and inventory planning.

Smart Software is designed for the modern marketplace, which is volatile and always changing. It can handle SKU proliferation, longer supply chains, less predictable lead times, and more intermittent and less forecastable demand patterns. It can also integrate with virtually every ERP solution on the market, by field-proven seamless connections or using a simple import/export process supported by Smart Software’s data model and data processing engine. By using Smart Software, companies can leverage inventory as a competitive asset, enhance customer satisfaction, drive up service levels, push down costs, and save substantial money.

Managing spare parts presents numerous challenges, such as unexpected breakdowns, changing schedules, and inconsistent demand patterns. Traditional forecasting methods and manual approaches are ineffective in dealing with these complexities. To overcome these challenges, this blog outlines key strategies that prioritize service levels, utilize probabilistic methods to calculate reorder points, regularly adjust stocking policies, and implement a dedicated planning process to avoid excessive inventory. Explore these strategies to optimize spare parts inventory and improve operational efficiency.

Bottom Line Upfront

​1.Inventory Management is Risk Management.

2.Can’t manage risk well or at scale with subjective planning – Need to know service vs. cost.

3.It’s not supply & demand variability that are the problem – it’s how you handle it.

4.Spare parts have intermittent demand so traditional methods don’t work.

5.Rule of thumb approaches don’t account for demand variability and misallocate stock.

6.Use Service Level Driven Planning  (service vs. cost tradeoffs) to drive stock decisions.

7.Probabilistic approaches such as bootstrapping yield accurate estimates of reorder points.

8.Classify parts and assign service level targets by class.

9.Recalibrate often – thousands of parts have old, stale reorder points.

10.Repairable parts require special treatment.

Do Focus on the Real Root Causes

Intermittent Demand

• Slow moving, irregular or sporadic with a large percentage of zero values.
• Non-zero values are mixed in randomly – spikes are large and varied.
• Isn’t bell shaped (demand is not Normally distributed around the average.)
• At least 70% of a typical Utility’s parts are intermittently demanded.

Normal Demand

• Very few periods of zero demand (exception is seasonal parts.)
• Often exhibits trend, seasonal, or cyclical patterns.
• Lower levels of demand variability.
• Is bell-shaped (demand is Normally distributed around the average.)

Don’t rely on averages

• OK for determining typical usage over longer periods of time.
• Often forecasts more “accurately” than some advanced methods.
• But…insufficient for determining what to stock.

Don’t Buffer with Multiples of Averages

Example:  Two equally important parts so let’s treat them the same.
We’ll order more  when On Hand Inventory ≤ 2 x Avg Lead Time Demand.

Do use Service Level tradeoff curves to compute safety stock

Standard Normal Probabilities

OK for normal demand. Doesn’t work with intermittent demand!

Don’t use Normal (Bell Shaped) Distributions

• You’ll get the tradeoff curve wrong:

– e.g., You’ll target 95% but achieve 85%.

– e.g., You’ll target 99% but achieve 91%.

• This is a huge miss with costly implications:

– You’ll stock out more often than expected.

– You’ll start to add subjective buffers to compensate and then overstock.

– Lack of trust/second-guessing of outputs paralyzes planning.

Why Traditional Methods Fail on Intermittent Demand: 

Traditional Methods are not designed to address core issues in spare parts management.

Need: Probability distribution (not bell-shaped) of demand over variable lead time.

• Get: Prediction of average demand in each month, not a total over lead time.
• Get: Bolted-on model of variability, usually the Normal model, usually wrong.

Need: Exposure of tradeoffs between item availability and cost of inventory.

• Get: None of this; instead, get a lot of inconsistent, ad-hoc decisions.

Do use Statistical Bootstrapping to Predict the Distribution:

Then exploit the distribution to optimize stocking policies.

How does Bootstrapping Work?

24 Months of Historical Demand Data.

Bootstrap Scenarios for a 3-month Lead Time.

Bootstrapping Hits the Service Level Target with nearly 100% Accuracy!

• National Warehousing Operation.

Task: Forecast inventory stocking levels for 12,000 intermittently demanded SKUs at 95% & 99% service levels

Results:

At 95% service level, 95.23% did not stock out.

At 99% service level, 98.66% did not stock out.

This means you can rely on output to set expectations and confidently make targeted stock adjustments that lower inventory and increase service.

Set Target Service Levels According to Order Frequency & Size

Recalibrate Reorder Points Frequently

• Static ROPs cause excess and shortages.
• As lead time increases, so should the ROP and vice versa.
• As usage decreases, so should the ROP and vice versa.
• Longer you wait to recalibrate, the greater the imbalance.
• Mountains of parts ordered too soon or too late.
• Wastes buyers’ time placing the wrong orders.
• Breeds distrust in systems and forces data silos.

Do Plan Rotables (Repair Parts) Differently

Summary

1.Inventory Management is Risk Management.

2.Can’t manage risk well or at scale with subjective planning – Need to know service vs. cost.

3.It’s not supply & demand variability that are the problem – it’s how you handle it.

4.Spare parts have intermittent demand so traditional methods don’t work.

5.Rule of thumb approaches don’t account demand variability and misallocate stock.

6.Use Service Level Driven Planning  (service vs. cost tradeoffs) to drive stock decisions.

7.Probabilistic approaches such as bootstrapping yield accurate estimates of reorder points.

8.Classify parts and assign service level targets by class.

9.Recalibrate often – thousands of parts have old, stale reorder points.

10.Repairable parts require special treatment.