In this blog, we will analyze the automatic forecasting for time series demand projections, focusing on key techniques, challenges, and best practices. There are multiple methods to predict future demand for an item, and this becomes complex when dealing with thousands of items, each requiring a different forecasting technique due to their unique demand patterns. Some items have stable demand, others trend upwards or downwards, and some exhibit seasonality. Selecting the right method for each item can be overwhelming. Here, we’ll explore how automatic forecasting simplifies this process.

Automatic forecasting becomes fundamental in managing large-scale demand projections. With thousands of items, manually selecting a forecasting method for each is impractical. Automatic forecasting uses software to make these decisions, ensuring accuracy and efficiency in the forecasting process. It’s importance lies in its ability to handle complex, large-scale forecasting needs efficiently. It eliminates the need for manual selection, saving time and reducing errors. This approach is particularly beneficial in environments with diverse demand patterns, where each item may require a different forecasting method.

Key Considerations for Effective Forecasting

1. Challenges of Manual Forecasting:
   • Infeasibility: Manually choosing forecasting methods for thousands of items is unmanageable.
   • Inconsistency: Human error can lead to inconsistent and inaccurate forecasts.
2. Criteria for Method Selection:
   • Error Measurement: The primary criterion for selecting a forecasting method is the typical forecast error, defined as the difference between predicted and actual values. This error is averaged over the forecast horizon (e.g., monthly forecasts over a year).
   • Holdout Analysis: This technique simulates the process of waiting for a year to elapse by hiding some historical data, making forecasts, and then revealing the hidden data to compute errors. This helps in choosing the best method in real-time.
3. Forecasting Tournament:
   • Method Comparison: Different methods compete to forecast each item, with the method producing the lowest average error winning.
   • Parameter Tuning: Each method is tested with various parameters to find the optimal settings. For example, simple exponential smoothing may be tried with different weighting factors.

The Algorithms Behind Effective Automatic Forecasting

Automatic forecasting is highly computational but feasible with modern technology. The process involves:

• Data Segmentation: Dividing historical data into segments helps manage and leverage different aspects of historical data for more accurate forecasting. For instance, for a product with seasonal demand, data might be segmented by seasons to capture season-specific trends and patterns. This segmentation allows forecasters to make and test forecasts more effectively.
• Repeated Simulations: Using sliding simulations involves repeatedly testing and refining forecasts over different periods. This method validates the accuracy of forecasting methods by applying them to different segments of data. An example is the sliding window method, where a fixed-size window moves across the time series data, generating forecasts for each position to evaluate performance.
• Parameter Optimization: Parameter optimization involves trying multiple variants of each forecasting method to find the best-performing one. By adjusting parameters, such as the smoothing factor in exponential smoothing methods or the number of past observations in ARIMA models, forecasters can fine-tune models to improve performance.

For instance, in our software, we allow various forecasting methods to compete for the best performance on a given item.  Knowledge of Automatic forecasting immediately carries over to Simple Moving Average, linear moving average, Single Exponential Smoothing, Double Exponential Smoothing, Winters’ Exponential Smoothing, and Promo forecasting. This competition ensures that the most suitable method is selected based on empirical evidence, not subjective judgment. The tournament winner is the closest method to predicting new data values from old. Accuracy is measured by average absolute error (that is, the average error, ignoring any minus signs). The average is computed over a set of forecasts, each using a portion of the data, in a process known as sliding simulation, which we have explained previously in a previous blog.

Methods used in Automatic forecasting

Normally, there are six extrapolative forecasting methods competing in the Automatic forecasting tournament:

• Simple moving average
• Linear moving average
• Single exponential smoothing
• Double exponential smoothing
• Additive version of Winters’ exponential smoothing
• Multiplicative version of Winters’ exponential smoothing

The latter two methods are appropriate for seasonal series; however, they are automatically excluded from the tournament if there are fewer than two full seasonal cycles of data (for example, fewer than 24 periods of monthly data or eight periods of quarterly data). These six classical, smoothing-based methods have proven themselves to be easy to understand, easy to compute and accurate. You can exclude any of these methods from the tournament if you have a preference for some of the competitors and not others.

Automatic forecasting for time series data is essential for managing large-scale demand projections efficiently and accurately. Businesses can achieve better forecast accuracy and streamline their planning processes by automating the selection of forecasting methods and utilizing techniques like holdout analysis and forecasting tournaments. Embracing these advanced forecasting techniques ensures that businesses stay ahead in dynamic market environments, making informed decisions based on reliable data projections.

​Demand planning and statistical forecasting software play a pivotal role in effective business management by incorporating features that significantly enhance forecasting accuracy. One key aspect involves the utilization of smoothing-based or extrapolative models, enabling businesses to quickly make predictions based solely on historical data. This foundation rooted in past performance is crucial for understanding trends and patterns, especially in variables like sales or product demand. Forecasting software goes beyond mere data analysis by allowing the blending of professional judgment with statistical forecasts, recognizing that forecasting is not a one-size-fits-all process. This flexibility enables businesses to incorporate human insights and industry knowledge into the forecasting model, ensuring a more nuanced and accurate prediction.

Features such as forecasting multiple items as a group, considering promotion-driven demand, and handling intermittent demand patterns are essential capabilities for businesses dealing with diverse product portfolios and dynamic market conditions.  Proper implementation of these applications empowers businesses with versatile forecasting tools, contributing significantly to informed decision-making and operational efficiency.

Extrapolative models

Our demand forecasting solutions support a variety of forecasting approaches including extrapolative or smoothing-based forecasting models, such as exponential smoothing and moving averages.  The philosophy behind these models is simple: they try to detect, quantify, and project into the future any repeating patterns in the historical data.

  There are two types of patterns that might be found in the historical data:

• Trend
• Seasonality

These patterns are illustrated in the following figure along with random data.

Illustrating trending, seasonal, and random time series data

If the pattern is a trend, then extrapolative models such as double exponential smoothing and linear moving average estimates the rate of increase or decrease in the level of the variable and project that rate into the future.

If the pattern is seasonality, then models such as Winters and triple exponential smoothing estimate either seasonal multipliers or seasonal add factors and then apply these to projections of the nonseasonal portion of the data.

Very often, especially with retail sales data, both trend and seasonal patterns are involved. If these patterns are stable, they can be exploited to give very accurate forecasts.

Sometimes, however, there are no obvious patterns, so that plots of the data look like random noise. Sometimes patterns are clearly visible, but they change over time and cannot be relied upon to repeat. In these cases, the extrapolative models don’t try to quantify and project patterns. Instead, they try to average through the noise and make good estimates of the middle of the distribution of data values. These typical values then become the forecasts.  Sometimes, when users see a historical plot with lots of ups and downs they are concerned when the forecast doesn’t replicate those ups and downs. Normally, this should not be a reason for concern.  This occurs when the historical patterns aren’t strong enough to warrant using a forecasting method that would replicate the pattern.  You want to make sure your forecasts don’t suffer from the “wiggle effect” that is described in this blog post.

Past as a predictor of the future

The key assumption implicit in extrapolative models is that the past is a good guide to the future. This assumption, however, can break down. Some of the historical data may be obsolete. For example, the data might describe a business environment that no longer exists. Or, the world that the model represents may be ready to change soon, rendering all the data obsolete. Because of such complicating factors, the risks of extrapolative forecasting are lower when forecasting only a short time into the future.

Extrapolative models have the practical advantage of being cheap and easy to build, maintain and use. They require only accurate records of past values of the variables you need to forecast. As time goes by, you simply add the latest data points to the time series and reforecast. In contrast, the causal models described below require more thinking and more data. The simplicity of extrapolative models is most appreciated when you have a massive forecasting problem, such as making overnight forecasts of demand for all 30,000 items in inventory in a warehouse.

Judgmental adjustments

Extrapolative models can be run in a fully automatic mode with Demand Planner with no intervention required. Causal models require substantive judgment for wise selection of independent variables. However, both types of statistical models can be enhanced by judgmental adjustments. Both can profit from your insights.

Both causal and extrapolative models are built on historical data. However, you may have additional information that is not reflected in the numbers found in the historical record. For instance, you may know that competitive conditions will soon change, perhaps due to price discounts, or industry trends, or the emergence of new competitors, or the announcement of a new generation of your own products. If these events occur during the period for which you are forecasting, they may well spoil the accuracy of purely statistical forecasts. Smart Demand Planner’ graphical adjustment feature lets you include these additional factors in your forecasts through the process of on- screen graphical adjustment.

Be aware that applying user adjustments to the forecast is a two-edged sword. Used appropriately, it can enhance forecast accuracy by exploiting a richer set of information. Used promiscuously, it can add additional noise to the process and reduce accuracy. We advise that you use judgmental adjustments sparingly, but that you never blindly accept the predictions of a purely statistical forecasting method.  It is also very important to measure forecast value add.  That is, the value added to the forecast process by each incremental step.  For example, if you are applying overrides based on business knowledge, it is important to measure whether those adjustments are adding value by improving forecast accuracy.  Smart Demand Planner supports measurement of forecast value add by tracking every forecast considered and automating the forecast accuracy reports. You can select statistical forecasts, measure their errors, and compare them to the overridden ones.  By doing so, you inform the forecasting process so that better decisions can be made in the future. 

Multiple-level forecasts

Another common situation involves multiple-level forecasting, where there are multiple items being forecast as a group or there may even be multiple groups, with each group containing multiple items. We will generally call this type of forecasting Multilevel Forecasting. The prime example is product line forecasting, where each item is a member of a family of items, and the total of all the items in the family is a meaningful quantity.

For example, as in the following figure, you might have a line of tractors and want forecasts of sales for each type of tractor and for the entire tractor line.

Illustrating multiple-level product forecasts

 Smart Demand Planner provides Roll Up/Roll Down Forecasting. This function is crucial for obtaining comprehensive forecasts of all product items and their group total. The Roll Down/Roll Up method within this feature offers two options for obtaining these forecasts:

Roll Up (Bottom-Up): This option initially forecasts each item individually and then aggregates the item-level forecasts to generate a family-level forecast.

Roll Down (Top-Down): Alternatively, the roll-down option starts by forming the historical total at the family level, forecasts it, and then proportionally allocates the total down to the item level.

When utilizing Roll Down/Roll Up, you have access to the full array of forecast methods provided by Smart Demand Planner at both the item and family levels. This ensures flexibility and accuracy in forecasting, catering to the specific needs of your business across different hierarchical levels.

Forecasting research has not established clear conditions favoring either the top-down or bottom-up approach to forecasting. However, the bottom-up approach seems preferable when item histories are stable, and the emphasis is on the trends and seasonal patterns of the individual items. Top-down is normally a better choice if some items have very noisy history or the emphasis is on forecasting at the group level. Since Smart Demand Planner makes it fast and easy to try both a bottom-up and a top- down approach, you should try both methods and compare the results.  You can use Smart Demand Planner’s “Hold back on Current”  feature in the “Forecast vs. Actual” to test both approaches on your own data and see which one yields a more accurate forecast for your business. 

In almost every business and industry, decision-makers need reliable forecasts of critical variables, such as sales, revenues, product demand, inventory levels, market share, expenses, and industry trends.

Many kinds of people make these forecasts. Some are sophisticated technical analysts, such as business economists and statisticians. Many others regard forecasting as an important part of their overall work: general managers, production planners, inventory control specialists, financial analysts, strategic planners, market researchers, and product and sales managers. Still, others seldom think of themselves as forecasters but often have to make forecasts on an intuitive, judgmental basis.

Because of the way we designed Smart Demand Planner, it has something to offer all types of forecasters. This design grows out of several observations about the forecasting process. Because we designed Smart Demand Planner with these observations in mind, we believe it has a style and content uniquely suited for turning your browser into an effective forecasting and planning tool:

Forecasting is an art that requires a mix of professional judgment and objective, statistical analysis.

It is often effective to begin with an objective statistical forecast that automatically accounts for trends, seasonality, and other patterns.  Then, apply adjustments or forecast overrides based on your business judgment. Smart Demand Planner makes it easy to execute graphical and tabular adjustments to statistical forecasts.

The forecasting process is usually iterative.

You will likely decide to make several refinements of your initial forecast before you are satisfied. You may want to exclude older historical data that you find to no longer be relevant.  You could apply different weights to the forecast model that put varying emphases on the most recent data. You could apply trend dampening to increase or decrease aggressively trending statistical forecasts.  You could allow the Machine Learning models to fine-tune the forecast selection for you and select the winning model automatically.  Smart Demand Planner’s processing speed gives you plenty of time to make several passes and saves multiple versions of the forecasts as “snapshots” so you can compare forecast accuracy later.

Forecasting requires graphical support.

The patterns evident in data can be seen by a discerning eye. The credibility of your forecasts will often depend heavily on graphical comparisons other business stakeholders make when they assess the historical data and forecasts. Smart Demand Planner provides graphical displays of forecasts, history, and forecast vs. actuals reporting.

Forecasts are never exactly correct.

Because some error always creeps into even the best forecasting process, one of the most useful supplements to a forecast is an honest estimate of its margin of error.

Smart Demand Planner presents both graphical and tabular summaries of forecast accuracy based on the acid test of predicting data held back from development of the forecasting model. 

Forecast intervals or confidence intervals are also very useful.  They detail the likely range of possible demand that is expected to occur.  For example, if actual demand falls outside of the 90% confidence interval more than 10% of the time then there is reason to investigate further.  

Forecasting requires a match of method to data.

One of the major technical tasks in forecasting is to match the choice of forecasting technique to the nature of the data. Features of a data series like trend, seasonality or abrupt shifts in level suggest certain techniques instead of others.

Smart Demand Planner’ Automatic forecasting feature makes this match quickly, accurately and automatically.

Forecasting is often a part of a larger process of planning or control.

For example, forecasting can be a powerful complement to spreadsheet-based financial analysis, extending rows of figures off into the future. In addition, accurate sales and product demand forecasts are fundamental inputs to a manufacturer’s production planning and inventory control processes. An objective statistical forecast of future sales will always help identify when the budget (or sales plan) may be too unrealistic. Gap analysis enables the business to take corrective action to their demand and marketing plans to ensure they do not miss the budgeted plan.

Forecasts need to be integrated into ERP systems
Smart Demand Planner can quickly and easily transfer its results to other applications, such as spreadsheets, databases and planning systems including ERP applications.  Users are able to export forecasts in a variety of file formats either via download or to secure FTP file locations.  Smart Demand Planner includes API based integrations to a variety of ERP and EAM systems including Epicor Kinetic and Epicor Prophet 21, Sage X3 and Sage 300, Oracle NetSuite, and each of Microsoft’s Dynamics 365 ERP systems. API based integrations enable customers to push forecast results directly back to the ERP system on demand.

The result is more efficient sales planning, budgeting, production scheduling, ordering, and inventory planning.

​When you should use traditional extrapolative forecasting techniques.

With so much hype around new Machine Learning (ML) and probabilistic forecasting methods, the traditional “extrapolative” or “time series” statistical forecasting methods seem to be getting the cold shoulder.  However, it is worth remembering that these traditional techniques (such as single and double exponential smoothing, linear and simple moving averaging, and Winters models for seasonal items) often work quite well for higher volume data. Every method is good for what it was designed to do.  Just apply each appropriately, as in don’t bring a knife to a gunfight and don’t use a jackhammer when a simple hand hammer will do. 

Extrapolative methods perform well when demand has high volume and is not too granular (i.e., demand is bucketed monthly or quarterly). They are also very fast and do not use as many computing resources as probabilistic and ML methods. This makes them very accessible.

Are the traditional methods as accurate as newer forecasting methods?  Smart has found that extrapolative methods do very poorly when demand is intermittent. However, when demand is higher volume, they only do slightly worse than our new probabilistic methods when demand is bucketed monthly.  Given their accessibility, speed, and the fact you are going to apply forecast overrides based on business knowledge, the baseline accuracy difference here will not be material.

The advantage of more advanced models like Smart’s GEN2 probabilistic methods is when you need to predict patterns using more granular buckets like daily (or even weekly) data.  This is because probabilistic models can simulate day of the week, week of the month, and month of the year patterns that are going to be lost with simpler techniques.  Have you ever tried to predict daily seasonality with a Winter’s model? Here is a hint: It’s not going to work and requires lots of engineering.

Probabilistic methods also provide value beyond the baseline forecast because they generate scenarios to use in stress-testing inventory control models. This makes them more appropriate for assessing, say, how a change in reorder point will impact stockout probabilities, fill rates, and other KPIs. By simulating thousands of possible demands over many lead times (which are themselves presented in scenario form), you’ll have a much better idea of how your current and proposed stocking policies will perform. You can make better decisions on where to make targeted stock increases and decreases.

So, don’t throw out the old for the new just yet. Just know when you need a hammer and when you need a jackhammer.

Probabilistic scenarios are sequences of data points generated to represent potential real-world situations. Unlike scenarios in war games or other simulations, these are synthetic time series used as inputs to system models or as intuition-builders for decision-makers.

For instance, scenarios of future item demand can be fed into Monte Carlo simulation models of inventory control systems, thereby creating a virtual laboratory in which to explore the consequences of management decisions, such as changing reorder points and/or order quantities. In addition, plots of metrics like on-hand inventory or stockouts can help inventory planners deepen their “feel” for the randomness inherent in their operations.

Figure 1 shows daily demand scenarios generated from a single observed demand series recorded over one year. Note that the same data generating process can “look quite different” in detail from sample to sample. This mimics real life.

Figure 1: An observed demand sequence and demand scenarios derived from it.

Figure 2 shows two demand scenarios and their consequences for stock on hand in a particular inventory control system. The difference between the two inventory plots illustrates the degree to which randomness in demand dominates the problem. The top plot shows two episodes of stockout, while the bottom plot shows nine. Averaging over many scenarios will clarify the typical values of Key Performance Metrics (KPIs) such as the average number of stockouts associated with any choice of Reorder Point and Order Quantity (which are 10 and 25, respectively, in Figure 2.)

Figure 2: Two demand scenarios and their consequences for on-hand inventory

In this note, we’ll describe techniques for creating scenarios and list criteria for evaluating scenario generators.

Criteria for Scenarios

As we’ll see below, there are several ways to create scenarios. No matter the source, what criteria define a “good” scenario? There are four main criteria: fidelity, variety, quantity, and cost. Fidelity summarizes how accurately a scenario imitates real-world situations. High fidelity means the scenarios mirror actual events closely, providing a solid foundation for analysis and decision-making. Variety describes the diversity of scenarios a generator can create. A versatile generator can simulate a wide range of potential situations, allowing for a thorough exploration of possibilities and risks. Quantity refers to how many scenarios a generator can produce. A generator that can create a large number of scenarios provides ample data for analysis. Cost considers both the computational and human resources required to produce the scenarios. An efficient scenario generator balances quality with resource usage, ensuring the effort is justified by the value and accuracy of the outcomes.

Scenario Generation

Again, think of a scenario as a time series. How are scenarios created?

1. Geppetto’s Workshop: This approach involves hand-crafting scenarios manually by experts. While it can yield high fidelity (realism), it is very resource-intensive and cannot easily generate variety, which requires a large number of scenarios.
2. Groundhog Day: This method involves repeatedly using a single real-world situation as input. While it’s realistic by definition and cost-effective (no resources are used beyond recording the data), this approach lacks variety and so cannot accurately reflect the diversity of real-world scenarios.
3. Parametric Models: Examples of parametric models are the classics studied in Statistics 101 classes: the Normal, exponential, Poisson, etc. The demand plots in Figure 2 are generated parametrically, being the squares of Poisson random variables. These models generate an unlimited number of low cost scenarios having good variety, but they may not always capture the complexity of real-world data, potentially compromising fidelity. When reality is more complicated, these models generate over-simplified scenarios.
4. Non-Parametric Time Series Bootstraps: This approach can score well on all criteria: fidelity, variety, quantity, and cost. It’s a versatile method that excels in creating massive numbers of realistic scenarios. The synthetic demand histories in Figure 1 are simple bootstrap samples based on the observed values in the top graph. (For some nitty-gritty details about generating scenarios, see the links below.)

Exploiting Scenarios

Scenarios prove their worth in two ways: As inputs to decision making and as intuition-builders. For instance, when demand scenarios are used as inputs to simulation models, they enable stress testing and performance estimation for system design. Scenarios can also serve as intuition-builders for decision-makers or system operators. Their visual representation aids in developing insight into and appreciation for the risks involved in making operational decisions, be they for demand forecasting or inventory management.

Scenario-based analysis is very computer intensive, especially when the scenarios are generated by bootstrapping. At Smart Software, computation happens in the cloud. Imagine the computational load involved in determining reorder points and order quantities for each of tens of thousands of inventory items using hundreds or thousands of demand simulations for each item. Further imagine the software not only evaluating a specific proposed reorder point/order quantity pair but roaming over the entire “design space” of pairs to find the best pair of control parameters for each item. To make this practical, we take advantage of the parallel processing power of the cloud. Essentially, each inventory item is assigned its own computer to use in the calculations, so that all that computing can happen simultaneously rather than sequentially. Now we can cut loose and really get you the results you need.

Learning More

Those interested in further technical details and references can find more information here.

What Makes a Probabilistic Forecast?

Probabilistic Forecasting for Intermittent Demand