What can you expect from this paper?
- Understanding how to Accurately Forecast in a COVID hit unstable market while eliminating guesswork for ramp-down pre-COVID and eventual revival
- Advantage of advanced machine learning Time-series models over expert opinions especially in an event such as COVID 19
- How to Tackle data quality issues faced while creating Time-Series Model
An accurate and robust forecasting capability is an asset to every organization indifferent to its industry or geography. A good forecast enables organizations to plan well in advance for resource management, market positioning and avert incurring of losses due to sudden shift from normalcy as recently seen during COVID 19.
An important application of this in the energy supply market is Consumption Forecasting i.e. how much energy our customers will consume over a period. Accurate forecasts can provide1 a superior edge over guesstimates by reducing over or under buying. Accurate energy forecasts also help with optimizing energy distribution, which has shown significant cost savings2 potential.
COVID 19 presented the industry with policy-based changes in consumption behavior. All other markets also saw their business performance plummet suddenly.
"COVID-19 has exposed the biggest flaw in expert’s estimate as a forecast. while guess based on historical data could lead to over-estimation, skepticism may lead to under-estimation. Both are based on guesses or biases of the forecaster"
Post-Black-Swan landscapes are complex to predict without data driven approach. Advanced techniques of time-series forecasting can help an organization rise above this difficulty and get a very robust revival forecast.
COVID 19 a Black Swan3 global pandemic shook all supply chain systems. Like many industries, B2B energy consumption behavior changed dramatically as shown in Figure 1.
The world is now moving towards a yet “new normal” of living with the virus, hence businesses are opening. Guessing “V-Shape” or “U-Shape” recovery is now becoming a trend that comes with a risk of under or over estimation, we try to address that risk in this paper.
“We at EXL developed a forecasting model to successfully forecast the ‘new normal’. We also provide insight into forecasting a ‘COVID revival’ market without any guesses, biases or estimates using the power of data insight and advanced forecasting techniques”
2. Phased Approach: “New Normal” and “Revival” phases
The shock to the system due to COVID 19 was a policy decision without the scope of gradual changes. Everything Shut and it shut on the same day. Plummet in the B2B energy consumption was a true reflection of this phenomenon. Nevertheless, was the shrinking permanent? Of course not, there was a lower bound as shown.
2.1 Bottomless Pit – forecasting “Rock Bottom” and “New Normal”
Places of congregation of people such as offices, gyms, pubs or factories were shutdown. Only essential services were running which meant a dip in energy consumption. Finding the end-point of this shrinkage was first challenge.
“A huge challenge was predicting when we would hit rock bottom without any guess work. Time-Series forecasting was able to forecast this accurately up to 97%”
This was achieved by combining the expert understanding of market within EXL and harnessing the power of the Machine learning forecasting algorithm.
- The industry experts’ team within EXL formulated industry Segmentation, bundling the business into “open” and “shut” based on in-depth analysis of policies of the lockdown. This helped in understanding how long it will take a business to shut operations, if it were to shut at all.
- As soon as the actual data of the “plummet” started coming in, the forecasting model was able to predict with 97% accuracy the entire fall. The crucial lower limit of consumption, marking the beginning of the new normal was also predicted with 97% accuracy.
2.2 Forecasting Revival
Actual data during the “plummet” was used for fine-tuning the Expert Judgment Segmentation by EXL.
“Revival from COVID shock revolves around terms such as V-shaped or U-shaped recovery. The biggest question was – when would this start? how can arbitrary guessing for revival start date be avoided?”
Unlike the uncharted territory of lockdown, revival has become a speculation free process thanks to a robust understanding of business health based on the EXL segmentation. There are two key elements in the forecasting solution design that gives the algorithm an edge over traditional and estimate/guessing approach.
A. Appreciating the gradual nature of revival
The classic debate of Expert Vs Algorithm gets heated when the future is uncertain. To address the issue, we investigated the “nature of policy-driven growth”. The “characteristic of growth” was incorporated in the algorithm as an input to avoid guesswork.
“Rookie mistake that was being made in the market was incorporating into algorithm the idea that there would be a single inflection point of revival for all businesses to come back to normal”
A rudimentary approach for introducing revival is using an artificial and speculative date of beginning of revival. This would not be an appropriate measure since it was contingent on both governing body’s policy decision and individual business’ decision and pace to open shop and come back to the market post removal of lockdown.
WEEK ON WEEK INCREASE
If Week on Week increase is unaccounted for, it will result in overestimation. We can see in Figure 3 how a single inflection point in energy consumption causes over-estimation. Such an approach expects entire market to come back to normal overnight. Figure 4 appreciates the Week on Week gradual growth, thereby reducing the over-estimation.
GROWTH AS A CHARECTERISTIC OF BUSINESS
Along with weekly and annual seasonality, this Week on Week growth too needs to be a part of the algorithm. In the energy market an added feature was different growth pace in Weekend and Weekday consumption as seen below.
To incorporate this idea of policy driven growth, the growth characteristics can be included in the model. We studied the growth in energy market once policy of lifting the lockdown was implemented. We studied the two policies of lockdown easing in the UK and its impact on behaviour of UK commercial energy consumption as show in Figure 6 and 7
We observed a very clear consumption pattern of policy simulation on consumption. This information is and fed it into the fForecasting algorithm. Our making the methodology: as below
Forecasted Energy = Historical Behaviour+COVID Shock+Weekday Growth+Weeknend Growth+Revival Policy-yB based growth
With growth factor already provided, we eliminate any speculation for growth post next policy. Growth either stimulated by, due to anticipation, or as a direct result of a future policy will already be accounted for. Thus, in the long term, any jump that a market sees will already be accounted for by the gradual growth incorporated in the forecast previously.
Such preparedness can now be used to model any further shocks such as second wave fairly in advance.
We now move on to address how to tackle the uncertainty in forecasting for an uncertain user-base.
B. Strength of the customer base
Another vital information incorporated into the algorithm is accounts that are bound to shut permanently. Advanced machine learning techniques were used to calculate the “survivability and ability to consume energy”. The models predicted likelihood of a customer to be a “COVID victim” i.e. never to recover from the shock and fail permanently.
It is not just COVID type events that take impact the strength of customer base. There are other factors such as loosing customer to competition or winning new ones.
If we begin with “n” customers, then total energy consumed can be written as:
Churn Analytics comes into picture to predict how many customers will remain. Machine learning can be used to understand churn behaviour of customers and stop the same from happening, keeping n almost static.
Same methodology can be used for customer acquisition too. Recent studies show that more customers are always on lookout for a better deal4 making this exercise important in order to prevent customer churn. In the wake of COVID 19 type event, this methodology is working to help predict how many businesses will survive the shock.
“Risk evaluation is a well-established domain of statistics. Hence, it is not required to reinvent the wheel by recreating bespoke Delphi like scores. Understanding whether an account remains with the business is the key insight that these models deliver”
Thus, acquisition & churn must be incorporated to get an estimate of consumers base.
“n” is unstable by its nature, any method for prediction which relies on the customer base strength will subsequently be unstable too. Time-Series Forecasting directly answers this. It uses the behaviour of the consumption itself independent of customers’ base strength. Stochasticity due to acquisition & churn are incorporated in the mathematical construct of the model.
We require clean and thick data free of missing values to create robust forecasting model. This is a major issue that data cleaning or data quality teams try to address across organisations. At EXL, we came up with a meta-learning technique to overcome this issue.
Most energy markets use meters to read and store energy consumption. There have been efforts5 to make these devices “smart” but the technology does not cover entire market. Therefore, not all customer’s consumption data is readily available. “More often than not, the data is unclean, unreliable or infrequent”6
A way to resolve this is by creating a Group Classification Profile. If we have sufficient data to get a “meta” understanding of the group, the behaviour of individual consumer can then extrapolated7.
“Meta heuristics” can be used to condense a group data (figure 9) using the below methodology to convert it into a “Meta” group profile (Figure 10)
Where “W” refers to the weight of the Consumer based on how close they are to average seasonality profile and “M” is the Market share of such customers in a given industry.
The Profile (Figure 10) is independent of scale. For an individual with missing or sparse data (Figure 11) this “meta” profile can fill the missing values very accurately.
Finally, we can add extra information for better accuracy. One such factor is weather, which has a profound effect on energy consumption. For example, the 2019 weather data of UK and the representative UK B2B energy market consumption behavior (Figure 12 and 13) shows how closely related the two are. Machine Learning based Time-Series Forecasting algorithms have proven to be quite accurate in predicting future of such data8.
All these factors combined give us a final model as below.
Forecasted Energy = Historical Behaviour + COVID Shock + Weekday Growth + Weeknend Growth + Revival Policy- Bbased Growth + Weather + market factors
In this paper we saw through the example of B2B UK energy Market how advanced data driven forecasting can help overcome uncertainty in decision making during Black Swans events such as COVID 19.
Where the traditional expert estimates failed or were prone to over or under estimation, Time-Series forecasting helps create a robust and speculation free forecast.
1. Accurate Forecasting in a COVID hit market WITHOUT ASSUMPTIONS regarding revival
- Expert Judgment is prone to over/under forecasting for Black Swan events
- AI/ML Forecasting eliminates pessimist or optimist view to provide a realistic forecast
- Training AI/ML model to see “Growth” as a “Feature” removes guesswork and biases
- Models trained to incorporate “Growth Characteristics” gave 97% accurate forecasts THROUGHOUT COVID
2. AI/ML driven Forecast is free from THESE speculations making it accurate Expert Judgment is prone to over/under forecasting for Black Swan events
- Revival Date – no single date from which markets revive. “Gradual Growth” methodology eliminates any speculation regarding “From When Will Market revive?”
- COVID Duration – Rookie mistake of assuming “How many days will a market be affected?” is eliminated by studying Week on Week market growth
- U, V, L Shaped Recovery – Macroeconomic behavior does not dictate specific market growth. Bespoke AI/ML model required to forecast a specific product market
- Stimulus Jump – AI/ML models include such “Positive Shock” events and prepare forecaster to be accurate during such an event
3. Tackling data quality issues faced while creating AI/ML driven Time-Series Model
- COVID 19 showed how businesses and industry sectors behave during such an event
- This data is now well catalogued at EXL and this information can help any industry prepare for Future Black Swans
- “Meta Learning” techniques can be used to tackle data quality issues eliminating using artificial data
Consultant II, Analytics
Consultant II, Analytics
Engagement Manager, Analytics