SPECIAL FEATURE
Forecasting: Academia versus Business
Reprinted from Foresight, Issue 41 (Spring 2016)
SUJIT SINGH
I n a recent discussion on LinkedIn, the question was
raised: Is there a disconnect between the academic
and the business world when it comes to forecasting?
As you can imagine, this is a matter of checking the
record and reporting on what one finds in the data.
Since I have neither the data nor the resources to
conduct this research, though, a comment from me
amounts perhaps to overreaching. Still, Ill try to
present what I personally see in the marketplace and
do so (hopefully) without any biases.
TYPES OF FORECASTING RESEARCH
In my mind, there are four types of research on
forecasting:
Statistics/mathematics-focused
- With many subfields, such as time-series fore-
casting and causal modeling
- For the most part, using computer software
Behavioral-focused
- Judgmental forecasts and judgmental adjust-
ments to statistical forecasts
- Behavioral biases and silo mentalities
Big-data-based
- A relatively new line of research
- Applies to both structured and unstructured data
Business performance focused
- Th
 e impact of better (or worse) forecasting
practices on the performance of the business
- Forecast performance assessment and
benchmarking
Statistical Research
For the most part, statistical research is ahead of
actual business practice. There appears to be a long
gap between the publication of research and its first
implementation in commercial software. I believe the
gap here is not only wide but growing wider as more
and more factors are brought into the forecasting
24 FORESIGHT Summer 2016
model, resulting in greater complexity and confusion
about the meaning of the results.
Very relevant, then, is the question of whether more
complex methods actually perform any better than
simpler alternatives. There is a body of evidence to
the contrary, including the very recent Foresight
article by Stephan Kolassa (2016), Sometimes Its
Better to Be Simple than Correct.
A proposed new method should demonstrably
improve upon existing (and simpler) alternatives,
not just on one specific sample but across a broader
range of data sets.
Behavioral Research
Similarly, behavioral research is ahead of the curve
compared to what is being used in the business world.
It has reached a critical mass whereby almost every
other book seems to be touching on how the human
brain works and how it affects the ways we do things.
With this awareness, I see much discussion and some
accommodation in the process of forecasting.
Big Data
Big-data-based research has come into its own in
the last decade. The essential idea is to incorporate
more detailed (and larger) data sets into forecasting
models. A newer variant involves the use of non-
structured data such as comments in tweets, blogs,
and sections of websites.
Business Performance
This fourth type of research is the most relevant to
our discussion. You would thinkand probably hope
that this type of research makes the connection
between theory and practice and points out the ben-
efits of good forecasting to the business user. It is in
this area that research can provide the most value to
the business user.
Forecasting deals with the future, and the future is
inherently uncertain. Forecasters with any reasonable
training and/or experience recognize that one cannot
codify uncertainty. As a result, no one really buys into
the idea that a particular forecast method is perfect.
The focus is put on reducing forecast error as much
as possible.
I think that traditionally this has been done by com-
paring forecast accuracy before and after a project
or across other comparative situations. Forecast-
accuracy improvements are fine as far as they go.
However, the business world is always interested in
the return on investment (ROI).
ROI is important for many reasons. During project
approval stages we need to calculate the anticipated
ROI in order to win managements approval for the
project. And its often the case that multiple projects
are vying to get access to money from the same pool.
Similarly, at the end of the project we need to show
the gains made and the percentage of them that can
be attributed to the forecast improvements.
THE GAPS
It is here that I see the biggest gap. The types of ques-
tions that I see practitioners asking include these:
What is a 10% improvement in forecast accuracy
worth to my business?
- At what level is this 10% accuracy improvement
taking place the overall business level or the
SKU-Customer level?
- At what time lag is this 10% accuracy improve-
ment? One month in advance or three months in
advance for companies with long lead times?
Is this the same if the starting forecast accuracy
is 20%? 40-60-80-90%? Is the impact easier to
achieve if the starting forecast accuracy is very
low?
What is the minimum/average/maximum improve-
ment one should expect using a commercial soft-
ware package relative to (a) what the organiza-
tion is doing currently and (b) a nave (no change)
model?

What is the minimum/average/maximum
improvement one should expect using bottom-up/
top-down/middle-out approaches?
Can forecast accuracy be correlated with a forecast-
ability score such as the Coefficient of Variation?
Theoretically, more-forecastable data should make
it easier to get to more accurate results. What are
the appropriate cutoffs of the said forecastability
scores to determine different levels of forecastabil-
ity? For each of these levels, what is an achievable
degree of accuracy in the forecast?
What is the expected improvement from collab-
orative input supplied by people in the field, such
as sales reps?
If I measure forecast accuracy at the top level of
my business, are there reliable ways to estimate
forecast accuracy at the lower levels of my forecast
hierarchy?
These questions elicit great interest from business
folks. Even when practitioners have no interest in
the details behind the science, they still need to know
the answers to these types of questions in order to
gain project approval from their management offices.
I see limited work along these lines coming out of
academia. On the other hand, a lot of the work in this
area is done by software vendors but can this work
be considered trustworthy? It is my hope that fore-
casting researchers will give more attention to these
practical aspects of forecasting research.
Sujit Singh, CFPIM, CSCP,
Chief Operating Officer of Arkieva
(www.arkieva.com), is respon-
sible for managing the delivery
of software and implementation
services as well as customer rela-
tionships and the day-to-day op-
erations of the corporation.
ssingh@arkieva.com
www.forecasters.org/foresight FORESIGHT 25
Copyright of Foresight: The International Journal of Applied Forecasting is the property of
International Institute of Forecasters and its content may not be copied or emailed to multiple
sites or posted to a listserv without the copyright holder's express written permission.
However, users may print, download, or email articles for individual use.