Posts Tagged ‘surveys of expectations’

Forecast Friday Topic: Expert Judgment

February 10, 2011

(Thirty-seventh in a series)

Last week, we began our discussion of judgmental forecasting methods, talking about judgmental extrapolation, which required no real understanding of the physical process behind the time series. Today, we will talk about more sophisticated judgmental techniques that are used in subjective forecasting; “sophisticated” only in the sense that the opinion of “experts” is used in trying to predict the future. The three techniques we will discuss are the Jury of Executive Opinion, sales force composite forecasts, and surveys of expectations.

Jury of Executive Opinion

The Jury of Executive Opinion is quite often seen in an organization’s budgeting and strategic planning process. The “jury” is often a group of high-level executives from all areas of the organization – marketing, finance, human resources, manufacturing, etc. – who come together to discuss their respective areas of business and work to come up with a composite forecast of where the organization’s business will be. Each executive shares his/her opinions and weighs and evaluates those of the other executive. After discussion, the executives write down their forecasts, which are then averaged.

One example of the Jury of Executive Opinion takes me back to 1999-2000, when I worked for catalog retailer Hammacher Schlemmer. Hammacher Schlemmer convened a weekly committee to estimate the orders coming in for the next two weeks from each of the active catalogs in circulation. The committee was made up of several marketing personnel, including myself (as I was the forecasting analyst!), and managers from the warehouse, in-bound call center, inventory control, and merchandising. We would begin every Wednesday morning reviewing the number of orders that came in for each active catalog, for the prior week and the first two days of the current week. Armed with that order information, and spreadsheets detailing order history for those catalogs’ prior years, each of us would indicate our orders forecasts for the next several weeks ahead. Our forecasts were then averaged, and we would then submit the composite forecasts to the warehouse and call center to assist with their staffing, and to inventory control to ensure adequate purchasing.

One of the nice things about the Jury of Executive Opinion is its simplicity. Getting executives to sound off is often pretty easy to do. Moreover, incorporating the experiences of a broad group into the forecasting process may enable companies to see the forest beyond the trees.

However, simple and broad-focused as it may be, the Jury of Executive Opinion is not without its flaws. These meetings can be time consuming, for one. Indeed, at Hammacher Schlemmer, during the last three months of the year – when the holiday season was in tow – those weekly meetings could take all morning, as nearly a dozen catalogs could be in circulation. Furthermore, group dynamics may actually lead to unwise consensus forecasts. The group is often at risk of being swayed by the opinions of those members who are most articulate, or with greater seniority or rank within the organization, or just by their own over-optimism. Another problem is that the passage of time makes it difficult to recognize those experts whose opinions were most reliable and whose should be given less weight. As a result, there’s no way to hold any individual member accountable for a forecast. Finally, executives are more comfortable with using their opinions for mid-and longer-range planning than for shorter period-to-period predictions, especially since recent unexpected events can also influence their opinion.

Sales Force Composite Forecasts

When companies have a product that is sold by sales agents in specific territories, it is not uncommon for them to seek the opinions of their sales representatives or branch/territory managers in developing forecasts for each product line. In fact, sales representatives’ opinions can be quite useful, since they are generally close to the customer, and may be able to provide useful insights into purchase intent. Essentially, these companies have their agents develop forecasts for each of the products they sell within a territory. The added benefit of this approach is that a company can develop a forecast for the entire market, as well as for individual territories.

Indeed, when I worked in the market research department of insurance company Bankers Life & Casualty during 1997-1999, we frequently conducted surveys of our sales force and branch managers to understand how many long-term care insurance policies, Medicare Supplement policies, and annuities were being sold within each market, and how much were being lost to the competition. These surveys would provide a read into the market size for each insurance product at both a regional and national level.

While the closeness to the customer is a great advantage of sales force composite surveys, they too have problems. Sales agents have a tendency to be overly optimistic in their forecasts and may set unrealistic goals. In addition, because sales agents are close to the customer, their opinions are likely to be swayed by microeconomic decision purchases, when in fact aggregate sales are often driven by macroeconomic factors. Supplementing sales force composite forecasts with more formal quantitative forecasting methods, if possible, is often recommended.

Surveys of Expectations

We actually covered surveys of expectations in our December 9, 2010 Forecast Friday post, but let me just quickly go through it. Sometimes when data isn’t available for forecasting, companies can conduct surveys to get opinions and expectations. Marketing research in this fashion is often expensive, so often surveys of expectations are used when it is believed they will provide valuable information. Surveys work well for new product development, brand awareness, and market penetration. In our December 9, 2010 Forecast Friday topic, the audience of the expectation survey was mostly executives and other business experts. In this post, the audience is consumers.

NCH Marketing Services, both the leading processor of grocery coupons and a leading coupon promotion firm – and also a former employer of mine – used surveys to obtain information on coupon usage. The company even asked persons how many coupons they took to the store in a typical month. From there, the company would estimate the number of coupons redeemed in the U.S. annually.


Companies often must rely solely on expert judgment for looking ahead. The Jury of Executive Opinion, sales force composite forecasts, and consumer surveys are just some of the approaches companies can take to predict the future when more formal quantitative methods are either unavailable or unreliable.

Next Forecast Friday Topic: The Delphi Method


Follow us on Facebook and Twitter!

For the latest insights on marketing research, predictive modeling, and forecasting, be sure to check out Analysights on Facebook and Twitter! “Like-ing” us on Facebook and following us on Twitter will allow you to stay informed of each new Insight Central post published, new information about analytics, discussions Analysights will be hosting, and other opportunities for feedback. So check us out on Facebook and Twitter!

Forecast Friday Topic: Leading Indicators and Surveys of Expectations

December 9, 2010

(Thirty-second in a series)

Most of the forecasting methods we have discussed so far deal with generating forecasts for a steady-state scenario. Yet the nature of the business cycle is such that there are long periods of growth, long periods of declines, and periods of plateau. Many managers and planners would love to know how to spot the moment when things are about to change for better or worse. Spotting these turning points can be difficult given standard forecasting procedures; yet being able to identify when business activity is going to enter a prolonged period of expansion or a protracted decline can greatly enhance managerial and organizational planning. Two of the most common ways managers anticipate turning points in a time series include leading economic indicators and surveys of expectations. This post discusses both.

Leading Economic Indicators

Nobody has a crystal ball. Yet, some time series exhibit patterns that foreshadow economic activity to come. Quite often, when activity turns positive in one time series, months later it triggers an appropriate response in the broader economy. When movements in a time series seem to anticipate coming economic activity, the time series is said to be a leading economic indicator. When a time series moves in tandem with economic activity, the time series is said to be a coincident economic
indicator; and when movements within a particular time series trails economic activity, the time series is said to be a lagging indicator. Economic indicators are nothing new. The ancient Phoenicians, whose empire was built on trading, often used the number of ships arriving in port as an indicator of trading and economic activity.

Economic indicators can be procyclic – that is they increase as economic activity increases and decrease when economic activity decreases; or countercyclic – meaning they decline when the economy is improving or increase when the economy is declining; or they can be acyclic, having little or no correlation at all with the broader economy. Acyclic indicators are rare, and usually are relegated to subsectors of the economy, to which they are either procyclic or countercyclic.

Since 1961, the U.S. Department of Commerce has published the Survey of Current Business, which details monthly changes in leading indicators. The Conference Board publishes a composite index of 10 leading economic indicators, whose activity suggests changes in economic activity six to nine months into the future. Those 10 components include (reprinted from

  1. the average weekly hours worked by manufacturing workers;
  2. the average number of initial applications for unemployment insurance;
  3. the amount of manufacturers’ new orders for consumer goods and materials;
  4. the speed of delivery of new merchandise to vendors from suppliers;
  5. the amount of new orders for capital goods unrelated to defense;
  6. the amount of new building permits for residential buildings;
  7. the S&P 500 stock index;
  8. the inflation-adjusted monetary supply (M2);
  9. the spread between long and short interest rates; and
  10. consumer sentiment


These indicators are used to measure changes in the broader economy. Each industry or organization may have its own indicators of business activity. For your business, the choice of the time series(‘) to use as leading indicators and the weight they receive depend on several factors, including:

  1. How well it tends to lead activity in your firm and industry;
  2. How easy the time series is to measure accurately;
  3. How well it conforms to the business cycle;
  4. The time series’ overall performance, not just turning points;
  5. Smoothness – no random blips that give misleading economic cues; and
  6. Availability of data.

Over time, the use of specific indicators, and their significance in forecasting do in fact change. You need to keep an eye on how well the indicators you select continue to foreshadow business activity in your industry.

Surveys of Expectations

Sometimes time series are not available for economic indicators. Changes in technology social structure may not be readily picked up in the existing time series. Other times, consumer sentiment isn’t totally represented in the economic indicators. As a result, surveys are used to measure business optimism, or expectations of the future. Economists and business leaders are often surveyed for their opinions. Sometimes, it’s helpful to know if business leaders anticipate spending more money on equipment purchases in the coming year; whether they plan to hire or lay off workers; or whether they intend to expand. While what respondents to these surveys say and what they really do can be quite different, overall, the surveys can provide some direction as to which way the economy is heading.

Next Forecast Friday Topic: Calendar Effects in Forecasting

Easter can fall in March or April; every four years, February has an extra day; in some years, months have four weekends; others years, five. These nuances can generate huge forecast errors. Next week’s Forecast Friday post discusses these calendar effects in forecasting and what you can do to adjust for them.


Be Sure to Follow us on Facebook and Twitter !

Thanks to all of you, Analysights now has nearly 200 fans on Facebook … and we’d love more! If you like Forecast Friday – or any of our other posts – then we want you to “Like” us on Facebook! And if you like us that much, please also pass these posts on to your friends who like forecasting and invite them to “Like” Analysights! By “Like-ing” us on Facebook, you and they will be informed every time a new blog post has been published, or when new information comes out. Check out our Facebook page! You can also follow us on Twitter. Thanks for your help!