Big Data Success Starts With Well-Defined Business Problem

April 18, 2014

(This post also appears on our successor blog, The Analysights Data Mine).

Lots of companies are jumping on the “Big Data” bandwagon; few of them, however, have given real thought to how they will use their data or what they want to achieve with the knowledge the data will give them.  Before reaping the benefits of data mining, companies need to decide what is really important to them.  In order to mine data for actionable insights, technical and business people within the organization need to discuss the business’ needs.

Data mining efforts and processes will vary, depending on a company’s priorities.  A company will use data very differently if its aim is to acquire new customers than if it wants to sell new products to existing customers, or find ways to reduce the cost of servicing customers.  Problem definition puts those priorities in focus.

Problem definition isn’t just about identifying the company’s priorities, however.  In order to help the business achieve its goals, analysts must understand the constraints (e.g., internal privacy policies, regulations, etc.) under which the company operates, whether the necessary data is available, whether data mining is even necessary to solve the problem, the audience at whom data mining is directed, and the experience and intuition of the business and technical sides.

What Does The Company Want to Solve?

Banks, cell phone companies, cable companies, and casinos collect lots of information on their customers.  But their data is of little value if they don’t know what they want to do with it.  In the banking industry, where acquiring new customers often means luring them away from another bank,  a bank’s objective might be to cross-sell, or get its current depositors and borrowers to acquire more of – its products, so that they will be less inclined to leave the bank.  If that’s the case, then the bank’s data mining effort will involve looking at the products its current customers have and the order and manner in which the customer acquired those products.

On the other hand, if the bank’s objective is to identify which customers are at risk of leaving, its data mining effort will examine the activity of departing households in the months leading up to their defection, and compare it to those households it retained.

If a casino’s goal is to decide on what new slot machines to install, its data mining effort will look at the slot machine themes its top patrons play most and use that in its choice of new slot machines.

Who is the Audience the Company is Targeting?

Ok, so the bank wants to prevent customers from leaving.  But do they want to prevent all customers from leaving?  Usually, only a small percentage of households account for all of a bank’s profit; many banking customers are actually unprofitable.  If the bank wants to retain its most profitable customers, it needs only analyze that subgroup of its customer base.  The bank’s predictions of its premier customers’ likelihood to leave based on a model developed on all its customers would be highly inaccurate.  In this case, the bank would need to build a model only on its most profitable customers.

Does the Problem Require Data Mining?

Data mining isn’t always needed.  Years ago, when I was working for a catalog company, I developed regression models to predict which customers were likely to order from a particular catalog.  When a model was requested for the company’s holiday catalog, I was told that it would go to 85 percent of the customer list.  When such a large proportion of the customer base – or the entire customer base for that matter – is to receive communication, then a model is not necessary.  More intuitive methods would have sufficed.

Is Data Available?

Before a data mining effort can be undertaken, the data necessary to solve the business problem must be available or obtainable.  If a bank wants to know the next best product to recommend to its existing customers, it needs to know the first product these customers acquired, how they acquired it, the length of time between their acquisition of their second product, then their third product, and so forth. The bank also needs to understand what products its customers acquired simultaneously (such as a checking account and a credit card), current activity with those products, and the sequence of product acquisition (e.g., checking account first, savings account second, certificate of deposit third, etc.).

It is extremely important that analysts consult both those on the business side and the IT department about the availability of data.  These internal experts often know what data is collected on customers, where it resides, and how it is stored.  In many cases, these experts may have access to data that doesn’t make it into the enterprise’s data warehouse.  And they may know what certain esoteric values for fields in the data warehouse mean.  Consulting these experts can save analysts a lot of time in understanding the data.

Under What Constraints Does the Business Operate?

Companies have internal policies regulating how their operation; are subject to regulations and laws governing the industries and localities in which they operate; and also are bound by ethical standards in those industries and locations.

Often, a company has access to data that, if used in making business decisions, can be illegal or viewed as unethical.  The company doesn’t acquire this data illegally; the data just cannot be used for certain business practices.

For example, I was building customer acquisition models for a bank a few years ago.  The bank’s data warehouse had access to summarized credit score statistics by block groups, as defined by the U.S. Bureau of the Census.  However, banks are subject to the Community Reinvestment Act (CRA), a 1977 law that was passed to prevent banks from excluding low- to moderate-income neighborhoods in their communities from lending decisions.  Obviously, credit scores are going to be lower in lower-income areas. Hence, under CRA guidelines, I could not use the summarized credit statistics to build a model for lending products.  I could, however, use those statistics for a model for deposit products; for post campaign analysis, to see which types of customers responded to the campaign; and also to demonstrate compliance with the CRA.

In addition, the bank’s internal policies did not allow the use of marital status in promoting products.  Hence, when using demographic data that the bank purchased, I had to ignore the field, “married” when building my model.  In cases like these, less direct approaches can be used.  The purchased data also contained a field called “number of adults (in the household).  This was totally appropriate to use, since it did not necessarily mean that a household with two adults was a married-couple household.

Again, the analyst must consult the company’s business experts so it can understand these operational constraints.

Are the Business Experts’ Opinions and Intuition Spot-On?

It’s often said that novices make mistakes out of ignorance and veterans make mistakes out of arrogance.  The business experts have a lot of experience in the company and a great deal of intuition, which can be very insightful.  However, they can be wrong too.  With every data mining effort, the data must be allowed to tell the story.  Does the data validate what the experts say?  For example, most checking accounts are automatically bundled with a debit card; a bank’s business experts know this; and the analysis will often bear this out.

However, if the business experts say that a typical progression in a customer’s banking relationship starts with demand deposit accounts (e.g., checking accounts) then consumer lending products (e.g., auto and personal loans), followed by time deposits (e.g., savings accounts and certificates of deposit), does the analysis confirm that?

 

Problem definition is the hardest, trickiest, yet most important, prerequisite to getting the most out of “Big Data.”  By knowing what the business needs to solve, analysts must also consider the audience the data mining effort is targeting; whether data mining is necessary; the availability of data and the conditions under which it may be used; and the experience of the business experts.  Effective problem definition begets data mining efforts that produce insights a company can act upon.

Advertisements

Mistakes Employers Make When Recruiting Analytics Talent

April 16, 2014

Check out our sister blog, the Analysights Data Mine to discover how employers are shooting themselves in the foot when trying to recruit analytics professionals.

Check out the Analysights Data Mine

March 26, 2014

As many of you have seen, the last post on Insight Central was in April 2011. I just wanted to take this moment and let you know that I have christened a brand new blog this morning, the Analysights Data Mine, which discusses trends and developments in the field of “Big Data.” If you love Insight Central, you’ll also love the Data Mine. Insight Central will remain up for your enjoyment, although I have no further plans to post on it. Thanks for your visits and loyalty to Insight Central; I look forward to seeing you again on the Data Mine.

Forecast Friday Topic: Evaluation of Forecasts

April 14, 2011

(Last in the series)

We have finally come to the end of our almost year-long Forecast Friday journey. During this period, we have discussed various forecasting methods, including regression analysis, exponential smoothing, moving average methods, the basics of both ARIMA and logistic regression models. We also discussed qualitative, or judgmental, forecasting methods; we discussed how to diagnose your regression models for violations such as multicollinearity, autocorrelation, heteroscedasticity, and specification bias; and we discussed a series of other topics in forecasting, like the identification problem, leading economic indicators, calendar effects in forecasting, and the combination of forecasts. Now, we move on to the last part of the forecasting process: evaluating forecasts.

How well does your forecast model perform? That question should be the crux of your evaluation. This criterion relates to your company’s bottom line. You need to consider the costs to your company of forecasting too high and of forecasting too low. If you own a toy store and your sales forecasts for some stock-keeping units (SKUs) is too high, you risk marking down those items on clearance. On the other hand, if your forecast is too low, you risk running out of stock. Which type of mistake is more costly to your company? How much error in each direction can you tolerate, affordably? These are questions you must consider.

Your models are useless if you don’t track how well they perform. Any time you generate a forecast, your model will not only give you a point forecast, but also a prediction interval associated with a given level of confidence. The point forecast is the midpoint of that prediction interval. Each time you generate a forecast, record the actual results. Did actuals fall within the prediction interval? If so, how close to the point forecast did they fall? If not, how far off were you?

As you keep track forecasts vs. actuals over time, determine how often your actuals fall within our outside your prediction intervals, and how close to the point forecast they are. If your forecasts are frequently far from your point estimate, especially near the upper or lower bounds of your prediction interval, that’s likely a sign that your model needs to be reworked. Indeed, model performance degrades over time. Technological advances, societal changes, changes in tastes, styles, and preferences, and random events can promote forecast error, because forecasting models are based on past data and assume that the future will continue to resemble the past.

Forecasting is as much an art as it is a science. And I hasten to add that the ability to forecast is like a muscle – you need to exercise it in order to strengthen it. Forecasts are never consistently perfect, but they can be frequently excellent. Don’t look to become a forecasting “guru.” It doesn’t last. Allow yourself to learn new things from every forecasting process you go through and each forecast evaluation you perform. And if you do that, becoming a great forecaster is in your forecast! And I can’t think of a better note on which to end the Forecast Friday series.

**********

Tell us what you thought of the Forecast Friday series!

We’ve been on a long road with Forecast Friday. I began the series last year because I believed that forecasting is an art that every business entity, or marketing, finance, production (etc.) professional could use to go far. Many of you have been tuning in to Forecast Friday each Thursday, so I would appreciate your honest feedback. Please leave comments. Let me know the topic(s) you found most helpful or useful. What could I have done better? What topic(s) should I have covered? Please don’t hold back. The purpose of Insight Central and Forecast Friday is to help you use analytics to advance your business and/or career.

Forecast Friday Will Resume April 14

April 6, 2011

I’ve been on assignment, and haven’t been able to devote time to writing this week’s Forecast Friday post, part one of “Evaluating Forecasts.”  So, what I will do is, next week,  write a complete post on the topic, and conclude the Forecast Friday series next week as planned.

Thanks for your patience and understanding, and for your continued interest in the Forecast Friday series.

Alex