Posts Tagged ‘problem definition’

Big Data Success Starts With Well-Defined Business Problem

April 18, 2014

(This post also appears on our successor blog, The Analysights Data Mine).

Lots of companies are jumping on the “Big Data” bandwagon; few of them, however, have given real thought to how they will use their data or what they want to achieve with the knowledge the data will give them.  Before reaping the benefits of data mining, companies need to decide what is really important to them.  In order to mine data for actionable insights, technical and business people within the organization need to discuss the business’ needs.

Data mining efforts and processes will vary, depending on a company’s priorities.  A company will use data very differently if its aim is to acquire new customers than if it wants to sell new products to existing customers, or find ways to reduce the cost of servicing customers.  Problem definition puts those priorities in focus.

Problem definition isn’t just about identifying the company’s priorities, however.  In order to help the business achieve its goals, analysts must understand the constraints (e.g., internal privacy policies, regulations, etc.) under which the company operates, whether the necessary data is available, whether data mining is even necessary to solve the problem, the audience at whom data mining is directed, and the experience and intuition of the business and technical sides.

What Does The Company Want to Solve?

Banks, cell phone companies, cable companies, and casinos collect lots of information on their customers.  But their data is of little value if they don’t know what they want to do with it.  In the banking industry, where acquiring new customers often means luring them away from another bank,  a bank’s objective might be to cross-sell, or get its current depositors and borrowers to acquire more of – its products, so that they will be less inclined to leave the bank.  If that’s the case, then the bank’s data mining effort will involve looking at the products its current customers have and the order and manner in which the customer acquired those products.

On the other hand, if the bank’s objective is to identify which customers are at risk of leaving, its data mining effort will examine the activity of departing households in the months leading up to their defection, and compare it to those households it retained.

If a casino’s goal is to decide on what new slot machines to install, its data mining effort will look at the slot machine themes its top patrons play most and use that in its choice of new slot machines.

Who is the Audience the Company is Targeting?

Ok, so the bank wants to prevent customers from leaving.  But do they want to prevent all customers from leaving?  Usually, only a small percentage of households account for all of a bank’s profit; many banking customers are actually unprofitable.  If the bank wants to retain its most profitable customers, it needs only analyze that subgroup of its customer base.  The bank’s predictions of its premier customers’ likelihood to leave based on a model developed on all its customers would be highly inaccurate.  In this case, the bank would need to build a model only on its most profitable customers.

Does the Problem Require Data Mining?

Data mining isn’t always needed.  Years ago, when I was working for a catalog company, I developed regression models to predict which customers were likely to order from a particular catalog.  When a model was requested for the company’s holiday catalog, I was told that it would go to 85 percent of the customer list.  When such a large proportion of the customer base – or the entire customer base for that matter – is to receive communication, then a model is not necessary.  More intuitive methods would have sufficed.

Is Data Available?

Before a data mining effort can be undertaken, the data necessary to solve the business problem must be available or obtainable.  If a bank wants to know the next best product to recommend to its existing customers, it needs to know the first product these customers acquired, how they acquired it, the length of time between their acquisition of their second product, then their third product, and so forth. The bank also needs to understand what products its customers acquired simultaneously (such as a checking account and a credit card), current activity with those products, and the sequence of product acquisition (e.g., checking account first, savings account second, certificate of deposit third, etc.).

It is extremely important that analysts consult both those on the business side and the IT department about the availability of data.  These internal experts often know what data is collected on customers, where it resides, and how it is stored.  In many cases, these experts may have access to data that doesn’t make it into the enterprise’s data warehouse.  And they may know what certain esoteric values for fields in the data warehouse mean.  Consulting these experts can save analysts a lot of time in understanding the data.

Under What Constraints Does the Business Operate?

Companies have internal policies regulating how their operation; are subject to regulations and laws governing the industries and localities in which they operate; and also are bound by ethical standards in those industries and locations.

Often, a company has access to data that, if used in making business decisions, can be illegal or viewed as unethical.  The company doesn’t acquire this data illegally; the data just cannot be used for certain business practices.

For example, I was building customer acquisition models for a bank a few years ago.  The bank’s data warehouse had access to summarized credit score statistics by block groups, as defined by the U.S. Bureau of the Census.  However, banks are subject to the Community Reinvestment Act (CRA), a 1977 law that was passed to prevent banks from excluding low- to moderate-income neighborhoods in their communities from lending decisions.  Obviously, credit scores are going to be lower in lower-income areas. Hence, under CRA guidelines, I could not use the summarized credit statistics to build a model for lending products.  I could, however, use those statistics for a model for deposit products; for post campaign analysis, to see which types of customers responded to the campaign; and also to demonstrate compliance with the CRA.

In addition, the bank’s internal policies did not allow the use of marital status in promoting products.  Hence, when using demographic data that the bank purchased, I had to ignore the field, “married” when building my model.  In cases like these, less direct approaches can be used.  The purchased data also contained a field called “number of adults (in the household).  This was totally appropriate to use, since it did not necessarily mean that a household with two adults was a married-couple household.

Again, the analyst must consult the company’s business experts so it can understand these operational constraints.

Are the Business Experts’ Opinions and Intuition Spot-On?

It’s often said that novices make mistakes out of ignorance and veterans make mistakes out of arrogance.  The business experts have a lot of experience in the company and a great deal of intuition, which can be very insightful.  However, they can be wrong too.  With every data mining effort, the data must be allowed to tell the story.  Does the data validate what the experts say?  For example, most checking accounts are automatically bundled with a debit card; a bank’s business experts know this; and the analysis will often bear this out.

However, if the business experts say that a typical progression in a customer’s banking relationship starts with demand deposit accounts (e.g., checking accounts) then consumer lending products (e.g., auto and personal loans), followed by time deposits (e.g., savings accounts and certificates of deposit), does the analysis confirm that?

 

Problem definition is the hardest, trickiest, yet most important, prerequisite to getting the most out of “Big Data.”  By knowing what the business needs to solve, analysts must also consider the audience the data mining effort is targeting; whether data mining is necessary; the availability of data and the conditions under which it may be used; and the experience of the business experts.  Effective problem definition begets data mining efforts that produce insights a company can act upon.

Advertisements

A Typical-Length Survey or A Few Shorter Ones?

June 28, 2010

Most online surveys today take between 10 and 15 minutes, with a few going as long as 25 to 30 minutes. As marketing researchers, we have long pontificated that surveys should be a reasonable length, as longer ones tend to cause respondents to disengage in many ways: speeding through, skipping questions, even abandoning the survey. Most marketers realize this, and the 10-15 minute survey seems to be the norm. But I wonder how many marketing researchers – on both the client and supplier side – have ever considered the length of a survey from a strategic, rather than a tactical, point of view.

Sure, a typical-length survey is not super long, and is often cost effective for a client. After all, the client can survey several people about several topics in a relatively short time, for a set price, and can get results quickly. But sometimes I believe that instead of one 15-minute survey, some clients might benefit more by conducting two 7- or 8-minute, or three 5-minute surveys, stretched out over time. Marketing researchers on both sides will likely disagree with me here. After all, multiple shorter surveys can cost more to administer. However, I believe that – in the long-run – clients will derive value from the more frequent, shorter surveys that would offset their cost. Multiple, shorter surveys will benefit clients in the following ways:

Focus

As marketing research suppliers, it is our job to make sure we understand the client’s key business problem. Many times, clients have several problems that must be addressed. We need to help clients look at all of their business problems and prioritize them in the order of benefit that their resolution would bring. If we could get the client’s survey focused on the one or two problems whose resolution would result in the most positive difference, we can keep the survey short, with more targeted questions. As a result, the client doesn’t get bombarded with tons of data tables or reports with lots of recommendations and end up immobilized wondering which ones should be implemented first. On the contrary, the client will receive a few, very direct, insights about how to respond to these key problems.

Reduced Incentive Costs

Since surveys are shorter, respondents may be willing to do them for little or no incentive. This can save the client money.

Higher Response Rates

Surveys that are 10-15 minutes long generally get decent response rates. However, a survey that’s 3, 5, or 7 minutes long will likely get excellent response rates. Why? Because they’re more convenient, straight to the point, and can be knocked off quickly. As a result, respondents are less willing to put it off. Respondents are also less likely to terminate the survey, speed through it, or skip questions.

Increased Trust by Respondents

Because you didn’t waste their time with the first survey, respondents may be more inclined to participate in your subsequent surveys. If they took your 5-minute survey today, then you send them another 5-minute survey four to six weeks from now, they are likely to trust that this survey won’t take long either, and will likely respond to it. Of course, the key here is to space the surveys out. You don’t want to send all three at once!

More Reliable Data

As mentioned above, respondents are less likely to speed, terminate, or skip questions to a short survey than they are with a longer one. As a result, there will be less non-response error and more truthful responses in the data, and hence more trustworthy findings.

Ability to Act on Results Faster

Because the survey is short and to-the-point, and response rates are higher, the client can achieve the desired number of completed surveys sooner than if the survey were longer, so the survey doesn’t have to be in the field as long. And because the survey is short, the time the marketing research firm needs to tabulate and analyze the data is much shorter. Hence the client can start acting on the insights and implementing the recommendations much sooner.

Discovery

What would happen if a client conducted a typical-length survey and found a theme emerging in open-ended questions or a trend in responses among a certain demographic group? The client may want to study that. But custom research is expensive. If the client did a typical-length survey, the budget may not be there to do another survey to investigate that newly discovered theme or trend. With a shorter survey, the cost may be somewhat lower, so funds might be left in the budget for another survey. In addition, if the client is scheduling subsequent shorter surveys, the learnings from the first survey can be used to shape questions for further investigation in those upcoming surveys.

The Shorter Survey May Be Enough

Several times, problems are interconnected, or generated by other problems. If research suppliers helped clients isolate their one or two biggest problems, and focused on those, the client might act on the insights and eliminate those problems. The resolution of those problems may also provide solutions to, or help extinguish, the lesser-priority problems. As a result, future surveys may not be needed. In that case, the research supplier did its job – solving the client’s problem in the shortest, most economical, and most effective manner possible.

Granted, many clients probably can’t do things this way. There are economies of scale in doing one longer survey as opposed to two or three shorter ones. Moreover, the client probably has several stakeholders, each of whom has a different opinion of which problem is most important. And each problem may have a different urgency to those stakeholders. This is why it is so important for the research supplier to get the client’s stakeholders and top management on board with this. As research suppliers, it is our job to inform and educate the client and its stakeholders on the research approach that maximizes the best interest of the client as a whole; and if that is not possible, work with those stakeholders to identify second-best solutions. But once the key issues – problems, budget, politics, and urgency – are on the table, research suppliers can work with the client to develop the shortest, most focused, most cost effective survey possible.

Free Online Survey Tools Can Yield Costly Useless Results if not Used Carefully

June 15, 2010

Thanks to online survey tools like Zoomerang, Surveymonkey, and SurveyPirate, the ability to conduct surveys has been greatly democratized. Small businesses, non-profits, and departments within larger firms can now conduct surveys that they would never have been able to do because of cost and lack of resources. Unfortunately, the greatest drawback of these free survey tools is the same as their greatest benefit: anyone can launch a survey. Launching an effective survey requires a clear definition of the business problem at hand; a carefully thought out discussion of the information needed to address the business problem, the audience of the survey, and how to reach it; determination of the sample size and how to select them; designing, testing, and implementing the questionnaire; and analyzing the results. Free online survey tools do not change this process.

Recently, a business owner from one of my networking groups sent me an online survey that he designed with one of these free tools. It was a questionnaire about children’s toys – which was the business he was in. He wasn’t sending me the survey to look at and give advice; he sent it to me as if I were a prospective customer. Unfortunately, I’m not married and don’t have kids; and all my nieces and nephews are past the age of toys. The survey was irrelevant to me. The toy purveyor needed to think about who his likely buyers were – and he should have good knowledge, based on his past sales, of who his typical buyers are. Then he could have purchased a list of people to whom he could send the survey. Even if that meant using a mail or phone survey, which could be costly, the owner could get more meaningful results. Imagine how many other irrelevant or uninterested recipients received the business owner’s survey. Most probably didn’t respond; but others might have responded untruthfully, giving the owner bogus results.

Also, the “toy-preneur’s” survey questions were poorly designed. A double-barreled question: “Does your child like educational or action toys?” What if a respondent’s child liked both educational and action toys? The owner should have asked two separate questions: “Does your child like educational toys?” and “Does your child like action toys?” Or he could have asked a multi-part question like, “Check the box next to each of the types of toys your child likes to play with,” followed with a list of the different types of toys.

The survey gets worse… questions like: “How much does your child’s happiness mean to you?” How many people are going to answer that question negatively? Hello? Another asking the respondent to rank-order various features of a toy for which there was no prototype pictured, and if that wasn’t bad enough, there were at least 9 items to rank? Most people can’t rank more than five items, especially not for an object they cannot visualize.

We also don’t know how the toy manufacturer selected his sample. My guess was that he sent it to everyone whose business card he collected. Hence, most of the people he was surveying were the wrong people. In addition to getting unacceptable results, another danger of these online survey tools is that people are more frequently bombarded with surveys that they stop participating in surveys altogether. Imagine if you were to receive five or more of these surveys in less than two weeks. How much time are you willing to give to answering these surveys? Then when a truly legitimate survey comes up, how likely are you to participate?

I think it’s great that most companies now have the ability to conduct surveys on the cheap. However, the savings can be greatly offset by the uselessness of the results if the survey is designed poorly or sent to the wrong sample. There is nothing wrong with reading up on how to do a survey and then executing it, as described, as long as the problem is well-defined, the relevant population is identified, and the sampling, execution, and analysis plans are in place. “Free” surveying isn’t good if it costs you money and time in rework and/or in faulty actions taken based on your findings.

Do you have trouble deciding whether you need to do a survey? Do you spend a lot of time trying to find out what you’re trying to learn from a survey? Or how many people to survey? Or the questions you need to ask? Or which people to survey? Let Analysights help. We have nearly 20 years of survey research experience and a strong background in data analysis. We can help you determine whether a survey is the best approach for your research needs, the best questions to ask to get the information you need, and help you understand what the findings mean. Feel free to call us at (847) 895-2565.

Conducting Market Research Without an Underlying Business Purpose Wastes Time and Money

April 21, 2010

The other day, a question was directed to me on the AllExperts.com Website, where I am a participating market research expert.  A student from the Netherlands was preparing a graduation thesis based on a large scale market research project he was doing for a manufacturer of water cooling systems for power plants.

His thesis topic was, “What does the northwest European market for cooling water systems in power plants look like?” 

While the manufacturer was good with the topic, his thesis advisor was not.  The advisor felt the topic wasn’t strategic.  And the advisor was right!

Market research has no value if a company can’t act on it.  Market research findings need to be taken to the next step – a recommended course of action for the client.  If the power plant water cooling system manufacturer knows what the market looks like, that’s great, but what does it do with that knowledge?  Many companies, for this very reason, end up suffering from “analysis paralysis.”

So I advised the student to reformulate the thesis topic into a strategic initiative, something like, “How can Company A achieve an X% share of the power plant water cooling systems market in northwest Europe?”  From there, he would still answer his research questions to understand what the market looked like, but then the findings would lead him to determine the strategy he should recommend to the manufacturer. (See his question and my response here).

Knowing what initiative the client wants to achieve is the starting point.  Once you know the business purpose, you can then determine the research questions that must be answered.  Once those questions are answered, you can then come up with the recommendations that lead to achievement of the business purpose.  Insights that can’t be acted on waste everyone’s time and money.

When NOT to do Marketing Research

October 2, 2009

Marketing research is an important part of a company’s decision making process.  However, there are times to do marketing research and times not to.  When marketing research keeps you on top of the markets in which your company operates; helps you achieve a strategic marketing advantage; enables you to select the course of action that achieves your key marketing objectives; or clarifies problems or investigates marketplace trends that affect your marketing goals, you should, by all means, conduct it.

However, there are certain times when you should NOT conduct marketing research.  First and foremost, you should not do marketing research if you have not first defined the problem you need to solve.  Problem definition is the single most important step in the marketing research process.  If not done – or done correctly – any research performed will be useless.  Granted, sometimes companies have no idea what the marketing problem is, so they must then do exploratory research, to help them identify the problem.  In that instance, there is a business problem, and that is to determine what is causing the company’s current marketing situation.

You probably also don’t need marketing research if:

You have access to readily available marketing information     

Your sales force may know its territories very well and each sales representative may understand the environment in which he or she calls on.  They may know the price of the competition’s products in those markets, as well as the relevant competitors there, and how much it costs to acquire customers there.  In addition, the Internet has made all kinds of marketing information freely available, and data sources like Dun & Bradstreet’s Million Dollar Database or ABI’s ReferenceUSA  has made finding information about prospective competition and customers a snap.  As a result, secondary research may be all you need to do to find the solutions to your marketing problems.

There’s not enough time or resources to conduct marketing research

If time is an issue, conducting elaborate marketing research will do no good.  Sometimes a situation arises where a decision must be made quickly.  In such a case, you might be better off convening the company’s business experts for an urgent discussion of the situation, alternative courses of action, and the selection of the course to take.  In other instances, you may lack the financial resources, or the internal staff for proper marketing research.  In these cases, you may also rely on the business experts and the secondary research already available to you.

The research adds little or no value

If the decision you want marketing research to help you make has little impact on sales, profit, market share, customer loyalty, brand equity, or any other marketing performance indicator, then it makes no sense to do marketing research.  Marketing research can be costly both in terms of time and money, so if the benefit of the research doesn’t at least pay for itself in the dollars and manpower expended to conduct it, it’s worthless.  You also need to consider the opportunity costs of that research.  If you do research on a problem whose solution adds little value, the time and money could have been better used to research a different problem with a bigger payoff, and that opportunity is lost.

Knowing when you need to do marketing research 

Develop an internal monitoring system of your marketing environment.  If you have a system in place to compile information about your company and your competition, it will alert top management to problems that marketing research can attack.  These days, you can set up e-mail alerts with Google and many major newspapers to keep you informed of any news or blog posts about your company, your competition, and your industry.   Also read your industry’s trade publications and get out to trade shows and conferences.  Talk to your sales force, your suppliers, and your customers.  You can get a wealth of information for free from these sources.

Knowing when not to do marketing research is just as important as knowing when to do it.  When marketing research adds significant value or improves your competitive position, it’s a go; when marketing research is just “nice to know,” it’s a no!