Archive for the ‘marketing’ Category

The Man Who Feared Analytics

June 9, 2010

A business owner had once been referred to me by a colleague with whom he had already been doing business. For many years, the businessman’s photography business had been sustained through direct mail advertising, and he often received a 5%-7% response rate, an accomplishment that would boggle most direct marketers. But the recent economic downturn combined with photography’s being a discretionary expense, he soon found his direct mail solicitations bringing in a puny 0.8% response rate. The business owner had a great product, a great price, and a great offer, but at that response rate, he was no longer breaking even.

My colleague and I spoke with the businessman about his dilemma. We talked through his business; we looked at his most recent mailer, learned how he obtained his mailing lists, and discussed his promotion schedule. We found that the photographer would buy a list of names, mail them once, and then use a different list, not giving people enough opportunity to develop awareness of his business. We also found that he didn’t have much information about the people he was mailing.

We recommended that analytics could help the photographer maximize his margin by improving both the top and bottom line. Analytics would first help him understand which customers were responding to his mailings. Then he could purchase lists of people with characteristics similar to those past respondents. His response rate would go up, since he would be sending to a list of people most receptive to his photography. He would also be able to mail fewer people, cutting out those with little likelihood of response. He could then use the savings to remail the members of his target segments who hadn’t responded to his earlier mailing, and thus increase their awareness. It all sounded good to the photographer.

And then, he decided he was going to wait to see if things got better!

Why the Fear of Analytics?

The photographer’s decision is a common refrain of marketers. Marketers and business owners who are introduced to analytics are like riders on a roller coaster: thrilled and nervous at the same time. While marketers are excited about the benefits of analytics, they are also concerned about its cost; they’re afraid of change; and they’re intimidated by the perceived complexity of analytics. We’ll tackle each of these fears here.

FEAR #1: Analytics could be expensive.

REALITY: Analytics is an investment that pays for itself.

The cost of analytics can appear staggering, especially in lean times. Some of the most sophisticated analytics techniques can run into tens – if not hundreds – of thousands of dollars for a large corporation. However, for many smaller companies, analytics can run a few thousand dollars, but still a lot of money. But analytics is not an expense; you are getting something great in return: the insights you need to make better informed marketing decisions and identify the areas in your marketing that you can improve or enhance; the ability to target customers and prospects more effectively, resulting in increased sales and reduced costs; and the chance to establish long-term continuous improvement systems.

Had the photographer gone through with the analytics for his upcoming mail, the entire analysis would have cost him somewhere between $1,300 and $1,800. But that fee would have enabled him to identify where his mailings were getting the greatest bang for his buck and he might have made up for it in reduced mailing costs and increased revenues. Once the analytics had saved or made the photographer at least $1,800, it would have paid for itself.

FEAR #2: Analytics means a change in the way we do things.

REALITY: Analytics brings about change gradually and seamlessly.
The photographer had been using direct mail over and over again, because it worked over and over again – until recently. In fact, having lost so much money on his recent direct mails, he’s probably leery of new approaches, so he stays the course out of familiarity. That’s quite common. But this is the nice part about analytics: change can be gradual! Analytics is about testing the waters, so to reduce risk. Perhaps the photographer could have done a test where half of his mailings were executed the traditional way, and half done the way the analytics recommended. Over the course of a short period, the photographer could then decide for himself what approach was working best.

FEAR #3: Analytics is “over my head.”
REALITY: You need only understand a few high level concepts.

Those complicated and busy mathematical formulas, in all their Greek symbol glory, can be intimidating to people who are not mathematicians, statisticians, or economists. In fact, even I get intimidated from those equations. We must remember, however, that these formulas were developed to improve the way we do things! With analytics, all you need to know is what approach was employed, what it does, why it’s important, and how to apply it – all of which are very simple. Analysts like me deal with all the complicated stuff – finding the approach, employing it, debugging it, refining it, and then packaging it in a way that you can apply seamlessly. And if you don’t understand something about the analytical approach employed, by all means, ask! And any good analyst will give you all the guidance you need until you’re able to apply the analytics on your own.

Forgoing Analytics Can Cost Your Business Three Times Over!

Analytics is one of those tools that many marketers know can enhance their businesses, yet decide to hold off on – either for cost, perceived complexity, or just plain fear. This inaction can be very dangerous. Analytics is not just a tool that improves your business decision making; it also helps you diagnose problems, identify opportunities, and make predictions about the future. Failure to do these properly costs you in three ways. First, you market incorrectly, wasting money. Second, you market to the wrong people; they don’t buy, and you lose revenue you could have made marketing to the right people. Third, you fail to recognize opportunities, and you forgo any sales those missed opportunities may have brought. Analytics is an investment that pays for itself, pays dividends down the road, brings about change in an easy and acceptable way, and whose benefits are easy to grasp and financially rewarding.

Advertisements

No Money to Conduct Primary Research? You May Have Done a Lot of it Already!

June 8, 2010

Last week, I wrote about an entrepreneur who was conducting secondary marketing research so he could develop his business plan. This week, I am writing to talk about primary research – data your company generates on its own. Often, we think of surveys and focus groups when we hear “primary research.” And those methods can indeed be costly. However, your business is probably generating volumes of primary data right under your nose. You’re out to hear the voice of your customers and prospects when you do primary research, and primary data is coming to you at nearly every touch point you have with them. Think of these sources:

Customer Service Calls

When customers call for customer service, or prospects call for information, what are the most common things they ask about? If your business sells handbags, which ones are frequently inquired about? Are the handbags most inquired about those that are higher or lower priced? Are they new handbags you’ve introduced? Are they mostly imported handbags? Also, who is making the inquiries? Are they long-term customers? Prospects? If long-term customers are inquiring about one line of handbags and prospects about another, you can tailor your marketing messages to their interests.

Customer Complaints

Nobody likes to be on the receiving end of a complaint. But complaints can be a great source of information. They can alert you to product defects, service breakdowns, and even give you ideas for enhancing your product or service. They can even help you save a long-term relationship and avoid bad word-of-mouth press. If women are complaining that the strap on one of the handbags you sell is uncomfortable to hang over their shoulders, that can prompt you to look for alternatives, or contact the supplier with that information. If a customer complains about the treatment an employee gave him/her, you might use that as an opportunity to either train your staff on improved customer service or discipline that employee.

It’s often said that 96% of a business’ dissatisfied customers will not complain; 91% will quietly go away; and those silent dissatisfied customers will likely communicate their dissatisfaction to at least nine other people. Encourage your customers to speak up when they’re not happy. Complaints can be a rich source of research.

Your Salespeople

Your salespeople are out in the field. They see everything at the frontlines. What successes are they having? What gripes do they have? Let’s say that a salesperson occasionally sells handbags to men, who are buying it for their wives, girlfriends, or mothers. You might have them inquire about the occasion. Perhaps it’s a birthday. When you know  the buyer’s spouse or significant other’s birthday, you might send a personal message to the gentleman around the same time next year, encouraging him to buy a new handbag. Salespeople can also tell you that they’re losing business to competitors because the sales cycle is too long, or too complicated, or there’s too much administrative work. They might also tell you that they’ve lost sales because your business doesn’t accept credit cards. All of these insights can be very helpful. You should encourage your salespeople to engage the customers and prospects, and also encourage them – without judgment – to share their successes, failures, and challenges with you.

Your Competition

Your competitors can be a great resource for your marketing research. Check out their Websites from time to time; follow them on Twitter; “Like” them on Facebook; read their blogs; subscribe to their newsletter; buy their products from time to time; drop in on them if they are a retailer, restaurant, etc. These techniques can alert you to their promotion schedule, the types of customers they are pursuing; the products and/or services they are emphasizing most heavily, what markets they’re in, and so forth. You might also be able to pick up the phone and talk to your competitors directly. It may be that they serve a different niche and that there’s plenty of business to go around. Plus, the fact that you are in the same business gives you an affinity that encourages both your competitors and you to help each other out.

Warranty Cards

Encouraging your customers to fill out a warranty card can also provide useful information: contact information, birthdate, age, type of product purchased, and other kinds of information. This will give you an idea of the type of customer that buys your product. Also, if customers invoke the warranty at some point, you can also get some idea for the products that are having the issue, the types of customers it has been happening with, and the most frequently occurring defects.

Previous Promotions

Look back at some of the ads you ran. How did they perform? Did you test two types of ads? Which one did better? Knowing which promotional tactics work well and which don’t can ensure that you’re directing your marketing dollars more effectively.

This list is far from comprehensive. You can also obtain primary research from trade and professional associations in your industry, as well as from chambers of commerce. You can also get information from your suppliers/vendors. And just plain old networking can give you information.

Primary research is generally expensive, but there’s so much of it that you’re likely already doing, that you may have a wealth of research right within your walls. Mining that information is like mining gold!

Do you have a lot of information you’re collecting that you’re not using to generate new or repeat business? Are you collecting mountains of information but can’t make any sense of it? Would this kind of primary research be of valuable to you, but you just don’t know where to start? Analysights can get you on the right track. Call us at (847) 895-2565 or visit our website at www.analysights.com.

Doing Market Research for Your Business Plan Need not be Expensive

June 2, 2010

Every business needs to do market research. Whether your company is a Fortune 500 corporation or the neighborhood bar, understanding the market or markets in which you operate is critical to your company’s success. Would you invest money in an oil company that didn’t research the fields where it wanted to drill? Would you buy a house in a neighborhood without checking out the schools, crime rate, or housing market? Would you open a restaurant if you knew nothing about the location, the traffic around it, or the prospective customers? You can be sure that if you wanted to open a business, no banker will loan you money without you having done proper, thorough market research.

When one hears the phrase “market research,” most often he/she thinks about surveys and focus groups. These are the most common, yet often most expensive types of market research. Surveys and focus groups are primary research methods, since they are conducted from scratch. Most market research that small businesses need is secondary, that is, research that has already been conducted, published, and available to the public. Often, secondary research can be found in libraries, online, or through other published sources. Secondary research is also much less expensive – sometimes even free – to obtain; however, sifting through it for information relevant to your business’ needs and analyzing it properly can be very time-consuming. In this post, we will discuss how someone starting a business can do market research without breaking the budget.

First Step: Decide on the Information You Need

Tom Johnson has decided to fulfill his dream of starting a comedy club. He’s purchased a book on writing a business plan, and finds that one section of a typical business plan is “Market Analysis.” Tom realizes he must get this section down pat in order to determine the viability of his business and make projections of his first few years of revenues, and convince a banker to lend him money. Tom needs to ask himself several questions: What type of customers am I catering to? What locations are most convenient for attracting those customers? What are the traffic patterns in those locations? What other comedy clubs and entertainment venues are in the area? What do they charge? How do they promote their businesses? What types of promotions do my target customers respond to? Tom writes down all the questions he can think of that will help him analyze his market.

Census Bureau

The first place Tom turns to is the U.S. Bureau of the Census. The bureau’s Web site, www.census.gov, provides a wealth of info for him. He looks at the Web site for demographics, and plugs in the ZIP codes for the locations he is considering, along with their adjacent ZIP codes. The Web site provides great insights into the number of households in the ZIP code, the age ranges, income levels, racial composition, and other demographic factors. Also from the bureau’s Web site, Tom obtains the latest “Consumer Expenditure Survey,” and finds out what the average family spends on entertainment each year.

Tom then notices that the bureau also does an Economic Census of businesses every five years. He finds the Web page for County Business Patterns and looks to see how many entertainment establishments are within the ZIP codes he is considering. He gets good insights about the number of establishments, their employee size, revenues, and payrolls. Tom also finds other interesting facts from the Economic Census – particularly what percentage of revenues entertainment establishments typically spend on various categories: advertising, salaries, maintenance, etc.

Local Library

Tom realizes the Census Bureau has provided him with data that is summarized and aggregated. He needs more information about specific competitors and business patterns in the areas he is considering. So he visits his local library, which has access to several different databases of small businesses, like Dun & Bradstreet’s Hoover’s, and Million Dollar Database. These databases provide information on several individual establishments, including revenues, owner/officer information, employees, and location. Tom does a search of all entertainment establishments in his locations of interest.

Tom also searches through local newspapers of the past few weeks to see which entertainment venues were advertising, how often they were advertising, what they were offering in their ads, etc. He then goes to the Yellow Pages to see if those prospective competitors advertise there as well.

Chambers of Commerce

Tom then contacts different chambers of commerce around his locations of interest. He finds out when their functions are and attends some of them. The local chambers of commerce are great sources for identifying the similar businesses in his area, meeting their owners directly, and finding other businesses that can be help to Tom in opening his business. For example, Tom could meet the general manager of a local movie theater, and might learn from him that the area seems to be pressed for customers, or is impacted by some local ordinance; Tom might also meet a banker or an attorney who specializes in helping new businesses start. Still, he might meet people from a local corporation who are seeking to do events for employees, of which a comedy club can be a great option. Tom might also find information on the cost of labor in the area, as well as commercial real estate rents in various areas. Chambers of commerce are ideal for networking, news, assistance, prospective customers, and other information.

Getting Out There

Tom has now done a lot of secondary research, an exhaustive amount if you ask me! But there is also some primary research he can – and must – do. Tom should drive the areas near the proposed locations for his comedy club. He should check out the other entertainment places nearby: restaurants, jazz/dance clubs, movie theaters, other comedy clubs, karaoke bars, etc. That is, he should mystery shop. Tom should go into some of these competitors and get a feel for the type of clientele to which they cater, the prices they charge, the quality of service they deliver, and how busy they are. He can also see the décor of these venues, their peak times, the outdoor signage, and the traffic around them. All of these can yield valuable clues about the venue’s degree of competitive threat to Tom’s comedy club, and the viability of the location.

Putting it all Together

While there are countless many more sources Tom can turn to for market research, we see he’s done quite an impressive amount already. While most of his sources were free, or of minimal cost, Tom’s real expense was the time and legwork he put into it; he must now synthesize all this information and analyze it to see which locations provide the best mix of traffic, revenue potential, rental costs, and demographics, and then use that information to create forecasts. Once he’s done that, Tom can write the Market Analysis section.

PlanPro Makes the Market Analysis Section of Your Business Plan a Snap!

Chances are you don’t have the time Tom did to do all of that research. Finding all that secondary information and making heads or tails of it is probably something you’d rather delegate to a professional. With PlanPro, Analysights conducts all the secondary research you need for your business, and provides you with templates for the primary research you need to do. Once all the research is compiled, we will analyze it and provide you with the findings, so that you could write the Market Analysis section of your business plan with ease. All for a flat $495! For an extra $125, we will also write the Market Analysis section for you. This way, you can spend more time on the elements of your business plan that make the best use of your time. To learn more about PlanPro, visit: http://analysights.com/PlanPro.aspx or call Analysights at (847) 895-2565.

Charities are Spying on You – But That’s Not Necessarily a Bad Thing!

May 26, 2010

The June 2010 issue of SmartMoney magazine contained an interesting article, “Are Charities Spying On You?,” which discussed the different ways nonprofit organizations are trying to find out information – available from public sources – on current and prospective donors. As one who has worked in the field of data mining and predictive analytics, I found the article interesting in large part because of how well the nonprofit sector has made use of these very techniques in designing their campaigns, solicitations, and programming.

At first glance, it can seem frightening what charities can learn about you. For instance, the article mentions how some charities’ prospect-research departments look at LinkedIn profiles, survey your salary history, and even use satellite images to get information on the home in which you live. And there is a wealth of information out there about us: Zillow.com gives info about the value of our homes and those around it; if you write articles or letters to the editor of your newspaper, online versions can often be found on Google; buy or sell any real estate? That too gets published in the online version of the newspaper; and online bridal and baby shower registries, graduation and wedding announcements, and any other news are fair game. And your shopping history! If you buy online or through a catalog, your name ends up on mailing lists that charities buy. Face it, there’s a lot of information about us that is widely and publicly available.

But is this so terrible? For the most part, I don’t think so. Surely, it’s bad if that information is being used against you. But think of the ways this data mining proves beneficial:

Customization

Let’s assume that you and I are both donors to the Republican National Committee. That suggests we’re both politically active and politically conservative. But are we engaged with the RNC in the same way? Most likely not. You might have donated to the RNC because you’re a wealthy individual who values low taxes and opposes a national health care plan; I might have donated because I am a social conservative who wants prayer in public schools, favors school choice, and opposes abortion. By seeking out information on us, the RNC can tailor its communications in a manner that speaks to each of us individually, sending you information about how it’s fighting proposed tax hikes in various states, and sending me information about school choice initiatives. In this way, the RNC maintains its relevance to each of us.

In addition, it’s very likely, in this example, that you’re donating a lot more money to the RNC than I am. Hence, that would likely lead the RNC to offer you special perks, such as free passes for you and a guest to meet various candidates or attend special luncheons or events. For me, I might at best be given an autographed photo of the event – in exchange for a donation of course – or an invite to the same events, but with a donation of a lot of money requested. I might get information about when the next Tea Party rally in my area will be held. Or even a brief newsletter. One can argue that the treatment you’re getting vs. that of what I’m getting is unfair. However, think of it like this: at a casino, people who gamble regularly and heavily are given all sorts of complimentary perks: drinks, food, a host to attend to their needs, and even special reduced rate stays. That’s because these gamblers are making so much money for the casino, that the cost of these “comps” is small in comparison. In addition, the casino wants to make it more fun for these gamblers to lose money, so that they’ll keep on playing. In short, the special treatment you’re getting is something you’re paying for, if indirectly. I’m getting less because I’m giving less; you’re getting more because you’re giving more. And the charity will give you more to keep you giving more!

Reduced Waste

Before direct marketing got so sophisticated, mass marketing was the only tactic. If you had a product to sell, you sent the same solicitation to thousands, if not millions of people and hoped for a 1-2% response rate. Most people simply threw your solicitation in the garbage when it came in the mail. Many recipients didn’t have a need for the item you were selling or the appeal for which you were soliciting, and disregarded your piece. As a result, lots of paper was wasted, and the phrase “junk mail” came into existence. In addition, if you used follow-up methods, such as phone calls after the mailing, that got costly trying to qualify the leads, just because of the labor involved.

Now, with targeted marketing and list rental, sales, and sharing, charities can build predictive models that estimate each current and prospective donor’s likelihood of responding to a promotion. As a result, the charity doesn’t need to send out quite a large mailing; it can mail solely to those with the best chance of responding, reducing the amount of paper, print, and postage involved, not to mention reduced labor costs involved, both in the production of the piece and in the staffing of the outbound call center. In short, the charity’s data mining is helping the environment, reducing overhead, and increasing the top and bottom lines.

Better Programming

By knowing more about you, the charity can know what makes you “tick,” so that it can come up with programs that fit your needs. Even if you’re not a large donor, if you and other donors feel strongly about certain issues, or value certain programs, the charity can develop programs that are suitable to its members at large. And while many larger donors may be granted special privileges, their large donations can help fund the programs of those who donate less. Everybody wins.

Not bad at all

The data mining tactics charities use aren’t bad. People don’t want to be bombarded with solicitations for which they see no value in it for themselves. Data mining makes it very possible to give you an offer that is relevant to your situation, is cost-effective and resource-efficient, and design programs from which you’re likely to benefit. It is important to note, that while major donors get several great perks, charities must not ignore those whose donations are smaller, for two reasons: first, they have the potential to become major donors, and second, because of their smaller donations, it’s very likely their frequency of giving is greater. This can mean a great stream of gifts to the charity over time. Hence, charities should do things that show these donors they’re appreciated – and, quite often, this too is often accomplished by data mining.

We welcome replies to our blog post!

Using Statistics to Evaluate a Promotion

May 25, 2010

Marketing – as much as cashflow – is the lifeblood of any business. No matter how good your product or service may be, it’s worthless if you can’t get it in front of your customers and get them to buy it. So all businesses, large and small, must engage in marketing. And we see countless types of marketing promotions or tactics being tried: radio and TV commercials, magazine and newspaper advertisements, public relations, coupons, email blasts, and so forth. But are our promotions working? The merchant John Wannamaker, often dubbed the father of modern advertising is said to have remarked, “Half the money I spend on advertising is wasted; the trouble is I don’t know which half.”

Some basic statistics can help you evaluate the effectiveness of your marketing and take away much of the mystique Wannamaker complained about. When deciding whether to do a promotion, managers and business owners have no way of knowing whether it will succeed; in fact, in today’s economy, budgets are still tight. The cost to roll out a full promotion can wipe out an entire marketing budget if it proves to be a fiasco. This is why many businesses do a test before doing a complete rollout. The testing helps to reduce the amount of uncertainty involved in an all-out campaign.

Quite often, large companies need to choose between two or more competing campaigns for rollout. But how do they know which will be effective? Consider the example of Jenny Kaplan, owner of K-Jen, a New Orleans-style restaurant. K-Jen serves up a tasty jambalaya entrée, which is priced at $10.00. Jenny believes that the jambalaya is a draw to the restaurant and believes that by offering a discount, she can increase the average amount of the table check. Jenny decides to issue coupons via email to patrons who have opted-in to receive such promotions. She wants to knock a dollar off the price of the jambalaya as the offer, but doesn’t know whether customers would respond better to an offer worded as “$1.00 off” or as “10% off.” So, Jenny decides to test the two concepts.

Jenny goes to her database of nearly 1,000 patrons and randomly selects 200 patrons. She decides to send half of those a coupon for $1.00 off for jambalaya, and the other half a coupon for 10% off. When the coupon offer expires 10 days later, Jenny finds that 10 coupons were redeemed for each offer – a redemption rate of 10% each. Jenny observes that either wording will get the same number of people to respond. But she wonders which offer generated the largest table check. So she looks at the guest checks to which the coupons were stapled. She notices the following:

Guest Check Amounts

 

Offer

 
 

$1.00 off

10% Off

 
 

$38.85

$50.16

 
 

$36.97

$54.44

 
 

$35.94

$32.20

 
 

$54.17

$32.69

 
 

$68.18

$51.09

 
 

$49.47

$46.18

 
 

$51.39

$57.72

 
 

$32.72

$44.30

 
 

$22.59

$59.29

 
 

$24.13

$22.94

 

 

Jenny quickly computes the average for each offer. The “$1.00 off” coupon generated an average table check of $41.44; the “10% off” coupon generated an average of $45.10. At first glance, it appears that the 10% off promotion generated a higher guest check. But is that difference meaningful, or is it due to chance? Jenny needs to do further analysis.

Hypothesis Testing

How does Jenny determine if the 10% off coupon really did better than the $1.00 off coupon? She can use statistical hypothesis testing, which is a structured analytical method for comparing the difference between two groups – in this case, two promotions. Jenny starts her analysis by formulating two hypotheses: a null hypothesis, which states that there is no difference in the average check amount for either offer; and an alternative hypothesis, which states that there is, in fact, a difference in the average check amount between the two offers. The null hypothesis is often denoted as H0, and the alternative hypothesis is denoted as HA. Jenny also refers to the $1.00 off offer as Offer #1, and the 10% off offer as Offer #2. She wants to compare the means of the two offers, the means of which are denoted as μ1 and μ2, respectively. Jenny writes down her two hypotheses:

H0: The average guest check amount for the two offers is equal.

HA: The average guest check amount for the two offers is not equal.

Or, more succinctly:

H0: μ12

HA: μ1≠μ2

 

Now, Jenny is ready to go to work. Note that the symbol μ denotes the population she wants to measure. Because Jenny did her test on a portion – a sample – of her database, the averages she computed were the sample average, which is denoted as . As we stated earlier, the average table checks for the “$1.00 off” and “10% off” offers were 1=$41.44 and 2=$45.10, respectively. Jenny needs to approximate μ using . She must also compute the sample standard deviation, or s for each offer.

Computing the Sample Standard Deviation

To compute the sample standard deviation, Jenny must subtract the mean of a particular offer from each of its check amounts in the sample; square the difference; sum them up; divide by the total observations minus 1(9) and then take the square root:

$1.00 Off

Actual Table Check

Average Table Check

Difference

Difference Squared

$38.85

$41.44

-$2.59

$6.71

$36.97

$41.44

-$4.47

$19.99

$35.94

$41.44

-$5.50

$30.26

$54.17

$41.44

$12.73

$162.03

$68.18

$41.44

$26.74

$714.97

$49.47

$41.44

$8.03

$64.46

$51.39

$41.44

$9.95

$98.98

$32.72

$41.44

-$8.72

$76.06

$22.59

$41.44

-$18.85

$355.36

$24.13

$41.44

-$17.31

$299.67

   

Total

$1,828.50

   

S21=

$203.17

   

S1=

$14.25

 

10% Off

Actual Table Check

Average Table Check

Difference

Difference Squared

$50.16

$45.10

$5.06

$25.59

$54.44

$45.10

$9.34

$87.22

$32.20

$45.10

-$12.90

$166.44

$32.69

$45.10

-$12.41

$154.03

$51.09

$45.10

$5.99

$35.87

$46.18

$45.10

$1.08

$1.16

$57.72

$45.10

$12.62

$159.24

$44.30

$45.10

-$0.80

$0.64

$59.29

$45.10

$14.19

$201.33

$22.94

$45.10

-$22.16

$491.11

   

Total

$1,322.63

   

S22=

$146.96

   

S2=

$12.12

 

Notice the denotation of S2. That is known as the variance. The variance and the standard deviation are used to measure the average distance between each data point and the mean. When data are normally distributed, about 95% of all observations fall within two standard deviations from the mean (actually 1.96 standard deviations). Hence, approximately 95% of the guest checks for the $1.00 off offer should fall between $41.44 ± 1.96*($14.25) or between $13.51 and $69.37. All ten fall within this range. For the 10% off offer, about 95% will fall between $45.10 ± 1.96*($12.12), or between $21.34 and $68.86. All 10 observations also fall within this range.

Degrees of Freedom and Pooled Standard Deviation

Jenny noticed two things immediately: first, that the 10% off coupon has the higher sample average, and second each individual table check is closer to it mean than it is for the $1.00 off coupon. Also notice that when we were computing the sample standard deviation for each offer, Jenny divided by 9, and not 10. Why? Because she was making estimates of the population standard deviation. Since samples are subject to error, we must account for that. Each observation gives us information into the population’s actual values. However, Jenny had to make an estimate based on that sample, so she gives up one observation to account for the sampling error – that is, she lost a degree of freedom. In this example, Jenny has 20 total observations; since she estimated the population standard deviation for both offers, she lost two degrees of freedom, leaving her with 18 (10 + 10 – 2).

Knowing the remaining degrees of freedom, Jenny must pool the standard deviations, weighting them by their degrees of freedom. This would be especially evident if the sample sizes of the two offers were not equal. The pooled standard deviation is given by:

FYI – n is simply the sample size. Jenny then computes the pooled standard deviation:

S2p = ((9 * $203.17) + (9 * $146.96))) / (10 + 10 – 2)

= ($1,828.53 + $1,322.64)/18

= $3,151.17/18

= $175.07

Now take the square root: $13.23

Hence, the pooled standard deviation is $13.23

Computing the t-Test Statistic

Now the fun begins. Jenny knows the sample mean of the two offers; she knows the hypothesized difference between the two population means (which we would expect to be zero, if the null hypothesis said they were equal); she knows the pooled standard deviation; she knows the sample size; and she knows the degrees of freedom. Jenny must now calculate the t-Test statistic. The t-Test Statistic, or the t-value, represents the number of estimated standard errors the sample average is from that of the population. The t-value is computed as follows:

 

So Jenny sets to work computing her t-Test Statistic:

t = (($41.44 – $45.10) – (0)) / ($13.23) * SQRT(1/10 + 1/10)

= -$3.66 / ($13.23 * SQRT(1/5))

=-$3.66 / ($13.23 * .45)

=-$3.66/$5.92

= -0.62

This t-statistic gives Jenny a basis for testing her hypothesis. Jenny’s t-statistic indicates that the difference in sample table checks between the two offers is 0.62 standard errors below the hypothesized difference of zero. We now need to determine the critical t – the value that we get from a t-distribution table that is available in most statistics textbooks and online. Since we are estimating with a 95% confidence interval, and since we must account for a small sample, our critical t-value is adjusted slightly from the 1.96 standard deviations from the mean. For 18 degrees of freedom, our critical t is 2.10. The larger the sample size, the closer to 1.96 the critical t would be.

So, does Jenny Accept or Reject her Null Hypothesis (Translation: Is the “10% Off” Offer Better than the “$1.00 Off” Offer)?

Jenny now has all the information she needs to determine whether one offer worked better than the other. What does the critical t of 2.10 mean? If Jenny’s t-statistic is greater than 2.10, or (since one offer can be lower than the other), less than -2.10, then she would reject her null hypothesis, as there is sufficient evidence to suggest that the two means are not equal. Is that the case?

Jenny’s t-statistic is -0.62, which is between -2.10 and 2.10. Hence, it is within the parameters. Jenny should not reject H0, since there is not enough evidence to suggest that one offer was better than the other at generating higher table checks. In fact, there’s nothing to say that the difference between the two offers is due to anything other than chance.

What Does Jenny Do Now?

Basically, Jenny can conclude that there’s not enough evidence that the “$1.00 off” coupon was worse/better than the “10% off” coupon in generating higher table check amounts, and vice-versa. This does not mean that our hypotheses were true or false, just that there was not enough statistical evidence to say so. In this case, we did not accept the null hypothesis, but rather, failed to reject it. Jenny can do a few things:

  1. She can run another test, and see if the same phenomenon holds.
  2. Jenny can accept the fact that both offers work equally well, and compare their overall average table checks to those of who ordered jambalaya without the coupons during the time the offer ran; if the coupons generated average table checks that were higher (using the hypothesis testing procedures outlined above) than those who paid full price, then she may choose to rollout a complete promotion using either or both of the offers described above.
  3. Jenny may decide that neither coupon offer raised average check amounts and choose not to do a full rollout after all.

So Why am I Telling You This?

The purpose of this blog post was to take you step-by-step into how you can use a simple concept like t-tests to judge the performance of two promotion concepts. Although a spreadsheet like Excel can run this test in seconds, I wanted to walk you through the theory in laymen’s terms, so that you can grasp the theory, and then apply it to your business. Analysights is in the business of helping companies – large and small – succeed at marketing, and this blog post is one ingredient in the recipe for your marketing success. If you would like some assistance in setting up a promotion test or in evaluating the effectiveness of a campaign, feel free to contact us at www.analysights.com.