Are Mail Surveys Useless?

These days, most surveys are delivered online. Researchers – especially with the proliferation of consumer panels – can now program a survey, administer it to a relevant sample and get results within a few days, for a relatively low cost per complete. This is a far cry from the day when most surveys were conducted by mail. Mail surveys often needed to be planned out well in advance, had to be kept in the field for several weeks – if not a few months, – required incentives and reminders, and often generated low response rates. Needless to say, mail surveys were also quite costly and could not be changed once in the field.

Most marketing research professionals don’t even consider conducting a survey by mail anymore; most now view mail surveys as obsolete. While I certainly favor online and social media surveys more than mail surveys, I caution not to dismiss mail surveys out of hand. They still have some relevance and, depending on the business objective, may be a better choice than the popular online survey methods.

There are several reasons why you might still consider doing a survey by mail:

1. Some people still aren’t online. What if you need to survey elderly persons? Or low-income households? Many persons in these groups do not have Internet access, so they cannot be reached online. Assuming they’re not homeless, virtually all of them live at a physical address with a mailbox.

    

2. Advance permission is often needed to send e-mail surveys. Because of the Controlling the Assault of Non-Solicited Pornography and Marketing (CAN-SPAM) Act of 2003, marketers cannot send promotional e-mail to prospects without permission. While a survey is not promotional, consumers upset about receiving an unsolicited survey might still report it as SPAM, getting you into trouble. This is why most e-mail surveys make use of pre-recruited panels. Mail surveys don’t require such permission.

    

3. Mailing lists can be obtained to conduct surveys. E-mail address lists cannot be sold. Quite often, you can rent mailing lists to send out surveys.

    

4. Mail surveys these days may get a better-than-expected response rate. Response rates likely won’t be double-digit, but since few mail surveys are sent these days, those few that are have a better chance of catching the respondent’s attention. And since the respondent isn’t being bombarded with mail surveys, he or she may be more inclined to answer.

    

5. Mail surveys offer greater perception of anonymity and confidentiality – and hence more truthful responses – than online surveys. Since surveys are administered online, it’s easy to tell who didn’t respond. When you send a respondent a reminder e-mail, the respondent knows his or her lack of response is known. This may lead him or her to feel that the answers he/she gives are also traceable back to him/her. As a result, he or she may be less-inclined to respond truthfully, let alone respond. Although tracking mechanisms have been placed on mail surveys, they’re not as easily discernable as they are for online surveys.

While online surveys appear to be the preferred survey method, there are still times when mail surveys are the better means of data collection. Sometimes, survey projects need to be multimodal in order to achieve a representative sample. Just because online surveys are faster and cheaper than mail surveys, you must consider the value of the insights each mode promises to bring to your business objective.

Insight Central Resumes Week of January 3, 2011!

In observance of the Christmas and New Years, Insight Central will resume the week of January 3, 2011.  We here at Analysights wish you and your family and friends a very Merry Christmas, and a Happy, Healthy, and Prosperous New Year! 

*************************

Be Sure to Follow us on Facebook and Twitter !

Thanks to all of you, Analysights now has more than 200 fans on Facebook … and we’d love more! If you like Forecast Friday – or any of our other posts – then we want you to “Like” us on Facebook! And if you like us that much, please also pass these posts on to your friends who like forecasting and invite them to “Like” Analysights! By “Like-ing” us on Facebook, you and they will be informed every time a new blog post has been published, or when new information comes out. Check out our Facebook page! You can also follow us on Twitter. Thanks for your help!

Analyzing Subgroups of Data

The data available to us has never been more voluminous. Thanks to technology, data about us and our environment are collected almost continuously. When we use a cell phone to call someone else’s cell phone, several pieces of information are collected: the two phone numbers involved in the call; the time the call started and ended; the cell phone towers closest to the two parties; the cell phone carriers; the distance of the call; the date; and many more. Cell phone companies use this information to determine where to increase capacity; refine, price, and promote their plans more effectively; and identify regions with inadequate coverage.

Multiply these different pieces of data by the number of calls in a year, a month, a day – even an hour – and you can easily see that we are dealing with enormous amounts of records and observations. While it’s good for decision makers to see what sales, school enrollment, cell phone usage, or any other pattern looks like in total, quite often they are even more interested in breaking down data into groups to see if certain groups behave differently. Quite often we hear decision makers asking questions like these:

• How do depositors under age 35 compare with those between 35-54 and 55 & over in their choice of banking products?
• How will voter support for Candidate A differ by race or ethnicity?
• How does cell phone usage differ between men and women?
• Does the length or severity of a prison sentence differ by race?

When we break data down into subgroups, we are trying to see whether knowing about these groups adds any additional meaningful information. This helps us customize marketing messages, product packages, pricing structures, and sales channels for different segments of our customers. There are many different ways we can break data down: by region, age, race, gender, income, spending levels; the list is limitless.

To give you an example of how data can be analyzed by groups, let’s revisit Jenny Kaplan, owner of K-Jen, the New Orleans-style restaurant. If you recall from the May 25 post, Jenny tested two coupon offers for her $10 jambalaya entrée: one offering 10% off and another offering $1 off. Even though the savings was the same, Jenny thought customers would respond differently. As Jenny found, neither offer was better than the other at increasing the average size of the table check. Now, Jenny wants to see if there is a preference for one offer over the other, based on customer age.

Jenny knows that of her 1,000-patron database, about 50% are the ages of 18 to 35; the rest are older than 35. So Jenny decides to send out 1,000 coupons via email as follows:

	$1 off	10% off	Total Coupons
18-35	250	250	500
Over 35	250	250	500
Total Coupons	500	500	1,000

Half of Jenny’s customers received one coupon offer and half received the other. Looking carefully at the table above, half the people in each age group got one offer and the other half got the other offer. At the end of the promotion period, Jenny received back 200 coupons. She tracks the coupon codes back to her database and finds the following pattern:

Coupons Redeemed (Actual)
	$1 off	10% off	Coupons Redeemed
18-35	35	65	100
Over 35	55	45	100
Coupons Redeemed	90	110	200

 

Exactly 200 coupons were redeemed, 100 from each age group. But notice something else: of the 200 people redeeming the coupon, 110 redeemed the coupon offering 10% off; just 90 redeemed the $1 off coupon. Does this mean the 10% off coupon was the better offer? Not so fast!

What Else is the Table Telling Us?

Look at each age group. Of the 100 customers aged 18-35, 65 redeemed the 10% off coupon; but of the 100 customers age 35 and up, just 45 did. Is that a meaningful difference or just a fluke? Do persons over 35 prefer an offer of $1 off to one of 10% off? There’s one way to tell: a chi-squared test for statistical significance.

The Chi-Squared Test

Generally, a chi-squared test is useful in determining associations between categories and observed results. The chi-squared – χ² – statistic is value needed to determine statistical significance. In order to compute χ², Jenny needs to know two things: the actual frequency distribution of the coupons redeemed (which is shown in the last table above), and the expected frequencies.

Expected frequencies are the types of frequencies you would expect the distribution of data to fall, based on probability. In this case, we have two equal sized groups: customers age 18-35 and customers over 35. Knowing nothing else besides the fact that the same number of people in these groups redeemed coupons, and that 110 of them redeemed the 10% off coupon, and 90 redeemed the $1 off coupon, we would expect that 55 customers in each group would redeem the 10% off coupon and 45 in each group would redeem the $1 off coupon. Hence, in our expected frequencies, we still expect 55% of the total customers to redeem the 10% off offer. Jenny’s expected frequencies are:

Coupons Redeemed (Expected)
	$1 off	10% off	Coupons Redeemed
18-35	45	55	100
Over 35	45	55	100
Coupons Redeemed	90	110	200

 

As you can see, the totals for each row and column match those in the actual frequency table above. The mathematical way to compute the expected frequencies for each cell would be to multiply its corresponding column total by its corresponding row total and then divide it by the total number of observations. So, we would compute as follows:

Frequency of:	Formula:	Result
18-35 redeeming $1 off:	=(100*90)/200	=45
18-35 redeeming 10% off:	=(100*110)/200	=55
Over 35 redeeming $1 off:	=(100*90)/200	=45
Over 35 redeeming 10% off:	=(100*110)/200	=55

 

Now that Jenny knows the expected frequencies, she must determine the critical χ² statistic to determine significance, then she must compute the χ² statistic for her data. If the latter χ² is greater than the critical χ² statistic, then Jenny knows that the customer’s age group is associated the coupon offer redeemed.

Determining the Critical χ² Statistic

To find out what her critical χ² statistic is, Jenny must first determine the degrees of freedom in her data. For cross-tabulation tables, the number of degrees of freedom is a straightforward calculation:

Degrees of freedom = (# of rows – 1) * (# of columns -1)

So, Jenny has two rows of data and two columns, so she has (2-1)*(2-1) = 1 degree of freedom. With this information, Jenny grabs her old college statistics book and looks at the χ² distribution table in the appendix. For a 95% confidence interval with one degree of freedom, her critical χ² statistic is 3.84. When Jenny calculates the χ² statistic from her frequencies, she will compare it with the critical χ² statistic. If Jenny’s χ² statistic is greater than the critical, she will conclude that the difference is statistically significant and that age does relate to which coupon offer is redeemed.

Calculating the χ² Value From Observed Frequencies

Now, Jenny needs to compare the actual number of coupons redeemed for each group to their expected number. Essentially, to compute her χ² value, Jenny follows a particular formula. For each cell, she subtracts the expected frequency of that cell from the actual frequency, squares the difference, and then divides it by the expected frequency. She does this for each cell. Then she sums up her results to get her χ² value:

	$1 off	10% off
18-35	=(35-45)^2/45 = 2.22	=(65-55)^2/55=1.82
Over 35	=(55-45)^2/45 = 2.22	=(45-55)^2/55=1.82
χ2=	2.22+1.82+2.22+1.82
=	8.08

 

Jenny’s χ² value is 8.08, much higher than the critical 3.84, indicating that there is indeed an association between age and coupon redemption.

Interpreting the Results

Jenny concludes that patrons over the age of 35 are more inclined than patrons age 18-35 to take advantage of a coupon stating $1 off; patrons age 18-35 are more inclined to prefer the 10% off coupon. The way Jenny uses this information depends on the objectives of her business. If Jenny feels that K-Jen needs to attract more middle-aged and senior citizens, she should use the $1 off coupon when targeting them. If Jenny feels K-Jen isn’t selling enough Jambalaya, then she might try to stimulate demand by couponing, sending the $1 off coupon to patrons over the age of 35 and the 10% off coupon to those 18-35.

Jenny might even have a counterintuitive use for the information. If most of K-Jen’s regular patrons are over age 35, they may already be loyal customers. Jenny might still send them coupons, but give the 10% off coupon instead. Why? These customers are likely to buy the jambalaya anyway, so why not give them the coupon they are not as likely to redeem? After all, why give someone a discount if they’re going to buy anyway! Giving the 10% off coupon to these customers does two things: first, it shows them that K-Jen still cares about their business and keeps them aware of K-Jen as a dining option. Second, by using the lower redeeming coupon, Jenny can reduce her exposure to subsidizing loyal customers. In this instance, Jenny uses the coupons for advertising and promoting awareness, rather than moving orders of jambalaya.

There are several more ways to analyze data by subgroup, some of which will be discussed in future posts. It is important to remember that your research objectives dictate the information you collect, which dictate the appropriate analysis to conduct.

*************************

If you Like Our Posts, Then “Like” Us on Facebook and Twitter!

Analysights is now doing the social media thing! If you like Forecast Friday – or any of our other posts – then we want you to “Like” us on Facebook! By “Like-ing” us on Facebook, you’ll be informed every time a new blog post has been published, or when other information comes out. Check out our Facebook page! You can also follow us on Twitter.

Using Statistics to Evaluate a Promotion

Marketing – as much as cashflow – is the lifeblood of any business. No matter how good your product or service may be, it’s worthless if you can’t get it in front of your customers and get them to buy it. So all businesses, large and small, must engage in marketing. And we see countless types of marketing promotions or tactics being tried: radio and TV commercials, magazine and newspaper advertisements, public relations, coupons, email blasts, and so forth. But are our promotions working? The merchant John Wannamaker, often dubbed the father of modern advertising is said to have remarked, “Half the money I spend on advertising is wasted; the trouble is I don’t know which half.”

Some basic statistics can help you evaluate the effectiveness of your marketing and take away much of the mystique Wannamaker complained about. When deciding whether to do a promotion, managers and business owners have no way of knowing whether it will succeed; in fact, in today’s economy, budgets are still tight. The cost to roll out a full promotion can wipe out an entire marketing budget if it proves to be a fiasco. This is why many businesses do a test before doing a complete rollout. The testing helps to reduce the amount of uncertainty involved in an all-out campaign.

Quite often, large companies need to choose between two or more competing campaigns for rollout. But how do they know which will be effective? Consider the example of Jenny Kaplan, owner of K-Jen, a New Orleans-style restaurant. K-Jen serves up a tasty jambalaya entrée, which is priced at $10.00. Jenny believes that the jambalaya is a draw to the restaurant and believes that by offering a discount, she can increase the average amount of the table check. Jenny decides to issue coupons via email to patrons who have opted-in to receive such promotions. She wants to knock a dollar off the price of the jambalaya as the offer, but doesn’t know whether customers would respond better to an offer worded as “$1.00 off” or as “10% off.” So, Jenny decides to test the two concepts.

Jenny goes to her database of nearly 1,000 patrons and randomly selects 200 patrons. She decides to send half of those a coupon for $1.00 off for jambalaya, and the other half a coupon for 10% off. When the coupon offer expires 10 days later, Jenny finds that 10 coupons were redeemed for each offer – a redemption rate of 10% each. Jenny observes that either wording will get the same number of people to respond. But she wonders which offer generated the largest table check. So she looks at the guest checks to which the coupons were stapled. She notices the following:

Guest Check Amounts
	Offer
	$1.00 off	10% Off
	$38.85	$50.16
	$36.97	$54.44
	$35.94	$32.20
	$54.17	$32.69
	$68.18	$51.09
	$49.47	$46.18
	$51.39	$57.72
	$32.72	$44.30
	$22.59	$59.29
	$24.13	$22.94

 

Jenny quickly computes the average for each offer. The “$1.00 off” coupon generated an average table check of $41.44; the “10% off” coupon generated an average of $45.10. At first glance, it appears that the 10% off promotion generated a higher guest check. But is that difference meaningful, or is it due to chance? Jenny needs to do further analysis.

Hypothesis Testing

How does Jenny determine if the 10% off coupon really did better than the $1.00 off coupon? She can use statistical hypothesis testing, which is a structured analytical method for comparing the difference between two groups – in this case, two promotions. Jenny starts her analysis by formulating two hypotheses: a null hypothesis, which states that there is no difference in the average check amount for either offer; and an alternative hypothesis, which states that there is, in fact, a difference in the average check amount between the two offers. The null hypothesis is often denoted as H₀, and the alternative hypothesis is denoted as H_A. Jenny also refers to the $1.00 off offer as Offer #1, and the 10% off offer as Offer #2. She wants to compare the means of the two offers, the means of which are denoted as μ₁ and μ₂, respectively. Jenny writes down her two hypotheses:

H₀: The average guest check amount for the two offers is equal.

H_A: The average guest check amount for the two offers is not equal.

Or, more succinctly:

H₀: μ₁=μ₂

H_A: μ₁≠μ₂

 

Now, Jenny is ready to go to work. Note that the symbol μ denotes the population she wants to measure. Because Jenny did her test on a portion – a sample – of her database, the averages she computed were the sample average, which is denoted as . As we stated earlier, the average table checks for the “$1.00 off” and “10% off” offers were ₁=$41.44 and ₂=$45.10, respectively. Jenny needs to approximate μ using . She must also compute the sample standard deviation, or s for each offer.

Computing the Sample Standard Deviation

To compute the sample standard deviation, Jenny must subtract the mean of a particular offer from each of its check amounts in the sample; square the difference; sum them up; divide by the total observations minus 1(9) and then take the square root:

$1.00 Off
Actual Table Check	Average Table Check	Difference	Difference Squared
$38.85	$41.44	-$2.59	$6.71
$36.97	$41.44	-$4.47	$19.99
$35.94	$41.44	-$5.50	$30.26
$54.17	$41.44	$12.73	$162.03
$68.18	$41.44	$26.74	$714.97
$49.47	$41.44	$8.03	$64.46
$51.39	$41.44	$9.95	$98.98
$32.72	$41.44	-$8.72	$76.06
$22.59	$41.44	-$18.85	$355.36
$24.13	$41.44	-$17.31	$299.67
		Total	$1,828.50
		S21=	$203.17
		S1=	$14.25

 

10% Off
Actual Table Check	Average Table Check	Difference	Difference Squared
$50.16	$45.10	$5.06	$25.59
$54.44	$45.10	$9.34	$87.22
$32.20	$45.10	-$12.90	$166.44
$32.69	$45.10	-$12.41	$154.03
$51.09	$45.10	$5.99	$35.87
$46.18	$45.10	$1.08	$1.16
$57.72	$45.10	$12.62	$159.24
$44.30	$45.10	-$0.80	$0.64
$59.29	$45.10	$14.19	$201.33
$22.94	$45.10	-$22.16	$491.11
		Total	$1,322.63
		S22=	$146.96
		S2=	$12.12

 

Notice the denotation of S². That is known as the variance. The variance and the standard deviation are used to measure the average distance between each data point and the mean. When data are normally distributed, about 95% of all observations fall within two standard deviations from the mean (actually 1.96 standard deviations). Hence, approximately 95% of the guest checks for the $1.00 off offer should fall between $41.44 ± 1.96*($14.25) or between $13.51 and $69.37. All ten fall within this range. For the 10% off offer, about 95% will fall between $45.10 ± 1.96*($12.12), or between $21.34 and $68.86. All 10 observations also fall within this range.

Degrees of Freedom and Pooled Standard Deviation

Jenny noticed two things immediately: first, that the 10% off coupon has the higher sample average, and second each individual table check is closer to it mean than it is for the $1.00 off coupon. Also notice that when we were computing the sample standard deviation for each offer, Jenny divided by 9, and not 10. Why? Because she was making estimates of the population standard deviation. Since samples are subject to error, we must account for that. Each observation gives us information into the population’s actual values. However, Jenny had to make an estimate based on that sample, so she gives up one observation to account for the sampling error – that is, she lost a degree of freedom. In this example, Jenny has 20 total observations; since she estimated the population standard deviation for both offers, she lost two degrees of freedom, leaving her with 18 (10 + 10 – 2).

Knowing the remaining degrees of freedom, Jenny must pool the standard deviations, weighting them by their degrees of freedom. This would be especially evident if the sample sizes of the two offers were not equal. The pooled standard deviation is given by:

FYI – n is simply the sample size. Jenny then computes the pooled standard deviation:

S²_p = ((9 * $203.17) + (9 * $146.96))) / (10 + 10 – 2)

= ($1,828.53 + $1,322.64)/18

= $3,151.17/18

= $175.07

Now take the square root: $13.23

Hence, the pooled standard deviation is $13.23

Computing the t-Test Statistic

Now the fun begins. Jenny knows the sample mean of the two offers; she knows the hypothesized difference between the two population means (which we would expect to be zero, if the null hypothesis said they were equal); she knows the pooled standard deviation; she knows the sample size; and she knows the degrees of freedom. Jenny must now calculate the t-Test statistic. The t-Test Statistic, or the t-value, represents the number of estimated standard errors the sample average is from that of the population. The t-value is computed as follows:

 

So Jenny sets to work computing her t-Test Statistic:

t = (($41.44 – $45.10) – (0)) / ($13.23) * SQRT(1/10 + 1/10)

= -$3.66 / ($13.23 * SQRT(1/5))

=-$3.66 / ($13.23 * .45)

=-$3.66/$5.92

= -0.62

This t-statistic gives Jenny a basis for testing her hypothesis. Jenny’s t-statistic indicates that the difference in sample table checks between the two offers is 0.62 standard errors below the hypothesized difference of zero. We now need to determine the critical t – the value that we get from a t-distribution table that is available in most statistics textbooks and online. Since we are estimating with a 95% confidence interval, and since we must account for a small sample, our critical t-value is adjusted slightly from the 1.96 standard deviations from the mean. For 18 degrees of freedom, our critical t is 2.10. The larger the sample size, the closer to 1.96 the critical t would be.

So, does Jenny Accept or Reject her Null Hypothesis (Translation: Is the “10% Off” Offer Better than the “$1.00 Off” Offer)?

Jenny now has all the information she needs to determine whether one offer worked better than the other. What does the critical t of 2.10 mean? If Jenny’s t-statistic is greater than 2.10, or (since one offer can be lower than the other), less than -2.10, then she would reject her null hypothesis, as there is sufficient evidence to suggest that the two means are not equal. Is that the case?

Jenny’s t-statistic is -0.62, which is between -2.10 and 2.10. Hence, it is within the parameters. Jenny should not reject H₀, since there is not enough evidence to suggest that one offer was better than the other at generating higher table checks. In fact, there’s nothing to say that the difference between the two offers is due to anything other than chance.

What Does Jenny Do Now?

Basically, Jenny can conclude that there’s not enough evidence that the “$1.00 off” coupon was worse/better than the “10% off” coupon in generating higher table check amounts, and vice-versa. This does not mean that our hypotheses were true or false, just that there was not enough statistical evidence to say so. In this case, we did not accept the null hypothesis, but rather, failed to reject it. Jenny can do a few things:

1. She can run another test, and see if the same phenomenon holds.
2. Jenny can accept the fact that both offers work equally well, and compare their overall average table checks to those of who ordered jambalaya without the coupons during the time the offer ran; if the coupons generated average table checks that were higher (using the hypothesis testing procedures outlined above) than those who paid full price, then she may choose to rollout a complete promotion using either or both of the offers described above.
3. Jenny may decide that neither coupon offer raised average check amounts and choose not to do a full rollout after all.

So Why am I Telling You This?

The purpose of this blog post was to take you step-by-step into how you can use a simple concept like t-tests to judge the performance of two promotion concepts. Although a spreadsheet like Excel can run this test in seconds, I wanted to walk you through the theory in laymen’s terms, so that you can grasp the theory, and then apply it to your business. Analysights is in the business of helping companies – large and small – succeed at marketing, and this blog post is one ingredient in the recipe for your marketing success. If you would like some assistance in setting up a promotion test or in evaluating the effectiveness of a campaign, feel free to contact us at www.analysights.com.

 

Don’t Confuse E-mail Selling with E-mail Marketing

In E-mail Marketing vs. E-mail Sales, e-mail marketing expert and independent consultant, Jeanne Jennings, discussed how some companies are confusing e-mail marketing with e-mail selling, and thus not reaping much of the benefits e-mail marketing can offer them.

Jennings points out that e-mail marketing is focused on longer-term objectives, while e-mail sales are geared towards immediate revenue, and that companies who send nothing but promotional e-mails tend to fatigue their lists, as well as limit their audience only to customers and prospects who are already at that stage in the purchasing cycle.  I could not agree more.

Jennings reiterates what we marketers must never forget: E-mail marketing – as any marketing –  is more than selling; it’s also brand-building, relationship-building, keeping your company at the top of your customers’ minds, and exchanging information between you and your customers.  Concentrating your e-mails solely on short-term sales can cost you greatly in foregone future repeat sales that often accompany good customer relationships.

The ideal proportion of your e-mail marketing messages that should be non-promotional vs. promotional varies by industry, product category, and other factors.  However, your company can reap great benefits from a healthy mix of these two types of messages.

Sending a promotional e-mail will likely succeed if a prospect is in the buying stage.  Another e-mail that offers news and tips on how to use your product may help increase product usage and create customer loyalty.  An e-mail that illustrates the benefits of your product or service may help a prospect who is still in the needs discovery phase of the buying cycle to think of your company when he/she is ready to buy.  Sending a confirmation e-mail after a purchase – coupled with additional information –  can trigger some impulse spending by your customer.

Another useful benefit of taking a long-term focus to e-mail marketing is that link-clicks are trackable.  You can track the behavior of someone on your e-mail list when he/she opens your e-mail and clicks on a link.  This can yield valuable clues about the type of content that interests the prospect, and can help you tailor both your non-promotional and promotional e-mails to the prospects’ preferences.  When you send your prospects and customers e-mail that interests them, they believe you have their best interest in mind, and they are more likely to buy from you.

In these recessionary times, companies need to make sales.  But hard selling, whether online or off, is a sign of desperation.  Companies whose marketing demonstrates  – in every channel – that they understand and care about their customers will more than make up for today’s lost sales tomorrow.

Three Metrics for E-mail Marketing Excellence

The principles of direct marketing apply just as much online as they do offline.  The process for tracking the performance of an e-mail campaign is essentially the same as for that of a direct mail campaign.

How do you know if your e-mail campaigns are working?  Start with three basic statistics: your bounce rate, your open rate, and your click-through rate.

Bounce Rate

The bounce rate tells you the percentage of your e-mails that were returned because they were undeliverable.  If you sent 10,000 e-mails and 1,000 were undeliverable, your bounce rate is 10%.  The 9,000 e-mails that were delivered are known as your non-bounce total.

Use the bounce rate to assess the quality and recency of your e-mail list.  Eyeball the list of e-mail addresses that bounced back.  You may find that some are simply invalid addresses (“,com” instead of “.com”) which can easily be rectified.  Others may be incomplete and thus useless.  Still other addresses might be old, which suggests you should have a continuous process in place for your customers and prospects to update their e-mail addresses.

Reducing bounce rate should be an ongoing objective of your e-mail marketing strategy.

Open Rate

The open rate is the number of recipients who opened an  HTML version of your e-mail, expressed as a percentage of your non-bounce total.  The open rate can give you an idea of how compelling and attention-getting your e-mail is.  Continuing with the example above, if 1,800 recipients opened your e-mail, then you have an open rate of 20% (1,800/9,000). 

The “HTML version” and the non-bounce total are very important components of this definition.  E-mail Service Providers (ESPs) can only track HTML e-mail messages, not text.  And the use of the non-bounce total has its own share of problems, because the non-bounce total isn’t synonymous with the total e-mails delivered.

E-mails may not be considered bounced because some e-mail servers inadvertently send them to a junk folder on the recipient’s computer, which he/she cannot access.  Furthermore, if the e-mail isn’t bounced by the server, but by a portable device or software on the recipient’s computer, it will not show up in your e-mail tracking report.  Hence, you are basing your open rate on the number of e-mails sent, as opposed to delivered.

An additional problem with the open rate lies in the definition of “opened.”  Your e-mail is considered “opened” if the recipient either 1) opens it in full view or lets its images display in the preview pane, or 2) clicks a link in the e-mail.   The preview pane is a double-edged sword: If the recipient let the images of your e-mail display in the pane, your open rate may be overstated.  On the other hand, if the recipient didn’t allow images to show in the pane, but scanned the e-mail, you open rate will be understated.

You might want to use some qualitative methods to estimate the degree to which these flaws exist.  For example, a survey may give you an idea of the percentage of your customers who use the preview pane and allow images to display; a test of 100 pre-recruited members of your list to receive your e-mail (who report whether or not they received it) might give you clues into how many non-bounced e-mails weren’t sent.  This will help you place a margin of error around your open rate.

If you find your open rates declining over several campaigns, that is a sign to make your messages more compelling.

 Click-Through Rate

Your click-through rate tells you the percentage of unique individuals who click at least one link in your opened e-mail.  If, from the 1,800 e-mails were opened, 180 recipients clicked at least one link, then your click-through rate is 10% (of your opened e-mails).  It’s important that you subtract multiple clicks by a recipient (whether he/she clicked more than one link, or one link several times), in order to prevent double counting.  Most ESPs do this for you seemlessly.

Your click-through rate is a measure of how well your e-mail calls your prospects to action.  Low or declining click-through rates suggest your e-mail message isn’t generating interest or desire.

Always remember to:

1. Track every e-mail campaign you do;
2. Look at non-click activity (increased store traffic, phone inquiries, etc.) that occur immediately following an e-mail campaign;
3. Look at activity to your Web site immediately following an e-mail campaign; and
4. Track your metrics over time.  Look for trends in these metrics to refine and improve your e-mail marketing results.