Posts Tagged ‘Facebook’

Avoiding Biased Survey Questions

July 19, 2010

Adequate thought must be given to designing a questionnaire. Ask the wrong questions, or ask questions the wrong way, and you can end up with useless information; make the survey difficult or cumbersome, and respondents won’t participate; put the questions in the wrong order and you can end up with biased results. The most common problem with wording survey questions is bias. Biased questions are frequently asked in surveys administered by groups or organizations that are seeking to advance their political or social action agendas, or by certain departments or units within a corporation or organization likewise seeking to improve their political standing within the organization. Consider the questions below:

“Do you think the senseless war in Iraq that President Bush insisted on starting is going to result in thousands of unnecessary deaths?”

“Do you think the unprecedented trillion-dollar federal deficit the Democrats are creating with their out-of-control spending is going to have disastrous consequences for our nation?”

“Do you favor repeal of the death tax, so that many families won’t be unfairly burdened with hefty taxes at the time of their grief?”

Could these questions be more biased? Notice the adjectives in the questions, words like “senseless,” “unnecessary,” “unprecedented,” “out-of-control,” “disastrous,” “unfairly,” “burdened,” and “hefty.” All of them make it clear that a certain answer to each question is expected.

Look also at the descriptive words in some of the questions: “trillion-dollar,” “death” (as opposed to “estate”). You can see further manipulation. Worded the way they are, these questions stir up the emotions, which surveys are not supposed to do.

Removing the Bias

Can these questions be improved? Depending on the objectives of the survey, most definitely. In the first question, we might simply change the question to a multiple choice and ask:

What is your opinion regarding President Bush’s decision to send troops to Iraq?

Totally Sensible

Mostly Sensible

Somewhat Sensible

Not Sure

Somewhat Senseless

Mostly Senseless

Totally Senseless

 

Notice the difference? Here, the question is neutral. It also opens the survey taker to options that reflect the degree to which he/she feels about President Bush’s decision.

How about the second question? Perhaps we can try this:

In your opinion, how serious will the consequences of the federal budget deficit be for the nation?

Very Serious (5)

Serious (4)

Slightly Serious (3)

Not Very Serious (2)

Not at All Serious (1)

 

Here, we again neutralize the tone of the question and we let the respondent decide how severe the impact of the deficit will be. Notice also that we used an unbalanced scale, like we discussed last week. That’s because we would expect more respondents to select choices on the left hand side of the scale. This revised question focused on the seriousness of the deficit. We could also ask respondents about their perceptions of the size of deficit:

How do you feel about the size of the federal budget deficit?

Too Large (5)

Very Large (4)

Slightly Large (3)

Just Right (2)

Too Small (1)

 

Again, we use an unbalanced scale for this one. If we asked both the revised questions, we can gain great insights into the respondent’s perceptions of both the size and seriousness of the deficit. Ideally, we would ask the question about the deficit’s size before the question about its consequences.

These two revised questions should also point out another flaw with the original question: not only was it worded with bias, but it was also multipurpose or double-barreled. It was trying to fuse two thoughts about the deficit: it was too large and it was going to have serious consequences. These two revised questions will give us another advantage: we can now see how many people think the deficit is too large but do not see it as a serious threat. After all, we may agree something is excessive but we may not necessarily agree about the impact of that excess.

Now let’s look at the last question. Perhaps we can focus on the fairness of the estate tax:

What is your opinion regarding the fairness of the estate tax?

Absolutely Fair

Mostly Fair

Not Sure

Mostly Unfair

Absolutely Unfair

 

Of course, some respondents might not know what the estate tax is, so we need to describe it to them. Even in describing or defining something, we can open the door to bias, so we must choose our words carefully:

When a person dies, his or her heirs pay taxes on the amount of his/her estate that exceeds $1 million. This is known as the “estate” tax. What is your opinion regarding the fairness of such a tax?

Absolutely Fair

Mostly Fair

Not Sure

Mostly Unfair

Absolutely Unfair

 

This does a good job of describing the estate tax, but putting in the $1 million dollar figure can bias the results. If a respondent’s net worth is nowhere close to $1 million, he or she may consider the estate tax fair, just his or her heirs are unlikely to be affected by it. Perhaps the question can be worded this way:

When a person dies, a portion of his or her estate is subject to an “estate” tax. Would you say that such a tax is:

Absolutely Fair

Somewhat Fair

Not Sure

Somewhat Unfair

Absolutely Unfair

 

I think this example would be better, since it says “a portion” rather than a specific amount. While the $1 million example is more factual, it also adds in more normative considerations. By using “a portion,” we respondents won’t concentrate on the dollar amount of the estate, but on the fairness of the estate tax.

The adage “It’s not what you say but how you say it,” rings very true in questionnaire design. You must choose your words carefully in order to get the information you need to make well-informed business decisions.

*************************

If you Like Our Posts, Then “Like” Us on Facebook and Twitter!

Analysights is now doing the social media thing! If you like Forecast Friday – or any of our other posts – then we want you to “Like” us on Facebook! By “Like-ing” us on Facebook, you’ll be informed every time a new blog post has been published, or when other information comes out. Check out our Facebook page! You can also follow us on Twitter.

Advertisements

No Money to Conduct Primary Research? You May Have Done a Lot of it Already!

June 8, 2010

Last week, I wrote about an entrepreneur who was conducting secondary marketing research so he could develop his business plan. This week, I am writing to talk about primary research – data your company generates on its own. Often, we think of surveys and focus groups when we hear “primary research.” And those methods can indeed be costly. However, your business is probably generating volumes of primary data right under your nose. You’re out to hear the voice of your customers and prospects when you do primary research, and primary data is coming to you at nearly every touch point you have with them. Think of these sources:

Customer Service Calls

When customers call for customer service, or prospects call for information, what are the most common things they ask about? If your business sells handbags, which ones are frequently inquired about? Are the handbags most inquired about those that are higher or lower priced? Are they new handbags you’ve introduced? Are they mostly imported handbags? Also, who is making the inquiries? Are they long-term customers? Prospects? If long-term customers are inquiring about one line of handbags and prospects about another, you can tailor your marketing messages to their interests.

Customer Complaints

Nobody likes to be on the receiving end of a complaint. But complaints can be a great source of information. They can alert you to product defects, service breakdowns, and even give you ideas for enhancing your product or service. They can even help you save a long-term relationship and avoid bad word-of-mouth press. If women are complaining that the strap on one of the handbags you sell is uncomfortable to hang over their shoulders, that can prompt you to look for alternatives, or contact the supplier with that information. If a customer complains about the treatment an employee gave him/her, you might use that as an opportunity to either train your staff on improved customer service or discipline that employee.

It’s often said that 96% of a business’ dissatisfied customers will not complain; 91% will quietly go away; and those silent dissatisfied customers will likely communicate their dissatisfaction to at least nine other people. Encourage your customers to speak up when they’re not happy. Complaints can be a rich source of research.

Your Salespeople

Your salespeople are out in the field. They see everything at the frontlines. What successes are they having? What gripes do they have? Let’s say that a salesperson occasionally sells handbags to men, who are buying it for their wives, girlfriends, or mothers. You might have them inquire about the occasion. Perhaps it’s a birthday. When you know  the buyer’s spouse or significant other’s birthday, you might send a personal message to the gentleman around the same time next year, encouraging him to buy a new handbag. Salespeople can also tell you that they’re losing business to competitors because the sales cycle is too long, or too complicated, or there’s too much administrative work. They might also tell you that they’ve lost sales because your business doesn’t accept credit cards. All of these insights can be very helpful. You should encourage your salespeople to engage the customers and prospects, and also encourage them – without judgment – to share their successes, failures, and challenges with you.

Your Competition

Your competitors can be a great resource for your marketing research. Check out their Websites from time to time; follow them on Twitter; “Like” them on Facebook; read their blogs; subscribe to their newsletter; buy their products from time to time; drop in on them if they are a retailer, restaurant, etc. These techniques can alert you to their promotion schedule, the types of customers they are pursuing; the products and/or services they are emphasizing most heavily, what markets they’re in, and so forth. You might also be able to pick up the phone and talk to your competitors directly. It may be that they serve a different niche and that there’s plenty of business to go around. Plus, the fact that you are in the same business gives you an affinity that encourages both your competitors and you to help each other out.

Warranty Cards

Encouraging your customers to fill out a warranty card can also provide useful information: contact information, birthdate, age, type of product purchased, and other kinds of information. This will give you an idea of the type of customer that buys your product. Also, if customers invoke the warranty at some point, you can also get some idea for the products that are having the issue, the types of customers it has been happening with, and the most frequently occurring defects.

Previous Promotions

Look back at some of the ads you ran. How did they perform? Did you test two types of ads? Which one did better? Knowing which promotional tactics work well and which don’t can ensure that you’re directing your marketing dollars more effectively.

This list is far from comprehensive. You can also obtain primary research from trade and professional associations in your industry, as well as from chambers of commerce. You can also get information from your suppliers/vendors. And just plain old networking can give you information.

Primary research is generally expensive, but there’s so much of it that you’re likely already doing, that you may have a wealth of research right within your walls. Mining that information is like mining gold!

Do you have a lot of information you’re collecting that you’re not using to generate new or repeat business? Are you collecting mountains of information but can’t make any sense of it? Would this kind of primary research be of valuable to you, but you just don’t know where to start? Analysights can get you on the right track. Call us at (847) 895-2565 or visit our website at www.analysights.com.