Posts Tagged ‘online marketing research’

Beware of “Professional” Survey Respondents!

April 3, 2009

Thanks to the Internet, conducting surveys has never been easier.  Being able to use the Web to conduct marketing research has greatly reduced the cost and time involved and has democratized the process for many companies.

While online surveys have increased simplicity and cost-savings, they have also given rise to a dangerous breed of respondents – “Professional” survey-takers.   

A “professional” respondent is one who actively seeks out online surveys offering paid incentives – cash, rewards, or some other benefit – for completing the survey.  In fact, many blogs and online articles tell of different sites people can go to find paid online surveys.

If your company conducts online surveys, “professionals” can render your findings useless.  In order for your survey to provide accurate and useful results, the people surveyed must be representative of the population you are measuring and selected randomly (that is, everyone from the population has an equal chance of selection).

“Professionals” subvert the sampling principles of representativeness and randomness simply because they self-select to take the survey.  The survey tool does not know that they are not part of the population to be measured, nor their probability of selection.  What’s more, online surveys exclude persons from the population without Internet access.  This results in a survey bias double-whammy.

In addition, “professionals” may simply go through a survey for the sake of the incentive.  Hence they may speed through it, paying little or no attention to the questions, or they may give untruthful answers.  Now your survey results are both biased and wrong.

 Minimizing the impact of “Professionals”

There are some steps you can take to protect your survey from “professionals,” including:

  • Maintain complete control of your survey distribution.  If possible, use a professional online survey panel company, such as e-Rewards, Greenfield Online, or Harris Interactive.  There are lots of others, and all maintain tight screening processes for their survey participants and tight controls for distribution of your survey;
  • If an online survey panel is out of your budget, perhaps you can build your own controlled e-mail list (following CAN-SPAM laws, of course).  E-mailing your survey is less prone to bias than keeping it on a Web site for anyone to join.
  • Have adequate screening criteria in your survey.  If you can get respondents to sign in using a passcode and/or ask questions at the beginning, which terminate the survey for people whose responses indicate they are not representative of the population, you can reduce the number of “professionals”;
  • Put “speed bumps” into your survey.  An example would be to have a dummy question inside that simply says: “Select the 3rd radio bottom from the top.”  Put two or three bumps in your survey.  A respondent who answers two or more of those bump questions incorrectly is likely to be a speeder and the survey can be instructed to terminate;
  • Ask validation questions.  That is, ask a question one way and then later in the survey ask it in another form, and see if the responses are consistent.  If they’re not, then the respondent may be a “professional” or a speeder.

The Internet may have made marketing research easier, but it has also made it more susceptible to bias.  The tools to conduct marketing research have become much easier and more user-friendly, but that doesn’t change the principles of statistics and marketing research.  Online surveys, no matter how easily, fast, or cheaply they can be implemented, will waste time and money if those principles are violated.

Online Marketing Research is Great, but it’s not the be-all and end-all

March 20, 2009

Today I came across a blog post by “Lawn Mower Belts” entitled “Role of Research in Online Marketing.”  Lawn Mower Belts stresses the importance of conducting marketing research, but seems to suggest that doing the research online is the best approach.

While the Internet has indeed made the collection of data much faster and easier, it has its own share of problems.  Doing a survey has never been simpler, thanks to the Internet.  But that does not change the principles of marketing research;  you still need to define your business problem, you must still decide the population you wish to research, and you must still select (randomly) a representative sample to whom you administer the survey. 

Even if using the Internet to conduct secondary market research, you must still keep in mind the objectives of the researcher whose work you referenced and you still need to assess the findings of that research critically.

To ensure that you get the best outcomes from online research, a good approach would be to:

  1. Start by using the Internet for exploratory research.  Look to see what other people are saying about your market, product type, or industry.  This can help you formulate hypotheses and define your business problem.  You can also informally ask people for in-depth information (qualitative research), which can also help you in this regard.
  2. Define and refine your business problem after #1, and determine your target population.
  3. Think of online places where members of your target population are likely to congregate.  See if you can make arrangements with the owners of those sites to administer your survey (or other research method) to their e-mail lists.
  4. Send your survey to the people on the e-mails you borrow or rent from those sites.

Although this process will still be slightly unrepresentative, it won’t be nearly as bad as if you simply stuck a survey on your web site and let others self-select to do the survey.  Online research may be cheap and easy, but it can cost you time and money if it’s not done carefully.