Results from Nonprobability Samples Can Be Quite Useful, Despite Limits

In marketing research, samples are often taken of the population because surveys of everyone is neither feasible nor cost-effective. Researchers often rely on two types of sampling methods: probability and nonprobability. Probability sampling
methods are those in which members of the population have a known chance – or probability – of being selected for the sample. Nonprobability sampling methods are more subjective. The probability of someone selected for a nonprobability sample is not known or cannot be determined. In fact, the sampling process for nonprobability samples is much less formal.

Wherever possible, researchers prefer probability samples because they give an idea of how effectively the sample represents the population, and their results can easily be generalized to the entire population. With nonprobability samples, however, there’s no way to know how well the sample represents the population; any statistical analysis on the sample cannot be extrapolated to the population. Yet there are times when probability samples are not possible: either funds are not available, or time is of concern, or the population is hard to find or otherwise hidden. In those cases, marketing researchers must rely on nonprobability sampling approaches, like mall intercept surveys, convenience sampling, or referral sampling.

Despite their inability for generalization to the population, nonprobability samples can still provide valuable information. While the findings from nonprobability samples cannot be used for inferential purposes, they can be used for exploratory purposes. For instance, the owner of a sports bar may invite some of his patrons to sample some new beer brands he’s considering putting on tap. He might ask the patrons what they like and dislike about the new brands. If the owner notices that middle-aged male customers seem to like the Imperial Stout brand that is being sampled, the owner may consider the times of evening and days of the week when many middle-aged men are in high patronage at the bar, and then offer the Imperial Stout to see if it has any takers. Similarly, if customers are gravitating towards lighter, or sweeter beers, the owner may look to see if his selections on tap are lacking in these qualities. He may even consider testing more beers of those varieties.  In this case, the nonprobability sampling produced directional insights.

Sometimes, a population is hidden. If, for instance, you were trying to market a home remedy type of product to alleviate an embarrassing condition (such as hemorrhoids, erectile dysfunction, jock itch, or nail fungus) and you wanted to see how receptive people would be for using it, you might rely on a self-selection or referral sampling approach. You might place a special coupon in or on the package encouraging the user to participate in a brief study. You might also invite people to the study through newspaper or billboard ads.

While the people you get to participate in your study may not be representative of all people who suffer from the malady your product claims to remedy, they are nonetheless, from your relevant population. If several people complain that your remedy burns, or leaves a foul odor, or doesn’t last long, it doesn’t matter whether or not the results can be generalized. After all, if you want your product to be well-received by its intended customer, you want to minimize those complaints before you go to market. So then you might look to see if there’s a less abrasive ingredient you can substitute without diluting the effectiveness of your remedy. Another nice thing about these “hidden population studies” is that those people who participate in the study could point you in the direction of others suffering from the condition, enabling you to build a database for future studies and marketing. They may even provide you with insights about the population you’re interested in, so that you can do a full-scale study using a probability sample later.

Although results drawn from nonprobability samples have severe limitations, they shouldn’t be  discounted. Results from nonprobability sampling can provide directional information, lead you to your relevant customer base, enable you to see a problem from a different angle, and discover new ways of approaching a business problem. Such results can even lay the foundation for future probability study surveys. In addition, you can still get a lot of useful information in a short time with limited funds. As long as you critically evaluate your findings by knowing which groups were systematically excluded and which were overrepresented, and verifying that other researchers – using different samples and approaches – have replicated the findings you have generated, you should be OK.

*************************

Help us Reach 200 Fans on Facebook!

Thanks to all of you, Analysights now has more than 150 Facebook fans! We had hoped to get up to 200 fans by this past Friday, but weren’t so lucky. Can you help us out? If you like Forecast Friday – and our other posts – then we want you to “Like” us on Facebook! And if you like us that much, please also pass these posts on to your friends who like Insight Central and invite them to “Like” Analysights! By “Like-ing” us on Facebook, you’ll be informed every time a new blog post has been published, or when new information comes out. Check out our Facebook page! You can also follow us on Twitter. Thanks for your help!

Tags: , , , , , , ,

4 Responses to “Results from Nonprobability Samples Can Be Quite Useful, Despite Limits”

  1. Rutubet Giderici Says:

    Hey great post! thanks Unfortunately I can not reach images of your blog in using Chorome browser.

    • analysights Says:

      Rutubet,
      I’m glad you liked it. I’m not exactly sure what you mean about Chrome browser and reaching images of the blog. To my knowledge, there are no images. Am I missing something?

  2. lilian Says:

    Greetings from Kuala Lumpur, Malaysia.

    I am a DBA student, and shall be doing my thesis on SCM. I am the Procurement Head of a local conglomerate, hence, I intend to use my company’s List of Registered Vendors as the sample for my research. The register has a total 861 companies all crossed reference with the Companies Commision of Malaysia for legitimacy. Do you have any views as to how I can prove that the sample is representative of all the companies in the country.

    thanks.

    • analysights Says:

      Lilian,

      What I would do is to first find out whether the Malaysian government compiles statistics on companies across various industries. Find out what the typical size of the companies are (number of employees, annual revenues, etc.) as well as the composition of firms (percentage that have 1-10 employees, 11-20 employees, and so on), and do this by industry, if statistics are broken down at that level. Once you know what a typical firm within each industry is like, you can compare the government statistics with those of your List of Registered Vendors. If the differences aren’t statistically significant and your List extends to most of the industries for which statistics are compiled, then it is a pretty safe assumption that your sample is representative.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: