How large should a survey sample be? Not as large as you think!

When conducting a survey, one of the key challenges a company faces is determining the appropriate sample size.  How many people should they survey?  Quite often, clients resort to subjective judgments, based on budget, past business processes, or corporate politics.

Some clients find safety in large sample sizes, and request 1,000 completed surveys.  But is a sample of 1,000 really necessary?  It depends on the level and sophistication of analysis the client needs to do.  If, for example, the client wants to compare market sizes for its product in five or six geographic regions within the country, a sample of 1,000 might be ideal for that level of analysis.  But what if the client is comparing at most two or three groups?  A sample size of 1,000 would be both overkill and a waste of money.

A scientific way to determine the ideal sample is to use a confidence interval approach.  This approach requires the client to know just three things: the desired level of confidence (a 95% confidence level means that if a sample was randomly drawn 100 times, we can be confident that 95 of them will contain the true population parameters); the variability (the degree to which respondents’ likelihood to answer your survey are similar or dissimilar to one another); and the desired level of error.

In most business cases, confidence intervals of 95% and 99% are common.  Insofar as variability is concerned, if you have no idea of the variability, you should assume maximum variability (50-50 chances).  And a 5% margin of error is pretty standard.

So, for a 95% confidence level, with maximum variability, and a 5% margin of error, a client would need only a sample size of 384.  At a 99% confidence level, the required sample size would only be 663.  Both are well below 1,000, and provide high levels of accuracy.  And a lot cheaper!    

But notice one thing: when the client increased its confidence by just 4% points, it needed to survey 279 more people!  The sample size had to be increased dramatically for just small increases in accuracy!  Simply put, the accuracy gained diminishes for each one-unit increase in sample.  In this case, the client needs to decide whether the benefit of the additional confidence justifies the cost of surveying an additional 279 people.  If millions of dollars are at stake, then the additional cost is justified.  If relatively few dollars or resources are at stake, probably not.

Your ideal sample size is that which provides you with the level of accuracy you need for the value you expect to receive.  If each addition to your accuracy increases the value of the research benefit, by all means, increase the sample size until the benefit of that accuracy is maximized.  But in many cases, even that point will be reached somewhere below 1,000.

Tags: , , , , , ,

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: