More Survey Rating Scale Discussions

Yesterday, we discussed situations in which it might be better to use a longer or shorter rating scale for particular survey questions. Today we’re going to build on that discussion, evaluating when it is best to use either a balanced or unbalanced scale; an even or an odd numbered scale; and a forced/unforced choice scale.

Scales in Balance

Most survey questions default to a balanced scale, meaning that there is an even number of points on the low, or negative, side as there is on the high, or positive, side. The agreement scale below is an example of a balanced rating scale:

Yesterday, we talked about how a longer scale is useful if your respondents are likely to skew heavily towards one end of the scale. Another approach would be to use an unbalanced rating
scale like this:

So, if you know that most of your respondents are going to agree with you, breaking down those who disagree is going to be of little value. But if most respondent will agree, the unbalanced scales achieve the same result as the longer-point scales, namely increasing the about of discrimination in their level of agreement. Generally speaking, if you know your respondents will skew heavily to one side, the unbalanced scale approach is preferable to the larger scale. However, if you are uncertain about how respondents will fall and you don’t have the benefit of past surveys or of doing a pilot survey, then go with a balanced, slightly larger scale. At the very worst, you can collapse the scales based on responses.

Important Note About Unbalanced Scales!

It is important to note that because there is no middle point in an unbalanced scale, you end up with a scale that is ordinal, as opposed to interval. Hence, you cannot properly compute a mean or average response. You must rely on the median response for measures of central tendency.

Odd vs. Even Scale Points

There is even some debate about whether to use even or odd numbers of points in a rating scale. The two examples above illustrate an odd number of scale points. The scale below is an example of an even one:

There are reasons a researcher might prefer an even scale over an odd one. In this example, it forces the respondent to draw his/her line in the sand about a particular point of view. This is particularly useful if the question is on something a person cannot be undecided about, or if an issue is highly charged. By not having a “Neutral” or “No Opinion” option in the middle of the scale, respondents cannot “cop out” by choosing it. However, forcing respondents to choose may cause some to skip the question, answer it incorrectly, or abandon the survey all together.

Forced vs. Unforced Choice Questions

Even point scales lead us into a discussion of forced vs. unforced choice questions. As you can see, the four-point scale example above shows a forced choice: respondents either agree or they disagree. The balanced odd-point scale in the first example illustrates an unforced choice question. You can still use an even point scale that is an unforced choice. You might place an option for “Don’t Know” or “Not Sure” after the “Strongly Agree” choice, and then code it with a “DK” or an “X”, rather than a “5.” Your reasons for choosing forced vs. unforced choices is largely the same as discussed above. It’s worth repeating that if you force respondents to make a choice, it could increase the incidence of non-response bias or incorrect selection, so choose carefully.

In summary, use unbalanced scales when you know in advance that your respondents will skew heavily on one end of the scale; use even-points and/or forced choice questions on topics for which one cannot be apathetic or for highly charged issues. However, be careful to know these conditions to be the case first. Guessing wrong can alienate respondents and cause you to lose their cooperation and honesty.

*************************************

Let Analysights Take the Pain out of Survey Design!

Rating scales are but one of the important things you need to consider when designing an effective survey. If you need to design a survey that gets to the heart of what you need to know in order for your company to achieve marketing success, call on Analysights. We will take the drudgery out of designing your survey, so you can concentrate on running your business. Check out our Web site or call (847) 895-2565.

Tags: Analysights, average, balanced and unbalanced rating scales, balanced rating scales, balanced vs. unbalanced rating scales, central tendency, even point scales, even vs. odd point scales, interval scale, market research, marketing research, mean, median, odd point scales, ordinal scale, rating scales, survey, survey research, Surveys, unbalanced rating scales