Each morning, I awake to my favorite radio station, and the last few days, I’ve awakened to a commercial about a teaming up of Feeding America and the reality show Biggest Loser to support food banks. While I think that’s a laudable joint venture, I have been somewhat puzzled by, if not leery of, a claim made in the commercial: that “49 million Americans struggled to put food on the table.” Forty-nine million? That’s one out of every six Americans!
Lots of questions popped into my head: Where did this number come from? How was it determined? How did the study define “struggling?” Why were the respondents struggling? How did the researcher define the implied “enough food?” What was the length of time these 49 million people went “struggling” for enough food? And most importantly, what was the motive behind the study?
The Biggest Loser/Feeding America commercial is a good reminder of why we should never take numbers or statistics at face value. Several things are fishy here. Does “enough food” mean the standard daily calorie intake (which, incidentally, is another statistic)? Or, given that two-thirds of Americans are either overweight or obese (another statistic I have trouble believing), is “enough food” defined as the average number of calories a person actually eats each day?
I also want to know how the people who conducted the study came up with 49 million people. Surely they could not have surveyed so many people. Most likely, they needed to survey a sample of people, and then make statistical estimations – extrapolations – based on the size of the population. In order to do that, the sample needed to be selected randomly: that is, every American had to have an equal chance of being selected for the survey. That’s the only way we could be sure the results are representative of the entire population.
Next, who and how many completed the survey? The issue of hunger is political in nature, and hence is likely to be very polarizing. Generally, people who respond to surveys based on such political issues have a vested interest in the subject matter. This introduces sample bias. Also, having an adequate sample size (neither too small nor too large) is important. There’s no way to know if the study that came up with the “49 million” statistic accounted for these issues.
We also don’t know how long a time these 49 million had to struggle in order to be counted? Was it just any one time during a certain year, or did it have to go for at least two consecutive weeks before it could be contacted? We’re not told.
As you can see, the commercial’s claim of 49 million “struggling to put food on the table” just doesn’t jive with me. Whenever you must rely on statistics, you must remember to:
- Consider the source of the statistic and its purpose in conducting the research;
- Ask how the sample was selected and the study executed, and how many responded;
- Understand the researcher’s definition of the variables being measured;
- Not look at just the survey’s margin of error, but also at the confidence level and the diversity within the population being sampled.
The Feeding America/Biggest Loser team-up is great, but that radio claim is a sobering example of how statistics can mislead as well as inform.