Pizza, burgers and bratwurst, and the magic of statistics

I was recently involved in an interesting discussion about how to interpret survey results. This is the gist of the discussion.

Let’s say you have a three question survey, “Do you like pizza,” “Do you like hamburgers,” and “Do you like bratwurst.”

You sent the survey out to a whole bunch of people, and exactly 500 people responded. However, not every person answered every question. All 500 people answered the pizza question. However, on the “hamburger” question, 225 people said “yes,” 250 said “no,” and 25 left the box blank.

Should you report this as 225/500 = 45.0% like hamburgers, reflecting that 45.0% of all those people who answered your study like burgers? Or as 225/475 = 47.4% of the people who answered that question like burgers? Which is it, 45.0% or 47.4%?

The correct answer, of course, is that you could respond either way, as long as you accurately explain what the percentage represents.

The most thorough response would be as 225/500 = 45.0% said they like burgers, 250/500 = 50.0% said they didn’t, and 25/500 = 5.0% didn’t specify.

Or you could say that, of those who replied to the question, 225/475 = 47.4% said they like burgers, while 250/475 = 52.6% said they do not like burgers.

But you can’t always give a long, thorough answer. If you’ve got to throw out one number, should it be 45.0% or 47.4%?

It gets more complicated when we look at bratwurst. Nine people said they like bratwurst, one person said he didn’t, and everyone else skipped the question.

If you just look at those who answered the question, then 9/10 = 90.0% like bratwurst, while 1/10 = 10.0% don’t like bratwurst. Those sausages are definitely popular, by a factor of ten-to-one.

If you look at it the other way, then 9/500 = 1.8% like bratwurst, 1/500 = 0.2% don’t like it, and 490/500 = 98.0% didn’t specify. Looks like very few people like bratwurst.

When you’re asked for the sound bite, “Do people like bratwurst?” what’s the answer? Do they (90.0%) or don’t they (0.2%)?

That all seems hypothetical, but what if you needed to make a decision? Perhaps you have to decide what to make for dinner based on what’s more popular, bratwurst or hamburgers.

If you consider only those who answered the question, then 90.0% like bratwurst while only 47.4% like hamburgers. Bratwurst wins. But if you look at it another way, 45.0% like hamburgers while only 1.8% like bratwurst. Hamburger wins.

What’s for dinner: burgers or bratwurst?