Vague Language – One of the Top 10 Mistakes Survey Writers Make

Editor’s Note: This is the fifth in a series of posts on common Survey writing mistakes. Click here to see the Mistake #4 and here for Mistake #6!

MISTAKE #5: YOU USE VAGUE LANGUAGE OR IMPRECISE WORDING

One of the most (in)famous  Survey “urban legends” is the story of the Soap Survey. It goes something like this: Several years ago a large consumer goods company contracted with a market research firm to study how their brand of dish soap stacked up on the marketplace against their competitors’ products. The research firm identified unemployed married women  between the ages of 20 and 40 with young children – aka “stay-at-home moms” – as the most influential decision-makers in household soap buying and commissioned a Survey of thousands of them to learn more about their preferences and buying habits.

The Survey took months to run and cost hundreds of thousands of dollars. The central question of the study was:

What is your favorite soap?

This was set up as a free response question – i.e. respondents were asked the question and allowed to answer however they wanted. Unfortunately, as soon as the results were compiled the Survey writers realized that they had made a horrible mistake. You see, although the stay-at-home moms had a clear favorite soap, it had nothing to do with dishes. The #1 response to the question was…

Days of Our Lives

Other popular responses included The Young and the Restless, The Bold and the Beautiful, and General Hospital. Palmolive, Dawn, and Ajax barely factored. The results were meaningless and the Survey was a complete waste of time and money.

What happened? Assumption bias caused by vague language. The writers of the Survey were so immersed in the topic that they stopped thinking of it as “dish soap.” To them, “soap” and “dish soap” had become synonymous, and they didn’t even think about what the word might mean to the people taking the Survey.

The most challenging aspect of Survey writing is anticipating all of the ways in which a respondent might interpret the wording of your questions. I call this projection, because I think of it as trying to project myself into the mind of the reader so I can read the questions from their perspective. If the Soap Survey writers had done a better job of projection, they might have realized that “soap” had multiple meanings to a stay-at-home mom. In fact, not only did they fail to recognize that soap might be interpreted as a “soap opera,” but they also neglected to differentiate between dish soap and all of the other types of soap used in the home!

Vague language and imprecise wording can doom even the best-constructed Survey. Choose your words carefully, define all terms as precisely as possible, and look for (and eliminate!) every possible way in which questions could be misinterpreted. If you do this, you just might avoid running your own Soap Survey.

By | 2017-09-05T01:48:13+00:00 July 14th, 2014|News, Tips & Tricks, Uncategorized|2 Comments

About the Author:

Wyatt Nordstrom is the co-founder and CEO of Maven. When he’s not mapping the world’s expertise he likes to row, brew beer, and make BBQ.

2 Comments

  1. Robin Rapture July 15, 2014 at 9:14 am - Reply

    Thank you Wyatt! This is a great recommendation for administrators working on customer feedback through surveys. My favorite find while doing research on writing effective surveys was from F. Thomas Juster, who devised the 11-point buyer intention scale. In my next survey, I will also keep your language tip in mind. *Cheers -RR

  2. Wyatt Nordstrom July 15, 2014 at 3:48 pm - Reply

    Thanks Robin! Be sure to check out the other tips as well. I’ll publish five more over the next couple weeks.

Leave A Comment

Interested in becoming a Maven customer?Request a Demo