Not even the finest musician can coax wonderful sounds from an instrument that’s poorly constructed or out of tune. If you want your survey instrument (the questionnaire) to yield information that’s clear, on-key and in harmony with your purposes, remember three basic parameters: Keep it short, simple, and professional. Review your draft through a survey respondent’s eyes, and you’ll see how important this is.
1. Keep it short.
Conventional wisdom used to be that people would complete a 20‑minute online survey. These days most market research professionals, including this one, recommend keeping it under 15 minutes. Five minutes is better yet. In fact, most AYTM panel surveys are completed in around five minutes. That might sound extreme, but you can collect a lot of information in five minutes. Besides, seven to ten solid questions in a five-minute questionnaire can get you better data than a 20-minute questionnaire that gets answered without the full focus and engagement of the respondent. Indeed, shorter is better, for three reasons:
- Response Rates and Completion Rates. You want to maximize the number who say “Sure, I'll do that!” when they see your survey invitation. People are more likely to agree to spend five minutes of their time than 15, so a shorter survey will maximize your actual response rate. It also helps your completion rate. Imagine the nice person who is willing to start your survey, but two or three questions in, starts getting a little bored… then the doorbell rings, the baby cries, the dog barks...and they’re gone. Now you have an incomplete survey, never a good thing. Think about this the next time you want to survey your list and you don’t have AYTM recruiting your respondents.
- Attention Span. You want to make sure that people read your questions, answer options and instructions carefully. Let's say you're doing some research to gather reactions to a new product concept. You want honest responses to a carefully worded description of your concept. You want your survey participants to read that description word for word, not skimming it. If they've already answered 15 questions before they get to it, guess what? They've started to zone out a bit. If you want people to read your carefully crafted questions, keep your survey short.
- Accurate Communication. Keeping it short increases the odds that people will follow instructions correctly. Is this a “select all that apply” question or a “select one” question? AYTM’s UI design helps enforce that logic, but you don't want people selecting only one item because they assumed that was the requirement when they could have selected more than one, resulting in a more accurate description of the behavior or attitude you’re measuring.
2. Keep it Simple
Remember, we're talking about online surveys here. Look through your respondent’s eyes: I’m on my computer, looking at a survey screen, maybe on a laptop or tablet with a very small screen. The first question of the questionnaire is a dense mass five lines long. Am I really going to read through that when one mouse click can banish it from my life? Besides making an unpleasant visual impact, those five dense lines suggest an onerous experience, not a simple, enjoyable task. People shouldn't have to work hard to take your survey. Now you understand why AYTM has taken a Twitter-like approach and enforces short character limits on questions and answers. This also explains our obsession with the user experience and UI design.
In addition to general wordiness, another common mistake to avoid: asking questions that require a lot of effort. Let’s say you're doing research on shopping behaviors, and you ask me to please go find my most recent clothes shopping receipt and tell you how much I spent on the last shirt or pair of jeans I bought. You’re making me work too hard, and I am likely to drop out or just guess. Instead, give me an approach that's simple and quick to respond to, but that will give you information precise enough to be useful. Maybe this would work instead, “Including tax, was the most recent shirt you bought less than $10.00, $10.00 to $20.00, $21.00 to $50.00, over $50.00?” Those ranges will be a lot easier for me to respond to.
3. Keep it Professional
The tone of any survey should be conversational—a comfortable discussion between friends sharing honest opinions. But do keep it professional.
- Triple check. A questionnaire with spelling and grammatical errors tells potential respondents that you want their opinion, but not enough to properly edit your questionnaire.
- Avoid intrusive questions. I recently saw a questionnaire draft in which the survey asked not just for net worth ranges and how much equity respondents had in their homes. Such intrusive questions lead respondents to wonder if this is a legitimate survey or sneaky lead generation—it’s a huge turn off.
Keep it Short, Simple, Professional
Keeping your instrument short, simple and professional will maximize responses, minimize dropouts and leave survey participants feeling good about their experience. So tune your instrument to perfect pitch, and your reward will be the sweet music of accurate data, and a satisfied audience.