Ask Your Target Market


Get inside the minds of your target market with AYTM's intuitive
online survey tool - the first affordable Internet survey software with a built-in consumer panel. Use the online survey creator to build your survey, define the criteria of your target market, and watch the results begin to pour in immediately.

Friday, April 29, 2011

Planning Survey Samples: Super Sizing Optional


So you’re planning to do an online survey and you’re wondering, “how many completes do I need?”  There is a short answer, but first, some context.

How confident is confident enough?

You want to have enough data so that you can feel confident in the results. The question is, how confident do you want to be? Are you looking for directional data or do you want something that you can calculate how well it actually represents the target population? 

When you watch the news, you sometimes hear polling results. A common description is, “this data is based on a 95 percent confidence interval plus or minus 3 percent.” What does that mean? It simply means that we know that if we were to do this same poll over and over again that we’re confident that the data would be replicated repeatedly with a precision level of plus or minus 3 percent. That’s pretty darn representative, and I’d feel good about that data.

So how would you figure out what that is? Well, if you really do want reliable data, something that represents a broader target market; then you need to have an estimate of that target market’s variability. In fact, one of the biggest myths in market research is that the sample size required is based on the population size (for the population of interest). That’s not quite true. The formula used by statisticians to determine the required sample size is based on the variability of the population. Now, those of us who do a lot of consumer market research know that there are common levels of variability that we’re likely to encounter. Based on this, many market researchers will use rules of thumb. Given the variability we typically encounter in consumer markets, we know that a sample size of 400 to 600 completes will often give us very nice data. In some cases though, that can be overkill. For example, if you’re studying a market where you know (from past research) that most people have very similar opinions and similar behaviors, then you can get away with a smaller sample size and still have representative data. 

POLITICAL REQUESTS

The reality is that sometimes the sample size requirement is not based on objective needs, but on political ones. You may have internal colleagues who are simply stuck on certain numbers when it comes to survey samples—we see it happen all of the time. Some people believe that they must have 500 completes to feel good about the data, or for some the number is 1,000 or more. If budgets permit, it’s certainly nice to have all that data to play with.

SUBGROUP ANALYSIS

If you plan to do any subgroup analysis, that becomes a sample size consideration. For example, if you know you’re going to want to analyze data by gender, you might want to make sure that you have at enough data so that you can compare the two groups. Or, if you have a question in your survey about purchase intent and you have 4 answer options, you may want enough data to use those 4 “buckets” for subgroup analysts. In reality, many market research sample sizes are based on subgroup analysis needs. 

SAMPLE SIZE CALULATORS

There are sample size calculators also, like this one. And if you’re really interested in getting into the statistics of sample size calculations here is a great article.

THE SHORT ANSWER

For some consumer studies, directional data of 100 responses can be used for quick hits. However, if you want data you that can be used to extrapolate your findings to the broader population, then a sample size of 400-600 is preferred…but add in some political needs or sub-group analysis, and you may need to go higher.



Tuesday, April 26, 2011

Market Research for Start-ups: Is My Baby Ugly?

Do you have a new company and perhaps a new product? Online surveys can be a great way to gather important information before you start marketing that product. Sure, you may think your baby is beautiful, but will everybody else? In your heart, you know you need a sanity check. You want a gauge on likelihood of success. And you may want to identify likely sales objections before you actually encounter them. Or, maybe you simply want to know if the product idea is easy to convey; will target customers embrace it readily, or is this going to be a long sales cycle? These are all great research goals.

Here are some examples of cool things you can do with online surveys before your reveal your new baby to the world.

Getting product concept feedback
In an online survey, you can describe a product and ask some follow up questions to get feedback. When you describe your product, keep in mind a few things: 
  • Keep the product description very brief, two to three sentences at most. Nobody wants to read a 100-word product description.
  • Make sure that your description is objective. This is not a sales pitch. If you want honest feedback, then you want to describe your product objectively. Instead of, "Hey we've invented the world's coolest widget; don't you want to buy one?” You want to say, “Here’s a description of a new widget. It has these features. It's different from other products in that it has feature Y.”

After the product description, you can ask some simple follow up questions. You might ask, “If this product were available today, how likely would you be to purchase it?” or perhaps, “How likely would you be to purchase this type of product in the next six months?” Notice that I am specifying time ranges. This is a way to keep it a little less hypothetical. If you simply say, “How likely are you to purchase this product?” It is very hypothetical: ever, in the next year, in the next five years, maybe? You get overly positive results that way, and you really do not want that. You want honest data. By putting in a timeframe, you will help mitigate that risk of getting overly rosy data.

Identifying Demand Drivers & Deterrents
You can also test possible demand drivers and deterrents. Use skip logic (http://askyourtargetmarket.com/pages/writing_and_editing) to have those people who indicate interest to answer a follow-up question and find out why they like the concept. Skip those who indicate little or no interest to a parallel question to measure demand deterrents. 

One approach is to simply ask, “Which of the following are reasons you are likely to purchase this product?” Now, you likely have some hypotheses to offer as answer options. You might have four different reasons for which people will buy this product:
  • I think it will save me time
  • I think it will look great in my home
  • I think that it will last longer than other similar products that I've own in the past
  • I like that it has feature X
  • Other (please specify)

In this example, your online survey would tell you what percent of people agree with each of those statements. Alternatively, you could have each of those items rated on a five-point scale.

Similarly, perhaps you have some hypotheses about what might prevent them from purchasing the product, such as:
§  My spouse wouldn’t want this in our house
§  It takes up too much space
§  It sounds hard to install
§  I don’t have time to use it
§  Other (please specify)
Measuring the prevalence of likely sales objections can be very helpful. Imagine if one of the objections can be addressed easily through modified packaging? Or, by adding or removing a simple feature or maybe just by describing the product differently? This kind of information could save many headaches.

So, Is Your Baby Ugly?

Hopefully not. But with a little customer insight, you will know how to dress your baby for the occasion.


Thursday, April 21, 2011

Selecting Survey Demographics

Demographics are those attributes or variables that describe the population of interest for your survey. Common demographic variables include age, income range, marital status, education level, ethnic group and gender.  The data collected in a survey is only valuable if you know who is answering your questions, so it’s vital to choose demographic criteria wisely when setting up a survey project.

To select demographics for your next survey, use these three considerations:

  1. What is your objective? The step of selecting demographics comes after setting project objectives. Based on those objectives, the type of population you need should be fairly easy to infer. If you’re doing research related to a bicycle accessory product, you’ll probably want people of a certain age range, and probably of a certain minimum income level so they can afford to spend on things like bicycle accessories.  If your product is designed to appeal more to women or to men, then you might choose to target your research toward one gender. Clear objectives will help define demographics.

  1. What is feasible? When selecting demographics for your survey, avoid the temptation of selecting too many. Why is this important?

·               The more demographic variables you require, the higher the cost. This is true with any sample source.
·               A large number of demographic variables can limit your pool of respondents drastically, reducing the feasibility of your project and increasing the study’s cost and timeline.  Just for context, a typical survey project has two to four demographic requirements..
·               An overly narrow focus becomes artificial. Unless you have a very large marketing budget, you likely can’t market only to men who make $50,000 to $150,000 a year, live in suburban areas, have at least two years of college, are married, and are of a specific ethnic group. With too many demographic selections, you quickly reach the point where your survey sample bears no relation to marketing reality.

  1. What does your internal audience need? You need to know the requirements of the people who will use your survey results—the internal clients. Here’s a common scenario:  You’re sharing the results of a survey project with some colleagues and hear disappointed utterances such as, “Oh, you didn’t include higher income ranges?” and “Oh, I see you included people who are retired…” The fact that your internal clients have questions about the demographics used to select participants will make them less likely to use the research results. You need to know early on about any specific demographic requirements so you can consider them in your planning process.

Conclusions: Having clear objectives, limiting criteria for feasibility (and reality), and identifying internal client needs are all important aspects of designing a good survey project. If you take these three items into consideration when selecting demographics, you’ll be able to prioritize the right demographic variables for your next AYTM survey.

Monday, April 18, 2011

How to Read Data


The idea of reading survey results seems simple enough. Until you do it. Even a “short” survey yields a lot of data. How can you simplify it so you can efficiently identify the most important results?
Let’s tackle this for two common situations.

1: Data from single choice questions
These questions are effective when you want people to prioritize a choice. Because each respondent can select only one answer, the responses add up to 100%, giving you an effective “rank order” of responses.  In some cases, one item clearly stands out at the top of the results—and you have a clear “winner.” But what if some of the items are close? Or if no single item really rises above the others?

The brutal reality is that there may not be a “winning” answer. Let us imagine a single choice question related to color preferences. The result may be that blue, green and purple are all within 5% of each other (ranging from 20-25% each), at the top of the list. You might have preferred to see a winner, but the data show that’s not reality among the population of interest. In this case, we have “bands”: band 1 colors are the top 3, band 2 colors are in the middle, and band 3 is at the bottom (those selected by fewer than 10%):
Blue 25%
Green 21%
Purple 20%
Red 12%
Orange 12%
Yellow 6%
Black 4%

Single choice questions don’t always yield a “winner,” and that’s ok. Using the “bands” approach can help you get to the “so what” quickly.

How do you define a band? Think about what is actionable for your topic area. For your topic, will a 10% difference in preferences cause you to make a different choice? To add a different feature? To offer a different color option? In most cases, probably not. A 20% difference is more likely to lead to action. A 30% difference? Almost certainly so. For example, if 60% of your customers like feature A and 30% like feature B, that will likely lead you to focus on feature A. If instead, that had been 40% versus 30%, the data alone would not lead you to choose feature A.

If you want to get really precise, you can import your data into a “crosstab” program and do actual significance testing. But in many cases, that can be overkill.  What most people want is to use the data to make decisions.  Not to know with precision if a 5% difference is attributable to minor variability in the data set.

2: Data from multiple-choice questions
Multiple-choice questions are great for capturing the realistic situation where more than one attitude, behavior or preference may exist. For an example, let’s imagine you have a question about favorite online news sources.  In this hypothetical case, you want to know “all that apply” because of two key hypotheses:
  •   You have a hypothesis that some customers are online news enthusiasts and visit many such sites.
  •  You have a hypothesis that some news sources attract the same people. For example, you think NYT.com and HuffingtonPost.com have a lot of overlap.


So in this case, a “check all that apply” makes sense.

But how would you interpret the data if you offered 5 news sources, and the percents add up to over 300%? That simply means many people checked most of your options, and supports hypothesis #1.
In this case, you may also want to look at some sub groups. This is the “filter by answers” option of AYTM. For example, take all of the people who selected NYT.com, and see what else they checked. Then take all of the people who selected  CNN.com and see what they selected. Comparing sub-groups can tell more of a story than looking only at the data in aggregate. Maybe the NYT subgroup also visits every other site, but the CNN group tends to focus on CNN alone? That would be fascinating!

But what if you offered 5 news source answer options, and the percents add up to over 400%, and that strikes you as a bit high? In such cases, a follow-up question could have been a safety net.  Question 1 would have been a check all that apply, “Which of the following news sources do you visit at least once a week?” And then question 2 could have been a single choice follow up, “Which of the following sources have you visited most recently?”  It gives you the best of both worlds.

In Either Case
In the case of single or multiple-choice questions, be prepared for unexpected results. Percents that are higher than you expected, or lower. And be open-minded.  The data may not tell the story you were hoping for, but it will tell a story. 

Thursday, April 14, 2011

Market Research Surveys: Keep Your Guests Happy


Do you know what’s worse than throwing a party and having nobody show up? Having one where guests are bored and unhappy.

Same thing applies to surveys. Once participants start your survey, you want them fully engaged—not holding up the walls. You want their active participation, so that they carefully read your questions and answer options, and give you thoughtful answers.


Survey Respondent Engagement

What are some strategies for keeping people fully engaged while completing your survey? Here are three.

Tip No. 1: Avoid Scary Starts. Make sure your first few questions are not so onerous that participants are scared off. If the first one, two, or even three questions seem too intrusive or boring, they’ll simply drop out. After all, at this point participants haven’t completed many questions, so they don’t have any real investment. If you do have questions that are a bit more cognitively difficult or lengthy, place them toward the middle or end of your survey instrument. At that point, there is a sense that, “I’ve already invested five minutes in this survey, so I might as well just finish it up.” In any case, we should keep such content to a bare minimum.

Tip No. 2: This Is Not a Test. Avoid questions that sound like you’re quizzing people. Few people have fond memories of tests like the SAT, so make sure the tone of your questions is friendly and simple. For example, examine your word choices. Change the word “utilize” to “use.” The word “enable” often can be “allow.” You get the point. Select words you would use in a real conversation. Avoid sentences that will need to be re-read three times to get the point.

Tip No. 3: Ask for Opinions. People don’t like to simply report what they do; it starts to feel a little intrusive. For example, asking too many questions in a row about what they’ve purchased recently or how much they spent on a recent grocery store trip gets boring and lacks emotional engagement. To keep them involved, include questions that ask for their opinions. Opinion-oriented questions might ask what brands they like best, or what they would like to see in a new product, or “What could this company do to improve your satisfaction?” Such questions are much more interesting. Sure, you may need some of the boring stuff—but mix it up.

Is Your Survey Boring?

Before you send out that survey, and we’ve mentioned this in other AYTM blog articles, have somebody read it out loud to you so you can hear how somebody else would actually read the questions, the instructions, and the answer options. If you find yourself zoning out after a couple of questions, that’s a good clue that your survey guests will be unhappy.




Tuesday, April 12, 2011

Big News from Ad:Tech in SF: Now Access Opinions from Millions of US Consumers through AYTM


 



Ask Your Target Market Integrates 
 uSamp’s SampleMarket 2.0Panel Access Platform to Serve Up Millions of uSamp’s U.S. Survey Respondents To AYTM's DIY Market Research Survey Solution

At ad:tech San Francisco, AYTM Showcases uSamp Relationship and
Unveils Major Product Upgrade

            SAN FRANCISCO (April 12, 2011) –  Ask Your Target Market (www.AYTM.com), a sophisticated new online survey platform, and uSamp (www.uSamp.com), one of the world's fastest growing technology and online sample companies, today announced a partnership that makes uSamp’s proprietary panel of more than 1.7 million screened and vetted U.S. survey respondents available to AYTM clients.  The companies made the announcement at ad:tech San Francisco, here through April 13.
            Powering the partnership is uSamp’s SampleMarket 2.0, the next generation of its market-leading sample access platform, which offers real-time, self-service access to the uSamp panel.  AYTM will seamlessly integrate access to the uSamp panel, with virtually no alteration to the elegant, intuitive user interface that has made AYTM popular with those needing quick, affordable market research surveys.
            “With the AYTM platform, we have created a service that heretofore didn't exist,” said Lev Mazin, CEO and co-founder, Ask Your Target Market.  “While there are web applications offering affordable, self-service email marketing, on-demand printing and online advertising, nothing else provides powerful, self-service access to affordable target market research. 
Our integration with uSamp makes projects of virtually any scale feasible on the AYTM platform, entirely within our environment.”
            For uSamp, the partnership is yet another example of the advantages of the company's technology, through its API (Application Programming Interface) open platform solution, and the quality of its millions of validated and screened panelists.
            “We’re delighted to provide AYTM with real-time access to millions of richly profiled respondents and our robust Sample Market platform interface,” said Matt Dusig, CEO and co-founder of uSamp.  “AYTM has opened an entirely new niche for businesses that previously lacked the access, expertise or budget to conduct market research.   We share that sensibility with AYTM – that the boundaries of market research are now expanding to encompass businesses of all sizes and complexities.”
            Separately, AYTM announced the addition of dozens of highly requested new features to its suite of online survey tools.  Survey authors can now choose among six types of questions, add videos or images to illustrate their survey, and analyze and customize results in more detail.  Agencies will be able to use the AYTM survey platform on a “white label” basis and offer self-branded market research services to their clients as part of their in-house portfolio of services.
            “Thanks to the uSamp relationship and our battery of enhancements, we believe AYTM will now deliver a significantly better market research experience for all of our stakeholders -- researchers, survey respondents and end clients,” AYTM’s Mazin said.  “With our easy interface, pleasing design palette, simple navigation and lightning-fast results, conducting market research has never been this easier.”

About AYTM.com
AYTM.com Ask Your Target Market is the leading innovator in DIY online market research. Only at AYTM can you define your exact target audience by drilling-down into a panel of millions of US consumers and find your ideal research respondents based upon their psychographic and demographic characteristics. Then, write a survey using up to 6 question types, skip-logic, images, and video.  Launch your survey and see results streaming back in minutes. Use our powerful, easy-to-learn analytic tools to help you understand your data in ways that deliver the greatest insight. AYTM has meticulously created the best user experience for researchers and survey takers alike. AYTM offers the ability to survey our respondents or your own. Agencies can even use the AYTM platform under white label and self-brand their research performed on behalf of their clients and benefit from a new revenue stream. AYTM.com market research has never been this easy.


About uSamp
uSamp (www.uSamp.com) is one of the world’s fastest growing technology and online sample companies, providing global survey panelists and an innovative sampling platform for use in market research. uSamp develops collaborative market research tools to foster more rewarding, profitable relationships between organizations and the people they serve. Founded in 2008, uSamp acquired DMS Insights in June 2010 and now has 175 team members worldwide and 5.1 million global market research panelists. The company’s web-based panel platform is transforming the management and delivery of online panel for market researchers, offering unprecedented access over their panel. uSamp’s deep well of proprietary technologies includes SampleMarket™, PanelNet™, PanelShield™, Opinion Place® River and real-time Panel Book Search – cutting-edge solutions for accessing, branding, sampling and managing panels. uSamp is based in Los Angeles, with offices in Dallas, London, New Delhi and Trumbull, CT.

# # #

Media Contacts:

For AYTM
Lisa Santos
(415) 364-8601

For uSamp
Ken Greenberg
Edge Communications, Inc. 
818/990-5001


Sunday, April 10, 2011

Survey Design Tips: Bad Questions Made Good

We can learn a lot about questionnaire design by looking at examples of what not to do. Below are three problematic questions. Take a look: can you spot the errors?
Example 1: Political Activity
Example 2: Social Media Behavior
Example 3: Clothing Purchase
This is from a survey that in previous questions had determined the participant had bought a pair of Brand X jeans within the past 30 days.
Survey Design Problems
Did you spot the errors?
In Example 1, no time frame is specified. Are we asking people if they have really ever done any of these things? Will it be useful if they did these things 20 years ago, but not within the past 5 years? Probably not.
In Example 2, there are a few issues. First, it is unclear if the participant should check one or “all that apply.” The second problem is that some of the items are vague. Researching what? Organizing what? If a lot of people select those items, how useful will that data be? We wouldn’t really know how to interpret it. And the most egregious error in example 2: no options for “Other, please specify” or “None of these.”
While we can’t say from these static pictures for sure, a possible issue with both examples 1 and 2 is whether the lists were randomized or not. Did every survey taker see those lists in the same order? If so, there is a risk that the responses would be subject to “first order bias.” Alas, we know that people tend to focus more on the top of a list than on the bottom. So randomizing lists is important. Of course, always anchor logical choices (like, "None of the above”) at the bottom of a list.
What about example 3? The problem here is that the question is too onerous. We can’t expect participants to hunt for receipts or to have accurate recall. And really, is it realistic for them to subtract out tax and, if applicable, shipping? This question is going to collect some very questionable data.

Questionnaire Design Solutions

These three examples point to some great checklist items for making sure your questions are precise enough to be meaningful and easy enough for the respondent to give accurate information.
1. Specify a time frame. If you are asking participants about past behaviors, or future ones, specify a time frame.
2. Make sure the answer options in a list are complete. When asking people to select attitude and behaviors from a list, consider an “Other, please specify” option. You might get some write-in answers that are unexpected but useful. Also, always offer a “None” or “None of these options.” You don’t want people simply picking an item so they can progress to the next question in the survey.
3. When applicable, randomize.
4. Give clear instructions. For list style questions, always state “Select one” or “Check al that apply.” Even if the survey software enforces logic (so that a person can not select more than one item if it is programmed as a single choose question), you don’t want people to only select one item when really you do want “all that apply.”
5. Keep it simple. Don’t make participants work hard to give you accurate information. If you really do need to collect pricing or budget information, ask for it in ranges. In our jeans example, perhaps something like,” How much did you spend on your most recent purchase of Brand X jeans?” With ranges of:
a. $100 or more
b. $80-$99
c. $60-$79
d. $40-$59
e. Under $40.

Good Surveys, Good Data
Poor questionnaire design can lead to high drop out rates (participants literally dropping out of a survey because it is to long, boring or onerous) and weak data. By following some basic best practices, you will avoid many common questionnaire design problems.



Kathryn Korostoff is the founder of Research Rockstar, an online training company dedicated to teaching market research best practices. She can be reached at KKorostoff@ResearchRockstar.com. For training classes, please visit www.ResearchRockstar.com/classes