Wade Armstrong

• Latest
• Archives
• Links
• About Me
• Projects
• Contact Me

Easy Survey Design

Posted Monday November 6, 2006 in Business

Gaining customer and stakeholder insights is an essential task for any company; yet the tools of which a smaller company can avail itself to learn these insights are limited. One of the best, easiest-to-use, and cheapest of these tools is the survey, but many companies design ineffective survey instruments that provide limited help in answering major business questions. Here’s the simple top down/bottom-up survey development methodology that has worked on dozens of surveys I’ve put in the field.

Now, many larger companies have someone on staff who has been trained specifically in survey development and administration, and I’d highly suggest you listen to experts in the field at your company if you have them. But, if you don’t have such an expert at your company, hey, you have one here — I was trained as an undergrad and as a grad student to design and administer surveys, and I’ve designed and administered surveys for dozens of companies, non-profits, national associations, and even governmental organizations. There’s a lot of technical knowledge that goes into designing a good survey, and working directly with an expert always helps, but if you don’t have access to, or can’t afford, an expert, the following approach will help you get the most out of the skills and information you have.

(Quick note: none of the above tackles complex questions like statistical validity. Following this methodology will get you a survey that works reasonably well, but it’s not a complete approach.)

Common Survey Mistakes

A typical, spontaneous survey development method often creates a survey that doesn’t work out well. Some of the most common problems encountered with poorly-planned surveys include:

• “Oh, This Seems Like A Good Question!”
• Insufficient Planning = Insufficient Data
• Write First, Figure Out How to Analyze Late

“Oh, This Seems Like A Good Question!”

One common approach is just to brainstorm questions for the survey and put them in, without any kind of particular order or goals. Brainstorming is often a good approach to use to develop survey goals, but throwing in varied questions willy-nilly leaves you with nothing other than a disorganized survey that confuses subjects and won’t fulfill your specific needs.

Insufficient Planning = Insufficient Data

A poorly-planned-out survey can leave you with big questions even after you’ve finished analyzing and digesting the data. There can be advantages to having multiple contacts with your customer via a survey, but coming back to the well again and again because you don’t get the results you need is not a good strategy. It takes time and money to administer every survey, so get good results — get the answers you need the first time.

Write First, Figure Out How to Analyze Later

Even good questions can turn out bad if you can’t turn your answers into specific, actionable data. Properly-designed, results to various questions can be compared and contrasted, good options identified, and clear answers gained from the responses you get. But, if you don’t plan out your analysis first, you may find that questions don’t actually give you results you can use — you can’t distinguish between preferences, customer feelings are unclear, and, at best, you spend hours re-coding results to allow any kind of analysis at all.

These three problems can result in a survey that doesn’t accomplish its key goal and doesn’t deliver you business value. Let’s look at that goal — the goal every single business survey shares and how pursuing it can deliver real value.

The Goal Every Survey Shares

Every survey distributed by a business is designed to inform some business decision. That decision may be “do we produce model x100 or 47Z?” or “what should R&D work on” or “how much should we charge”, but there’s always a business decision that needs to be made. Since making this decision is your goal, your entire survey should be designed around the decision.

What Do You Need to Decide?

The first step here is to be clear about the decision you need to make. Write it down on a whiteboard, run it by a few people, make sure that you’ve expressed it as clearly and simply as possible. You’ll do yourself a favor if you’re clear on what the range of possible options is.

What Information Do You Need to Make that Decision?

So: why can’t you make that decision now? What else do you need to know? In general, you need to gather certain information before you can make that decision well. What specific information would be sufficient to make that decision? Who can you get that information from?

The Pyramid

The Pyramid is a methodology I’ve used to successfully write surveys for dozens of companies, non-profits, national associations, and even governmental organizations. You start from the apex and build down; when you’ve reached the ground, your survey is ready to hit the streets, and, when analyzed, your survey will help you build back up to the apex — making your business decision.

Working Backwards

So, start with the business decision. From there:

1. What big questions do you need to answer to make that decision?
2. What are the possible answers for those big questions? Enumerate all of the important ones.
3. Develop survey questions that allow you to measure, or compare, the answers above. You’ll typically need a number of questions that lies between “one survey question that compares all of the options” and “one survey question to measure the accuracy of each possible answer to the big questions that confront you,” although, if some of the answers are complex multi-part concepts, you might need a question or two for each concept. Each survey question should give you, or at least contribute to, a fairly clear yes, no, or maybe answer to one of your big questions.

Because you started with your business decision, your analysis can follow the reverse path:

1. Collect responses.
2. Analyze responses to determine which answers to the big questions are correct.
3. Plug the answers to your big questions into your decision-making process.
4. Decide!

Again, because you started with your business decision, you can be sure that, at the end, you’ll be able to come back and make that decision, because you’ll have a complete, meaningful set of data — complete and meaningful as defined by you.

In later entries, I’ll look at some tips and tricks about survey administration — the part where you get your nice new survey instrument out in the field and actually gather the useful information — and some hints at how to write good survey questions for different kinds of business questions. But the Pyramid is robust, and you’ll get good results using it in your next survey even if you have some weaknesses in your questions or sample.

Powered by Movable Type and PmWiki
Hosted By DreamHost
Copyright © Wade Armstrong