A question isn’t just about the question
A survey isn’t simply a set of questions. Often there will be context and narrative critical to the response. Never is this truer than in consultations. For years the Gunning Principles have been etched on my brain. One of these principles focuses on providing sufficient information ‘to permit intelligent consideration’. Basically, how do you expect someone to tell you whether they agree with something if you don’t give them enough information to weigh it up?
You can have the greatest set of questions ever seen. But if you don’t get a couple of things right, your efforts may well be in vein. After all, the whole point of developing a survey is for people to answer your questions.
Identify your target audience and come up with a plan to reach them. Some market research ahead of launching a product aimed at people under 25 isn’t going to be much use if you don’t have anyone that age in your sample.
Give your survey period some thought. Allow enough time for you to reach your audience. And think about those times when you could be chasing your tail. Pushing a survey out during the Christmas and summer holidays is usually tough.
Timing is also critical if you’re evaluating something. You need to strike that balance between allowing time for impact and not leaving it too long that the ‘something’ could be a distant memory.
Get ahead of yourself
This is not a typo.
There are times when you absolutely should not get ahead of yourself. Like the 8 year old me in the closing stages of the 95/96 Premier League season, when I thought my team, Newcastle United, was destined to be champions under Keegan. I would’ve loved it too. I’m still not over it.
But there are times when you should. And survey design is one of those times. Whenever I’m developing a survey I always think ahead to the survey closing and ask myself what will the responses to the questions tell me? And how will I analyse it and make it mean something?
This isn’t because I plan to fudge the results. But I like to imagine what the analysis and report might look like to understand if I’m asking the right questions.
Take it right back to the core objectives of the survey or research. Will it actually address these? If not, it needs more work. And there’s no shame in redrafting. I once got to version 19 of a survey. It was miles better than version 1.
Also ask yourself, are there any other relevant data sources? This is also a question to ask yourself at the very start; you might be asking questions you already have the answers to. But it’s also one to keep in mind throughout the process. Is there a national dataset lacking a local sample that you would like to compare against? Is there historic local data to compare against?
So, what’s the point?
The point is if you’re designing a survey, or any research for that matter, don’t rush. Question design is important. Once they go out, you’re stuck with them. Think about your audience, how to reach them and the information they need. Think about the platform, what works and what doesn’t. Think ahead to the analysis and how your questions link to your objectives.
The analysis of any research tends to gets the most attention. It’s the end goal. Powerful findings, new insights.
The fieldwork and promotion tend to get the most resource. After all, you need some data to work with.
The value of planning and design can be overlooked or compromised in an effort to ‘get it out’. But these are your foundations. If they’re not right, like a house, your research will fall down.