Before we start asking questions on behalf of our clients, we recommend the clients to first ask themselves some questions: How will the data be used? What decision are you trying to make? Who do you need to ask? What hypothesis do you want to test?

The answers to questions like these help lay the foundation for putting together a survey that will provide you with the quality data you need to make the right decision, whether you’re trying to better understand brand loyalty or looking for input on package design for a new produ

Some clients have already thought a lot about these things before they reach out to us. That being said, it’s not uncommon for marketers to need a helping hand when it comes to topics, target groups, and testing hypotheses.

Minimising bias

One of the most important considerations when choosing questions is taking steps to minimise bias. Questions that are leading, pre-defined answers, or poorly designed surveys can all lead to skewed answers and data that are not truly representative.

It’s easy to overlook how our perspectives can affect the choices we make in terms of phrasing, question design, and answer choices. People’s personal biases have a way of creeping into how they think and frame their inquiries. Helping identify and address those biases is one of the most important functions of our ongoing dialogue with clients.

With decades of experience to lean on, we can help clients keep questions as neutral as possible and construct surveys with as little bias as possible. While some level of bias may be unavoidable, there are additional steps you can take to make sure your questions minimise the risk of bias in your results.

Keep it simple

At Norstat, we have a responsibility not only to our clients but also to our panel members. Our ability to deliver quality data depends on the quality of our panels. Thus, we like to remind clients that survey questions are answered by the real people who make up our carefully curated panels. Just like you, they can be confused by questions that are overly complicated or unclear.

Thus, we recommend that clients keep their questions simple. Word choice matters – and our survey experts can help you make choices that bring clarity, rather than confusion, to respondents. Ask a question you could easily answer yourself. Avoid overly complex wording or trying to ask about two different things with the one question. If respondents don’t understand what you are trying to ask them, the quality of the resulting data may be compromised.

Question formats

In addition to word choice, it’s also important to consider how questions are formatted and presented to respondents. Norstat surveys feature a range of different question types to help our clients collect the type of data they need.

The way questions are presented or displayed can affect how respondents engage with a survey, as well as the answers they provide. If respondents come face to face with a large grid of questions with 20 statements they need to evaluate, it may feel too overwhelming, causing them to drop out or speed through their answers without really thinking. On the other hand, seeing a survey with 20 individual questions may give the impression that the survey is actually longer than it really is.

Generally, we recommend striking a balance between length and simplicity, relying on different formats to help hold our panel members’ interest and keep them engaged. Alternating between different question formats, for example, can help avoid boredom. Less boredom reduces the risk of dropouts that can skew the survey results.

Answer options

Bias can also slip into survey results through pre-determined or poorly thought-through answer options. Even a coherent and varied questionnaire consisting of simple and well-worded questions can result in low-quality data if respondents aren’t given appropriate or comprehensive answer choices.

While pre-defined lists can help provide granular data points for different options, such as the colour of a new package, for example, it’s important to provide an open-ended option to accurately capture people’s sentiments. Otherwise, the preference for one colour may be biased by respondents who preferred a different option but were forced to choose one of the options included in the pre-defined list.

When measuring positive or negative attitudes, it can be useful to allow respondents to answer on a scale rather than a binary “like/dislike” option. By offering a scale, clients can get a better idea of what percentage of respondents have attitudes that lean strongly in one direction or another. They may also discover that a large percentage of respondents are actually indifferent but felt forced to choose one of the two available options.

Delivering the data clients need

To sum up, generating high-quality data you can use relies on asking the right questions. And to ask the right questions you need to consider several factors such as wording, format, and answer options. The first step involves clarifying the purpose of the data and the decisions they will inform. Minimizing bias is crucial, as it can lead to skewed answers and unreliable data.

With Norstat’s Full Service offering, we can help you at every step of the way, from project management and survey design to data collection and delivery. Asking questions is at the core of what we do, both when engaging with our clients to help them figure out exactly what questions they want to ask, and by delivering surveys to our panel members that provide our clients with the answers and high-quality data they need to make the right decision.