‘You can’t make the right decisions without the right data’

The market research industry has evolved a great deal since Erling J.B. Eriksen joined Norstat back in 2005. In the years since, an increasing share of people’s lives has migrated online.

While online behaviours can generate lots of data, the trend created something of a challenge for market research companies accustomed to simply opening a phonebook and calling people to get the data they needed.

“It’s harder and harder to find fully representative registers of people that you can reach,” explains Eriksen, who took over as CEO of Norstat Group in 2021. “What used to be offline is now digitalized and that’s one of the main reasons the industry started recruiting panels consisting of people who have given their consent to be interviewed online”

The magic of statistics

The use of consumer panels is now common in the industry. But not all panels are created equal.

And when Eriksen begins to describe the Norstat approach to consumer panels, it’s hard not to conclude that he considers them one of the company’s most prized strategic assets.

“The foundation of everything we do is built on delivering high-quality, representative data you can trust. And our panels are central to that mission,” he explains.

“You can’t make the right decisions without the right data. That’s where it all starts.”

The “magic of statistics”, says Eriksen, is that interviewing 1,000 people and aggregating those responses to an entire population allows one to say with 95 percent confidence “this is what this population thinks or feels”.

“Statistics can deliver the truth, basically. But this statistical magic only happens – this ability to extrapolate the responses 1,000 Swedes to the entire country, for example – if you have a truly representative sample,” he explains.

“If those 1,000 Swedes aren’t representative, you get skewed data – and then you can’t be sure that the results are really an accurate reflection of how Swedes think or feel about the issue at hand. You get something other than the truth.”

Representative consumer panels

That’s why Norstat invests so heavily in creating – and curating – the consumer panels that are truly representative to provide answers to questions about everything from packaging design to public policy.

“Our panels are proprietary. We own them and have control at every step, from recruitment to survey collection, ”says Eriksen.

While some market research companies rely on open panels that anyone can join, Norstat’s panels are closed.

“You can’t just raise your hand and volunteer to join one of our panels. It’s by invitation only,” he explains.

“We strive to mirror the population as a whole because high-quality data equals representative data. And if you want to predict or understand something, you need to use data that’s representative of the population you’re looking at.”

A high-quality panel bring high-quality data

Norstat also takes great pains to minimize the risk of fraud or “unserious” responses through measures such as 2-step verification, dummy questions, and monitoring how much time it takes panellists to complete a survey.

“If a panellist spends too short time on a survey, they get a warning – gets excluded from the sample – and repeat offenders are kicked out,” says Eriksen. “We also operate as a default with cheater and unserious detection algorithms that works across surveys and panels. Everything to ensure the highest possible quality data”.

“It’s not only important to control who gets recruited to the panel, you also need to control what happens inside the panel.”

Thus, Norstat strives to provide panellists with a positive experience every step of the way in hopes of cultivating a culture of quality and sense of belonging among its panellists.

“You need to treat panellists like people and not like cattle,” he says.

“If we expect them to provide us high-quality responses, we need to provide them with a high-quality environment. The questions need to make sense, be in the local language, and they need to know they can get quick feedback from us when they have questions of their own.” The entire quality culture that surrounds the Norstat panels contributes to the giving the clients accurate representative data.

The high costs of skewed data

While recruiting and managing Norstat’s closed consumer panels is resource intensive, Eriksen wouldn’t have it other way.

“If you don’t have measures to protect against fraud and encourage participation, you don’t get the representative data you need to ensure your delivering truthful answers to your clients,” says Eriksen.

“And if you don’t do that, you end up making the wrong decisions. It’s as simple as that.”

And making the wrong decisions can have serious consequences.

To illustrate his point, Eriksen describes a hypothetical scenario involving a consumer goods company that needs to bring new products out quickly and thus conducts a lot of product testing before deciding to launch at scale.

“They test for taste, colour, price, package design – a range of different attributes that could affect consumer behaviour,” he explains.

“And if you deliver the wrong data – data that’s not representative – you can end up with a production line making green bottles when the market actually wants blue cans. And bad decisions like those can cost millions.”

Delivering ‘the truth’ with high-quality data

Skewed data can not only lead to costly decisions for businesses. It can also lead policymakers to support initiatives that lack support among voters or give media outlets the wrong assessment of which candidate is likely to win an election.

Delivering a representative sample is the fundament for high quality data. The job Norstat does for their clients also involves advising on the questionnaire, making sure the layout of the surveys is mobile-friendly and appealing, translate correctly and of course testing the logic between the various questions. Making sure the correct information is derived from respondents can be complex job, but one Norstat is happy and capable of helping their clients with.

At the end of the day, says Eriksen, Norstat’s job is to provide businesses, organizations, governments, and society at large “with the truth”.

“That is a huge responsibility that we obviously take very seriously,” he says.

However, Eriksen emphasizes that Norstat’s mission begins and ends with delivering the high-quality data decision makers need.

“We’re not telling any decision maker how to act upon the data, but we need to provide the truth about what people are thinking and feeling,” he explains.

“That’s how we help decision makers know their world, and it’s that knowledge that helps them make the right decision.”