To ensure a balanced composition, panel members should be recruited from a variety of sources. The panel must cover all population groups and provide the opportunity to target surveys to specific demographic subgroups. This ensures the quality of data collection and contributes to the panel’s capacity.

It is also important that the panel contains a large number of background variables. These provide direct access to narrow target groups, as well as control in recruitment for surveys that are specified and targeted. With a wide list of background variables, you have more opportunities to target your surveys. For the best quality, it is important that the background information comes directly from the members.

Important measures that should form the basis for research panel quality:

A controlled composition of the panel is a quality measure in itself. However, there are other important measures that should also form the basis for ensuring the quality of the panel. This is something Norstat has implemented and is working on daily:

• Duplicate checks of members
• Automated systems that detect and prevent fraud
• Checking response quality through regular monitoring questions
• Manual checks for suspicious answers
• Detection and prevention of cheating and fraud
• Speed detection and exclusion
• Attention tests
• Removal of inactive members

There are many panels to choose from on the market, but not all of them focus on quality or have systems that facilitate this. Ask yourself these questions the next time you consider using a research panel:

1. Is the panel built as a “probability panel”? Does the panel’s distribution reflect the population? This provides better and more precise sampling features and is a prerequisite for representativeness.
2. Is the panel sufficiently large and capable? The size of the panel influences quality and capacity. Smaller panels are more likely to have biases and imprecise distributions, especially when the target audience is narrow.
3. Does the panel provider allow self-recruitment? This gives less control over member recruitment and increases the risk of “cheaters” and skewed distribution.
4. Does the panel provider have a “double-opt-in” recruitment process? This helps detect and reduce cheating and duplicates by requiring members to verify their mobile number.
5. Our recommendation is to always go with a single panel provider! If your panel supplier uses so-called “panel blends” (data collected from a combination of panels), the control and oversight of the data basis is weaker, and the likelihood of skewed distributions and lack of control is higher.

Paal Melbye, Head of Business Development, Norstat, on what he thinks is the most important thing to consider before choosing a research panel.

“Choose a research panel that has the same gross distribution of gender, age and geography as the population. There aren’t many of them. Make sure your data comes from one panel, avoid “panel blends”. By using a quality panel, you will get data that can be trusted and that is representative – i.e. valid for the population you are studying.”