What Are Your Research Partners Doing to Ensure Data Quality?

At CMB, we strive for the highest standards of research and data collection. But all of us are dealing with historically high rates of data fraud. While there are many ways of mitigating your data fraud risks, when you are looking to implement strategic projects that involve data, considering asking these questions  to your trusted research partners to ensure that they’re doing all they can to combat data fraud, including:

1. In field reports, are you willing to share total completes and subsequent “actual” completes [how many were lost after quality check (QC) process?] Can I routinely see into the QC process and how many respondents are being removed and replaced?
2. Which third party sample fraud tools are you aware of? Which ones are you investigating? Which ones are you using? Do you have agreements in place with specific sample vendors for this?
3. Do you have success stories to share that illustrate your ability to tackle fraudulent data?

Communicating openly about the percent of fraudulent respondents uncovered is akin to a badge of honor these days, and clients appreciate the honesty and transparency in our process. Finding and removing imposters with ever-improving tools can be achieved. In B2B research, this is even more necessary because much higher incentives attract much more sophisticated “incentive-harvesting” operations. For tough audiences in Tech, B2B “panels” are often abandoned and instead rely exclusively on Expert Networks and other specialty recruiters to ensure data quality.

Kudos to all those implementing third party tools. Speaking from experience, it’s tough onboarding these data tools. If sample providers and third-party vendors do not have existing agreements in place, it can wreak havoc on study launches. There are plenty of moving parts on a questionnaire script with invites, redirects, and quotas. When additional third-party programming is implemented, there are new potential points of error with the up-front data fraud screening. On a recent study where we had multiple tools implemented with one third-party vendor, three individual tools in the programming were not working with the redirects. In cases like this, if programmers and vendors are not properly aligned, they may substantially delay a study launch.

Having multiple QC measures in place to prevent fraudsters from making it to your survey is a lofty but attainable goal. While it may not sound like success, revealing that 50% of the respondents were proactively prevented from taking the questionnaire with third-party tools and another 10% of completes failed internal QC and needed replacement is an example of success when you know you’ve improved data quality to an extent that could have altered the key findings of a study.

So, what are the best pieces of armor for protection? It’s most likely that the more layers of protection implemented, the better.

Are your partners using a three-pronged approach at minimum?

1. A strong, smart questionnaire containing red herrings and other internal QC pieces
2. Third party sample fraud tools as appropriate
3. Internal, proprietary QC protocols to ensure the best data quality for your research

Let’s abandon the old trope of “garbage in, garbage out.” How about high-quality respondents into the questionnaire, quality data out? Time to ask your vendors what they are doing to help combat data fraud. Transparency is key. And if your vendors are failing you, you can rely on your partners at CMB to do what’s right. Contact us.