How to Prevent Survey Fraud

The point of a survey is to understand the experiences and opinions of the members of a group by asking questions of a representative subset of these people. Surveys can only achieve this goal if those completing the survey are honest about who they are, or if there is a technical way to make lying impossible.

Survey fraud is unfortunately rampant in the market research industry. Surveys these days are typically administered online to “panel members” (people who have signed up to take surveys for money) and there is almost always a financial incentive attached—a quite large one for “B2B” surveys, i.e., surveys about a particular job. A survey targeting cardiologists, for example, could pay $200 for ten minutes of effort. If you are a member of a survey panel, and a survey starts by asking whether you are a cardiologist or whether you have purchased a sports car in the last month, well, you might be tempted to say “yes” even if it’s not true. This happens sometimes, and the impact on research quality is tremendous.

So, what can researchers do? The following is a list of ways to prevent survey fraud. For the most part, these techniques focus on whether a respondent is who they say they are at least in terms of some essential criteria. As to whether respondents are honest about their experiences and opinions, able to remember past events in sufficient detail, or are misled by imperfect survey questions, that is a topic for another day.

Unfortunately, many panel companies also use vague language to suggest a level of rigor in validating respondents’ identities that doesn’t exist. It is likely that every major survey panel operating in the US would say that they validate panelists’ identities (just look at their websites!). The details of what they actually do, however, are essential to understand.

Consider the following a checklist. Does your panel company offer any of the following?

Survey Fraud Prevention Checklist

o   Online activity tracking

o   GPS location tracking

o   Government-issued ID verification

o   ID verification before panel registration

o   ID verification after panel registration

o   LinkedIn profile review

o   VPN blocking

o   Face scanning

o   Digital fingerprinting

o   Facebook or Google sign-on

Online activity tracking – Some panels’ survey apps ask users for permission to track their activity on other apps on their phone. This allows panels to see, for example, whether certain panelists visited nike.com or saw an ad about shampoo or use TikTok frequently. As a result, the panels can target members precisely for related surveys. Want to survey people that you know saw your shampoo ad? No problem.

Pros: Hard to cheat. Your internet activity is what it is. Panelists can get paid simply for sharing their data.

Cons: Many panel members do not want their activity data tracked and will not allow it, even if they get paid a modest amount.

GPS location tracking (different from IP address tracking) – Like activity tracking, this involves allowing an app to track the location of your phone using GPS. Again, you can target panelists for surveys precisely. I.e., I know you went to Wegman’s yesterday. Here’s a survey about your experience.

Pros: Hard to cheat. Panelists can get paid more for qualifying for so-called low incidence surveys.

Cons: A lot of people find location tracking creepy and will not enable this feature. You might, for example, have just twenty percent of your panel available as a baseline for these types of surveys.

Government-issued ID verification – Some panels, like Survey Diem, require panelists to verify their identity by scanning their driver’s license or state-issued ID card. The goal typically is to confirm their location in broad terms (they are located in the US) and prevent one person for signing up for multiple panel accounts.

Pros: Good ID verification services can spot fake IDs.

Cons: This approach can feel invasive. Panels need to reassure people that their name and address are not being sold or hacked.

ID verification before panel registration – Survey Diem requires an ID to be scanned before a user of the Survey Diem app can register for the panel.

Pros: Gets this step out of the way and ensures that all survey respondents have been ID verified.

Cons: Can feel like a barrier to panel registration.

ID verification after panel registration – Some panels wait to verify IDs until the panelist has completed some surveys and wants to redeem their incentives. Other panels reserve the right to verify IDs if they suspect someone is not providing honest data. This latter approach is very common!

Pros: Limits the barrier to panel registration that ID verification presents.

Cons: Many survey respondents will never have their ID actually verified, especially now that it’s getting harder to spot fraudsters.

LinkedIn profile review – Some panels allow you to register by linking to your LinkedIn account. The theory is that fraudsters are not likely to go through the trouble of creating one or more fake LinkedIn accounts. A good B2B panel will also review panelists’ LinkedIn profiles to find the best fits for a study. In other words, they won’t just ask in the survey whether you’re a cardiologist. They’ll already know that you’re a cardiologist from your LinkedIn profile.

Pros: This is an essential step for B2B surveys. I think it’s true that fraudsters are not likely to create fake LinkedIn pages.

Cons: You never know. Anyone can create a LinkedIn profile or several if they really want to.

VPN blocking – A lot of fraudsters taking US surveys are not actually in the US. Most or all survey administration platforms track respondents’ IP addresses, so it’s quite easy to remove bad responses based on IP address. The catch is that it’s easy to use a VPN connection to give you a fake US-based IP address. Fortunately, the most common VPN providers are known and can also be blocked.

Pros: Common VPN providers are known and can be blocked. If you limit a survey to US IP addresses and block VPNs, you are going to cut a lot of the fraud that currently happens.

Cons: VPN providers are getting more sophisticated too. It’s a game of cat and mouse. You can try to block VPNs but can’t guarantee you are doing so 100% of the time.

Face scanning – Technology has developed to such an extent that you can scan and store data about a survey taker’s face at a not-totally-unreasonable cost. The idea is that by scanning someone’s face you can guarantee that a real person is taking the survey (and not a bot) and check to see if that person is taking your survey multiple times.

Pros: Guarantees that a real person is taking your survey.

Cons: This is fairly invasive and doesn’t preclude someone abroad from taking US surveys. Should be used in conjunction with other fraud prevention approaches.

Digital fingerprinting and database reference – A digital fingerprint is a string of characters that is intended to be unique to each digital device used to take surveys. Note that your iPhone doesn’t have an actual number embedded in it called a “digital fingerprint.” A digital fingerprint is made up based on your phone’s model, browser installation, and a variety of other features. There are several databases of digital fingerprints that researchers can use to screen out known bad actors.

Pros: It’s easy to assign a device a digital fingerprint and track survey-taking activities on that device.

Cons: There is no comprehensive digital fingerprint database for the market research industry. I suggest that there should be, but that’s an uphill battle. You can also have someone buy ten iPhones and create ten panel accounts.

Facebook or Google sign-on – It is common for panels to allow people to register by signing into their Facebook or Google account.

Pros: I guess it’s better than nothing.

Cons: It’s easy to create multiple Facebook or Google accounts. I’m not a fraudster and I have five Google accounts I could use to register for the same panel five times. Why do I have five Google accounts? Email info@surveydiem.com or info@olasurveys.com and I’ll tell you.

In the survey: Trap questions – Savvy researchers know to include trap questions in their surveys. At my old company, we used to include a fake brand in the answer options for surveys, knowing that if someone says they have used the brand, they’re either lying or not paying attention. There are many fun and clever questions you can ask to try to catch people in a lie. Just don’t get too clever and boot everyone from your survey.

Pros: It’s easy to add these questions to any survey.

Cons: Fraudsters are getting smarter, and when they use AI agents to take surveys, the AI is smarter still. Again, there is a never-ending game of cat and mouse.

Other things – I’m sure I’ve missed some things (e.g., speed traps, mobile-only surveys, app-based surveys, hashed survey links), but this post is already long.

Have questions? We’re happy to hear from you. Hit us up at info@surveydiem.com.