Where Good Survey Respondents Come From

Last updated: 27 August 2026

Respondent sourcing is the process of deciding who gets invited into a study. It runs through four routes: a panel the vendor owns, a sample exchange or marketplace, fresh recruitment through channels like phone and messaging, or the client's own customer lists. Each route fails in a different way, and the failure is usually decided before the first question is asked.

The evidence for that is unusually clean. In a study of more than 60,000 interviews across six online sources, Pew Research Center found 4% to 7% bogus respondents in widely used opt-in samples, against roughly 1% in panels recruited offline from random residential addresses. Same questionnaires, same analysis, different sourcing: a five-fold difference in fake participation.

Panel size never entered the equation, which is worth remembering in a market where vendors advertise panels in the tens of millions. Provenance and verification are the buying criteria. Headcount is the brochure.

Address-based recruitment is the archetype behind that 1% figure, and it is how national statistical programs have always worked. The US Census Bureau's Current Population Survey draws from residential addresses rather than volunteers, which is expensive, slow and the reason its samples behave.

The Four Sources of Respondents, and How Each Fails

Every sourcing conversation with a vendor resolves to one of these rows, whatever the branding says.

The point of the table is that no row is safe by default. An owned panel gives the vendor control over who joined, and simultaneously breeds the practiced respondent who has seen a thousand screeners. An exchange delivers scale, and hides where people came from. Fresh recruitment reaches people no panel contains, at a price. Client lists are perfect provenance and a biased frame. Buyers who understand which failure they are accepting choose better than buyers comparing panel sizes.

Among named providers, YouGov, Dynata, aytm and Prodege operate their own large consumer panels, while Respondent and Prolific compete on verification discipline rather than headcount, publishing identity and screening controls as the product. Verasight, Outsized Insights and PickFu sit in the same verification-first group. Which of those is right depends on whether your constraint is scale or provenance, and those are usually different vendors.

What Happens When Sources Are Blended?

Blends are the practical norm on larger studies, which adds a reporting requirement rather than removing one. When a vendor fills a quota from two panels, an exchange and a client list, ask for completion quality and removal rates by source, not blended. A single blended number is how a weak supplier hides inside a strong one. The consent and disclosure mechanics are covered in consent and disclosure in AI-moderated research.

Does Owning a Panel Guarantee Better Data?

No. Ownership improves provenance, because the vendor controls sign-up and profiling, but it introduces the panel's own pathologies, and it says nothing about verification discipline. The two questions are separate: where do people come from, and how does anyone know they are who they claim?

The professional respondent is the owned panel's characteristic risk: someone who takes surveys at volume, learns what screeners want to hear, and qualifies for studies they should not enter. Repeated participation also shifts behavior in subtler ways.

Pew's American Trends Panel documentation discusses panel conditioning openly, along with the finding that it does not seriously degrade their data. That openness is itself the tell of a well-run panel. Rotation rules, participation caps and refresh cohorts are the machinery that keeps a panel honest, and a vendor who owns one should be able to describe theirs unprompted.

Sample exchanges invert the trade. Scale and speed are real, and the ESOMAR code and guidelines exist in part because aggregated supply chains need auditable duties around consent and data handling. When a vendor fields through an exchange, the useful question is which suppliers were used on your project and what quality controls applied at the join, because the exchange's average hides the variance you will actually receive.

How Do Platforms Verify That Respondents Are Real?

Through layers, and no single one is sufficient. Identity signals at sign-up, device and network fingerprinting to catch duplicates, screening logic that traps impossible claims, in-survey behavior checks, and post-field review of open-ended answers. Vendors serious about this publish their approach; some go further, with fraud detection backed by refunds, as Outset advertises for LLM-generated responses. The strength of the discipline varies more than the marketing does.

The Five-Question Verification Audit

A workable buyer's audit fits in five questions:

• Identity: what stops one person holding five accounts, and is any check anchored to something costly to fake, such as a verified phone number?
• Geography: how are location claims verified, given that Pew's bogus-respondent criteria included people outside the survey's country entirely?
• Attention: which in-survey checks run, and what disqualifies a complete?
• Open ends: who or what reads them, and what happens to non sequitur answers?
• Consequences: what is removed, refunded or re-fielded when fraud is found after delivery?

Standards give the audit a floor: AAPOR's standards and ethics and the Insights Association's work on data quality both treat transparency about sourcing and exclusions as professional duties, not competitive secrets. AAPOR's Transparency Initiative turns that into a publishable checklist, which is a fair template for what a vendor should hand over with a dataset. A vendor who cannot answer the five questions is not necessarily fraudulent. They are unverified, which for a research buyer is the same purchasing decision.

What Do AI Bots Change About Survey Fraud?

They collapse the cost of producing plausible fake responses, which moves the burden of proof from the answer to the respondent. A decade ago a fake complete usually looked fake: random grids, gibberish open ends. Today a language model writes a fluent, on-topic paragraph about a detergent it has never used.

The scale of the shift is measurable at the edges. When researchers reran a text summarization task on Amazon Mechanical Turk, keystroke detection and classifier estimates put LLM use at 33% to 46% of crowd workers on that task. The authors are careful to note the task was unusually suited to LLM help, so the figure is a ceiling for tasks like it rather than a universal rate.

Survey-side reporting points the same way. A Johns Hopkins Bloomberg School of Public Health article on data integrity in online surveys reported 34% of online survey participants saying they had used AI tools to answer open-ended questions, citing speed and writing quality.

What Should Buyers Do Differently?

Two consequences follow for buyers. First, open-ended fluency is no longer evidence of a real, attentive human, so verification has to lean on identity, device and behavioral layers rather than answer quality. Voice and video modes raise the cost of faking further, since a live conversation in a respondent's own language and register is far harder to manufacture at scale than typed text.

Second, the positivity bias Pew documented gets worse, not random. Their bogus respondents approved of everything, saying yes to opposing political items at 78% and 84%, the kind of systematic skew that inflates purchase intent and concept scores precisely when the numbers matter. The defense is structural rather than clever: source respondents through routes that are costly to fake, verify identity at entry, and treat post-field review as part of fieldwork rather than an afterthought.

Recruitment Channels Decide Who You Can Even Reach

Verification protects against the people who should not be in the study. Recruitment decides who can be, and for most consumer categories the binding constraint is that panel-joiners are not the population. Pew's mobile technology fact sheet reports 16 percent of US adults as smartphone-only internet users, rising to 34 percent in households under $30,000 a year, and the ITU's Facts and Figures 2025 counts 2.2 billion people offline worldwide. Every one of those people is invisible to a browser-recruited panel, and so are the far larger numbers who are online but will never join one.

Channel-based recruitment is the working answer: reaching people over the phone numbers and messaging apps they already use. Health researchers recruiting migrant and mobile populations have used WhatsApp precisely because it retains contact with participants no fixed sampling frame holds. The same logic runs in commercial research: interviews running natively inside WhatsApp, with no link and no app, and AI phone interviews that reach any working number, recruit from the population rather than from the panel-joining slice of it. A phone-number-anchored invitation also carries a verification bonus: the identity is attached to a number that is costly to fake at scale. Research pairing donated messaging data with survey responses shows how much more is verifiable when a study is tied to a real account rather than an anonymous web session.

How Does Channel-Based Recruitment Run in Practice?

Alchemic runs this as one system. Recruiting draws on its own panel network, client lists or a hybrid top-up, with live screening and quotas, incentives paid over bank transfer, local rails or global payment systems, and automated fraud and quality flags feeding an audit trail.

The design premise is the one this article has been arguing: sourcing, reach and verification are a single problem, and vendors should be evaluated on all three at once. A structured way to do that in a vendor call is this reach checklist.

Where Every Sourcing Model Falls Short

• Fraud is only half of sample quality. A study can be fraud-free and still miss the people who matter, because coverage bias is a different defect from fake participation. The companion problem, who a sampling mode structurally excludes, is covered in sample validity in AI-moderated research.
• Verification adds friction, and friction biases. Every identity check deters some legitimate respondents, usually the busiest and most privacy-conscious. The craft is layering checks so the honest path stays short.
• No detection layer is final. Fraud methods and detection methods co-evolve, and any vendor claiming a solved problem is describing last quarter. What a buyer can reasonably demand is disclosed method, measured rates and refund-backed consequences.
• Coverage costs money, not cleverness. The World Bank's Living Standards Measurement Study is the reference case: reaching a genuinely representative household sample means funding in-country field operations, and no online shortcut has repealed that.
• Fresh recruitment does not scale infinitely. Phone and messaging recruitment reach non-panel populations at real cost per complete, so large multi-market quotas usually blend routes. Ask how the blend is reported, because a blended sample with one quality number is hiding its variance.