Last updated: 7 October 2026
Quick Answer: Sampling bias is a systematic error that makes some members of a population more likely to be sampled than others, tilting results toward them. Unlike random error, it does not shrink as the sample grows. The 2020 US Census, for example, undercounted Hispanic residents by 4.99% while overcounting non-Hispanic White residents by 1.64%.
Sampling bias is what happens when the way you collect a sample decides part of the answer before anyone responds. Some people are easier to list, easier to reach or keener to take part, and if they differ from everyone else on what you are measuring, the result leans their way.
Size is no defense. A biased sample of 10,000 gives a precise estimate of the wrong number, which makes it more dangerous than a small sample, because the narrow margin of error looks like proof.
How Much Can Sampling Bias Shift a Result?
Sampling bias shifts a result by giving one group more weight in the sample than it has in the population, so the figure drifts toward that group's answers. How far it drifts depends on how over-represented the group is and how differently it answers, and the shift can easily reach several points.
A simple example shows the arithmetic. Suppose 30% of a brand's customers are heavy users, 80% of whom are satisfied, while 50% of light users are satisfied. True satisfaction is 0.3 × 80 + 0.7 × 50, or 59%. Now suppose heavy users are keener to answer and make up 60% of the sample. The survey reports 0.6 × 80 + 0.4 × 50, or 68%.
That 9-point gap is pure bias. Doubling the sample size halves nothing; only fixing who gets in, or weighting heavy users back to 30%, removes it.
Bias can run the other way too. If unhappy customers are keener to respond, for example after a well-publicized outage, the same arithmetic pushes the score below the truth.
How Is Sampling Bias Different From Sampling Error?
Sampling bias is systematic and pushes results in one direction, while sampling error is random and scatters results around the true value. Sampling error shrinks as the sample grows and is measured by the margin of error; sampling bias does not shrink with size and is not captured by the margin of error at all.
So a survey can report a margin of error of 2 points and still be 9 points off. The margin of error answers "how much might chance move this?", never "did we sample the right people?" In practice, report both: the margin of error for chance, and a plain statement of who the sample could not reach.
What Are the Types of Sampling Bias?
The main types of sampling bias are undercoverage, self-selection, nonresponse, convenience, survivorship and healthy-volunteer bias, plus the exclusion and attrition problems that creep in later. The table runs roughly in the order they arise, from building the list to analyzing results.
| Type | What happens | Typical example |
|---|---|---|
| Undercoverage | Part of the population is missing from the list | A landline-only phone list in a mostly mobile population |
| Exclusion | Screening rules drop eligible people | Excluding anyone who bought in the last 30 days |
| Convenience | Only easy-to-reach people are recruited | Surveying shoppers at one flagship store |
| Self-selection | People choose whether to take part | An open poll on a brand's social account |
| Voluntary response | Strong opinions are likelier to reply | A product page that invites reviews |
| Nonresponse | Selected people do not answer | Low reply rates among busy or unhappy customers |
| Healthy volunteer | Participants are healthier or keener than average | Long-term health cohorts |
| Survivorship | Only cases that lasted are observed | Studying only customers who stayed |
| Attrition | People drop out partway through a study | Diary studies that lose participants by week three |
Nonresponse has its own causes and cures, covered in the guide to survey fatigue and who stops answering.
What Is Undercoverage Bias?
Undercoverage bias occurs when some members of the target population are missing from the list or source a sample is drawn from, so they have no chance of selection. It is the hardest type to spot, because the missing people leave no trace in the data.
Even the US census faces it. The Census Bureau's Post-Enumeration Survey found no net national undercount in the 2020 Census, at -0.24%. Yet it undercounted the Black population by 3.30%, the Hispanic population by 4.99% and American Indians living on reservations by 5.64%, while overcounting non-Hispanic White residents by 1.64%.
A national total can look right while specific groups are badly covered, and any survey built on a similar list inherits the same gaps.
Business lists have the same blind spots. A customer database misses guest checkouts and cash buyers, and a landline-era phone list misses most adults: the CDC's National Health Interview Survey found 78.7% of US adults lived in wireless-only households in the second half of 2024. Name the missing group before fielding and find a second source that reaches it, rather than hoping weighting will cover it.
What Is Self-Selection and Voluntary Response Bias?
Self-selection bias arises when people decide for themselves whether to join a study, and voluntary response bias is the version where those with strong views are likeliest to reply. In both, the act of volunteering is linked to the answer, so the sample over-represents the motivated, the healthy or the angry.
UK Biobank shows the effect clearly. About 9.2 million people were invited and 5.5% took part. A 2017 comparison in the American Journal of Epidemiology found participants less likely to be obese, smoke or drink daily than the general population. Their all-cause mortality at ages 70 to 74 was lower by 46.2% in men and 55.5% in women. The authors called it a healthy volunteer selection bias.
Customer research has its own versions. Open web polls, app-store reviews and social media surveys all let people choose themselves, and they skew toward fans and critics with something to say. The fix is to invite a drawn sample rather than wait for volunteers, and to check whether early responders differ from late ones.
How Is Sampling Bias Different From Selection Bias?
Selection bias is the umbrella term for errors in how people get into a study or an analysis; sampling bias is the part that happens when the sample is drawn. Epidemiologists use selection bias more widely, including people lost to follow-up after a study begins.
In market research the two are often used interchangeably. The distinction matters when diagnosing a problem: sampling bias points to the frame and recruitment, while selection bias can also point to who was kept, cleaned out or analyzed.
Attrition is the common case of selection bias that is not a sampling problem. A diary study may start from a well-drawn sample, but if busy parents drop out by week three, the final analysis over-represents people with spare time, even though the original draw was sound.
What Are Some Real Sampling Bias Examples?
Real sampling bias examples range from national elections to product decisions, with one pattern: one group was easier to reach, and its answers dominated.
- The 2016 US election polls. An AAPOR evaluation found that many polls, especially at the state level, did not adjust their weights for the over-representation of college graduates in their samples.
- Survivorship in World War II. Statistician Abraham Wald's well-known analysis of damaged returning bombers argued for armor where returning planes showed no hits, because planes hit there were the ones that never came back.
- Customer feedback from loyal users. A churn study that surveys only active customers learns why people stay, not why they leave.
- Voluntary employee surveys. When participation is optional, engaged staff and those with grievances answer most, so the middle of the workforce drops out of the results.
Each is a selection bias example in the broad sense: the data was real, but it came from the wrong mix of people.
Which People Does Sampling Bias Usually Leave Out?
Sampling bias usually leaves out people who are harder to list, harder to contact or less inclined to answer in the format offered. That often means lower-income, older, less connected or non-English-speaking groups, exactly the people a single online channel tends to miss.
Format choice alone can shift who answers. In a 2021 study of 387 adults aging with long-term physical disability, about 40% chose phone over web. The phone group was older, had less education and rated their health lower, and people earning $10,008 or less a year were 5.22 times as likely to pick phone. A web-only survey would have quietly dropped them.
Offering more than one channel narrows that gap. WhatsApp-native interviews reach people inside a chat app they already use, with no link and no app, and AI phone interviews reach people who prefer to talk. Alchemic publishes 60+ languages including Spanish and Hindi, with managed fieldwork or bring your own sample across 14 markets including the USA and the UK.
How Can You Reduce Sampling Bias?
You reduce sampling bias by fixing who can be selected, who is invited and who answers, then weighting only for what remains. No single step removes it, but together they shrink the gap between the people you reached and the people you meant to study.
- Start with the best frame: Use the list that covers most of the target population, and combine sources where one misses a group.
- Select by chance where possible: Use probability sampling or tight quotas on the traits that drive the answer.
- Offer more than one channel: Web, chat and phone each reach different people.
- Chase non-responders: Reminders and a second channel recover the people who would otherwise drop out.
- Weight carefully: Weight to education and other traits linked to the outcome, as pollsters learned after 2016.
- Test the sample first: Compare it with census or customer-base shares, and investigate any gap of more than a few points.
- Report the limits: State the frame, response rate and weighting with every result.
Once it has the client's brief, Alchemic designs and tailors the sampling plan to handle these risks before fielding, rather than leaving that work to the buyer. When a decision rests on an exact population figure, a probability-based panel or official statistics are the better source, even at higher cost.