Last updated: 1 September 2026
A usability testing service supplies three things: participants who match your users, a moderated or unmoderated study format, and a written diagnosis that ranks what broke by severity and ties each finding to the moment it happened. What moves the price is almost never the software.
It is the recruit: who you need in the session, how hard those people are to find, and how many distinct groups of them you have to cover.
That last point is where quotes diverge. Two vendors can look at the same prototype, agree on the same method, and return numbers that differ by a factor of four. One assumed a general-population recruit and the other read the brief closely enough to notice that you need eight radiologists, four of whom use a screen reader.
Sample size is the other place buyers get quietly overcharged and quietly under-served, usually in the same study. The "five users is enough" rule that most vendors still quote is a summary of a finding about averages, and the average is not the number you are buying.
Why Two Vendors Quote the Same Study So Differently
The recruit dominates the invoice. A usability study on a consumer checkout flow with general-population participants is a commodity, and vendors price it like one. The same study specified for a defined professional group, or for people using assistive technology, or across three language markets, changes the cost base entirely, because the vendor moves from drawing on a standing panel to actively finding people. The wider version of that interrogation is in 14 questions to ask a research vendor about reach.
Three specification details do most of the work in a quote:
- Number of distinct user groups. Managers and frontline employees use the same system differently, and each group needs its own sessions. The US federal guidance summarized in Jean E. Fox's Bureau of Labor Statistics paper on usability testing methods lists the number of user groups as the first factor pushing participant counts up.
- Heterogeneity inside a group. A diverse group produces more distinct approaches to the same task, so it takes more sessions to reach the point where new problems stop appearing.
- System complexity. More paths through a product means more places for a problem to hide.
None of these are platform features, which is why comparing usability testing services on their software tour tells you little about what you will pay. The distinction between a tool and a service is drawn in platforms that run the study for you.
Moderated or Unmoderated: Which Study Are You Buying?
Moderated testing puts a researcher in the session, live, able to ask why a participant hesitated. Unmoderated testing sends participants a task script and records what they do without intervention. They answer different questions and they are not interchangeable, though vendors often quote them as if they were.
| Format | What it tells you well | What it misses | Typical use |
|---|---|---|---|
| Moderated, in person | Behavior with physical context, assistive technology in real use, complex or sensitive tasks | Expensive, slow, geographically narrow | Accessibility work, medical and industrial systems, high-stakes flows |
| Moderated, remote | The reasoning behind a hesitation, follow-up on anything unexpected | Requires a stable connection and a device the participant can screen-share from | Most product and prototype work |
| Unmoderated, remote | Task success rates, time on task, where people drop out, at volume and low cost | The why. A participant who quits tells you nothing about the reason | Benchmarking, large-sample validation, A/B of two flows |
| In-product intercept | Real behavior with real stakes, no recruitment lag | Self-selected respondents, no control over who answers | Continuous monitoring on a live product |
| AI-moderated | Follow-up questions at unmoderated cost and volume; latitude to leave the guide varies considerably by system | Depends on the system's probing rules and the modes it supports | Larger qualitative samples, multi-language studies |
Best for accessibility work: moderated and in person, still, and it is worth saying plainly because it is the row a research vendor has the least commercial reason to recommend. Watching someone navigate with a screen reader they have configured themselves, on their own hardware, surfaces problems that no remote session reproduces reliably. Section508.gov's guidance on usability testing with people with disabilities is explicit that participants should be recruited on functional ability and assistive technology use rather than on diagnosis, and that setup time in these sessions is not overhead to be trimmed.
How Many Participants Does a Usability Test Need?
More than five, and the honest answer depends on how much risk you are willing to carry. The five-user guideline traces to Virzi's work around 1990 and 1992, which found that four or five participants revealed about 80 percent of the usability problems, on average, with diminishing returns after that. The word doing the work in that sentence is "average."
Faulkner's 2003 study is the one worth quoting to a stakeholder who wants to cut the sample. She ran a usability test of a timesheet system with 60 participants, then drew 100 random samples of five from that pool. The five-person samples uncovered 85 percent of the problems on average, which is consistent with Virzi.
But the worst-performing group of five found just 55 percent. At groups of ten, the average rose to 95 percent and the minimum to 82 percent.
A third study cited in the same federal paper is starker. Testing e-commerce sites that sold music and video, researchers identified 378 usability problems in total; the first five participants on one site uncovered 35 percent of them, and new problems were still appearing at the eighteenth participant.
The practical reading: five participants per group buys you a good chance of finding most of the big problems and a real chance of finding barely half of them. If the study is informing a reversible design decision, that risk is fine. If it is signing off a launch, it is not. Budget ten per distinct user group where the decision is expensive, and treat five as the floor for a quick diagnostic round.
What Drives the Price of a Usability Study?
In rough order of impact:
- Recruitment difficulty. General consumers are cheap. Verified professionals, low-incidence conditions, assistive technology users and specific job titles are not, and incentives rise with the specificity.
- Number of user groups multiplied by participants per group. This is the line that compounds. Three groups at ten participants is thirty sessions, not ten.
- Moderation. A moderated session costs researcher hours; an unmoderated one does not.
- Markets and languages. Each additional market is a separate recruit, and a moderator who genuinely works in the language.
- Analysis depth. A severity-ranked written diagnosis with video evidence costs more than a highlight reel, and is worth more.
Note what is not on that list: the platform license, which is usually the smallest line and the one vendors compete hardest on. What people will actually pay is a separate instrument, covered in pricing research methods.
Who Does a Browser-Based Usability Test Miss?
Every remote usability study inherits the reach of its delivery channel, and a browser-based session with screen sharing is a narrower channel than it appears.
The clearest evidence is domestic to the United States, not a developing-market story. Pew Research Center's mobile technology fact sheet reports that 16 percent of US adults are smartphone-only internet users, meaning they own a smartphone but have no home broadband. Broken down by income, that is 34 percent of adults in households under $30,000 a year, against 4 percent of those above $100,000.
A recruitment method that assumes a laptop, a stable connection and the confidence to screen-share is not sampling the general population. It is sampling the top of the income distribution, and the gap widens exactly where affordability research matters most.
The same structural exclusion runs through accessibility. Research on inclusion in study design, including work on conducting accessible research in public health and outcomes studies, documents that inaccessible recruitment, consent and measurement each independently filter out disabled participants before a study begins. Work on universal design of research argues for multi-sensory recruitment and instrument formats as the routine default rather than an accommodation on request.
A parallel argument in the visualization research literature, set out in a position paper on inclusion and accessibility in that field, notes that small participant numbers are themselves a recurrent obstacle to accessibility work.
How Does Interview Channel Change Who Is in the Sample?
This is where interview channel becomes a sampling decision rather than a logistics preference. Studies that need to reach beyond the laptop-and-broadband segment increasingly run the qualitative layer over channels those respondents already use. That is the reasoning behind interviews conducted natively inside WhatsApp with no link and no app to install, and behind outbound AI phone research for people who are reachable by voice and not by browser.
Neither replaces a screen-sharing usability session when the task is watching someone use an interface.
Both change who is available to be asked what happened afterward, and for a study spanning several income bands or several language markets, that is the difference between a finding and an artifact.
Where a study genuinely needs prototype interaction observed rather than reported, Alchemic's UI and UX testing work probes during use, triggered on hesitation, mis-clicks and back-outs against Figma, Framer, Webflow and coded prototypes. Sample norms run at 8 to 15 per persona and 30 to 50 per variant. Recruitment runs as managed fieldwork or bring your own, which matters most on exactly the specialist recruits that dominate a quote.
How Do You Read a Severity Ranking?
A severity ranking is the deliverable that decides whether a study changes anything, and it is the least standardized part of the category. Most vendors grade on some combination of how many participants hit the problem, how badly it blocked the task, and how easily a user recovered. None of that is an industry standard, so two vendors can rank the same finding two levels apart in good faith.
Ask three things of any severity scale before you accept the report. What evidence sits behind each rating, and can you get to the session moment that produced it? Does the scale separate frequency from impact, or collapse them into one number that hides which is which? And does a low-frequency, high-impact failure, the kind that blocks one participant completely, survive the ranking or get averaged away?
The professional standards worth holding a vendor to on evidence handling and participant treatment are published by ESOMAR in its code and guidelines. The US government's usability guidance on digital.gov is a reasonable public baseline for what a defensible process looks like. Testing the execution itself is covered in creative and ad testing before launch.
Where Usability Testing Gives You the Wrong Answer
Usability testing tells you whether people can use a thing. It does not tell you whether they want it, and buying it for the second question is the most common way the money is wasted.
Four honest limits:
- It does not measure demand. A checkout flow can test perfectly and sell nothing. Concept and proposition work answers that question, and it is a different study.
- Task-based testing rewards the tasks you thought to write. Problems live in the paths you did not script.
- Lab and remote sessions both carry an observation effect. Participants who know they are being watched persist through friction they would abandon in private, which biases task success upward.
- Small samples do not support statistical claims. Eight participants finding a problem is strong qualitative evidence and is not a percentage of your user base. Reporting it as one, which happens routinely in decks, converts a genuine finding into a number that will not survive scrutiny.
There is also a limit on the AI-moderated end of the table above. How much latitude a system has to leave the discussion guide varies considerably between tools, and a system that is rigid about it will hold the script when the interesting answer is one question sideways. That is a property of the specific system and its probing rules rather than of automated moderation generally, so it is a question to put to a vendor in a pilot rather than an assumption to carry into one.
Where the study is run as a managed service, the researchers design and tailor the guide to those risks from the brief before fielding, rather than leaving that work to the buyer.
For teams weighing the broader category rather than a single study, Alchemic's overview of AI-moderated interviews sets out where conversational depth and scale trade against each other, and the qualitative research primer covers the method foundations this article assumes.