Research guide

Sample size, percentages & multi-select questions

Why the denominator matters, why some percentages exceed 100% and why a large total sample does not mean every chart uses the same base.

Survey percentages look simple because they compress a count and a denominator into one number. The denominator is the part most likely to disappear in a headline, yet it is essential for interpreting the result correctly.

The basic calculation

A survey percentage is usually a count divided by an eligible base. If 4,113 out of 4,532 recent online shoppers report using a platform, the reported share is about 90.8%. The denominator is 4,532 because that is the group eligible for the platform question.

If the total study contained 5,077 respondents, dividing by 5,077 would answer a different question: the share of the entire analytical sample who selected that platform, including people who did not recently shop online.

Total sample versus analytical base

The total analytical sample is the set of records included in the overall study after applicable quality rules. A question-level analytical base can be smaller because of routing, eligibility or intentionally excluded responses.

Whenever a chart uses a subgroup, the best practice is to show the subgroup base near the result. This prevents readers from assuming every percentage is based on the headline sample size.

Why sample size matters

Large bases generally make descriptive percentages less sensitive to the answer of any one respondent. In a base of 20, a single response changes a percentage by five points. In a base of 2,000, one response changes it by only 0.05 points.

But sample size does not solve every design issue. A large non-probability sample can still differ systematically from the population a reader hopes to describe. The number of observations and the method used to obtain them should be considered together.

Counts are useful companions to percentages

Publishing a count alongside a percentage makes the result easier to audit and helps readers understand scale. It also exposes small subgroup bases that might otherwise look more authoritative than they are.

Counts can differ slightly from a percentage reconstructed by multiplying because the displayed percentage may be rounded.

Rounding

Percentages are often rounded to one decimal place or to whole numbers. As a result, single-choice percentages may add to 99.9% or 100.1% instead of exactly 100%. This is usually a display effect, not a data problem.

When precision matters, counts and the stated base provide the clearest reconstruction of the underlying result.

What is a multi-select question?

A multi-select question allows a respondent to choose more than one answer. “Which shopping platforms did you use in the past 30 days?” is a typical example. A respondent might select Trendyol, Hepsiburada and a retailer’s own website.

Each platform percentage is calculated independently against the same eligible base. Because the same respondent can contribute to several options, the percentages can add to far more than 100%.

Why multi-select percentages are not market share

Market share usually implies mutually exclusive allocation of a total, such as one primary choice or a measured share of transactions or revenue. Multi-select survey usage is different: it measures reach or reported usage. Several platforms can simultaneously have high reach among the same respondents.

For this reason, a multi-select chart should use wording such as “share of recent shoppers who used each platform,” not “share of the market,” unless the questionnaire actually measured exclusive share.

Single-choice questions

A single-choice question asks each eligible respondent to select one answer. “Which payment method do you use most often?” is an example. Here, the categories are intended to be mutually exclusive, so the percentages should normally total approximately 100% after rounding and any valid missing-response handling.

Maximum-choice questions

Some questions allow several responses but impose a limit, such as “select up to three factors.” These remain multi-select questions. The percentages describe how frequently each factor was selected, not the percentage of all selections. A person can contribute to as many options as the rule allows.

Routing changes interpretation

Imagine a study with 5,077 included respondents. Q1 asks whether they shopped online recently; 4,532 say yes. Q2–Q7 are then shown only to those recent shoppers. Q8 returns to all respondents. The base changes across the questionnaire even though all questions belong to the same study.

A publication should therefore avoid a single generic “n=5,077” label on every chart. The correct base depends on the specific question.

Missing responses and exclusions

Some studies may exclude invalid or unusable responses from a particular calculation. If those exclusions are material, the publication should explain them and identify the effective base. The denominator should always correspond to the records that could legitimately contribute to the result.

Comparing percentages across studies

Two percentages are most comparable when they use similar question wording, timeframes, eligibility rules, collection modes and analytical definitions. A 30-day usage measure and an “ever used” measure are not the same metric even if both are labelled “usage.”

Quick reading rules

  • Look for the base before interpreting the percentage.
  • Check whether the question was routed to a subgroup.
  • Use counts to understand the scale behind the percentage.
  • Expect small rounding differences.
  • Do not force multi-select percentages to total 100%.
  • Do not call multi-select reach “market share” without an exclusive measure.
  • Be cautious with percentages from very small subgroups.

See these conventions in published research → · Online panels explained →