Sampling error
Quantify uncertainty in a repeated Bernoulli average without mistaking variance for bias.
13 minutes · advanced · Original Poker4Quant lessonWhat you will learn
Calculate a model standard error and explain its independence and sampling assumptions.
Prerequisites: Repeated bets and dependence · Calibration and scoring
A sample fraction is a random quantity
For n independent identically distributed Bernoulli trials with event probability p, the sample fraction has expectation p and variance p(1−p)/n. Its standard deviation, often called the standard error in this model, is the square root of that variance.
At p=1/2 and n=100, the standard error is sqrt(0.25/100)=0.05. This describes the sampling distribution over repeated datasets. It does not mean every observed fraction is within five percentage points of the truth.
More observations do not fix every problem
Quadrupling n halves the standard error under the same independent model. Correlation between trials changes the variance; selected or biased observations can shift the mean away from the intended target. Neither issue is repaired merely by plugging a larger n into the formula.
A formal interval needs a justified construction and coverage interpretation. For example, the distribution-free Chebyshev bound gives P(|sample fraction−p|≥ε)≤p(1−p)/(nε²), under the stated finite-variance independence model. It may be loose; do not relabel one standard error as a 95% interval.
Worked example
Under independent Bernoulli p=1/2 sampling, compare n=100 with n=400 and bound a deviation of at least 0.10 for n=100.
- At n=100, variance is 0.25/100=0.0025, so standard error is 0.05.
- At n=400, variance is 0.000625, so standard error is 0.025.
- Chebyshev at n=100 and ε=0.10 gives 0.0025/0.01=0.25.
The standard error halves, and the displayed bound is at most 25% for that deviation. The bound is not an exact tail probability or a claim of a 95% interval.
Hands-on: Spend a sample budget and inspect repeated datasets
Resize recorded samples, compare seeded sampling distributions and distinguish fresh source groups, selective inclusion, standard error, bias and observed batch variation.
Reading sampling experiment…
Common mistakes
- Confusing standard error with a guaranteed error limit.
- Applying independent-trial variance to correlated data.
- Expecting more biased observations to remove selection bias.
Keep these ideas
- State the sampling model.
- Standard error scales like 1/sqrt(n) under independence.
- Bias, dependence and interval coverage need separate analysis.
Further reading
- MIT OpenCourseWare — Probability Recitation 21: Inequality Examples ↗
University examples distinguish probability bounds from distributional approximations. Linked for further study of the inequality used in the original sampling-error lesson. Checked 2026-10-21.