Consistency and information growth

What you will learn

Prove or refute consistency for sample-size-dependent anchor schedules, distinguish nominal records from independent information, and construct a finite bias-plus-fluctuation certificate.

Prerequisites: Shrinkage, bias and squared error · Moment tail bounds and tightness

A learning claim concerns an entire sequence

Consistency asks whether an estimation procedure learns a fixed parameter as the amount of information grows. Write T_n for the rule used with nominal sample size n. At any fixed population p, every positive tolerance ε must eventually have P_p(|T_n−p|>ε) approaching zero. This permits occasional bad datasets. It does not promise that the next estimate is closer than the previous estimate, nor that a particular finite dataset is accurate. Both the rule and the sampling mechanism must be specified for every n, not merely at a few convenient sample sizes.

Our experiment has a fixed unknown success rate p in [0,1]. Fresh observations are independent Bernoulli trials at that same rate, recorded without selection or error. There is no population drift, adaptive stopping or posterior distribution for p. The anchor is the stipulated number 1/2. This is a synthetic learning experiment, not evidence that poker actions or financial returns satisfy these assumptions. We compare two collection policies: collect n fresh observations, or copy a single observed bit into n record positions. The second policy deliberately violates independence between records and lets us isolate information from file size.

Build a rule whose anchor influence can disappear

Let V_n be the fresh sample proportion, or the single bit under the copy policy. Add an anchor mass a_n, selected before any observations, and report T_n=(nV_n+a_n/2)/(n+a_n). The anchor mass is algebraic weight, not additional independent data. Put w_n=n/(n+a_n). For either policy the sampling mean is w_np+(1−w_n)/2, and signed bias is (1−w_n)(1/2−p). For fresh data, variance is w_n²p(1−p)/n. For copies, variance is w_n²p(1−p), because averaging identical copies returns the original bit.

Compare three complete schedules: a_n=4, a_n=√n, and a_n=n. The first two have a_n/n tending to zero, so w_n tends to one and anchor bias disappears. Proportional anchoring instead leaves w_n=1/2 forever. With fresh data it settles near p/2+1/4, which equals the target only at p=1/2. A shrinking variance can therefore produce increasing certainty around the wrong value. The important comparison is anchor mass relative to data mass; requiring the anchor mass itself to approach zero would incorrectly reject the square-root schedule.

Prove learning and expose an information failure

For fresh data under either sublinear anchor schedule, variance tends to zero and squared bias tends to zero. Their sum is the mean squared error R_n. For a fixed ε>0, applying a nonnegative tail bound to squared estimation error gives P_p(|T_n−p|>ε)≤R_n/ε². This supplies a sufficient proof of consistency at each p. A finite table may illustrate the terms, but the argument about all larger n establishes the limit. An asymptotic property alone also gives no universal claim that one schedule has lower finite-sample risk than another.

Copying one bit defeats that proof: for sublinear anchoring, T_n approaches that same random bit, not its unknown mean. At p=1/2, the absolute error approaches 1/2 with probability one. Every tolerance below 1/2 eventually fails almost surely, even though the sampling mean is exactly p for every n. Thus unbiasedness is insufficient. At p=0 or 1 the bit is deterministic, so those two boundary populations are exceptions. With proportional anchoring and copies, the limiting outputs are 1/4 and 3/4; no p makes this rule consistent. Checking degenerate boundaries prevents an overbroad claim about the whole family.

Construct a finite certificate with two error budgets

A proof of eventual learning and a usable finite guarantee answer different questions. Split ε into a deterministic bias allowance b and a random allowance r=ε−b, with both positive. If |E_p[T_n]−p|≤b, an error larger than ε requires a centered fluctuation larger than r. Consequently its probability is at most min(1,Var_p(T_n)/r²). If the bias allowance fails, this particular certificate reports the trivial bound one. That failure does not prove the actual error probability is large. Increasing the bias allowance makes the deterministic test easier but leaves less random allowance and raises the variance bound.

For the fixed-four schedule with fresh data, p=1/4, ε=1/4 and equal allowances b=r=1/8, bias magnitude is 1/(n+4) and variance is 3n/[16(n+4)²]. At n=16, bias is 1/20 and variance is 3/400. Bias fits, but the tail bound is (3/400)/(1/64)=12/25, exceeding the desired δ=1/8. At n=256, bias is 1/260 and variance is 3/4225. The bound is 192/4225, approximately 0.045444, below 1/8. These are model guarantees, not exact tail probabilities or confidence statements conditional on an observed estimate.

Audit a ladder without confusing it with infinity

The workshop lets you construct a complete anchor schedule, switch collection policies, inspect a generating population and transfer tolerance markers between the bias and fluctuation budgets. Its five horizons are 4,16,64,256,1024. Mark every horizon where both certificate conditions hold, then report the limiting sampling mean, limiting MSE and first certified horizon in that displayed ladder. Use zero when no displayed horizon qualifies. At the worked settings n=64 has bound 48/289, greater than 1/8; n=256 is therefore the first qualifying rung. It is not claimed to be the smallest qualifying integer sample size.

Classify consistency over the whole parameter range separately from the finite certificate. The choices distinguish all populations, the anchor alone, endpoints alone and none. Changing a model or budget clears previous claims so that stale arithmetic cannot certify a different experiment. Fractions and finite decimals are accepted within one millionth; hints and reset support revision. Drafts last for this visit and do not award account credit. A large copied dataset, a favorable anchor population or a successful finite certificate at one tolerance cannot replace the required statement about every fixed positive tolerance and the actual information-generating mechanism.

Hands-on: Build an information-growth certificate

Construct an anchor schedule and collection policy, transfer tolerance allowances, certify sample-size rungs and classify the entire sequence.

Common mistakes

• Using nominal record count in an independence-based variance formula after copying one bit.
• Confusing vanishing variance with convergence to the correct target.
• Requiring absolute anchor mass to vanish instead of its ratio to data mass.
• Treating a failed upper-bound certificate as proof of a large exact error probability.
• Calling the first qualifying displayed rung the minimum over all integers.
• Replacing every positive tolerance in consistency with one convenient tolerance.

Keep these ideas

• Specify an estimator sequence and its information-generating policy.
• Sublinear anchor mass permits consistency with fresh data; proportional anchor mass generally does not.
• Unbiased copied records can retain permanent uncertainty.
• Split finite error into deterministic bias and random fluctuation, and audit both allowances.
• Finite certificates and asymptotic learning are separate conclusions.

Further reading

• UC Berkeley STAT 135: Consistency ↗

  Read Consistency and convergence-in-probability definitions, lines 115–125. Used for the every-positive-tolerance sequence criterion. Source exercises are not reused. Anchor-growth schedules, copied-bit counterexample and tolerance-transfer workshop are original. No video reviewed. Checked 2026-10-25.

• UC Berkeley STAT 135: Vanishing MSE implies consistency ↗

  Read the squared-error Markov argument in lines 103–122. Used for the sufficient implication from MSE to convergence in probability. Our two-allowance finite certificate, sample-size ladder and rational derivation are original; no general MLE consistency or video claim. Checked 2026-10-25.