Predicting with uncertain models

What you will learn

Compute prior and posterior predictive probabilities in a finite latent-rate model without replacing uncertain type by a single guess.

Prerequisites: Sequential updates · Uncertain opponent frequencies · Testing event independence

Two different unknowns

Suppose an artificial opponent has one persistent hidden type. Low type produces an action called success with probability 1/4; high type produces it with probability 3/4. Each type has prior probability 1/2. Actions are independent conditional on type. Success is only a binary action label, not a profitable bet or a hand-strength claim. These supplied rates are not observations about real players.

Even knowing the type would leave the next action random. Not knowing the type adds uncertainty about which action distribution applies. A posterior probability of high type answers a question about the hidden model; the next-action predictive probability answers a different question about an outcome.

Weight the future by the current information

Before data, the next-success probability is (1/2)(1/4)+(1/2)(3/4)=1/2. This is a prior predictive calculation. After a first success, Bayes gives high-type posterior (1/2×3/4)/(1/2×1/4+1/2×3/4)=3/4. The next-success probability becomes (1/4)(1/4)+(3/4)(3/4)=5/8.

The most likely type now has rate 3/4, but predicting 3/4 would discard the remaining low-type possibility. The weight 3/4 is also not itself the predictive probability. For this one Bernoulli action, averaging rates works exactly; that does not justify plugging an average parameter into every nonlinear prediction.

A shared type links fresh observations

For two fresh successes after the first observation, first square each type rate, then average: (1/4)(1/4)²+(3/4)(3/4)²=7/16. Multiplying the two marginal predictions gives (5/8)²=25/64 instead, which is smaller. The draws are independent given type, not after mixing over an unknown shared type.

If a new independent type were sampled for each action, the joint prediction would describe a different experiment. State whether a latent parameter persists. All calculations here are exact consequences of a stipulated two-type model; they do not establish its empirical fit, calibration or usefulness for an actual opponent.

Hands-on: Construct predictions through a shared hidden type

Filter equal-mass model worlds, select fresh-success slots and distinguish posterior weights, single predictions and two-action joint probabilities.

Reading predictive workshop…

Common mistakes

• Using the most likely type as if it were known.
• Confusing the posterior probability of a type with that type’s action rate or the next-action prediction.
• Squaring a marginal prediction when both future actions share an unknown persistent type.
• Treating eight possible worlds as eight empirical observations or carrying evidence between the separate workshop models.

Keep these ideas

• Predict outcomes by averaging their conditional probabilities over current model weights.
• Separate posterior model uncertainty from fresh outcome randomness.
• For nonlinear events, average the event probability within each model before mixing.
• Conditional independence can disappear when averaging over a persistent hidden type.

Further reading

• Stan User’s Guide 2.40 — Posterior predictive distribution and sampling ↗

  Bounded review of the predictive-distribution definition and parameter-versus-outcome uncertainty sections. The workshop uses an original exact two-type finite model, not Stan inference or the guide’s Poisson example. No whole-guide or video review claim. Checked 2026-10-22.