Posterior beliefs and error costs

Build a complete decision policy from posterior probabilities and an explicit asymmetric loss table.

15 minutes · advanced · Original Poker4Quant lesson

What you will learn

Derive a two-action minimum-expected-loss threshold, preserve boundary ties and explain why a probability forecast alone does not determine an action.

Prerequisites: Predicting with uncertain models · Expected value as a weighted average

Separate inference from the decision objective

Let p be the posterior probability of a state labeled positive. There are two possible reports: positive or negative. A false positive reports positive when the true state is negative; a false negative does the reverse. In this artificial exercise, correct reports cost zero and the two errors have stipulated nonnegative loss points. Positive is a state name, not a good outcome.

The posterior summarizes the model’s beliefs about states. A loss table specifies how decisions are evaluated. Holding p fixed while changing an error cost can change the preferred report without changing what the evidence says. A posterior of 60% does not by itself specify the action when errors have different costs. We minimize expected loss; we are not inferring a learner’s personal preferences.

Weight each error by the state where it happens

Write cFP for the false-positive cost and cFN for the false-negative cost. Reporting positive has expected loss (1−p)cFP, because it is wrong only in negative states. Reporting negative has expected loss p cFN. These are expectations over mutually exclusive states, not costs to be added from two simultaneous errors.

For positive costs, a positive report is strictly preferred when (1−p)cFP < p cFN, or p > cFP/(cFP+cFN). Below that threshold a negative report is preferred; at equality both minimize loss. The familiar 50% cutoff is a special case of equal error costs, not a universal consequence of Bayes’ rule.

A complete policy includes its boundaries and assumptions

The workshop’s nine posterior cards p=0/8 through8/8 are separate supplied contexts. Select every minimizing action at each card. Both actions are legitimate minimizers at an exact threshold; calling a tie a strict recommendation loses information. Costs are positive in all supplied models, so the threshold denominator is nonzero.

The loss table omits additional actions, constraints and future consequences. It also assumes p is the relevant state probability and that these costs represent the objective. A precise optimum under a poor model remains a conditional result. This exercise is not a medical, hiring, financial or gambling recommendation, and its artificial labels are not a trained classifier.

Worked example

Keep p=3/8. Compare equal unit error costs with a false-negative cost of3and false-positive cost1.

With equal costs, positive-report loss is5/8and negative-report loss3/8; report negative minimizes loss.
After changing only cFN to3, positive-report loss stays5/8. Negative-report loss becomes(3/8)×3=9/8.
The new indifference point is1/(1+3)=1/4. Since3/8>1/4, reporting positive now minimizes expected loss.
Neither calculation updates p. Only the cost of one kind of mistake changed.

The same posterior below50% supports different reports under different loss tables. Minimize the stated objective rather than guessing the most likely state by habit.

Hands-on: Build a loss table and a complete posterior policy

Allocate separate error costs, select every minimizing report at nine supplied posterior cases, inspect both true-state alternatives and reconcile expected losses and boundary probabilities.

Reading loss-policy workshop…

Common mistakes

Using p to weight a false positive even though that error happens in negative states.
Using50%regardless of error costs.
Claiming cost changes are new evidence and should alter the posterior.
Selecting only one action at a genuine indifference boundary.

Keep these ideas

Beliefs and decision costs are separate inputs.
Positive-report loss is(1−p)cFP; negative-report loss is p cFN.
For positive costs, the report threshold is cFP/(cFP+cFN).
A minimum-loss policy is conditional on its model, action set and objective.