Score, curvature and information

Build local likelihood derivatives, distinguish observed curvature from expected information and test an erasing sensor.

42 minutes · intermediate · Original Poker4Quant lesson

What you will learn

Derive report-level scores and curvature, reconcile Fisher information through two expectations, and diagnose zero observed slope versus a parameter-free acquisition model.

Prerequisites: Likelihood from coarsened observations · Consistency and information growth · Variance and standard deviation

Specify a local question and an acquisition mechanism

A likelihood tells us how plausible a fixed report sequence is across candidate parameter values. A local question asks how that likelihood changes when we move a short distance from a stated reference point. The score is the first derivative of its natural logarithm; negative curvature is minus the second derivative. If derivatives are new, read the first as instantaneous slope and the second as change in slope. We use d log(p)/dp=1/p and d log(1−p)/dp=−1/(1−p), differentiating once more to obtain the curvature contributions below. These identities are enough to follow the complete calculation without a general calculus course.

Assume four independent probes. Each underlying binary result has the same fixed success probability p, strictly between zero and one. A sensor independently retains each result with known probability r, regardless of whether it succeeded. Otherwise it emits Erased, with no hidden outcome revealed. Retention decisions are independent across probes and independent of all results. Thus the three report probabilities are rp, r(1−p), and 1−r. The sensor mechanism is stipulated, not estimated. Outcome-dependent missingness, shared sensor failures and drifting rates would require a different likelihood. This fictional acquisition example is not an empirical model of poker decisions or market observations.

Differentiate the report that was actually observed

For an ordered dataset with s successes, f failures and e erasures, the likelihood is (rp)^s[r(1−p)]^f(1−r)^e. Only feasible reports are allowed: r=1 excludes erasures, and r=0 allows erasures only. At interior p and positive retention, its p-dependent log terms are s log p+f log(1−p). The score is S=s/p−f/(1−p), and observed negative curvature is J=s/p²+f/(1−p)². The sensor factors disappear from these derivatives because r is known and does not change with p. Erased records are present in the acquisition ledger but add zero to either derivative.

The score is signed: positive slope favors a small upward move locally. Curvature measures how rapidly that slope bends downward. In this model J is nonnegative, but it is not the square of S. With s=1 and f=3 at p=1/4, the slopes cancel to zero while J=64/3 is positive. The log likelihood has a stationary peak there. Calling this dataset uninformative because its slope is zero confuses being at a peak with having a flat curve. An all-erased dataset has S=J=0, although its sensor could have produced informative reports on another run.

Average over possible reports to measure the design

Before seeing data, the single-report slope takes values 1/p, −1/(1−p), and zero with probabilities rp, r(1−p), and 1−r. Its mean is r−r=0. Its second moment is rp/p²+r(1−p)/(1−p)²=r/[p(1−p)]. This is the single-probe Fisher information in the p coordinate. Independence makes the four scores add, and their zero means cancel cross products in the expected square of the sum. Hence I_4(p)=4r/[p(1−p)]. Averaging negative curvature gives the same result: four times [rp/p²+r(1−p)/(1−p)²]. These are two routes through the whole report space, not a replacement of expected counts by observed counts.

The reference parameter inside the derivatives must match the parameter generating the averaging probabilities for these identities. A score evaluated at an arbitrary reference can have nonzero mean under a different population. Information also depends on parameter units: changing coordinates changes derivative scale. It is neither payoff value nor a probability that an estimate is correct. At fixed interior p, halving independent retention halves expected information per attempted probe. At r=0 every possible report is Erased and the likelihood is constant in p. This is genuine nonidentification, unlike a zero slope for one dataset from a design with r>0.

Derive and independently check a local prediction

Take r=1/2, p=1/4 and the four reports Success, Failure, Erased, Erased. Their probabilities are 1/8, 3/8, 1/2, 1/2. The ordered likelihood is 3/256. Observed score is 4−4/3=8/3; observed curvature is 16+16/9=160/9. Expected information is 4(1/2)/[(1/4)(3/4)]=32/3. The observed curvature exceeds its design average here. Independently average all three single-report cases: their mean score is (1/8)4+(3/8)(−4/3)=0, and expected squared score is 2+2/3=8/3. Multiplying by four independently checked probes gives 32/3 again.

For a small move h, the quadratic prediction of log-likelihood change is hS−h²J/2. With h=1/16 the prediction is 1/6−5/144=19/144, approximately 0.131944. Check against a calculation that never differentiates: the new point is 5/16, so the success likelihood factor changes by 5/4 and the failure factor by 11/12. Erasure factors cancel. The exact likelihood ratio is 55/48, with natural logarithm approximately 0.136132. The approximation misses by about 0.004188 in log units. A visible difference is expected at a finite step; this calculation supplies neither a rigorous remainder bound nor a confidence interval.

Use edits to separate data evidence from expected sensitivity

In the workshop, cycle each of four report markers through the outcomes allowed by the selected sensor. This changes the observed likelihood without changing its design. Keep p and r fixed, make all reports Erased, and compare the zero observed derivatives with positive expected information. Next restore a success and three failures and inspect p=1/4: zero slope accompanies positive curvature. Finally choose complete erasure; every feasible dataset now has zero derivatives and the model contains no p. These controlled changes distinguish a dataset accident, a stationary point, and an acquisition mechanism that cannot identify its parameter.

Move the reference point and reverse the small step, then audit score, curvature, expected information and quadratic log change using fractions or finite decimals within one millionth. The exact log change is displayed for comparison. Each model or report edit clears old answers. The endpoint buttons intentionally leave the interior calculation: at p=0 or p=1 some report probabilities vanish and our common-support derivative argument no longer applies. Diagnose the excluded boundary instead of interpreting an infinity as usable precision. Drafts last for this visit and confer no account credit. This lesson establishes exact finite-model derivative identities and tests a local approximation; it does not establish an estimator’s finite-sample error law, confidence coverage or a universal accuracy claim.

Worked example

Four independent half-retained probes report Success, Failure, Erased, Erased. At p=1/4, derive observed and expected sensitivity and compare an upward step of 1/16 with the exact likelihood change.

Write L(p)=(p/2)((1−p)/2)(1/2)²=p(1−p)/16, giving L(1/4)=3/256.
Differentiate log L: S=1/p−1/(1−p)=8/3. Differentiate again and negate: J=1/p²+1/(1−p)²=160/9.
Average single-report slopes with masses (1,3,4)/8. Their mean is zero and second moment is 8/3. Four independent probes give I_4=32/3.
The local quadratic log change is (1/16)(8/3)−(1/512)(160/9)=19/144.
Independently L(5/16)=55/4096, so the exact ratio is (55/4096)/(3/256)=55/48. Its log is approximately 0.136132.
Changing the observed reports changes S and J, while expected information stays 32/3 at the same p and sensor.

Observed curvature 160/9 and expected information 32/3 answer different questions. The local prediction 19/144 approximates, rather than equals, the exact log ratio.

Hands-on: Edit a local likelihood

Manipulate four report markers and sensor retention, reconcile observed derivatives with expected information, and test a local quadratic prediction.

Edit a local likelihood

Four independent probes have success rate p. Known retention r is independent of results. The sensor reports Success, Failure or Erased. Edit reports independently of the reference point.

Choose the acquisition design

Retention r = 0.5. Changing sensors resets the four reports to a feasible sequence.

Cycle the four report markers

Each press cycles one actual report through the outcomes this sensor permits. This changes the dataset, not its acquisition probabilities.

Move the reference and local step

At reference p=0.25, h=0.0625. Exact log-likelihood change to p+h: 0.136132174. Compare your quadratic prediction below with this exact finite step.

Read the single-report derivative ledger

Weights describe possible reports, not this dataset’s histogram. Each row gives probability, score and negative curvature.

Success: probability 0.125000; slope 4.000000; negative curvature 16.000000.

Failure: probability 0.375000; slope -1.333333; negative curvature 1.777778.

Erased: probability 0.500000; slope 0.000000; negative curvature 0.000000.

Diagnose the model before calculating

For interior p, enter the four totals. Fractions or finite decimals are accepted within 0.000001. At an excluded boundary, only the diagnosis is audited.

Edit four reports, then audit local sensitivity. Draft lasts for this visit.

Common mistakes

Squaring a dataset’s total score and calling it observed negative curvature.
Replacing expected retention by the number retained in this run.
Treating a zero score at a peak as zero Fisher information.
Averaging under a different parameter while invoking the zero-mean-score identity.
Using interior formulas at endpoints with vanishing report probabilities.
Interpreting a quadratic approximation as exact or as confidence coverage.

Keep these ideas

Differentiate the observed-data log likelihood with the acquisition mechanism fixed.
Observed score and curvature depend on the actual reports; Fisher information averages over possible reports.
Independent zero-mean scores give additive information.
Zero observed slope, a flat realized likelihood and a parameter-free design are distinct.
State the interior parameter domain and compare local approximations with exact changes.