Covariance through conditioning and sign reversal

Separate paired residual products from moving report centers, then construct and remove a pooled association reversal.

34 minutes · intermediate · Original Poker4Quant lesson

What you will learn

Derive total covariance and explain a negative-within, positive-pooled sign reversal by independently manipulating within-report pairing and mixture weights.

Prerequisites: Conditional variance and the variance budget · Repeated bets and dependence

Define paired measurements before interpreting association

Assume an exact two-stage experiment. First select report A with probability w=1/3 and report B with probability 2/3. Within A select uniformly from the two paired slots (X,Y)=(0,2),(2,0). Within B select uniformly from four distinct slots (4,6),(4,6),(6,4),(6,4). Repeated pairs occupy different slots. At this initial mixture every slot happens to have mass 1/6, but that equality disappears when w changes. X and Y are stipulated numerical measurements on the same selected slot. They are not independent draws, estimated returns or causal effects. All moments exist; use population probability weights without a sample correction.

Covariance averages a product of two centered measurements: Cov(X,Y)=E[(X−μX)(Y−μY)]. A positive product occurs when both measurements lie on the same side of their respective population means; a negative product occurs when they lie on opposite sides. Unlike a squared deviation, a product can be negative. Its units multiply the units of X and Y. Equal units here are a convenience, not a requirement. Changing the pairing can change covariance even when each separate measurement distribution stays fixed. Reading only the two marginal histograms therefore cannot answer the association question.

Separate local pairing from movement of report centers

Inside A, both conditional means are 1. The centered pairs are (−1,+1) and (+1,−1), so each residual product is −1 and Cov(X,Y given A)=−1. Inside B, both means are 5. Its four centered pairs again have opposite signs and product −1. Thus the probability-weighted within covariance W=(1/3)(−1)+(2/3)(−1)=−1. This averages covariances, not correlations or absolute products. Taking a square or removing the minus signs would answer a different question.

Now replace each measurement by its report-dependent conditional mean. The pair of centers is (1,1) in A and (5,5) in B. Both centers rise together across reports, creating positive between covariance. Since μX=μY=(1/3)1+(2/3)5=11/3, the between term is B=(1/3)(1−11/3)²+(2/3)(5−11/3)²=64/27+32/27=32/9. The squares appear only because the two conditional centers coincide in this particular example. In a different model the centers might move in opposite directions and the between covariance would be negative. Do not import the nonnegative-component guarantee from total variance.

Derive total covariance and check the pooled sign independently

Write aS=E[X given S] and bS=E[Y given S]. For a fixed report, expand XY=(X−aS)(Y−bS)+aS(Y−bS)+bS(X−aS)+aS bS. The two middle averages vanish within that report: aS and bS are fixed there, while each conditional residual has mean zero. Average over reports to obtain E[XY]=E[Cov(X,Y given S)]+E[aS bS]. Subtract E[X]E[Y]. Iterated expectation identifies the means of aS and bS with the unconditional means. Therefore Cov(X,Y)=E[Cov(X,Y given S)]+Cov(aS,bS)=W+B. Residuals need not be independent of each other or of the report.

For the stipulated slots, the result is −1+32/9=23/9, a positive pooled covariance despite negative covariance inside both reports. Independently calculate products without centering: XY values are 0,0,24,24,24,24. Their uniform average is 96/6=16. The raw means both equal 22/6=11/3. Thus E[XY]−E[X]E[Y]=16−121/9=23/9. A direct centered-product check gives numerators 55,55,7,7,7,7 over denominator 9, whose average is 138/54=23/9. These checks use different intermediate quantities, making a missed weight or incorrect sign easier to detect.

Change the mixture without changing either local relationship

Keep every conditional slot fixed but vary w. Each A slot has mass w/2 and each B slot mass (1−w)/4. Both local covariances remain −1, so their weighted average remains −1. The two-center covariance becomes 16w(1−w). Consequently C(w)=−1+16w(1−w). At w=1/2 it is 3; at w=1/12 it is 2/9; at w=0 it is −1 because only B occurs. Setting C(w)=0 gives w=(2−√3)/4 or (2+√3)/4. The pooled covariance is positive strictly between these roots, approximately 0.067 and 0.933. At the roots it is zero, not positive.

This sign reversal describes different comparison populations. It does not show that increasing X causes Y to fall within a report or rise in the pooled population. No intervention or causal assignment was specified. Nor does the identity tell us which reporting partition is scientifically appropriate. If a real dataset motivated this calculation, its sampling process, measurement definitions and relevant grouping variables would require separate justification. At w=0 or w=1, a report has zero probability and cannot be interpreted as an observed conditional population. Its specified slot rule remains available for a later mixture, but contributes nothing to the current pooled calculation.

Use controlled pairing swaps to diagnose the explanation

In the workshop select two slots in the same report to exchange their Y measurements. X values, report masses and both conditional marginal distributions remain fixed. Cross-report exchanges are rejected because they would change conditional means. Swapping A slots turns its local covariance from −1 to +1. Swapping B slots 3 and 5 produces residual products +1,−1,+1,−1, whose average is zero. With the initial w=1/3 and A unchanged, that B swap changes W to −1/3 while B remains 32/9; pooled covariance becomes 29/9. The manipulation isolates pairing effects from report-center effects.

Construct a reversal first, then remove it by setting the A mixture tokens to zero. Return to four tokens and compare a within-report pairing swap. Enter the weighted within covariance, the between covariance and the pooled covariance, then decide whether every active report is negative while the pooled covariance is positive. Changing a pairing or weight clears prior answers. The displayed ledger gives conditional residual products and slot masses, so reconstruct each weighted contribution rather than copying a final total. A zero within covariance can reflect cancellation of opposite products; it alone cannot certify independence. Completion is a local reasoning audit, separate from lesson-check or account progress.

Worked example

For reports A=(0,2),(2,0) and B=(4,6),(4,6),(6,4),(6,4), use w=1/3. Reconcile pooled covariance and then exchange Y at B slots 3 and 5.

A centers are (1,1), B centers (5,5); every initial conditional residual product is −1, so W=−1.
Both pooled means equal 11/3. Between covariance is (1/3)(−8/3)²+(2/3)(4/3)²=32/9.
Total covariance is −1+32/9=23/9. Independently, E[XY]=96/6=16 and 16−(11/3)²=23/9.
Exchange B Y values at slots 3 and 5: B pairs become (4,4),(4,6),(6,6),(6,4). Their residual products are +1,−1,+1,−1, giving covariance zero.
A retains covariance −1, so W=−1/3. The centers and B=32/9 stay fixed; total is 29/9.
Independently, the new six raw products are 0,0,16,24,36,24. Their average is 50/3; subtract 121/9 to obtain 29/9.

Within-report pairing and movement of report centers are separate contributions. Their sum can reverse the local association sign without establishing causation.

Hands-on: Separate pairing from mixture effects

Exchange paired Y measurements within reports, reweight the report mixture, reconstruct both covariance terms and diagnose a pooled sign reversal.

Separate pairing from mixture effects

Choose report A with mass w, then uniformly choose one of its two slots; B has four slots. Exchange Y values only within a report. Slot probabilities change with the mixture. Draft lasts for this visit; no account credit.

A mixture tokens: 4/12. w=0.333333. Both pooled means: 3.666667. Display rounded to six decimals.

Report A: mass 0.333333, centers (1, 1), conditional covariance -1.

Report B: mass 0.666667, centers (5, 5), conditional covariance -1.

Pairing ledger

Residual product uses each report’s own centers. Multiply it by slot mass to reconstruct the within term.

Slot 1: mass 0.166667, residual product -1.

Slot 2: mass 0.166667, residual product -1.

Slot 3: mass 0.166667, residual product -1.

Slot 4: mass 0.166667, residual product -1.

Slot 5: mass 0.166667, residual product -1.

Slot 6: mass 0.166667, residual product -1.

Select two slots in one report to exchange their Y measurements.

Common mistakes

Squaring residuals instead of multiplying the X and Y residuals.
Using an unweighted average of report covariances after mixture weights change.
Assuming both total-covariance components must be nonnegative like variance components.
Calling a pooled association a causal effect or proof of within-report behavior.
Conditioning on an inactive report or treating zero covariance as proof of independence.

Keep these ideas

Total covariance equals weighted conditional covariance plus covariance of conditional means.
A mixture can reverse association signs when between-center movement dominates.
Local pairing swaps preserve marginal distributions but change conditional covariance.
Changing report weights changes the population being compared.
Zero-mass groups contribute nothing, and association alone gives no causal conclusion.