Transforming distributions and losing information

What you will learn

Derive the law of a nonlinear transformation from an existing PMF, reconcile its expectation and prove event recoverability or information loss with a collision witness.

Prerequisites: From outcomes to random variables · Expected value as a weighted average

Start from a law, then change the report

Assume a finite numerical measurement X with possible values −3, −1, 0, 1 and 3. Their probabilities, in that order, are 1/8, 2/8, 1/8, 3/8 and 1/8. These are stipulated model weights, not estimated frequencies, poker winnings or a uniform distribution over five labels. All calculations refer to one observation. No independence assumption is needed because we are transforming the same observation, rather than adding another random experiment. The input could represent a signed error in a toy prediction system. Its sign records direction, while its magnitude records size.

A report Y=g(X) applies a fixed numerical function to that measurement. We already know the law of X, so we need not reconstruct every physical outcome that originally produced it. List the supported x values, evaluate g at each, and combine the original masses whenever reported values coincide. In symbols, pY(y)=Σ pX(x), with the sum over all supported x satisfying g(x)=y. This finite formula uses addition of masses. It requires neither derivatives nor a density correction. It also does not square, clip or otherwise transform the probabilities themselves.

The starting PMF is enough to determine the law of any deterministic function of X. It would not be enough for a report that also depended on an unspecified second variable. Keep the function and its domain explicit: an upper clip min(X,2) changes only values above 2 and leaves negative values alone. An absolute-value clip would be a different operation.

Collisions change support, not total probability

For Y=X², the input values become 9, 1, 0, 1 and 9. The supported report values are therefore 0, 1 and 9, with probabilities 1/8, 5/8 and 2/8. The two signs of magnitude one collide, as do the two signs of magnitude three. The middle report is much more likely than either extreme even though there are three report labels. A complete output table must retain those unequal masses. Its sum is (1+5+2)/8=1.

For a general threshold t, find P(Y≤t) by including every supported input with g(x)≤t. Squaring an inequality requires care: X²≤1 means −1≤X≤1. It does not mean merely X≤1, which would incorrectly include −3. Here the selected input weights are 2+1+3=6, so P(Y≤1)=6/8. At t=0 the probability is 1/8; just below zero it is zero. Between 1 and 9 it remains 6/8. These flat regions and jumps are exact consequences of the finite support.

An affine report such as 2−X is one-to-one on this support. It changes the labels and reverses their numerical ordering, but each report still identifies one input value. Squaring behaves differently because distinct supported inputs share a report. Normalization cannot distinguish these information properties: both reports have total mass one.

Transformation order is part of the model

Consider two processing instructions: square a number, and clip it from above at 2. Squaring first produces Z=min(X²,2), with outputs 2, 1, 0, 1 and 2. The resulting law has masses 1/8 at 0, 5/8 at 1 and 2/8 at 2. Clipping first produces W=min(X,2)², with outputs 9, 1, 0, 1 and 4. Its masses are 1/8 at 0, 5/8 at 1, 1/8 at 4 and 1/8 at 9. The same named operations do not define the same report unless their order is specified.

The input −3 is a useful boundary probe. An upper clip leaves it at −3, so clipping first and then squaring gives 9. Squaring first gives 9 and clipping afterward gives 2. The input 3 gives another probe: the respective outputs are 4 and 2. At −1, 0 and 1 the pipelines agree, so checking only small inputs would miss the difference. Testing a convenient example is weaker than checking the whole supported domain.

Neither pipeline changes how much original probability each input carries. What changes is which inputs become indistinguishable. The first pipeline merges the two extreme signs into output 2. The second retains their distinction as outputs 9 and 4. Report design should therefore consider the question that the downstream learner or decision system must answer.

Recover an event or prove that the report erased it

Say that an original event can be recovered from Y alone when there is a set of report values B such that the event is exactly {Y∈B} on the stated support. Inspect each output bucket. If every input in that bucket gives the same yes-or-no answer to the original event, keep the bucket when the answer is yes and discard it when the answer is no. This gives a constructive recovery procedure. If a bucket mixes yes and no, no selection of complete report buckets can reproduce the original event.

For the squared report, the event X≠0 is recoverable: select Y=1 or Y=9. The event X<0 is not recoverable. Inputs −1 and 1 have the same report 1, but only −1 satisfies the event. They form a collision witness. Choosing report 1 includes both; omitting it excludes both. Unequal probabilities do not repair this pointwise ambiguity. Under the specified PMF we can still calculate P(X<0)=3/8, but that population probability does not let us decide whether a particular reported 1 came from a negative input.

For W=min(X,2)², the event X=3 is recoverable by selecting report 4. For Z=min(X²,2), it is not recoverable because −3 and 3 both report 2. This is a question about the supported values, not whether the formula has an inverse on every real number. Even squaring is invertible on an entirely nonnegative support. Always state the support before declaring information lost.

Average the transformed values, then cross-check

To compute an expected report, multiply g(x) by the original pX(x) and add. Equivalently, first construct the report PMF and average its output values. The two calculations must agree because grouping a finite sum does not change it. For the squared report, the input-row calculation is (9×1+1×2+0×1+1×3+9×1)/8=23/8. The grouped calculation is (0×1+1×5+9×2)/8=23/8.

The input mean is (−3−2+0+3+3)/8=1/8. Squaring that mean gives 1/64, which is not the squared-report mean 23/8. Linearity permits moving an affine transformation through expectation; it does not grant that permission for arbitrary nonlinear functions. The discrepancy here is a concrete calculation, not a rounding issue. After clipping the square, the mean is 9/8; clipping before squaring gives 18/8. A change of processing order can change both support and average.

In the activity, change the report and the original query. Construct the set of report values that recovers the query, or mark it unrecoverable and select two input values as a collision witness. Then independently enter the mean and cutoff probability. The displayed law helps you reason, but a correct average alone cannot substitute for the event proof. Use the hint after diagnosing whether your mistake concerns masses, operation order or information loss.

Hands-on: Recover events or construct a collision witness

Compare three processing pipelines, select complete report buckets or prove information loss, and audit transformed means and cutoff probabilities.

Common mistakes

• Transforming probabilities instead of input values. Original mass travels unchanged.
• Using only the positive square root when recovering a squared event on signed support.
• Assuming min(X²,2) equals min(X,2)²; probe −3 as well as 3.
• Calling an event recoverable because its population probability is known. Recovery requires a consistent answer within every report bucket.
• Replacing E[g(X)] with g(E[X]) for a nonlinear g.

Keep these ideas

• Apply the function to supported inputs, then sum masses at coincident outputs.
• Operation order and support determine the transformed law.
• Recoverable events are unions of whole output buckets; mixed buckets supply impossibility witnesses.
• Check the transformed mean both rowwise and from its grouped law.

Further reading

• Karl B. Gregory, University of South Carolina: Transformations of a random variable (2022) ↗

  Read PDF slides 2–4: inverse images, event probabilities and transformed discrete PMFs. Bounded review only; no continuous transformation theorem or source exercises reused. Original signed-error laws, processing pipelines and event-recovery activity. Checked 2026-10-24.

• Data 140: Expectations of Functions (Spring 2017) ↗

  Read linear and nonlinear function rules, domain/range grouping and the finite expectation method. Original examples and collision-witness proofs; no textbook exercise or video review claimed. Checked 2026-10-24.