Statistical independence

Similarly, when we assert that two random variables are independent, we intuitively mean that knowing something about the value of one of them does not yield any information about the value of the other. For example, the number appearing on the upward face of a die the first time it is thrown and that appearing the second time are independent.

Independent events

The standard definition says:

Two events A and B are independent iff P(A ∩ B)=P(A)P(B).

More generally, any collection of events -- possibly more than just two of them -- are mutually independent iff for any finite subset A₁, ..., A_n of the collection we have

If two events A and B are independent, then the conditional probability of A given B is the same as the "unconditional" (or "marginal") probability of A, that is,

When one recalls that the conditional probability P(A | B) is given by

Independent random variables

Two random variables X and Y are independent iff for any numbers a and b the events [X ≤ a] and [Y ≤ b] are independent events as defined above. Similarly an arbitrary collection of random variables -- possible more than just two of them -- is independent precisely if for any finite collection X₁, ..., X_n and any finite set of numbers a₁, ..., a_n, the events [X₁ ≤ a₁], ..., [X_n ≤ a_n] are independent events as defined above.

The measure-theoretically inclined may prefer to substitute events [X ∈ A] for events [X ≤ a] in the above definition, where A is any Borel set. That definition is exactly equivalent to the one above when the values of the random variables are real numbers. It has the advantage of working also for complex-valued random variables or for random variables taking values in any topological space.

If any two of a collection of random variables are independent, they may nonetheless fail to be mutually independent; this is called pairwise independence.

If X and Y are independent, then the expectation operator E has the nice property

E[X· Y] = E[X] · E[Y]

var(X + Y) = var(X) + var(Y).

var(X + Y) = var(X) + var(Y) + 2 cov(X, Y).

Furthermore, random variables X and Y with distribution functionss F_X(x) and F_Y(y), and probability densities f_X(x) and f_Y(y), are independent if and only if the combined random variable (X,Y) has a joint distribution

Conditionally independent random variables

Intuitively, two random variables X and Y are conditionally independent given Z if, once Z is known, the value of Y does not add any additional information about X. For instance, two measurements X and Y of the same underlying quantity Z are not independent, but they are conditionally independent given Z (unless the errors in the two measurements are somehow connected).

The formal definition of conditional independence is based on the idea of conditional distributions. If X, Y, and Z are discrete random variables, then we define X and Y to be conditionally independent given Z if
: P(X = x, Y = y | Z = z) = P(X = x | Z = z) · P(Y = y | Z = z)
for all x, y and z such that P(Z = z) > 0. On the other hand, if the random variables are continuous and have a joint probability density function p, then X and Y are conditionally independent given Z if
: p_XY|Z(x, y | z) = p_X|Z(x | z) · p_Y|Z(y | z)
for all real numbers x, y and z such that p_Z(z) > 0.

If X and Y are conditionally independent given Z, then
: P(X = x | Y = y, Z = z) = P(X = x | Z = z)
for any x, y and z with P(Z = z) > 0. That is, the conditional distribution for X given Y and Z is the same as that given Z alone. A similar equation holds for the conditional probability density functions in the continuous case.

Independence can be seen as a special kind of conditional independence, since probability can be seen as a kind of conditional probability given no events.

See also:

• Independent identically-distributed random variables.