Convolution: Addition of independent random variables

When X, Y are independent variables and Z=X+Y we can easily calculate the mean and variance of Z as

(1)   \begin{align*} E(Z) &= E(X) + E(Y) \\ \Var(Z) &= \Var(X) + \Var(Y)  \end{align*}

Many times however we are also interested in other characteristics of Z. This information is incorporated in its distribution function F_{Z} and definitely in its density function f_{Z}. The question addressed here is to derive f_{Z} whenever the density functions of X,Y, respectively, f_{X} and f_{Y} are known. Independence does NOT imply that f_{Z} = f_{X} + f_{Y}. Note that, we then have \int_{\Zc} f_{Z}(z) dz = 2 and rescaling by a .5 won’t save us either. To derive the density for Z explicitly we use the definition of density functions, ie, that function f_{Z} for which we have

(2)   \begin{align*} P(Z \leq s)= \int_{-\infty}^{s} f_{Z}(z) dz. \end{align*}

As Z depends on both the values of X,Y we have to condition on either of these variables, in this case, Y=y. Let us use these ideas as follows:

(3)   \begin{align*} P(Z \leq s) & = P(X+Y \leq s) = P( X \leq s - y \, | \, Y=y) P(Y=y), \\ & = \int_{\Yc} \int_{-\infty}^{s-y} f_{X\, | \, Y=y}(x\, | \, y) dx f_{Y}(y) dy = \int_{\Yc} \int_{-\infty}^{s-y} f_{X}(x ) dx f_{Y}(y) dy, \\ & = \int_{\Yc} \int_{-\infty}^{s} f_{X}(x -y) dx f_{Y}(y) dy, \\ & = \int_{\Yc} F_{X}(s-y) f_{Y}(y) dy, \end{align*}

where the equality on the second line is due to independence of X,Y and as usual we have written F_{X} for the distribution function of X. The density function for Z is thus given by differentiation with respect to s.

(4)   \begin{align*} f_{Z}(s) & = \frac{d}{d s} F_{Z}(Z \leq s) = \frac{d}{d s} \int_{\Yc} F_{X}(s-y) f_{Y}(y) dy = \int_{\Yc} f_{X}(s-y) f_{Y}(y) dy \\ & = \int_{\Xc} f_{Y}(s-x) f_{X}(x) dx, \end{align*}

where we have used the symmetry to derive the latter. Hence, if we know the form of f_{X}, f_{Y} we can mix these density with each other, regardless of order, to derive the density function f_{Z}.

Leave a Reply