eBook › Chapter 6 · Sample Statistics

Section 6.1

Moment-Generating and Characteristic Functions

Consider two independent random variables \(X\) and \(Y\) with PDFs \(f_X(x)\) and \(f_Y(y)\), respectively. Let \(Z = X+Y\) be the sum of the two random variables. We know from Chapter 5 that the PDF of \(Z\), \(f_Z\), is the convolution of \(f_X\) and \(f_Y\). However, we think you will agree that convolutions are not easy to compute. Especially when the sum involves more random variables, computing the convolution would be tedious. So how should we proceed in this case? One approach is to use some kind of “frequency domain” method that transforms the PDFs to another domain and then perform multiplication instead of the convolution to make the calculations easy or at least easier. The moment-generating functions and the characteristic functions are designed for this purpose.

6.1.1Moment-generating function

Definition 6.1

For any random variable \(X\), the moment-generating function (MGF) \(M_X(s)\) is

M_X(s) = \E\left[e^{sX}\right].

The definition says that the moment-generating function (MGF) is the expectation of the random variable taken to the power \(e^{sX}\) for some \(s\). Effectively, it is the expectation of a function of random variables. The meaning of the expectation can be seen by writing out the definition. For the discrete case, the MGF is

M_X(s) = \sum_{x \in \Omega} e^{sx} p_X(x),

whereas in the continuous case, the MGF is

M_X(s) = \int_{-\infty}^{\infty} e^{sx} f_X(x) \;dx.

The continuous case should remind us of the definition of a Laplace transform. For any function \(f(t)\), the Laplace transform is

\calL[f](s) = \int_{0}^{\infty} f(t) e^{-st} \;dt.

From this perspective, we can interpret the MGF as the Laplace transform of the PDF. The argument \(s\) of the output can be regarded as the coordinate in the Laplace space. If \(s = -j\omega\), then \(M_X(-j\omega)\) becomes the Fourier transform of the PDF.

Example 6.1

Consider a random variable \(X\) with three states \(0, 1, 2\) and with probability masses \(\frac{2}{6}, \frac{3}{6}, \frac{1}{6}\) respectively. Find the MGF.

Solution

The moment-generating function is

\begin{aligned} M_X(s) = \E[e^{sX}] &= e^{s0} \cdot \frac{2}{6} + e^{s1} \cdot \frac{3}{6} + e^{s2} \cdot \frac{1}{6} \\ &= \frac{1}{3} + \frac{e^s}{2} + \frac{e^{2s}}{6}. \end{aligned}

Practice Exercise 6.1

Find the MGF for a Poisson random variable.

Solution

The MGF of Poisson random variable can be found as

\begin{aligned} M_X(s) = \E[e^{sX}] &= \sum_{x=0}^{\infty} e^{sx} \frac{\lambda^x e^{-\lambda}}{x!} = \sum_{x=0}^{\infty} \frac{\left(\lambda e^{s}\right)^x}{x!} e^{-\lambda} = e^{\lambda e^{s}} e^{-\lambda}. \end{aligned}

Practice Exercise 6.2

Find the MGF for an exponential random variable.

Solution

The MGF of an exponential random variable can be found as

\begin{aligned} M_X(s) = \E[e^{sX}] = \int_0^{\infty} e^{sx} \lambda e^{-\lambda x} \;dx = \int_0^{\infty} \lambda e^{(s-\lambda)x} \;dx = \frac{\lambda}{\lambda-s}, \quad\quad \mbox{if} \;\; \lambda > s. \end{aligned}

Why are moment-generating functions so called? The following theorem reveals the reason.

Theorem 6.1

The MGF has the properties that

sep0ex
\(M_X(0) = 1\),
\(\frac{d}{ds} M_X(s)|_{s=0} = \E[X]\), \(\frac{d^2}{ds^2} M_X(s)|_{s=0} = \E[X^2]\),
\(\frac{d^k}{ds^k} M_X(s)|_{s=0} = \E[X^k]\), for any positive integer \(k\).

Proof. The first property can be proved by noting that

\begin{aligned} M_X(0) = \E[e^{0X}] = \E[1] = 1. \end{aligned}

The third property holds because

\begin{aligned} \frac{d^k}{ds^k} M_X(s) = \int_{-\infty}^{\infty} \frac{d^k}{ds^k} e^{sx} f_X(x) \;dx = \int_{-\infty}^{\infty} x^k e^{sx} f_X(x) \;dx. \end{aligned}

Setting \(s = 0\) yields

\begin{aligned} \frac{d^k}{ds^k} M_X(s) |_{s=0} = \int_{-\infty}^{\infty} x^k f_X(x) \;dx = \E[X^k]. \end{aligned}

The second property is a special case of the third property.

■

The theorem tells us that if we take the derivative of the MGF and set \(s = 0\), we will obtain the moment. The order of the moment depends on the order of the derivative. As a result, the MGF can “generate moments” by taking derivatives. This happens because of the exponential function \(e^{sx}\). Since \(\frac{d}{ds}e^{sx} = xe^{sx}\), the variable \(x\) appears whenever we take the derivative.

Practice Exercise 6.3

Let \(X\) be a Bernoulli random variable with parameter \(p\). Find the first two moments using MGF.

Solution

The MGF of a Bernoulli random variable is

\begin{aligned} M_X(s) &= \E[e^{sX}] \\ &= e^{s0}p_X(0) + e^{s1}p_X(1) \\ &= (1)(1-p) + (e^s)(p) \\ &= 1-p + pe^s. \end{aligned}

The first and the second moment, using the derivative approach, are

\begin{aligned} \E[X] &= \frac{d}{ds}M_X(s) \bigg|_{s=0} = \frac{d}{ds} \bigg( 1-p + pe^s \bigg)\bigg|_{s=0} = pe^s \bigg|_{s=0} = p,\\ \E[X^2] &= \frac{d^2}{ds^2}M_X(s) \bigg|_{s=0} = \frac{d^2}{ds^2} \bigg( 1-p + pe^s \bigg)\bigg|_{s=0} = pe^s \bigg|_{s=0} = p. \end{aligned}

To facilitate our discussions of MGF, we summarize a few MGFs in the table below.

Distribution	PMF / PDF	\(\E[X]\)	\(\Var[X]\)	\(M_X(s)\)
Bernoulli	\(p_X(1) = p\) and \(p_X(0) = 1-p\)	\(p\)	\(p(1-p)\)	\(1-p+pe^s\)
Binomial	\(p_X(k) = {n \choose k}p^k(1-p)^{n-k}\)	\(np\)	\(np(1-p)\)	\((1-p+pe^s)^n\)
Geometric	\(p_X(k) = p(1-p)^{k-1}\)	\(\displaystyle \frac{1}{p}\)	\(\displaystyle \frac{1-p}{p^2}\)	\(\displaystyle \frac{pe^s}{1-(1-p)e^s}\)
Poisson	\(\displaystyle p_X(k) = \frac{\lambda^k e^{-\lambda}}{k!}\)	\(\lambda\)	\(\lambda\)	\(\displaystyle e^{\lambda(e^s-1)}\)
Gaussian	\(\displaystyle f_X(x) = \frac{1}{\sqrt{2\pi\sigma^2}}\exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}\)	\(\mu\)	\(\sigma^2\)	\(\displaystyle \exp\left\{\mu s + \frac{\sigma^2s^2}{2}\right\}\)
Exponential	\(f_X(x) = \lambda\exp\left\{-\lambda x\right\}\)	\(\displaystyle \frac{1}{\lambda}\)	\(\displaystyle \frac{1}{\lambda^2}\)	\(\displaystyle \frac{\lambda}{\lambda-s}\)
Uniform	\(\displaystyle f_X(x) = \frac{1}{b-a}\)	\(\displaystyle \frac{a+b}{2}\)	\(\displaystyle \frac{(b-a)^2}{12}\)	\(\displaystyle \frac{e^{sb}-e^{sa}}{s(b-a)}\)

Table 6.1. Moment-generating functions of common random variables.

6.1.2Sum of independent variables via MGF

MGFs are most useful when analyzing the PDF of a sum of two random variables. The following theorem highlights the result.

Theorem 6.2

Let \(X\) and \(Y\) be independent random variables. Let \(Z = X+Y\). Then

$$M_Z(s) = M_X(s)M_Y(s).$$

Proof. By the definition of MGF, we have that

\begin{aligned} M_Z(s) = \E\left[e^{s(X+Y)}\right] \overset{(a)}{=} \E\left[e^{sX}\right] \E\left[e^{sY}\right] = M_X(s) M_Y(s), \end{aligned}

where (a) is valid because \(X\) and \(Y\) are independent.

■

Corollary 6.1

Consider independent random variables \(X_1,\ldots,X_N\). Let \(Z = \sum_{n=1}^N X_n\) be the sum of random variables. Then the MGF of \(Z\) is

\begin{aligned} M_Z(s) = \prod_{n=1}^N M_{X_n}(s). \end{aligned}

If these random variables are further assumed to be identically distributed, the MGF is

\begin{aligned} M_Z(s) = \left(M_{X_1}(s)\right)^N. \end{aligned}

Proof. This follows immediately from the previous theorem:

\begin{aligned} M_Z(s) = \E[e^{s(X_1+\cdots+X_N)}] = \E[e^{sX_1}] \E[e^{sX_2}] \cdots \E[e^{sX_N}] = \prod_{n=1}^N M_{X_n}(s). \end{aligned}

If the random variables \(X_1,\ldots,X_N\) are i.i.d., then the product simplifies to

\prod_{n=1}^N M_{X_n}(s) = \prod_{n=1}^N M_{X_1}(s) = \left(M_{X_1}(s)\right)^N.

■

Theorem 6.3 (Sum of Bernoulli = binomial)

Let \(X_1\), …, \(X_N\) be a sequence of i.i.d. Bernoulli random variables with parameter \(p\). Let \(Z = X_1+\cdots+X_N\) be the sum. Then \(Z\) is a binomial random variable with parameters \((N,p)\).

Proof. Let us consider a sequence of i.i.d. Bernoulli random variables \(X_n \sim \mathrm{Bernoulli}(p)\) for \(n = 1,\ldots, N\). Let \(Z = X_1 + \cdots + X_N\). The moment-generating function of \(Z\) is

\begin{aligned} M_Z(s) = \E[e^{s(X_1+\cdots+X_N)}] &= \prod_{n=1}^N \E[e^{sX_n}]\\ &= \prod_{n=1}^N \left( pe^{s1} + (1-p)e^{s0} \right) = \left( pe^{s} + (1-p) \right)^N. \end{aligned}

Now, let us check the moment-generating function of a binomial random variable: If \(Z \sim \mathrm{Binomial}(N,p)\), then

\begin{aligned} M_Z(s) = \E[e^{sZ}] &= \sum_{k=0}^N e^{sk} {N \choose k} p^k (1-p)^{N-k} \\ &= \sum_{k=0}^N {N \choose k} (pe^s)^k (1-p)^{N-k} = \left(pe^s + (1-p) \right)^N, \end{aligned}

where the last equality holds because \(\sum_{k=0}^N {N \choose k} a^k b^{N-k} = (a+b)^N\). Therefore, the two moment-generating functions are identical.

■

Theorem 6.4 (Sum of binomial = binomial)

Let \(X_1\), …, \(X_N\) be a sequence of i.i.d. binomial random variables with parameters \((n,p)\). Let \(Z = X_1+\cdots+X_N\) be the sum. Then \(Z\) is a binomial random variable with parameters \((Nn,p)\).

Proof. The MGF of a binomial random variable is

\begin{aligned} M_{X_i}(s) = \left(pe^s + (1-p) \right)^n. \end{aligned}

If we have \(N\) of these random variables, then \(Z = X_1 + \cdots + X_N\) will have the MGF

\begin{aligned} M_Z(s) = \prod_{i=1}^N M_{X_i}(s) = \left(pe^s + (1-p) \right)^{Nn}. \end{aligned}

Note that this is just the MGF of another binomial random variable with parameter \((Nn,p)\).

■

Theorem 6.5 (Sum of Poisson = Poisson)

Let \(X_1\), …, \(X_N\) be a sequence of i.i.d. Poisson random variables with parameter \(\lambda\). Let \(Z = X_1+\cdots+X_N\) be the sum. Then \(Z\) is a Poisson random variable with parameter \(N\lambda\).

Proof. The MGF of a Poisson random variable is

\begin{aligned} M_X(s) = \E[e^{sX}] &= \sum_{k=0}^{\infty} e^{sk} \frac{\lambda^k}{k!} e^{-\lambda } \\ &= e^{-\lambda} \sum_{k=0}^{\infty} \frac{(\lambda e^s)^k}{k!} \\ &= e^{-\lambda} e^{\lambda e^s} = e^{\lambda(e^s-1)}. \end{aligned}

Assume that we have a sum of \(N\) i.i.d. Poisson random variables. Then, by the main theorem, we have that

\begin{aligned} M_Z(s) = [M_X(s)]^N = e^{N\lambda(e^s-1)}. \end{aligned}

Therefore, the resulting random variable \(Z\) is a Poisson with parameter \(N\lambda\).

■

Theorem 6.6 (Sum of Gaussian = Gaussian)

Let \(X_1\), …, \(X_N\) be a sequence of independent Gaussian random variables with parameters \((\mu_1,\sigma_1^2)\), …, \((\mu_N,\sigma_N^2)\). Let \(Z = X_1+\cdots+X_N\) be the sum. Then \(Z\) is a Gaussian random variable:

Z = \text{Gaussian}\bigg(\sum_{n=1}^N \mu_n, \sum_{n=1}^N \sigma_n^2\bigg).

Proof. We skip the proof of the MGF of a Gaussian. It can be shown that

M_X(s) = \exp\left\{\mu s + \frac{\sigma^2 s^2}{2}\right\}.

When we have a sequence of Gaussian random variables, then

\begin{aligned} M_Z(s) &= \E[e^{s(X_1+\cdots+X_N)}] \\ &= M_{X_1}(s) \cdots M_{X_N}(s) \\ &= \bigg(\exp\left\{\mu_1 s + \frac{\sigma_1^2 s^2}{2}\right\}\bigg)\cdots\bigg(\exp\left\{\mu_N s + \frac{\sigma_N^2 s^2}{2}\right\}\bigg)\\ &= \exp\left\{ \left(\sum_{n=1}^N \mu_n\right)s + \left(\sum_{n=1}^N \sigma_n^2 \right)\frac{s^2}{2} \right\}. \end{aligned}

Therefore, the resulting random variable \(Z\) is also a Gaussian. The mean and variance of \(Z\) are \(\sum_{n=1}^N \mu_n\) and \(\sum_{n=1}^N \sigma_n^2\), respectively.

■

6.1.3Characteristic functions

Moment-generating functions are the Laplace transforms of the PDFs. However, since the Laplace transform is defined on the entire right half-plane, not all PDFs can be transformed. One way to mitigate this problem is to restrict \(s\) to the imaginary axis, \(s = j\omega\). This will give us the characteristic function.

Definition 6.2 (Usual definition)

The characteristic function of a random variable \(X\) is

\Phi_X(j\omega) = \E[e^{j\omega X}].

However, we note that since \(\omega\) can take any value in \((-\infty,\infty)\), it does not matter if we consider \(\E[e^{-j\omega X}]\) or \(\E[e^{j\omega X}]\). This leads to the following equivalent definition of the characteristic function:

Definition 6.3 (Alternative definition (for this book))

The characteristic function of a random variable \(X\) is

\Phi_X(j\omega) = \E[e^{-j\omega X}].

If we follow this definition, we see that the characteristic function can be written as

\Phi_X(j\omega) = \E[e^{-j\omega X}] = \int_{-\infty}^{\infty} e^{-j \omega x} f_X(x) \;dx.

This is exactly the Fourier transform of the PDF. The reason for introducing this alternative characteristic function is that \(\E[e^{-j\omega X}]\) is the Fourier transform of \(f_X(x)\) but \(\E[e^{j\omega X}]\) is the inverse Fourier transform of \(f_X(x)\). The former is more convenient (in terms of notation) for students who have taken a course in signals and systems. However, we should stress that the usual way of defining the characteristic function is \(\E[e^{j\omega X}]\).

A list of common Fourier transforms is shown in the table below. Additional identities can be found in standard signals and systems textbooks.

Fourier Transforms \(f(t) \longleftrightarrow F(\omega)\)
lightgray 1.	\(e^{-at}u(t) \longleftrightarrow \frac{1}{a+j\omega}\), \(a > 0\)	10.	\(\mathrm{sinc}^2(\frac{Wt}{2}) \longleftrightarrow \frac{2\pi}{W}\Delta(\frac{\omega}{2W})\)
2.	\(e^{at}u(-t) \longleftrightarrow \frac{1}{a-j\omega}\), \(a > 0\)	11.	\(e^{-at}\sin(\omega_0 t)u(t) \longleftrightarrow \frac{\omega_0}{(a+j\omega)^2+\omega_0^2}\)
lightgray 3.	\(e^{-a\|t\|} \longleftrightarrow \frac{2a}{a^2+\omega^2}\), \(a > 0\)	12.	\(e^{-at}\cos(\omega_0 t)u(t) \longleftrightarrow \frac{a+j\omega}{(a+j\omega)^2+\omega_0^2}\)
4.	\(\frac{a^2}{a^2+t^2} \longleftrightarrow \pi a e^{-a\|\omega\|}\), \(a>0\)	13.	\(e^{-\frac{t^2}{2\sigma^2}} \longleftrightarrow \sqrt{2\pi}\sigma e^{-\frac{\sigma^2\omega^2}{2}}\)
lightgray 5.	\(te^{-at}u(t) \longleftrightarrow \frac{1}{(a+j\omega)^2}\), \(a>0\)	14.	\(\delta(t) \longleftrightarrow 1\)
6.	\(t^n e^{-at}u(t) \longleftrightarrow \frac{n!}{(a+j\omega)^{n+1}}\), \(a>0\)	15.	\(1 \longleftrightarrow 2\pi \delta(\omega)\)
lightgray 7.	\(\mathrm{rect}(\frac{t}{\tau}) \longleftrightarrow \tau \mathrm{sinc}(\frac{\omega\tau}{2})\)	16.	\(\delta(t-t_0) \longleftrightarrow e^{-j\omega t_0}\)
8.	\(\mathrm{sinc}(Wt) \longleftrightarrow \frac{\pi}{W}\mathrm{rect}(\frac{w}{2W})\)	17.	\(e^{j\omega_0t} \longleftrightarrow 2\pi \delta(\omega-\omega_0)\)
lightgray 9.	\(\Delta(\frac{t}{\tau}) \longleftrightarrow \frac{\tau}{2}\mathrm{sinc}^2(\frac{\omega\tau}{4})\)	18.	\(f(t)e^{j\omega_0t} \longleftrightarrow F(\omega-\omega_0)\)

Table 6.2. Fourier transform pairs of commonly used functions.

Example 6.2

Let \(X\) be a random variable with PDF \(f_X(x) = \lambda e^{-\lambda x}\) for \(x \ge 0\). Find the characteristic function.

Solution

The Fourier transform pair is

\lambda e^{-\lambda x} \longrightarrow \lambda \cdot \calF\bigg\{e^{-\lambda x}\bigg\} = \lambda \cdot \frac{1}{\lambda+j\omega}.

Therefore, the characteristic function is \(\Phi_X(j\omega) = \frac{\lambda}{\lambda+j\omega}\).

Example 6.3

Let \(X\) and \(Y\) be independent, and let

f_X(x) = \begin{cases} \lambda e^{-\lambda x}, &\quad x \ge 0,\\ 0 , &\quad x < 0, \end{cases} \quad\quad f_Y(y) = \begin{cases} \lambda e^{-\lambda y}, &\quad y \ge 0,\\ 0 , &\quad y < 0. \end{cases}

Find the PDF of \(Z = X+Y\).

Solution

The characteristic function of \(X\) and \(Y\) can be found from the Fourier table:

\begin{aligned} \Phi_X(j\omega) = \frac{\lambda}{\lambda+j\omega} \qquad \mbox{and} \qquad \Phi_Y(j\omega) = \frac{\lambda}{\lambda+j\omega}. \end{aligned}

Therefore, the characteristic function of \(Z\) is

\begin{aligned} \Phi_Z(j\omega) &= \Phi_X(j\omega) \Phi_Y(j\omega) = \frac{\lambda^2}{(\lambda + j\omega)^2}. \end{aligned}

By inverse Fourier transform, we have that

\begin{aligned} f_Z(z) = \calF^{-1}\left\{\frac{\lambda^2}{(\lambda + j\omega)^2}\right\} = \lambda^2 ze^{-\lambda z},\;\; z \ge 0. \end{aligned}

Why \(\Phi_X(j\omega)\) but not \(M_X(s)\)? As we said, the MGF is not always defined. Recall that the expectation \(\E[X]\) exists only when \(f_X(x)\) is absolutely integrable, or \(\E[|X|] < \infty\). For a characteristic function, the expectation is valid because \(\E[|e^{j\omega X}|] = \E[1] = 1\). However, for a moment-generating function, \(\E[|e^{s X}|]\) could be unbounded. To see a counterexample, we consider the Cauchy distribution.

Theorem 6.7

Consider the Cauchy distribution with PDF

f_X(x) = \frac{1}{\pi(x^2+1)}.

The MGF of \(X\) is undefined but the characteristic function is well defined.

Proof. The MGF is

\begin{aligned} M_X(s) &= \int_{-\infty}^{\infty} e^{sx} \frac{1}{\pi(x^2+1)} \;dx \ge \int_{1}^{\infty} e^{sx} \frac{1}{\pi(x^2+1)} \;dx \\ &\ge \int_{1}^{\infty} \frac{(sx)^3}{6\pi(x^2+1)} \;dx, \quad\mbox{because}\quad e^{sx} \ge \frac{(sx)^3}{6}\\ &\ge \int_{1}^{\infty} \frac{(sx)^3}{6\pi(2x^2)} \;dx = \frac{s^3}{12\pi} \int_{1}^{\infty}x \;dx = \infty. \end{aligned}

Therefore, the MGF is undefined. On the other hand, by the Fourier table we know that

\Phi_X(j\omega) = \calF\left\{ \frac{1}{\pi(x^2+1)} \right\} = e^{-|\omega|}.

■

Example 6.4

Let \(X_0,X_1,\ldots\) be a sequence of independent random variables with PDF

f_{X_k}(x) = \frac{a_k}{\pi(a_k^2+x^2)}, \quad a_k = \frac{1}{2^{k+1}} \quad\mbox{for} \ k= 0, 1, \dots.

Find the PDF of \(Y\), where \(Y = \sum_{k=0}^{\infty}X_k.\)

Solution

From the Fourier transform table, we know that

\begin{aligned} \frac{a_k}{\pi(a_k^2+x^2)} = \frac{1}{a_k \pi} \cdot \frac{a_k^2}{(a_k^2+x^2)} \quad \overset{\calF}{\longleftrightarrow} \quad \frac{1}{a_k \pi} \cdot \pi a_k e^{-a_k|\omega|} = e^{-a_k|\omega|}. \end{aligned}

The characteristic function of \(Y\) is

\begin{aligned} \Phi_Y(j\omega) = \prod_{k=0}^{\infty} \Phi_{X_k}(j\omega) = \exp\left\{-|\omega|\sum_{k=0}^{\infty} a_k\right\}. \end{aligned}

Since \(\sum_{k=0}^{\infty} a_k = \sum_{k=0}^{\infty} \frac{1}{2^{k+1}} = \frac{1}{2}+\frac{1}{4} + \cdots = 1\), the characteristic function becomes \(\Phi_Y(j\omega) = e^{-|\omega|}\). The inverse Fourier transform gives us

\begin{aligned} e^{-|\omega|} = \frac{1}{\pi} \cdot \pi e^{-|\omega|} \;\; \overset{\calF}{\longleftrightarrow} \;\; \frac{1}{\pi} \cdot \frac{1}{1+x^2}. \end{aligned}

Therefore the PDF of \(Y\) is

f_Y(y) = \frac{1}{\pi(1+y^2)}.

Example 6.5

Two random variables \(X\) and \(Y\) have the PDFs

f_X(x) = \begin{cases} e^{-x}, &\quad x \ge 0,\\ 0, &\quad x< 0, \end{cases} \quad\mbox{and}\quad f_Y(y) = \begin{cases} e^{-y}, &\quad y \ge 0,\\ 0, &\quad y < 0. \end{cases}

Find the PDF of \(Z = \max(X,Y) - \min(X,Y)\).

Solution

We first show that

\begin{aligned} Z = \max(X,Y) - \min(X,Y) = |X-Y|. \end{aligned}

Suppose \(X > Y\), then \(\max(X,Y) = X\) and \(\min(X,Y) = Y\). So \(Z = X - Y\). If \(X < Y\), then \(\max(X,Y) = Y\) and \(\min(X,Y) = X\). So \(Z = Y- X\). Combining the two cases gives us \(Z = |X-Y|\). Now, consider the Fourier transform of the PDFs:

\begin{aligned} e^{-x} \;\; \overset{\calF}{\longleftrightarrow}\;\; \frac{1}{1+j\omega}. \end{aligned}

Let \(U = X-Y\), and let \(Z = |U|\). The characteristic function is

\begin{aligned} \Phi_U(j\omega) &= \E[e^{-j\omega (X-Y)}] = \E[e^{-j\omega X}] \E[e^{j\omega Y}]\\ &= \frac{1}{1+j\omega} \cdot \frac{1}{1-j\omega} = \frac{1}{1+\omega^2} \;\; \overset{\calF}{\longleftrightarrow} \;\; f_U(u) = \frac{1}{2}e^{-|u|}. \end{aligned}

With the PDF of \(U\), we can find the CDF of \(Z\):

\begin{aligned} F_Z(z) = \Pb[Z \le z] &= \Pb[|U| \le z]\\ &= \int_{-z}^{z} f_U(u) \;du \\ &= \int_{-z}^{z} \frac{1}{2}e^{-|u|} \;du \\ &= 2 \int_0^{z} \frac{1}{2}e^{-u} \;du = 1-e^{-z}. \end{aligned}

Hence, the PDF is

\begin{aligned} f_Z(z) = \frac{d}{dz} F_Z(z) = e^{-z}. \end{aligned}

Closing remark. Moment-generating functions and characteristic functions are useful tools. In this section, we have confined our discussion to using them to compute the sum of two random variables. Later sections and chapters will explain further uses for these functions. For example, we use the MGFs when proving Chernoff's bound and proving the Central Limit Theorem.