Expectations

This page covers how to calculate expectations of random variables and some of the more important properties of expectations. The appendix will include more in depth coverage and proofs. As always, this serves as a reference guide. For a more holistic understanding, please visit our lessons.

Note: Unless otherwise specified, the properties and calculations below work on any random variable (independent and dependent) with any shaped distribution (except distributions with no means like Cauchy) with one exception.  The exception is where the distribution has no mean and this occurs if both the expectations of positive and negative values are infinite. This special case is discussed below.

Calculating Expectations

The expected value of discrete random variable X is given below.  In a nutshell, we just sum up the product of x and its respective probability.

$E(X) = \sum_{all\, x}xf(x)$

Where X is a discrete random variable, f(X) is the probability mass function of X, and one of the following is finite:

$\sum_{negative\, x}xf(x)$

$\sum_{positive\, x}xf(x)$

If both of the above are infinite, then $E(X)$ does not exist.  If only one is infinite and the other is finite, then the expectation is just positive or negative infinity. 

The expected value of continuous random variable X is given below.  In a nutshell, we just sum up the product of x and its respective likelihood.

$E(X) = \int_{-\infty}^\infty xf(x)dx$

Where X is a continuous random variable, f(X) is the probability density function of X, and one of the following is finite:

$\int_0^\infty xf(x)dx$

$\int_{-\infty}^0 xf(x)dx$

If both of the above are infinite, than $E(X)$ does not exist.  If only one is infinite and the other is finite, then the expectation is just infinity

The expected value of a function of a single random variable is given below.  In a nutshell, just multiply that function by the pdf or pmf and then sum up the values. This makes intuitive sense.  The outcome of r(X) has a probability of occurring equal to the probability of X occurring, and thus, we just calculate the weighted average of r(X).  You can think of it as first we transform X, and then we determine the expected value of transformed X using the respective probabilities. 

For a discrete random variable:

$E[r(X)] = \sum_{all\, r(x)}xf(x)$

For a continuous random variable:

$E[r(X)] = \int_{-\infty}^\infty r(x)f(x)dx$

Where in both of the above, r(X) is a real value function of X, f(X) is either a pdf or pmf, and the mean of f(X) exists.

Expectation of a function with more than one random variables:  

Just like in the single random variable case, we just multiply the function r(X) by the pdf or pmf (in this case joint) and then sum up the values.  The outcome of r(X) has a probability of occuring equal to the probability of x occurring, and thus, we just calculate the weighted average of r(X).

For a discrete joint distribution:

$E[r(X)] = \sum_{all\, x_1,\dots,x_n} r(x_1,\dots,x_n)f(x_1,\dots,x_n)$

For a continuous distribution, we have an integral for each random variable.

$E[r(X)] = \idotsint r(x_1,\dots,x_n)f(x_1,\dots,x_n)dx_1,\dots,dx_n$

Where in both of the above, r(X) is a real value function of n random variables, f(X) is either a joint pdf or pmf, and the mean of f(X) exists.

Expectation of the sum of several different random variables: This is a general property and applies in all cases except for the non-finite mean case.  Applies even if your variables are not independent.

If we have X1 to Xn different random variables, and each of the random variables have finite expectations, then the expectation of the sum of the random variables equals the sum of the expectations of each random variable. That is:

$E(X_1 + \dots + X_n) = E(X_1) + \dots + E(X_n)$

Note that the expectation of the sum of several random variables always equals the sum of their individual expectations, regardless of what their joint distribution is.  That is, the above holds true even IF the random variables themselves are dependent or correlated to each other.  A real life example is the expected return of a portfolio is just the sum of the expected returns of the stocks, even though we know some returns are correlated with each other.  Dependency only affects variance calculations.  Also, this means that when it comes to sampling with replacement, we are also able to add up the expectations of each trial.

Thus, I believe, the expectation of the SUM (linear combination) of random variables is just equal to the sum of the expected values of each random variable.  

No comment yet, add your voice below!


Have a suggestion, question, or found an error?