Random Variables

Definition: Quantities of interest that are a function of a random outcome are called random variables.

Definition: $F(x)=P\{X\le x\}$ is called the cumulative distribution function of the random variable $X$. $F(x)$ is a nondecreasing function of $x$

Definition: A random variable that takes on a countable number of possible values is said to be discrete. $p_X(a)=P\{X=a\}$ is the probability mass function of $X$.

We can say $\sum _{i=1}^{\infty }p(x_i)=1$ where $X$ assumes one of $x_1,x_2,\dots$

Definition: $E[X]=\sum _{x}xp(x)$ is the expectation of the discrete random variable $X$

We can also calculate the expected value of a function over a random variable, $E[g(X)]=\sum _{i}g(x_i)p(x_i)$

Some random variables are uncountable, say the time that a train arrives at a specified stop.

Definition: $X$ is a continuous random variable if there exists a nonnegative function $f$ defined for all $x\in (-\infty ,\infty )$ such that for any set $B$ of real numbers, $P\{X\in B\}={\int }_B^{}f(x)dx$. We call $f_X$ the probability density function of $X$

We get $1=P\{X\in (-\infty ,\infty )\}={\int }_{-\infty }^{\infty }f(x)dx$

Any question about $X$ can be answered in terms of $f$

The expectation of a continuous random variable is $E[X]={\int }_{-\infty }^{\infty }xf(x)dx$, and likewise the expectation over a function of $X$ is $E[g(x)]={\int }_{-\infty }^{\infty }g(x)f(x)dx$

While expectation is a useful measure of a random variable’s statistical properties, we’d also like to have some measure of its spread.

Definition: If $X$ is a random variable with mean $\mu =E[X]$, then the variance of $X$ is $\text{Var}(x)=E[(X-\mu {)}^2]$

We can derive that $\text{Var}(x)=E[X^2]-(E[X]{)}^2$

Definition: A Bernoulli random variable has PMF $p_X(0)=1-p$, $p_X(1)=p$ for some $0\le p\le 1$

Definition: A binomial random variable represents the number of successes in $n$ independent trials with success probability $p$. We get $p(i)=\left(\genfrac{}{}{0ex}{}{n}{i}\right)p^i(1-p{)}^{n-i}$ for $i=0,1,\dots ,n$.

A Bernoulli random variable is a binomial random variable with $n=1$

We can derive that $E[X]=np$ where $X$ is a binomial random variable
We can also derive $\text{Var}(X)=np(1-p)$

Definition: A Poisson random variable has $p_X(i)=P\{X=i\}=e^{-\lambda }\frac{{\lambda }^i}{i!}$ for $i=0,1,2,\dots$

For a large $n$ and smaller $p$ such that $\lambda =np$ is of “moderate” size, the Poisson distribution approximates the binomial distribution

Interestingly, $E[X]=\text{Var}(X)=\lambda$

It turns out the Poisson distribution is a good approximation, even when the trials are weakly dependent: Consider $n$ events, with $p_i$ equal to the probability that event $i$ occurs. If all $p_i$ are “small” and the trials are at most “weakly dependent,” then the number of these events that occur has a Poisson distribution with mean ${\sum }_{i=1}^n{p}_i$.

Other common distributions include,

• The geometric random variable $P\{X=n\}=(1-p{)}^{n-1}p$ where $X$ is the number of trials required until a success occurs. $E[X]=\frac{1}{p}$, $\text{Var}(X)=\frac{1-p}{p^2}$
• The negative binomial random variable $P\{X=n\}=\left(\genfrac{}{}{0ex}{}{n-1}{r-1}\right)p^r(1-p{)}^{n-r}$ where $X$ is the number of trials required to accumulate $r$ successes. $E[X]=\frac{r}{p}$ and $\text{Var}(X)=\frac{r(1-p)}{p^2}$
• The hypergeometric random variable $P\{X=i\}=\frac{\left(\genfrac{}{}{0ex}{}{m}{i}\right)\left(\genfrac{}{}{0ex}{}{N-m}{n-i}\right)}{\left(\genfrac{}{}{0ex}{}{N}{n}\right)}$ where $X$ can represent the number of white balls selected from an urn with $N$ balls (of which $m$ are white). $E[X]=\frac{nm}{N}$ and $\text{Var}(x)=np(1-p)\left(1-\frac{n-1}{N-1}\right)$ where $p=\frac{m}{N}$
• The Zeta distribution, $p\{X=k\}=\frac{C}{k^{\alpha +1}}$