Statistical Measures - Expectation, Variance & Standard Deviation
Random Variables & Measures
A random variable is just a real valued function over a population:
\[r:\Omega\rightarrow\Bbb R\]Using a probability distribution:
\[P:\Omega\rightarrow [0,1]\]we can find the expected value $E[X]$ of a random variable.
This is also sometimes called the average value or the mean value.
This is calculated using the following summation:
\[E[X]=\sum_{X\in\Omega}P[X]r(X)\]This is the probability of choosing an element multiplied by the value of the associated random variable.
This $E[X]$ represents the typical value or what we expect to happen.
Dice Example
The expected value of throwing a fair die is $\frac{7}{2}$. This is calculated by:
\[E[X]=\sum^6_{k=1}\frac{1}{6}k=\frac{21}{6}=3.5\]Suppose we are using a biased die with the following probabilities:
\[P[x]=\begin{cases}\frac14 &\text{if }X\in\{1,2,3\}\\\frac1{12}&\text{if }X\in\{4,5,6\}\end{cases}\]We can then work out the expected value to be:
\[E[X]=\frac64+\frac{15}{12}=\frac{11}4=2.75\]Mode & Median
Sometimes means can be misleading due to outliers; mode and median don’t take into account these outliers. You calculate them as you expect.
Variance
You may find that for a dice (with the same population and basis for random variables) you get distinct outcomes.
The idea of variance allows you to see the spread of the random variables are.
Formal Definition
We have a population $\Omega$ and a random variable $r(X)$; leading to expectation $E[X]$ using probability distribution $P[X]$.
The variance $\text{Var}(X)$ is defined to be:
\[\sum_{X\in\Omega}(r(X)-E[X])^2\]Variance gives a measure of by how much the population as a whole differs from the expected value.
This is always non-negative.
Standard Deviation
Formally the exact standard deviation is:
\[\sigma \equiv\sqrt{\text{Var}(X)}\]The variance is defined with respect to the whole population but it is not always feasible to compute that.
Estimated Standard Deviation
One way of estimating is by using a sampling method according the the probability distribution.
We take $N$ samples from $\Omega$:
\[\langle y_1,y_2,\ldots,y_N\rangle:y_k=r(X_k)\]The estimated standard deviation is:
\[S_N\equiv\sqrt{\frac{\sum^N_{i=1}(y_i-E[Y])^2}N}\]This is the variance of the sample, divided by the number of samples chosen. This is then rooted to find the standard deviation.
Notice that:
\[E[Y]=\frac{\sum^N_{i=1}y_i}N\]This method captures how the population behaves b looking at a sample of its members.
This method sometimes gives underestimates.
Bessel’s Correction
This method attempts to correct for underestimates when estimating standard deviation.
Take $N$ sample from $\Omega$:
\[\langle y_1,y_2,\ldots,y_N\rangle:y_k=r(X_k)\]The Bessel’s correction for estimated standard deviation:
\[S^B_N\equiv\sqrt{\frac{\sum^N_{i=1}(y_i-E[Y])^2}{N-1}}\]The change is that we divide by the number of samples minus one.
Significance Testing
A typical experiment will have:
- A predicted outcome: $X$
- An actual outcome: $Y$
We want to know the chance of out prediction begin accurate given the outcome is likely.
To assess this we count the number of standard deviations by which $Y$ differs from $X$. If there are too many then the hypothesis isn’t valid.