B Probability Reminders

B.1 Independence

Definition B.1 Two random variables \(X\) and \(Y\) are (statistically) independent, if \(P(X\in A, Y\in B) = P(X\in A) P(Y\in B)\) for all sets \(A\) and \(B\).

We list some properties of independent random variables:

  • If \(X\) and \(Y\) are independent, and if \(f\) and \(g\) are functions, then \(f(X)\) and \(g(Y)\) are also independent.

B.2 Expectation

The expectation or mean of a random variable \(X\) is denoted \(E(X)\). Some properties of expectations are:

  • Linearity: For all constants \(a\) and \(b\), and random variables \(X\) and \(Y\) we have \[\begin{equation*} \mathbb{E}(aX + bY) = a\mathbb{E}(X) + b\mathbb{E}(Y). \end{equation*}\]

  • For every constant \(c\) we have \[\begin{equation*} \mathbb{E}(c) = c. \end{equation*}\]

  • For independent random variables \(X\) and \(Y\) we have \[\begin{equation*} \mathbb{E}(XY) = \mathbb{E}(X)\mathbb{E}(Y). \end{equation*}\]

B.3 Variance

The variance of a random variable \(X\), denoted \(\mathop{\mathrm{Var}}(X)\), measures its spread around the mean. Important properties include:

  • For constants \(a\) and \(b\), and a random variable \(X\) we have \[\begin{equation*} \mathop{\mathrm{Var}}(aX + b) = a^2 \mathop{\mathrm{Var}}(X). \end{equation*}\]

  • For every constant \(c\) we have \[\begin{equation*} \mathop{\mathrm{Var}}(c) = 0 \end{equation*}\]

  • For independent random variables \(X\) and \(Y\) we have \[\begin{equation*} \mathop{\mathrm{Var}}(X + Y) = \mathop{\mathrm{Var}}(X) + \mathop{\mathrm{Var}}(Y). \end{equation*}\]

  • The variance can be computed as \[\begin{equation*} \mathop{\mathrm{Var}}(X) = \mathbb{E}\bigl( (X - \mathbb{E}(X))^2 \bigr) = \mathbb{E}(X^2) - (\mathbb{E}(X))^2. \end{equation*}\]

B.4 Covariance

The covariance between two random variables \(X\) and \(Y\), denoted \(\mathop{\mathrm{Cov}}(X, Y)\), measures their linear relationship. Key properties are:

  1. Symmetry: \[\begin{equation*} \mathop{\mathrm{Cov}}(X, Y) = \mathop{\mathrm{Cov}}(Y, X) \end{equation*}\]

  2. For constants \(a\), \(b\), \(c\) and \(d\), and random variables \(X\) and \(Y\) we have \[\begin{equation*} \mathop{\mathrm{Cov}}(aX + b, cY + d) = ac \mathop{\mathrm{Cov}}(X, Y). \end{equation*}\]

  3. For independent random variables \(X\) and \(Y\) we have \[\begin{equation*} \mathop{\mathrm{Cov}}(X, Y) = 0. \end{equation*}\]

  4. The covariance can be computed as \[\begin{equation*} \mathop{\mathrm{Cov}}(X, Y) = \mathbb{E}\bigl( (X - \mathbb{E}(X)) (Y - \mathbb{E}(Y)) \bigr) = \mathbb{E}(XY) - \mathbb{E}(X) \mathbb{E}(Y). \end{equation*}\]

  5. We have \(Var(X) = Cov(X, X)\).

B.5 Well-Known Distributions

B.5.1 The Chi-Squared Distribution

Definition B.2 Let \(X_1, \ldots, X_\nu \sim \mathcal{N}(0, 1)\) be i.i.d. Then the distribution of \(\sum_{i=1}^\nu X_i^2\) is called the \(\chi^2\)-distribution with \(\nu\) degrees of freedom. The distribution is denoted by \(\chi^2(\nu)\).

Some important properties of the \(\chi^2\)-distribution are:

  • \(\chi^2\)-distributed random variables are always positive.

  • If \(Y\sim \chi^2(\nu)\), then \(\mathbb{E}(Y) = \nu\) and \(\mathop{\mathrm{Var}}(Y) = 2\nu\).

  • The R command pchisq(|\(x\),\(\nu\)) gives the value \(\Phi_\nu(x)\) of the CDF of the \(\chi^2(\nu)\)-distribution.

  • The R command qchisq(\(\alpha\),\(\nu\)) can be used to obtain the \(\alpha\)-quantile of the \(\chi^2(\nu)\)-distribution.

  • More properties can be found on Wikipedia.

B.5.2 The t-distribution

Definition B.3 Let \(Z \sim \mathcal{N}(0,1)\) and \(Y \sim \chi^2(\nu)\) be independent. Then the distribution of \[\begin{equation} T = \frac{\,Z\,}{\,\sqrt{Y / \nu}\,} \tag{B.1} \end{equation}\] is called the \(t\)-distribution with \(\nu\) degrees of freedom. This distribution is denoted by \(t(\nu)\).

Some important properties of the \(t\)-distribution are:

  • The \(t\)-distribution is symmetric: if \(T \sim t(\nu)\), then \(-T \sim t(\nu)\)

  • If \(T\sim t(\nu)\), then \(\mathbb{E}(T) = 0\).

  • The R command pt(|\(x\),\(\nu\)) gives the value \(\Phi_\nu(x)\) of the CDF of the \(t(\nu)\)-distribution.

  • The R command qt(\(\alpha\),\(\nu\)) can be used to obtain the \(\alpha\)-quantile of the \(t(\nu)\)-distribution.

  • More properties can be found on Wikipedia.