Lecture 13 The Cramér–Rao Inequality

In lecture 10 we have found the best unbiased estimators by conditioning on a complete sufficient statistic. In contrast, in this lecture we will discuss a question which could be asked about any unbiased estimator: What are the limitations of unbiased estimators? In the situation of lecture 11, we will see that the variance of an unbiased estimator cannot be smaller than $1/\mathcal{I}_{n}(\theta)$ . This is the Cramér–Rao inequality, which gives a benchmark against which every unbiased estimator can be measured and which we will use in lecture 17 to characterise the maximum likelihood estimator. In this lecture we will prove the inequality, in a form which covers estimators for functions of the parameter, and then see which estimators achieve the bound. We will conclude by considering the uniform distribution, where the conditions for the bound to hold are violated and where it is possible to beat the bound.

13.1 The Inequality

Throughout this lecture, $X_{1},\dots,X_{n}$ are i.i.d. with density or probability weights $f(x;\theta)$ , the parameter space $\Theta\subseteq\mathbb{R}$ is an open interval, and the conditions of section 11.1 hold. The score $\ell^{\prime}(\theta)$ and the Fisher information $\mathcal{I}_{n}(\theta)=\mathop{\mathrm{Var}}\nolimits_{\theta}\bigl{(}\ell^{% \prime}(\theta)\bigr{)}$ are as in lecture 11; we assume in addition that $0<\mathcal{I}_{n}(\theta)<\infty$ for all $\theta\in\Theta$ , and we write $f(x;\theta)=\prod_{i=1}^{n}f(x_{i};\theta)$ for the joint density of a value $x=(x_{1},\dots,x_{n})$ of the sample.

The idea of the proof can be stated in advance: the covariance between an unbiased estimator $W$ for $g(\theta)$ and the score equals $g^{\prime}(\theta)$ , for any model, and then the Cauchy–Schwarz inequality shows that a small Fisher information forces a large variance. We state the result for the covariance as a separate lemma.

Lemma 13.1.

Let $W=W(X_{1},\dots,X_{n})$ be a statistic such that $\mathop{\mathrm{Var}}\nolimits_{\theta}(W)<\infty$ for all $\theta\in\Theta$ , and assume that the expectation of $W$ can be differentiated under the integral sign, i.e. that

equation (13.1) (13.1)

\frac{\mathrm{d}}{\mathrm{d}\theta}\int W(x)\,f(x;\theta)\,\mathrm{d}x=\int W(% x)\,\frac{\partial}{\partial\theta}f(x;\theta)\,\mathrm{d}x,

with a sum in place of the integral for discrete models. Then we have

\frac{\mathrm{d}}{\mathrm{d}\theta}\mathbb{E}_{\theta}(W)=\mathop{\mathrm{Cov}% }\nolimits_{\theta}\bigl{(}W,\ell^{\prime}(\theta)\bigr{)}

for all $\theta\in\Theta$ .

Proof.

Since the score has mean zero (lemma 11.1), the covariance is given by the expectation of the product $\mathop{\mathrm{Cov}}\nolimits_{\theta}\bigl{(}W,\ell^{\prime}(\theta)\bigr{)}% =\mathbb{E}_{\theta}\bigl{(}W\ell^{\prime}(\theta)\bigr{)}$ . As in equation (11.2), the score of the sample is the ratio $\ell^{\prime}(\theta)=\frac{\partial}{\partial\theta}f(X;\theta)\big{/}f(X;\theta)$ of the joint density and thus the density cancels when we take the expectation:

\displaystyle\mathbb{E}_{\theta}\bigl{(}W\ell^{\prime}(\theta)\bigr{)}

\displaystyle=\int W(x)\,\frac{\frac{\partial}{\partial\theta}f(x;\theta)}{f(x% ;\theta)}\,f(x;\theta)\,\mathrm{d}x

\displaystyle=\int W(x)\,\frac{\partial}{\partial\theta}f(x;\theta)\,\mathrm{d}x

\displaystyle=\frac{\mathrm{d}}{\mathrm{d}\theta}\int W(x)\,f(x;\theta)\,% \mathrm{d}x

\displaystyle=\frac{\mathrm{d}}{\mathrm{d}\theta}\mathbb{E}_{\theta}(W),

where the integrals are over the support of the distribution, which does not depend on $\theta$ . ∎

The assumption (13.1) is an additional exchange of differentiation and integration. For exponential families, this relation holds for all statistics with finite variance, by a similar argument from analysis as given in the proof of lemma 7.10, but we will not verify this relation for individual examples.

Often the quantity of interest is a function $g(\theta)$ of the parameter rather than $\theta$ itself, for example the probability $e^{-\theta}$ of zero counts in example 10.2. We therefore state the inequality for an unbiased estimator for $g(\theta)$ .

Theorem 13.2 (Cramér–Rao inequality).

Assume the conditions of section 11.1 and $0<\mathcal{I}_{n}(\theta)<\infty$ , and let $g\colon\Theta\to\mathbb{R}$ be differentiable. Let $W$ be an unbiased estimator for $g(\theta)$ with $\mathop{\mathrm{Var}}\nolimits_{\theta}(W)<\infty$ which satisfies (13.1). Then we have

\mathop{\mathrm{Var}}\nolimits_{\theta}(W)\geq\frac{g^{\prime}(\theta)^{2}}{% \mathcal{I}_{n}(\theta)}

for all $\theta\in\Theta$ .

Proof.

Since $W$ is unbiased, we have $\mathbb{E}_{\theta}(W)=g(\theta)$ , and lemma 13.1 gives $\mathop{\mathrm{Cov}}\nolimits_{\theta}\bigl{(}W,\ell^{\prime}(\theta)\bigr{)}% =g^{\prime}(\theta)$ . Using the Cauchy–Schwarz inequality, lemma A.8, for $W$ and $\ell^{\prime}(\theta)$ , we find

g^{\prime}(\theta)^{2}=\mathop{\mathrm{Cov}}\nolimits_{\theta}\bigl{(}W,\ell^{% \prime}(\theta)\bigr{)}^{2}\leq\mathop{\mathrm{Var}}\nolimits_{\theta}(W)\,% \mathop{\mathrm{Var}}\nolimits_{\theta}\bigl{(}\ell^{\prime}(\theta)\bigr{)}=% \mathop{\mathrm{Var}}\nolimits_{\theta}(W)\,\mathcal{I}_{n}(\theta),

where we used definition 11.2 for the last step. Dividing by the positive number $\mathcal{I}_{n}(\theta)$ gives the claim. This completes the proof. ∎

The most important special case is the estimation of $\theta$ itself.

Corollary 13.3.

Assume the conditions of section 11.1 and $0<\mathcal{I}_{n}(\theta)<\infty$ . Let $W$ be an unbiased estimator for $\theta$ with $\mathop{\mathrm{Var}}\nolimits_{\theta}(W)<\infty$ which satisfies (13.1). Then we have

\mathop{\mathrm{Var}}\nolimits_{\theta}(W)\geq\frac{1}{\mathcal{I}_{n}(\theta)}

for all $\theta\in\Theta$ .

Proof.

This is the case $g(\theta)=\theta$ of theorem 13.2, where $g^{\prime}(\theta)=1$ . ∎

Two comments help to understand the corollary. First, by proposition 11.4 the bound is $1/\bigl{(}n\mathcal{I}(\theta)\bigr{)}$ : no unbiased estimator can have a standard deviation which decreases faster than the $1/\sqrt{n}$ of the sample mean (lemma 2.2). Second, the bound is independent of the estimator used, and only depends on the model: the bound measures how much the data can reveal about $\theta$ , regardless of the unbiased method used to estimate it. For the normal mean with known variance the bound is $\sigma^{2}/n=\mathop{\mathrm{Var}}\nolimits_{\mu}(\bar{X})$ , as shown in example 11.7, and thus $\bar{X}$ achieves this bound.

It is the general form of theorem 13.2 which is worth memorising: identify the function $g$ the estimator is unbiased for, differentiate, square and divide by the information. Both numerator and denominator depend on how the model is parametrised, but by proposition 11.9 the bound itself does not: it is a property of the estimator and the model, not of the name we give to the parameter. Exercise 13.6 verifies this.

13.2 Efficiency and Attainment

We give a name to the estimators which attain the bound.

Definition 13.4.

Let $W$ be an unbiased estimator for $g(\theta)$ that satisfies the assumptions of theorem 13.2. Then the efficiency of $W$ is given by

\operatorname{eff}_{\theta}(W)=\frac{g^{\prime}(\theta)^{2}/\mathcal{I}_{n}(% \theta)}{\mathop{\mathrm{Var}}\nolimits_{\theta}(W)}.

The estimator $W$ is called efficient, if $\operatorname{eff}_{\theta}(W)=1$ for all $\theta\in\Theta$ , i.e. if its variance equals the Cramér–Rao bound for every $\theta$ .

From theorem 13.2 we know that the efficiency lies in the interval $[0,1]$ . An efficient estimator is a UMVUE in the sense of definition 10.4 amongst the unbiased estimators satisfying (13.1), since none of these can beat the bound. The converse is not true, because the bound is not always attained: in exercise 13.4, the UMVUE for the rate of the exponential distribution has efficiency $(n-2)/n$ .

The proof of theorem 13.2 suggests when the bound is attained: equality in the Cauchy–Schwarz step requires $W-g(\theta)$ to be a multiple of the score. To make this precise, we compare $W-g(\theta)$ with the multiple $c(\theta)\,\ell^{\prime}(\theta)$ , where $c(\theta)=g^{\prime}(\theta)/\mathcal{I}_{n}(\theta)$ is the regression coefficient $\mathop{\mathrm{Cov}}\nolimits_{\theta}\bigl{(}W,\ell^{\prime}(\theta)\bigr{)}% \big{/}\mathop{\mathrm{Var}}\nolimits_{\theta}\bigl{(}\ell^{\prime}(\theta)% \bigr{)}$ of $W$ on the score, and we show that the variance of $W$ exceeds the bound by the mean squared difference between the two.

Proposition 13.5.

Under the assumptions of theorem 13.2, the estimator $W$ is efficient if and only if there is a function $c(\theta)$ such that, for every $\theta\in\Theta$ , we have

equation (13.2) (13.2)

W(X)-g(\theta)=c(\theta)\,\frac{\partial}{\partial\theta}\log f(X;\theta)

with probability one, where $X=(X_{1},\dots,X_{n})$ and $f(x;\theta)$ is the joint density of the sample. In this case we have $c(\theta)=g^{\prime}(\theta)/\mathcal{I}_{n}(\theta)$ .

Proof.

The derivative in (13.2) is the score $\ell^{\prime}(\theta)$ of the sample. We first derive an identity which holds for every unbiased estimator $W$ , efficient or not, and then use it to prove the two directions of the claim. Consider the random variable

R=W-g(\theta)-\frac{g^{\prime}(\theta)}{\mathcal{I}_{n}(\theta)}\,\ell^{\prime% }(\theta).

Since $W$ is unbiased and the score has mean zero (lemma 11.1), we have $\mathbb{E}_{\theta}(R)=0$ and thus $\mathbb{E}_{\theta}(R^{2})=\mathop{\mathrm{Var}}\nolimits_{\theta}(R)$ . By lemma 13.1 we have $\mathop{\mathrm{Cov}}\nolimits_{\theta}\bigl{(}W,\ell^{\prime}(\theta)\bigr{)}% =g^{\prime}(\theta)$ , and by definition 11.2 we have $\mathop{\mathrm{Var}}\nolimits_{\theta}\bigl{(}\ell^{\prime}(\theta)\bigr{)}=% \mathcal{I}_{n}(\theta)$ . Expanding the variance of $R$ thus gives

\displaystyle\mathbb{E}_{\theta}(R^{2})

\displaystyle=\mathop{\mathrm{Var}}\nolimits_{\theta}(W)-2\,\frac{g^{\prime}(% \theta)}{\mathcal{I}_{n}(\theta)}\mathop{\mathrm{Cov}}\nolimits_{\theta}\bigl{% (}W,\ell^{\prime}(\theta)\bigr{)}+\frac{g^{\prime}(\theta)^{2}}{\mathcal{I}_{n% }(\theta)^{2}}\mathop{\mathrm{Var}}\nolimits_{\theta}\bigl{(}\ell^{\prime}(% \theta)\bigr{)}

\displaystyle=\mathop{\mathrm{Var}}\nolimits_{\theta}(W)-2\,\frac{g^{\prime}(% \theta)^{2}}{\mathcal{I}_{n}(\theta)}+\frac{g^{\prime}(\theta)^{2}}{\mathcal{I% }_{n}(\theta)}

\displaystyle=\mathop{\mathrm{Var}}\nolimits_{\theta}(W)-\frac{g^{\prime}(% \theta)^{2}}{\mathcal{I}_{n}(\theta)}.

Now assume that $W$ is efficient. Then the right-hand side is zero for every $\theta$ . Since the non-negative random variable $R^{2}$ then has expectation zero, we have $R=0$ with probability one, which is (13.2) with $c(\theta)=g^{\prime}(\theta)/\mathcal{I}_{n}(\theta)$ . Conversely, assume that (13.2) holds for some function $c(\theta)$ . Taking the covariance of both sides with $\ell^{\prime}(\theta)$ gives $g^{\prime}(\theta)=c(\theta)\mathcal{I}_{n}(\theta)$ by lemma 13.1, and thus $c(\theta)=g^{\prime}(\theta)/\mathcal{I}_{n}(\theta)$ . Then $R=0$ with probability one, the variance of $W$ equals the bound and $W$ is efficient. This completes the proof. ∎

Since the left-hand side $\mathbb{E}_{\theta}(R^{2})$ of the identity in the proof is non-negative, the identity also gives a second proof of theorem 13.2. The bound is the part of the variance of $W$ which is explained by the score, and an efficient estimator is one for which nothing is left over.

In an exponential family, the attainment condition identifies an efficient estimator directly.

Corollary 13.6.

Let the model be a one-parameter exponential family as in definition 7.1, with density

f(x;\theta)=h(x)\exp\bigl{(}\eta(\theta)T(x)-B(\theta)\bigr{)},

where $\eta$ is twice continuously differentiable with $\eta^{\prime}(\theta)\neq 0$ for all $\theta$ , and let $T_{n}=\sum_{i=1}^{n}T(X_{i})$ be the natural statistic. Then $T_{n}/n$ is an efficient estimator for the mean $\mu(\theta)=\mathbb{E}_{\theta}\bigl{(}T(X_{1})\bigr{)}$ .

Proof.

The score of a single observation is $\ell_{i}^{\prime}(\theta)=\eta^{\prime}(\theta)T(X_{i})-B^{\prime}(\theta)$ . Since the score has mean zero (lemma 11.1), taking expectations gives $B^{\prime}(\theta)=\eta^{\prime}(\theta)\mu(\theta)$ , and thus $\ell_{i}^{\prime}(\theta)=\eta^{\prime}(\theta)\bigl{(}T(X_{i})-\mu(\theta)% \bigr{)}$ . Summing over $i$ and dividing by $n\eta^{\prime}(\theta)\neq 0$ , we find

\frac{T_{n}}{n}-\mu(\theta)=\frac{1}{n\,\eta^{\prime}(\theta)}\,\ell^{\prime}(% \theta),

which is (13.2) for $W=T_{n}/n$ , $g=\mu$ and $c(\theta)=1/\bigl{(}n\eta^{\prime}(\theta)\bigr{)}$ . It remains to check the assumptions of theorem 13.2. The conditions of section 11.1 hold for exponential families, as noted there. The estimator $T_{n}/n$ is unbiased for $\mu(\theta)$ , and it has finite variance, since we have $T(X_{i})=\mu(\theta)+\ell_{i}^{\prime}(\theta)/\eta^{\prime}(\theta)$ and the score has finite variance by the conditions of section 11.1. Finally, $T_{n}/n$ satisfies (13.1) by the remark after lemma 13.1. Thus, by proposition 13.5, the estimator $T_{n}/n$ is efficient. This completes the proof. ∎

The condition (13.2) is restrictive. With the score from the proof of corollary 13.6, an efficient estimator $W$ for $g(\theta)$ in an exponential family satisfies

W=g(\theta)+c(\theta)\,n\,\eta^{\prime}(\theta)\Bigl{(}\frac{T_{n}}{n}-\mu(% \theta)\Bigr{)}.

Since $W$ is a statistic, it cannot depend on $\theta$ , and thus $W=a+bT_{n}/n$ for constants $a$ and $b$ . Then $g(\theta)=\mathbb{E}_{\theta}(W)=a+b\,\mu(\theta)$ : the only functions of $\theta$ which have an efficient estimator are the affine functions of the mean of the natural statistic. For the Poisson distribution, an exponential family with $T(x)=x$ and $\mu(\theta)=\theta$ , the sample mean is thus an efficient estimator for $\theta$ ; exercise 13.3 considers $g(\theta)=e^{-\theta}$ instead.

13.3 When the Conditions Fail

As we have seen, the proof of the inequality in this lecture involves two instances of differentiating under the integral sign, once in lemma 11.1 and once in the assumption (13.1). If the support of the distribution moves as $\theta$ does, both of these exchanges can fail, since differentiating an integral whose limits depend on $\theta$ produces an additional term from the boundary, which the exchange omits.

Example 13.7.

Let $X_{1},\dots,X_{n}$ be i.i.d. with a uniform distribution on $(0,\theta)$ and $\theta>0$ . The support of the distribution depends on $\theta$ and the conditions from section 11.1 are violated. Thus, theorem 13.2 does not apply. We can still see what the formula would be for this model: in example 11.10 the three expressions of lemma 11.3 had three different values, and taking the only positive one of these three values, $\mathbb{E}_{\theta}\bigl{(}\ell_{1}^{\prime}(\theta)^{2}\bigr{)}=1/\theta^{2}$ , and multiplying by $n$ as in proposition 11.4, we find the formal expression $n/\theta^{2}$ and the formal bound $\theta^{2}/n$ .

Now consider the estimators for $\theta$ based on the maximum $M=\max_{i}X_{i}$ . The MLE $M$ is biased by exercise 2.2, and thus its small mean squared error does not violate the inequality, which is a statement about unbiased estimators only. The bias-corrected estimator $W=(n+1)M/n$ is unbiased and by exercise 10.4 has variance

\mathop{\mathrm{Var}}\nolimits_{\theta}(W)=\frac{\theta^{2}}{n(n+2)}<\frac{% \theta^{2}}{n}\qquad\text{for every }n\geq 1,

which is of order $1/n^{2}$ instead of $1/n$ . An unbiased estimator with variance smaller than the formal bound in this way is said to be super-efficient.

There is no contradiction: the inequality has hypotheses, and the uniform model violates these hypotheses. The simulation in lecture 9 has already shown the $1/n^{2}$ decay for the mean squared error of $M$ . Exercise 16.3 proves the consistency of $M$ by a direct argument.

Summary.

•

Under the conditions of section 11.1, all unbiased estimators $W$ for $g(\theta)$ with (13.1) satisfy $\mathop{\mathrm{Var}}\nolimits_{\theta}(W)\geq g^{\prime}(\theta)^{2}/\mathcal% {I}_{n}(\theta)$ . For $g(\theta)=\theta$ we get the bound $1/\mathcal{I}_{n}(\theta)$ . The proof uses $\mathop{\mathrm{Cov}}\nolimits_{\theta}\bigl{(}W,\ell^{\prime}(\theta)\bigr{)}% =g^{\prime}(\theta)$ and the Cauchy–Schwarz inequality.
•

The efficiency of an unbiased estimator is the ratio of the bound to the variance, with values in $[0,1]$ . An efficient estimator is a UMVUE amongst the estimators satisfying (13.1), but a UMVUE need not be efficient.
•

The bound is attained if and only if $W-g(\theta)=c(\theta)\ell^{\prime}(\theta)$ with probability one. For a one-parameter exponential family, the average $T_{n}/n$ of the natural statistic is efficient for its mean, and the only functions of $\theta$ which can be used to construct an efficient estimator are affine functions of the mean.
•

The bound depends on regularity conditions. For example, for the uniform distribution on $(0,\theta)$ , the support of the distribution depends on the parameter. The unbiased estimator $(n+1)M/n$ has variance $\theta^{2}/\bigl{(}n(n+2)\bigr{)}$ , which is smaller than the formal value $\theta^{2}/n$ . Thus, the estimator is super-efficient.

Exercise 13.1.

Let $X_{1},\dots,X_{n}$ be i.i.d. exponentially distributed with rate $\theta>0$ . Define $f(x;\theta)=\theta e^{-\theta x}$ for $x\geq 0$ and $T=\sum_{i=1}^{n}X_{i}$ . Use the fact that the chi-squared distribution $\chi^{2}_{k}$ from definition A.2 coincides with the $\text{Gamma}(k/2,1/2)$ distribution from table A.2 and that $T$ is $\text{Gamma}(n,\theta)$ distributed.

1.

Show that $Y_{1}=2\theta X_{1}$ has density $\tfrac{1}{2}e^{-y/2}$ for $y\geq 0$ and consequently $Y_{1}\sim\chi^{2}_{2}$ .
2.

Using definition A.2, show that $2\theta T\sim\chi^{2}_{2n}$ .
3.

Using the gamma density of $T$ and the change of variables formula from proposition A.19, show that you get the same result as in the previous part.
4.

Using the mean and variance of the chi-squared distribution, find $\mathbb{E}_{\theta}(T)$ and $\mathop{\mathrm{Var}}\nolimits_{\theta}(T)$ . Check your values by computing the moments of a single observation.

Exercise 13.2.

Let $X_{1},\dots,X_{n}$ be i.i.d. Bernoulli with success probability $\theta\in(0,1)$ , and let $T_{n}=\sum_{i=1}^{n}X_{i}$ .

1.

Write down the Cramér–Rao bound for $\theta$ , using example 11.5.
2.

Show that $\bar{X}$ is an efficient estimator for $\theta$ .
3.

Verify condition (13.2) for $W=\bar{X}$ directly, by writing the score $\ell^{\prime}(\theta)$ in terms of $\bar{X}-\theta$ , and check that the resulting $c(\theta)$ equals $g^{\prime}(\theta)/\mathcal{I}_{n}(\theta)$ .
4.

Write down the Cramér–Rao bound for the odds $g(\theta)=\theta/(1-\theta)$ , and explain why no efficient estimator for the odds exists.
5.

Show that in fact no unbiased estimator for the odds exists at all.

Exercise 13.3.

Let $X_{1},\dots,X_{n}$ be i.i.d. Poisson with mean $\theta>0$ and let $T_{n}=\sum_{i=1}^{n}X_{i}$ .

1.

Write down the Cramér–Rao bound for $\theta$ , using example 11.6, and show that $\bar{X}$ achieves this bound.
2.

Verify condition (13.2) for $W=\bar{X}$ directly, by writing the score $\ell^{\prime}(\theta)$ in terms of $\bar{X}-\theta$ , and by checking that the resulting $c(\theta)$ equals $g^{\prime}(\theta)/\mathcal{I}_{n}(\theta)$ .
3.

Explain why no efficient estimator for $g(\theta)=e^{-\theta}$ exists, and why this does not contradict the fact that $(1-1/n)^{T_{n}}$ is the UMVUE for $e^{-\theta}$ , using example 10.7.

Exercise 13.4.

Let $X_{1},\dots,X_{n}$ , where $n\geq 3$ , be i.i.d. exponential with rate $\theta>0$ , let $T=\sum_{i=1}^{n}X_{i}$ and let $\tilde{\theta}=(n-1)/T$ be the unbiased version of the MLE from lecture 5.

1.

Write down the Cramér–Rao bound for $\theta$ , using example 11.8.
2.

In exercise 5.3 we found $\mathop{\mathrm{Var}}\nolimits_{\theta}(\tilde{\theta})=\theta^{2}/(n-2)$ . Determine the efficiency of $\tilde{\theta}$ and describe how it behaves as $n\to\infty$ .
3.

Show that no unbiased estimator for $\theta$ can be efficient.
4.

Show that $\tilde{\theta}$ is the UMVUE for $\theta$ .
5.

Show that $\bar{X}$ is an efficient estimator for the mean $\mu=1/\theta$ , and explain how this is compatible with the previous results.

Exercise 13.5.

Let $X_{1},\dots,X_{n}$ be i.i.d. $N(\mu,\sigma^{2})$ . Assume first that $\mu$ is known and $\sigma^{2}>0$ is the parameter.

1.

Show that $\mathcal{I}_{n}(\sigma^{2})=n/(2\sigma^{4})$ and find the Cramér–Rao bound for $\sigma^{2}$ .
2.

Show that $\hat{\sigma}^{2}=\frac{1}{n}\sum_{i=1}^{n}(X_{i}-\mu)^{2}$ is an efficient estimator for $\sigma^{2}$ , using definition A.2 and lemma A.3.
3.

Now assume that $\mu$ is unknown. From lemma 2.3 we know that the sample variance $S^{2}$ is an unbiased estimator for $\sigma^{2}$ . Using proposition A.4, find $\mathop{\mathrm{Var}}\nolimits(S^{2})$ and compare the result with the bound from the first part of the question. Lecture 14 shows that this bound still holds when $\mu$ is unknown. What can you say about $S^{2}$ , in light of example 10.8?

Exercise 13.6.

Let $\psi=g(\theta)$ where $g\colon\Theta\to g(\Theta)$ is a twice continuously differentiable bijection with $g^{\prime}(\theta)\neq 0$ for all $\theta\in\Theta$ . Furthermore, let $\mathcal{I}_{n,\psi}$ and $\mathcal{I}_{n,\theta}$ be the Fisher information of the sample with model parametrised by $\psi$ and by $\theta$ , respectively.

1.

Using propositions 11.4 and 11.9, show that $\mathcal{I}_{n,\psi}(\psi)=\mathcal{I}_{n,\theta}(\theta)/g^{\prime}(\theta)^{2}$ .
2.

Let $W$ be an unbiased estimator for $g(\theta)$ which satisfies the assumptions of theorem 13.2. Show that the bound from corollary 13.3, for $W$ as an unbiased estimator for the parameter $\psi$ , coincides with the bound from theorem 13.2 for $g(\theta)$ .