Processing math: 100%

Example Problems on Estimators

ST702 Homework 5 on Estimators

7.45

Let X1,X2,Xn be iid from a distribution with mean μ and variance σ2, and let S2 be the usual unbiased estimator of σ2. In Example 7.3.4 we saw that, under normality, the MLE has smaller MSE than S2. In this exercise we will explore variance estimators some more.

a

Show that, for any estimator of the form aS2, where a is some constant,

MSE(aS2)=E[aS2σ2]2=a2Var(S2)+(a1)2σ4 MSE(aS2)=E[Wθ]2=E[aS2σ2]2=Var(aS2)(Bias(aS2))2=a2Var(S2)(E(aS2)σ2)2=a2Var(S2)(a2E(S2)22aE(S2)σ2+σ4)=a2Var(S2)(a2(σ2)22aσ2σ2+σ4)unbiased=a2Var(S2)+(a1)2σ4

b

Show that

Var(S2)=1n(κn3n1)σ4

where κ=E[Xμ]4/σ4 is the kurtosis.

V(S2)=V(1n1(Xi¯X)2)=1(n1)2V((Xi¯X)2)=1(n1)2[E(((Xi¯X)2)2)E((Xi¯X)2)2]=1(n1)2[E((Xi¯X)4)E((Xi¯X)2)2]=1(n1)2κσ4(σ2)2=1(n1)2(nκσ4nσ4)

c

Show that under normality, the kurtosis is 3 and establish that, in this case, the estimator of the form aS2 with the minimum MSE is n1n+1S2. (Lemma 3.6.5 may be helpful)

We can bring the σ4 inside the fraction to get

κ=E[(Xμσ)4].

Notice that the inside function now follows a N(0,1) distribution. We can also make use of Lemma 3.6.5 which says

E[g(x)(xμ)]=σ2E(g(x)).

Now we can take g(Z)=Z3. Then,

κ=E[Z4]=E[g(Z)(Z0)]=σ2E[g(Z)]=σ2E[3Z2]=3σ2E[Z2]=3σ2(V(Z)+E(Z)2)=31(1+0)=3.

Now we can use our MSE equation from (a) and our variance equation from (b). We are trying to optimize with respect to a.

MSE=a21n(κn3n1)σ4+(a1)2σ4MSEa=2a1n(κn3n1)σ4+2(a1)σ4=0a=σ41n(κn3n1)σ4+σ4=σ41n(3n3n1)σ4+σ4=11n(3n3n1)+1=n1n+1

d

If normality is not assumed, show that MSE(aS2) is minimized at

a=n1n+1+(κ3)(n1)n

which is useless as is depends on a parameter.

From above

a=σ41n(κn3n1)σ4+σ4=11n(κn3n1)+1=n1n+1+(κ3)(n1)n

e

Show that

i

for distributions with κ>3, the optimal a will satisfy a<n1n+1.

Taking κ>3 will make the second term in our denominator positive, which will make the denominator larger than n+1. So our overall expression will be less than n1n+1.

ii

for distributions with κ<3, the optimal a will satisfy n1n+1<a<1.

Similarly, making κ<3 will make the second term in our denominator negative, which will make the denominator smaller than n+1. So our overall expression will be greater than n1n+1.

7.48

Suppose that Xi, i=1,,n are iid Bernoulli(p).

a

Show that the variance of the MLE of p attains the Cramér-Rao Lower Bound.

The MLE is,

L(p|x)=f(xi|p)=pxi(1p)1xi=pxi(1p)1xi(p|x)=xilog(p)+(nxi)log(1p)p=xi1p+(nxi)11p0=xi1p+(nxi)11pxi1p=(nxi)11p1pp=nxixiˆp=¯X.

The Cramér-Rao lower bound is,

Vθ(W(X))(θEθ(W(X)))2Eθ(2θ2log(f(x|θ)))Eθ(2θ2log(f(x|θ)))=Eθ(θxip(nxi)11p)=Eθ(xip2+(nxi)(1p)2)=1p2E(xi)+E(n=xi(1p)2)=np2p+nnp(1p)2=np+n(1p)(1p)2CRLB=1np+n(1p)(1p)2=p(1p)n.

Now we can find the variance of our estimator.

V(ˆp)=V(xin)=1n2V(xi)=1n2np(1p)=p(1p)n=CRLB

b

For n4, show that the product X1X2X3X4 is an unbiased estimator of p4. and use this fact to find the best unbiased estimator of p4.

E[X1X2X3X4]=E(X1)E(X2)E(X3)E(X4)independence=p4.

Also ni=1xi is sufficient for p by the Factorization Theorem.

f(X|p)=pxi(1p)1xi=pxi(1p)1xi

We can use the Rao-Blackwell theorem to find a UMVUE.

ϕ(T)=E(X1X2X3X4|ni=1Xi=t)=P(X1,X2,X3,X4=1ni=1Xi=t)P(ni=1Xi=t)=P(X1,X2,X3,X4=1ni=5Xi=t4)P(ni=1Xi=t)=P(X1,X2,X3,X4=1)P(ni=5Xi=t4)P(ni=1Xi=t)independence=p4(n4t4)pt4(1p)n4(t4)(nt)pt(1p)nt=(n4t4)(nt)

7.49

Let X1,,Xn be iid exponential(λ).

a

Find an unbiased estimator of λ based only on Y=minX1,,Xn.

FY=P(X<Xi)=[1F(X)]nfy=nf(x)(1F(X))n1=n1λe1λx(1(1e1λx))n1=n1λen1λx.

So YExp(λ/b). So nY is unbiased.

E(nY)=nE(Y)=nλ/n=λ

b

Find a better estimator than the one in part (a). Prove that it is better.

Since it’s an exponential family, xi is sufficient and complete. Notice that ¯X is a function of xi and is unbiased.

E(¯X)=1nE(xi)=1nnλ=λ

By Lehmann-Scheffe, ¯X is UMVUE, so it is better than the estimator in (a).

c

The following data are high-stress failure times (in hours) of Kevlar/epoxy spherical vessels used in a sustained pressure environment on the space shuttle:

50.1,70.1,137.0,166.9,170.5,152.8,80.5,123.5,112.6,148.5,160.0,125.4.

Failure times are often modeled with the exponential distribution. Estimate the mean failure time using the estimators from parts (a) and (b).

From (a) ˆλ=50.112=601.2. For (b) we can take the average ˆλ=124.825.

4

Consider a random sample X1,X2,Xn from the density f(x|θ)=exp[(xθ)], θ<x<, and 0 elsewhere. Show that the first order statistic X(1)=minXi is a complete sufficient statistic for θ, and find the UMVUE of θ.

f(X|θ)=e(xθ)I[θ<x<]=exinθI[θ<X(1)]=exienθI[θ<X(1)]

By the factorization theorem X(1) is sufficient. Now we can show that it is complete. First, we need its distribution.

fX(1)=nfx(1FX)n1=ne(xθ)[1xθe(tθ)dt]n1=ne(xθ)[1(1ex+θ)]n1=ne(xθ)[ex+θ]n1=nen(x+θ)

Now we can show completeness.

θg(x)nen(θt)dx=nenθθg(x)entdx

This is only 0 if g(x) is 0. Now we can look at the expectation of the minimum order statistic.

E(X(1))=θxne(x+θ)ndx=n(1+nθ)n2=1n+θ

So X(1)1n in unbiased.