Let X1, . . . , Xn denote a random sample from a N(μ, σ2) distribution, where the mean μ and the variance σ2 are both unknown so that the param- eter vector is given by θ = (μ, σ2)T (i) Show that the joint p.d.f. of X1, . . . , Xn belongs to the two-parameter regular exponential family, stating what the canonical parameter is equal to in terms of θ. (ii) Use the result in (i) to derive the maximum likelihood (ML) estimate of θ, θˆ = (μˆ, σˆ2)T ,
Question
Let X1, . . . , Xn denote a random sample from a N(μ, σ2) distribution, where the mean μ and the variance σ2 are both unknown so that the param- eter vector is given by θ = (μ, σ2)T (i) Show that the joint p.d.f. of X1, . . . , Xn belongs to the two-parameter regular exponential family, stating what the canonical parameter is equal to in terms of θ. (ii) Use the result in (i) to derive the maximum likelihood (ML) estimate of θ, θˆ = (μˆ, σˆ2)T ,
Solution
(i) The joint probability density function (pdf) of X1, ..., Xn from a normal distribution N(μ, σ^2) is given by:
f(x1, ..., xn; μ, σ^2) = (1/(2πσ^2)^(n/2)) * exp{-∑(xi - μ)^2 / (2σ^2)}
This can be rewritten in the form of a two-parameter regular exponential family as:
f(x1, ..., xn; μ, σ^2) = h(x1, ..., xn) * c(μ, σ^2) * exp{∑[η(μ, σ^2) * T(xi) - A(μ, σ^2)]}
where h(x1, ..., xn) = 1, c(μ, σ^2) = (1/(2πσ^2)^(n/2)), η(μ, σ^2) = [μ/σ^2, -1/(2σ^2)], T(xi) = [xi, xi^2], and A(μ, σ^2) = nμ^2/(2σ^2).
(ii) The maximum likelihood estimates of μ and σ^2 are obtained by taking the derivative of the log-likelihood function with respect to μ and σ^2, setting them equal to zero, and solving for μ and σ^2.
The log-likelihood function is given by:
L(μ, σ^2) = -n/2 * log(2π) - n/2 * log(σ^2) - ∑(xi - μ)^2 / (2σ^2)
Taking the derivative with respect to μ and setting it equal to zero gives:
∂L/∂μ = ∑(xi - μ) / σ^2 = 0
Solving for μ gives the maximum likelihood estimate of μ:
μˆ = ∑xi / n
Taking the derivative with respect to σ^2 and setting it equal to zero gives:
∂L/∂σ^2 = -n / (2σ^2) + ∑(xi - μ)^2 / (2σ^4) = 0
Solving for σ^2 gives the maximum likelihood estimate of σ^2:
σˆ^2 = ∑(xi - μˆ)^2 / n
So, the maximum likelihood estimates of μ and σ^2 are μˆ = ∑xi / n and σˆ^2 = ∑(xi - μˆ)^2 / n, respectively.
Similar Questions
Let X1, . . . , Xn denote a random sample from a N (µ, σ2) distribution,where the mean µ and the variance σ2 are both unknown so that the param-eter vector is given byθ = (µ, σ2)T .(i) Show that the joint p.d.f. of X1, . . . , Xn belongs to the two-parameterregular exponential family, stating what the canonical parameter isequal to in terms of θ.
how that the joint p.d.f. of X1, . . . , Xn belongs to the two-parameterregular exponential family, stating what the canonical parameter isequal to in terms of θ.
Let X1, . . . , Xn denote a random sample from a N (μ, σ2) distribution, where the mean μ and the variance σ2 are both unknown so that the param- eter vector is given by θ = (μ, σ2)T .ive the parametric function g(θ) that defines the quantile of order α of the normal distribution in terms of μ, σ, and the corresponding quantile zα of the standard normal distribution. (viii) Find the ML estimate of g(θ). (x) Derive the bias of g( ˆθ) as an estimator of g(θ) and use it to provide a bias-corrected estimator ̃g of g(θ). (xi) Derive the standard deviation of ̃g and the consequent standard error of ̃g. [NOTE: You may assume the distribution of ˆσ2, but you need to derive the expressions where necessary for the moments of ˆσ and ˆσ2.
1. Suppose X = (X1, . . . , Xn) is a random sample from the Gamma distribution withshape α and rate θ, i.e.,fX (x) = θαΓ(α)xα−1e−xθ, x > 0, α > 0, θ > 0.Each Xi has expectation E(X) = α/θ, variance Var(X) = α/θ2, and momentgenerating function MX (t) = (1 − t/θ)−α.(a) Assuming both α and θ to be unknown, write down the log likelihood functionℓ(θ, α; X) and the corresponding score functions ∂ℓ∂θ and ∂ℓ∂α .(b) Verify that each score function has zero expectation.(c) Assuming α to be known:(i) Calculate the Cramer Rao Lower Bound (CRLB) for the variance of anunbiased estimator of θ.(ii) Is there any unbiased estimator of θ whose variance attain the CRLB?(iii) Show thatS = α − 1nnXi=1 1Xiis an unbiased estimator for θ. What is the MVU estimator for θ?(iv) Identify a change of parameter η = η(θ) for which there exists an unbiasedestimator with variance attained the CRLB.(v) For the parameter in the previous part, identify the MVU estimator whosevariance attains the CRLB. Compute this variance
the moment estimator of θ;• the MLE of θ
Upgrade your grade with Knowee
Get personalized homework help. Review tough concepts in more detail, or go deeper into your topic by exploring other relevant questions.