Suppose you have data x coming from a N (µ, σ2) distribution where the variance σ2 isknown but you want to perform Bayesian inference on the mean µ. We are going toconsider two different scenarios of prior knowledge:• The first scenario involves using a normal prior i.e. π(µ) = N (ν, w) and lettingw → ∞ to represent lack of knowledge.• In the second scenario, you know that there is probability p that µ = 0, but there islittle prior knowledge about µ̸ = 0. This prior information will be represented by amixture distribution, with a discrete probability at µ = 0 (i.e. a probability atom)and π(µ) = N (0, w) for µ̸ = 0 and let w → ∞ to represent our lack of knowledge forµ̸ = 0.(a) [6 marks] Analytically calculate the posterior probability π(µ|x) for the first sce-nario. Comment on whether the prior and posterior are proper or improper.(b) [5 marks] Analytically calculate the posterior probability π(µ = 0|x) for the sec-ond scenario. Give an explanation in words (1-2 sentences) of what this posteriorindicates and whether this is sensible.(c) [3 marks] Give an explanation as to what properties of the prior in (b) give rise tothe behaviour of the posterior in (b), compared to the prior in (a).(d) [2 marks] Suggest an alternative prior in the spirit of (b) and show that it gives amore sensible posterior π(µ = 0|x) than the one calculated in (b)

(a) The posterior distribution π(µ|x) is calculated using Bayes' theorem, which states that the posterior is proportional to the likelihood times the prior. In this case, the likelihood is the normal distribution N(x|µ, σ2) and the prior is N(µ|ν, w). When we multiply these two together and normalize, we get another normal distribution. This is because the normal distribution is a conjugate prior for the normal likelihood. The mean of the posterior distribution will be a weighted average of the sample mean and the prior mean, and the variance will be less than the minimum of the sample variance and the prior variance. If w → ∞, the prior becomes improper (not a valid probability distribution because it does not integrate to 1), but the posterior is still proper because it is a weighted average of proper distributions.

(b) The posterior probability π(µ = 0|x) is calculated in a similar way, but now the prior is a mixture of a point mass at µ = 0 and a normal distribution N(0, w). The posterior will be a weighted average of these two components, with weights determined by the likelihood of the data under each component. If the data are very unlikely under the point mass (i.e., if the data are far from 0), then the posterior will be dominated by the normal component. This is sensible because it reflects the fact that the data are providing strong evidence against µ = 0.

(c) The prior in (b) gives more weight to µ = 0 than the prior in (a), so the posterior in (b) will also give more weight to µ = 0, all else being equal. This is because Bayes' theorem updates our beliefs in a way that is proportional to the prior probabilities.

(d) An alternative prior could be a mixture of a point mass at µ = 0 and a wider normal distribution, say N(0, 2w). This would give less weight to extreme values of µ, and therefore lead to a posterior that is less influenced by outliers in the data. The posterior π(µ = 0|x) would then be a weighted average of the likelihood under the point mass and the likelihood under the wider normal distribution, which would be more sensible than the posterior calculated in (b).