Bayesian improvements of a MRE estimator of a bounded location parameter

Éric Marchand^{a}, Amir T. Payandeh Najafabadi^{b} a Département de mathématiques, Université de Sherbrooke, Sherbrooke, CANADA (eric.marchand@usherbrooke.ca) b Amir T. Payandeh Najafabadi, Department of Mathematical Sciences, Shahid Beheshti University, G.C. Evin, 1983963113, Tehran, IRAN (amirtpayandeh@sbu.ac.ir)

Abstract We study the frequentist risk performance of Bayesian estimators of a bounded location parameter, and focus on conditions placed on the shape of the prior density guaranteeing dominance over the minimum risk equivariant (MRE) estimator. For a large class of even and logconcave densities, even convex loss functions, we demonstrate in a unified manner that symmetric priors which are bowled shaped and logconcave lead to Bayesian dominating estimators. The results generalize similar results obtained by Marchand and Strawderman for the fully uniform prior, as well as those obtained by Kubokawa for squared error loss. Finally, we present a detailed example and several remarks.

AMS 2010 subject classifications. 62F10, 62F30, 62C20. Keywords and phrases: Bayes estimator; bounded mean; dominance; location family; logconcavity; minimum risk equivariant; restricted parameter space.

1. The Problem

Consider the restricted parameter space estimation problem:

X ∼ f_{0}(x − θ), loss is ρ(d − θ), θ ∈ [a, b],  (1)

where f_{0} is a positive Lebesgue density and ρ is convex. Without the restriction θ ∈ [a, b], a benchmark estimator is the minimum risk equivariant estimator δmre, which is also minimax and Bayes with respect to the flat (or noninformative) prior on (−∞, ∞). In view of the compact interval restriction, the frequentist risk performance of Bayesian alternatives δ_{π} with respect to prior densities π supported on [a, b], or a subset of [a, b], is of interest. Characterizing Bayesian estimators δ_{π}, or the prior densities π themselves, that guarantee that δ_{π} dominate δmre is of particular interest. In this regard, Marchand and Strawderman (2005A) (see also Kubokawa, 2005) showed that indeed that the fully uniform prior Bayes estimator δ_{U} dominates δmre quite generally with respect to f_{0} and convex ρ, (or again logconcave f_{0} and strict bowl-shaped ρ). The result is achieved, in an unified way with respect to f_{0} and ρ and by using Kubokawa’s (2004) IERD method, via general conditions for an estimator δ to dominate δmre and showing then that δ_{U} satisfies these conditions. By making use of such conditions, Kubokawa (2005A, Proposition 3.1) provides quite elegant, simple and useful conditions on f_{0} and π for δ_{π} to dominate δmre for squared error ρ, reproduced here in a slightly weaker version.

Lemma 1. (Kubokawa, 2005A) For problem (1) with f_{0} even, unimodal and logconcave^{1}, and squared error ρ, δ_{π} dominates δmre whenever the density π is symmetric about \frac{a+b}{2}, logconcave on (a, b), and nondecreasing on (\frac{a+b}{2}, b).

We see indeed that the conditions for dominance are qualitatively appealing. Moreover, we argue that they capture the essential features of priors which lead to dominance. Indeed, these prior densities are bowled shaped in contrast to unimodal priors π which may lead to large frequentist risk R(θ, δ_{π}) for θ on or near the boundary {a, b}. They also do not rise too sharply from the center, as controlled by the logconcavity condition, as otherwise, such as the case of too sharply increasing densities moving away from the center, the frequentist risk may be too large in the center \frac{a+b}{2} of the parameter space. However, while Marchand and Strawderman’s dominance conditions and proof of the superiority of δ_{U} on δmre applies for a large class of losses ρ, Lemma 1 is limited to squared error ρ and Kubokawa’s proof does indeed exploit analytical properties of δ_{π} specific to squared error loss. The main motivation and finding below thus consists of a generalization (Theorem 1) of Lemma 1 to a larger class of losses, where convex ρ and logconcave ρ′ (the latter is a weak condition) are required. Moreover, the proof is unified with respect to (f_{0}, ρ), and we believe that it even simplifies some of the intricate analysis of Kubokawa’s proof for the squared error case. It also brings into focus precise analytical properties (Lemmas 4 and 5) of Bayes estimators in restricted parameter spaces which will be of use in similar problems. An illustration of our results (Example 1) completes the presentation. In contrast to the problem of dominating the (more plausible) truncation of δmre, where conditions applicable to δ_{π} (Marchand and Perron, 2001, 2005, 2009; Marchand et al., 2008) are limited to not too large parameter spaces (i.e., b − a not too large), the results of Lemma 1 and those below apply regardless of the given parameter space [a, b]. There has been a fair amount of work on decision theoretic approaches to restricted parameter space problems (see for instance van Eeden, 2004; Marchand and Strawderman, 2004), with many remaining challenges such as those solved with Lemma 1 and its generalization below. As an example, Hartigan (2004) studies a multivariate version of problem (1) with X ∼ N_{p}(θ, I_{p}), loss ∥d − θ∥^{2}, θ ∈ C where C is a convex set with a non-empty interior, and shows that the fully uniform Bayes estimator δ_{U} (although here the prior can be improper) dominates quite generally X regardless of C and p. His results apply to problem (1) but have not been extended to other priors, or to other models or losses for p > 1.

1It is true that the symmetry and unimodality would suffice as these imply unimodality.

2. Main results

For problem (1), a minimum risk equivariant estimator δmre exists and is, under mild conditons which we assume, uniquely given by X + c_{0}, where c_{0} minimizes the constant risk R(θ, X + c) = E_{0}[ρ(X + c)] in c. Unless specified otherwise, we take throughout in (1) a = −m, b = m without loss of generality, we assume f_{0} to be even, logconcave, and we further assume that ρ is absolutely continuous, symmetric about 0, and strict bowled-shaped such that ρ ≥ 0, ρ(0) = 0, ρ′(u) < 0 for u < 0 and ρ′(u) > 0 for u > 0. We point out that the logconcave assumption on f_{0} equates to a strict monotone likelihood ratio (mlr) property for the family of densities of X. Under the above assumptions, the MRE estimator is simply X. Marchand and Strawderman’s conditions for an estimator X + h(X) to dominate δmre specialize as follows for Bayesian estimators δ_{π} associated with symmetric densities π. The conditions are rather simple, qualitatively appealing, and bring into play the benchmark fully uniform Bayes estimator δ_{U}. Lemma 2. For problem (1) with f_{0} even, with ρ symmetric about 0, logconcave and strict bowledshaped, the Bayes estimator δ_{π}(x) = x + h_{π}(x) with respect to a symmetric about 0 prior density π, dominates δmre(X) = X whenever

(i) h_{π}(x) decreases in x, for x > 0; (ii) and |δ_{π}| ≥ |δ_{U}|.

Proof. From Marchand and Strawderman’s (2005A, Theorem 5.1(ii) and Remark 5.1) sufficient conditions for dominance, we require for X + h_{π}(X) to dominate δmre(X): (a) h_{π}(·) to be nonincreasing on ℜ, (b) there exists x_{0} such that δ_{π}(x_{0}) = δ_{U}(x_{0}) = δmre(x_{0}), and (c) and |h| ≤ |h_{U}|; where x + h_{U}(x) is the fully uniform Bayes estimator. Now, under the given symmetry assumptions on f_{0}, ρ, and π, δ_{π}(·) will be an odd function (i.e., equivariant with respect to a sign change) and hence δ_{π}(x) = −δ_{π}(−x), or equivalently h_{π}(x) = −h_{π}(−x). Thus, (a) is equivalent to (i), (b) is satisfied with x_{0} = 0, and (c) is equivalent to (ii), thus establishing the result. Synthesizing the previous lemma, our task is simple to describe: select Bayesian estimators which shrink (towards 0) on the δmre and increasingly as |x| increases (i.e., |x − δ_{π}(x)| increases in |x|), but at the same time expand (away from 0) on the fully uniform Bayes estimator δ_{U}. The difficulty, of course, is relating these features to the prior π. We pursue with a useful lemma.

Lemma 3. Suppose g_{1} and g_{0} are two distinct, positive densities on (a, b) with respect to a σ− finite measure μ, such that \frac{g_{1}}{g_{0}} increases on (a, b). Consider a function k(·) that changes signs exactly once from − to + on (a, b), in the sense that there exists y_{0} ∈ (a, b) such that k(y) < 0 for y < y_{0}, and k(y) > 0 for y > y_{0}. Then, we have E_{g_{0}}[k(Y)] = 0 ⇒ E_{g_{1}}[k(Y)] > 0, and E_{g_{1}}[k(Y)] = 0 ⇒ E_{g_{0}}[k(Y)] < 0.

Proof. The result is known. Here is a short proof for completeness. We have

E_{g_{0}}[k(Y)] = ∫_{a}^{y_{0}} k(y)\frac{g_{0}(y)}{g_{1}(y)} g_{1}(y) dμ(y) + ∫_{y_{0}}^{b} k(y)\frac{g_{0}(y)}{g_{1}(y)} g_{1}(y) dμ(y)
< \frac{g_{0}(y_{0})}{g_{1}(y_{0})} ∫_{a}^{b} k(y) g_{1}(y)dμ(y) = \frac{g_{0}(y_{0})}{g_{1}(y_{0})}E_{g_{1}}[k(Y)],

and the result follows directly. Here below is a key lemma which addresses condition (i) of Lemma 2, showing that priors with logconcave densities on (−m, m) lead to Bayes estimates δ_{π}(x) such that δ_{π}(x) − x decreases in x. The result, which applies for location models and invariant losses with strict mlr densities and strict bowl-shaped losses, or positive densities and convex losses; generalizes Marchand and Strawderman’s (2005A, Lemma 5.1) uniform prior result, as well as Kubokawa’s (2005A, Proposition 3.1.) squared error loss result. We do not assume for this lemma that f_{0} is symmetric, nor that ρ or π is symmetric so that the phenomenon is actually much more general than required here. Lemma 4. For problem (1) with b= -a=m, logconcave f_{0}, and strict bowled-shaped ρ, the Bayes estimator δ_{π}(x) = x + h_{π}(x) possesses the property that h_{π}(x) = δ_{π}(x) − x decreases in x, for all x ∈ ℜ, as long as the prior density π is logconcave on (−m, m).

Proof. The Bayes estimate δ_{π}(x) minimizes the expected posterior loss E(ρ(d − θ)|x) in d, and hence satisfies the equation ∫_{(−m,m)} ρ′(δ_{π}(x) − θ) f_{0}(x − θ) π(θ)dθ = 0, for all x, or equivalently

E_{x}[ρ′(h_{π}(x) + c_{0} + U)] = 0; x ∈ ℜ;  (2)

where U =^{d} X −θ|x has density f_{U,x}(u) proportional to f_{0}(u) π(−u+x) 1_{(x−m,x+m)}(u). Now observe that, for x_{1} > x_{0}, the ratio \frac{f_{U,x_{1}}(u)}{f_{U,x_{0}}(u)} ∝ \frac{π(−u+x_{1}) 1_{(x_{1}−m,x_{1}+m)}(u)}{π(−u+x_{0}) 1_{(x_{0}−m,x_{0}+m)}(u)} is increasing in u given the logconcavity of π. Hence, the family of densities f_{U,x} possesses an increasing monotone likelihood ratio (in U) with parameter x. Now, take any x_{1} > x_{0} and suppose in order to arrive at a contradiction that a′ = h_{π}(x_{1}) − h_{π}(x_{0}) > 0. Then, on one hand, with equation (2) and the mlr property of the densities f_{U,x}, Lemma 3 would tell us that

0 = E_{x_{1}}[ρ′(h_{π}(x_{1}) + c_{0} + U)] > E_{x_{0}}[ρ′(h_{π}(x_{1}) + c_{0} + U)] .

On the other hand, observe that the family of densities of T = h_{π}(x_{0}) + c_{0} + a + U, with a > 0 possesses an increasing mlr property as well with parameter a, so that a further application of (2) and Lemma 3 would tell us that

E_{x_{0}}[ρ′(h_{π}(x_{1}) + c_{0} + U)] = E_{a′}[ρ′(T)] > E_{a=0}[ρ′(T)] = E_{x_{0}}[ρ′(h_{π}(x_{0}) + c_{0} + U)] = 0 ,

and would thus lead to a contradiction. Hence, we must have h_{π}(x_{1}) ≤ h_{π}(x_{0}). Remark 1. The result also holds for positive densities and convex losses as the key property of a decreasing monotone likelihood ratio of the posterior densities of U = θ − x, with parameter x, follows from the logconcavity of the prior, and the sign change argument of Lemma 3 is not required when ρ′ is increasing. In both this case, and the situation in the lemma, the Bayes estimators δ_{π} are (essentially) unique (see Marchand and Strawderman, 2005A, page 133, for a similar situation).

There remains to address condition (ii) of Lemma 2. We prove in what follows a more general result ordering the absolute value of Bayes estimates δ_{π_{1}} and δ_{π_{0}} in cases where the ratio of densities \frac{π_{1}(θ)}{π_{0}(θ)} is monotone in |θ|. This quite plausible property, which we will exploit for δ_{π_{0}} ≡ δ_{U}, is very useful as it involves simple conditions for which δ_{π_{1}} expands, or shrinks, on δ_{π_{0}}. A squared error loss version of the following lemma was given by Marchand et al. (2008), while a multivariate normal and squared error loss version was previously established by Marchand and Perron (2001), so that the novel feature below lies with the departure from squared error loss. We assume below that ρ is convex, even, and that ρ′ is logconcave on (0, ∞). The latter assumption is weak, includes L^{p} losses |d − θ|^{p}, with p ≥ 1, Linex loss ρ_{α}(t) = e^{αt} − αt − 1 and the symmetrized version ρ_{α}(t) + ρ_{α}(−t) = e^{αt} + e^{−αt} − 2 , but discounts very sharp penalizing losses such as e^{|d−θ|^{p}} with p > 1.

Lemma 5. Consider problem (1) with b=-a=m, f_{0} logconcave and even, ρ convex and even, and such that ρ′ is logconcave on (0, ∞). Suppose that π_{0} and π_{1} are symmetric about 0 prior densities with respect to a σ-finite measure μ such that \frac{π_{1}(θ)}{π_{0}(θ)} increases in θ ∈ [0, m]. Then, we have |δ_{π_{1}}(x)| ≥ |δ_{π_{0}}(x)| for all x ∈ ℜ.

Proof. As in Lemma 2, δ_{π_{1}}(·) and δ_{π_{0}}(·) will be odd given the assumptions on (f_{0}, ρ, π_{1}, π_{0}), so that we only need consider x > 0. Now, for an even density π, the Bayes estimator δ_{π} satisfies, for all x > 0:

∫_{[−m,m]} f_{0}(x − θ) π(θ) ρ′(δ_{π}(x) − θ) dμ(θ) = 0,

⟺ ∫_{[0,m]} k_{π}(x, λ) π(λ) dμ(λ) = 0 ,  (3)

where k_{π}(x, λ) = ρ′(δ_{π}(x) − λ) f_{0}(x − λ) + ρ′(δ_{π}(x) + λ) f_{0}(x + λ), x > 0, λ ∈ [0, m]. Turning to the sign of k_{π}(x, ·) for a fixed x > 0, observe that k_{π}(x, ·) must change sign at least once on [0, m] given (3), and that k_{π}(x, ·) is positive on [0, δ_{π}(x)] given the properties of ρ. For λ > δ_{π}(x), we write with f_{0} even and ρ′ odd:

k_{π}(x, λ) = \{ \frac{ρ′(λ + δ_{π}(x))}{ρ′(λ − δ_{π}(x))} − \frac{f_{0}(λ − x)}{f_{0}(λ + x)} \}f_{0}(λ + x) ρ′(λ − δ_{π}(x)) .

We infer from the above that k_{π}(x, ·) changes signs once from + to − on (δ_{π}(x), m] since: (i) \frac{ρ′(λ+δ_{π}(x))}{ρ′(λ−δ_{π}(x))} decreases in λ given the logconcavity of ρ′, (ii) \frac{f_{0}(λ−x)}{f_{0}(λ+x)} increases in λ given the logconcavity of f_{0}, and (iii) ρ′(λ − δ_{π}(x)) > 0 for λ > δ_{π}(x). Now, we fix x > 0 and we assume in order to arrive at a contradiction that δ_{π_{1}}(x) < δ_{π_{0}}(x) which would imply k_{π_{1}}(x, λ) < k_{π_{0}}(x, λ) given that ρ′ is increasing (i.e., ρ convex). We apply Lemma 3 to −k_{π}(x, λ) to infer that under this assumption, we would have

∫_{[0,m]} k_{π_{0}}(x, λ)π_{0}(λ)dμ(λ) > ∫_{[0,m]} k_{π_{0}}(x, λ)π_{1}(λ)dμ(λ) > ∫_{[0,m]} k_{π_{1}}(x, λ)π_{1}(λ)dμ(λ) , which is not possible given (3) applied to π_{0} and π_{1}, and concludes the proof. Remark 2. We point out that Lemma 5 holds for non-symmetric π_{0} or π_{1} by replacing the condition of the monotone increasing ratio \frac{π_{1}(·)}{π_{0}(·)} by the monotone increasing ratio \frac{π_{1}^{∗}(·)}{π_{0}^{∗}(·)} (on [0, m]), where π^{∗} is the density of λ = |θ|.

Having addressed conditions (i) and (ii) of Lemma 2, our main result, which generalizes Lemma 1, follows immediately from Lemma 2, Lemma 4 and Lemma 5. Theorem 1. For problem (1) with f_{0} even and logconcave, with ρ convex and even, and such that ρ′ is logconcave on (0, ∞), we have that δ_{π} dominates δmre whenever the density π is symmetric about \frac{a+b}{2}, logconcave on (a, b), and nondecreasing on (\frac{a+b}{2}, b).

We pursue with an illustration. Example 1. We study applications of our findings for a normal model with X ∼ N(θ, 1) and θ ∈ [−m, m], m > 0. Consider prior densities π_{a}(θ) ∝ e^{a|θ|}1_{(−m,m)}(θ) with a ≥ 0. Such choices satisfy the conditions of Theorem 1 (nondecreasing on (0, m), even, logconcave) and include the uniform on (−m, m) case for a = 0. Noting ϕ and Φ the pdf and cdf respectively of a standard normal distribution, we obtain that the posterior density is given by

π_{a}(θ|x) = \frac{1}{k}\{ϕ(θ − (x − a)) 1_{(−m,0)}(θ) + ϕ(θ − (x + a)) 1_{(0,m)}(θ)\} ,

with k = Φ(a−x)−Φ(−m+a−x)+Φ(m−x−a)−Φ(−x−a) . Interestingly, this posterior density is seen to as a superimposition of two truncated normals on (−m, 0) and (0, m) with means x − a and x + a respectively. When a = 0, these means coincide and we simply obtain a truncated N(x, 1) on (−m, m). Now, Theorem 1 tells us that the associated Bayes estimators δ_{π_{a}} dominates necessarily δmre for all a ≥ 0 and for losses ρ that satisfy the given conditions. Such losses include the interesting case of absolute value loss. In this case, the posterior median is computable directly from the above posterior density yielding the dominating estimator δ_{π_{a}}(x) = x−a+Φ^{−1}(\frac{k}{2}+Φ(−m+a−x)) , for x ≤ 0, and δ_{π_{a}}(x) = −δ_{π}(−x) for x > 0. Recapitulating the dominance findings of this paper in an historical context which are applicable to the normal case and to the π_{a}’s above, we begin by pointing out that the dominance result for the specific case a = 0 and ρ(t) = t^{2} is due to Gatsonis, MacGibbon and Strawderman (1987). Extensions for a = 0 to other losses ρ are due to Marchand and Strawderman (2005A), while extensions to a > 0 with ρ(t) = t^{2} follow from Kubokawa (2005A) (i.e., Lemma 1). The main dominance findings of this paper unifies and generalizes the above for a > 0 and for losses ρ satisfying the conditions of Theorem 1. Finally, we do reemphasize the greater generality of Kubokawa’s and our findings to other (than normal) symmetric and logconcave densities (and to other priors); as well as Marchand and Strawderman’s results for the uniform prior with respect to many other asymmetric pairs (f_{0}, ρ).

3. Concluding remarks

We have demonstrated that essential features of the prior density π on [−m, m], namely symmetry, bowl-shapedness and logconcavity are persistent conditions for a Bayes estimator δ_{π} to dominate δmre in problem (1), for a wide class of models f_{0} and losses ρ. Moreover, the approach is unified and extends or complements previous results of Kubokawa, and Marchand and Strawderman. The main application for a finite i.i.d. sample undoubtedly arises for a normal model f_{0} giving sufficiency (but see Kubokawa, 2005A or Marchand and Strawderman, 2005A for some developments in this regard). Implications for some scale parameter problems X_{1} ∼ \frac{1}{σ}f_{1}(\frac{x_{1}}{σ}) with σ restricted to a interval are available as corollaries with the transformation X_{1} → log(X_{1}) to a location problem (see Marchand and Strawderman, 2005B), but are not pursued here. Given the symmetric condition on the location density, the original scale parameter problem will require the equidistributional property \frac{X_{1}}{σ} ∼ (\frac{X_{1}}{σ})^{−1} with common examples of such distributions given by lognormal, half-Cauchy and Fisher with equal degrees of freedom on numerator and denominator. Potential findings for other unsolved problems may well benefit by the methods and results of this paper. These include: (i) asymmetric versions where either f_{0} or ρ is not symmetric; and (ii) extensions to location-scale problems with a bounded coefficient of variation (e.g., Kubokawa, 2005B). Finally, with Hartigan’s result applying for the model X ∼ N_{p}(θ, I_{p}) with a constraint to a ball, such as ∥θ∥ ≤ m, and guaranteeing that the fully uniform Bayes estimator dominates X under loss ∥d − θ∥^{2}, extensions to other spherically symmetric priors, or to other losses, or to other spherically symmetric models, such as those obtained here for p = 1, seem plausible and quite interesting to pursue.

Acknowledgments

The research support of NSERC of CANADA for Marchand is gratefully acknowledged. This research occurred during visits of Amir Payandeh at the mathematics department of the Université de Sherbrooke, and we gratefully acknowledge its hospitality and financial support.

References

Gatsonis, C., MacGibbon, B., & Strawderman, W.E. (1987). On the estimation of a restricted normal mean. Statistics & Probability Letters, 6, 21-30.

Hartigan, J.A. (2004). Uniform priors on convex sets improve risk. Statistics & Probability Letters, 67, 285-288.

Kubokawa, T. (1994). A unified approach to improving on equivariant stimators. Annals of Statistics, 22, 290-299.

Kubokawa, T. (2005A). Estimation of bounded location and scale parameters. Journal of the Japanese Statistical Society, 35, 221-249.

Kubokawa, T. (2005B). Estimation of a mean of a normal distribution with a bounded coefficient of variation, Sankhyā: The Indian Journal of Statistics, 67, 499-525.

Marchand, É. & Perron, F. (2001). Improving on the MLE of a bounded normal mean. Annals of Statistics, 29, 1078-1093.

Marchand, É. & Perron, F. (2005). Improving on the MLE of a bounded mean for spherical distributions. Journal of Multivariate Analysis, 92, 227-238.

Marchand, É. & Perron, F. (2009). Estimating a bounded parameter for symmetric distributions. Annals of the Institute of Mathematical Statistics, 61, 215-234.

Marchand, É., Ouassou, I., Payandeh, A.T. & Perron, F. (2008). On the estimation of a restricted location parameter for symmetric distributions. Journal of the Japanese Statistical Society, 38, 293-309.

Marchand, É. & Strawderman, W. E. (2004). Estimation in restricted parameter spaces: A review. Festschrift for Herman Rubin, IMS Lecture Notes-Monograph Series, 45, pp. 21-44.

Marchand, É. & Strawderman, W. E. (2005A). Improving on the minimum risk equivariant estimator of a location parameter which is constrained to an interval or a half-interval. Annals of the Institute of Mathematical Statistics, IMS Lecture Notes-Monograph Series, 57, 129-143.

Marchand, É. & Strawderman, W. E. (2005B). On improving on the minimum risk equivariant estimator of a scale parameter under a lower bound constraint, Journal of Statistical Planning and Inference. 134, 90-101.

van Eeden, C. (2006). Restricted parameter space problems. Admissibility and minimaxity properties. Lecture Notes in Statistics, 188, Springer.