On estimation with weighted balanced type loss function

Mohammad Jafari Jozani,^{a} Éric Marchand,^{b,}^{1} Ahmad Parsian,^{c,}^{2}

a Department of Statistics, Faculty of Mathematical Sciences, Shahid Beheshti University, Tehran, IRAN (e-mail: mjafari2002@yahoo.com)

b University of New Brunswick and Université de Sherbrooke, Département de mathématiques, Sherbrooke Qc, CANADA, J1K 2R1 (e-mail: eric.marchand@usherbrooke.ca)

c School of Mathematical Sciences, Isfahan University of Technology, Isfahan, 84156, IRAN (e-mail: ahmad_{−}p@cc.iut.ac.ir)

ABSTRACT

For estimating an unknown parameter θ, we introduce and motivate the use of the balanced type loss function: L_{w,δ_{0}}(θ, δ) = ω q(θ)(δ − δ_{0})^{2} + (1 − ω)q(θ)(δ − θ)^{2}, where 0 ≤ ω ≤ 1, q(θ) is a positive weight function, and δ_{0} is a general “target” estimator. Developments and various examples are given with regards to the issues of admissibility, dominance, Bayesianity, and minimaxity. In many cases, as in Dey, Ghosh, and Strawderman (1999), we show that results for loss L_{w,δ_{0}} may be inferred directly from corresponding results for weighted squared error loss (i.e, ω = 0). Specific issues related to constrained parameter spaces, which include the choice of the target estimator, are addressed. Finally, we derive minimax estimators of a bounded normal mean θ under loss L_{w,δ_{0}} with δ_{0} being the maximum likelihood estimator of θ.

Keywords: Balanced loss function; Admissibility; Bayes estimator; Minimax estimation; Constrained parameter spaces

MSC: 62C10; 62C15; 62C20; 62F10; 62F30

1research supported by NSERC of Canada

2research supported by a grant of the Research Council of the Isfahan University of Technology

1 Introduction

This paper deals with the estimation of an unknown parameter θ under the model X = (X_{1}, …, X_{n}) ∼ F_{θ}, and with loss

L_{w,δ_{0}}(θ, δ) = ω q(θ)(δ − δ_{0})^{2} + (1 − ω)q(θ)(δ − θ)^{2};  (1)

where 0 ≤ ω ≤ 1, q(θ) is a positive weight function, and δ_{0} is a “target” estimator. This loss, which depends on the observed value of δ_{0}(X), reflects a desire of closeness of δ both to: (i) θ in terms of weighted squared error loss, and (ii) the target estimator δ_{0} in terms of weighted squared distance. The ωq(θ) (δ − δ_{0})^{2} part of the loss is analogous to a penalty term for lack of smoothness in non-parametric regression. The weight ω in (1) calibrates the relative importance of these two criteria. The choice of loss (1) is inspired by the Zellner’s (1994) balanced loss function, which has been studied by several others and which corresponds to the case q(θ) = 1, and a least squares δ_{0}. Indeed, for the particular case where δ_{0}(X) = X̄, loss (1) becomes equivalent to loss:

\frac{ω}{n} ∑_{i=1}^{n}(X_{i} − δ)^{2} + (1 − ω)(δ − θ)^{2};

highlighting a goodness of fit term (i.e, \frac{ω}{n} ∑_{i=1}^{n}(X_{i} − δ)^{2}) with the least squares estimates. However, it is useful to view δ_{0} as a more general target estimator than least squares as a unifying scheme for establishing decision theoretic properties such as admissibility, Bayesianity, and minimaxity. Moreover, we study the application of (1) in the context of restricted parameter spaces Θ, where it will be desirable for the target estimator to take values solely in Θ.

As shown by Dey, Ghosh and Strawderman (1999), the issues of admissibility and dominance, as well as Bayesianity for BLF essentially does not depend on ω, and can be inferred by previously or to be established results under squared error loss (i.e, ω = 0). As one may anticipate, similar connections can be drawn for losses in (1) for more general (q(θ), δ_{0}). Section 2 deals with such connections, and expands on various illustrations. Moreover, we also develop new connections with regards to minimaxity in Section 3, and present various applications, which include the problem of estimating a bounded normal mean θ under loss L_{ω,δ_{0}}, with q ≡ 1 and δ_{0} the maximum likelihood estimator.

2 Admissibility and Dominance

For an estimator δ(X), we will denote the risk under loss (1) as R_{w,δ_{0}}(θ, δ(X)). We assume throughout risk finiteness of R_{w,δ_{0}}(θ, δ(X)). For the case ω = 0, we will simply write R_{0}(θ, δ(X)) as the risk becomes independent of δ_{0}. For similar reasons, we will write L_{0,δ_{0}} as L_{0}, and the difference in losses below Δ_{0,δ_{0}} as Δ_{0}. As in Dey, Ghosh, and Strawderman (1999), the connections between losses L_{ω,δ_{0}}; 0 ≤ ω < 1; are driven by Lemma 1’s decompositions of the risk R_{w,δ_{0}}(θ, δ(X)) and of the difference in losses:

Δ_{ω,δ_{0}}(θ, δ_{1}, δ_{2}) = L_{ω,δ_{0}}(θ, δ_{1}) − L_{ω,δ_{0}}(θ, δ_{2}).

Lemma 1 We have

(a) Δ_{ω,δ_{0}}(θ, δ_{0} + (1 − ω)g, δ_{0}) = (1 − ω)^{2}Δ_{0}(θ, δ_{0} + g, δ_{0});

(b) Δ_{ω,δ_{0}}(θ, δ_{0} + (1 − ω)g_{1}, δ_{0} + (1 − ω)g_{2}) = (1 − ω)^{2}Δ_{0}(θ, δ_{0} + g_{1}, δ_{0} + g_{2});

(c) R_{ω,δ_{0}}(θ, δ_{0}(X) + (1 − ω)g(X)) = ω(1 − ω)R_{0}(θ, δ_{0}(X)) + (1 − ω)^{2}R_{0}(θ, δ_{0}(X) + g(X)) .

Proof. In order to establish part (c), first take expectations on both sides of the equality in part (a) to obtain

R_{ω,δ_{0}}(θ, δ_{0}(X)+(1−ω)g(X)) = R_{ω,δ_{0}}(θ, δ_{0}(X))+(1−ω)^{2}{R_{0}(θ, δ_{0}(X)+g(X))−R_{0}(θ, δ_{0}(X))} .

Now, part (c) follows given that R_{ω,δ_{0}}(θ, δ_{0}(X)) = (1 − ω)R_{0}(θ, δ_{0}(X)). Part (b) follows directly from (a) as Δ_{ω,δ_{0}}(θ, δ_{0} + (1 − ω)g_{1}, δ_{0} + (1 − ω)g_{2}) = Δ_{ω,δ_{0}}(θ, δ_{0} + (1 − ω)g_{1}, δ_{0}) − Δ_{ω,δ_{0}}(θ, δ_{0} + (1 − ω)g_{2}, δ_{0}). Finally, to establish part (a), a straightforward expansion leads to

L_{ω,δ_{0}}(θ, δ_{0} + (1 − ω)g) = q(θ){g^{2}(1 − ω)^{2} + 2g(1 − ω)^{2}(δ_{0} − θ) + (1 − ω)(δ_{0} − θ)^{2}}
= q(θ)(1 − ω)^{2}{g^{2} + 2g(δ_{0} − θ)} + L_{ω,δ_{0}}(θ, δ_{0}),

which implies Δ_{ω,δ_{0}}(θ, δ_{0} + (1 − ω)g, δ_{0}) = q(θ)(1 − ω)^{2}{g^{2} + 2g(δ_{0} − θ)}, and the desired result.

Now, the following is immediate.

Corollary 1 (a) The estimator δ_{0}(X) + (1 − ω)g(X) is admissible under loss L_{ω,δ_{0}} if and only if the estimator δ_{0}(X) + g(X) is admissible under loss L_{0};

(b) δ_{0}(X) + (1 − ω)g_{1}(X) dominates δ_{0}(X) + (1 − ω)g_{2}(X) under loss L_{ω,δ_{0}} if and only if δ_{0}(X) + g_{1}(X) dominates δ_{0}(X) + g_{2}(X) under loss L_{0}.

To complement the above with regards to admissibility and dominance, similar connections between Bayes estimators for varying ω are easily derived and given in the next lemma. In particular, the Bayes (or Generalized Bayes) estimator under loss L_{ω,δ_{0}} may be expressed simply as a convex linear combination of the Bayes (or Generalized Bayes) estimator under loss L_{0} (weight 1 − ω) and δ_{0} (weight ω).

Lemma 2 Under loss L_{ω,δ_{0}} (as in (1)), the Bayes estimator of θ under prior π admits representation:

δ_{π}(X) = ωδ_{0}(X) + (1 − ω)\frac{E_{π}(θq(θ)|X)}{E_{π}(q(θ)|X)};

or, equivalently,

δ_{π}(X) = δ_{0}(X) + (1 − ω)\{\frac{E_{π}(θq(θ)|X)}{E_{π}(q(θ)|X)} − δ_{0}(X)\};  (2)

as long as E_{π}(θ^{i}q(θ)|X) < ∞; i = 0, 1; with probability one.

Proof. The proof, which is straightforward and similar to the one given by Dey, Ghosh and Strawderman (1999) for q(·) = 1, is omitted. (It even follows from Dey, Ghosh and Strawderman’s result for q(·) = 1 given that Bayesian estimators for (q, π) are necessarily equivalent to Bayesian estimators for (q ≡ 1, π′ ≡ q × π).)

Results concerning dominance, admissibility, and Bayesianity of a scalar parameter under squared error or weighted squared losses are, of course, plentiful in the literature. For instance, many such results may be found in the textbook of Lehmann and Casella (1998), as well in various papers dating back at least 50 years. Corollary 1 and Lemma 2 imply that, necessarily, such results have easily stated analogs under loss (1), and for arbitrary δ_{0}. The reverse is also true. Hence, problems that address the issues of dominance, admissibility, and/or Bayesianity, with respect to a loss function L_{ω,δ_{0}} can be handled, and perhaps have already been handled, by an analysis for loss L_{0}.

A lengthy enumeration of examples is not pertinent here. Rather, we illustrate the above connections with a few immediate examples, and we emphasize with later examples in Section 3 other related issues such as the minimax criterion and the choice of the target estimator in restricted parameter spaces.

Example 1 (Affine linear estimators) A sampling of familiar results concerning the performance of affine linear estimators under the assumptions E_{θ}(X) = θ, Var_{θ}(X) = σ^{2}(θ) = σ^{2} (e.g., Lehmann and Casella, 1998; Gupta, 1966) indicate that, for estimating θ under squared error loss:

(i) cX + d is inadmissible whenever c > 1, c < 0, or c = 1, d ≠ 0;

(ii) cX + d is, for X ∼ N(θ, 1), admissible iff c = 1, d = 0 or c ∈ [0, 1), d ∈ ℜ;

(iii) cX + d is, for X ∼ Gamma(α, θ/α), admissible iff c = α/(α + 1), d = 0 or c ∈ (0, α/(α + 1)], d > 0.

The inadmissibility results in (i) translates, for an arbitrary target estimator δ_{0}, to the inadmissibility under loss L_{ω,δ_{0}} of estimators of the form δ_{0}(X) + (1 − ω)(cX + d − δ_{0}(X)) = ωδ_{0}(X)+(1−ω)(cX +d) for c > 1, c < 0, or c = 1, d ≠ 0. In contrast, given (ii), estimators of the form ωδ_{0}(X) + (1 − ω)(cX + d) are, under loss L_{ω,δ_{0}} with X ∼ N(θ, 1), admissible iff c = 1, d = 0; or c ∈ [0, 1), d ∈ ℜ. In particular, if δ_{0}(X) = X, we obtain the inadmissibility of c′X + d′ (under loss L_{ω,δ_{0}} with δ_{0}(X) = X) for c′ > 1, c′ < ω, or c′ = ω, d′ ≠ 0; and the admissibility for X ∼ N(θ, 1) of c′X + d′ iff c′ = 1, d′ = 0; or c′ ∈ [ω, 1), d′ ∈ ℜ. Similar inferences can be drawn from (iii). Namely for X ∼ Gamma(α, θ/α), c′X + d′ is under loss L_{ω,δ_{0}}, with δ_{0}(X) = X, admissible for estimating θ iff c′ = (α + ω)/(α + 1), d′ = 0 or c′ ∈ (ω, (α + ω)/(α + 1)], d > 0.

Example 2 (Minimum risk equivariant estimators) Consider the location model X ∼ f_{0}(x− θ); with E_{0}(X^{2}) < ∞; and take L_{ω,δ_{0}} as in (1) with q ≡ 1. It is well known that, under loss L_{0}, location invariant estimators of the form X + c have constant risk, and that an optimal choice (MRE) is given by δ^{∗}(X) = X +c^{∗} (where c^{∗} = −E_{0}(X)). Furthermore, the estimator δ^{∗}(X) is, under loss L_{0}, minimax, admissible, and Bayes with respect to the right-invariant Haar prior π^{∗}(θ) = 1. The results of this section tell us that, under loss L_{ω,δ_{0}}, the estimator δ^{∗∗}(X) = ωδ_{0}(X) + (1 − ω)δ^{∗}(X) is admissible, and Bayes with respect to the right-invariant Haar prior π^{∗}(θ) = 1. In particular, if the target estimator δ_{0} is location invariant of the form X + c_{0}, then δ^{∗∗}(X) (given by X + ωc_{0} + (1 − ω)c^{∗}; and equal, of course, to δ^{∗∗}(X) if c_{0} = c^{∗}) also: (i) has constant risk (given part (c) of Lemma 1); (ii) is optimal among location-invariant estimators X +c; (iii) is, by virtue of Theorem 1 below, minimax. Finally, we point out that similar developments for observables X_{1}, …, X_{n} and for scale parameter models follow along the same lines (also see Example 6).

3 Minimaxity

Here, we capitalize on the results of Lemma 1 to present general results and various examples pertaining to minimax estimators under loss L_{ω,δ_{0}}. The first part of the section deals with cases where the target estimator δ_{0} has constant risk under loss L_{0}, while the second part addresses more general cases and concludes with an application concerning the estimation of a bounded normal mean.

Theorem 1 Suppose δ_{0}(X) has constant risk γ under loss L_{0}; then δ_{0}(X) + (1 − ω)g(X) is minimax under loss L_{ω,δ_{0}}, with minimax risk (1−ω)(ωγ+(1−ω)r), if and only if δ_{0}(X)+g(X) is minimax under loss L_{0}, with minimax risk r.

Proof. Since R_{0}(θ, δ_{0}(X)) = γ by assumption, we obtain from part (c) of Lemma 1

R_{ω,δ_{0}}(θ, δ_{0}(X) + (1 − ω)g(X)) = ω(1 − ω)γ + (1 − ω)^{2}R_{0}(θ, δ_{0}(X) + g(X)) ;  (3)

telling us that

sup_{θ}{R_{ω,δ_{0}}(θ, δ_{0}(X) + (1 − ω)g(X))} = ω(1 − ω)γ + (1 − ω)^{2} sup_{θ}{R_{0}(θ, δ_{0}(X) + g(X))} ,

and yielding the result.

Corollary 2 Suppose the estimator δ_{0}(X) has constant risk γ under loss L_{0}. Then δ_{0}(X) is minimax under loss L_{ω,δ_{0}} with constant (and minimax) risk (1 − ω)γ if and if only δ_{0}(X) is minimax under loss L_{0} with constant (and minimax) risk γ.

Proof. Apply Theorem 1 with g(X) = 0.

We pursue with a selection of applications of Theorem 1 and Corollary 2.

Example 3 (Poisson) Let X ∼ Poisson(θ); θ ∈ (0, ∞); with L_{0}(θ, δ) = \frac{(δ−θ)^{2}}{θ} (i.e., informationnormalized loss). Since δ_{0}(X) = X is minimax under loss L_{0} with constant (and minimax) risk 1, Corollary 2 tells us that δ_{0}(X) = X is also minimax under any loss L_{ω,δ_{0}}; (ω ∈ (0, 1], weight q(θ) = \frac{1}{θ}); with constant (and minimax) risk 1 − ω.

Example 4 (Binomial) Let X ∼ Bi(n, θ); θ ∈ (0, 1); with L_{0}(θ, δ) = \frac{(δ−θ)^{2}}{θ(1−θ)} (i.e., informationnormalized loss). Since δ_{0}(X) = \frac{X}{n} is minimax under loss L_{0} with constant (and minimax) risk \frac{1}{n}, Corollary 2 tells us that δ_{0}(X) = \frac{X}{n} is also minimax under any loss L_{ω,δ_{0}}; (ω ∈ (0, 1], weight q(θ) = \frac{1}{θ(1−θ)}); with constant (and minimax) risk \frac{1−ω}{n}. In contrast, if L_{0}(θ, δ) = (δ − θ)^{2}, Corollary 2 applies for the constant risk and minimax under L_{0} estimator δ_{0}(X) = \frac{X+(\sqrt{n}/2)}{n+\sqrt{n}}, and implies the minimaxity of this δ_{0}(X) for all losses L_{ω,δ_{0}}; ω ∈ (0, 1]. Finally, if L_{0}(θ, δ) = (δ − θ)^{2} but δ_{0}(X) = \frac{X}{n}, Theorem 1 does not apply, and one can proceed directly in finding a constant risk proper Bayes (and hence) minimax estimator (see Sanjari Farsipour and Asgharzadeh, 2003).

In the next example, the target estimator δ_{0} is not minimax, but we can still make use of Theorem 1.

Example 5 (Negative Binomial) Consider X ∼ NBi(α, p); α > 0 known, p ∈ (0, 1) unknown, with θ = E(X) = α(\frac{1}{p}−1); Var_{θ}(X) = αθ(1+θ); and P_{θ}(X = x) = \frac{Γ(α+x)}{x!Γ(α)}(\frac{α}{α+θ})^{α}(\frac{θ}{α+θ})^{x}I_{\{0,1,...\}}(x).

Now, consider L_{0}(θ, δ) = \frac{(δ−θ)^{2}}{Var_{θ}(X)} for which the maximum likelihood estimator δ_{0}(X) = X has

constant risk equal to γ = 1. Given that, under loss L_{0}, \frac{X}{1+α} (= X − \frac{α}{1+α}X) is minimax, with minimax risk r = \frac{α}{1+α}, (e.g., Ferguson, 1967), Theorem 1 tells us that X − (1 − ω)\frac{α}{1+α}X = (\frac{1+ωα}{1+α})X is minimax under loss L_{ω,δ_{0}}; with minimax risk (1 − ω)((ω × 1) + (1 − ω)(\frac{α}{1+α})) = \frac{(1−ω)(α+ω)}{1+α}. Moreover, since \frac{X}{1+α} is admissible and dominates δ_{0}(X) = X under loss L_{0}, it follows from Corollary 1 that (\frac{1+ωα}{1+α})X is admissible and dominates δ_{0}(X) = X for any loss L_{ω,δ}; ω ∈ (0, 1). Finally, we point out that other admissibility and Bayesianity results under loss L_{0}, such as those given by Chou (1995), can be translated via Corollary 1 and Lemma 2 to admissibility and Bayesianity results for loss L_{ω,δ_{0}}.

In the next example, there exists many minimax estimators.

Example 6 (Lower bounded scale parameter) For the model X ∼ Gamma(α, θ); (density proportional to x^{α−1}e^{−x/θ}); consider estimating a lower bounded θ, θ ≥ a > 0 (a known), with loss L_{ω,δ_{0}} as in (1) with q(θ) = θ^{−2}. Set h_{α}(z) = \frac{z^{α+1}e^{−z}}{∫_{0}^{z} t^{α+1}e^{−t}dt}. Van Eeden (1995) showed that,

under loss L_{0}, the Bayes estimator δ_{a}^{∗}(X) with respect to the prior π_{a}^{∗}(θ) = θ^{−1}1_{(a,∞)}(θ), given by δ_{a}^{∗}(x) = \frac{x}{α+1}(1+h_{α+2}(\frac{x}{a})), is an admissible and minimax estimator of θ, with minimax risk r = \frac{1}{α+1} (also see Jafari Jozani, Nematollahi, and Shafie 2002). For general target estimator δ_{0}, the results of Section 2 tell us that the estimator δ_{a}^{∗∗}(X) = ωδ_{0}(X) + (1 − ω)δ_{a}^{∗}(X) is an admissible Generalized Bayes estimator with respect to π_{a}^{∗}. If the target estimator δ_{0} is chosen to be the MRE estimator \frac{X}{α+1}, then it follows from Theorem 1 that δ_{a}^{∗∗}(X); given now by \frac{X}{α+1} + (1 − ω)δ_{a}^{∗}(X) = \frac{X}{α+1}(1 + (1 − ω)h_{α+2}(\frac{X}{a})); is not only admissible, but also minimax, with minimax risk \frac{1−ω}{α+1} (since γ = R_{0}(θ, \frac{X}{α+1}) = \frac{1}{α+1})). Observe as well that there exists many minimax estimators under loss L_{0}, and hence many under loss L_{ω,δ_{0}}, when δ_{0} is the MRE estimator. Similar developments for lower bounded scale or location parameters under losses L_{0}, which can be applied to losses L_{ω,δ_{0}}, are given by Marchand and Strawderman (2005a, 2005b).

Remark 1 With respect to the previous example, if the target estimator δ_{0} takes values outside the parameter space [a, ∞), there is no guarantee for instance that P_{θ}(δ_{a}^{∗∗}(X) ∈ [a, ∞)) = 1. In fact, if δ_{0}(X) = \frac{X}{α+1}, then

lim_{x→0} δ_{a}^{∗∗}(x) = (1 − ω) lim_{x→0} δ_{a}^{∗}(x) = (1 − ω)\frac{α + 2}{α + 1}a,

since lim_{x→0} δ_{a}^{∗}(x) = \frac{α+2}{α+1}a (van Eeden, 1995). This tells us that δ_{a}^{∗∗}(x) would take values within the parameter space if and only if ω ≤ \frac{1}{α+2}. This phenomenon is not restricted to this example, and occurs since the loss function L_{ω,δ_{0}} reflects the closeness of δ to the target δ_{0}, rather than the criterion δ ∈ Θ. It was our motivation to study, present examples, and have results available for general target estimators δ_{0}, which are not necessarily least-squares, and which take values solely in the parameter space. For instance, the general admissibility and Bayesian properties of δ_{a}^{∗∗}(X) in Example 6 hold for the truncated version of the MRE estimator δ_{0}(X) = max(a, \frac{X}{α+1}); or the maximum likelihood estimator δ_{0}(X) = max(a, X/α).

Notwithstanding the various minimax estimation applications of Theorem 1 and Corollary 2 given in Examples 2 to 6, they are limited to cases where the target estimator δ_{0}(X) has constant risk. In contrast, the following development relates to situations where δ_{0}(X) does not have constant risk, but rather possesses risk properties permitting us to exploit the following well-known criterion for minimaxity (e.g., Lehmann and Casella, 1998, section 5.1).

Lemma 3 If δ_{π} is a Bayes estimator with respect to a proper prior π, and S_{π} = {θ ∈ Θ : sup_{θ}{R(θ, δ_{π}); θ ∈ Θ} = R(θ, δ_{π})}, then δ_{π} is minimax whenever P_{π}(θ ∈ S_{π}) = 1.

To pursue, we define for loss L_{ω,δ_{0}} and for an estimator δ(X):

S_{ω,δ_{0}}(δ) = {θ ∈ Θ : sup_{θ}{R_{ω,δ_{0}}(θ, δ(X)); θ ∈ Θ} = R_{ω,δ_{0}}(θ, δ(X))};

(i.e., the set of θ’s such that δ(X) attains its maximum risk under loss L_{ω,δ_{0}}); and write S_{0}(δ) for S_{0,δ_{0}}(δ).

Lemma 4 Suppose δ_{π}(X) = δ_{0}(X) + g_{π}(X) is unique Bayes under loss L_{0} for proper prior π. Suppose further that P_{π}(θ ∈ S_{0}(δ_{π})) = P_{π}(θ ∈ S_{0}(δ_{0})) = 1, then the estimator δ_{π}(X) = δ_{0}(X) + (1 − ω)g_{π}(X) is unique minimax under loss L_{ω,δ}.^{3}

Proof. Given that under loss L_{ω,δ_{0}}, δ′_{π}(X) = δ_{0}(X) + (1 − ω)g_{π}(X) is unique Bayes with respect to π by virtue of (2), it suffices to show that

P_{π}(θ ∈ S_{ω,δ_{0}}(δ′_{π})) = 1,  (4)

since we could then apply Lemma 3 to infer the desired result. But (4) follows by Lemma 1 part (c)’s representation of the risk R_{ω,δ_{0}}(θ, δ′_{π}(X)) given that, by assumption, both R_{0}(θ, δ_{0}(X)) and R_{0}(θ, δ_{π}(X)) attain their supremum on the same probability one set.

3we note that these assumptions imply that δ_{0}(X) + g_{π}(X) is minimax under loss L_{0}

We now give an application of Lemma 4 to the normal distribution case X ∼ N(θ, σ^{2}), where θ is bounded to an interval [−m, m], and where the target estimator δ_{0} is chosen as the maximum likelihood estimator δmle(X) = (m ∧ X)sgn(X). Incidentally, Marchand and Perron (2001) provide various dominating estimators (δ_{0}(X)+g(X)) of δmle(X) under squared error loss L_{0}, which translate to various dominating estimators (δ_{0}(X) + (1 − ω)g(X)) of δmle(X) under loss L_{ω,δ_{0}} with δ_{0} = δmle. An example is given by the minimax estimator of Theorem 2 when m ≤ σ. Returning to the minimax estimation of θ, we capitalize on Casella and Strawderman’s (1981) work giving conditions for which the Bayes estimator, δ_{BU}(X), with respect to the (boundary) uniform prior π_{BU} on {−m, m} is minimax.

Theorem 2 For estimating θ ∈ [−m, m] with loss L_{ω,δ_{0}}, as in (1) with q(θ) = 1 and δ_{0} = δmle, under the model: X ∼ N(θ, σ^{2}) (with pdf \frac{1}{σ}ϕ(\frac{x−θ}{σ}); cdf Φ(\frac{x−θ}{σ})) with known σ, the estimator (1 − ω)δ_{BU}(X) + ωδmle(X) is unique minimax as long as m ≤ m_{0}σ, with m_{0} ≈ 1.0567 and δ_{BU}(X) = \frac{m}{σ} tanh(mX/σ^{2}).

Proof. We use Lemma 4 above, and Lemma 5 which is stated and proven in the Appendix. From Casella and Strawderman (1981), we have P_{π}(θ ∈ S_{0}(δ_{π})) = 1, for π = π_{BU} and m ≤ m_{0}σ. Furthermore, we show in Lemma 5 below that the maximum value of R(θ, δmle(X)) is necessarily attained at θ = ±m whenever m ≤ \sqrt{2}σ. Since \sqrt{2} ≥ m_{0}, in other words P_{π}(θ ∈ S_{0}(δ_{0})) = 1 for π = π_{BU} and m ≤ m_{0}σ, the result follows directly as a consequence of Lemma 4.

References

Casella, G., Strawderman, W.E. (1981). Estimating a bounded normal mean. Annals of Statistics, 9, 870-878.

Chou, J.P. (1995). Admissibility of conjugate Bayes estimators for the mean of a negative binomial distribution. Statistics & Decisions, 13, 301-306.

Dey, D., Ghosh, M., Strawderman, W.E. (1999). On estimation with balanced loss functions. Statistics & Probability Letters, 45, 97-101.

Ferguson, T. S. (1967). Mathematical statistics: A decision theoretic approach. Academic Press, New York-London.

Gupta, M.K. (1966). On the admissibility of linear estimates for estimating the mean of distributions of the one parameter exponential family, Calcutta Statistical Association Bulletin, 15, 14-19.

Jafari Jozani, M., Nematollahi, N., Shafie, K. (2002). An admissible minimax estimator of a bounded scale-parameter in a subclass of the exponential family under scale-invariant squared error loss. Statistics & Probability Letters, 60, 437-444.

Lehmann, E. L., Casella, G. (1998). Theory of Point Estimation. Springer-Verlag, New York, 2nd edition.

Marchand, Éric, Perron, François (2001). Improving on the MLE of a Bounded Normal Mean. Annals of Statistics, 29, 1078-1093.

Marchand, É., Strawderman, W. E. (2005a). On Improving on the Minimum Risk Equivariant Estimator of a Location Parameter which is Constrained to an Interval or a Half-Interval. Annals of the Institute of Statistical Mathematics, 57, 129-143.

Marchand, É., Strawderman, W. E. (2005b). On Improving on the Minimum Risk Equivariant Estimator of a Scale Parameter under a Lower-Bound Constraint. Journal of Statistical Planning and Inference, 134, 90-101.

Sanjari Farsipour, N. and Asgharzadeh, A. (2004). Estimation of the parameter of a Bernouilli distribution using a balanced loss function. Personal communication.

van Eeden, C. (1995) Minimax estimation of a lower-bounded scale parameter of a gamma distribution for scale-invariant squared error loss. Canadian Journal of Statistics, 23, 245-256.

Zellner, A. (1994). Bayesian and Non-Bayesian estimation using balanced loss functions. Statistical Decision Theory and Methods V, (J.O. Berger and S.S. Gupta Eds). NewYork: Springer-Verlag, 337-390.

4 Appendix

Lemma 5 For X ∼ N(θ, σ^{2}) with |θ| ≤ m, the maximum risk under squared error loss of the estimator δmle(X) is attained on the boundary \{−m, m\}, whenever m ≤ \sqrt{2}σ.

Proof. Without loss of generality, set σ = 1. We first show that, for all m ∈ (0, \sqrt{2}),

(A)  sup_{θ∈[−m,m]}{ R_{0}(θ, δmle(X)) } = max{ R_{0}(0, δmle(X)), R_{0}(m, δmle(X)) }.

Secondly, we show

(B) R_{0}(0, δmle(X)) ≤ R_{0}(m, δmle(X)), iff m ≤ m_{1} ≈ 1.96422.

Clearly, (A) and (B) (which are both of independent interest), will then suffice to establish the lemma. To establish (A), we proceed directly by showing that, for m ≤ \sqrt{2},

\frac{∂}{∂^{3}θ}R_{0}(θ, δmle(X)) ≥ 0,

for θ ∈ [0, m]. Indeed, observe that either: (i) the convexity of R_{0}(θ, δmle(X)) for θ ∈ [0, m]; or (ii) a change from concavity to convexity of R_{0}(θ, δmle(X)) on [0, m] both imply (A). Note that the sufficiency of (ii) exploits the fact that the risk of δmle is an even function of θ, which implies that its derivative at θ = 0^{+} is 0; and that the risk will be decreasing in θ for positive θ in a neighbourhood of 0.

Expanding directly the risk E_{θ}[(δmle(X) − θ)^{2}], we obtain

R_{0}(θ, δmle(X)) = ∫_{−m}^{m}(x − θ)^{2}ϕ(x − θ)dx + (m − θ)^{2}Φ(θ − m) + (m + θ)^{2}Φ(−(θ + m)). (5)

Direct computations now yield

\frac{1}{2} \frac{∂}{∂θ}R_{0}(θ, δmle(X)) = (m + θ)Φ(−(θ + m)) + (θ − m)Φ(θ − m) ;
\frac{1}{2} \frac{∂}{∂^{2}θ}R_{0}(θ, δmle(X)) = Φ(−θ − m) + Φ(θ − m) − (m + θ)ϕ(m + θ) + (θ − m)ϕ(θ − m) ;
\frac{1}{2} \frac{∂}{∂^{3}θ}R_{0}(θ, δmle(X)) = ϕ(θ − m)(2 − (m − θ)^{2})) − ϕ(θ + m)(2 − (m + θ)^{2})) .

Finally, result (A) follows as ϕ(θ − m) ≥ ϕ(θ + m) ≥ 0; 2 − (m − θ)^{2} ≥ 2 − (m + θ)^{2}; and 2 − (m − θ)^{2} ≥ 0; for θ ∈ [0, m] with m ≤ \sqrt{2}.

To establish (B), use (5) to write

R_{0}(0, δmle(X)) = 2m^{2}Φ(−m) + 2 ∫_{0}^{m} x^{2}ϕ(x)dx;
and R_{0}(m, δmle(X)) = 4m^{2}Φ(−2m) + ∫_{0}^{2m} x^{2}ϕ(x)dx.

From this, work with the difference in risks D(m) = R_{0}(m, δmle(X)) − R_{0}(0, δmle(X)) to obtain

\frac{∂}{∂m}D(m) = 4m\{2Φ(−2m) − mϕ(2m) − Φ(−m)\} = 4mN(m) (say);

and \frac{∂}{∂m}N(m) = ϕ(2m) \{e^{3m^{2}} + 4m^{2} − 5\}.

Now, observe that N(0) = \frac{1}{2} > 0, lim_{m→∞} N(m) = 0, and that \frac{∂}{∂m}N(m) changes signs from - to + on (0, ∞). These conditions imply that N(m) changes signs once from + to - for m ∈ (0, ∞), which in turn tells us that D(m) has at most one signs change from + to - for m ∈ (0, ∞) (as D(0) = 0). Finally, a numerical evaluation yields D(m) = 0 for m = m_{1} ≈ 1.96422.