On Bayesian credible sets in restricted parameter space problems and lower bounds for frequentist coverage ^{1}

Éric Marchand^{a}, William E. Strawderman^{b}

a Université de Sherbrooke, Département de mathématiques, Sherbrooke Qc, CANADA, J1K 2R1

(e-mail: eric.marchand@usherbrooke.ca)

b Rutgers University, Department of Statistics and Biostatistics, 501 Hill Center, Busch Campus, Piscataway, N.J., USA, 08855 (e-mail: straw@stat.rutgers.edu)

Summary

For estimating a lower bounded parametric function in the framework of Marchand and Strawderman (2006), we provide through a unified approach a class of Bayesian confidence intervals with credibility 1 − α and frequentist coverage probability bounded below by \frac{1−α}{1+α}. In cases where the underlying pivotal distribution is symmetric, the findings represent extensions with respect to the specification of the credible set achieved through the choice of a spending function, and include Marchand and Strawderman’s HPD procedure result. For non-symmetric cases, the determination of a such a class of Bayesian credible sets fills a gap in the literature and includes an “equal-tails” modification of the HPD procedure. Several examples are presented demonstrating wide applicability.

AMS 2000 subject classifications: 62C10, 62F15, 62F25, 62F30 Key words and phrases: Bayesian methods, Credible sets, Frequentist coverage probability, Lower bound, Restricted Parameter, Spending function

1December 17, 2012

1. Introduction

Bayesian credible sets are not designed (e.g., Robert, 2011) and are far from guaranteed (Fraser, 2011) to have satisfactory, exact or precise frequentist coverage but it is nevertheless of interest to investigate (Wasserman, 2011) to what extent there is convergence or divergence in various situations. A historically resonating example where there is exact convergence arises for estimating the mean of a N(μ, σ^{2}) distribution, and where the use of the non-informative prior leads to a (1−α)×100% HPD credible set (i.e. the z or t confidence interval) with exact frequentist coverage. This, however, is very much the exception. Even, in the simple presence of a lower bound on the mean parameter μ (e.g., Mandelkern, 2002), with the prior taken to be the truncation of the noninformative prior onto the restricted parameter space, the frequentist coverage of the (1−α)×100% HPD credible set fluctuates from its credibility (or nominal coverage) 1 − α. However, the HPD procedure does not fare poorly as a frequentist procedure for large 1 − α as witnessed by the lower bound \frac{1−α}{1+α} on its frequentist coverage due to Roe and Woodroofe (2000, known σ^{2}) and Zhang and Woodroofe (2003, unknown σ^{2}), as well as the better lower bound 1 − \frac{3α}{2} (for α < 1/3, known σ^{2}) obtained by Marchand et al. (2008). In a generalization of the above, Marchand and Strawderman (MS 2006) introduced a unified framework for which the (1 − α) × 100% HPD credible set of a lower bounded parametric function has frequentist coverage greater than \frac{1−α}{1+α} for all values lying in the restricted parameter space. This framework, as well as its various applications, will be revisited in Sections 2 and 5, but let us consider for sake of illustration the basic examples: (i) X ∼ f_{0}(x − θ) with known f_{0}, θ ≥ 0; and (ii) X ∼ Gamma(α, θ) with θ ≥ 1 with known α. For location family densities as in (i) with f_{0} unimodal and symmetric, Marchand and Strawderman’s results apply for the flat prior on the truncated parameter space [0, ∞) and the corresponding (1 − α) × 100% HPD credible set, with the guarantee that the actual frequentist coverage is bounded below by \frac{1−α}{1+α} for all θ ≥ 0. However, if f_{0} is not symmetric, such a result does not hold in general (MS 2006, Example 1). The same is true for a vast number of so-called non-symmetric situations arising in Marchand and Strawderman’s framework, including the Gamma models in (ii) where the prior is given by \frac{1}{θ}𝕀_{[1,∞)}(θ), that is the truncation on [1, ∞) of the usual non-informative prior \frac{1}{θ}𝕀_{(0,∞)}(θ). It is true that the bound holds for certain specific classes of f_{0}’s (MS 2006, Theorem 2, a), and it is also the case that numerical evaluations of a theoretical and unexplicit lower bound for frequentist coverage provides further evidence for satisfactory coverage for a specific Gamma model in (ii) (MS 2006, Example 2). Nevertheless, a clear analytical result or lower bound for frequentist coverage in such non-symmetric cases is lacking, and it our motivation here to try to fill this gap. For a large variety of situations with a lower bounded parametric constraint, we obtain here a class of Bayesian (1 − α) × 100% credible sets which provide minimal frequentist probability coverage exceeding \frac{1−α}{1+α}. These Bayesian confidence intervals include an “equal-tails” modification, or approximation, of the HPD credible set, which also coincides with the latter in situations of underlying symmetry. Our findings are achieved by introducing and exploiting a spending function interpretation of Bayesian confidence intervals, and lead to a class of procedurees (rather than a single one) which share the above lower bound for frequentist coverage. The rest of the paper is organized as follows. Preliminary results, definitions and model assumptions, including those related to the spending function associated with a Bayesian credible interval, are presented in Section 2, while Bayesian credible interval representations are outlined in Section 3. The main findings concerning frequentist coverage appear in Section 4 and various applications are presented and commented on in Section 5.

2. Definitions and preliminary results

2.1. Assumptions, invariance, pivot, prior, and implications

As in basic examples (i) and (ii), we consider model densities f(x; θ); x ∈ X, θ ∈ Θ ⊂ ℝ^{p}; for an observable X, and we are concerned with interval estimation of a parametric function τ(θ) (ℝ^{p} → ℝ) with the additional constraint τ(θ) ≥ 0. We assume there exists a pivot of the form T (X, θ) = \frac{a_{1}(X)−τ(θ)}{a_{2}(X)}; a_{2}(·) > 0; such that −T (X, θ) has cdf G and Lebesgue density g_{0}. This pivot assumption means that the frequentist or conditional distribution of T (X, θ), or −T (X, θ), given θ does not depend on θ, θ ∈ ℝ^{p}. We can thus set G as the common cdf of −T (X, θ). In the basic location-family example (i) with X ∼ f_{0}(x − θ)(= g_{0}(θ − x), say), the above is illustrated by the fact that −T (X, θ) = θ − X is a pivot with cdf G and pdf g_{0}. In the Gamma example, or more generally scale families with X ∼ \frac{1}{θ} f_{1}(\frac{x}{θ}), θ ≥ 1, a corresponding T (X, θ) pivot is obtained with a_{1}(X) = log(X), a_{2}(X) = 1, τ(θ) = log(θ). We further assume that the unrestricted decision problem is invariant under a group G of transformations and that the pivot satisfies the invariance requirement T (x, θ) = T (gx, ḡθ), for all x ∈ X, θ ∈ Θ, g ∈ G, ḡ ∈ Ḡ, with X, Θ, G, and Ḡ being isomorphic. For instance, in basic example (i), the invariance is achieved with the additive group G on ℝ^{p} and since T (x, θ) = x−θ = (x+g)−(θ+g) = T (gx, ḡθ) for all group elements g. Collecting the above assumptions, we have for further reference. Assumption 1. We have a model density f(x; θ); x ∈ X, θ ∈ Θ; for an observable X, with both X and θ being vectors, and we seek to estimate a parametric function τ(θ) (ℝ^{p} → ℝ) with the constraint τ(θ) ≥ 0. We assume there exists a pivot T (X, θ) = \frac{a_{1}(X)−τ(θ)}{a_{2}(X)}; a_{2}(·) > 0; such that −T (X, θ) has cdf G and Lebesgue density g_{0}. We further assume that the decision problem is invariant under a group G of transformations and that the pivot satisfies the invariance requirement T (x, θ) = T (gx, ḡθ), for all x ∈ X, θ ∈ Θ, g ∈ G, ḡ ∈ Ḡ, with X, Θ, G, and Ḡ being isomorphic.

We consider prior measures π_{H} and π_{0}, where π_{0}(θ) = π_{H}(θ)I_{[0,∞)}(τ(θ)), and π_{H} is the Haar right invariant measure which satisfies the property π_{H}(A ḡ) = π_{H}(A) for every measurable subset A of Θ, and for every g ∈ G. The right Haar measure π_{H} exists and is unique up to a multiplicative constant for locally compact groups such as location, scale, and location-scale. For the basic location and the Gamma model (or scale model) examples of the Introduction, right Haar invariant measures are given by π_{H}(θ) = 1 and π_{H}(θ) = \frac{1}{θ} respectively. For a sample from a location-scale family with X_{i} ∼ind. \frac{1}{θ_{2}} f_{2}(\frac{x_{i}−θ_{1}}{θ_{2}} ) , i = 1, …, n, the common non-informative prior π(θ) = \frac{1}{θ_{2}} is right Haar invariant. We refer to Berger (1985) or Eaton (1989) for detailed treatments of invariance and Haar invariant measures. A key feature relative to Assumption 1 and the choice of the right Haar invariant measure is that the frequentist distribution of T (X, θ); which is free of θ by virtue of the pivot assumption for T (X, θ); coincides with the posterior distribution of T (x, θ) under π_{H} for any given x, i.e.,

T (x, θ)|x =^{d} T (X, θ)|θ , for all x, θ.  (1)

We will pursue, after the next Lemma, by illustrating the above and drawing implications of immediate interest. For sake of completeness, we reproduce here a key lemma from MS(2006) justifying (1) and we refer to their work for further details. Lemma 1. (MS, 2006, Corollary 1) Suppose X, Θ, G, and Ḡ are all isomorphic, and that T (X, θ) is a function for which T (x, θ) = T (gx, ḡθ), for all x ∈ X, θ ∈ Θ, g ∈ G, ḡ ∈ Ḡ. Then condition (1) holds, that is P_{θ}[T (X, θ) ∈ B] = P^{π_{H}(θ|x)}[T (X, θ) ∈ B] for each measurable set B.

Now, for the basic unrestricted location family example with the flat prior π_{H}(θ) = 1, which is Haar right invariant, observe that the posterior density of θ is given by

π_{H}(θ|x) = \frac{f_{0}(x − θ)}{∫_{θ} f_{0}(x − θ) dθ} = f_{0}(x − θ) = g_{0}(θ − x) ,

so that the posterior density of −T (x, θ) = θ − x associated with π_{H} is given by g_{0} as well. This correspondence for basic example (i) illustrates property (1) which is, of course, more general under Assumption 1. In general, observe that the posterior cdf under π_{H} for τ(θ) is available from the fact that −T (X, θ) = \frac{τ(θ)−a_{1}(X)}{a_{2}(X)} ∼ G yielding

P_{π_{H}}(τ(θ) ≤ y|x) = G(\frac{y − a_{1}(x)}{a_{2}(x)}) .  (2)

Now, under the truncation π_{0} of π_{H}, the above correspondence between the frequentist and posterior distributions of −T (X, θ) does not hold, and the posterior cdf under π_{0} of τ(θ) differs. However, we can still express the posterior distribution of τ(θ) under π_{0} in terms of π_{H} and G. Indeed, with π_{0}(θ) = π_{H}(θ) 𝕀_{[0,∞)}(τ(θ)) and \frac{τ(θ)−a_{1}(x)}{a_{2}(x)}|x ∼ G under π_{H}, we have for a measurable set A ⊂ Θ_{0} = {θ ∈ Θ : τ(θ) ≥ 0}, and for any x:

P_{π_{0}}(θ ∈ A|x) = ∫_{A} π_{0}(θ|x) dθ
= \frac{∫_{A} π_{0}(θ) f(x|θ) dθ}{∫_{Θ_{0}} π_{0}(θ) f(x|θ) dθ}
= \frac{∫_{A} π_{H}(θ) f(x|θ) dθ}{∫_{Θ_{0}} π_{H}(θ) f(x|θ) dθ}
= \frac{P_{π_{H}}(θ ∈ A|x)}{P_{π_{H}}(θ ∈ Θ_{0}|x)} .

In terms of the posterior survival function of τ(θ) under π_{0}, the above yields along with (2), for y ≥ 0,

P_{π_{0}}(τ(θ) ≥ y|x) = \frac{P_{π}(τ(θ) ≥ y|x)}{P_{π}(τ(θ) ≥ 0|x)} = \frac{1 − G(\frac{y−a_{1}(x)}{a_{2}(x)})}{1 − G(\frac{−a_{1}(x)}{a_{2}(x)})} .  (3)

We will make use, in Section 3, of the above in setting and describing the bounds of Bayesian credible sets for τ(θ) under π_{0}.

2.2. The spending function associated with a Bayesian credible set

With the objective of constructing a (1−α)×100% Bayesian credible set or region, the determination of a posterior distribution for τ(θ) supported on [0, ∞) leaves open many choices and various different approaches (e.g., Berger, 1985, section 4.3.2). The HPD credible set is one such region chosen to minimize volume and leading to intervals for unimodal posterior densities. In our set-up, (1−α)×100% Bayesian credible intervals are, more generally, of the form [l(x), u(x)], x ∈ ℝ, where P (l(x) ≤ τ(θ) ≤ u(x)|x) = 1−α. An alternative (and equivalent) way to set or view the bounds l(x) and u(x), for a given x, is to focus on the complementary set [0, l(x))∪(u(x), ∞) and to allocate (or “spend”) probabilities α−α(x) and α(x) respectively on its two disjoint parts, with α(x) ∈ [0, α]. It is clear (when the posterior density is absolutely continuous) that the choice α(x) leads to a unique choice of [l(x), u(x)], and vice-versa. Since we are interested in the frequentist properties of such Bayesian credible intervals, we will represent this allocation as a spending function. Moreover, our findings guaranteeing minimal frequentist coverage of at least \frac{1−α}{1+α} for a class of Bayesian credible sets will be conveniently expressed as conditions on the corresponding spending function. Definition 1. For a given prior π for θ and a credibility coefficient 1 − α, a spending function α(·) : ℝ^{p} → [0, α] is a function such that, for all x, P_{π}(τ(θ) ≥ u(x)|x) = α(x), P_{π}(τ(θ) ≤ l(x)|x) = α − α(x), and [l(x), u(x)] is a (1 − α) × 100% Bayesian credible interval for τ(θ).

For example, a lower-tailed credible interval for a given x corresponds to the selection α(x) = α, an upper tailed credible interval corresponds to α(x) = 0, and an equal tailed (based on the posterior π) corresponds to α(x) = α/2.

2.3. Checklist

To facilitate the further presentation of the results, here is a list of definitions and notations used. Cheklist

• 1 − α: credibility or posterior coverage or nominal frequentist coverage (α ∈ (0, 1))

• T (X, θ) = \frac{a_{1}(X)−τ(θ)}{a_{2}(X)}: pivot

• π_{H}: unrestricted prior density chosen as the right Haar invariant measure

• π_{0} : prior density given by the truncation of π_{H} onto the restricted parameter space

• G: cumulative distribution function (cdf) of −T (X, θ)|x and of −T (X, θ)|θ under π_{H} (which coincide for all x, θ)

• g_{0} = G′: probability density function (pdf) of −T (X, θ)

• G^{−1}: inverse cdf

• α(·): spending function

• I_{π_{0},α(·)}(X) = [l(X), u(X)]: Bayesian credible set of credibility 1 − α associated with the prior π_{0} and the spending function α(·)

• C(θ): the frequentist coverage at θ of the confidence interval I_{π_{0},α(·)}(X) given by C(θ) = P_{θ}(I_{π_{0},α(·)}(X) ∋ τ(θ))

• y_{0} = −G^{−1}(\frac{α}{1+α})

• t(x) = \frac{a_{1}(x)}{a_{2}(x)}

• Δ_{0}(x) = (1 − α)(1 − G(−t(x)))

3. Bayesian credible intervals: representations and proper-

ties

In this section, we expand upon two different, yet equivalent, and instructive approaches to constructing a credible set for τ(θ) associated with prior π_{0}. These are: (A) the spending function approach, and (B) the approach based on the quantiles of the pivot.

(A) (Spending function approach) As seen above, a (1 − α) × 100% credible interval for τ(θ) associated with prior π_{0} can be generated by a spending function α(·) : ℝ^{p} → [0, α], such that I_{π_{0},α(·)}(X) = [l(X), u(X)] with P_{π_{0}}(τ(θ) ≥ u(x)|x) = α(x). More precisely, we have the following under Assumption 1.

Lemma 2. For a given spending function α(·), the bounds of I_{π_{0},α(·)}(x) are given by: l_{α(·)}(x) = a_{1}(x) + a_{2}(x)G^{−1}{G(−t(x)) + (α − α(x))(1 − G(−t(x))} and u_{α(·)}(x) = a_{1}(x) + a_{2}(x)G^{−1}{1 − α(x)(1 − G(−t(x))\}, with t(x) = \frac{a_{1}(x)}{a_{2}(x)}.

Proof. With the survival function P_{π_{0}}(τ(θ) ≥ y| x) = ^{\frac{1− G( \frac{y−a_{1}(x)}{a_{2}(x)} )}{1−G(−t(x))}} , as given in (3), we obtain for β ∈ (0, 1), y > 0, P_{π_{0}}(τ(θ) ≥ y| x) = β ⇔ y = a_{1}(x) + a_{2}(x)G^{−1}{1 − β + βG(−t(x))}, and the result follows with the choices β = α(x) and β = 1 − (α − α(x)) for u(x) and l(x) respectively.

Example 1. The HPD procedures studied by MS (2006) for symmetric about 0 and unimodal g_{0} are given by the bounds l(x) = max\{0, a_{1}(x)+a_{2}(x)G^{−1}(\frac{1−(1−α)G(t(x))}{2})\} and u(x) = a_{1}(x)+ a_{2}(x) min\{G^{−1}(1 − αG(t(x))), G^{−1}(\frac{1+(1−α)G(t(x))}{2})\} . With these given bounds, one may verify directly from (3) that the corresponding spending function is equal to

min\{α, \frac{α}{2} + \frac{G(−t(x)}{2(1 − G(−t(x)))}\},  (4)

with α(x) = α if and only if t(x) ≤ −G^{−1}(\frac{α}{1+α}) = G^{−1}(\frac{1}{1+α}) since g_{0} is symmetric about 0. Conversely, applying Lemma 2 with the spending function choice α(·) in (4) leads to the HPD procedure above (using the equality of G(·) and 1 − G(−·) for symmetric about 0 g_{0}’s).

(B) (Approach based on quantiles of the pivot) Alternatively, a second approach for cases where l(x) > 0 begins with choices γ_{1} and γ_{2}, which will be made for each x, such that G(γ_{2}) − G(−γ_{1}) = Δ, for a given Δ ∈ (0, 1). Since, for any x, we require 1 − α = P_{π_{0}}(l(x) ≤ τ(θ) ≤ u(x)|x), we must have by (3):

G(\frac{u(x) − a_{1}(x)}{a_{2}(x)}) − G(\frac{l(x) − a_{1}(x)}{a_{2}(x)}) = (1 − α)(1 − G(−t(x))),

and this can be achieved with choices −γ_{1} and γ_{2} above for Δ = Δ_{0}(x) = (1−α)(1−G(−t(x))) yielding \frac{u(x)−a_{1}(x)}{a_{2}(x)} = γ_{2}(Δ_{0}(x)) and \frac{l(x)−a_{1}(x)}{a_{2}(x)} = −γ_{1}(Δ_{0}(x)), in other words

l(x) = a_{1}(x) − a_{2}(x) γ_{1}(Δ_{0}(x)), and u(x) = a_{1}(x) + a_{2}(x) γ_{2}(Δ_{0}(x)) ,  (5)

whenever l(x) > 0. In view of the lower bound restriction on τ(θ) (i.e., τ(θ) ≥ 0), and the corresponding requirement that l(X) ≥ 0, observe that not all choices of −γ_{1} (and hence of γ_{2}) are feasible in (5) and that we must have

−γ_{1}(Δ_{0}(x)) ≥ −\frac{a_{1}(x)}{a_{2}(x)} .

Example 2. With the above construction in (5), an equal-tails choice of −γ_{1} and γ_{2}, that is −γ_{1}(Δ) = G^{−1}(\frac{1−Δ}{2}) and γ_{2}(Δ) = G^{−1}(\frac{1+Δ}{2}), leads to the credible interval bounds

l(x) = a_{1}(x) + a_{2}(x) G^{−1}(\frac{1 − Δ_{0}(x)}{2}), and u(x) = a_{1}(x) + a_{2}(x) G^{−1}(\frac{1 + Δ_{0}(x)}{2}),  (6)

when l(x) > 0. These above bounds coincide with those of the HPD procedure (when l(x) > 0) in the symmetric case of Example 1, as well as the spending function given in (4) as can be verified directly from (3). NOTE: We wish to emphasize that the terminology “equal tails” does not mean α(x) = α/2 (i.e., equal tails under the posterior distribution), but rather refers to the choice of (equal tails) quantiles −γ_{1} and γ_{2} under G.

The next section’s lower bound of \frac{1−α}{1+α} on frequentist coverage applies to a class of Bayesian credible intervals. This class will include an equal-tails credible interval I_{π_{0},α_{eqt}(·)} which relates to both approaches presented in this section. On one hand, it borrows the bounds (and hence the spending function) of the HPD procedure for symmetric about 0 unimodal densities and, on the other hand, it is defined through the above equal-tailed choice (whenever l(x) > 0).

Definition 2. In the context of Assumption 1, the G-equal-tails credible interval I_{π_{0},α_{eqt}(·)}(X) is given by the bounds l(x) = max\{0, a_{1}(x)+a_{2}(x)G^{−1}(\frac{1−(1−α)G(t(x))}{2})\} and u(x) = a_{1}(x)+a_{2}(x) min\{G^{−1}(1− αG(t(x))), G^{−1}(\frac{1+(1−α)G(t(x))}{2})\} . Equivalently, I_{π_{0},α_{eqt}(·)}(X) is given by the spending function

α_{eqt}(x) = min\{α, \frac{α}{2} + \frac{G(−t(x)}{2(1 − G(−t(x)))}\}.  (7)

4. Frequentist coverage properties

We study here the frequentist coverage properties, under Assumption 1, of a class of Bayesian credible intervals which includes the equal-tails credible interval I_{π_{0},α_{eqt}(·)}(X). This procedure, as well as Example 1’s HPD procedure for symmetric g_{0}, produces estimates of the form [0, u(x)] if and only if t(x) ≤ y_{0}, where y_{0} = −G^{−1}(\frac{α}{1+α}) (and t(x) = \frac{a_{1}(x)}{a_{2}(x)} as above). We thus focus on a class of credible intervals with the same behaviour. Said otherwise in terms of the spending function, we impose the choice α(x) = α whenever t(x) ≤ y_{0}. We hence seek conditions on α(x), for those x’s such that t(x) ≥ y_{0}, for which minimal frequentist coverage is bounded below by \frac{1−α}{1+α}.

Theorem 1. Under the conditions of Theorem 1 of Marchand and Strawderman (2006), that is Assumption 1, consider Bayesian credible intervals I_{π_{0},α(·)} associated with prior π_{0} and a spending function α(·) such that α(x) = α for all x with t(x) ≤ y_{0}. For the frequentist coverage C(θ) = P_{θ}(I_{π_{0},α(·)}(X) ∋ τ(θ)), we then have

(a) C(θ) = \frac{1}{1+α}(> \frac{1−α}{1+α}) for all θ such that τ(θ) = 0; (b) Moreover, we have C(θ) > \frac{1−α}{1+α} for all θ such that τ(θ) ≥ 0 as long as α(x) satisfies, for all x,

\frac{(1 − α) G(−t(x)) + \frac{α^{2}}{1+α}}{1 − G(−t(x))} ≤ α(x) ≤ (\frac{α}{1 + α})\frac{1}{1 − G(−t(x))}.  (8)

Proof.

(a) First, observe that for θ such that τ(θ) = 0, the pivot assumption for −T (X, θ) = \frac{τ(θ)−a_{1}(X)}{a_{2}(X)} implies that −t(X) = −\frac{a_{1}(X)}{a_{2}(X)} has cdf G whenever τ(θ) = 0. Hence, for θ such that τ(θ) = 0, we have

P_{θ}(I_{π_{0},α(·)}(X) ∋ 0) = P_{θ}(α(X) = α) = P_{θ}(t(X) ≤ y_{0}) = 1 − G(−y_{0}) = \frac{1}{1 + α} .

(b) With the case τ(θ) = 0 addressed in part (a), we consider τ(θ) > 0. First, observe that the confidence interval I_{1}(X) = [l_{1}(X), u_{1}(X)] = max\{0, a_{1}(X) + a_{2}(X)G^{−1}(\frac{α}{1+α})\}, a_{1}(X) + a_{2}(X)G^{−1}(\frac{1}{1+α})\} has the same frequentist coverage as I_{1}^{∗}(X) = [a_{1}(X)+a_{2}(X)G^{−1}(\frac{α}{1+α})\}, a_{1}(X)+ a_{2}(X)G^{−1}(\frac{1}{1+α})\}] equal to P_{θ}(G^{−1}(\frac{α}{1+α}) ≤ \frac{τ(θ)−a_{1}(X)}{a_{2}(X)} ≤ G^{−1}(\frac{1}{1+α})) = G(G^{−1}(\frac{1}{1+α}))−G(G^{−1}(\frac{α}{1+α})) = \frac{1−α}{1+α}. Now, we show that the given conditions on α(·) imply that I_{π_{0},α(·)} ⊇ I_{1}; with the inclusion being strict with probability greater than 0 for all θ; which will lead to the result directly. Indeed, we have by the upper bound in (8) and Lemma 2: u_{α(·)}(x) ≥ a_{1}(x) + a_{2}(x)G^{−1}(1 − \frac{α}{1+α}) = u_{1}(x). Similarly, from the lower bound (8) and Lemma 2 we obtain l(x) ≤ a_{1}(x) + a_{2}(x)G^{−1}\{G(−t(x) + α(1 − G(−t(x))) − \frac{α^{2}}{1+α} − (1 − α)G(−t(x))\} = a_{1}(x) + a_{2}(x)G^{−1}(\frac{α}{1+α}) = l_{1}(x).

Corollary 1. Under Assumption 1, the G-equal-tails credible interval I_{π_{0},α(·)}, given in Definition 7, has minimum frequentist coverage C(θ) greater than \frac{1−α}{1+α} for all θ such that τ(θ) ≥ 0.

Proof. It suffices to show directly that (8) is satisfied for the selection α(x) = αeqt(x) given in (4) for x such that t(x) ≥ y_{0}. Indeed, we have for such x’s:

αeqt(x)(1 − G(−t(x))) = \frac{α}{2} + \frac{1 − α}{2} G(−t(x)) ≤ \frac{α}{2} + \frac{1 − α}{2} G(−y_{0}) = \frac{α}{1 + α} ,

and

αeqt(x)(1 − G(−t(x))) − (1 − α) G(−t(x)) − \frac{α^{2}}{1 + α} = \frac{α(1 − α)}{2(1 + α)} − \frac{1 − α}{2} G(−t(x))
≥ \frac{α(1 − α)}{2(1 + α)} − \frac{1 − α}{2} G(−y_{0})) = 0.  □

Remark 1. In cases where the underlying pivotal distribution is non-symmetric, Corollary 1 is a new result, generalizing Theorem 1 of MS (2006), and is widely applicable given the lack of assumptions on g_{0}. Also, the bounds of the equal-tails procedure are easier to evaluate than that of the HPD credible interval. And the findings of Theorem 1 go beyond a single procedure, even in the symmetric case, by providing a class of credible sets, as specified by a spending function, with frequentist coverage bounded below by \frac{1−α}{1+α}.

We do not have a recommended prescription for the choice of the spending function among those specified by Theorem 1 as guaranteeing minimal frequentist coverage of at least \frac{1−α}{1+α}. The G−equaltails choice is simple, intuitively appealing and matches the HPD procedure under symmetry of the pivotal density, while upper tailed and lower tailed choices are not allowed for x such that t(x) ≥ y_{0}. The bounds in (8): (i) α_{1}(x) = ^{\frac{(1−α) G(−t(x))+ \frac{α^{2}}{1+α}}{1−G(−t(x))}} and (ii) α_{2}(x) = (\frac{α}{1+α})\frac{1}{1−G(−t(x))}, are other interesting choices which push extremally I_{π,α(·)} towards +∞ and 0 respectively. Finally, along with these choices, it might be feasible to minimize the length of the credible interval under the restrictions imposed by Theorem 1.

5. Examples

At the risk of some redundancy with the examples provided by MS (2006), it is still beneficial here to present various applications with accompanying commentary. Assumption 1 is satisfied in all of the examples below with the underlying family of transformations (distributions) being either the location family, the scale family, or the location-scale family. In all of the examples, Theorem 1 and Corollary 1 provide conditions on the spending function α(·) so that the Bayesian intervals I_{π_{0},α(·)}(X) have minimal frequentist coverage greater than \frac{1−α}{1+α} for all θ such that τ(θ) ≥ 0. These intervals include the equal-tails procedure given in Definition 2 and can be evaluated in general using the expression given in Lemma 2.

(A) (a single location parameter) X ∼ f_{0}(x − θ); τ(θ) = θ ≥ 0; T (X, θ) = X − θ; π_{H}(θ) = 𝕀_{ℝ}(θ), π_{0}(θ) = 𝕀_{[0,∞)}(θ). In such cases, all Bayes credible sets I_{π_{0},α(·)} (with credibility 1 − α), with the spending function α(·) satisfying the conditions of Theorem 1 and the bounds in (8), have necessarily minimum frequentist coverage bounded below by \frac{1−α}{1+α}. Through the transformations X → X − a and X → −X + a, one can reduce all lower bounded restrictions θ ≥ a and upper bounded restrictions θ ≤ a to the case θ ≥ 0 considered here and we will not make further explicit mention of such transformations below.

Remark 2. Results such as those in (A) are applicable as well for several observations by conditioning on a maximal invariant statistic V. Such a maximal invariant statistic V is an ancillary statistic and specifically an invariant function such that every other invariant statistic is a function of V. Indeed, suppose that X = (X_{1}, …, X_{n}) ∼ f_{0}(x_{1} − θ, …, x_{n} − θ), where f_{0} is known and where the X_{i}’s are not necessarily independently distributed. Here, V = (X_{2} − X_{1}, …, X_{n} − X_{1}) is a maximal invariant statistic. One can then proceed, for a given value v of V, with an interval estimate I_{π_{0},α(·,v)}(X_{1}, v) as given in Lemma 2 with G ≡ G_{v} representing the cdf of the pivot X_{1}−θ conditional on V = v, and α(x, v) satisfying the conditions of Theorem 1 and (8). This is feasible by the pivot and ancillarity property with the joint distribution of (X_{1} − θ, V) independent of θ. In such a case, Theorem 1 applies to the conditional frequentist coverage C(θ, v) = P_{θ}(I_{π_{0},α(·,v)}(X, v) ∋ τ(θ)|V = v) yielding the inequality C(θ, v) > \frac{1−α}{1+α} for all θ ≥ 0. Since this is true for all v, the unconditional frequentist coverage C(θ) of the Bayes credible set I_{π_{0},α(·,·)}(X, V) will also exceed \frac{1−α}{1+α} for all θ ≥ 0 (see MS 2006, for more details related to a multivariate Student model). In the same vein, all the scenarios below (B to G), although presented for simplicity in the single observation case, are also applicable in presence of a sample by conditioning on a maximal invariant statistic.

(B) (a lower bounded scale parameter) X ∼ \frac{1}{θ}f_{1}(\frac{x}{θ}) 𝕀_{(0,∞)}(x) with θ ≥ a; τ(θ) = log(θ) − log(a) ≥ 0; T (X, θ) = log(X) − log(a) − τ(θ); π_{H}(θ) = \frac{1}{θ}𝕀_{(0,∞)}(θ), π_{0}(θ) = \frac{1}{θ}𝕀_{[0,∞)}(τ(θ)). Here, an interval estimate of τ(θ) provides an interval estimate of θ. Important models include Gamma, Weibull, Fisher, among others. A familiar set-up where the results can be applied arises in random effects analysis of variance models with a Fisher distributed pivot (see Zhang and Woodroofe, 2002, for details). As in (A), for a sample X = (X_{1}, …, X_{n}) ∼ \frac{1}{θ^{n}} f_{1}(\frac{x_{1}}{θ} , …, \frac{x_{n}}{θ}) ∏_{i} 𝕀_{(0,∞)}(x_{i}), Theorem 1 and Corollary 1 are applicable by conditioning on the maximal invariant statistic V = (\frac{X_{1}}{X_{n}} , …, \frac{X_{n−1}}{X_{n}}).

Remark 3. Further applications consist of power parameter families where we have a scale family for an observable Y and the model of interest are the distributions for X = e^{Y}. As as simple illustration, consider the Pareto model for X with densities \frac{γ}{x^{γ+1}} 1_{(1,∞)}(x) and the parametric constraint γ ∈ (1, γ_{0}). In such cases, we have that γ_{0} log(X) ∼ Exp(θ) with θ = \frac{γ_{0}}{γ} ≥ 1 and the results in (B) apply.

(C) (location-scale families ) (X_{1}, X_{2}) ∼ \frac{1}{θ_{2}^{2}} f_{2}(\frac{x_{1}−θ_{1}}{θ_{2}} , \frac{x_{2}}{θ_{2}} ) 𝕀_{(0,∞)}(x_{2}); τ(θ) = θ_{1} ≥ 0; T (X, θ) = \frac{X_{1}−θ_{1}}{X_{2}} ; π_{H}(θ) = \frac{1}{θ_{2}} 𝕀_{(0,∞)}(θ_{2})𝕀_{(−∞,∞)}(θ_{1}), π_{0}(θ) = \frac{1}{θ_{2}} 𝕀_{(0,∞)}(θ_{2})𝕀_{[0,∞)}(θ_{1}). This set-up encompasses, but is not limited to, the basic normal case: Y_{1}, … Y_{n} ∼^{ind.} N(μ, σ^{2}) with σ^{2} unknown and μ ≥ 0, and by taking X_{1} and X_{2} respectively as the sample mean and standard deviation of the Y_{i}’s. More generally, the results apply for linear models Y = Zβ + ϵ, ϵ ∼ N(0, σ^{2}I_{n}) where the objective is to estimate a lower-bounded linear combination τ(θ) = l′β, by setting X_{1} = β̂(Z′Z)^{−1}Z′Y, X_{2}^{2} = ∥Y − Zβ∥^{2}, θ_{1} = β, θ_{2} = σ. Here, the pivot T (X, θ) has a Student distribution. Alternatively, if the objective is to estimate a lower bounded scale θ_{2}, one can proceed as in (B).

(D) (linear combination of several location parameters) X = (X_{1}, …, X_{p}) ∼ f_{0}(x_{1} − θ_{1}, …, x_{p}−θ_{p}); τ(θ) = ∑_{i=1}^{p} a_{i}θ_{i}; π_{H}(θ) = 𝕀_{ℝ^{p}}(θ), π_{0}(θ) = 𝕀_{[0,∞)}(τ(θ)), T (X, θ) = (∑_{i=1}^{p} a_{i}X_{i})− τ(θ). This set-up includes, for instance, estimating a difference θ_{1}−θ_{2} with an order constraint θ_{1} ≥ θ_{2}.

(E) (multivariate location-scale families with homogeneous scale ) In (D), we can incorporate a common scale and apply the results of this paper for estimating a lower bounded linear combination with X = (X_{1}, …, X_{p} , X_{p+1}) ∼ f_{0}(\frac{x_{1}−θ_{1}}{θ_{p+1}} , …, \frac{x_{p}−θ_{p}}{θ_{p+1}} , \frac{x_{p+1}}{θ_{p+1}} ), τ(θ) = ∑_{i=1}^{p} a_{i}θ_{i}, T (X, θ) = \frac{(∑_{i=1}^{p} a_{i}X_{i})−τ(θ)}{X_{p+1}}, and π_{0}(θ) = \frac{1}{θ_{p+1}} 1_{(0,∞)}(θ_{p+1})1_{[0,∞)}(τ(θ)).

(F) (several scale parameters ) (X_{1}, …, X_{p}) ∼ (Π_{i=1}^{p} \frac{1}{θ_{i}}) f_{1}(\frac{x_{1}}{θ_{1}} , …, \frac{x_{p}}{θ_{p}}); τ(θ) = ∑_{i=1}^{p} a_{i} log(θ_{i}), π_{H}(θ) = ∏_{i} \frac{1}{θ_{i}} 𝕀_{(0,∞)}(θ_{i}), π_{0}(θ) = π_{H}(θ) 𝕀_{[0,∞)}(τ(θ)). This can consist, for instance with p = 2, a_{1} = 1, a_{2} = −1, of estimating a lower bounded ratio \frac{θ_{2}}{θ_{1}} ≥ 1 of two scale parameters.

(G) (quantiles in location-scale families) X_{i} ∼ind. N(μ, σ), i = 1, … n, θ = (μ, σ), τ(θ) = μ + ησ ≥ 0, π_{H}(θ) = \frac{1}{θ_{2}} 𝕀_{(0,∞)}(θ_{2})𝕀_{(−∞,∞)}(θ_{1}), π_{0}(μ, σ) = \frac{1}{σ}𝕀_{(0,∞)}(σ)𝕀_{[0,∞)}(μ + ησ). T (X, θ) = \frac{X̄−μ−ησ}{S}. Here, T (X, θ) is distributed as non-central Student. The applications are not restricted to normality and are applicable in general for location-scale families as in (C).

6. Concluding remarks

For a large variety of situations with a lower bounded parametric constraint, we have obtained a class of Bayesian (1 − α) × 100% credible sets which provide minimal frequentist probability coverage exceeding \frac{1−α}{1+α}. These Bayesian confidence intervals include an equal tailed modification or approximation of the HPD credible set which coincides with the latter when the distribution of the underlying pivot is symmetric. In non-symmetric cases not covered by Marchand and Strawderman (2006), our findings provide instances of Bayesian credible sets with given minimal frequentist coverage and hence fill a gap in the literature. In comparison to earlier results for normal models, as well as the symmetric models considered by Marchand and Strawderman (2006), the findings here relative to the HPD are not new, but those related to other Bayesian credible sets are an addition. In seeking to evaluate the frequentist performance of Bayesian confidence intervals, our results illustrate that the choice of bounds or spending function matters, so that there does not necessarily exist a single universal assessment of their frequentist performance even in a given specific problem.

Acknowledgements

The authors are grateful to two reviewers, an associate editor, and the editor for useful comments and suggestions which led to a more self-contained and readable manuscript. Eric Marchand’s research is supported in part by a grant from the Natural Sciences and Engineering Research Council of Canada, and William Strawderman’s research is partially supported by a grant from the Simons

Foundation (#209035).

References

Berger, J.O. (1985). Statistical Decision Theory and Bayesian Analysis. Springer-Verlag, New York, 2nd edition.

Eaton, M. L. (1989). Group invariance Applications in Statistics. NSF-CBMS Regional Conference Series in Probability and Statistics 1. Hayward, CA.

Feldman, G.J. and Cousins, R. (1998). Unified approach to the classical statistical analysis of small signals. Physical Review D, 57, 3873-3889.

Fraser, D.A.S. (2011). Is Bayes posterior just quick and dirty confidence? Statistical Science, 26, 299-316.

Mandelkern, M. (2002). Setting Confidence Intervals for Bounded Parameters with discussion. Statistical Science, 17, 149-172.

Marchand, É., Strawderman, W. E., Bosa, K., and Lmoudden, A. (2008). On the frequentist coverage of Bayesian credible intervals for lower bounded means. Electronic Journal of Statistics, 2, 1028-1042.

Marchand, É. and Strawderman, W. E. (2006). On the behaviour of Bayesian credible intervals for some restricted parameter space problems. Recent Developments in Nonparametric Inference and Probability : A Festschrift for Micheal Woodroofe,, IMS Lecture Notes-Monograph Series, 50, pp. 112-126.

Robert, C.P. (2011). Discussion of “Is Bayes posterior just quick and dirty confidence?” by D.A.S. Fraser. Statistical Science, 26, 317-318.

Roe, B. and Woodroofe, M. (2000). Setting confidence belts. Physical Review D, 63, 013009/01-09.

Wasserman, L. (2011). Frasian inference. Statistical Science, 26, 322-325.

Zhang, T. and Woodroofe, M. (2003). Credible and confidence sets for restricted parameter spaces. Journal of Statistical Planning and Inference, 115, 479-490.

Zhang, T. and Woodroofe, M. (2002). Credible and confidence sets for the ratio of variance components in the balanced one-way model. Sankhyā: Special issue in memory of D. Basu, 64, 545-560.