Measuring the Discrepancy of a Parametric Model via Local Polynomial Smoothing

Anouar El Ghouch^{1} , Marc G. Genton^{2} and Taoufik Bouezmarni^{3}

1 Institut de Statistique, Université Catholique de Louvain, B-1348 Louvain-la-Neuve,

Belgium. E-mail: Anouar.Elghouch@uclouvain.be

2 Department of Statistics, Texas A&M University, College Station, TX 77843-3143, U.S.A.

E-mail: genton@stat.tamu.edu

3 Département de mathématiques, Université de Sherbrooke, Québec, Canada, E-mail: taoufik.bouezmarni@usherbrooke.ca.

September 27, 2010

Abstract

In the context of multivariate mean regression we propose a new estimator of the minimum L^{2}-distance between the true but unknown regression curve and a given parametric family. The method is based on local polynomial averaging of residuals with a polynomial degree that increases with the dimension of the covariate. Under some weak assumptions we give a Bahadur-type representation of the estimated distance from which root-n consistency and asymptotic normality are derived for strongly mixing variables. We then show how to use the proposed method to: (i) measure the explanatory power of a given set of covariates or a given parametric model; and (ii) test the goodness-of-fit hypothesis. We conclude with a simulation study that aims at checking the finite sample properties of these techniques.

KEY WORDS: Explanatory power; Goodness-of-fit; Model misspecification; Multivariate local polynomial smoothing; Strong mixing sequences; Hypothesis test. Short title: Discrepancy of a Parametric Model

1 Introduction

In the context of regression with high dimensional predictors, it is difficult to get an efficient nonparametric estimator for the true regression function because of the sparsity of the data, the so-called curse of dimensionality. For that reason and for the purpose of interpretability, simple parametric models with few covariates are usually preferred to a purely nonparametric fit or to complex and possibly redundant parametric models. Therefore, the selection of an appropriate set of covariates to be used and the selection of an adequate parametric function to fit and make inference about the data are two of the most important and challenging problems in real data analysis. Typically, attention is focused on checking the adequacy of a particular parametric model without inquiring about the overall quality of the regressors. However, if the covariates at hand only capture a small amount of the variability of the output variable then it is obviously difficult to justify the choice of any particular parametric model. The problem becomes even more difficult with a small sample size or with a complex data structure such as in the presence of dependency among the observations. It turns out that one should consider both aspects in order to make the best decision by first selecting a subset of regressors without any parametric restrictions and then choosing an appropriate model taking into account the explanatory power of the selected covariates. Although covariate selection and model selection are of different nature, the main question in both problems is whether the variance explained by the function/covariates is relevant when compared to the variance due to errors.

During the last decades a very large amount of research related to this topic has been proposed with a variety of procedures, justifications and assumptions. The traditional literature includes the use of model selection criteria such as Akaike and Bayesian information criteria and the use of test statistics such as Wald and likelihood ratio tests. In the first case the selection is done by choosing the model with the smallest criterion (error) among competing models/regressors while in the second case the selection is based on a test statistic that measures the departure from the null hypothesis in the direction of an alternative. As mentioned by Taylor (2009), “the ordering of statistical tests and the choice of the significance level can influence the outcome, as can the choice of the model under the null hypothesis”. For an excellent review and a detailed discussion of model/covariate selection procedures and tests we refer to Lavergne (1998). To be consistent, the classical approaches impose severe restrictions on the true underling parametric structure and depend heavily on some strong assumptions about data such as normality of residuals, homoscedasticity, or the fact that the correct model belongs to the set of candidate models. Modern literature avoids these drawbacks by using nonparametric techniques that allow for great flexibility. Methods such as kernel smoothing and splines have become widely used to justify covariate and model selection. The literature related to this subject is vast and includes the work of Härdle and Mammen (1993), Hong and White (1995), Zheng (1996), Li and Wang (1998) and Jun and Pinkse (2009) for consistently testing a parametric regression functional form and Fan and Li (1996), Lavergne and Vuong (1996) and Gu et al. (2007) for testing relevant regressors.

In this context, we propose a new method based on a minimum L^{2}-distance between the parametric family and the unknown true regression function. This basic idea is behind many goodness-of-fit tests proposed in the literature. Commonly, these classical methods focus on the behavior of a test statistic under the null hypothesis that the given model is correct. Our approach differs from existing methods in many aspects. Rather than a testing problem, our purpose is to construct an estimator of the distance between the parametric model and the target function with some good statistical properties regardless of whether the given parametric model is correct or not. In that sense, the proposed estimator can be seen as a selection criterion and, so, be used to discriminate between several models. Under some weak assumptions, our estimator is shown to be consistent and asymptotically unbiased. We also prove its asymptotic normality with the optimal root-n convergence rate. These results are stated under a random design and we allow for weakly dependent data which means that our method can be applied also in time series or spatial frameworks. Unlike many existing methods that treat only some particular parametric functions such as linear or polynomial functions, our approach can be applied to check the quality of any smooth parametric model without further restriction on its form and without the need of any bias correction or bootstrap procedure. The proposed distance estimator is then used to estimate an “inadequacy index” which serves as a kind of an inverse coefficient of determination (R^{2}): it takes values in [0, 1] and when its value is close to 0 the parametric fit becomes better. Based on this result we are able to test the adequacy of a given parametric model using the asymptotic normality of that estimator. The main difficulty here is degeneracy of the test statistic under correct specification. To bypass this problem without sacrificing the power and the rate of convergence we adopt the concept of neighborhood hypotheses; see Dette and Munk (2003) for a very nice discussion. This method allows for the validation of a given parametric model which cannot be done with the classical goodness-of-fit tests. Finally we also discuss the problem of variable selection and show how the methodologies described above can be directly adapted to that.

The rest of the paper is organized as follows. In Section 2 we introduce the L^{2}-distance between the mean regression function and a parametric model. The nonparametric estimation procedure is also described in this section. Section 3 is devoted to the asymptotic properties of the proposed estimator. In Section 4 we define an inadequacy index and show how it can be used to validate a given model via neighborhood hypotheses testing. In that section, we also discuss the problem of covariate selection. The performance of the proposed method is examined in Section 5 via a Monte Carlo simulation study. The proofs of the asymptotic results are collected in the Appendix.

2 Estimation Procedure

2.1 Problem setting

Let (X, Y) be a random vector in ℝ^{d} × ℝ. For a given x ∈ ℝ^{d}, we denote by m(x) the conditional mean of Y given that X = x. Define ϵ = Y − m(X) and denote by f the marginal density of X. Let the function m(θ, x) be a parametric model. This function is known up to the finite parameter θ that belongs to the parameter space Θ which is assumed to be a compact subset of ℝ^{q}. We consider the function m(θ, x) as a member of the family of parametric functions M = {m(θ, x), θ ∈ Θ}.

The data are given by (X_{i}, Y_{i}), i = 1, …, n and have the same distribution as (X, Y). As dependent observations are considered in this paper, we introduce here the mixing coefficient. Let F_{I}^{L} (−∞ ≤ I, L ≤ ∞) denote the σ-field generated by the family {(X_{t}, Y_{t}), I ≤ t ≤ L}. The stochastic processes {(X_{t}, Y_{t})} is said to be strongly mixing if the α-mixing coefficient α(t) = sup_{A∈F_{−∞}^{0},B∈F_{t}^{∞}} |P (A ∩ B) − P (A)P (B)| converges to 0 as t → ∞. This dependency structure includes numerous random sequences. Among them are the independent and mdependent variables and, under some weak conditions, the classical linear and nonlinear ARMA and (G)ARCH time series; see, for example, Fan and Yao (2003) and Carrasco and Chen (2002) for further details. As we will see later, the dependency among observations does not have any impact on the asymptotic results, provided that the degree of the dependence, as measured by the mixing coefficient α(t), is weak enough such that assumption (A6) given below is satisfied.

2.2 The parameter of interest

As mentioned in the introduction, our main objective here is to assess the adequacy of the function m(θ, x) as an approximation of the true but unknown mean regression function m(x). We measure the distance between the regression function m and m(θ, x) by T (θ) = 𝔼[(m(X) − m(θ, X))^{2}]. This is nothing but the L^{2}-distance, with respect to f, that separates m(θ, X) from m(X). Because in many situations one is only interested in measuring the quality of m(θ, x) within a subset of the support of the design density f, it is better to work with the following weighted version

T_{φ}(θ) = 𝔼[Δ^{2}(θ, X)φ^{2}(X)],

where Δ(θ, x) = m(x) − m(θ, x) and φ(x) is a known weight function. In this way only the set {x : φ(x) ≠ 0} matters. In practice, the weighted version may also be used to avoid highly uncertain estimation in regions with sparse or noisy data. For example, by choosing φ = I_{{f>δ}}, with I being the standard indicator function and {f > δ} = {x : f(x) > δ}, for some δ > 0, one may avoid troubles that arise when the density f approaches 0, which is especially problematic in the context of nonparametric inference. From now on we will only consider the weighted version T_{φ} of T, so all upcoming formulas will depend on φ, but for notational convenience we will suppress φ in all our notations. Let θ̂ denote an estimator of θ. The parametric estimation procedure will be discussed latter. For now, the only assumption that we need is that θ̂ is pseudo-consistent. By this, we mean that there exists a unique parameter θ^{∗} in the interior of Θ such that θ̂ converges in probability to θ^{∗}, i.e. θ̂ = θ^{∗} +o_{p}(1). Our parameter of interest is T (θ^{∗}) = T_{φ}(θ^{∗}), i.e., the L^{2}-distance between m(·) and m(θ^{∗}, ·).

2.3 The local polynomial smoother

We now explain the kernel smoothing procedure that is used in the estimation step. Let K denote a nonnegative kernel function defined on ℝ^{d}, 0 < h_{n} ≡ h → 0 be a bandwidth parameter and K_{h}(x) = h^{−d}K(x/h). For any θ ∈ Θ, denote Y (θ) = Y − m(θ, X). By definition, Δ(θ, x) = 𝔼[Y (θ)|X = x]. If a parameter θ is available and if we consider (X_{i}, Y_{i}(θ)), i = 1, …, n, as the observed sample and Δ(θ, x) as the objective function, then we could directly apply classical smoothing techniques to construct a valid nonparametric estimator of Δ(θ, x). In fact, using the local averaging principle, we propose to estimate Δ(θ, x) by

Δ̂(θ, x) = ∑_{j=1}^{n} w_{j}(x)Y_{j}(θ),  (1)

where w_{j}(x), j = 1, …, n, are local weight functions depending on x, on {X_{1}, …, X_{n}}, on the bandwidth parameter h and on the kernel function K. The form of (1) is shared by many nonparametric estimators of a regression function; see, for example, Fan and Gijbels (1996) for more details. In particular, the multivariate local polynomial estimator of order p, p ∈ ℕ, which is based on a multivariate local polynomial approximation, i.e., a Taylor expansion, of the target function Δ(θ, x), can be expressed as (1). In this case the weight functions, w_{j}(x), take different forms depending on the dimension d and the value of p. For the local constant regressor, i.e., p = 0, w_{j}(x) = K_{h}(X_{j} − x)/ ∑_{i=1}^{n} K_{h}(X_{j} − x). For the univariate local linear estimator, i.e., p = 1, and d = 1, w_{j}(x) = n^{−1}K_{h}(X_{j}−x)[s_{n,2}(x)−\frac{X_{j}−x}{h}s_{n,1}(x)]/[s_{n,0}(x)s_{n,2}(x)− s_{n,1}^{2}(x)], where s_{n,k}(x) = n^{−1} ∑_{j=1}^{n} [(X_{j} − x)/h]^{k} K_{h}(X_{j} − x). The general expression of the local polynomial multivariate weight function w_{j}(x), for any p ≥ 0 and d ≥ 1, can be found in the Appendix, see (9). The estimator given by (1) is only available when θ is known, which, of course, is not the case here. So, we use the pseudo-consistent estimator θ̂ to get the following feasible estimator: Δ̂(θ̂, x) = ∑_{j=1}^{n} w_{j}(x)Y_{j}(θ̂) = m̂(x)−m̂(θ̂, x), where m̂(x) = ∑_{j=1}^{n} w_{j}(x)Y_{j} is the standard (nonparametric) local polynomial estimator of the mean regression function m(x) and m̂(θ̂, x) = ∑_{j=1}^{n} w_{j}(x)m(θ̂, X_{j}) is a smooth version of the parametric estimator m(θ̂, x). It is known that smoothing the parametric estimator makes it asymptotically biased exactly as the standard nonparametric fit m̂(x). Such a procedure is largely used in the theory of goodness-of-fit test to improve power.

2.4 Our estimator of T (θ^{∗})

Now that we have a valid estimator of Δ(θ^{∗}, x), and given the fact that T (θ) = 𝔼[Δ^{2}(θ, X)φ^{2}(X)] we may consider estimating T (θ^{∗}) using the obvious statistic

T_{n}^{0}(θ̂) = n^{−1} ∑_{i=1}^{n} Δ̂^{2}(θ̂, X_{i})φ^{2}(X_{i}).  (2)

Up to the factor h^{d/2}, this is a discrete (Riemann sum) version of the test statistic that was proposed by Härdle and Mammen (1993) in the context of goodness-of-fit test. They used the local constant weight function w_{j}(x), as defined above, and under some regularity assumptions, they showed the asymptotic normality of their test statistic when m ∈ M. In the present work, the primary objective is not about testing but about constructing an estimator of T (θ^{∗}) with some “good” properties. To this end, we consider here another estimator of T (θ^{∗}) given by T_{n}(θ̂), with

T_{n}(θ) = n^{−1} ∑_{i=1}^{n} (2Y_{i}(θ)Δ̂(θ, X_{i}) − Δ̂^{2}(θ, X_{i})) φ^{2}(X_{i}).  (3)

It is straightforward to show that this estimator is simply the empirical version of the following expression:

T (θ) = 𝔼 [(2Y (θ) − Δ(θ, X)) Δ(θ, X)φ^{2}(X)] .

Later, see Remark 2 and Section 5, the advantages of T_{n} over T_{n}^{0} will become clear. Another way to motivate the choice of this estimation procedure is via the influence function approach. In fact, it can bee shown that T_{n}(θ) is the one-step estimator of T (θ) based on its influence function. More details can be found in Doksum and Samarov (1995). This approach is widely used in parametric and semiparametric theory to construct asymptotically linear estimators with high efficiency; see Bickel et al. (1993).

3 Main Results

Before starting with the study of the asymptotic properties of the proposed estimator we need first to introduce some notations and give a set of sufficient regularity conditions needed for the results to hold. For a d-tuple k = (k_{1}, …, k_{d}) ∈ ℕ^{d} and a d-vector x = (x_{1}, …, x_{d}) ∈ ℝ^{d}, we write

x^{k} = x_{1}^{k_{1}} × ⋯ × x_{d}^{k_{d}}, |k| = ∑_{l=1}^{d} k_{l}, and (D^{k}m)(x) = \frac{∂^{k}m(x)}{∂x_{1}^{k_{1}} ⋯ x_{d}^{k_{d}}} .

ASSUMPTIONS (A)

(A1) x → m(θ, x) is a continuous function on S ⊂ ℝ^{d} for each θ in Θ. θ → m(θ, x) is twice differentiable on Θ for each x in S. The functions ṁ := \frac{∂m}{∂θ} and m̈ := \frac{∂^{2}m}{∂θθ^{T}} are continuous on Θ × S.

(A2) φ has a compact support D ⊂ int(S), where int(S) is the interior of S.

(A3) The marginal density f of X is bounded, uniformly continuous, and for all x ∈ D f(x) > L for some L > 0. For every l ≥ 1, the joint density of (X_{1}, X_{l+1}) is bounded.

(A4) The conditional density of X given Y exists and is bounded. For every l ≥ 1, the conditional density of (X_{1}, X_{l+1}) given (Y_{1}, Y_{l+1}) exists and is bounded.

(A5) For every k with |k| = p + 1, (D^{k}m) is a bounded Lipschitz function.

(A6) 𝔼|Y |^{δ} < ∞ for some δ > 2, h_{n} ∼ (n^{−1} ln n)^{a} for some 0 < a < d^{−1}(1 − 2/δ) and α(t) = O(t^{−ā}), with ā > max (\frac{2ν}{ν−2}, \frac{δ(7+2d)−4}{δ(1−ad)−2}) fore some ν ∈ (2, δ].

(A7) The kernel K is a bounded nonnegative function with compact support, say [−1, 1]^{⊗d} and for every k with 0 ≤ |k| ≤ 2p the function u → u^{k}K(u) is Lipschitz .

Some comments on our assumptions are worth noting. Assumption (A1) is mainly needed to apply the mean value theorem. The compactness stipulation in (A2) is used to derive asymptotic uniform bounds. Assumptions (A3)-(A7) are largely used in the theory of kernel regression with dependent data. Those assumptions can be found, for example, in Masry (1996). The stipulations about the bandwidth and the mixing coefficient in (A6) are just a simple, i.e., stronger, version of the necessary assumptions given by Conditions (7d), (4.5) and (4.7) in Masry (1996).

Our first result is formulated in the following Lemma.

\underline{Lemma} 1 Under assumptions (A), if 𝔼[φ^{2}(X)] < ∞ and 𝔼(|ϵ|φ^{2}(X)) < ∞ then

T_{n}(θ̂) = T_{n}(θ^{∗}) − 2B^{T} (θ̂ − θ^{∗}) + o_{p}(||θ̂ − θ^{∗}||),

where B = 𝔼 [ṁ(θ^{∗}, X)Y (θ^{∗})φ^{2}(X)] and ϵ = Y − m(X).

This Lemma states that, asymptotically, the only impact of using the estimator θ̂ instead of θ^{∗} is to shift T_{n} by the term 2B^{T} (θ̂ − θ^{∗}). This quantity vanishes whenever m ∈ M since in that case B = 0. Otherwise B can be easily estimated by its empirical version B̂ = n^{−1} ∑_{i=1}^{n} ṁ(θ̂, X)Y (θ̂)φ^{2}(X).

The next Lemma gives a Bahadur-type representation of the estimator T_{n}(θ) .

\underline{Lemma} 2 Under assumptions (A2)-(A7), if (i) \frac{ln n}{n^{1/2}h^{d}} = o(1), (ii) n^{1/2}h^{2(p+1)} = o(1), (iii) 𝔼|ϵ|^{ν} < ∞, 𝔼|φ^{2}(X)|^{ν} < ∞ and 𝔼|ϵφ^{2}(X)|^{ν} < ∞, and (iv) for any t > 1, 𝔼|ϵ_{1}ϵ_{t}φ^{2}(X_{t})|^{ν} < ∞ and 𝔼|ϵ_{1}ϵ_{t}φ^{2}(X_{1})|^{ν} < ∞, then for any θ ∈ Θ,

T_{n}(θ) = n^{−1} ∑_{i=1}^{n} [2Y_{i}(θ)Δ(θ, X_{i}) − Δ^{2}(θ, X_{i})] φ^{2}(X_{i}) + o_{p}(n^{−1/2}).

This is a very simple asymptotic representation of T_{n}(θ) as a sum of weakly dependent random variables whose mean is exactly T (θ). The simplicity of this representation comes from the fact that it is free from the bandwidth parameter h and the fact that it depends only on Y (θ), Δ(θ, x) and on the known function φ.

\underline{Remark} 1

• Assumptions (i) and (ii) imply that n^{1/2}h^{d} → ∞ and h^{2(p+1)−d} → 0. For this conditions to hold, we need that p > d/2 − 1. In other words, to guarantee the optimal root-n convergence rate, the order of the local polynomial approximation should increase as the dimension d of the covariates X increases. This result appears to be new in the kernel smoothing literature.

• All the bandwidth restrictions given in (A6), (i) and (ii) are fulfilled whenever the assumption (i’) given bellow is satisfied: (i’) δ ≥ 4, p > d/2 − 1 and h_{n} ∼ (n^{−1} ln n)^{a} for some \frac{1}{4(p+1)} < a < \frac{2}{d} .

• From the proofs given in the Appendix, it easy to see that Lemma 1 remains valid if instead of the statistic T_{n} we use T_{n}^{0}. However this is not the case when we consider Lemma 2. In fact, without adding extra assumptions, one can only state that,

T_{n}^{0}(θ^{∗}) = n^{−1} ∑_{i=1}^{n} Δ^{2}(θ^{∗}, X_{i})φ^{2}(X_{i}) + sup_{x∈D} |Δ(θ^{∗}, x)| {O_{p}((ln n/(nh^{d}))^{1/2}) + O_{p}(h^{p+1})} .

From this expression it is clear that in order to achieve a higher rate of convergence for T_{n}^{0}, one needs to impose some restrictions on Δ(θ^{∗}, x), such as for example Δ(θ^{∗}, x) = c_{n}Δ_{n}(x), for certain sequences c_{n} → 0 and a bounded function Δ_{n}(x).

Of course, all the results discussed above are only available under our restrictions on the bandwidth parameter as formulated in the assumptions given in Lemma 2. Other bandwidth choices may also be considered but this will inevitably affect the limiting distribution of the estimator. It is also important to note that no restriction was made on the parametric estimation procedure and so one can use any available parametric method. Here, for its simplicity and desirable properties, we suggest to use the least squares technique. Thus we propose to estimate θ by

θ̂ = arg min_{θ∈Θ} n^{−1} ∑_{i=1}^{n} Y_{i}^{2}(θ)φ^{2}(X_{i}).  (4)

In this definition we used the weighted version of the least squares estimator, since, as motivated in Section 4.2, we are interested in assessing the quality of the parametric model m(θ, x) within the support of φ(x). From Corollary 3.1 in Domowitz and White (1982) we claim that, under Assumptions (A), θ̂ converges with probability 1 to

θ^{∗} = arg min_{θ∈Θ} T (θ).  (5)

Interestingly, in this particular case, the parameter of interest T (θ^{∗}) coincides with min_{θ∈Θ} T (θ). In other words, our parameter of interest becomes exactly the minimum L^{2}-distance between m and the parametric family M. Moreover, since θ^{∗} minimize T (θ) in the interior of Θ, Assumption (A1) implies that,

−2B^{T} = 𝔼 [−2ṁ^{T} (θ^{∗}, X)Δ(θ^{∗}, X)φ^{2}(X)]

= 𝔼 [\frac{∂Δ^{2}(θ^{∗}, X)}{∂θ}φ^{2}(X)] = \frac{dT (θ^{∗})}{dθ} = 0,

so no bias estimation or correction will be needed. This result, together with Lemma 1 and Lemma 2, leads to the following theorem.

\underline{Theorem} 1 Under Assumptions (A), if the conditions (i)-(iv) given in Lemma 2 are satisfied, then

T_{n}(θ̂) = n^{−1} ∑_{i=1}^{n} [2Y_{i}(θ^{∗})Δ(θ^{∗}, X_{i}) − Δ^{2}(θ^{∗}, X_{i})] φ^{2}(X_{i}) + o_{p}(n^{−1/2}),

where θ̂ and θ^{∗} are given by (4) and (5), respectively. In the next section we demonstrate some applications of this result.

4 Closeness of Parametric Approximation and Explanatory Power of a Covariate

4.1 An inadequacy index

Following an original idea of Doksum and Samarov (1995), we introduce a measure of model deficiency based on the L^{2} loss function. The idea is in the spirit of the wellknown Pearson’s correlation ratio η^{2} = \frac{𝕍ar[m(X)]}{𝕍ar(Y)} = 1 − \frac{𝕍ar(ϵ)}{𝕍ar(Y)}. This coefficient gives the fraction of variability of Y explained by X through the true mean regression function m(x). In other words this coefficient measures the ability of the covariates, in a correctly specified model, to distinguish differing outcomes. It is a direct consequence of the ANOVA decomposition 𝕍ar(Y) = 𝕍ar(m(X)) + 𝕍ar(ϵ). This equality is just a special case of a more general one given by 𝔼(Y − g(X))^{2} = 𝔼(m(X) − g(X))^{2} + 𝔼(ϵ^{2}), where g is any real-valued function with 𝔼(g^{2}(X)) < ∞. Now, letting g(x) = m(θ, x) we get 𝔼(Y − m(θ, X))^{2} = 𝔼(m(X) − m(θ, X))^{2} + 𝔼(ϵ^{2}). This is a decomposition of the parametric residual variation into unexplained variation due to model misspecification and error variation. Therefore, the coefficient

ζ^{2}(θ) = \frac{𝔼(m(X) − m(θ, X))^{2}}{𝔼(Y − m(θ, X))^{2}} = 1 − \frac{𝔼(Y − m(X))^{2}}{𝔼(Y − m(θ, X))^{2}}

is the fraction of the parametric residual variation that can be completely attributed to the lack of fit in the parametric function m(θ, x) which will be described shortly as the missed fraction of variation or the inadequacy index. In particular, the index ζ^{2} := ζ^{2}(θ^{∗}) is the missed fraction of variation after adjusting the best possible parametric model from the family M = {m(θ, x), θ ∈ Θ }. It is interesting to note that η^{2} and ζ^{2} behave in an opposite way in the sense that while a large value of η^{2} indicates a good explanatory power of the covariates, the smallest the value of ζ^{2} the best the model is. Both of these coefficients range from 0 to 1. However, if M includes constants (which should always be the case) then ζ^{2} ≤ η^{2}, with equality if m(θ^{∗}, X) = 𝔼(Y). This is the case when the parametric model fails to capture any variability in the data.

4.2 An estimator of ζ^{2} and its asymptotic variance

For the same reasons as previously mentioned we prefer to work with a weighted version of ζ^{2}, which corresponds to T (θ^{∗})/S^{2}(θ^{∗}), where S^{2}(θ) = 𝔼 [(Y − m(θ, X))^{2}φ^{2}(X)]. Hereafter T (θ^{∗})/S^{2}(θ^{∗}) will be denoted by ζ^{2} ≡ ζ^{2}(θ^{∗}). An obvious estimator of this index is provided by ζ̂^{2} := T_{n}(θ̂)/S_{n}^{2}(θ̂), where T_{n}(θ) is given by (3), θ̂ is given by (4) and S_{n}^{2}(θ) = n^{−1} ∑_{i=1}^{n}(Y_{i} − m(θ, X_{i}))^{2}φ^{2}(X_{i}). Based on the result of Theorem 1, the next theorem gives a very useful asymptotic expression for ζ̂^{2}.

\underline{Theorem} 2 Under Assumptions (A), if the conditions (i)-(iv) given in Lemma 2 are satisfied, then

ζ̂^{2} − ζ^{2} = n^{−1} ∑_{i=1}^{n} ξ_{i} + o_{p}(n^{−1/2}),

where ξ_{i} is a shortcut for ξ_{i}(θ^{∗}), ξ_{i}(θ) = [(1 − ζ^{2})Y^{2}(θ) − ϵ_{i}^{2}] φ^{2}(X_{i})/S^{2}(θ) with Y_{i}(θ) = Y_{i} − m(θ, X_{i}), and ϵ_{i} = Y_{i} − m(X_{i}). A direct consequence of this theorem is the asymptotic normality of ζ̂^{2}. In fact, applying the central limit theorem to the strong mixing sequence {ξ_{t}}, see for example Theorem 2.21 in Fan and Yao (2003), we have that under the assumptions of Lemma 2:

\sqrt{n}(ζ̂^{2} − ζ^{2}) ⟶^{d} N (0, σ^{2}),

where the asymptotic variance σ^{2} ≡ σ^{2}(θ^{∗}), with σ^{2}(θ) := lim_{n→∞} n^{−1}𝕍ar (∑_{t=1}^{n} ξ_{t}(θ)) = 𝕍ar[ξ_{1}(θ)] + 2 ∑_{t>1} ℂov(ξ_{1}(θ), ξ_{t}(θ)).

To use this property in practice we need a consistent estimator for the asymptotic variance σ^{2}. In the case of i.i.d. data this can be done by using the classical sample variance estimator.

In the presence of correlated data, we adopt here the moving block bootstrap (MBB) procedure as proposed by Künsch (1989) and Liu and Singh (1992). This approach allows us to estimate σ^{2} without making any parametric model restriction and without resort to any Monte Carlo simulation. A detailed description of this method and its merits over other competing methods can be found in the book by Lahiri (2003). To fix ideas, we start by splitting the “data” {ξ_{i}}_{1≤i≤n} into N := n−l+1 blocks B_{i} = {ξ_{i}, …, ξ_{i+l−1}}, i = 1, …, N, of length l ≡ l_{n} ∈ [1, n]. We require that l → 0 and l = o(n^{−1}). Let U_{i} = l^{−1} ∑_{j=i}^{i+l−1} ξ_{j} be the sample mean of the i-th block and Ū the sample mean of {U_{1}, …, U_{N}}. Like in the i.i.d. case (l = 1), the MBB estimator of σ^{2} is σ̂^{2} = lN^{−1} ∑_{i=1}^{N} (U_{i} − Ū)^{2} . By Theorem 3.1 in Lahiri (2003), one can easily check that under the assumption of Lemma 2, σ̂^{2} converges in probability to σ^{2}. However, this estimator depends on the unknown parameter θ^{∗}. To overcome this problem, we simply suggest to plugging-in θ̂ into the definition of σ̂^{2}(θ) to get σ̂^{2}(θ̂) as our feasible estimator of the asymptotic variance. Following similar techniques as those used in the Appendix, it can be shown that σ̂^{2}(θ̂) − σ̂^{2}(θ^{∗}) = o_{p}(1) and so that σ̂^{2}(θ̂) is consistent.

4.3 Validation of a parametric model

A first and direct application of the previous results is that one can construct an asymptotically valid Wald-type confidence interval for ζ that is given by ζ̂ ± \frac{σ̂}{\sqrt{n}}z_{1−α/2}, where z_{α} is the αquantile of the standard normal distribution and σ̂^{2} = σ̂^{2}(θ̂). Although this confidence interval gives us valuable information about the quality of the parametric model m(θ, x), we still need a formal approach to test the goodness-of-fit hypothesis

H_{0} : m ∈ M versus H_{1} : m ∉ M.

In term of ζ, this hypothesis can be formulated as

H_{0} : ζ^{2} = 0 versus H_{1} : ζ^{2} > 0.  (6)

Unfortunately, ζ̂^{2} cannot be directly used as a test statistic for (6) since under the null hypothesis ξ vanishes and so does σ^{2}. The asymptotics in such a form have been noted before by many authors; see for example Fan and Li (1996). This degeneracy can be handled by considering higher-order terms in the expansion of T_{n}(θ̂). In fact, under H_{0}, it can be shown that T_{n}(θ̂) = Jn,1^{′′} + o_{p}(n^{−1}h^{−d/2}), where Jn,1^{′′} is a degenerate U-statistics as defined by (21) in the Appendix. This remark can be used to prove the asymptotic normality of nh^{d/2}ζ̂^{2} under correct specification and so to get a valid test statistic for the hypothesis (6). Such an approach will lead inevitably to the curse of dimensionality as the convergence rate decreases with d. Here instead of (6) we propose to test the following hypothesis

H_{π,0} : ζ^{2} ≥ π versus H_{π,1} : ζ^{2} < π,  (7)

where π ∈ (0, 1) is a small constant that can be considered by the analyst as a tolerable missed fraction of variation. In the literature, (7) is known as a neighborhood hypothesis or “precise” hypothesis; see Hodges and Lehmann (1954). The drawbacks of testing the classical hypothesis (6) with the hypothesis (7) were largely documented by many authors; see for example Dette and Munk (1998) and the references given therein. To cite just an argument in favor of the concept of neighborhood testing, observe that (7) is designed to provide evidence in favor of the tested model m(θ, x) while the latter cannot be confirmed even if the p-value associated with (6) is large. For a detailed discussion of many other aspects related with neighborhood hypothesis we refer to Dette and Munk (2003).

As noted by those authors, the main difficulty with neighborhood testing is the need of the asymptotic distribution of the test statistic not only under the assumption that m ∈ M, as is classically done in the literature of goodness-of-fit testing, but at any point in the model space M. The method adopted in this work consists in studying the estimated distance T_{n}(θ̂) without restrictions on the model specification and allows us to easily overcome this difficulty.

In fact, by Theorem 2, we directly conclude that a consistent critical region for H_{π,0} is provided by

ζ̂^{2} < π + z_{α} \frac{σ̂}{\sqrt{n}}.  (8)

Another difficulty usually associated with this procedure is the selection of π. In our case, this is facilitated by the fact that the coefficient ζ^{2} is a proportion bounded above by 1 and hence π should be as well. One can also get around this difficulty by reformulating the problem of testing (7) in terms of interval estimation. In fact, an asymptotic 100 × (1 − α)% upper confidence interval for ζ is given by [0, ζ_{n,+}^{2}], with

ζ_{n,+}^{2} = ζ̂^{2} + \frac{σ̂}{\sqrt{n}}z_{1−α}.

Of course, the rule given by (8) is equivalent to ζ_{n,+}^{2} ≤ π. In any case, given a sufficiently large data set, since P (ζ^{2} ≤ ζ_{n,+}^{2}) → 1 − α, one can state, at risk α × 100%, that the missed fraction of variation does not exceed ζ_{n,+}^{2}. According to the value of the latter, the tested model can be judged as admissible or not.

\underline{Remark} 2 If one is interested in the explanatory power of a random covariate X then, instead of ζ^{2}, η^{2} should be used. An estimator of the weighted version of the latter, i.e., 𝕍ar(m(X)φ(X))/𝕍ar(Y φ(X)), is given by η̂^{2} = T_{n} − Ȳ^{2}/S_{n}^{2}, where Ȳ = n^{−1} ∑_{i} Y_{i}φ(X_{i}), S_{n}^{2} = n^{−1} ∑_{i=1}^{n}(Y_{i}φ(X_{i}) − Ȳ)^{2}, and T_{n} = n^{−1} ∑_{i=1}^{n} (2Y_{i}m̂(X_{i}) − m̂^{2}(X_{i})) φ^{2}(X_{i}), with m̂(x) being the local polynomial estimator of m(x) of order p. Then T_{n} can be seen as a special case of T_{n}(θ) with m(θ, X) ≡ 0. Therefore, as a direct consequence of Lemma 2, it follows that

T_{n} = n^{−1} ∑_{i}(2Y_{i}m(X_{i}) − m^{2}(X_{i}))φ(X_{i}) + o_{p}(n^{−1/2}).

From this result and following a similar approach as in the proof of Theorem 2, it can be shown that, under the assumptions of Lemma 2,

η̂^{2} − η^{2} = σ^{−2}n^{−1} ∑_{i=1}^{n} [(1 − η^{2})(Y_{i}φ(X_{i}) − μ)^{2} − ϵ_{i}^{2}φ^{2}(X_{i})] + o_{p}(n^{−1/2}),

where μ = 𝔼(Y φ(X)) and σ^{2} = 𝕍ar(Y φ(X)). As we have done for ζ^{2}, this asymptotic expression can be used to provide consistent confidence intervals and tests procedures for η^{2}.

5 Monte Carlo Simulations

In this section we report the results of an extensive simulation study that was designed to evaluate the finite sample performance of η̂^{2} and ζ̂^{2} and their asymptotic properties as stated in the previous sections. The simulation considers univariate and multivariate cases with both i.i.d. data and weak dependent data using the weight function φ(t) = I(0 ≤ t ≤ 1) and N := 2000 replications. We generate n := 200 data according to the following model

Y_{t} = m_{1}(X_{t}) + λm_{2}(X_{t}) + τϵ_{t},

where X_{t} ∼ Unif[−ε, 1+ε] and ϵ_{t} ∼ N (0, 1). Here ε was chosen so that P (0 ≤ X_{t} ≤ 1) = 0.95. For d = 1, m_{1} and m_{2} are given by

m_{1}(x) = 6 + 2x, m_{2}(x) = sin(\sqrt{(3πx + π)^{2}}).

We are interested in measuring and testing the quality of the covariate X and of the linear parametric model m(θ, x) = θ_{0} + θ_{1}x. To this end we vary the values of λ and τ. The linear model m(θ, x) is correct only when λ = 0. In this case ζ^{2} = 0, but as λ increases, m(θ, x) becomes more and more inadequate and ζ^{2} ↗ 1. On the other hand, by increasing τ, X becomes more and more irrelevant and η^{2} ↘ 0. Table 1 shows the values of τ and λ used to generate the data together with the corresponding values of η^{2} and ζ^{2}. In the two-dimensional case, we choose

m_{1}(x) = 6 + 2x_{1} + 2x_{2}, m_{2}(x) = sin(\sqrt{(3πx_{1} + π)^{2} + (3πx_{2} + π)^{2}}),

and for d = 3, m_{1}(x) = 6 + 2x_{1} + 2x_{2} + 2x_{3}, m_{2}(x) = sin(\sqrt{(3πx_{1} + π)^{2} + (3πx_{2} + π)^{2} + (3πx_{3} + π)^{2}}).

The covariates are independent of each other, independent of the error variable ϵ_{t} and are Unif[−ε, 1 + ε].

Table 1: The true values of η^{2} and ζ^{2} in %

η^{2}  ζ^{2}
λ  0  0.8 1.5 2.5  5  0  0.8 1.5 2.5  5
τ  d=1  d=1
0.5 95.44 95.48 95.88 96.72 98.39  0.00 53.82 80.38 91.93 97.85
1 83.95 84.09 85.33 88.04 93.84  0.00 22.56 50.60 74.00 91.93
2 56.66 56.93 59.26 64.79 79.21  0.00 6.79 20.38 41.56 74.00
τ d=2 d=2
0.5 96.58 96.78 97.11 97.68 98.77  0.00 55.28 81.30 92.36 97.98
1 87.59 88.24 89.36 91.31 95.25  0.00 23.60 52.07 75.12 92.36
2 63.83 65.22 67.73 72.42 83.36  0.00 7.17 21.36 43.00 75.12
τ d=3 d=3
0.5 97.32 97.38 97.56 97.94 98.82  0.00 56.01 81.74 92.56 98.03
1 90.09 90.29 90.92 92.25 95.46  0.00 24.14 52.81 75.67 92.56
2 69.45 69.93 71.47 74.84 84.01  0.00 7.37 21.86 43.73 75.67

To calculate each estimator we use the local linear smoother with the Epanechnikov kernel function. As a data-driven bandwidth selection criterion, we use the least squares crossvalidation method; see Xia and Li (2002) and also Li and Racine (2004). In the multivariate cases, we use the product kernel and let all components of each bandwidth vector to be equal.

Our first objective is to perform a comparison between two estimators of ζ^{2}: ζ̂^{2} = T_{n}(θ̂)/S_{n}^{2}(θ̂) and ζ̂_{0}^{2} = T_{n}^{0}(θ̂)/S_{n}^{2}(θ̂) and two estimators of η^{2}: η̂^{2} = T_{n}/S_{n}^{2} and η̂_{0}^{2} = T_{n}^{0}/S_{n}^{2}, where T_{n}^{0} = n^{−1} ∑_{i=1}^{n} m̂^{2}(X_{i})φ^{2}(X_{i}). In Table 2 we report the root mean squared error (RMSE) for all those estimators, for d = 1, 2, 3, τ = 0.5, 1, 2 and λ = 0, 0.8, 1.5, 2.5, 5. From this table we observe that both η̂^{2} and η̂_{0}^{2} perform very well with respect to the MSE criterion, with a clear advantage of η̂^{2} over η̂_{0}^{2}. This is also the case for ζ^{2}. In fact, we obtain almost systematically a smaller MSE when we use our estimators T_{n}(θ̂) instead of T_{n}^{0}(θ̂). The only exception happened with a very small value of λ (ζ^{2} → 0) where ζ̂_{0}^{2} provided a slightly better result. The performances of all these estimators are clearly affected by the true values of the corresponding parameters and by the dimensionality d, along with interaction between this two factors. For example, when d = 1 or 2, as ζ increases the MSE of ζ̂^{2} initially increases and then rapidly decreases. Also ζ̂_{0}^{2} behaves similarly but its MSE increases rapidly and then decreases slowly. For d = 3, we can see that ζ̂_{0}^{2} behaves badly as ζ^{2} increases and approaches 1. As expected, increasing the covariate dimensionality d causes the MSE to increase but η̂_{0}^{2} and ζ̂_{0}^{2} are clearly more sensitive to the curse of dimensionality. For example, for τ = 1 and λ = 1.5, when d moves from 1 to 3, the MSE of ζ̂_{0}^{2} increases by a factor of 6.8 whereas the MSE of ζ̂^{2} increases by only a factor of 1.8. This becomes even more striking when we consider the optimal bandwidths (the results are not shown here). To give just an example, under the same scenario as above and using the optimal bandwidth for both estimators, the MSE of ζ̂_{0}^{2} increases by a factor of 12.3 whereas the MSE of ζ̂^{2} increases by only a factor of 0.8. This definitely demonstrates the advantages of the proposed estimators over the “classical” ones.

Table 3 gives the optimal root mean squared error (RMSE^{∗}) obtained using the optimal bandwidth (the one that minimize the MSE). For easy comparison, the table shows the RMSE obtained using the data-driven bandwidth. We also report the bias and the standard deviation (S.D.) for both η̂^{2} and ζ̂^{2} when d = 2. The other results are not displayed here for the sake of brevity. Both the absolute value of the bias and the variance increase with d. For d = 3, we have also noticed that both η̂^{2} and ζ̂^{2} become more biased as λ increases. Globally, the variance is the main component of the mean squared error. Typically, its contribution decreases with λ and increases with τ and it is more sensitive to the alteration in τ. Regarding the usefulness of the bandwidth selection procedure, we have observed that, in general, the resulting values for the two estimators η̂^{2} and ζ̂^{2} obtained using the data driven bandwidth were quite close to the optimal ones obtained using the optimal bandwidth. However, the dimensionality has again a negative impact. The observed averaged value of the difference in mean squared error |MSE − MSE^{∗}| was 0.004 for η̂^{2} and 0.021 for ζ̂^{2} with maximum value equal to 0.015 and 0.062 for η̂^{2} and ζ̂^{2}, respectively. We have also observed that the loss of efficiency due to the estimated bandwidth is larger for η̂_{0}^{2} and ζ̂_{0}^{2}. In fact, for these estimators the average (maximum) of |MSE − MSE^{∗}| was 0.019 (0.074) and 0.048 (0.137), respectively. This indicates a greater robustness of η̂^{2} and ζ̂^{2} to bandwidth misspecification.

Another objective of this simulation study is to verify the validity of the proposed testing procedures. As the estimation of the asymptotic variance plays a crucial role, we start by checking the finite sample performance of our variance estimator of ζ̂^{2} as introduced in Section 4.2 . To estimate the asymptotic variance of η̂^{2} we use exactly the same procedure. Table 3 illustrates the consistent nature of the these estimators via a small mean squared error. The behavior of the latter differs for the various cases but we observe that the MSE performance is better than expected even for d = 3. To complete the picture, Table 5 shows the coverage probability for the lower and upper confidence intervals (LCI and UCI) for η^{2} and ζ^{2}, respectively, of nominal level 95% computed using η̂^{2} and ζ̂^{2}. The accuracy of confidence limits was assessed by calculating the proportion of times the true value was above or below the confidence limit. For both estimators, the observed coverage probability was often different from the expected values especially for τ = 2. For τ = 0.5 or for λ = 0, our intervals appear overly conservative but as d increases they become anti-conservative. For d = 3 the results were unsatisfactory especially for ζ^{2} (the results are not shown). This is not really surprising given that we use the same bandwidth parameter that we used to estimate our parameters. As we have seen, this bandwidth is appropriate for MSE minimization but now we need a balance between narrow confidence interval and minimum coverage error. To illustrate the benefit of our method when the bandwidth is correctly specified, Table 6 gives the optimal coverage probability obtained using a fixed but optimal bandwidth (the one that minimize the coverage error) for d = 3. These results clearly demonstrate the usefulness and the good performance of the Normal approximation and the resulting confidence limits given a “good” bandwidth parameter. Theoretically, the optimal bandwidth parameter can be determined by studying how fast \sqrt{n}(ζ̂− ζ) converges to its limit using, for instance, Edgeworth expansions; see for example Hall (1992). Practically, bootstrap methods may be used to estimate the coverage error associated with a given bandwidth and so to approximate the optimal one, leading to further improvements of the confidence intervals. This is however beyond the scope of the present work but may be a topic of further research.

Finally, the entire simulation study was re-run using data with correlated errors generated according to an autoregressive AR(ρ) process of order 1 with different values of the autocorrelation parameter ρ. To be more precise, we generate ϵ_{t} according to the model ϵ_{t} = ρϵ_{t−1} + ω_{t}, with ω_{t} ∼ i.i.d. N (0, 1). To choose the block length needed for the asymptotic variance estimator (see Section 4.2), we use the block selection method of Patton et al. (2009) provided by the R package np of Hayfield and Racine (2008). The results for the dependent case were globally similar to those obtained with i.i.d. data and so we only provide here a brief summary given in Table 7 for the case ρ = 0.95 and d = 1. This table (and other results not shown here) clearly indicate that this dependency structure has almost no effect on our estimators and the proposed confidence intervals. Nevertheless, comparing the i.i.d. case and the dependent case is difficult here because changing ρ affects the variation in Y and so it also affects η^{2}, ζ^{2} and the variance of their corresponding estimators. For example, when d = 1, τ = 0.5 and λ = 0, η^{2} ≈ 0.67 and 𝕍ar(η̂^{2}) ≈ 0.55 for ρ = 0.95, while for ρ = 0 (i.i.d.) η^{2} ≈ 0.95 and 𝕍ar(η̂^{2}) ≈ 0.02.

Table 2: 100 × RMSE for η̂_{0}^{2}, η̂^{2}, ζ̂_{0}^{2} and ζ̂^{2}.

λ  λ
0  0.8  1.5  2.5  5  0  0.8  1.5  2.5  5
τ  d=1  d=1
0.5 η̂_{0} 3.702 3.057 2.302 1.331 1.825 ζ̂_{0} 1.044 9.286 8.043 6.192 3.897
η̂ 1.228 1.078 0.880 0.618 0.411 ζ̂ 1.682 5.146 2.574 1.131 0.320
1 η̂_{0} 6.862 5.749 4.619 2.824 2.637 ζ̂_{0} 1.044 7.255 9.226 8.615 6.187
η̂ 3.319 3.086 2.628 1.858 0.810 ζ̂ 1.682 6.295 5.486 3.322 1.131
2 η̂_{0} 9.688 8.872 7.552 5.433 3.904 ζ̂_{0} 1.044 4.392 6.953 8.990 8.603
η̂ 5.765 5.747 5.312 4.483 2.635 ζ̂ 1.682 4.085 6.342 6.051 3.322
τ d=2 d=2
0.5 η̂_{0} 3.507 3.966 5.999 8.450 11.083 ζ̂_{0} 1.921 17.825 20.987 19.361 14.968
η̂ 0.936 0.743 0.657 0.591 0.874 ζ̂ 2.217 5.849 2.710 1.069 1.552
1 η̂_{0} 6.650 6.352 7.594 10.436 14.313 ζ̂_{0} 1.873 9.962 17.225 20.657 19.334
η̂ 2.578 2.276 1.908 1.417 0.727 ζ̂ 2.127 7.788 6.207 3.541 1.074
2 η̂_{0} 10.206 9.969 9.476 11.070 16.310 ζ̂_{0} 1.936 4.660 9.839 15.186 20.649
η̂ 5.289 5.349 4.752 3.998 2.321 ζ̂ 2.235 5.937 7.822 7.035 3.537
τ d=3 d=3
0.5 η̂_{0} 3.150 6.270 10.475 16.454 56.755 ζ̂_{0} 2.294 25.516 35.952 38.987 40.211
η̂ 0.725 0.819 1.640 3.282 7.043 ζ̂ 3.248 7.165 6.897 8.187 9.923
1 η̂_{0} 5.910 8.864 11.893 17.537 56.415 ζ̂_{0} 2.350 15.263 24.139 33.452 39.586
η̂ 1.980 2.111 1.962 2.546 6.092 ζ̂ 3.356 9.988 7.455 6.524 8.188
2 η̂_{0} 9.452 10.900 13.533 20.256 29.625 ζ̂_{0} 2.308 5.709 14.164 20.291 33.186
η̂ 4.637 5.115 5.251 4.548 5.064 ζ̂ 3.265 5.000 10.243 8.413 6.039

Table 3: 100 × RMSE^{∗}, 100 × RMSE, 100 × Bias and 100 × S.D. for η̂^{2} and ζ̂^{2} with d = 2

η̂^{2}  ζ̂^{2}
τ  λ  0  0.8  1.5  2.5  5  0  0.8 1.5 2.5  5
0.5 RMSE^{∗} 0.662 0.596 0.602 0.926 1.927  0.000 4.069 2.145 1.369 2.398
RMSE 0.936 0.743 0.657 0.591 0.874  3.217 5.849 2.710 1.069 1.552
Bias -0.513 -0.282 -0.337 -0.418 -0.669  1.435 2.754 1.231 0.283 -1.103
S.D. 0.783 0.688 0.564 0.418 0.563  2.879 5.160 2.414 1.031 1.092
1 RMSE^{∗} 2.349 2.127 1.799 1.346 1.250  0.000 4.356 4.154 2.537 1.369
RMSE 2.578 2.276 1.908 1.417 0.727  3.127 7.788 6.207 3.541 1.074
Bias -0.514 -0.098 0.101 0.098 -0.268  1.401 3.298 2.904 1.655 0.290
S.D. 2.526 2.274 1.905 1.414 0.676  2.796 7.056 5.486 3.131 1.034
2 RMSE^{∗} 5.116 4.794 4.267 3.411 1.825  0.000 2.590 4.274 4.533 2.537
RMSE 5.289 5.349 4.752 3.998 2.321  3.235 6.937 7.822 7.035 3.537
Bias 0.010 -0.087 0.960 1.300 0.822  1.429 0.902 3.176 3.242 1.656
S.D. 5.289 5.348 4.654 3.780 2.171  2.902 6.878 7.148 6.244 3.125

Table 4: 100 × RMSE for the estimated asymptotic variance of η̂^{2} and ζ̂^{2}.

η̂^{2}  ζ̂^{2}
τ|λ  0  0.8  1.5  2.5  5  0  0.8  1.5  2.5  5
d = 1 0.5  1.940 1.364 0.691 0.211 0.028  3.856 12.327 3.307 0.646 0.049
1  11.328 8.623 5.044 2.008 0.381  3.856 18.337 13.444 5.336 0.646
2  14.488 13.073 10.533 7.826 3.462  3.856 13.657 18.189 16.023 5.336
d = 2 0.5  0.988 0.618 0.353 0.138 16.652  7.119 19.859 4.214 0.652 22.416
1  7.115 5.061 3.014 1.325 0.239  6.970 28.357 21.812 7.320 0.653
2  15.214 14.212 11.016 7.873 3.084  7.105 16.824 28.711 27.034 7.315
d = 3 0.5  0.553 0.563 13.454 9.476 15.720  8.429 18.980 3.059 2.182 4.886
1  4.390 4.034 3.068 2.677 14.501  8.471 35.351 21.536 4.772 2.221
2  13.721 14.352 12.178 7.598 2.679  8.486 20.825 35.183 27.299 5.315

Table 5: Lower and upper confidence intervals (LCI and UCI) for η^{2} and ζ^{2}, respectively, using data driven bandwidth. Nominal coverage = 95%.

LCI for η^{2}  UCI for ζ^{2}
τ|λ  0  0.8 1.5 2.5  5  0  0.8 1.5 2.5  5
d = 1 0.5  97.71 96.24 96.27 97.20 99.93  98.55 96.76 98.47 98.87 99.20
1  93.62 90.33 90.13 90.13 90.67  100.00 93.71 96.80 98.00 98.87
2  93.14 89.67 87.93 86.33 84.47  98.90 83.90 93.40 96.33 98.00
d = 2 0.5  98.60 97.20 97.53 99.53 100.00  98.52 94.80 95.67 94.60 80.33
1  93.47 89.73 87.67 87.87 95.27  98.84 90.47 94.87 95.47 94.40
2  91.87 89.87 85.60 83.33 82.31  98.79 82.67 90.00 93.87 95.47

Table 6: Lower and upper confidence intervals (LCI and UCI) for η^{2} and ζ^{2}, respectively using optimal bandwidth. Nominal coverage = 95%.

LCI for η^{2}  UCI for ζ^{2}
τ|λ  0  0.8 1.5 2.5  5  0  0.8 1.5 2.5  5
0.5  95.10 95.10 96.00 98.89 99.90  100.00 98.20 99.60 97.80 100.00
1  94.60 94.40 94.40 98.80 93.89  100.00 96.20 93.30 99.10 97.80
2  92.50 95.00 95.00 96.20 92.60  100.00 96.20 93.60 93.70 99.10

Table 7: 100 × RMSE and lower and upper confidence intervals (LCI and UCI) for η^{2} and ζ^{2}, respectively, using data driven bandwidth. Nominal coverage = 95%.

η̂^{2}  ζ̂^{2}
τ|λ  0  0.8 1.5 2.5  5  0  0.8 1.5 2.5  5
0.5 RMSE 1.23 1.08 0.88 0.62 0.41 RMSE 1.68 5.14 2.57 1.13 0.32
LCI 97.71 96.23 96.26 97.20 99.93 UCI 100.00 96.76 98.46 98.86 99.20
1 RMSE 3.32 3.88 2.63 1.86 0.81 RMSE 1.69 2.52 5.48 3.32 1.13
LCI 93.62 93.81 90.13 90.13 90.67 UCI 100.00 95.00 96.80 98.00 98.87

6 Appendix

This appendix collects proofs of the main results stated in the previous sections. Throughout, when we evaluate the order of some terms, the symbol C denotes a generic constant. For a given u = 0, …, p, let N_{u} := \frac{(u+d−1)!}{(d−1)!u!} be the number of distinct d−tuples k with |k| = u. Arrange the N_{u} elements of {k, |k| = u} in a lexicographical descending order. Let l_{u} denote this one-to-one mapping, i.e., l_{u}(1) = (0, …, 0, u), …, l_{u}(N_{u}) = (u, 0, …, 0). Now for a given x, define γ_{u,h}(x) to be the N_{u}×1 vector of the lexicographical arrangement of {(x/h)^{k}, |k| = u}, i.e., γ_{u,h}(x) = ((x/h)^{l_{u}(1)}, …, (x/h)^{l_{u}(N_{u})})^{T}. Put γ_{h}(X_{j} −x) = (γ_{0,h}^{T}(X_{j} −x), …, γ_{p,h}^{T}(X_{j} −x))^{T}. This is a column vector of dimension N := ∑_{u=0}^{p} N_{u}. Let X_{n,u}(x) be the n × N_{u} matrix [γ_{u,h}(X_{1} − x) … γ_{u,h}(X_{n} − x)]^{T}, X_{n}(x) ≡ X be the N × N matrix [X_{n,0}(x) … X_{n,p}(x)], W be the n × n diagonal matrix with a diagonal given by {n^{−1}K_{h}(X_{j} − x), j = 1, …, n}, Y be the n × 1 vector (Y_{1}, …, Y_{n})^{T}, m(X, θ) be the the n × 1 vector (m(X_{1}, θ), …, m(X_{n}, θ))^{T}, and Y(θ) = Y − m(X, θ).

By definition, see e.g. Masry (1996), the local multivariate polynomial estimator of Δ(θ, x) is the first element of the N × 1 vector S_{n}^{−1}(x)X^{T} WY(θ), with S_{n}(x) = X^{T} WX. Thus, Δ̂(θ, x) = e_{N,1}^{T}S_{n}^{−1}(x)X^{T} WY(θ) = ∑_{j=1}^{n} w_{j}(x)Y_{j}(θ), with

w_{j}(x) = e_{N,1}^{T}S_{n}^{−1}(x)X^{T} We_{n,j}  (9)
= n^{−1}e_{N,1}^{T}S_{n}^{−1}(x)γ_{h}(X_{j} − x)K_{h}(X_{j} − x)

where, for r = n, N and l = 1, … r, e_{r,l} is the r × 1 vector with the lth element being 1 and the rest of elements being zero. First note that, ∑_{j=1}^{n} w_{j}(x) = 1. Also, observe that

∑_{j=1}^{n} |w_{j}(x)| ≤ n^{−1}||e_{N,1}^{T}S_{n}^{−1}(x)|| ∑_{j=1}^{n} ||γ_{h}(X_{j} − x)||K_{h}(X_{j} − x).

It is easy to check that the elements of the matrix S_{n}(x) are s_{n,k}(x) = n^{−1} ∑_{j=1}^{n} (\frac{X_{j}−x}{h})^{k} K_{h}(X_{j}− x), 0 ≤ |k| ≤ 2p. On the other hand, by assumption (A7), ||γ_{h}(X_{j} − x)|| ≤ C. So, n^{−1} ∑_{j=1}^{n} ||γ_{h}(X_{j} − x)||K_{h}(X_{j} − x) ≤ Cs_{0,n}(x). Now, by Corollary 1 in Masry (1996), sup_{D} |s_{n,k}(x) − f(x)μ_{k}| = o_{p}(1), where μ_{k} = ∫ u^{k}K(u)du. So,

sup_{x∈D} ∑_{j=1}^{n} |w_{j}(x)| = O_{P} (1).  (10)

\underline{Remark} 3 X^{T} WY corresponds to the vector τ_{n}(x) as defined by equation (2.2) in Masry (1996). Also Δ(θ, x) = e_{N,1}^{T}S_{n}^{−1}(x)X^{T} W[Y − m(θ, X)] = m̂(x) − e_{N,1}^{T}S_{n}^{−1}(x)X^{T} Wm(θ, X) = m̂(x) − m̂(θ̂, x).

Proof of Lemma 1 By the mean value Theorem,

T_{n}(θ̂) − T_{n}(θ^{∗}) = (θ̂ − θ^{∗})^{T} Ṫ_{n}(θ̃_{n}),  (11)

where θ̃_{n} = θ^{∗} + η(θ̂ − θ^{∗}), for some η ∈ (0, 1), and

Ṫ_{n}(θ) = 2n^{−1} ∑_{i} [Ẏ_{i}(θ)Δ̂(θ, X_{i}) + (Y_{i}(θ) − Δ̂(θ, X_{i})) ˙Δ̂(θ, X_{i})] φ^{2}(X_{i}),  (12)

with ˙ Δ̂(θ, x) = − ∑_{j} w_{j}(x)ṁ(θ, X_{j}), and Ẏ_{i}(θ) = −ṁ(θ, X_{i}). From (11), it is clear that Lemma 1 is equivalent to say that Ṫ_{n}(θ̃_{n}) = −2B + o_{p}(1). Let I_{n} = n^{−1} ∑_{i=1}^{n}(Y_{i}(θ̃_{n})−Δ̂(θ̃_{n}, X_{i})) ˙Δ̂(θ̃_{n}, X_{i})φ^{2}(X_{i}). Then Ṫ_{n}(θ̃_{n}) = 2n^{−1} ∑_{i} Ẏ_{i}(θ̃_{n})Δ̂(θ̃_{n}, X_{i})φ^{2}(X_{i})+ 2I_{n}. First we will show that I_{n} = o_{p}(1). Observe that,

I_{n} = I_{n,1} + I_{2,n} + I_{3,n} − I_{4,n} − I_{5,n},

with I_{n,1} = n^{−1} ∑_{i} ϵ_{i} ˙Δ̂(θ^{∗}, X_{i})φ^{2}(X_{i}), I_{n,2} = n^{−1} ∑_{i} ϵ_{i}( ˙Δ̂(θ̃_{n}, X_{i}) − ˙ Δ̂(θ^{∗}, X_{i}))φ^{2}(X_{i}), I_{n,3} = n^{−1} ∑_{i}(Δ(θ̃_{n}, X_{i})−Δ(θ^{∗}, X_{i})) ˙Δ̂(θ̃_{n}, X_{i})φ^{2}(X_{i}), I_{n,4} = n^{−1} ∑_{i}(Δ̂(θ̃_{n}, X_{i})−Δ̂(θ^{∗}, X_{i})) ˙Δ̂(θ̃_{n}, X_{i})φ(^{2}X_{i}), and I_{n,5} = n^{−1} ∑_{i}(Δ̂(θ^{∗}, X_{i}) − Δ(θ^{∗}, X_{i})) ˙Δ̂(θ̃_{n}, X_{i})φ^{2}(X_{i}). For some given ϵ > 0, let K = {u : |u − x| ≤ ϵ and x ∈ D}. By choosing ϵ sufficiently small, K becomes a compact subset of S. For any x in D, any θ ∈ Θ and for n sufficiently large,

|| ˙Δ̂(θ, x)|| ≤ ∑_{j} |w_{j}(x)|||ṁ(θ, X_{j})||

≤ sup_{(θ,u)∈Θ×K} ||ṁ(θ, u)|| ∑_{j} |w_{j}(x)|.

So, by (10) and assumption (A1),

sup_{(θ,x)∈Θ×D} || ˙Δ̂(θ, x)|| = O_{p}(1).  (13)

By the mean value Theorem, for any x in D and for n sufficiently large, there exists a θ̃_{n,x} ∈ Θ such that |Δ(θ̃_{n}, x) − Δ(θ^{∗}, x)| = |m(θ̃_{n}, x) − m(θ^{∗}, x)| ≤ ||θ̂ − θ^{∗}||||ṁ(θ̃_{n,x}, x)||. So, by assumption (A1)

sup_{x∈D} |Δ(θ̃_{n}, x) − Δ(θ^{∗}, x)| = O_{p}(||θ̂ − θ^{∗}||).  (14)

Using (13) we can also check that

sup_{x∈D} |Δ̂(θ̃_{n}, x) − Δ̂(θ^{∗}, x)| = O_{p}(||θ̂ − θ^{∗}||).  (15)

Similarly one can prove that,

sup_{x∈D} || ˙Δ̂(θ̃_{n}, x) − ˙ Δ̂(θ^{∗}, x)|| = O_{p}(||θ̂ − θ^{∗}||).  (16)

From the definition of Δ̂(θ, x), see (1), we have that Δ̂(θ^{∗}, x) − Δ(θ^{∗}, x) = (m̂(x) − m(x)) − ∑_{j=1}^{n} w_{j}(x)(m(θ^{∗}, X_{j}) − m(θ^{∗}, x)). By Theorem 6 in Masry (1996), sup_{x∈D} |m̂(x) − m(x)| = o_{p}(1). On the other hand, | ∑_{j} w_{j}(x)(m(θ^{∗}, X_{j}) − m(θ^{∗}, x))| ≤ sup_{|u−x|≤h_{n}} ||m(θ^{∗}, u) − m(θ^{∗}, x)|| ∑_{j} |w_{j}(x)|. So using (10) and assumption (A1), we have that

sup_{x∈D} |Δ̂(θ^{∗}, x) − Δ(θ^{∗}, x)| = o_{p}(1).  (17)

Similarly,

sup_{x∈D} || ˙Δ̂(θ^{∗}, x) + ṁ(θ^{∗}, x)| = o_{p}(1).  (18)

Now we have all the ingredients needed to show that, I_{n,l} = o_{p}(1), for, l = 1, …, 5. In fact, using the LLN and (13), we get from (18), (16), (14), (15) and (17) that, respectively,

I_{n,1} = n^{−1} ∑_{i} ϵ_{i}( ˙Δ̂(θ^{∗}, X_{i}) + ṁ(θ^{∗}, X_{i}))φ^{2}(X_{i}) − n^{−1} ∑_{i} ϵ_{i}ṁ(θ^{∗}, X_{i})φ^{2}(X_{i})
= o_{p}(1) − o_{p}(1) = o_{p}(1),
|I_{n,2}| ≤ sup_{x∈D} || ˙Δ̂(θ̃_{n}, x) − ˙ Δ̂(θ^{∗}, x)||n^{−1} ∑_{i} |ϵ_{i}|φ^{2}(X_{i})
= O_{p}(||θ̂ − θ^{∗}||)O_{p}(1) = o_{p}(1),
|I_{n,3}| ≤ sup_{x∈D} |Δ(θ̃_{n}, x) − Δ(θ^{∗}, x)| sup_{(θ,x)∈Θ×D} || ˙Δ̂(θ, x)||n^{−1} ∑_{i} φ^{2}(X_{i})
= O_{p}(||θ̂ − θ^{∗}||)O_{p}(1)O_{p}(1) = o_{p}(1),
I_{n,4} = o_{p}(1), and I_{n,5} = o_{p}(1).

So, we have established that I_{n} = o_{p}(1), hence, by (12)

Ṫ_{n}(θ̃_{n}) = 2n^{−1} ∑_{i} Ẏ_{i}(θ̃_{n})Δ̂(θ̃_{n}, X_{i})φ^{2}(X_{i}) + o_{p}(1)

= −2n^{−1} ∑_{i} ṁ(θ̃_{n}, X_{i})Δ̂(θ̃_{n}, X_{i})φ^{2}(X_{i}) + o_{p}(1)

= −2n^{−1} ∑_{i} ṁ(θ^{∗}, X_{i})Δ(θ^{∗}, X_{i})φ^{2}(X_{i}) + o_{p}(1)  (19)

= −2B + o_{p}(1),

where in (19) we have used (15), sup_{x∈D} ||ṁ(θ̃_{n}, x) − ṁ(θ^{∗}, x)|| = O_{p}(||θ̃_{n} − θ^{∗})||) and the fact that both ṁ(θ^{∗}, x) and Δ(θ^{∗}, x) are bounded in D. This together with (11) concludes the proof of Lemma 1. □

Proof of Lemma 2 Substituting Δ̂(θ, x) in (3) by Δ̂(θ, x) − Δ(θ, x) + Δ(θ, x), we obtain

T_{n}(θ) =n^{−1} ∑_{i=1}^{n} [2Y_{i}(θ)Δ(θ, X_{i}) − Δ^{2}(θ, X_{i})] φ^{2}(X_{i})  (20)

+ 2n^{−1} ∑_{i=1}^{n} [Δ̂(θ, X_{i}) − Δ(θ, X_{i})] ϵ_{i}φ^{2}(X_{i})

− n^{−1} ∑_{i=1}^{n} [Δ̂(θ, X_{i}) − Δ(θ, X_{i})]^{2} φ^{2}(X_{i}).

By Theorem 6 in Masry (1996), sup_{x∈D} |m̂(x) − m(x)| = O_{p}((ln n/(nh^{d}))^{1/2}) + O_{p}(h^{p+1}) = o_{p}(n^{−1/4}). Similar arguments as those used in the proof of that result lead to the property sup_{x∈D} | ∑_{j=1}^{n} w_{j}(x)m(θ^{∗}, X_{j}) − m(θ^{∗}, x)| = O_{p}(h^{p+1}) = o_{p}(n^{−1/4}). So,

n^{−1} ∑_{i=1}^{n} [Δ̂(θ, X_{i}) − Δ(θ, X_{i})]^{2} φ^{2}(X_{i}) = o_{p}(n^{−1/2}).

It remains to show that J_{n}(θ) := n^{−1} ∑_{i=1}^{n} [Δ̂(θ, X_{i}) − Δ(θ, X_{i})] ϵ_{i}φ^{2}(X_{i}) = o_{p}(n^{−1/2}). To do so, we need the following Lemma.

\underline{Lemma} 3 Under the conditions of Lemma 2, for any θ ∈ Θ

Δ̂(θ, x) − Δ(θ, x) =d_{n,0}(x)n^{−1} ∑_{j=1}^{n} γ_{h}(X_{j} − x)K_{h}(X_{j} − x)ϵ_{j}
+ h^{p+1}d_{n,1}(θ, x) + d_{n,2}(θ, x) + r_{n}(θ, x),

where d_{n,0}(x) := e_{N,1}^{T}𝔼(S_{n}^{−1}(x)) is a deterministic 1×N vector that satisfies sup_{x∈D} ||d_{n,0}(x)|| = O(1), d_{n,1}(θ, x) and d_{n,2}(θ, x) are two deterministic scalars that satisfy sup_{x∈D} |d_{n,1}(θ, x)| = O(1) and sup_{x∈D} |d_{n,2}(θ, x)| = o(h^{p+1}). The remainder term r_{n} satisfies sup_{x∈D} |r_{n}(θ, x)| = O_{p}(ln n/(nh^{d})) + O_{p}(h^{p+1}(ln n/(nh^{d}))^{1/2}) = o_{p}(n^{1/2}). The proof of this Lemma is omitted since it follows directly from equation (2.13) in Masry (1996) through his Theorem 2, Theorem 4, Theorem 5, Proposition 1, Corollary 2 and Corollary 3.

From Lemma 3 we can decompose J_{n}(θ) as J_{n,1} + J_{n,2}(θ) + J_{n,3}(θ) + J_{n,4}(θ), with J_{n,1} = n^{−2} ∑_{i=1}^{n} ∑_{j=1}^{n} d_{n,0}(X_{i})γ_{h}(X_{j}−X_{i})K_{h}(X_{j}−X_{i})ϵ_{i}ϵ_{j}φ^{2}(X_{i}), J_{n,2}(θ) = h^{p+1}n^{−1} ∑_{i=1}^{n} d_{n,1}(θ, X_{i})ϵ_{i}φ^{2}(X_{i}), J_{n,3}(θ) = n^{−1} ∑_{i=1}^{n} d_{n,2}(θ, X_{i})ϵ_{i}φ^{2}(X_{i}) and J_{n,4}(θ) = n^{−1} ∑_{i=1}^{n} r_{n}(θ, X_{i})ϵ_{i}φ^{2}(X_{i}). Clearly

|J_{n,4}(θ)| ≤ sup_{x∈D}|r_{n}(θ, x)|n^{−1} ∑_{i=1}^{n} |ϵ_{i}|φ^{2}(X_{i})
= o_{p}(n^{−1/2})O_{p}(1) = o_{p}(n^{−1/2}).

Now consider J_{n,3}(θ). By Lemma 3 and Lemma 7 in Doukhan and Louhichi (1999), using assumption (A6)

𝔼[J_{n,3}^{2}(θ)] ≤ Cn^{−1}(𝔼|d_{n,2}(θ, X_{i})ϵ_{i}φ^{2}(X_{i})|^{ν})^{2/ν} ∑_{t≥1} α^{1−2/ν}(t)

≤ Cn^{−1}h^{2(p+1)}(𝔼|ϵ_{i}φ^{2}(X_{i})|^{ν})^{2/ν} ∑_{t≥1} α^{1−2/ν}(t)

≤ Cn^{−1}h^{2(p+1)}.

We conclude that J_{n,3}(θ) = O_{p}(n^{−1/2}h^{(p+1)}) = o_{p}(n^{−1/2}). Similarly, one can check that J_{n,2}(θ) = o_{p}(n^{−1/2}). To get the desired result, it remains to prove that J_{n,1} = o_{p}(n^{−1/2}). Let Jn,1^{′} = n^{−2} ∑_{i=1}^{n} d_{n,0}(X_{i})γ_{h}(0)K_{h}(0)ϵ_{i}^{2}φ^{2}(X_{i}). Observe that, for n sufficiently large,

|Jn,1^{′}| ≤ n^{−1}h^{−d}K(0)||γ_{h}(0)|| sup_{x∈D} ||d_{n,0}(X_{i})|| (n^{−1} ∑_{i=1}^{n} ϵ_{i}^{2}φ^{2}(X_{i}))

≤ Cn^{−1}h^{−d} (n^{−1} ∑_{i=1}^{n} ϵ_{i}^{2}φ^{2}(X_{i}))

= O_{p}(n^{−1}h^{−d}) = o_{p}(n^{−1/2}).

So, we have that J_{n,1} = Jn,1^{′′} + o_{p}(n^{−1/2}), with

Jn,1^{′′} := n^{−2} ∑_{i=1}^{n} ∑_{j≠i} d_{n,0}(X_{i})γ_{h}(X_{j} − X_{i})K_{h}(X_{j} − X_{i})ϵ_{i}ϵ_{j}φ^{2}(X_{i}).  (21)

Put ξ_{i} = (X_{i}, ϵ_{i})^{T} and η(ξ_{i}, ξ_{j}) = d_{n,0}(X_{i})γ_{h}(X_{j} − X_{i})K_{h}(X_{j} − X_{i})ϵ_{i}ϵ_{j}φ^{2}(X_{i}). Let h(ξ_{i}, ξ_{j}) = η(ξ_{i}, ξ_{j}) + η(ξ_{j}, ξ_{i}). With these notations, we can write Jn,1^{′′} as Jn,1^{′′} = ∑_{1≤i<j≤n} h(ξ_{i}, ξ_{j}).

Observe that h(ξ_{i}, ξ_{j}) is symmetric and 𝔼[h(ξ_{i}, v)] = 0, for any v. So by Lemma C.2(ii) in Gao and King (2006), using assumption (A6)

𝔼[(Jn,1^{′′})^{2}] =n^{−4}𝔼[( ∑_{1≤i<j≤n} h(ξ_{i}, ξ_{j}))^{2}] ≤ Cn^{−2}M_{n}^{2/ν},

with M_{n} = max_{1<i<j≤n} max {E|h(ξ_{i}, ξ_{j})|^{ν}, ∫ |h(ξ_{i}, ξ_{j})|^{ν}dP (ξ_{i})dP (ξ_{j})}. Under our assumptions (iii) and (iv) given in Lemma 2, using the fact that, for n sufficiently large, |d_{n,0}(X_{i})γ_{h}(X_{j}−

X_{i})K_{h}(X_{j} − X_{i})| ≤ C/h^{d}, one can easily verify that M_{n}^{2/ν} ≤ Ch^{−2d}. So, Jn,1^{′′} = O_{p}(n^{−1}h^{−d}) =

o_{p}(n^{−1/2}) which concludes the proof of Lemma 2. □

Proof of Theorem 2 Using Theorem 1, and the fact that S_{n}^{2}(θ̂) − S^{2}(θ^{∗}) = (S_{n}^{2}(θ̂) − S^{2}(θ^{∗})) + (S_{n}^{2}(θ^{∗}) − S^{2}(θ^{∗})) = o_{p}(||θ̂ − θ^{∗}||) + O_{p}(n^{−1/2}), the desired result follows directly from the identity

\frac{â}{b̂} = \frac{a}{b} + b̂ [â − a − (b̂ − b)\frac{a}{b}] .

Details are omitted. □

References

Bickel, P. J., C. A. J. Klaassen, Y. Ritov, and J. A. Wellner (1993). Efficient and adaptive estimation for semiparametric models. Johns Hopkins Series in the Mathematical Sciences. Baltimore, MD: Johns Hopkins University Press.

Carrasco, M. and X. Chen (2002). Mixing and moment properties of various GARCH and stochastic volatility models. Econometric Theory 18, 17–39.

Dette, H. and A. Munk (1998). Validation of linear regression models. Ann. Statist. 26 (2), 778–800.

Dette, H. and A. Munk (2003). Some methodological aspects of validation of models in nonparametric regression. Statist. Neerlandica 57 (2), 207–244.

Doksum, K. and A. Samarov (1995). Nonparametric estimation of global functionals and a measure of the explanatory power of covariates in regression. Ann. Statist. 23 (5), 1443–1473.

Domowitz, I. and H. White (1982). Misspecified models with dependent observations. J. Econometrics 20 (1), 35–58.

Doukhan, P. and S. Louhichi (1999). A new weak dependence condition and applications to moment inequalities. Stochastic Process. Appl. 84 (2), 313–342.

Fan, J. and I. Gijbels (1996). Local Polynomial Modelling and its Applications, Volume 66 of Monographs on Statistics and Applied Probability. London: Chapman & Hall.

Fan, J. and Q. Yao (2003). Nonlinear Time Series. Springer Series in Statistics. Springer-Verlag. Nonparametric and Parametric Methods.

Fan, Y. and Q. Li (1996). Consistent model specification tests: omitted variables and semiparametric functional forms. Econometrica 64 (4), 865–890.

Gao, J. and M. King (2006). Estimation and model specification testing in nonparametric and semiparametric econometric models. Mpra paper, University Library of Munich, Germany.

Gu, J., D. Li, and D. Liu (2007). Bootstrap non-parametric significance test. J. Nonparametr. Stat. 19 (6-8), 215–230.

Hall, P. (1992). On bootstrap confidence intervals in nonparametric regression. Ann. Statist. 20 (2), 695–711.

Härdle, W. and E. Mammen (1993). Comparing nonparametric versus parametric regression fits. Ann. Statist. 21 (4), 1926–1947.

Hayfield, T. and J. S. Racine (2008). Nonparametric econometrics: The np package. Journal of Statistical Software 27 (5).

Hodges, J. L. and E. L. Lehmann (1954). Testing the approximative validity of statistical hypotheses. Journal of the Royal Statistical Society, Series B 16, 261–268.

Hong, Y. and H. White (1995). Consistent specification testing via nonparametric series regression. Econometrica 63 (5), 1133–1159.

Jun, S. J. and J. Pinkse (2009). Semiparametric tests of conditional moment restrictions under weak or partial identification. J. Econometrics 152 (1), 3–18.

Künsch, H. (1989). The jackknife and the bootstrap for general stationary observations. Ann. Statist. 17, 1217–1241.

Lahiri, S. N. (2003). Resampling methods for dependent data. Springer Series in Statistics. New York: Springer-Verlag.

Lavergne, P. (1998). Selection of regressors in econometrics: parametric and nonparametric methods selection of regressors in econometrics. Econometric Reviews 17 (3), 227–273.

Lavergne, P. and Q. H. Vuong (1996). Nonparametric selection of regressors: the nonnested case. Econometrica 64 (1), 207–219.

Li, Q. and J. Racine (2004). Cross-validated local linear nonparametric regression. Statist. Sinica 14 (2), 485–512.

Li, Q. and S. Wang (1998). A simple consistent bootstrap test for a parametric regression function. J. Econometrics 87 (1), 145–165.

Liu, R. and K. Singh (1992). Moving blocks jackknife and bootstrap capture weak dependence. In Exploring the Limits of Bootstrap (East Lansing, MI, 1990), pp. 225–248. Wiley.

Masry, E. (1996). Multivariate local polynomial regression for time series: uniform strong consistency and rates. J. Time Ser. Anal. 17 (6), 571–599.

Patton, A., D. N. Politis, and H. White (2009). Correction to “Automatic block-length selection for the dependent bootstrap” by D. Politis and H. White [mr2041534]. Econometric Rev. 28 (4), 372–375.

Taylor, L. W. (2009). Penalized-r^{2} criteria for model selection. Manchester School 77 (6), 699–717.

Xia, Y. and W. K. Li (2002). Asymptotic behavior of bandwidth selected by the crossvalidation method for local polynomial fitting. J. Multivariate Anal. 83 (2), 265–287.

Zheng, J. X. (1996). A consistent test of functional form via nonparametric estimation techniques. J. Econometrics 75 (2), 263–289.