P Values for Composite Null Models
Journal article by M. J. Bayarri, James O. Berger; Journal of the American Statistical Association, Vol. 95, 2000
Journal Article Excerpt
P Values for Composite Null Models. by M. J. BAYARRI , JAMES O. BERGER The problem of investigating compatibility of an assumed model with the data is investigated in the situation when the assumed model has unknown parameters. The most frequently used measures of compatibility are p values, based on statistics T for which large values are deemed to indicate incompatibility of the data and the model. When the null model has unknown parameters, p values are not uniquely defined. The proposals for computing a p value in such a situation include the plug-in and similar p values on the frequentist side, and the predictive and posterior predictive p values on the Bayesian side. We propose two alternatives, the conditional predictive p value and the partial posterior predictive p value, and indicate their advantages from both Bayesian and frequentist perspectives. KEY WORDS: Bayes factors; Bayesian p values; Conditioning; Model checking; Predictive distributions. 1. INTRODUCTION 1.1 Background In parametric statistical analysis of data X, one is frequently working at a given moment with an entertained model or hypothesis [H.sub.0]: X [sim] f(x; [theta]). We will call this the null model or null hypothesis, even though no alternative is explicitly formulated. We assume that f(x; [theta]) is either a discrete density or a continuous density (with respect to Lebesgue measure). A statistic T = t(X) is chosen to investigate compatibility of the model with the observed data, [X.sub.obs]. We assume that T has been expressed in such a way that large values of T indicate less compatibility with the model. The most commonly used measure of compatibility is the p value, defined as p = Pr(t(X) [greater than or equal to] t([X.sub.obs])). (1) When [theta] is known, the probability computation in (1) is with respect to f(x; [theta]). The focus in this article is on the choice of the probability distribution used to compute (1) when [theta] is unknown. In Section 2 we present two new types of p values, which we argue are superior to existing choices. The rest of this section describes the most common of the existing choices. We abuse notation by using f(t; [theta]) and f(t; [theta]) to denote the marginal density of t(X) and the conditional density of t(X) given u(X) = u. The most obvious way to deal with an unknown [theta] in computation of the p value is to replace [theta] in (1) by some estimate, [theta]. In this article we consider only the usual choice for [theta], namely the maximum likelihood estimator (MLE). We call the resulting p value the plug-in p value ([p.sub.plug]). Using a superscript to denote the density with respect to which the p value in (1) is computed, the plug-in p value is thus defined as [P.sub.plug] =[Pr.sup.f(.;[theta])](t(X) [greater than or equal to] t([X.sub.obs]). (2) The main strengths of [P.sub.plug] are its simplicity and intuitive appeal. Its main weakness appears to be a failure to account for uncertainty in the estimation of [theta], although as we show, this issue is rather involved. Another natural device for eliminating the unknown [theta] is to condition on a sufficient statistic, U, for [theta]. Then f(x\[u.sub.obs;][theta]) does not depend on [theta], and computations in (1) can be carried out using the completely specified f(X\[u.sub.obs]). [In fact, U need only be sufficient for [theta] with respect to f(t; [theta]).] We call these p values similar p values, a term borrowed from the related notion of similar tests and confidence regions. A similar p value is thus defined as [p.sub.sim] = [Pr.sup.f(.\[u.sub.obs])](t(X) t([X.sub.obs])). (3) The main strength of [p.sub.sim] is that it is based on a proper probability computation, which imbues the end result with various desirable properties (discussed later). Its main weaknesses are that the computation can be burdensome and that a suitable sufficient U typically does not exist. Bayesians have a natural way to eliminate nuisance parameters: integrate them out. Thus if [pi])[theta]) is a prior distribution for 9, then the marginal or (prior) predictive distribution is m(x) = [integral of] f(x; [theta])[pi]([theta]) d[theta]. (4) Because this is free of [theta], it can be used to compute a p value, leading to the prior predictive p value, given by [p.sub.prior] = [Pr.sup.m](.)(t(X)[greater than or equal to] t([X.sub.obs])). (5)The main strengths of [p.sub.prior] are that it is also based on a proper probability computation (at least if [pi]([theta]) is proper), and that it suggests a natural and simple T, namely t(x) = 1/m(x). The main weakness Of [p.sub.prio]r for pure model checking is its ... |
End of free preview...
To continue reading this publication, you must have a Questia Subscription.
Try Us Today! Click Here
Questia provides the world's largest online library of scholarly books and journal articles, with integrated footnote and bibliography tools, highlighting, note taking and book marking. With a Questia subscription, you'll have access to the full text of more than 67,000 books and 1.5 million articles.
Already a subscriber? Login:
Sponsored Links
Free Newsletter
Get helpful how-to's, writing tips, search strategies, quizzes & more!
Search the Library
