Flexible Discriminant Analysis by Optimal Scoring

By Hastie, Trevor; Tibshirani, Robert et al. | Journal of the American Statistical Association, December 1994 | Go to article overview

Flexible Discriminant Analysis by Optimal Scoring


Hastie, Trevor, Tibshirani, Robert, Buja, Andreas, Journal of the American Statistical Association


ANDREAS BUJA*

Fisher's linear discriminant analysis is a valuable tool for multigroup classification. With a large number of predictors, one can find a reduced number of discriminant coordinate functions that are "optimal" for separating the groups. With two such functions, one can produce a classification map that partitions the reduced space into regions that are identified with group membership, and the decision boundaries are linear. This article is about richer nonlinear classification schemes. Linear discriminant analysis is equivalent to multiresponse linear regression using optimal scorings to represent the groups. In this paper, we obtain nonparametric versions of discriminant analysis by replacing linear regression by any nonparametric regression method. In this way, any multiresponse regression technique (such as MARS or neural networks) can be postprocessed to improve its classification performance.

KEY WORDS: Classification; Discriminant analysis; Nonparametric regression; MARS.

1. INTRODUCTION

Multigroup classification or discrimination is an important problem with applications in many fields. In the generic problem, the outcome of interest G falls into J unordered classes, which for convenience we denote by the set J = {1, 2, 3, ... J}. We wish to build a rule for predicting the class membership of an item based on p measurements of predictors or features X [epsilon] [R.sup.p]. Our training sample consists of the class membership and predictors for N items. Traditional statistical methods for this problem include linear discriminant analysis and multiple logistic regression. Neural network classifiers have become a powerful alternative, with the ability to incorporate a very large number of features in an adaptive nonlinear model. Ripley (1994) gave an informative survey from a statistician's viewpoint. The recent success and popularity of neural networks led us to look for similar methodologies in the statistical literature, but this seems to be a relatively unexplored area. One significant appr oach is the classification and regression tree (CART) methodology of Breiman, Friedman, Olshen, and Stone (1984), which is well known to statisticians and is becoming popular in the artificial intelligence community.

There have been a number of recent advances in the nonparametric multiple regression literature. These include projection pursuit regression (Friedman and Stuetzle 1981), the ACE algorithm (Breiman and Friedman 1985), additive models (Hastie and Tibshirani 1990), multivariate adaptive regression splines (MARS; Friedman 1991), Breiman's (1991) II method, the interaction spline methodology of Wahba (1990), and more recently the hinging hyperplanes of Breiman (199la). Neural networks (e.g., Barron and Barron 1988; Lippman 1989, and Hinton 1989) can be viewed as yet another approach to nonparametric regression. In this article we describe methods for multigroup classification that use these tools to generalize linear discriminant analysis.

The foundations for the developments described here can be found in the nonlinear scaling literature, notably the work of Gifi (1981, 1990). Our work was motivated by the unpublished paper by Breiman and Ihaka (1984). Section 6.3 details the connection with their work. Ripley and Hjovt (1994) were similarly motivated.

This article focuses on adaptive classification procedures. A companion article, "Penalized Discriminant Analysis" (Hastie, Buja, and Tibshirani 1994), gives a more technical basis for some of the procedures described here and focuses on obtaining smooth, interpretable canonical variates for high-dimensional problems such as spectral and image analysis. Both articles rely on the connections between penalized optimal scoring and penalized discriminant analysis. Hereafter we will refer to this companion article as PDA.

2. LINEAR DISCRIMINANT ANALYSIS AND GENERALIZATIONS

2. …

The rest of this article is only available to active members of Questia

Already a member? Log in now.

Notes for this article

Add a new note
If you are trying to select text to create highlights or citations, remember that you must now click or tap on the first word, and then click or tap on the last word.
One moment ...
Default project is now your active project.
Project items

Items saved from this article

This article has been saved
Highlights (0)
Some of your highlights are legacy items.

Highlights saved before July 30, 2012 will not be displayed on their respective source pages.

You can easily re-create the highlights by opening the book page or article, selecting the text, and clicking “Highlight.”

Citations (0)
Some of your citations are legacy items.

Any citation created before July 30, 2012 will labeled as a “Cited page.” New citations will be saved as cited passages, pages or articles.

We also added the ability to view new citations from your projects or the book or article where you created them.

Notes (0)
Bookmarks (0)

You have no saved items from this article

Project items include:
  • Saved book/article
  • Highlights
  • Quotes/citations
  • Notes
  • Bookmarks
Notes
Cite this article

Cited article

Style
Citations are available only to our active members.
Buy instant access to cite pages or passages in MLA, APA and Chicago citation styles.

(Einhorn, 1992, p. 25)

(Einhorn 25)

1. Lois J. Einhorn, Abraham Lincoln, the Orator: Penetrating the Lincoln Legend (Westport, CT: Greenwood Press, 1992), 25, http://www.questia.com/read/27419298.

Cited article

Flexible Discriminant Analysis by Optimal Scoring
Settings

Settings

Typeface
Text size Smaller Larger Reset View mode
Search within

Search within this article

Look up

Look up a word

  • Dictionary
  • Thesaurus
Please submit a word or phrase above.
Print this page

Print this page

Why can't I print more than one page at a time?

Help
Full screen

matching results for page

    Questia reader help

    How to highlight and cite specific passages

    1. Click or tap the first word you want to select.
    2. Click or tap the last word you want to select, and you’ll see everything in between get selected.
    3. You’ll then get a menu of options like creating a highlight or a citation from that passage of text.

    OK, got it!

    Cited passage

    Style
    Citations are available only to our active members.
    Buy instant access to cite pages or passages in MLA, APA and Chicago citation styles.

    "Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences." (Einhorn, 1992, p. 25).

    "Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences." (Einhorn 25)

    "Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences."1

    1. Lois J. Einhorn, Abraham Lincoln, the Orator: Penetrating the Lincoln Legend (Westport, CT: Greenwood Press, 1992), 25, http://www.questia.com/read/27419298.

    Cited passage

    Thanks for trying Questia!

    Please continue trying out our research tools, but please note, full functionality is available only to our active members.

    Your work will be lost once you leave this Web page.

    Buy instant access to save your work.

    Already a member? Log in now.

    Oops!

    An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.