The Receiver-Operating Characteristic (Roc) Analysis: Fundamentals and Applications in Clinical Psychology

By Pintea, Sebastian; Moldovan, Ramona | Journal of Evidence-Based Psychotherapies, March 2009 | Go to article overview

The Receiver-Operating Characteristic (Roc) Analysis: Fundamentals and Applications in Clinical Psychology


Pintea, Sebastian, Moldovan, Ramona, Journal of Evidence-Based Psychotherapies


Abstract

The Receiver-Operating Characteristic (ROC) analysis has been long used in Signal Detection Theory to depict the tradeoff between hit rates and false alarm rates of classifiers. In the last years, ROC analysis has become largely used in the medical community for visualizing and analyzing the performance of diagnostic tests. Our article points out some fundamental aspects of ROC analysis underlying the importance of using ROC analysis in evaluating the diagnostic validity of tests commonly used in clinical psychology. The main statistical programs available for this type of analysis, with their advantages and deficiencies are also discussed. In order to illustrate how ROC analysis works in clinical research, we also describe an application of ROC analysis in evaluating scales generally related to depression.

Keywords: receiver operating characteristics (ROC), ROC analysis, area under the curve (AUC), diagnostic performance, sensitivity, specificity, clinical psychology, depression

THE ROC ANALYSIS: FUNDAMENTALS

Receiver Operating Characteristic (ROC) analysis is a procedure used in assessing diagnostic properties of tests, namely in assessing the way various measures generally discriminate between different categories of subjects. In order to do this, a cut-off point needs to be established; based on the cut-off point, we can determine whether a person with a certain score belongs to one category or another (e.g. normal/non-clinical or clinical group). ROC analysis may also be used when comparing the diagnostic performance of two or more tests (Westin, 2001).

ROC analysis was used for the first time in the military field, for the analysis of radar images, during The Second World War (Westin, 2001). In medicine, the procedure has been used since the 1960s, and there is an extensive literature on the use of ROC graphs for diagnostic testing (Fawcett, 2006). In chemistry, ROC curves analysis is used to solve dichotomous decision problems such as: the presence or absence of a protein marker, whether the structure of a molecule is X or Y, whether a reaction obeys first order kinetics or second, should a reaction be terminated or continued, etc.? (Brown & Davis, 2006).

In clinical psychology, ROC analysis is being used with increased frequency, particularly in examining the utility or performance of diagnostic or screening tools for: future difficulties in reading comprehension (Shapiro, Solari & Petscher, 2008), alcohol and drug abuse (Kills Small et al., 2007), neuropsychological impairment (Horwitz et al., 2008; O'Brien et al., 2007), depression (Benazzi, 2008; Serrano-Duenas & Serrano, 2008; Stafford, Berk & Jackson, 2007; Ballesteros et al., 2007; Walsh et al., 2006), obsessive-compulsive disorder (Ivarsson & Larsson, 2008), bipolar disorder (Parker et al., 2008;), suicide (Jokinen, Nordstrom & Nordstrom, 2008), dementia (Chiu et al., 2008; Giaquinto & Parnetti, 2006), dropout risk from different treatments such as cognitive-behavior therapy for insomnia (Ong, Kuo & Manber, 2008) and so on.

Two of the more important literature reviews on ROC curves analysis have been conducted by McFall & Treat (1999) and Streiner & Cairney (2007). McFall and Treat follow a broader objective in their work, as they themselves state, "about the functions of clinical assessment, the standards by which methods can be evaluated, and the most promising approaches to achieving the broad goals of clinical assessment" (p. 216). Compared to their article and also to the one of Streiner and Cairney, our paper is intended to be more applicative. We use actual data collected from a Romanian sample to illustrate a number of ROC concepts reviewed in this paper; we also discuss an extended set of uses, from establishing cutoff-points to comparing different tests to overall and partial ROC areas, to specific points of the curves.

Measures related to ROC curves

The dichotomous decision process is based on a threshold value ? …

The rest of this article is only available to active members of Questia

Sign up now for a free, 1-day trial and receive full access to:

  • Questia's entire collection
  • Automatic bibliography creation
  • More helpful research tools like notes, citations, and highlights
  • Ad-free environment

Already a member? Log in now.

Notes for this article

Add a new note
If you are trying to select text to create highlights or citations, remember that you must now click or tap on the first word, and then click or tap on the last word.
One moment ...
Default project is now your active project.
Project items

Items saved from this article

This article has been saved
Highlights (0)
Some of your highlights are legacy items.

Highlights saved before July 30, 2012 will not be displayed on their respective source pages.

You can easily re-create the highlights by opening the book page or article, selecting the text, and clicking “Highlight.”

Citations (0)
Some of your citations are legacy items.

Any citation created before July 30, 2012 will labeled as a “Cited page.” New citations will be saved as cited passages, pages or articles.

We also added the ability to view new citations from your projects or the book or article where you created them.

Notes (0)
Bookmarks (0)

You have no saved items from this article

Project items include:
  • Saved book/article
  • Highlights
  • Quotes/citations
  • Notes
  • Bookmarks
Notes
Cite this article

Cited article

Style
Citations are available only to our active members.
Sign up now to cite pages or passages in MLA, APA and Chicago citation styles.

(Einhorn, 1992, p. 25)

(Einhorn 25)

1

1. Lois J. Einhorn, Abraham Lincoln, the Orator: Penetrating the Lincoln Legend (Westport, CT: Greenwood Press, 1992), 25, http://www.questia.com/read/27419298.

Cited article

The Receiver-Operating Characteristic (Roc) Analysis: Fundamentals and Applications in Clinical Psychology
Settings

Settings

Typeface
Text size Smaller Larger Reset View mode
Search within

Search within this article

Look up

Look up a word

  • Dictionary
  • Thesaurus
Please submit a word or phrase above.
Print this page

Print this page

Why can't I print more than one page at a time?

Full screen

matching results for page

Cited passage

Style
Citations are available only to our active members.
Sign up now to cite pages or passages in MLA, APA and Chicago citation styles.

"Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences." (Einhorn, 1992, p. 25).

"Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences." (Einhorn 25)

"Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences."1

1. Lois J. Einhorn, Abraham Lincoln, the Orator: Penetrating the Lincoln Legend (Westport, CT: Greenwood Press, 1992), 25, http://www.questia.com/read/27419298.

Cited passage

Welcome to the new Questia Reader

The Questia Reader has been updated to provide you with an even better online reading experience.  It is now 100% Responsive, which means you can read our books and articles on any sized device you wish.  All of your favorite tools like notes, highlights, and citations are still here, but the way you select text has been updated to be easier to use, especially on touchscreen devices.  Here's how:

1. Click or tap the first word you want to select.
2. Click or tap the last word you want to select.

OK, got it!

Thanks for trying Questia!

Please continue trying out our research tools, but please note, full functionality is available only to our active members.

Your work will be lost once you leave this Web page.

For full access in an ad-free environment, sign up now for a FREE, 1-day trial.

Already a member? Log in now.