Putting Machine Testing to the Test: Next-Generation Educational Standards Meet Next-Generation Scoring Methods, but with Controversy

By Mayes, Randall | The Futurist, January-February 2014 | Go to article overview

Putting Machine Testing to the Test: Next-Generation Educational Standards Meet Next-Generation Scoring Methods, but with Controversy


Mayes, Randall, The Futurist


The goal of raising academic achievement in the United States has led to a number of remedies, ranging from the No Child Left Behind Act (2001) to the Common Core State Standards (2010). This has meant not only more testing, but also more-complex testing of students. In order to keep up with grading these tests, a growing trend is to use machine scoring--even on the essay portion of standardized tests.

The need to do something about declining and mediocre standardized scores means that future testing and assessment in K-12 will require more-complex tasks than the current selected-response method, where students can derive correct answers by guessing. New testing methods will utilize more performance-based tasks and constructed responses. Essay questions will require students to demonstrate independent thinking, selecting and organizing information from provided references, and using reasoning skills for development.

[ILLUSTRATION OMITTED]

Since 2005, the Scholastic Aptitude Test (SAT) for college entrance has included an essay section to measure writing skills. Graduate and professional schools are also utilizing writing as part of the admissions process. A more recent phenomenon is universities experimenting with Massive Open Online Courses (MOOCs), some with more than 100,000 students. Testing in these educational programs generates an extremely large number of essays.

All of these trends--new educational standards, compulsory statewide student assessments, the college admissions process, and the emergence of MOOCs--have created a new business model in testing, and several scenarios could potentially develop for assessing students.

The market for standardized educational testing has led to commercial ventures and has become a multimillion-dollar business. Through bidding, educational testing companies have entered into contracts with individual states and private schools using thousands of readers.

For overall student assessment, grading essays is the most expensive component of standardized educational testing. For the last three decades, essay scoring has relied on human readers, who have college degrees from different fields, have demonstrated writing ability, and are qualified through scoring-agreement rates with other readers in practice sets.

In addition to paying readers for training and scoring essays, educational testing companies develop rubrics that address which features they want to score and the characteristics of each score on a scale of 1 to 6 points. Assessing essays also requires the time-consuming process of developing prompts that pose essay questions.

With the demands on teachers' time, the need for quick feedback, and the labor costs for human scoring, educational researchers need to determine the most efficient way to assess standardized essays. From a business perspective, using artificial intelligence dramatically reduces the cost and time required to evaluate student writing. The algorithm developed by Educational Testing Service for the General Management Aptitude Test can score 16,000 essays in 20 seconds.

Given the efficiency of AI scoring, it is unlikely that using only human readers will make business sense or satisfy the requirements for next-generation educational assessment.

Yet, technological innovation is often disruptive. AI evaluators unfortunately will displace thousands of reader jobs. On a positive note, this disruption in the existing market creates new opportunities in computer programming, artificial intelligence, linguistics, business development, psychometrics, and Web design.

The History and Future of Machine Scoring

In 1967, former high school English teacher Ellis Page developed Project Essay Grade (PEG), the first successful automated essay scoring system, but it required IBM punch cards and mainframe computers. In the 1990s, this would all change with advances in computing, AI, and natural language processing. …

The rest of this article is only available to active members of Questia

Already a member? Log in now.

Notes for this article

Add a new note
If you are trying to select text to create highlights or citations, remember that you must now click or tap on the first word, and then click or tap on the last word.
One moment ...
Default project is now your active project.
Project items

Items saved from this article

This article has been saved
Highlights (0)
Some of your highlights are legacy items.

Highlights saved before July 30, 2012 will not be displayed on their respective source pages.

You can easily re-create the highlights by opening the book page or article, selecting the text, and clicking “Highlight.”

Citations (0)
Some of your citations are legacy items.

Any citation created before July 30, 2012 will labeled as a “Cited page.” New citations will be saved as cited passages, pages or articles.

We also added the ability to view new citations from your projects or the book or article where you created them.

Notes (0)
Bookmarks (0)

You have no saved items from this article

Project items include:
  • Saved book/article
  • Highlights
  • Quotes/citations
  • Notes
  • Bookmarks
Notes
Cite this article

Cited article

Style
Citations are available only to our active members.
Buy instant access to cite pages or passages in MLA, APA and Chicago citation styles.

(Einhorn, 1992, p. 25)

(Einhorn 25)

1. Lois J. Einhorn, Abraham Lincoln, the Orator: Penetrating the Lincoln Legend (Westport, CT: Greenwood Press, 1992), 25, http://www.questia.com/read/27419298.

Cited article

Putting Machine Testing to the Test: Next-Generation Educational Standards Meet Next-Generation Scoring Methods, but with Controversy
Settings

Settings

Typeface
Text size Smaller Larger Reset View mode
Search within

Search within this article

Look up

Look up a word

  • Dictionary
  • Thesaurus
Please submit a word or phrase above.
Print this page

Print this page

Why can't I print more than one page at a time?

Help
Full screen

matching results for page

    Questia reader help

    How to highlight and cite specific passages

    1. Click or tap the first word you want to select.
    2. Click or tap the last word you want to select, and you’ll see everything in between get selected.
    3. You’ll then get a menu of options like creating a highlight or a citation from that passage of text.

    OK, got it!

    Cited passage

    Style
    Citations are available only to our active members.
    Buy instant access to cite pages or passages in MLA, APA and Chicago citation styles.

    "Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences." (Einhorn, 1992, p. 25).

    "Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences." (Einhorn 25)

    "Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences."1

    1. Lois J. Einhorn, Abraham Lincoln, the Orator: Penetrating the Lincoln Legend (Westport, CT: Greenwood Press, 1992), 25, http://www.questia.com/read/27419298.

    Cited passage

    Thanks for trying Questia!

    Please continue trying out our research tools, but please note, full functionality is available only to our active members.

    Your work will be lost once you leave this Web page.

    Buy instant access to save your work.

    Already a member? Log in now.

    Oops!

    An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.