Assessing the Accuracy of Automated Twitter Sentiment Coding

By Davis, Joel J.; O'Flaherty, Shannon | Academy of Marketing Studies Journal, January 1, 2012 | Go to article overview

Assessing the Accuracy of Automated Twitter Sentiment Coding


Davis, Joel J., O'Flaherty, Shannon, Academy of Marketing Studies Journal


ABSTRACT

Social media have provided consumers with numerous outlets for disseminating their brand-related comments. Given the impact of these comments on brand image and brand success, marketers have begun to rely on third-party companies to automatically track, collect and analyze these comments with regard to their content and sentiment. There is, however, an absence of controlled research which objectively and systematically assesses the accuracy of these companies' automated sentiment coding. This research evaluated the automated sentiment coding accuracy and misclassification errors of six leading third-party companies for a broad range of comment types and forms. Overall, automated sentiment coding appears to have limited reliability and appears to be accurately accomplished only for very simple statements in which a keyword is used to convey its typical meaning. Statements without keywords or statements in which keyword meaning is reversed through negation or context are accurately coded at very low levels. Neutral statements appear to be problematic for some, but not all, companies. Implications of the research for the use of automated sentiment analysis for brand decisionmaking are presented.

INTRODUCTION

Marketers' attempt to manage brand image in the pre-social media era was straightforward. Brand managers would decide what they wanted consumers to think about their brand and then, through advertising and other consumer-directed communications, would tightly control the messages to which consumers were exposed. Ideally, these communications would create or reinforce the desired brand image. Communications planning took the perspective of "we ?? tell you the things that we think are really important [and what you should think] about our brand and product" (Bostic, 2010) as consumers had little opportunity to disseminate their own perspective.

Social media have fundamentally altered how brand image is created, maintained and changed. Today's successful marketers understand that social media have reduced their control and they realize that they are but one (albeit important) contributor to a dialogue about the brand. These marketers realize that their brand's image will not only be the result of what they say on their brand's own behalf, but additionally, what consumers say and how they interact with and respond to these consumers' comments (Davenport & Beck, 2002; Esch et al., 2006). Consumers are now in at least a shared leadership position with regard to brand image creation, in essence telling marketers "your brand is whatever [we] say it is" (Li & Bernoff, 2008).

Li and Bernoff (2008) refer to the transfer of power from businesses to individuals as "the groundswell." Examples of the groundswell are the abundance of online brand and product reviews as well as online communities such as Get Satisfaction (http://www. getsatisfaction.com), which provides a forum for customers to raise questions or complain about a wide range of companies, and for the resulting discussions to be displayed for other consumers to search and view. Online product reviews and sites such as Get Satisfaction can have a profound effect on brand image and brand success given that the consumer decision-making process is greatly influenced by other consumers' comments (Goldberg et al., 2001; Kelsey Group, 2007; Jansen et al., 2009), especially at the time of product evaluation and purchase (Zabin and Jefferies, 2008).

Clearly, it is important for marketers to understand what consumers are saying about their brands and products. Specifically, marketers need to be aware of what consumers are saying (i.e., the specific comments about the brand) and they need to understand what consumers are feeling (i.e., the sentiment associated with a brand comment, that is, whether a particular comment is positive, negative or neutral). Cai et al. (2010) provide a rationale for the specific focus on sentiment monitoring: "The voice of the web to gain consumer, brand and market insights can be truly differentiating and valuable to today's corporations [and] one important form of insights can be derived from sentiment analysis. …

The rest of this article is only available to active members of Questia

Already a member? Log in now.

Notes for this article

Add a new note
If you are trying to select text to create highlights or citations, remember that you must now click or tap on the first word, and then click or tap on the last word.
One moment ...
Default project is now your active project.
Project items
Notes
Cite this article

Cited article

Style
Citations are available only to our active members.
Buy instant access to cite pages or passages in MLA 8, MLA 7, APA and Chicago citation styles.

(Einhorn, 1992, p. 25)

(Einhorn 25)

(Einhorn 25)

1. Lois J. Einhorn, Abraham Lincoln, the Orator: Penetrating the Lincoln Legend (Westport, CT: Greenwood Press, 1992), 25, http://www.questia.com/read/27419298.

Note: primary sources have slightly different requirements for citation. Please see these guidelines for more information.

Cited article

Assessing the Accuracy of Automated Twitter Sentiment Coding
Settings

Settings

Typeface
Text size Smaller Larger Reset View mode
Search within

Search within this article

Look up

Look up a word

  • Dictionary
  • Thesaurus
Please submit a word or phrase above.
Print this page

Print this page

Why can't I print more than one page at a time?

Help
Full screen
Items saved from this article
  • Highlights & Notes
  • Citations
Some of your highlights are legacy items.

Highlights saved before July 30, 2012 will not be displayed on their respective source pages.

You can easily re-create the highlights by opening the book page or article, selecting the text, and clicking “Highlight.”

matching results for page

    Questia reader help

    How to highlight and cite specific passages

    1. Click or tap the first word you want to select.
    2. Click or tap the last word you want to select, and you’ll see everything in between get selected.
    3. You’ll then get a menu of options like creating a highlight or a citation from that passage of text.

    OK, got it!

    Cited passage

    Style
    Citations are available only to our active members.
    Buy instant access to cite pages or passages in MLA 8, MLA 7, APA and Chicago citation styles.

    "Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences." (Einhorn, 1992, p. 25).

    "Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences." (Einhorn 25)

    "Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences." (Einhorn 25)

    "Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences."1

    1. Lois J. Einhorn, Abraham Lincoln, the Orator: Penetrating the Lincoln Legend (Westport, CT: Greenwood Press, 1992), 25, http://www.questia.com/read/27419298.

    Cited passage

    Thanks for trying Questia!

    Please continue trying out our research tools, but please note, full functionality is available only to our active members.

    Your work will be lost once you leave this Web page.

    Buy instant access to save your work.

    Already a member? Log in now.

    Search by... Author
    Show... All Results Primary Sources Peer-reviewed

    Oops!

    An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.