Multimedia Stories for Deaf Children

By Andrews, Jean F.; Jordan, Daonald L. | Teaching Exceptional Children, May/June 1998 | Go to article overview

Multimedia Stories for Deaf Children

Andrews, Jean F., Jordan, Daonald L., Teaching Exceptional Children

It's hard to boat the feel of a book between the hands-feeling Its soft cover, turning crisp pages, and smelling the ink. The low cost of books and their easy portability-carrying them to the couch, back yard, bathtub, pool, and the beach-entice us. But the whir and wonder of technology has captivated ni many teachers. Multimedia stories on the Web, for instance, can lead children with language and reading disabilities to read print augmented with graphics, animation, and movies.

Using Multimedia Technology

Multimedia technology allows authors to develop stories in two or more languages. Each language, then, can be accessed by the click of a button on each page. Multimedia applications are especially useful for deaf children because video dictionaries of sign language can be built right into the stories. This article describes our U.S. Department of Education-funded project and shows how teachers can use the Web to research their stories-and put these stories on the Web.

Phase 1: We set up a multimedia laboratory with state-of-the-art hardware and software.

Phase 2: Our staff (graduate students in deaf education and computer science) developed scripts and multimedia stories centering on the Mexican-American culture. These student-authors used library sources, as well as the Web, for information (see Figures 1 and 2 for Web sources). Native users of American Sign Language provided sign language translations and native-Spanish-speaking students provided the Spanish translations. Our computer science students set up a Web server to distribute our stories. Phase 3: We provided summer workshops for teachers, teaching them to use the Web and to develop their own multimedia stories for their students. We placed teacher-authored stories on the Web.

Phase 4: We set up a research plan to follow the progress of Mexican-American deaf children over 1 year to see how they learned language using this new technology.

Designing Projects for MexicanAmerican Deaf Children

We developed materials and activities for Mexican-American deaf children because they are the fastest growing minority group in the U.S. school-age population of deaf children, particularly in Texas, where we work. In fact, there are more than 7,000 deaf children from Spanishspeaking homes in the United States, and this number is growing (Schildroth & Hotto, 1996). These children have difficulty learning English; and on standardized tests that measure reading, language and mathematics, many score 2-3 years below their Anglo peers who are deaf (Allen, 1994; Gerner de Garcia, 1993).

School has been difficult for Hispanic deaf students because of cultural and linguistic differences. Cultural influences can be Spanish, Mexican, Puerto Rican, Dominican Republican, Cuban, Latin or South American origin, or mixed. These students' language learning may be fragmented. For example, children may use some spoken and written English, American Sign Language (ASL), gestures, and home signs. They may also speak and lipread the Spanish language. In addition, they may use some spoken English and sign language, or a mixture of these. And even further, if families recently emigrated from Mexico or South America, these deaf children may use an indigenous sign language. Such a mixture of codes and languages can make learning academic subjects in English difficult for deaf Mexican-American students.

Mexican-American deaf youth must also navigate through three different cultures-Hispanic, American, and deaf cultures. Even though they might eat ethnic foods and celebrate the religious and historical holidays of their families, these cultural events have little meaning because few family members can explain these events to them in sign language.

Consequently, many deaf Mexican-American children have grown up not fully understanding their home culture. To meet their language and cultural needs, we designed a project to develop stories centered on Mexican-American cultural themes written at different reading levels-elementary, junior high, and high school. We added translations in ASL. We also provided written and spoken texts in Spanish and English because some Mexican-American deaf and hardof-hearing children may benefit from hearing and reading Spanish words. We wanted our multimedia stories to be accessible to children with a wide range of hearing losses. (See Figures 3 and 4, page 30, for other uses of multimedia stories.)

Personalizing Dictionaries and Stories

Multimedia technology lets you explore information at your own pace. It combines printed text, narration, words, sounds, music, graphics, photos, movies, and animation on one computer "page." These pages can be linked together sequentially or can branch off into new pages called hypermedia.

For deaf children who use ASL, printed texts can be supported with sign language video (or movie) dictionaries. These videos can include facial expressions, head tilts, eyebrow raises, and body movements, the elements that encode the grammar of ASL (Pollard, 1993). No longer must deaf students turn to the teacher or sign language interpreter to ask what a word in a story means. They can simply press a button-and a person will appear on the screen, explaining the word in sign. This person could be the teacher, thus personalizing the dictionary.

For example, one Mexican-American folktale we designed is called "The Tracks" or "Las Vias." In this story, on one page is the phrase "piled up." If the student does not know the meaning, he can click on this "hot word" (which is colored red to differentiate from the other black text). By clicking on this hot word, the student is linked to a movie clip of a deaf adult signing the concept "piled up." All pages contain hot words where the child can get a sign language translation. Students can also click on the button and have whole paragraphs signed to them (see Figure 5).

Hard-of-hearing children can choose these sign language translations or turn up the sound track volume of the story or use both. If the Hispanic child knows some Spanish words, he or she may click on a button to translate the paragraph into Spanish. Stories formatted in multiple ways provide options for children to choose the mode that best meets their needs. Along the way, they learn about their Mexican-American culture in stories that are motivating and entertaining to read.

Creating Mexican-Amer&an Faculty and graduate students of Mexican-American heritage decided what Mexican-American cultural themes should be used. We purchased a library of 150 books, including Hispanic history, literature, encyclopedias, and references. We bought books on holidays, food, and traditions, and we purchased videotapes and magazines. We also found Internet resources on Hispanic and deaf cultures (see Figures 1 and 2).

With Web-related information, we developed stories about folktales, animal stories, Mexican-American history, famous Hispanic Americans, holidays, crafts, foods, and entertainers. An important topic was also added-successful Hispanic-deaf persons to provide role modeling to deaf children.

Our graduate students teamed up to write short two-page scripts. They calculated a reading grade level using the Flesch-Kincaid readability formula on Microsoft Word 6.0. After stories were edited, computer-science research assistants designed a "book" using the ToolBook software. Then the computergraphics students designed pictures for the story. We also scanned pictures from books, calendars, and magazines. Teachers who made materials for classroom-use only could use pictures from books and magazines. However, we wrote original stories or rewrote folktales which had been written more than 75 years ago in order to not violate copyright laws. These stories we will commercially market.

Other graduate students worked on the sign language videos of the script. Deaf students fluent in ASL signed scripts in American Sign Language. Other students used videocapturing equipment to mesh the sign language with the text of each story. One of our graduate students from Mexico who was fluent in Spanish translated our stories into written and spoken Spanish.

Other graduate students designed comprehension tests or games for each story (Pollard, 1993). The games provided the teachers with a tool to measure reading comprehension. The students could push a button to see how many points they scored after each game. For example, in one game, the student pressed a button to see a sentence signed into ASL. The student's task was to translate the ASL sentence into English. At the bottom of the screen were a group of scrambled words. The student had to drag the words and put them into the right slots that would show a grammatically correct English sentence.

For example, in "The Tracks," students come upon scrambled words. They push on the "hot button" PLAY and see a video clip of a sentence signed in ASL. It is their job to unscramble the words to make a correct English sentence. If they succeed, they get a smiley face on the screen (see Figure 6).

Producing a CD-ROM mad a Public Web She

After we edited the stories, we transferred the files and pressed them to a CD-ROM disk (Andrews & Jordan, 1998). We also made copies of the CD-ROMs for the teachers participating in the research component of the project.

Mexican-American deaf children in the area were invited to the multimedia lab at our university to read the stories and comment on them. We have also loaded up some of the stories on the World Wide Web for national distribution. You can access our stories through our Web page (http://www.deafed.lamar. edu/).

An important part of our project was to teach educators how to develop their own multimedia stories. During two summers, 20 teachers who worked primarily with Mexican-American deaf children attended a 2-week multimedia workshop. Some teachers had no experience with Windows applications; others had used ToolBook before. In the two summers, the teachers learned basic and advanced competencies with ToolBook software.

Skills included accessing the Internet (e-mail, newsgroups, file transfer protocol, World Wide Web). Teachers also learned how to operate a CD-ROM drive, view the internal and external components of the computer, use scanners, and create and save simple books on ToolBook. Further, they learned the basics of ToolBook software, viewed commercial CD-ROM software, and used the Internet to download and save audio and video files. Teachers created text and buttons in ToolBook; and they recorded, saved, and captured sound and video and incorporated them into their books. They composed a story with a Mexican-American theme and storyboarded it into pages, shared files over the network, transferred files over a network, and completed a CDROM storybook. Teachers completed one story, and our staff pressed it to a CDROM so they could use it with their Mexican-American students. See our Web page for titles of stories our teachers created.

Coding Res_sh

Of the 20 participating teachers, 7 took multimedia computers from our university setting to their home classrooms. Teachers would use the computers with the students to create additional books using ToolBook software. These teachers worked in classrooms with at least 80% Hispanic-deaf children in McAllen, Corpus Christi, Zapata, Austin, Baytown, San Antonio, and Beaumont, Texas. These 7 teachers will study the language learning of 10 Hispanic-deaf children as they use multimedia technology during the third and final year of the project. So far, commentary cn the use of tech nology in the schools has been anecdotal or testimonial, rather than data on the children's performance in reading. (An exception to this is data on math score improvements using computers.) In view of this lack of data, our study will use indepth literacy assessments to determine how deaf children learn language from reading multimedia stories in Phase 4 of our project.

The seven teachers who received computers and took them to their classrooms will follow the progress of two or three students over a full school year (September 1997-June 1998). A set of literacy measures will be given to the children. These will be standardized tests and portfolio or performance assessments.

The literacy portfolio is a collection of a student's work and records of progress of achievement assembled over time. In contrast to standardized tests, literacy portfolios measure students on high-quality, performance-based, meaningful tasks. These tasks include reading and discussing significant books and articles, writing reflective responses to meaningful topics, researching and writing reports, and compiling a log of books and stories read (Valencia & Calfee, 1991). Based on current reading research, we put together a battery of different tasks to measure literacy development.

Using these techniques, we can provide a detailed description of 10 MexicanAmerican deaf students and how they develop English literacy skills using technology, as well as other literacy activities in their curriculum over a full school year. Even though researchers have reported smaller studies of the language and communication abilities of Hispanic deaf children (Gerner de Garcia, 1993; LuetkeStahlman & Weiner, 1984), to our knowledge, no one has examined MexicanAmerican deaf students' literacy development over time. Cnkng Early Results

We have started to collect data on Mexican-American deaf children using our CD-ROM multimedia stories. More detailed studies will be made in the 199899 school year, the final year of our project. So far, teachers report that their students enjoy using the sign language videos with the English print. This has increased vocabulary learning. One teacher reported that one of her students used the vocabulary hot words independently by pushing the ASL translation button to get the meaning in sign. Another teacher reported that the stories with Mexican-American themes have generated class discussions about identity and customs (pinatas, 15th birthday party, immigrants, and language). For example, a Vietnamese deaf child who recognized that her skin color was the same as the Mexican-American signer in one story, asked if she was Mexican too. This started a class discussion on ethnic background and skin color. One teacher's story about the Quinceanera, or the 15th birthday party, an important rite of passage for young Hispanic girls, generated lots of discussion. Two students who were going through the ceremonies themselves did not understand the meaning behind it. The CD-ROM story provided explanations to these girls-for example, the meaning of the white dress, the church service, and the necklace worn by the girl in the story.

One teacher reported that her deaf students were learning Spanish words and bringing them home to show to their parents. Many of these Mexican-American deaf children came from homes where neither English nor signing was used. Thus, the CD-ROM stories raised the children's awareness of their home family's native language.

Another teacher reported that groups of her students used the CD-ROM stories independently. Students would gather around each other and take turns as the "teacher" to see if their classmates could read the text, then check the ASL translation for meaning. Our anecdotal reports are still preliminary, but we plan more comprehensive portfolio assessments of students' reading skills; and we plan to build case studies of Mexican-American deaf children and their language development. Find Notes

The Web offers a valuable resource for researching stories. Teachers and students have access to libraries of print, graphics, and videos on the deaf culture, sign language, and the Mexican-American culture.

The Web can also be used to distribute teacher and student stories worldwide. It is easy to download our stories with the graphics, print, and Spanish translations. The sign language videos, however, may take longer to download. The technology for crisp, clear sign language videos with fast transmission rates is not here yet. But with compressed video techniques and faster computers and modems, this technology is emerging fast. Schools must have computers with high-speed modems or direct lines to the Internet. Currently, the download time for our sign language translations is slow and cumbersome. We anticipate that the technology will soon allow faster transmission of sign language videos. In the meantime, we will continue our development of stories and put them on the Web for experimentation in preparation for new technology.

Our four-phase project-setting up a multimedia lab, developing stories, training teachers, and assessing the progress of deaf children as they use our CD-ROM stories-is an attempt to make a positive impact in improving literacy for deaf children from the Mexican-American heritage. With computer costs decreasing and with improvements in technology, multimedia on CD-ROMs and on the Web will continue to be excellent tools to bring I 4

*To order books marked by an asterisk (*), please call 24 hrs/365 days: 1-800-BOOKS-NOW (266-5766) or (702) 258-3338 and ask for ext, 1212; or visit them on the Web at http://www. BooksNow. com/TeachingExceptional.htm. Use VISA, M/C, or AMEX or send check or money order + $4.95 S&H ($2.50 each add'l item) to: Books Now, 448 East 6400 South, Suite 125, Salt Lake City, UT 84107.




Allen, T. (1994). Who are the deaf and hardof-hearing students leaving high school and entering postsecondary education? Washington, DC: Office of Special Education and Rehabilitative Services, U.S. Department of Education.

Andrews, J. & Jordan, D. (1998). The tracks and the wise stones: 2 Mexican American folktales retold in American Sign Language, English and Spanish. A CD-ROM available now. Lamar University, Beaumont, TX 77710.


Cummins, J. (1988). Second language acquisition with bilingual education programs. In L. Beebe (Ed.), Issues in second language acquisition (pp. 145-166). New York: Newbury.* Gerner de Garcia, B. (1993). Language in use in Spanish-speaking families with deaf children. Unpublished doctoral dissertation, Boston University.

Krashen, S. (1996). Under attack: The case against bilingual education. Culver City, CA: Language Education Associates.* Luetke-Stahlman, B., & Weiner, F. (1984). Language and/or system assessment for Spanish preschoolers. In G. Delgado (Ed.), The Hispanic deaf: Issues and challenges for


bilingual special education (pp. 106-121). Washington, DC: Gallaudet University Press.

Pollard, G. (1993). Making accessible to the deaf CD-ROM reading software. Austin, TX: Texas School for the Deaf.* Schildroth, A., & Hotto, S. (1996). Changes in student and program characteristics. American Annals of the Deaf, 141(2), 68-71. ToolBook II Assistant, Software. (1996, 1997). Published by Asymetrix Corp., Belview, WA.

Valencia, S., & Calfee, R. (1991). The development and use of literacy portfolios for students, classes, and teachers. Applied Measurement in Education, 4(4), 333-345.



Vernon M & Andrews J ( 1990). The psychology of deafness: Understanding deaf and hard of hearing persons. White plains, NY: Longman.*

[Author Affiliation]

Jean E Andrews (CEC Texas Federation), Department of Communication Disorders and Deafness, Lamar University, Beaumont, Texas. Donald L. Jordan, Department of Business and Management Information Systems, Lamar University, Beaumont, Texas. Address correspondence to Jean E Andrews, Department of Communication Disorders and Deafness, Lamar University PO. Box 20076, Beaumont, TX 77710 (e-mail: JPhelan2OO@

Copyright 1998 CEC.

The rest of this article is only available to active members of Questia

Sign up now for a free, 1-day trial and receive full access to:

  • Questia's entire collection
  • Automatic bibliography creation
  • More helpful research tools like notes, citations, and highlights
  • Ad-free environment

Already a member? Log in now.

Notes for this article

Add a new note
If you are trying to select text to create highlights or citations, remember that you must now click or tap on the first word, and then click or tap on the last word.
One moment ...
Default project is now your active project.
Project items

Items saved from this article

This article has been saved
Highlights (0)
Some of your highlights are legacy items.

Highlights saved before July 30, 2012 will not be displayed on their respective source pages.

You can easily re-create the highlights by opening the book page or article, selecting the text, and clicking “Highlight.”

Citations (0)
Some of your citations are legacy items.

Any citation created before July 30, 2012 will labeled as a “Cited page.” New citations will be saved as cited passages, pages or articles.

We also added the ability to view new citations from your projects or the book or article where you created them.

Notes (0)
Bookmarks (0)

You have no saved items from this article

Project items include:
  • Saved book/article
  • Highlights
  • Quotes/citations
  • Notes
  • Bookmarks
Cite this article

Cited article

Citations are available only to our active members.
Sign up now to cite pages or passages in MLA, APA and Chicago citation styles.

(Einhorn, 1992, p. 25)

(Einhorn 25)


1. Lois J. Einhorn, Abraham Lincoln, the Orator: Penetrating the Lincoln Legend (Westport, CT: Greenwood Press, 1992), 25,

Cited article

Multimedia Stories for Deaf Children


Text size Smaller Larger Reset View mode
Search within

Search within this article

Look up

Look up a word

  • Dictionary
  • Thesaurus
Please submit a word or phrase above.
Print this page

Print this page

Why can't I print more than one page at a time?

Full screen

matching results for page

    Questia reader help

    How to highlight and cite specific passages

    1. Click or tap the first word you want to select.
    2. Click or tap the last word you want to select, and you’ll see everything in between get selected.
    3. You’ll then get a menu of options like creating a highlight or a citation from that passage of text.

    OK, got it!

    Cited passage

    Citations are available only to our active members.
    Sign up now to cite pages or passages in MLA, APA and Chicago citation styles.

    "Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences." (Einhorn, 1992, p. 25).

    "Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences." (Einhorn 25)

    "Portraying himself as an honest, ordinary person helped Lincoln identify with his audiences."1

    1. Lois J. Einhorn, Abraham Lincoln, the Orator: Penetrating the Lincoln Legend (Westport, CT: Greenwood Press, 1992), 25,

    Cited passage

    Thanks for trying Questia!

    Please continue trying out our research tools, but please note, full functionality is available only to our active members.

    Your work will be lost once you leave this Web page.

    For full access in an ad-free environment, sign up now for a FREE, 1-day trial.

    Already a member? Log in now.