Strategies for Natural Language Processing

Anatole V. Gershman Schlumberger-Doll Research Center

Natural-language communication with computers is one of the most practical and at the same time one of the most difficult problems of computer science. The first thing that we have to do in such a communication is to look at the natural-language input and decide what this text means. Natural-language understanding involves two components: knowledge about concepts being communicated and knowledge about the ways these concepts are encoded in language. Without these two components, understanding is impossible. A person who has never heard of American football will not be able to understand a radio report on a football game even if it is given in that person's native language. Conversely, no one can understand a sentence in an unfamiliar language. What kinds of knowledge do we need to understand a natural-language text, how much knowledge, and how to apply it? These are the central problems of natural-language processing.

There are many ways of attacking the problem of natural-language understanding. At one end of the spectrum are analyzers that read the input sentences from left to right, very closely following every twist in syntax, trying to interpret every bit of information contained in the sentence. In most cases, these analyzers separate the syntactic and semantic parts of the analysis into separate consecutive stages, paying much more attention to the syntactic part at the expense of semantics ( Ginsparg, 1978; Marcus, 1977).

At the other end of the spectrum are the analyzers that skim through the text looking for certain types of information and paying attention only to the words and expressions relevant to the task ( DeJong, 1979, Schank, Lebovitz, &

-177-