Magazine article Information Today

The Little Search Engine That Could: HotBot Is a Powerful Alternative to 'Dumbed-Down' Online Utilities

Magazine article Information Today

The Little Search Engine That Could: HotBot Is a Powerful Alternative to 'Dumbed-Down' Online Utilities

Article excerpt

Martha Williams, program chair at the National Online Meeting this past May in New York, asked me once again to chair a session, and, as usual, I agreed. Generally, Martha will try to find a subject I'm interested in; she truly succeeded this time. It was titled "Internet and Search Engines: Producer Views," which was to be followed the next day with "Internet and Search Engines: User Views."

I must admit, I was a bit wary about my session. Martha managed to get speakers from Inktomi Corp. (makers of HotBot), Lycos, AltaVista, and Excite to fill a half-day session. Each was scheduled to speak about his company's search engine for at least 45 minutes, which is a long time to listen to product reviews. Lucky for us, this wasn't the case.

The speakers turned out to be top non-marketing types. They were, of course, self-serving but willing to reveal the present state of their search engines and--most importantly--their intentions for the future. Only one speaker--AltaVista's Bob Lehmenkuler--backed out that morning. Fortunately, Infoseek (which should have been included in the first place) came to the rescue at the last minute and filled in brilliantly. Not surprisingly, AltaVista got a lot of flack for pulling out. Many who came especially to hear them and get answers to some knotty questions were disappointed.

The Scalability Dilemma

The session--standing room only, I'm glad to say--turned out to be the best I've chaired and attended at National Online. Why? Because the speakers were so honest and open about their products that it was possible to detect some important trends to watch for. First, the lineup: Besides the Infoseek presenter, there were Eric Brewer, chief technology officer at HotBot's Inktomi Corporation; Mark Simmer, vice president of online publishing at Lycos, Inc.; and Graham Spencer, chief technology officer and founder of Excite, Inc.

All three were excellent, but Eric Brewer, the first speaker, stole the show. First, he surprised me by saying that HotBot now covered more of the text of the Internet and World Wide Web than any of the other search engines. I had always thought that was AltaVista's big advantage. Apparently that advantage had quickly disappeared, again pointing out today's volatile environment and what is now being called "hypercompetition." Brewer maintained that scalability was a big factor and that HotBot was technologically ahead of the others. He said the HotBot architecture was different and thus better able to handle both the increase of Internet users and text.

The other speakers didn't refute his boast. In fact, Lycos' Simmer said they would solve the scalability problem with "customization." Excite's Graham Spencer proposed a solution that was stated in terms of "caching and pruning." AltaVista took a beating the next day at the User View session when it was revealed that a report had shown that only 600 of 6,000 pages of promised text had been picked up by AltaVista. Scalability problems? You betcha!

What caught my ear in particular was Brewer's statement that HotBot was targeting the "serious user" (his words). This must have delighted NOM's mostly information-professional audience who are unlike those who only occasionally access the Internet for quick answers.

On the other hand, Simmer and Spencer both emphasized that they were targeting the "consumer." Simmer spoke of the "quick-answer" consumer and the "entertainment" area and suggested that, for them, Boolean searching would be good enough. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed


An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.