By Romano, Catherine
Management Review , Vol. 85, No. 9
It's no news flash that companies are jumping on the World Wide Web at exponential speeds. Consider the progress that has been made in just two short years. In early 1994, no one knew what a Web site was. In early 1995, a handful of companies started showing up on the Net, and in 1996, there were more than 100,000 commercial sites.
But in 1996, companies also began to realize that, while the Internet was the place to be, it was not the immediate profit center they had envisioned. In fact, says John Nardone, director of media and research services at Modem Media in Westport, Conn., "there were Web sites up and no one was coming."
To attract attention to their pages, companies started buying advertising-placing banners on related sites. In theory, it was a good idea. Well-placed ads would certainly stimulate interest that would potentially generate revenue.
But there were drawbacks--namely the cost. Those selling ad space were the only ones really making any money. In addition to design and production costs, companies were now spending additional advertising dollars on the Web. And still they had very little to show for it. "All of the sudden, between on-line production costs and advertising, companies are spending real money. If they spend $1 million, what's their return on investment?" asks Nardone. "Our clients are spending significant money and they need answers to questions." Specifically, is their Web site effective? What should the Web site be accomplishing and, furthermore, does it?
Marketers have been trying to find the answers by applying traditional principles of market research to the new medium. Focus groups conducted in chat rooms and on-line surveys are the current mainstays, but a few adventurous companies are trying new methods. AT&T's college division wanted to do more than focus groups, so they put students in a "reaction theater," equipped with a large-screen monitor and reaction dials. Moderators could walk participants through the Web site and students could input their immediate reactions and opinions.
There are, however, some very practical considerations. For instance, only 20 percent of the population is on-line, which limits the projectability of the sample. Granted, all subsets of the U.S. adult population are represented to some degree on-line. "They tend to be boomers, wealthier, male, employed, with children, better-educated, regard themselves as smart, fairly social, individualistic," says Charles D'Oyly, director of on-line research at Yankelovich Partners in Norwalk, Conn. But unless a company is working with a very specific target population, the results will not be projectable to the general population.
"There's no point in doing research on anybody unless you know something about the people doing the answers," says Andrew Watt, COO of Cyber Dialogue in New York. If a random sample isn't feasible, marketers have to know how the sample they are working with relates to the general population. For example, if a company chooses to use a sample of 500 mountain bike enthusiasts, it must understand how these cyclists compare to the overall population. Are they older or younger, are there more males than females, and is their income more or less than the average? Then a company can weigh the responses they receive accordingly.
For this reason, says Watt at Cyber Dialogue, most on-line research tends to be qualitative. "It works well with focus groups, where you don't need general population sample," he says. "If you're a manufacturer and need to set strategy for the next three years and you're only seeking 20 percent of the population. This may not be the best place to design strategy."
If a company is willing to accept working with 20 percent of the population, there will still be difficulties in choosing the sample. Because Internet users are so inclusive, companies could end up with self-selected respondents. Some companies have attempted to do their research by buying a list of e-mail names and randomly sending e-mail questionnaires to a portion of that list. …