SOME recent articles and blog postings are finally starting to challenge the Google search religion. I've provided a bunch of links for you in the webliography at the end of this column. The real issue is one of search engine spam and the serving up of questionable results. This is a real issue for any kind of searcher, learner, educator, or researcher. It runs the risk of ruining the usefulness of search engines and, in particular, Google. Some scenarios could happen:
1. Google could become a massive dark hole of lousy content driven by the needs of advertisers, marketers, and special interest groups. Users may or may not notice.
2. New competitors could arrive and address these weaknesses and create market options for search that drive improvements across the board. Perhaps Bing or blekko are already starting this.
3. Search engine optimizers could become regulated or self-manage to address the threats to their own interests.
4. Content reputation management systems that have been tried over the years (like a Good Housekeeping Seal) that have been tried over the years may finally come alive.
5. Recommendation systems that rely on the value of the recommender, your own social connections or respected groups, leaders, or professions could influence the relevancy of search results. This shows some potential in recommendations tied to your own contacts in such environments as Facebook; Linkedln; StumbleUpon; Digg, Inc.; Quora; or even a renewed Delicious. Peer recommendations are already working better in music, movies, and recreational reading than they are in the research and question and answer space.
Each of the above potential opportunity scenarios has some chance of occurring. Some are desirable goals, but most also run the risk of being double-edged swords. While you could get better answers under some scenarios, it comes at a cost of narrowness, a dependence on group-sourcing answers, and/or a reduction in innovative thought and serendipity. So, what to do?
I'd suggest that what is most important, in the near term, is to build credulity skills in learners and researchers about what's behind the results they get from web search engines. To do this, we must add a greater dimension to the teaching of searching and information literacies. We must move beyond the teaching of raw searching skills and the retrieval of information, simple content quality evaluations, and the narrowly based search training for media literacy to avoid the dangerous, prurient, and gambling aspects of the web. These skills are important, but there are more fundamental insights that can be gained by understanding the business models behind search engines. Learners and researchers should know and be able to ask themselves who or what chooses to promote that link on the pages of search results they are seeing. Are those links driven by simple mathematical relevancy or a search algorithm? Are special interest groups, political parties, individuals, lobbyists, or commercial advertising interests determining the results searchers are finding?
So, here are some insights into what we need to be teaching, in addition to all the good stuff we're already doing now.
First, every hour, 1 million spam pages of content are created. Spammers are out to harm users, steal publisher traffic, and defraud legitimate advertisers. A new search engine has created a spam clock to highlight this issue (see Figure 1).
For starters, every searcher should know who creates spam pages, why, and how they influence search results. For instance, did you know that Yahoo! owns one of the largest content and article creation companies that are designed to drive traffic to advertisers? Can you name the other majors? This is an important issue. These so-called content farms are companies such as Demand Media and Answers.com. Each creates thousands of pieces of content per day. This …