The Scholar versus the Invisible

In refuting the findings of Bergman’s size of the unreachable areas of the internet called the “invisible web,” Exploring the Academic Invisible Web argues that that no single library alone will be able to index the invisible web, let alone the Academic Invisible Web (AIW).

The article’s arguments are interesting when used to analyze the effectiveness of search engines, such as Google and its counterpart, Google Scholar. In the author’s opinions, the AIW can be effectively indexed only through cooperation of portals, that is, combining Google with Scirus, BASE, and Vascoda. Only by balancing the strengths and weaknesses of each can the AIW be properly mined.

The article offers a fascinating complement to a recent article, The Depth and Breadth of Google Scholar, which argues that one of Scholar’s weaknesses is its bias towards the sciences. However, that is just the tip of the iceberg; here are others:

(1) Broken Links – What makes Google strong also makes it weak. While citation analysis offers integrity to its sources, it unfortunately has a “blindspot” to sources that are single-paged entities which are put up on the internet solely not for linkage. There are numerous pages like this on the so-called “invisible” web, which Google does not catch, while others such as Yahoo! might.

(2) Currency - While Google Scholar has a plethora of sources, not all of it is recent. In fact, it might be detrimental for the health sciences if new discoveries are left at the bottom of the pile because they are not as cited as much as the older documents.

(3) Law of Diminishing Marginal Utility – With information saturation, the tradeoffs are up to the user’s needs, and requires Rangananthian philosophical debate to resolve. While Google provides a great deal of information, it requires users themselves to sort through the mass of information.

(4) Indexing – Although citation analysis is reliable, there is still too much grey literature floating in cyberspace. Until Google can somehow reach into these areas, it is still an incomplete tool. One possibility is its emulation of the Dublin Core, which can create a generic uniformity for everything posted on the internet. Although only apipedream, imagine the possibilities!

As someone once commented, relying Google for critical information can be like using an “axe instead of a scalpel” in surgery. Although it cannot replace the librarian professional, it does offer a nice complement to the team.