Searching the Invisible Web
Did you know that most of the information on the Internet is "hidden" in databases? This information is invisible to the standard search engines (Google, AltaVista, Excite, etc). Archived databases of articles, health information, addresses, government documents -- piles of useful information -- are not indexed in standard search engines. In fact, I've seen estimates that the invisible Web is 500 times larger than what we see in search engines. This may seem depressing, most of my searches already seem to return 342,000 + hits. But the invisible Web has a major benefit. Many of these databases are extremely targeted, and they return highly relevant information.
We recently started working with a company called Intelliseek (they've integrated our SurfSaver product into BullsEye Plus, their meta-search application). Intelliseek has been working to bring the invisible Web out into the open through their products:
1. Invisible Web - a Web site directory which catalogs over 10,000 Invisible Web sources.
2. Profusion - an online meta-search site that sends search requests to over 1,000 search engines and databases.
3. BullsEye - a desktop meta-search application that helps you search and organize results from over 800+ databases and search engines.
The first Web site, the Invisibleweb (http://www.invisibleweb.com), is a directory of over 10,000 databases and searchable sources that have been frequently overlooked by traditional search engines. Each source is analyzed and described by editors. At this site, you simply type in the general subject of your search, and you obtain links to many databases, archives, and search engines. The InvisibleWeb provides limited, targeted results that should get you more quickly to the specific information you are seeking.
A second site operated by Intelliseek is ProFusion at: http://www.profusion.com.
Profusion is more than just a directory of Invisible Web databases, Profusion actually lets you search through the invisible Web. You can do a general search or search in specific categories. Profusion.com accesses over 1,000 targeted sources of information, including over 500 Invisible Web sources.
Bullseye is a desktop application that you install on your computer. Similar to the Profusion Web site, BullsEye can search through 800+ targeted sources of information, including all major search engines. BullsEye offers more features and conveniences than Web-based services like Profusion.
In BullsEye, the search results are displayed at the top of the screen, and a built in browser displays the pages at the bottom of the screen. Hits in the retrieved pages are even highlighted. In addition, BullsEye removes duplicates from the search results. Another great feature is the ability to eliminate dead links from your search results. This means no more clicking on a link just to find a "404 File Not Found error". You can even save searches and manage them so you can avoid duplication of effort on searches you commonly perform. If you regularly need new information on a specific topic, you can use BullsEye Tracker to automatically perform scheduled searches for you. You can then access the search results at your convenience. This is extremely useful if your job or hobby involves searching the Web everyday or every week.
For novice users, BullsEye has predefined search categories (for example news, shopping, health, computers, and more). You simply choose one of these categories, and you're able to search specific resources in these areas. For more advanced users, BullsEye allows you to customize what search engines you wish to utilize for every different category of search available in the product. The end result for the user is a listing of search results that will get you exactly what you are looking for nearly every time.
The invisible Web is growing - more databases, directories, and other bits of hidden treasure. I was pleasantly surprised at some of the great resources I found while researching this article. I've included links below to additional resources on the "Invisible Web". Happy searching.
Related Links
http://www.invisibleweb.com
http://www.profusion.com
http://www.surfsaver.com/bullseye/default.asp (special offer for SurfSaver users)
http://gwis2.circ.gwu.edu/%7Egprice/direct.htm
http://websearch.about.com/library/weekly/aa061199.htm
http://www.calvin.edu/library/searreso/internet/as/
http://library.rider.edu/scholarly/rlackie/Invisible/Inv_Web.html
http://cgi.zdnet.com/slink?/adeskb/adt0706/2782865:6601785
|
