What is The Problem With Search Engines?
The standard search engines setup crawlers that index websites and create a database of information that are then used to provide results for a search.
The problem is that the crawlers, originally called spiders (web) only index standard websites. But there is another level, called the deep web, that crawlers do not index. This is the deep web also called deepnet, the invisible web, or the hidden web.
Estimates are that the surface web are at the 167 terabyte level, the library of congress by comparison is 3000 terabytes in size (as of 1997). The deep web is 7,500 terabytes in size.
The Number of Web Sites
What is federated search?
Federated Search is the simultaneous search of multiple online databases or web resources and is an emerging feature of automated, web-based library and information retrieval systems. Microsoft Windows 7 adds Federated Search to its windows Search 4.0.
Windows Search Create osdx
With Federated Search one can search for items in remote locations from within explorer by using connectors. It is based on Open Search and RSS. You use connectors for the remote locations. For example if you have the connectors you can use Twitter or Flickr within explorer.
How to use Federated Search
To work with federated search one can add a connection to explorer. There are several available:
•ISOHunt.com •Yahoo Image •Technet-Edge • MSDN Channel9 • Bing •eBay • SevenForum’s • Amazon Neowin •Neowin •Flickr Search •YouTube Search •Google News Search •Google Blogs Search •Twitter Search •MSN Search •TechRepublic •Wired
Add the Flickr connector to Windows 7 Explorer
Download the Flickr Connection
Save the File to Local Disk
Install the Connector to Windows 7 Explorer
Flickr Connector Installed
Use the Flickr search
Results of the Flickr Search
Federated search with Windows 7 is just getting started. As more sites add connectors, the ability to expand into the deep web will be enhanced. In 2009, this is just the beginning.
Sources:
http://www.therawfeed.com/2008/02/how-big-is-web-155583825-sites-report.html
http://www.infotoday.com/linkup/lud031506-goldsborough.shtml
http://www.nytimes.com/2009/02/23/technology/internet/23search.html
http://quod.lib.umich.edu/cgi/t/text/text-idx?c=jep;view=text;rgn=main;idno=3336451.0007.104
Connectors
Flickr: http://www.istartedsomething.com/flickrsearch/
Deviant Art: http://brandonlive.com/files/osd/Deviant%20Art.osdx
You Tube: http://www.sevenforums.com/sfp/youtube.osdx
Amazon Search: http://www.sevenforums.com/sfp/amazon.osdx
