Do search engines search the entire Web?
In a word, no. Search engines have millions of pages in their databases (Google alone indexes close to eight billion), but none of them even come close to indexing the whole Web, let alone the Internet. What are some pages that they exclude?
- The invisible web,, a vast repository of information that search engines don't have access to, such as databases
- Private networks, called intranets, that are not actually hooked up to the Web
- Forms, like ColdFusion or CGI
- Password-protected sites, like a university library
- Sites that intentionally, for various reasons, keep their information from being indexed by search engine spiders
