Spider Identification: What Is A Search Engine Spider?
Tuesday January 3, 2006
A spider, also known as a robot or a crawler, is actually just a program that follows, or "crawls", links throughout the Internet, grabbing content from sites and adding it to search engine indexes. Learn more about search engine spiders.
Related: And speaking of search engines, why not try a new search engine in 2006? Pick one from the 100 Search Engines in 100 Days Table of Contents.
Comments
We also decided to write an article slightly more indepth about what is a spider.. We narrow down “crawlers” and even explain how they are used by search engines to roam the World Wide Web via the Internet. I hope our article helps expand the answer to those reading this article. Another article which seems to be of interest to our readers who find what is a spider, is our extremely in-depth article about how Google makes money.