Definition: The term crawler is just another word for spider and robot. These are basically software programs that crawl the Web and index site information for search engine databases.
Next word: Dead Link
Also Known As: spider, robot
Crawler
By Wendy Boswell, About.com
How to Use Crawlers to Your Advantage
Blogs and Search Engine OptimizationSEO ToolsGoogle Search Engine Optimization On A Budget
Explore Web Search
Must Reads
About.com Special Features
Make Meals Easy
An easy way to put delicious meals together.Build Your Business
Get tips on starting up and staying competitive.Love Your Living Space
Achieve your dream home with advice from the pros.Financial Planning
Stay on top with our smart financial tips.
©2009 About.com, a part of The New York Times Company.
All rights reserved.