Definition: The term crawler is just another word for spider and robot. These are basically software programs that crawl the Web and index site information for search engine databases.
Next word: Dead Link
Also Known As: spider, robot
Crawler
By Wendy Boswell, About.com
How to Use Crawlers to Your Advantage
Blogs and Search Engine OptimizationSEO ToolsGoogle Search Engine Optimization On A Budget
Explore Web Search
Must Reads
More from About.com
Work Hard, Travel Easy
The best tips for business travelers.Dog Care
Get tips on training and caring for dogs of all ages.48 Hours in Vegas
Plan a hot weekend in Sin City.Hotel Advice
The must-stay hotels worldwide.
©2008 About.com, a part of The New York Times Company.
All rights reserved.