Spider

A spider is a program that visits Web sites and reads their pages and other information in order to create entries for a search engine index. All major search engines on the Web have these kinds of programs, which are also known as "crawlers" or a "bots". Spiders are usually programmed to visit sites that have been marked by their owners as fresh or modernized.