|
Spiders are automatic programs that search engine indices use to collect the web site pages information and include into their databases. Spiders find web pages either by working through pages submitted to them on purpose of improving rankings, or by following links between web pages. They are often called by quirky names; for example, Google's spider is called 'googlebot', Yahoo!'s is 'slurp', AltaVista's is 'scooter'.
Spiders have difficulty in following certain HTML code.As a matter of fact ,they could be considered as very primitive browsers. Spiders cannot follow JavaScript links or index content in images. Professional search engine optimization should remove these obstacles and revise code that allows spiders to freely roam the web site. |
