Reply To: Bots …

Keith Saunders
  • Hampshire, England
  • Topics Started 14
  • Total Posts 89

Friendly BOTS are fine provided I have allowed them to crawl the site.

Most BOTS you see will actually be ‘web crawlers’  such as Google and Bing

Web Crawler bots use data provided by the webmaster (me) of the web site which is provided by a combination of a site map, robot.txt and the webmaster accounts on sites like Google.

Currently, I have set all crawlers not to index the forum until I have removed all the test posts over the next week.