Updated Robots and Crawlers List

Today we publish our updated robots and crawlers list that can be used with Advanced Robots.txt Generator Professional.

You can download our complete list from http://www.basisoft.com/botslist.html

In order to use the most recent list, download the XML document, and place it in the installation directory of Advanced Robots.txt Generator.

Let us know of any other additions so they can be used in the next update.

robots.txt