Today we publish a new updated robots and crawlers list that can be used with Advanced Robots.txt Generator Professional.
You can download our complete list from http://www.basisoft.com/botslist.html
In order to use the most recent list, download the XML document, and place it in the installation directory of Advanced Robots.txt Generator.
Let us know of any other additions so they can be used in the next update.
1 response so far ↓
Jack // August 15, 2008 at 6:29 am
This XML file does not appear to have any style information associated with it. The document tree is shown below.
Dude there is something wrong with the downloading of the xml file as the above response indicates.
Could some one look into it.
Thanks
Jack