FutureQuest, Inc. FutureQuest, Inc. FutureQuest, Inc.
Robots.txt?
Posted on 31 January 2004 12:46 AM
Question:

I checked my FutureQuest directory: big/dom/xMyDomain/www and see that there is a robots.txt file.

Should the robots.txt file be there?

Can I safely delete the robot.txt file?

Actually, as you can probably guess, I have no idea what the robots.txt file is or does.

Answer:

For information on the robots.txt file you may want to visit the FutureQuest Community Support Forums at http://www.FutureQuest.net/forums/ and do a search for "robots.txt"

You'll also want to view sites such as:
http://www.thesitewizard.com/archive/robotstxt.shtml
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=156449
http://www.rietta.com/robogen/index.shtml

The robots.txt file can be deleted if you like, but we would advise learning about it before doing so. It really can be a valuable tool for you if used properly.