FutureQuest, Inc. FutureQuest, Inc. FutureQuest, Inc.
Proper method of specifying Error Documents in .htaccess?
Posted on 26 January 2004 09:36 AM
Question:

I want to specify a 403 Error Document for spiders that view my web site. What is the proper method of configuring my .htaccess file for this?

Answer:

When specifying any custom or other error documents in .htaccess never use your full domain path as this may result in a looping situation:

***WRONG***
ErrorDocument 403 http://www.example.com/403.html
***WRONG***

Some of of the more error prone spiders will loop (MSIECrawler comes to mind) infinitely until someone stops it from either end...

Spider Loop Scenario:
spider --> index.html
denied : return 403 (see 403.html)
spider --> 403.html
denied : return status 403 (see 403.html)
spider --> 403.html
denied : return status 403 (see 403.html)
spider --> 403.html
denied : return status 403 (see 403.html)
and on into infinity

To avoid this the correct code would be:

***RIGHT***
ErrorDocument 403 /403.html
***RIGHT***

This will avoid the redirect and send back the 403.html page **without** redirecting...