Tips for debugging htaccess rewrite rules

Question

Many posters have problems debugging their RewriteRule and RewriteCond statements within their  htaccess files   Most of these are using a shared hosting service and therefore don t have access to the root server configuration  They cannot avoid using  htaccess files for rewriting and cannot enable a RewriteLogLevel quot  as many respondents suggest   Also there are many  htaccess-specific pitfalls and constraints are aren t covered well   Setting up a local test LAMP stack involves too much of a learning curve for most  So my Q here is how would we recommend that they debug their rules themselves   I provide a few suggestions below   Other suggestions would be appreciated   Understand that the mod rewrite engine cycles through  htaccess files  The engine runs this loop  do   execute server and vhost rewrites  in the Apache Virtual Host Config    find the lowest  quot Per Dir quot   htaccess file on the file path with rewrites enabled   if found  htaccess       execute  htaccess rewrites  in the user s directory  while rewrite occurred  So your rules will get executed repeatedly and if you change the URI path then it may end up executing other  htaccessfiles if they exist  So make sure that you terminate this loop  if necessary by adding extra RewriteCond to stop rules firing   Also delete any lower level  htaccess rewrite rulesets unless explicitly intent to use multi-level rulesets   Make sure that the syntax of each Regexp is correct by testing against a set of test patterns to make sure that is a valid syntax and does what you intend with a fully range of test URIs   See answer below for more details   Build up your rules incrementally in a test directory   You can make use of the  quot execute the deepest  htaccess file on the path feature quot  to set up a separate test directory  tree  and debug rulesets here without screwing up your main rules and stopping your site working   You have to add them one at a time because this is the only way to localise failures to individual rules   Use a dummy script stub to dump out server and environment variables   See Listing 2 If your app uses  say  blog index php then you can copy this into test blog index php and use it to test out your blog rules in the test subdirectory   You can also use environment variables to make sure that the rewrite engine in interpreting substitution strings correctly  e g  RewriteRule       -  E TEST0   DOCUMENT ROOT  blog html cache  1 html   and look for these REDIRECT   variables in the phpinfo dump   BTW  I used this one and discovered on my site that I had to use   ENV DOCUMENT ROOT REAL  instead   In the case of redirector looping  REDIRECT REDIRECT   variables list the previous pass  Etc    Make sure that you don t get bitten by your browser caching incorrect 301 redirects   See answer below   My thanks to Ulrich Palha for this   The rewrite engine seems sensitive to cascaded rules within an  htaccess context   that is where a RewriteRule results in a substitution and this falls though to further rules   as I found bugs with internal sub-requests  1   and incorrect PATH INFO processing which can often be prevents by use of the  NS    L  and  PT  flags    Any more comment or suggestions  Listing 1 -- phpinfo  lt  php phpinfo INFO ENVIRONMENT INFO VARIABLES

User · Answer

If you are working with url  You might want to check if you  Enable Mod Rewrite

User · Answer

Make sure that the syntax of each Regexp is correct by testing against a set of test patterns to make sure that is a valid syntax and does what you intend with a fully range of test URIs  See regexpCheck php below for a simple script that you can add to a private test directory in your site to help you do this   I ve kept this brief rather than pretty   Just past this into a file regexpCheck php in a test directory to use it on your website   This will help you build up any regexp and test it against a list of test cases as you do so   I am using the PHP PCRE engine here  but having had a look at the Apache source  this is basically identical to the one used in Apache   There are many HowTos and tutorials which provide templates and can help you build your regexp skills  Listing 1 -- regexpCheck php  lt html gt  lt head gt  lt title gt Regexp checker lt  title gt  lt  head gt  lt body gt   lt  php       a pattern  isset   POST  pattern        POST  pattern      quot  quot        a ntests   isset   POST  ntests        POST  ntests     1       a test     isset   POST  test        POST  test     array              res   array     maxM -1       foreach  a test as  t             rtn    preg match      a pattern      t  m           if  rtn    1                maxM max  maxM count  m                 res   array merge  array  matched      m              else                res   array   rtn     FALSE    invalid     non-matched                        gt   lt p gt  amp nbsp   lt  p gt   lt form method  quot post quot  action  quot  lt  php echo   SERVER  SCRIPT NAME     gt  quot  gt       lt label for  quot pl quot  gt Regexp Pattern   lt  label gt       lt input id  quot p quot  name  quot pattern quot  size  quot 50 quot  value  quot  lt  php echo htmlentities  a pattern ENT QUOTES  quot UTF-8 quot      gt  quot    gt       lt label for  quot n quot  gt  amp nbsp   amp nbsp  Number of test vectors   lt  label gt       lt input id  quot n quot  name  quot ntests quot   size  quot 3 quot  value  quot  lt  php echo  a ntests   gt  quot   gt       lt input type  quot submit quot  name  quot go quot  value  quot OK quot   gt  lt hr  gt  lt p gt  amp nbsp  lt  p gt       lt table gt  lt thead gt  lt tr gt  lt td gt  lt b gt Test Vector lt  b gt  lt  td gt  lt td gt  amp nbsp   amp nbsp   lt b gt Result lt  b gt  lt  td gt   lt  php      for    i 0   i lt  maxM   i     echo  quot  lt td gt  amp nbsp   amp nbsp   lt b gt    i lt  b gt  lt  td gt  quot       echo  quot  lt  tr gt  lt tbody gt  n quot       for   i 0   i lt  a ntests   i              echo   lt tr gt  lt td gt  amp nbsp  lt input name  quot test   quot  value  quot                 htmlentities  a test  i   ENT QUOTES  quot UTF-8 quot     quot    gt  lt  td gt            foreach   res  i  as  v    echo   lt td gt  amp nbsp   amp nbsp    htmlentities  v  ENT QUOTES  quot UTF-8 quot     amp nbsp   amp nbsp   lt  td gt             echo  quot  lt  tr gt  n quot           gt   lt  table gt  lt  form gt  lt  body gt  lt  html gt

User · Answer

One from a couple of hours that I wasted   If you ve applied all these tips and are only going on 500 errors because you don t have access to the server error log  maybe the problem isn t in the  htaccess but in the files it redirects to     After I had fixed my  htaccess-problem I spent two more hours trying to fix it some more  even though I simply had forgotten about some permissions

User · Answer

Regarding 4   you still need to ensure that your  dummy script stub  is actually the target URL after all the rewriting is done  or you won t see anything   A similar related trick  see this question  is to insert a temporary rule such as   RewriteRule       show php url  1  END    Where show php is some very simple script that just displays its   GET parameters  you can display environment variables too  if you want    This will stop the rewriting at the point you insert it into the ruleset  rather like a breakpoint in a debugger   If you re using Apache  lt 2 3 9  you ll need to use  L  rather than  END   and you may then need to add   RewriteRule  show php  -  L    At the very top of your ruleset  if the URL  show php is itself being rewritten

User · Answer

Similar to Doin idea  To show what is being matched  I use this code   keys   array keys   GET   foreach  keys as  i  gt  key       echo   i   gt   key  lt br gt        Save it to r php on the server root and then do some tests in  htaccess For example  i want to match urls that do not start with a language prefix  RewriteRule      en de          r php  1 amp  2  L    1 amp  2 amp     RewriteRule         r php nomatch  L   report nomatch and exit

User · Answer

I found this question while trying to debug my mod rewrite issues  and it definitely has some helpful advice  But in the end the most important thing is to make sure you have your regex syntax correct  Due to problems with my own RE syntax  installing the regexpCheck php script was not a viable option   But since Apache uses Perl-Compatible Regular Expressions  PCRE s  any tool which helps writing PCREs should help  I ve used RegexPlanet s tool with Java and Javascript REs in the past  and was happy to find that they support Perl as well   Just type in your regular expression and one or more example URLs  and it will tell you if the regex matches  a  1  in the      column  and if applicable  any matching groups  the numbers in the  split  column will correspond to the numbers Apache expects  e g   1   2 etc   for each URL  They claim PCRE support is  in beta   but it was just what I needed to solve my syntax problems   http   www regexplanet com advanced perl index html  I d have simply added a comment to an existing answer but my reputation isn t yet at that level  Hope this helps someone

User · Answer

Make sure you use the percent sign in front of variables  not the dollar sign   It s   HTTP HOST   not   HTTP HOST   There will be nothing in the error log  there will be no Internal Server Errors  your regexp is still correct  the rule will just not match  This is really hideous if you work with django   genshi templates a lot and have     for variable substitution in muscle memory

User · Answer

Don t forget that in  htaccess files it is a relative URL that is matched   In a  htaccess file the following RewriteRule will never match   RewriteRule             something  s

User · Answer

Here are a few additional tips on testing rules that may ease the debugging for users on shared hosting  1  Use a Fake-user agent  When testing a new rule  add a condition to only execute it with a fake user-agent that you will use for your requests   This way it will not affect anyone else on your site   e g    protect with a fake user agent RewriteCond   HTTP USER AGENT    my-fake-user-agent   Here is the actual rule I am testing RewriteCond   HTTP HOST    www  domain  com   NC   RewriteRule   http   www domain com  REQUEST URI   L R 302     If you are using Firefox  you can use the User Agent Switcher to create the fake user agent string and test       2  Do not use 301 until you are done testing  I have seen so many posts where people are still testing their rules and they are using 301 s  DON T    If you are not using suggestion 1 on your site  not only you  but anyone visiting your site at the time will be affected by the 301    Remember that they are permanent  and aggressively cached by your browser  Use a 302 instead till you are sure  then change it to a 301   3  Remember that 301 s are aggressively cached in your browser  If your rule does not work and it looks right to you  and you were not using suggestions 1 and 2  then re-test after clearing your browser cache or while in private browsing   4  Use a HTTP Capture tool  Use a HTTP capture tool like Fiddler to see the actual HTTP traffic between your browser and the server    While others might say that your site does not look right  you could instead see and report that all of the images  css and js are returning 404 errors  quickly narrowing down the problem   While others will report that you started at URL A and ended at URL C  you will be able to see that they started at URL A  were 302 redirected to URL B and 301 redirected to URL C  Even if URL C was the ultimate goal  you will know that this is bad for SEO and needs to be fixed   You will be able to see cache headers that were set on the server side  replay requests  modify request headers to test

User · Answer

Online  htaccess rewrite testing I found this Googling for RegEx help  it saved me a lot of time from having to upload new  htaccess files every time I make a small modification  from the site   htaccess tester To test your htaccess rewrite rules  simply fill in the url that you re applying the rules to  place the contents of your htaccess on the larger input area and press  quot Check Now quot  button

User · Answer

as pointed out by  JCastell  the online tester does a good job of testing individual redirects against an  htaccess file   However  more interesting is the api exposed which can be used to batch test a list of urls using a json object   However  to make it more useful  I have written a small bash script file which makes use of curl and jq to submit a list of urls and parse the json response into a CSV formated output with the line number and rule matched in the htaccess file along with the redirected url  making it quite handy to compare a list of urls in a spreadsheet and quickly determine which rules are not working

User · Answer

I ll leave this here  maybe obvious detail  but got me banging my head for hours  be careful using   REQUEST URI  because what  Krist van Besien say in his answer is totally right  but not for the REQUEST URI string  because the out put of this TestString starts with a    So take care   RewriteCond   REQUEST URI    assets                                                                 check this pesky fella right here if missing

User · Answer

If you re creating redirections  test with curl to avoid browser caching issues  Use -I to fetch http headers only  Use -L to follow all redirections

User · Answer

Some mistakes I observed happens when writing  htaccess  Using of        repetitively in multiple rules  using        causes other rules to be impotent in most cases  because it matches all of the url in single hit   So  if we are using rule for this url sapmle url it will also consume this url sapmle url string      L  flag should be used to ensure our rule has done processing     Should know about   Difference in  n and  n   n is matched during   RewriteCond  part and  n is matches on   RewriteRule  part   Working of RewriteBase     The RewriteBase directive specifies the URL prefix to be used for   per-directory  htaccess  RewriteRule directives that substitute a   relative path       This directive is required when you use a relative path in a   substitution in per-directory  htaccess  context unless any of the   following conditions are true       The original request  and the substitution  are underneath the   DocumentRoot  as opposed to reachable by other means  such as Alias     The filesystem path to the directory containing the RewriteRule    suffixed by the relative substitution is also valid as a URL path on   the server  this is rare   In Apache HTTP Server 2 4 16 and later    this directive may be omitted when the request is mapped via Alias or   mod userdir

User · Answer

Set environment variables and use headers to receive them   You can create new environment variables with RewriteRule lines  as mentioned by OP   RewriteRule       -  E TEST0   DOCUMENT ROOT  blog html cache  1 html    But if you can t get a server-side script to work  how can you then read this environment variable  One solution is to set a header   Header set TEST FOOBAR    REDIRECT TEST0 e    The value accepts format specifiers  including the   NAME e specifier for environment variables  don t forget the lowercase e   Sometimes  you ll need to add the REDIRECT  prefix  but I haven t worked out when the prefix gets added and when it doesn t

User · Answer

If you re planning on writing more than just one line of rules in  htacesss  don t even think about trying one of those hot-fix methods to debug it   I have wasted days setting multiple rules  without feedback from LOGs  only to finally give up  I got Apache on my PC  copied the whole site to its HDD  and got the whole rule-set sorted out  using the logs  real fast  Then I reviewed my old rules  which been working  I saw they were not really doing what was desired  A time bomb  given a slightly different address   There are so many pit falls in rewrite rules  it s not a straight logic thing at all  You can get Apache up and running in ten minutes  it s 10MB  good license   NIX WIN MAC ready  even without install  Also  check the header lines of your server and get the same version of Apache from their archive if it s old  My OP is still on 2 0  many things are not supported

[apache] Tips for debugging .htaccess rewrite rules

Examples related to apache

Examples related to .htaccess

Examples related to mod-rewrite