Is a slash equivalent to an encoded slash 2F in the path portion of an HTTP URL

Question

I have a site that treats  quot   quot  and  quot  2F quot  in the path portion  not the query string  of a URL differently   Is this a bad thing to do according to either the RFC or the real world  I ask because I keep running into little surprises with the web framework I m using  Ruby on Rails  as well as the layers below that  Passenger  Apache  e g   I had to enable  quot ALLOW ENCODED SLASHES quot  for Apache    I am now leaning toward getting rid of the encoded slashes completely  but I wonder if I should be filing bug reports where I see weird behavior involving the encoded slashes  As to why I have the encoded slashes in the first place  basically I have routes such as this   controller  foo  bar  where  foo is something like a path that can contain slashes   I thought the most straightforward thing to do would be to just URL escape foo so the slashes are ignored by the routing mechanism   Now I am having doubts  and it s pretty clear that the frameworks don t really support this  but according to the RFC is it wrong to do it this way  Here is some information I have gathered  RFC 1738  URLs    Usually a URL has the same interpretation when an octet is represented by a character and when it encoded  However  this is not true for reserved characters  encoding a character reserved for a particular scheme may change the semantics of a URL   RFC 2396  URIs    These characters are called  quot reserved quot   since their usage within the URI component is limited to their reserved purpose  If the data for a URI component would conflict with the reserved purpose  then the conflicting data must be escaped before forming the URI    does escaping here mean something other than encoding the reserved character   RFC 2616  HTTP 1 1    Characters other than those in the  quot reserved quot  and  quot unsafe quot  sets  see RFC 2396  42   are equivalent to their  quot  quot   quot  HEX HEX quot  encoding   There is also this bug report for Rails  where they seem to expect the encoded slash to behave differently   Right  I d expect different results because they re pointing at different resources  It s looking for the literal file  foo bar  in the root directory  The non escaped version is looking for the file bar within directory foo   It s clear from the RFCs that raw vs  encoded is the equivalent for unreserved characters  but what is the story for reserved characters

User · Answer

I also have a site that has numerous urls with urlencoded characters. I am finding that many web APIs (including Google webmaster tools and several Drupal modules) trip over urlencoded characters. Many APIs automatically decode urls at some point in their process and then use the result as a URL or HTML. When I find one of these problems, I usually double encode the results (which turns %2f into %252f) for that API. However, this will break other APIs which are not expecting double encoding, so this is not a universal solution.

Personally I am getting rid of as many special characters in my URLs as possible.

Also, I am using id numbers in my URLs which do not depend on urldecoding:

example.com/blog/my-amazing-blog%2fstory/yesterday

becomes:

example.com/blog/12354/my-amazing-blog%2fstory/yesterday

in this case, my code only uses 12354 to look for the article, and the rest of the URL gets ignored by my system (but is still used for SEO.) Also, this number should appear BEFORE the unused URL components. that way, the url will still work, even if the %2f gets decoded incorrectly.

Also, be sure to use canonical tags to ensure that url mistakes don't translate into duplicate content.

User · Answer

From the data you gathered  I would tend to say that encoded     in an uri are meant to be seen as     again at application cgi level   That s to say  that if you re using apache with mod rewrite for instance  it will not match pattern  expecting slashes against URI with encoded slashes in it  However  once the appropriate module cgi     is called to handle the request  it s up to it to do the decoding and  for instance  retrieve a parameter including slashes as the first component of the URI   If your application is then using this data to retrieve a file  whose filename contains a slash   that s probably a bad thing   To sum up  I find it perfectly normal to see a difference of behaviour in     or   2F  as their interpretation will be done at different levels

User · Answer

The story of  2F vs   was that  according to the initial W3C recommendations  slashes   must imply a hierarchical structure     The slash   quot   quot   ASCII 2F hex  character is reserved for the delimiting of substrings whose relationship is hierarchical  This enables partial forms of the URI  Example 2 The URIs http   www w3 org albert bertram marie-claude and http   www w3 org albert bertram 2Fmarie-claude are NOT identical  as in the second case the encoded slash does not have hierarchical significance

User · Answer

What to do if  foo in its natural form contains slashes   You wouldn t want it to  Isn t that the distinction the recommendation is attempting to preserve   It specifically notes      The similarity to unix and other disk operating system filename conventions should be taken as purely coincidental  and should not be taken to indicate that URIs should be interpreted as file names    If one was building an online interface to a backup program  and wished to express the path as a part of the URL path  it would make sense to encode the slashes in the file path  as that is not really part of the hierarchy of the resource - and more importantly  the route    backups 2016-07-28content  home dan  loses the root of the filesystem in the double slash   Escaping the slashes is the appropriate way to distinguish  as I read it

User · Answer

encodeURI   decodeURI and encodeURIComponent   decodeURIComponent are utility functions to handle this  Read more here https   stackabuse com javascripts-encodeuri-function

User · Answer

If you use Tomcat  add  -Dorg apache tomcat util buf UDecoder ALLOW ENCODED SLASH true  in VM properties   https   tomcat apache org tomcat-7 0-doc config systemprops html Security

[http] Is a slash ("/") equivalent to an encoded slash ("%2F") in the path portion of an HTTP URL

Examples related to http

Examples related to url

Examples related to encoding