Why am I getting 304 Not Modified error on some links when using HttpWebRequest

Question

Any ideas why on some links that I try to access using HttpWebRequest I am getting  The remote server returned an error   304  Not Modified   in the code   The code I m using is from Jeff s post here  the page seems to have disappeared  see an archive copy at the Wayback Machine    Note the concept of the code is a simple proxy server  so I m pointing my browser at this locally running piece of code  which gets my browsers request  and then proxies it on by creating a new HttpWebRequest  as you ll see in the code   It works great for most sites links  but for some this error comes up   You will see one key bit in the code is where it seems to copy the http header settings from the browser request to it s request out to the site  and it copies in the header attributes   Not sure if the issue is something to do with how it mimics this aspect of the request and then what happens as the result comes back   case  If-Modified-Since      request IfModifiedSince   DateTime Parse listenerContext Request Headers key       break    I get the issue for example from http   en wikipedia org wiki Main Page  PS  UPDATE HERE   Still can t work this out   Basically I can identify 1 link which has an issue  and it seems to work fine  2nd time it gets the error  3rd time OK  4th time gets the error  5th time OK etc   As if there is some state not getting cleared or something in the code   I ve tried to clean up the code a bit using  using  type statements etc   Here s the code   If anyone can spot why every 2nd time I browse to a link like http   newsimg bbc co uk css screen 1 0 16 nol v4 story css  starting at the 2nd time  not the first  via this proxy code I get the error I d love to hear   class Program       static void Main string   args                Proxy p   new Proxy 8080            Thread proxythread   new Thread new ThreadStart p Start            proxythread Start             Console WriteLine  Proxy Started  Press Any Key To Stop               Console ReadKey             p Stop              public class Proxy       private HttpListener  listener      private int  port       public Proxy int port                int defaultport   8080              Setup Thread Pool         System Threading ThreadPool SetMaxThreads 50  1000           System Threading ThreadPool SetMinThreads 50  50               Sanitize Port Number         if  port  lt  1024    port  gt  65535              port   defaultport              Create HttpListener Prefix         string prefix   string Format  http      0     port            port   port              Create HttpListener          listener   new HttpListener             listener Prefixes Add prefix              public void Start                  listener Start             while  true                        HttpListenerContext request   null               try                               request    listener GetContext                        Statistics  by Greg                  int availThreads   -1                  int compPortThreads   -1                  ThreadPool GetAvailableThreads out availThreads  out compPortThreads                   log  INFO   request Request Url ToString     START -      availThreads                          ThreadPool QueueUserWorkItem ProcessRequest  request                             catch  HttpListenerException ex                                log  ERROR    NA    INFO  HttpListenerException -     ex Message                   break                            catch  InvalidOperationException ex                                log  ERROR    NA    INFO  InvalidOperationException -     ex Message                   break                                     public void Stop                  listener Stop               private void log string sev  string uri  string message                Console Out WriteLine Process GetCurrentProcess   Id     -     sev          uri           message              private void ProcessRequest object  listenerContext                 region local variables         HttpWebRequest psRequest                       Request to send to remote web server         HttpWebResponse psResponse                     Response from remote web server                  List lt byte gt  requestBody   new List lt byte gt         Byte array to hold the request s body         List lt byte gt  responseBody   new List lt byte gt        Byte array to hold the response s body         byte   buffer          string uri                endregion          var listenerContext    HttpListenerContext  listenerContext          uri   listenerContext Request Url ToString   Replace string Format    0     port                    Create Interent Request          HttpWebRequest internetRequest    HttpWebRequest WebRequest Create uri            region Build Request Up         internetRequest Method   listenerContext Request HttpMethod          internetRequest ProtocolVersion   listenerContext Request ProtocolVersion          internetRequest UserAgent   listenerContext Request UserAgent          foreach  string key in listenerContext Request Headers AllKeys                        try                               switch  key                                        case  Proxy-Connection                       case  Connection                           internetRequest KeepAlive    listenerContext Request Headers key  ToLower       keep-alive     true   false                          break                       case  Content-Length                           internetRequest ContentLength   listenerContext Request ContentLength64                          break                       case  Content-Type                           internetRequest ContentType   listenerContext Request ContentType                          break                       case  Accept                           internetRequest Accept   listenerContext Request Headers key                           break                       case  Host                           break                       case  Referer                           internetRequest Referer   listenerContext Request Headers key                           break                       case  If-Modified-Since                           internetRequest IfModifiedSince   DateTime Parse listenerContext Request Headers key                            break                       default                          internetRequest Headers Add key  listenerContext Request Headers key                            break                                              catch  Exception ex                                Console WriteLine  Error settup up psRequest object  Error       ex Message     n    ex StackTrace                                    endregion           region Copy content into request         buffer   new byte 1024           using  Stream instream   listenerContext Request InputStream                        int incount   instream Read buffer  0  buffer Length               while  incount  gt  0                                internetRequest GetRequestStream   Write buffer  0  incount                   incount   instream Read buffer  0  buffer Length                                    endregion             Get Internet Response         HttpWebResponse internetResponse   null          try                       using  internetResponse    HttpWebResponse internetRequest GetResponse                                   region Configure Local Response Header Keys                 foreach  string key in internetResponse Headers Keys                                        try                                               switch  key                                                        case  Transfer-Encoding                                   listenerContext Response SendChunked    internetResponse Headers key  ToLower       chunked     true   false                                  break                               case  Content-Length                                   listenerContext Response ContentLength64   internetResponse ContentLength                                  break                               case  Content-Type                                   listenerContext Response ContentType   internetResponse Headers key                                   break                               case  Keep-Alive                                   listenerContext Response KeepAlive   true                                  break                               default                                  listenerContext Response Headers Add key  internetResponse Headers key                                    break                                                                      catch  Exception ex                                                log  ERROR   uri   Error settup up listenerContext Response objects  Error       ex Message     n    ex StackTrace                                                            endregion                  try                                          Transfer the body data from Internet Response to Internal Response                     buffer   new byte 1024                       using  Stream inputStream   internetResponse GetResponseStream                                                  int outcount   inputStream Read buffer  0  buffer Length                           while  outcount  gt  0                                                        listenerContext Response OutputStream Write buffer  0  outcount                               outcount   inputStream Read buffer  0  buffer Length                                                                                     catch  Exception ex                                        log  ERROR   uri   Could not obtain response from URI      ex Message                                     finally                                       listenerContext Response OutputStream Close                                                      catch  Exception ex                          if  ex is InvalidOperationException                      ex is ProtocolViolationException                      ex is WebException                                    log uri   Could not successfully get response      ex GetType       -     ex Message                     listenerContext Response Close                      return                                else   throw                 log  ERROR   uri   Could not successfully get response      ex GetType       -     ex Message               listenerContext Response Close                        And here is an example of what I see - first hit is good  2nd has error     Proxy Started  Press Any Key To Stop    2080 - INFO  http   newsimg bbc co uk 8080 css screen 1 0 16 nol v4 story css   START -  50  2080 - INFO  http   newsimg bbc co uk 8080 css screen 1 0 16 nol v4 story css   START -  50  2080 - ERROR  http   newsimg bbc co uk css screen 1 0 16 nol v4 story css   Could not successfully get response  System Net WebException - The remote server returned an error   304  Not Modified

User · Answer

This is intended behavior   When you make an HTTP request  the server normally returns code 200 OK  If you set If-Modified-Since  the server may return 304 Not modified  and the response will not have the content   This is supposed to be your cue that the page has not been modified   The authors of the class have foolishly decided that 304 should be treated as an error and throw an exception  Now you have to clean up after them by catching the exception every time you try to use If-Modified-Since

User · Answer

It is not an issue it is because of caching    To overcome this add a timestamp to your endpoint call  e g  axios get   api products    After timestamp it should be axios get  api products   Date now     It will resolve your 304 status code

User · Answer

Just pressing F5 is not always working   why   Because your ISP is also caching web data for you   Solution  Force Refresh   Force refresh your browser by pressing CTRL   F5 in Firefox or Chrome to clear ISP cache too  instead of just pressing F5  You then can see 200 response instead of 304 in the browser F12 developer tools network tab   Another trick is to add question mark   at the end of the URL string of the requested page   http   localhost 52199 Customers Create   The question mark will ensure that the browser refresh the request without caching any previous requests   Additionally in Visual Studio you can set the default browser to Chrome in Incognito mode to avoid cache issues while developing  by adding Chrome in  Incognito mode as default browser  see the steps  self illustrated

User · Answer

I think you have not installed these features  see below in picture     I also suffered from this problem some days ago  After installing this feature then I solved it  If you have not installed this feature then installed it   Install Process    go to android studio Tools Android SDK Manager Appearance  amp  Behavior Android SDK

User · Answer

First  this is not an error  The 3xx denotes a redirection  The real errors are 4xx  client error  and 5xx  server error    If a client gets a 304 Not Modified  then it s the client s responsibility to display the resouce in question from its own cache  In general  the proxy shouldn t worry about this  It s just the messenger

[http] Why am I getting "(304) Not Modified" error on some links when using HttpWebRequest?

Examples related to http

Examples related to httpwebrequest

Examples related to http-status-code-304

Examples related to if-modified-since