Cannot set some HTTP headers when using System Net WebRequest

Question

When I try to add a HTTP header key value pair on a WebRequest object  I get the following exception      This header must be modified using the appropriate property   I ve tried adding new values to the Headers collection by using the Add   method but I still get the same exception   webRequest Headers Add HttpRequestHeader Referer   http   stackoverflow com      I can get around this by casting the WebRequest object to a HttpWebRequest and setting the properties such as httpWebReq Referer   http   stackoverflow com   but this only works for a handful of headers that are exposed via properties   I d like to know if there s a way to get a finer grained control over modifying headers with a request for a remote resource

User · Answer

You can just cast the WebRequest to an HttpWebRequest showed below:

var request = (HttpWebRequest)WebRequest.Create(myUri);

and then instead of trying to manipulate the header list, apply it directly in the request property request.Referer:

request.Referer = "yourReferer";

These properties are available in the request object.

User · Answer

If you need the short and technical answer go right to the last section of the answer   If you want to know better  read it all  and i hope you ll enjoy       I countered this problem too today  and what i discovered today is that    the above answers are true  as   1 1 it s telling you that the header you are trying to add already exist and you should   then modify its value using the appropriate property  the indexer  for instance   instead of  trying to add it again   1 2 Anytime you re changing the headers of an HttpWebRequest  you need to use the appropriate properties on the object itself  if they exist    Thanks FOR and Jvenema for the leading guidelines      But  What i found out  and that was the missing piece in the puzzle is that   2 1 The WebHeaderCollection class is generally accessed through WebRequest Headers or WebResponse Headers  Some common headers are considered restricted and are either exposed directly by the API  such as Content-Type  or protected by the system and cannot be changed    The restricted headers are    Accept Connection Content-Length Content-Type Date Expect Host If-Modified-Since Range Referer Transfer-Encoding User-Agent Proxy-Connection     So  next time you are facing this exception and don t know how to solve this  remember that there are some restricted headers  and the solution is to modify their values using the appropriate property explicitly from the WebRequest HttpWebRequest class     Edit   useful  from comments  comment by user Kaido      Solution is to check if the header exists already or is restricted  WebHeaderCollection IsRestricted key   before calling add

User · Answer

Anytime you re changing the headers of an HttpWebRequest  you need to use the appropriate properties on the object itself  if they exist  If you have a plain WebRequest  be sure to cast it to an HttpWebRequest first  Then Referrer in your case can be accessed via   HttpWebRequest request  Referrer  so you don t need to modify the header directly - just set the property to the right value  ContentLength  ContentType  UserAgent  etc  all need to be set this way   IMHO  this is a shortcoming on MS part   setting the headers via Headers Add   should automatically call the appropriate property behind the scenes  if that s what they want to do

User · Answer

Note  this solution will work with WebClientSocket as well as with HttpWebRequest or any other class that uses WebHeaderCollection to work with headers   If you look at the source code of WebHeaderCollection cs you will see that Hinfo is used to keep information of all known headers    private static readonly HeaderInfoTable HInfo   new HeaderInfoTable      Looking at HeaderInfoTable class  you can notice all the data is stored into hash table   private static Hashtable HeaderHashTable    Further  in static contructor of HeaderInfoTable  you can see all known headers are added in HeaderInfo array and then copied into hashtable   Final look at HeaderInfo class shows the names of the fields   internal class HeaderInfo        internal readonly bool IsRequestRestricted      internal readonly bool IsResponseRestricted      internal readonly HeaderParser Parser                 Note that the HeaderName field is not always valid  and should not        be used after initialization  In particular  the HeaderInfo returned        for an unknown header will not have the correct header name              internal readonly string HeaderName      internal readonly bool AllowMultiValues                  So  with all the above  here is a code that uses reflection to find static Hashtable in HeaderInfoTable class and then changes every request-restricted HeaderInfo inside hash table to be unrestricted             use reflection to remove IsRequestRestricted from headerInfo hash table         Assembly a   typeof HttpWebRequest  Assembly          foreach  FieldInfo f in a GetType  System Net HeaderInfoTable   GetFields BindingFlags NonPublic   BindingFlags Static                         if  f Name     HeaderHashTable                                 Hashtable hashTable   f GetValue null  as Hashtable                  foreach  string sKey in hashTable Keys                                         object headerInfo   hashTable sKey                         Console WriteLine String Format   0    1    sKey  hashTable sKey                         foreach  FieldInfo g in a GetType  System Net HeaderInfo   GetFields BindingFlags NonPublic   BindingFlags Instance                                                  if  g Name     IsRequestRestricted                                                         bool b    bool g GetValue headerInfo                               if  b                                                                g SetValue headerInfo  false                                   Console WriteLine sKey         g Name     changed to false

User · Answer

I ran into this problem with a custom web client  I think people may be getting confused because of multiple ways to do this   When using WebRequest Create   you can cast to an HttpWebRequest and use the property to add or modify a header  When using a WebHeaderCollection you may use the  Add  referer   my url     Ex 1  WebClient client   new WebClient    client Headers Add  referer    http   stackoverflow com    client Headers Add  user-agent    Mozilla 5 0      Ex 2  HttpWebRequest request    HttpWebRequest WebRequest Create url   request Referer    http   stackoverflow com   request UserAgent    Mozilla 5 0   response    HttpWebResponse request GetResponse

User · Answer

WebRequest being abstract  and since any inheriting class must override the Headers property    which concrete WebRequest are you using   In other words  how do you get that WebRequest object to beign with    ehr   mnour answer made me realize that the error message you were getting is actually spot on  it s telling you that the header you are trying to add already exist and you should then modify its value using the appropriate property  the indexer  for instance   instead of trying to add it again  That s probably all you were looking for   Other classes inheriting from WebRequest might have even better properties wrapping certain headers  See this post for instance

User · Answer

I ran into same issue below piece of code worked for me HttpWebRequest request    HttpWebRequest WebRequest Create url    request Headers  quot UserAgent quot      quot Mozilla 5 0  compatible  MSIE 9 0  Windows NT 6 1   Trident 5 0  quot

User · Answer

I ran into same issue below piece of code worked for me HttpWebRequest request    HttpWebRequest WebRequest Create url    request Headers  quot UserAgent quot      quot Mozilla 5 0  compatible  MSIE 9 0  Windows NT 6 1   Trident 5 0  quot

User · Answer

Anytime you re changing the headers of an HttpWebRequest  you need to use the appropriate properties on the object itself  if they exist  If you have a plain WebRequest  be sure to cast it to an HttpWebRequest first  Then Referrer in your case can be accessed via   HttpWebRequest request  Referrer  so you don t need to modify the header directly - just set the property to the right value  ContentLength  ContentType  UserAgent  etc  all need to be set this way   IMHO  this is a shortcoming on MS part   setting the headers via Headers Add   should automatically call the appropriate property behind the scenes  if that s what they want to do

User · Answer

WebRequest being abstract  and since any inheriting class must override the Headers property    which concrete WebRequest are you using   In other words  how do you get that WebRequest object to beign with    ehr   mnour answer made me realize that the error message you were getting is actually spot on  it s telling you that the header you are trying to add already exist and you should then modify its value using the appropriate property  the indexer  for instance   instead of trying to add it again  That s probably all you were looking for   Other classes inheriting from WebRequest might have even better properties wrapping certain headers  See this post for instance

User · Answer

Basically  no  That is an http header  so it is reasonable to cast to HttpWebRequest and set the  Referer  as you indicate in the question    HttpWebRequest req       req Referer    your url

User · Answer

All the previous answers describe the problem without providing a solution  Here is an extension method which solves the problem by allowing you to set any header via its string name    Usage  HttpWebRequest request   WebRequest Create url  as HttpWebRequest  request SetRawHeader  content-type    application json      Extension Class  public static class HttpWebRequestExtensions       static string   RestrictedHeaders   new string                  Accept                Connection                Content-Length                Content-Type                Date                Expect                Host                If-Modified-Since                Keep-Alive                Proxy-Connection                Range                Referer                Transfer-Encoding                User-Agent              static Dictionary lt string  PropertyInfo gt  HeaderProperties   new Dictionary lt string  PropertyInfo gt  StringComparer OrdinalIgnoreCase        static HttpWebRequestExtensions                 Type type   typeof HttpWebRequest           foreach  string header in RestrictedHeaders                        string propertyName   header Replace  -                    PropertyInfo headerProperty   type GetProperty propertyName               HeaderProperties header    headerProperty                       public static void SetRawHeader this HttpWebRequest request  string name  string value                if  HeaderProperties ContainsKey name                         PropertyInfo property   HeaderProperties name               if  property PropertyType    typeof DateTime                   property SetValue request  DateTime Parse value   null               else if  property PropertyType    typeof bool                   property SetValue request  Boolean Parse value   null               else if  property PropertyType    typeof long                   property SetValue request  Int64 Parse value   null               else                 property SetValue request  value  null                     else                       request Headers name    value                      Scenarios  I wrote a wrapper for HttpWebRequest and didn t want to expose all 13 restricted headers as properties in my wrapper  Instead I wanted to use a simple Dictionary lt string  string gt    Another example is an HTTP proxy where you need to take headers in a request and forward them to the recipient    There are a lot of other scenarios where its just not practical or possible to use properties  Forcing the user to set the header via a property is a very inflexible design which is why reflection is needed  The up-side is that the reflection is abstracted away  it s still fast   001 second in my tests   and as an extension method feels natural   Notes  Header names are case insensitive per the RFC  http   www w3 org Protocols rfc2616 rfc2616-sec4 html sec4 2

User · Answer

All the previous answers describe the problem without providing a solution  Here is an extension method which solves the problem by allowing you to set any header via its string name    Usage  HttpWebRequest request   WebRequest Create url  as HttpWebRequest  request SetRawHeader  content-type    application json      Extension Class  public static class HttpWebRequestExtensions       static string   RestrictedHeaders   new string                  Accept                Connection                Content-Length                Content-Type                Date                Expect                Host                If-Modified-Since                Keep-Alive                Proxy-Connection                Range                Referer                Transfer-Encoding                User-Agent              static Dictionary lt string  PropertyInfo gt  HeaderProperties   new Dictionary lt string  PropertyInfo gt  StringComparer OrdinalIgnoreCase        static HttpWebRequestExtensions                 Type type   typeof HttpWebRequest           foreach  string header in RestrictedHeaders                        string propertyName   header Replace  -                    PropertyInfo headerProperty   type GetProperty propertyName               HeaderProperties header    headerProperty                       public static void SetRawHeader this HttpWebRequest request  string name  string value                if  HeaderProperties ContainsKey name                         PropertyInfo property   HeaderProperties name               if  property PropertyType    typeof DateTime                   property SetValue request  DateTime Parse value   null               else if  property PropertyType    typeof bool                   property SetValue request  Boolean Parse value   null               else if  property PropertyType    typeof long                   property SetValue request  Int64 Parse value   null               else                 property SetValue request  value  null                     else                       request Headers name    value                      Scenarios  I wrote a wrapper for HttpWebRequest and didn t want to expose all 13 restricted headers as properties in my wrapper  Instead I wanted to use a simple Dictionary lt string  string gt    Another example is an HTTP proxy where you need to take headers in a request and forward them to the recipient    There are a lot of other scenarios where its just not practical or possible to use properties  Forcing the user to set the header via a property is a very inflexible design which is why reflection is needed  The up-side is that the reflection is abstracted away  it s still fast   001 second in my tests   and as an extension method feels natural   Notes  Header names are case insensitive per the RFC  http   www w3 org Protocols rfc2616 rfc2616-sec4 html sec4 2

User · Answer

I m using just    request ContentType    application json  charset utf-8

User · Answer

I ran into this problem with a custom web client  I think people may be getting confused because of multiple ways to do this   When using WebRequest Create   you can cast to an HttpWebRequest and use the property to add or modify a header  When using a WebHeaderCollection you may use the  Add  referer   my url     Ex 1  WebClient client   new WebClient    client Headers Add  referer    http   stackoverflow com    client Headers Add  user-agent    Mozilla 5 0      Ex 2  HttpWebRequest request    HttpWebRequest WebRequest Create url   request Referer    http   stackoverflow com   request UserAgent    Mozilla 5 0   response    HttpWebResponse request GetResponse

User · Answer

The above answers are all fine  but the essence of the issue is that some headers are set one way  and others are set other ways   See above for  restricted header  lists   FOr these  you just set them as a property   For others  you actually add the header   See here       request ContentType    application x-www-form-urlencoded        request Accept    application json        request Headers Add HttpRequestHeader Authorization   Basic     info clientId         info clientSecret

User · Answer

Basically  no  That is an http header  so it is reasonable to cast to HttpWebRequest and set the  Referer  as you indicate in the question    HttpWebRequest req       req Referer    your url

User · Answer

Basically  no  That is an http header  so it is reasonable to cast to HttpWebRequest and set the  Referer  as you indicate in the question    HttpWebRequest req       req Referer    your url

User · Answer

Anytime you re changing the headers of an HttpWebRequest  you need to use the appropriate properties on the object itself  if they exist  If you have a plain WebRequest  be sure to cast it to an HttpWebRequest first  Then Referrer in your case can be accessed via   HttpWebRequest request  Referrer  so you don t need to modify the header directly - just set the property to the right value  ContentLength  ContentType  UserAgent  etc  all need to be set this way   IMHO  this is a shortcoming on MS part   setting the headers via Headers Add   should automatically call the appropriate property behind the scenes  if that s what they want to do

User · Answer

I had the same exception when my code tried to set the  Accept  header value like this   WebRequest request   WebRequest Create  http   someServer 6405 biprws logon long    request Headers Add  Accept    application json      The solution was to change it to this   HttpWebRequest request    HttpWebRequest WebRequest Create  http   someServer 6405 biprws logon long    request Accept    application json

User · Answer

Basically  no  That is an http header  so it is reasonable to cast to HttpWebRequest and set the  Referer  as you indicate in the question    HttpWebRequest req       req Referer    your url

User · Answer

I had the same exception when my code tried to set the  Accept  header value like this   WebRequest request   WebRequest Create  http   someServer 6405 biprws logon long    request Headers Add  Accept    application json      The solution was to change it to this   HttpWebRequest request    HttpWebRequest WebRequest Create  http   someServer 6405 biprws logon long    request Accept    application json

User · Answer

Anytime you re changing the headers of an HttpWebRequest  you need to use the appropriate properties on the object itself  if they exist  If you have a plain WebRequest  be sure to cast it to an HttpWebRequest first  Then Referrer in your case can be accessed via   HttpWebRequest request  Referrer  so you don t need to modify the header directly - just set the property to the right value  ContentLength  ContentType  UserAgent  etc  all need to be set this way   IMHO  this is a shortcoming on MS part   setting the headers via Headers Add   should automatically call the appropriate property behind the scenes  if that s what they want to do

User · Answer

I m using just    request ContentType    application json  charset utf-8

User · Answer

WebRequest being abstract  and since any inheriting class must override the Headers property    which concrete WebRequest are you using   In other words  how do you get that WebRequest object to beign with    ehr   mnour answer made me realize that the error message you were getting is actually spot on  it s telling you that the header you are trying to add already exist and you should then modify its value using the appropriate property  the indexer  for instance   instead of trying to add it again  That s probably all you were looking for   Other classes inheriting from WebRequest might have even better properties wrapping certain headers  See this post for instance

User · Answer

You can just cast the WebRequest to an HttpWebRequest showed below:

var request = (HttpWebRequest)WebRequest.Create(myUri);

and then instead of trying to manipulate the header list, apply it directly in the request property request.Referer:

request.Referer = "yourReferer";

These properties are available in the request object.

User · Answer

The above answers are all fine  but the essence of the issue is that some headers are set one way  and others are set other ways   See above for  restricted header  lists   FOr these  you just set them as a property   For others  you actually add the header   See here       request ContentType    application x-www-form-urlencoded        request Accept    application json        request Headers Add HttpRequestHeader Authorization   Basic     info clientId         info clientSecret

User · Answer

Note  this solution will work with WebClientSocket as well as with HttpWebRequest or any other class that uses WebHeaderCollection to work with headers   If you look at the source code of WebHeaderCollection cs you will see that Hinfo is used to keep information of all known headers    private static readonly HeaderInfoTable HInfo   new HeaderInfoTable      Looking at HeaderInfoTable class  you can notice all the data is stored into hash table   private static Hashtable HeaderHashTable    Further  in static contructor of HeaderInfoTable  you can see all known headers are added in HeaderInfo array and then copied into hashtable   Final look at HeaderInfo class shows the names of the fields   internal class HeaderInfo        internal readonly bool IsRequestRestricted      internal readonly bool IsResponseRestricted      internal readonly HeaderParser Parser                 Note that the HeaderName field is not always valid  and should not        be used after initialization  In particular  the HeaderInfo returned        for an unknown header will not have the correct header name              internal readonly string HeaderName      internal readonly bool AllowMultiValues                  So  with all the above  here is a code that uses reflection to find static Hashtable in HeaderInfoTable class and then changes every request-restricted HeaderInfo inside hash table to be unrestricted             use reflection to remove IsRequestRestricted from headerInfo hash table         Assembly a   typeof HttpWebRequest  Assembly          foreach  FieldInfo f in a GetType  System Net HeaderInfoTable   GetFields BindingFlags NonPublic   BindingFlags Static                         if  f Name     HeaderHashTable                                 Hashtable hashTable   f GetValue null  as Hashtable                  foreach  string sKey in hashTable Keys                                         object headerInfo   hashTable sKey                         Console WriteLine String Format   0    1    sKey  hashTable sKey                         foreach  FieldInfo g in a GetType  System Net HeaderInfo   GetFields BindingFlags NonPublic   BindingFlags Instance                                                  if  g Name     IsRequestRestricted                                                         bool b    bool g GetValue headerInfo                               if  b                                                                g SetValue headerInfo  false                                   Console WriteLine sKey         g Name     changed to false

User · Answer

WebRequest being abstract  and since any inheriting class must override the Headers property    which concrete WebRequest are you using   In other words  how do you get that WebRequest object to beign with    ehr   mnour answer made me realize that the error message you were getting is actually spot on  it s telling you that the header you are trying to add already exist and you should then modify its value using the appropriate property  the indexer  for instance   instead of trying to add it again  That s probably all you were looking for   Other classes inheriting from WebRequest might have even better properties wrapping certain headers  See this post for instance

User · Answer

If you need the short and technical answer go right to the last section of the answer   If you want to know better  read it all  and i hope you ll enjoy       I countered this problem too today  and what i discovered today is that    the above answers are true  as   1 1 it s telling you that the header you are trying to add already exist and you should   then modify its value using the appropriate property  the indexer  for instance   instead of  trying to add it again   1 2 Anytime you re changing the headers of an HttpWebRequest  you need to use the appropriate properties on the object itself  if they exist    Thanks FOR and Jvenema for the leading guidelines      But  What i found out  and that was the missing piece in the puzzle is that   2 1 The WebHeaderCollection class is generally accessed through WebRequest Headers or WebResponse Headers  Some common headers are considered restricted and are either exposed directly by the API  such as Content-Type  or protected by the system and cannot be changed    The restricted headers are    Accept Connection Content-Length Content-Type Date Expect Host If-Modified-Since Range Referer Transfer-Encoding User-Agent Proxy-Connection     So  next time you are facing this exception and don t know how to solve this  remember that there are some restricted headers  and the solution is to modify their values using the appropriate property explicitly from the WebRequest HttpWebRequest class     Edit   useful  from comments  comment by user Kaido      Solution is to check if the header exists already or is restricted  WebHeaderCollection IsRestricted key   before calling add

[c#] Cannot set some HTTP headers when using System.Net.WebRequest

Examples related to c#

Examples related to header

Examples related to webrequest