setting request headers in selenium

Question

I m attempting to set the request header  Referer  to spoof a request coming from another site  We need the ability test that a specific referrer is used  which returns a specific form to the user  otherwise an alternative form is given   I can do this within poltergeist by   page driver headers     Referer    gt  referer string    but I can t find the equivalent functionality for the selemium driver   How can I set request headers in the capybara selenium driver

User · Answer

With the solutions already discussed above the most reliable one is using Browsermob-Proxy

But while working with the remote grid machine, Browsermob-proxy isn't really helpful.

This is how I fixed the problem in my case. Hopefully, might be helpful for anyone with a similar setup.

Add the ModHeader extension to the chrome browser

How to download the Modheader? Link

ChromeOptions options = new ChromeOptions();
options.addExtensions(new File(C://Downloads//modheader//modheader.crx));

// Set the Desired capabilities 
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(ChromeOptions.CAPABILITY, options);

// Instantiate the chrome driver with capabilities
WebDriver driver = new RemoteWebDriver(new URL(YOUR_HUB_URL), options);

Go to the browser extensions and capture the Local Storage context ID of the ModHeader

Navigate to the URL of the ModHeader to set the Local Storage Context

.

// set the context on the extension so the localStorage can be accessed
driver.get("chrome-extension://idgpnmonknjnojddfkpgkljpfnnfcklj/_generated_background_page.html");

Where `idgpnmonknjnojddfkpgkljpfnnfcklj` is the value captured from the Step# 2

Now add the headers to the request using Javascript

.

   ((Javascript)driver).executeScript(
         "localStorage.setItem('profiles', JSON.stringify([{  title: 'Selenium', hideComment: true, appendMode: '', 
             headers: [                        
               {enabled: true, name: 'token-1', value: 'value-1', comment: ''},
               {enabled: true, name: 'token-2', value: 'value-2', comment: ''}
             ],                          
             respHeaders: [],
             filters: []
          }]));");

Where token-1, value-1, token-2, value-2 are the request headers and values that are to be added.

Now navigate to the required web-application.

driver.get("your-desired-website");

User · Answer

I had the same issue  I solved it downloading modify-headers firefox add-on and activate it with selenium   The code in python is the following  fp   webdriver FirefoxProfile   path modify header    C  xxxxxxx modify headers-0 7 1 1-fx xpi  fp add extension path modify header   fp set preference  modifyheaders headers count   1  fp set preference  modifyheaders headers action0    Add   fp set preference  modifyheaders headers name0    Name of header     Set here the name of the header fp set preference  modifyheaders headers value0    value of header     Set here the value of the header fp set preference  modifyheaders headers enabled0   True  fp set preference  modifyheaders config active   True  fp set preference  modifyheaders config alwaysOn   True   driver   webdriver Firefox firefox profile fp

User · Answer

For those people using Python  you may consider using Selenium Wire  which can set request headers  as well as provide you with the ability to inspect requests and responses  from seleniumwire import webdriver    Import from seleniumwire    Create a new instance of the Firefox driver  or Chrome  driver   webdriver Firefox      Create a request interceptor def interceptor request       del request headers  Referer      Delete the header first     request headers  Referer      some referer     Set the interceptor on the driver driver request interceptor   interceptor    All requests will now use  some referer  for the referer driver get  https   mysite

User · Answer

If you use the HtmlUnitDriver  you can set request headers by modifying the WebClient  like so    final case class Header name  String  value  String   final class HtmlUnitDriverWithHeaders headers  Seq Header   extends HtmlUnitDriver     super modifyWebClient       val client   super getWebClient     headers foreach h   gt  client addRequestHeader h name  h value       client         The headers will then be on all requests made by the web browser

User · Answer

Had the same issue today  except that I needed to set different referer per test  I ended up using a middleware and a class to pass headers to it  Thought I d share  or maybe there s a cleaner solution     lib request headers rb   class CustomHeadersHelper   cattr accessor  headers end  class RequestHeaders   def initialize app  helper   nil       app   helper   app  helper   end    def call env      if  helper       headers    helper headers        if headers is a  Hash          headers each do  k v            env  HTTP   k upcase gsub  -             v         end       end     end       app call env    end end     config initializers middleware rb  require  request headers   if  w test cucumber  include  Rails env    Rails application config middleware insert before Rack  Lock   RequestHeaders   CustomHeadersHelper end     spec support capybara headers rb  require  request headers   module CapybaraHeaderHelpers   shared context  navigating within the site  do     before  each    add headers  Referer    gt  Capybara app host            end    def add headers custom headers      if Capybara current driver     rack test       custom headers each do  name  value          page driver browser header name  value        end     else       CustomHeadersHelper headers   custom headers     end   end end     spec spec helper rb      config include CapybaraHeaderHelpers   Then I can include the shared context wherever I need  or pass different headers in another before block  I haven t tested it with anything other than Selenium and RackTest  but it should be transparent  as header injection is done before the request actually hits the application

User · Answer

Webdriver doesn t contain an API to do it  See issue 141 from Selenium tracker for more info  The title of the issue says that it s about response headers but it was decided that Selenium won t contain API for request headers in scope of this issue  Several issues about adding API to set request headers have been marked as duplicates  first  second  third   Here are a couple of possibilities that I can propose    Use another driver library instead of selenium Write a browser-specific plugin  or find an existing one  that allows you to add header for request  Use browsermob-proxy or some other proxy    I d go with option 3 in most of cases  It s not hard   Note that Ghostdriver has an API for it but it s not supported by other drivers

User · Answer

I wanted something a bit slimmer for RSpec Ruby so that the custom code only had to live in one place  Here s my solution    spec support selenium rb     RSpec configure do  config    config after  suite  do      custom headers   nil   end end  module RequestWithExtraHeaders   def headers      custom headers each do  key  value        self set header  HTTP   key    value     end if  custom headers      super   end end class ActionDispatch  Request   prepend RequestWithExtraHeaders end   Then in my specs    specs features something spec rb      custom headers     Referer    gt  referer string

User · Answer

You can do it with PhantomJSDriver   PhantomJSDriver pd     PhantomJSDriver    WebDriverFacade  getDriver    getProxiedDriver     pd executePhantomJS               this onResourceRequested   function request  net                       net setHeader  header-name    header-value                           Using the request object  you can filter also so the header won t be set for every request

User · Answer

check this out  chrome options   Options    chrome options add argument  --headless   chrome options add argument  user-agent  Mozilla 5 0  Macintosh  Intel Mac OS X 10 14 4  AppleWebKit 537 36  KHTML  like Gecko  Chrome 73 0 3683 103 Safari 537 36      Problem solved

[webdriver] setting request headers in selenium

Examples related to webdriver

Examples related to capybara

Examples related to selenium-webdriver