Change user-agent for Selenium web-driver

Question

I have the following code in Python   from selenium webdriver import Firefox from contextlib import closing  with closing Firefox    as browser    browser get url    I would like to print the user-agent HTTP header and possibly change it  Is it possible

User · Answer

This is a short solution to change the request UserAgent on the fly.

Change UserAgent of a request with Chrome

from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

driver = webdriver.Chrome(driver_path)
driver.execute_cdp_cmd('Network.setUserAgentOverride', {"userAgent":"python 2.7", "platform":"Windows"})
driver.get('http://amiunique.org')

then return your useragent:

agent = driver.execute_script("return navigator.userAgent")

Some sources

The source code of webdriver.py from SeleniumHQ (https://github.com/SeleniumHQ/selenium/blob/11c25d75bd7ed22e6172d6a2a795a1d195fb0875/py/selenium/webdriver/chrome/webdriver.py) extends its functionalities through the Chrome Devtools Protocol

def execute_cdp_cmd(self, cmd, cmd_args):
        """
        Execute Chrome Devtools Protocol command and get returned result

We can use the Chrome Devtools Protocol Viewer to list more extended functionalities (https://chromedevtools.github.io/devtools-protocol/tot/Network#method-setUserAgentOverride) as well as the parameters type to use.

User · Answer

To build on Louis s helpful answer     Setting the User Agent in PhantomJS  from selenium import webdriver from selenium webdriver common desired capabilities import DesiredCapabilities     caps   DesiredCapabilities PHANTOMJS caps  phantomjs page settings userAgent      whatever you want  driver   webdriver PhantomJS desired capabilities caps    The only minor issue is that  unlike for Firefox and Chrome  this does not return your custom setting   driver execute script  return navigator userAgent     So  if anyone figures out how to do that in PhantomJS  please edit my answer or add a comment below  Cheers

User · Answer

To build on JJC s helpful answer that builds on Louis s helpful answer     With PhantomJS 2 1 1-windows this line works   driver execute script  return navigator userAgent     If it doesn t work  you can still get the user agent via the log  to build on Mma s answer     from selenium import webdriver import json from fake useragent import UserAgent  dcap   dict DesiredCapabilities PHANTOMJS  dcap  phantomjs page settings userAgent      UserAgent   random  driver   webdriver PhantomJS executable path r your path   desired capabilities dcap  har   json loads driver get log  har   0   message      get the log print  user agent     har  log    entries   0   request    headers   1   value

User · Answer

There is no way in Selenium to read the request or response headers  You could do it by instructing your browser to connect through a proxy that records this kind of information   Setting the User Agent in Firefox  The usual way to change the user agent for Firefox is to set the variable  general useragent override  in your Firefox profile  Note that this is independent from Selenium   You can direct Selenium to use a profile different from the default one  like this   from selenium import webdriver profile   webdriver FirefoxProfile   profile set preference  general useragent override    whatever you want   driver   webdriver Firefox profile    Setting the User Agent in Chrome  With Chrome  what you want to do is use the user-agent command line option  Again  this is not a Selenium thing  You can invoke Chrome at the command line with chrome --user-agent foo to set the agent to the value foo   With Selenium you set it like this   from selenium import webdriver from selenium webdriver chrome options import Options opts   Options   opts add argument  user-agent whatever you want    driver   webdriver Chrome chrome options opts    Both methods above were tested and found to work  I don t know about other browsers   Getting the User Agent  Selenium does not have methods to query the user agent from an instance of WebDriver  Even in the case of Firefox  you cannot discover the default user agent by checking what general useragent override would be if not set to a custom value   This setting does not exist before it is set to some value    Once the browser is started  however  you can get the user agent by executing   agent   driver execute script  return navigator userAgent     The agent variable will contain the user agent

[python] Change user-agent for Selenium web-driver

Examples related to python

Examples related to selenium

Examples related to http-headers

Examples related to user-agent