[python] How to deal with certificates using Selenium?

I am using Selenium to launch a browser. How can I deal with the webpages (URLs) that will ask the browser to accept a certificate or not?

In Firefox, I may have a website like that asks me to accept its certificate like this:

Firefox

On the Internet Explorer browser, I may get something like this:

Enter image description here

On Google Chrome:

Google Chrome

I repeat my question: How can I automate the acceptance of a website's certificate when I launch a browser (Internet Explorer, Firefox and Google Chrome) with Selenium (Python programming language)?

This question is related to python selenium python-3.x selenium-webdriver browser

The answer is


It looks like it still doesn't have a standard decision of this problem. In other words - you still can't say "Okay, do a certification, whatever if you are Internet Explorer, Mozilla or Google Chrome". But I found one post that shows how to work around the problem in Mozilla Firefox. If you are interested in it, you can check it here.


For Firefox:

ProfilesIni profile = new ProfilesIni();
FirefoxProfile myprofile = profile.getProfile("default");
myprofile.setAcceptUntrustedCertificates(true);
myprofile.setAssumeUntrustedCertificateIssuer(true);
WebDriver driver = new FirefoxDriver(myprofile);

For Chrome we can use:

DesiredCapabilities capabilities = DesiredCapabilities.chrome();
capabilities.setCapability("chrome.switches", Arrays.asList("--ignore-certificate-errors"));
driver = new ChromeDriver(capabilities);

For Internet Explorer we can use:

DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);      
Webdriver driver = new InternetExplorerDriver(capabilities);

Delete all but the necessary certificate from your browser's certificate store and then configure the browser to automatically select the certificate when only one certificate is present.


Creating a profile and then a driver helps us get around the certificate issue in Firefox:

var profile = new FirefoxProfile();
profile.SetPreference("network.automatic-ntlm-auth.trusted-uris","DESIREDURL");
driver = new FirefoxDriver(profile);

Javascript:

const capabilities = webdriver.Capabilities.phantomjs();
capabilities.set(webdriver.Capability.ACCEPT_SSL_CERTS, true);
capabilities.set(webdriver.Capability.SECURE_SSL, false);
capabilities.set('phantomjs.cli.args', ['--web-security=no', '--ssl-protocol=any', '--ignore-ssl-errors=yes']);
const driver = new webdriver.Builder().withCapabilities(webdriver.Capabilities.chrome(), capabilities).build();

Just an update regarding this issue.

Require Drivers:

Linux: Centos 7 64bit, Window 7 64bit

Firefox: 52.0.3

Selenium Webdriver: 3.4.0 (Windows), 3.8.1 (Linux Centos)

GeckoDriver: v0.16.0 (Windows), v0.17.0 (Linux Centos)

Code

System.setProperty("webdriver.gecko.driver", "/home/seleniumproject/geckodrivers/linux/v0.17/geckodriver");

ProfilesIni ini = new ProfilesIni();


// Change the profile name to your own. The profile name can 
// be found under .mozilla folder ~/.mozilla/firefox/profile. 
// See you profile.ini for the default profile name

FirefoxProfile profile = ini.getProfile("default"); 

DesiredCapabilities cap = new DesiredCapabilities();
cap.setAcceptInsecureCerts(true);

FirefoxBinary firefoxBinary = new FirefoxBinary();

GeckoDriverService service =new GeckoDriverService.Builder(firefoxBinary)
    .usingDriverExecutable(new 
File("/home/seleniumproject/geckodrivers/linux/v0.17/geckodriver"))
    .usingAnyFreePort()
    .usingAnyFreePort()
    .build();
try {
    service.start();
} catch (IOException e) {
    e.printStackTrace();
}

FirefoxOptions options = new FirefoxOptions().setBinary(firefoxBinary).setProfile(profile).addCapabilities(cap);

driver = new FirefoxDriver(options);
driver.get("https://www.google.com");

System.out.println("Life Title -> " + driver.getTitle());
driver.close();

For people coming to this question related to headless chrome via python selenium, you may find https://bugs.chromium.org/p/chromium/issues/detail?id=721739#c102 to be useful.

It looks like you can either do

chrome_options = Options()
chrome_options.add_argument('--allow-insecure-localhost')

or something along the lines of the following (may need to adapt for python):

ChromeOptions options = new ChromeOptions()
DesiredCapabilities caps = DesiredCapabilities.chrome()
caps.setCapability(ChromeOptions.CAPABILITY, options)
caps.setCapability("acceptInsecureCerts", true)
WebDriver driver = new ChromeDriver(caps)

And in C# (.net core) using Selenium.Webdriver and Selenium.Chrome.Webdriver like this:

ChromeOptions options = new ChromeOptions();
options.AddArgument("--ignore-certificate-errors");
using (var driver = new ChromeDriver(Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location),options))
{ 
  ...
}

    ChromeOptions options = new ChromeOptions().addArguments("--proxy-server=http://" + proxy);
    options.setAcceptInsecureCerts(true);

Whenever I run into this issue with newer browsers, I just use AppRobotic Personal edition to click specific screen coordinates, or tab through the buttons and click.

Basically it's just using its macro functionality, but won't work on headless setups though.


In selenium python, you need to set desired_capabilities as:

desired_capabilities = {
    "acceptInsecureCerts": True
}

WebDriverManager.chromedriver().setup();
ChromeOptions options = new ChromeOptions();
options.addArguments("--ignore-certificate-errors");
driver = new ChromeDriver(options);

I have used it for Java with Chrome browser it is working nice


I was able to do this on .net c# with PhantomJSDriver with selenium web driver 3.1

 [TestMethod]
    public void headless()
    {


        var driverService = PhantomJSDriverService.CreateDefaultService(@"C:\Driver\phantomjs\");
        driverService.SuppressInitialDiagnosticInformation = true;
        driverService.AddArgument("--web-security=no");
        driverService.AddArgument("--ignore-ssl-errors=yes");
        driver = new PhantomJSDriver(driverService);

        driver.Navigate().GoToUrl("XXXXXX.aspx");

        Thread.Sleep(6000);
    }

I had the exact same issue. However when I tried opening the website manually in the browser the certificate was correct, but in the details the name was "DONOTTRUST".

The difference of certificate was caused by Fiddler that was running in background and decrypting all HTTPS content before reencrypting it.

To fix my problem, just close Fiddler on machine. If you need to keep Fiddler opened, then you can uncheck Decrypt SSL in Fiddler Settings.


I ran into the same issue with Selenium and Behat. If you want to pass the parameters via behat.yml, here is what it needs to look like:

default:
    extensions:
        Behat\MinkExtension:
            base_url: https://my-app.com
            default_session: selenium2
            selenium2:
                browser: firefox
                capabilities:
                    extra_capabilities:
                        acceptInsecureCerts: true

For Firefox Python:

The Firefox Self-signed certificate bug has now been fixed: accept ssl cert with marionette firefox webdrive python splinter

"acceptSslCerts" should be replaced by "acceptInsecureCerts"

from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary

caps = DesiredCapabilities.FIREFOX.copy()
caps['acceptInsecureCerts'] = True
ff_binary = FirefoxBinary("path to the Nightly binary")

driver = webdriver.Firefox(firefox_binary=ff_binary, capabilities=caps)
driver.get("https://expired.badssl.com")

For those who come to this issue using Firefox and the above solutions don't work, you may try the code below (my original answer is here).

from selenium import webdriver

profile = webdriver.FirefoxProfile()
profile.DEFAULT_PREFERENCES['frozen']['marionette.contentListener'] = True
profile.DEFAULT_PREFERENCES['frozen']['network.stricttransportsecurity.preloadlist'] = False
profile.DEFAULT_PREFERENCES['frozen']['security.cert_pinning.enforcement_level'] = 0
profile.set_preference('webdriver_assume_untrusted_issuer', False)
profile.set_preference("browser.download.folderList", 2)
profile.set_preference("browser.download.manager.showWhenStarting", False)
profile.set_preference("browser.download.dir", temp_folder)
profile.set_preference("browser.helperApps.neverAsk.saveToDisk",
                   "text/plain, image/png")
driver = webdriver.Firefox(firefox_profile=profile)

Examples related to python

programming a servo thru a barometer Is there a way to view two blocks of code from the same file simultaneously in Sublime Text? python variable NameError Why my regexp for hyphenated words doesn't work? Comparing a variable with a string python not working when redirecting from bash script is it possible to add colors to python output? Get Public URL for File - Google Cloud Storage - App Engine (Python) Real time face detection OpenCV, Python xlrd.biffh.XLRDError: Excel xlsx file; not supported Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation

Examples related to selenium

SessionNotCreatedException: Message: session not created: This version of ChromeDriver only supports Chrome version 81 session not created: This version of ChromeDriver only supports Chrome version 74 error with ChromeDriver Chrome using Selenium Selenium: WebDriverException:Chrome failed to start: crashed as google-chrome is no longer running so ChromeDriver is assuming that Chrome has crashed WebDriverException: unknown error: DevToolsActivePort file doesn't exist while trying to initiate Chrome Browser Class has been compiled by a more recent version of the Java Environment How to configure ChromeDriver to initiate Chrome browser in Headless mode through Selenium? How to make Firefox headless programmatically in Selenium with Python? element not interactable exception in selenium web automation Selenium Web Driver & Java. Element is not clickable at point (x, y). Other element would receive the click How do you fix the "element not interactable" exception?

Examples related to python-3.x

Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation Replace specific text with a redacted version using Python Upgrade to python 3.8 using conda "Permission Denied" trying to run Python on Windows 10 Python: 'ModuleNotFoundError' when trying to import module from imported package What is the meaning of "Failed building wheel for X" in pip install? How to downgrade python from 3.7 to 3.6 I can't install pyaudio on Windows? How to solve "error: Microsoft Visual C++ 14.0 is required."? Iterating over arrays in Python 3 How to upgrade Python version to 3.7?

Examples related to selenium-webdriver

SessionNotCreatedException: Message: session not created: This version of ChromeDriver only supports Chrome version 81 Selenium: WebDriverException:Chrome failed to start: crashed as google-chrome is no longer running so ChromeDriver is assuming that Chrome has crashed WebDriverException: unknown error: DevToolsActivePort file doesn't exist while trying to initiate Chrome Browser How to configure ChromeDriver to initiate Chrome browser in Headless mode through Selenium? How to make Firefox headless programmatically in Selenium with Python? Selenium Web Driver & Java. Element is not clickable at point (x, y). Other element would receive the click How do you fix the "element not interactable" exception? Scrolling to element using webdriver? Only local connections are allowed Chrome and Selenium webdriver Check if element is clickable in Selenium Java

Examples related to browser

How to force reloading a page when using browser back button? How do we download a blob url video How to prevent a browser from storing passwords How to Identify Microsoft Edge browser via CSS? Edit and replay XHR chrome/firefox etc? Communication between tabs or windows How do I render a Word document (.doc, .docx) in the browser using JavaScript? "Proxy server connection failed" in google chrome Chrome - ERR_CACHE_MISS How to check View Source in Mobile Browsers (Both Android && Feature Phone)