[python] How to save and load cookies using Python + Selenium WebDriver

How can I save all cookies in Python's Selenium WebDriver to a .txt file, and then load them later?

The documentation doesn't say much of anything about the getCookies function.

This question is related to python python-2.7 selenium webdriver

The answer is


Based on the answer by Eduard Florinescu, but with newer code and the missing imports added:

$ cat work-auth.py
#!/usr/bin/python3

# Setup:
# sudo apt-get install chromium-chromedriver
# sudo -H python3 -m pip install selenium

import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options

chrome_options = Options()
chrome_options.add_argument("--user-data-dir=chrome-data")
driver = webdriver.Chrome('/usr/bin/chromedriver',options=chrome_options)
chrome_options.add_argument("user-data-dir=chrome-data")
driver.get('https://www.somedomainthatrequireslogin.com')
time.sleep(30)  # Time to enter credentials
driver.quit()

$ cat work.py
#!/usr/bin/python3

import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options

chrome_options = Options()
chrome_options.add_argument("--user-data-dir=chrome-data")
driver = webdriver.Chrome('/usr/bin/chromedriver',options=chrome_options)
driver.get('https://www.somedomainthatrequireslogin.com')  # Already authenticated
time.sleep(10)
driver.quit()

Remember, you can only add a cookie for the current domain.

If you want to add a cookie for your Google account, do

browser.get('http://google.com')
for cookie in cookies:
    browser.add_cookie(cookie)

Just a slight modification for the code written by Roel Van de Paar, as all credit goes to him. I am using this in Windows and it is working perfectly, both for setting and adding cookies:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

chrome_options = Options()
chrome_options.add_argument("--user-data-dir=chrome-data")
driver = webdriver.Chrome('chromedriver.exe',options=chrome_options)
driver.get('https://web.whatsapp.com')  # Already authenticated
time.sleep(30)

This is a solution that saves the profile directory for Firefox (similar to the user-data-dir (user data directory) in Chrome) (it involves manually copying the directory around. I haven't been able to find another way):

It was tested on Linux.


Short version:

  • To save the profile
driver.execute_script("window.close()")
time.sleep(0.5)
currentProfilePath = driver.capabilities["moz:profile"]
profileStoragePath = "/tmp/abc"
shutil.copytree(currentProfilePath, profileStoragePath,
                ignore_dangling_symlinks=True
                )
  • To load the profile
driver = Firefox(executable_path="geckodriver-v0.28.0-linux64",
                 firefox_profile=FirefoxProfile(profileStoragePath)
                )

Long version (with demonstration that it works and a lot of explanation -- see comments in the code)

The code uses localStorage for demonstration, but it works with cookies as well.

#initial imports

from selenium.webdriver import Firefox, FirefoxProfile

import shutil
import os.path
import time

# Create a new profile

driver = Firefox(executable_path="geckodriver-v0.28.0-linux64",
                  # * I'm using this particular version. If yours is
                  # named "geckodriver" and placed in system PATH
                  # then this is not necessary
                )

# Navigate to an arbitrary page and set some local storage
driver.get("https://DuckDuckGo.com")
assert driver.execute_script(r"""{
        const tmp = localStorage.a; localStorage.a="1";
        return [tmp, localStorage.a]
    }""") == [None, "1"]

# Make sure that the browser writes the data to profile directory.
# Choose one of the below methods
if 0:
    # Wait for some time for Firefox to flush the local storage to disk.
    # It's a long time. I tried 3 seconds and it doesn't work.
    time.sleep(10)

elif 1:
    # Alternatively:
    driver.execute_script("window.close()")
    # NOTE: It might not work if there are multiple windows!

    # Wait for a bit for the browser to clean up
    # (shutil.copytree might throw some weird error if the source directory changes while copying)
    time.sleep(0.5)

else:
    pass
    # I haven't been able to find any other, more elegant way.
    #`close()` and `quit()` both delete the profile directory


# Copy the profile directory (must be done BEFORE driver.quit()!)
currentProfilePath = driver.capabilities["moz:profile"]
assert os.path.isdir(currentProfilePath)
profileStoragePath = "/tmp/abc"
try:
    shutil.rmtree(profileStoragePath)
except FileNotFoundError:
    pass

shutil.copytree(currentProfilePath, profileStoragePath,
                ignore_dangling_symlinks=True # There's a lock file in the
                                              # profile directory that symlinks
                                              # to some IP address + port
               )

driver.quit()
assert not os.path.isdir(currentProfilePath)
# Selenium cleans up properly if driver.quit() is called,
# but not necessarily if the object is destructed


# Now reopen it with the old profile

driver=Firefox(executable_path="geckodriver-v0.28.0-linux64",
               firefox_profile=FirefoxProfile(profileStoragePath)
              )

# Note that the profile directory is **copied** -- see FirefoxProfile documentation
assert driver.profile.path!=profileStoragePath
assert driver.capabilities["moz:profile"]!=profileStoragePath

# Confusingly...
assert driver.profile.path!=driver.capabilities["moz:profile"]
# And only the latter is updated.
# To save it again, use the same method as previously mentioned

# Check the data is still there

driver.get("https://DuckDuckGo.com")

data = driver.execute_script(r"""return localStorage.a""")
assert data=="1", data

driver.quit()

assert not os.path.isdir(driver.capabilities["moz:profile"])
assert not os.path.isdir(driver.profile.path)

What doesn't work:

  • Initialize Firefox(capabilities={"moz:profile": "/path/to/directory"}) -- the driver will not be able to connect.
  • options=Options(); options.add_argument("profile"); options.add_argument("/path/to/directory"); Firefox(options=options) -- same as above.

When you need cookies from session to session, there is another way to do it. Use the Chrome options user-data-dir in order to use folders as profiles. I run:

# You need to: from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("user-data-dir=selenium") 
driver = webdriver.Chrome(chrome_options=chrome_options)
driver.get("www.google.com")

Here you can do the logins that check for human interaction. I do this and then the cookies I need now every time I start the Webdriver with that folder everything is in there. You can also manually install the Extensions and have them in every session.

The second time I run, all the cookies are there:

# You need to: from selenium.webdriver.chrome.options import Options    
chrome_options = Options()
chrome_options.add_argument("user-data-dir=selenium") 
driver = webdriver.Chrome(chrome_options=chrome_options)
driver.get("www.google.com") # Now you can see the cookies, the settings, extensions, etc., and the logins done in the previous session are present here. 

The advantage is you can use multiple folders with different settings and cookies, Extensions without the need to load, unload cookies, install and uninstall Extensions, change settings, change logins via code, and thus no way to have the logic of the program break, etc.

Also, this is faster than having to do it all by code.


This is code I used in Windows. It works.

for item in COOKIES.split(';'):
    name,value = item.split('=', 1)
    name=name.replace(' ', '').replace('\r', '').replace('\n', '')
    value = value.replace(' ', '').replace('\r', '').replace('\n', '')
    cookie_dict={
            'name':name,
            'value':value,
            "domain": "",  # Google Chrome
            "expires": "",
            'path': '/',
            'httpOnly': False,
            'HostOnly': False,
            'Secure': False
        }
    self.driver_.add_cookie(cookie_dict)

Examples related to python

programming a servo thru a barometer Is there a way to view two blocks of code from the same file simultaneously in Sublime Text? python variable NameError Why my regexp for hyphenated words doesn't work? Comparing a variable with a string python not working when redirecting from bash script is it possible to add colors to python output? Get Public URL for File - Google Cloud Storage - App Engine (Python) Real time face detection OpenCV, Python xlrd.biffh.XLRDError: Excel xlsx file; not supported Could not load dynamic library 'cudart64_101.dll' on tensorflow CPU-only installation

Examples related to python-2.7

Numpy, multiply array with scalar Not able to install Python packages [SSL: TLSV1_ALERT_PROTOCOL_VERSION] How to create a new text file using Python Could not find a version that satisfies the requirement tensorflow Python: Pandas pd.read_excel giving ImportError: Install xlrd >= 0.9.0 for Excel support Display/Print one column from a DataFrame of Series in Pandas How to calculate 1st and 3rd quartiles? How can I read pdf in python? How to completely uninstall python 2.7.13 on Ubuntu 16.04 Check key exist in python dict

Examples related to selenium

SessionNotCreatedException: Message: session not created: This version of ChromeDriver only supports Chrome version 81 session not created: This version of ChromeDriver only supports Chrome version 74 error with ChromeDriver Chrome using Selenium Selenium: WebDriverException:Chrome failed to start: crashed as google-chrome is no longer running so ChromeDriver is assuming that Chrome has crashed WebDriverException: unknown error: DevToolsActivePort file doesn't exist while trying to initiate Chrome Browser Class has been compiled by a more recent version of the Java Environment How to configure ChromeDriver to initiate Chrome browser in Headless mode through Selenium? How to make Firefox headless programmatically in Selenium with Python? element not interactable exception in selenium web automation Selenium Web Driver & Java. Element is not clickable at point (x, y). Other element would receive the click How do you fix the "element not interactable" exception?

Examples related to webdriver

WebDriverException: unknown error: DevToolsActivePort file doesn't exist while trying to initiate Chrome Browser Selenium Web Driver & Java. Element is not clickable at point (x, y). Other element would receive the click How do you fix the "element not interactable" exception? ImportError: No module named 'selenium' how to use List<WebElement> webdriver How to get attribute of element from Selenium? Selenium Finding elements by class name in python Open web in new tab Selenium + Python When running WebDriver with Chrome browser, getting message, "Only local connections are allowed" even though browser launches properly How can I start InternetExplorerDriver using Selenium WebDriver