Validate SSL certificates with Python

Question

I need to write a script that connects to a bunch of sites on our corporate intranet over HTTPS and verifies that their SSL certificates are valid  that they are not expired  that they are issued for the correct address  etc   We use our own internal corporate Certificate Authority for these sites  so we have the public key of the CA to verify the certificates against   Python by default just accepts and uses SSL certificates when using HTTPS  so even if a certificate is invalid  Python libraries such as urllib2 and Twisted will just happily use the certificate   Is there a good library somewhere that will let me connect to a site over HTTPS and verify its certificate in this way   How do I verify a certificate in Python

User · Answer

I have added a distribution to the Python Package Index which makes the match_hostname() function from the Python 3.2 ssl package available on previous versions of Python.

http://pypi.python.org/pypi/backports.ssl_match_hostname/

You can install it with:

pip install backports.ssl_match_hostname

Or you can make it a dependency listed in your project's setup.py. Either way, it can be used like this:

from backports.ssl_match_hostname import match_hostname, CertificateError
...
sslsock = ssl.wrap_socket(sock, ssl_version=ssl.PROTOCOL_SSLv3,
                      cert_reqs=ssl.CERT_REQUIRED, ca_certs=...)
try:
    match_hostname(sslsock.getpeercert(), hostname)
except CertificateError, ce:
    ...

User · Answer

Here s an example script which demonstrates certificate validation   import httplib import re import socket import sys import urllib2 import ssl  class InvalidCertificateException httplib HTTPException  urllib2 URLError       def   init   self  host  cert  reason           httplib HTTPException   init   self          self host   host         self cert   cert         self reason   reason      def   str   self           return   Host  s returned an invalid certificate   s   s n                     self host  self reason  self cert    class CertValidatingHTTPSConnection httplib HTTPConnection       default port   httplib HTTPS PORT      def   init   self  host  port None  key file None  cert file None                               ca certs None  strict None    kwargs           httplib HTTPConnection   init   self  host  port  strict    kwargs          self key file   key file         self cert file   cert file         self ca certs   ca certs         if self ca certs              self cert reqs   ssl CERT REQUIRED         else              self cert reqs   ssl CERT NONE      def  GetValidHostsForCert self  cert           if  subjectAltName  in cert              return  x 1  for x in cert  subjectAltName                            if x 0  lower       dns           else              return  x 0  1  for x in cert  subject                               if x 0  0  lower       commonname        def  ValidateCertificateHostname self  cert  hostname           hosts   self  GetValidHostsForCert cert          for host in hosts              host re   host replace            replace                           if re search    s      host re    hostname  re I                   return True         return False      def connect self           sock   socket create connection  self host  self port           self sock   ssl wrap socket sock  keyfile self key file                                            certfile self cert file                                            cert reqs self cert reqs                                            ca certs self ca certs          if self cert reqs  amp  ssl CERT REQUIRED              cert   self sock getpeercert               hostname   self host split      0  0              if not self  ValidateCertificateHostname cert  hostname                   raise InvalidCertificateException hostname  cert                                                     hostname mismatch     class VerifiedHTTPSHandler urllib2 HTTPSHandler       def   init   self    kwargs           urllib2 AbstractHTTPHandler   init   self          self  connection args   kwargs      def https open self  req           def http class wrapper host    kwargs               full kwargs   dict self  connection args              full kwargs update kwargs              return CertValidatingHTTPSConnection host    full kwargs           try              return self do open http class wrapper  req          except urllib2 URLError  e              if type e reason     ssl SSLError and e reason args 0     1                  raise InvalidCertificateException req host                                                        e reason args 1               raise      https request   urllib2 HTTPSHandler do request   if   name         main         if len sys argv     3          print  usage  python  s CA CERT URL    sys argv 0          exit 2       handler   VerifiedHTTPSHandler ca certs   sys argv 1       opener   urllib2 build opener handler      print opener open sys argv 2   read

User · Answer

pyOpenSSL is an interface to the OpenSSL library  It should provide everything you need

User · Answer

You can use Twisted to verify certificates   The main API is CertificateOptions  which can be provided as the contextFactory argument to various functions such as listenSSL and startTLS   Unfortunately  neither Python nor Twisted comes with a the pile of CA certificates required to actually do HTTPS validation  nor the HTTPS validation logic   Due to a limitation in PyOpenSSL  you can t do it completely correctly just yet  but thanks to the fact that almost all certificates include a subject commonName  you can get close enough   Here is a naive sample implementation of a verifying Twisted HTTPS client which ignores wildcards and subjectAltName extensions  and uses the certificate-authority certificates present in the  ca-certificates  package in most Ubuntu distributions   Try it with your favorite valid and invalid certificate sites      import os import glob from OpenSSL SSL import Context  TLSv1 METHOD  VERIFY PEER  VERIFY FAIL IF NO PEER CERT  OP NO SSLv2 from OpenSSL crypto import load certificate  FILETYPE PEM from twisted python urlpath import URLPath from twisted internet ssl import ContextFactory from twisted internet import reactor from twisted web client import getPage certificateAuthorityMap      for certFileName in glob glob   etc ssl certs   pem          There might be some dead symlinks in there  so let s make sure it s real      if os path exists certFileName           data   open certFileName  read           x509   load certificate FILETYPE PEM  data          digest   x509 digest  sha1             Now  de-duplicate in case the same cert has multiple names          certificateAuthorityMap digest    x509 class HTTPSVerifyingContextFactory ContextFactory       def   init   self  hostname           self hostname   hostname     isClient   True     def getContext self           ctx   Context TLSv1 METHOD          store   ctx get cert store           for value in certificateAuthorityMap values                store add cert value          ctx set verify VERIFY PEER   VERIFY FAIL IF NO PEER CERT  self verifyHostname          ctx set options OP NO SSLv2          return ctx     def verifyHostname self  connection  x509  errno  depth  preverifyOK           if preverifyOK              if self hostname    x509 get subject   commonName                  return False         return preverifyOK def secureGet url       return getPage url  HTTPSVerifyingContextFactory URLPath fromString url  netloc   def done result       print  Done    len result  secureGet  https   google com    addCallback done  reactor run

User · Answer

From release version 2 7 9 3 4 3 on  Python by default attempts to perform certificate validation   This has been proposed in PEP 467  which is worth a read  https   www python org dev peps pep-0476   The changes affect all relevant stdlib modules  urllib urllib2  http  httplib    Relevant documentation   https   docs python org 2 library httplib html httplib HTTPSConnection     This class now performs all the necessary certificate and hostname checks by default  To revert to the previous  unverified  behavior ssl  create unverified context   can be passed to the context parameter    https   docs python org 3 library http client html http client HTTPSConnection     Changed in version 3 4 3  This class now performs all the necessary certificate and hostname checks by default  To revert to the previous  unverified  behavior ssl  create unverified context   can be passed to the context parameter    Note that the new built-in verification is based on the system-provided certificate database  Opposed to that  the requests package ships its own certificate bundle  Pros and cons of both approaches are discussed in the Trust database section of PEP 476

User · Answer

Or simply make your life easier by using the requests library   import requests requests get  https   somesite com   cert   path server crt   verify True    A few more words about its usage

User · Answer

The following code allows you to benefit from all SSL validation checks  e g  date validity  CA certificate chain      EXCEPT a pluggable verification step e g  to verify the hostname or do other additional certificate verification steps   from httplib import HTTPSConnection import ssl   def create custom HTTPSConnection host        def verify cert cert  host             Write your code here           You can certainly base yourself on ssl match hostname           Raise ssl CertificateError if verification fails         print  Host    host         print  Peer cert    cert      class CustomHTTPSConnection HTTPSConnection  object           def connect self               super CustomHTTPSConnection  self  connect               cert   self sock getpeercert               verify cert cert  host       context   ssl create default context       context check hostname   False     return CustomHTTPSConnection host host  context context    if   name         main           try expired badssl com or self-signed badssl com       conn   create custom HTTPSConnection  badssl com       conn request  GET            conn getresponse   read

User · Answer

PycURL does this beautifully   Below is a short example  It will throw a pycurl error if something is fishy  where you get a tuple with error code and a human readable message   import pycurl  curl   pycurl Curl   curl setopt pycurl CAINFO   myFineCA crt   curl setopt pycurl SSL VERIFYPEER  1  curl setopt pycurl SSL VERIFYHOST  2  curl setopt pycurl URL   https   internal stuff     curl perform     You will probably want to configure more options  like where to store the results  etc  But no need to clutter the example with non-essentials   Example of what exceptions might be raised    60   Peer certificate cannot be authenticated with known CA certificates    51   common name  CN something else stuff O Example Corp C SE  does not match  internal stuff      Some links that I found useful are the libcurl-docs for setopt and getinfo    http   curl haxx se libcurl c curl easy setopt html http   curl haxx se libcurl c curl easy getinfo html

User · Answer

M2Crypto can do the validation  You can also use M2Crypto with Twisted if you like  The Chandler desktop client uses Twisted for networking and M2Crypto for SSL  including certificate validation   Based on Glyphs comment it seems like M2Crypto does better certificate verification by default than what you can do with pyOpenSSL currently  because M2Crypto checks subjectAltName field too   I ve also blogged on how to get the certificates Mozilla Firefox ships with in Python and usable with Python SSL solutions

User · Answer

I was having the same problem but wanted to minimize 3rd party dependencies  because this one-off script was to be executed by many users   My solution was to wrap a curl call and make sure that the exit code was 0  Worked like a charm

User · Answer

Jython DOES carry out certificate verification by default  so using standard library modules  e g  httplib HTTPSConnection  etc  with jython will verify certificates and give exceptions for failures  i e  mismatched identities  expired certs  etc   In fact  you have to do some extra work to get jython to behave like cpython  i e  to get jython to NOT verify certs    I have written a blog post on how to disable certificate checking on jython  because it can be useful in testing phases  etc   Installing an all-trusting security provider on java and jython  http   jython xhaus com installing-an-all-trusting-security-provider-on-java-and-jython

[python] Validate SSL certificates with Python

Examples related to python

Examples related to https

Examples related to ssl-certificate

Examples related to verification