CERTIFICATE_VERIFY_FAILED in Python 3.6, the official installer for macOS

The contents of this article are all written in ** Important Information ** of the installer, but I will leave it because I am addicted to it if I do not notice it.

problem

Using Python 3.6 installed with the official installer for macOS distributed on python.org, when trying to get the https: // web page with ʻurllib.request.urlopen () `, I get the following error: Occurs.

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1318, in do_open
    encode_chunked=req.has_header('Transfer-encoding'))
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1239, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1285, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1234, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1026, in _send_output
    self.send(msg)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 964, in send
    self.connect()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/http/client.py", line 1400, in connect
    server_hostname=server_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 401, in wrap_socket
    _context=self, _session=session)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 808, in __init__
    self.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 1061, in do_handshake
    self._sslobj.do_handshake()
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/ssl.py", line 683, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 526, in open
    response = self._open(req, data)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 544, in _open
    '_open', req)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 504, in _call_chain
    result = func(*args)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1361, in https_open
    context=self._context, check_hostname=self._check_hostname)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/urllib/request.py", line 1320, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:749)>

Other modules that use the ssl module should get the same error when validating the server certificate.

Cause

Since OpenSSL installed by default on macOS is too old, since Python 3.6, the installer for macOS includes OpenSSL and the system's OpenSSL is no longer referenced.

As a result, the root certificate installed in the OS is not referenced, and the root certificate is not included in the state immediately after installation [^ 1]. As a result, TLS server certificate validation fails.

approach

The following command will download the certifi module and reference the root certificate contained therein.

$ /Applications/Python\ 3.6/Install\ Certificates.command

Before execution:

$ ls -l /Library/Frameworks/Python.framework/Versions/3.6/etc/openssl/

After execution:

$ ls -l /Library/Frameworks/Python.framework/Versions/3.6/etc/openssl/
total 8
lrwxr-xr-x  1 orange  admin  52  3 22 23:00 cert.pem -> ../../lib/python3.6/site-packages/certifi/cacert.pem

In this case, it says to subscribe to the certifi project mailing list to properly update as the root certificate renews.

from now on

I don't want the current situation where users have to update their certificates individually, so DSAS Developer's Room: Recent Python-dev (2017-03) /archives/2017-03/python-dev-201703.html) Make OS certificates available using TLS implementations other than OpenSSL PEP 543 I think it will lead to the story of org / dev / peps / pep-0543 /).

reference

[^ 1]: pip ships with a root certificate, so pip install works.