PEP 493: First draft as a standards track PEP
- now has 2 feature sections, and 3 backport recommendations - features are for a new Python API and environment setting - backports are all for targeting versions < 2.7.9 - Python 3 is excluded from scope
This commit is contained in:
Nick Coghlan
parent
da92741a4b
commit
348bbf8137
339
pep-0493.txt
339
pep-0493.txt
|
|
@ -1,78 +1,91 @@
|
|||
PEP: 493
|
||||
Title: HTTPS verification recommendations for Python 2.7 redistributors
|
||||
Title: HTTPS verification migration tools for Python 2.7
|
||||
Version: $Revision$
|
||||
Last-Modified: $Date$
|
||||
Author: Nick Coghlan <ncoghlan@gmail.com>,
|
||||
Robert Kuska <rkuska@redhat.com>,
|
||||
Marc-André Lemburg <mal@lemburg.com>
|
||||
Status: Draft
|
||||
Type: Informational
|
||||
Type: Standards Track
|
||||
Content-Type: text/x-rst
|
||||
Created: 10-May-2015
|
||||
Python-Version: 2.7.12
|
||||
Post-History: 06-Jul-2015, 11-Nov-2015, 24-Nov-2015
|
||||
|
||||
|
||||
Abstract
|
||||
========
|
||||
|
||||
PEP 476 updated Python's default handling of HTTPS certificates to be
|
||||
appropriate for communication over the public internet. The Python 2.7 long
|
||||
term maintenance series was judged to be in scope for this change, with the
|
||||
new behaviour introduced in the Python 2.7.9 maintenance release.
|
||||
PEP 476 updated Python's default handling of HTTPS certificates in client
|
||||
modules to align with certificate handling in web browsers, by validating
|
||||
that the certificates received belonged to the server the client was attempting
|
||||
to contact. The Python 2.7 long term maintenance series was judged to be in
|
||||
scope for this change, with the new behaviour introduced in the Python 2.7.9
|
||||
maintenance release.
|
||||
|
||||
This PEP provides recommendations to downstream redistributors wishing to
|
||||
provide a smoother migration experience when helping their users to manage
|
||||
this change in Python's default behaviour.
|
||||
|
||||
*Note that this PEP is not currently accepted, so it is a *proposed*
|
||||
recommendation, rather than an active one.*
|
||||
This has created a non-trivial barrier to adoption for affected Python 2.7
|
||||
maintenance releases, so this PEP proposes additional Python 2.7 specific
|
||||
features that allow system administrators and other users to more easily
|
||||
decouple the decision to verify server certificates in HTTPS client modules
|
||||
from the decision to update to newer Python 2.7 maintenance releases.
|
||||
|
||||
|
||||
Rationale
|
||||
=========
|
||||
|
||||
PEP 476 changed Python's default behaviour to better match the needs and
|
||||
expectations of developers operating over the public internet, a category
|
||||
which appears to include most new Python developers. It is the position of
|
||||
the authors of this PEP that this was a correct decision.
|
||||
PEP 476 changed Python's default behaviour to align with expectations
|
||||
established by web browsers in regards to the semantics of HTTPS URLs:
|
||||
starting with Python 2.7.9 and 3.4.3, HTTPS clients in the standard library
|
||||
validate server certificates by default.
|
||||
|
||||
However, it is also the case that this change *does* cause problems for
|
||||
infrastructure administrators operating private intranets that rely on
|
||||
self-signed certificates, or otherwise encounter problems with the new default
|
||||
certificate verification settings.
|
||||
|
||||
The long term answer for such environments is to update their internal
|
||||
certificate management to at least match the standards set by the public
|
||||
internet, but in the meantime, it is desirable to offer these administrators
|
||||
a way to continue receiving maintenance updates to the Python 2.7 series,
|
||||
without having to gate that on upgrades to their certificate management
|
||||
infrastructure.
|
||||
To manage these kinds of situations, web browsers provide users with "click
|
||||
through" warnings that allow the user to add the server's certificate to the
|
||||
browser's certificate store. Network client tools like ``curl`` and ``wget``
|
||||
offer options to switch off certificate checking entirely (by way of
|
||||
``curl --insecure`` and ``wget --no-check-certificate``, respectively).
|
||||
|
||||
PEP 476 did attempt to address this question, by covering how to revert the
|
||||
new settings process wide by monkeypatching the ``ssl`` module to restore the
|
||||
At a different layer of the technology stack, Linux security modules like
|
||||
`SELinux` and `AppArmor`, while enabled by default by distribution vendors,
|
||||
offer relatively straightforward mechanisms for turning them off.
|
||||
|
||||
At the moment, no such convenient mechanisms exist to disable Python's
|
||||
default certificate checking for a whole process.
|
||||
|
||||
PEP 476 did attempt to address this question, by covering how to revert to the
|
||||
old settings process wide by monkeypatching the ``ssl`` module to restore the
|
||||
old behaviour. Unfortunately, the ``sitecustomize.py`` based technique proposed
|
||||
to allow system administrators to disable the feature by default in their
|
||||
Standard Operating Environment definition has been determined to be
|
||||
insufficient in at least some cases. The specific case of interest to the
|
||||
authors of this PEP is the one where a Linux distributor aims to provide
|
||||
their users with a
|
||||
insufficient in at least some cases. The specific case that led to the
|
||||
initial creation of this PEP is the one where a Linux distributor aims to
|
||||
provide their users with a
|
||||
`smoother migration path <https://bugzilla.redhat.com/show_bug.cgi?id=1173041>`__
|
||||
than the standard one provided by consuming upstream CPython 2.7 releases
|
||||
directly, but other potential challenges have also been pointed out with
|
||||
updating embedded Python runtimes and other user level installations of Python.
|
||||
|
||||
Rather than allowing a plethora of mutually incompatibile migration techniques
|
||||
to bloom, this PEP proposes two alternative approaches that redistributors
|
||||
may take when addressing these problems. Redistributors may choose to implement
|
||||
one, both, or neither of these approaches based on their assessment of the
|
||||
needs of their particular userbase.
|
||||
to bloom, this PEP proposes an additional feature to be added to Python 2.7.12
|
||||
to make it easier to revert a process to the past behaviour of skipping
|
||||
certificate validation in HTTPS client modules. It also provides additional
|
||||
recommendations to redistributors backporting these features to versions of
|
||||
Python prior to Python 2.7.9.
|
||||
|
||||
These designs are being proposed as a recommendation for redistributors, rather
|
||||
than as new upstream features, as they are needed purely to support legacy
|
||||
environments migrating from older versions of Python 2.7. Neither approach
|
||||
is being proposed as an upstream Python 2.7 feature, nor as a feature in any
|
||||
version of Python 3 (whether published directly by the Python Software
|
||||
Foundation or by a redistributor).
|
||||
These designs are being proposed purely as tools for helping to manage the
|
||||
transition to the new default certificate handling behaviour in the context
|
||||
of Python 2.7. They are not being proposed as new features for Python 3, as
|
||||
it is expected that the vast majority of client applications affected by this
|
||||
problem without the ability to update the application itself will be Python 2
|
||||
applications.
|
||||
|
||||
It would likely be desirable for a future version of Python 3 to allow default
|
||||
certificate handling for secure protocols to be configurable on a per-protocol
|
||||
basis, but that question is beyond the scope of this PEP.
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
|
|
@ -84,20 +97,20 @@ their customers. The main approaches available are:
|
|||
* Continuing to rebase on new Python 2.7.x releases, while providing no
|
||||
additional assistance beyond the mechanisms defined in PEP 476 in migrating
|
||||
from unchecked to checked hostnames in standard library HTTPS clients
|
||||
* Gating availability of the improved default handling of HTTPS connections on
|
||||
upgrading from Python 2 to Python 3
|
||||
* For Linux distribution vendors, gating availability of the improved default
|
||||
* Gating availability of the changes in default handling of HTTPS connections
|
||||
on upgrading from Python 2 to Python 3
|
||||
* For Linux distribution vendors, gating availability of the changes in default
|
||||
handling of HTTPS connections on upgrading to a new operating system version
|
||||
* Implementing one or both of the design suggestions described in this PEP,
|
||||
* Implementing one or both of the backport suggestions described in this PEP,
|
||||
regardless of the formal status of the PEP
|
||||
|
||||
|
||||
Requirements for capability detection
|
||||
=====================================
|
||||
|
||||
As these recommendations are intended to cover backports to earlier Python
|
||||
As the proposals in this PEP aim to facilitate backports to earlier Python
|
||||
versions, the Python version number cannot be used as a reliable means for
|
||||
detecting them. Instead, the recommendations are defined to allow the presence
|
||||
detecting them. Instead, they are designed to allow the presence
|
||||
or absence of the feature to be determined using the following technique::
|
||||
|
||||
python -c "import ssl; ssl._relevant_attribute"
|
||||
|
|
@ -108,57 +121,99 @@ relevant capability is not available.
|
|||
The marker attributes are prefixed with an underscore to indicate the
|
||||
implementation dependent nature of these capabilities - not all Python
|
||||
distributions will offer them, only those that are providing a multi-stage
|
||||
migration process from the legacy HTTPS handling to the new default behaviour.
|
||||
migration process from the original Python 2.7 HTTPS handling to the new
|
||||
default behaviour.
|
||||
|
||||
Feature: Configuration API
|
||||
==========================
|
||||
|
||||
This change is proposed for inclusion in CPython 2.7.12 and later CPython 2.7.x
|
||||
releases. It consists of a new ``ssl._verify_https_certificates()`` to specify
|
||||
the default handling of HTTPS certificates in standard library client libraries.
|
||||
|
||||
It is not proposed to forward port this change to Python 3, so Python 3
|
||||
applications that need to support skipping certificate verification will still
|
||||
need to define their own suitable security context.
|
||||
|
||||
Feature detection
|
||||
-----------------
|
||||
|
||||
The marker attribute on the ``ssl`` module related to this feature is the
|
||||
``ssl._verify_https_certificates`` function itself.
|
||||
|
||||
Specification
|
||||
-------------
|
||||
|
||||
The ``ssl._verify_https_certificates`` function will work as follows::
|
||||
|
||||
def _verify_https_certificates(enable=True):
|
||||
"""Verify server HTTPS certificates by default?"""
|
||||
global _create_default_https_context
|
||||
if enable:
|
||||
_create_default_https_context = create_default_context
|
||||
else:
|
||||
_create_default_https_context = _create_unverified_context
|
||||
|
||||
If called without arguments, or with ``enable`` set to a true value, then
|
||||
standard library client modules will subsequently verify HTTPS certificates by default, otherwise they will skip verification.
|
||||
|
||||
If called with ``enable`` set to a false value, then standard library client
|
||||
modules will subsequently skip verifying HTTPS certificates by default.
|
||||
|
||||
Security Considerations
|
||||
-----------------------
|
||||
|
||||
The inclusion of this feature will allow security sensitive applications to
|
||||
include the following forward-compatible snippet in their code::
|
||||
|
||||
if hasattr(ssl, "_verify_https_certificates"):
|
||||
ssl._verify_https_certificates()
|
||||
|
||||
Some developers may also choose to opt out of certificate checking using
|
||||
``ssl._verify_https_certificates(enable=False)``. This doesn't introduce any
|
||||
major new security concerns, as monkeypatching the affected internal APIs was
|
||||
already possible.
|
||||
|
||||
|
||||
Recommendation for an environment variable based security downgrade
|
||||
===================================================================
|
||||
Feature: environment based configuration
|
||||
========================================
|
||||
|
||||
Some redistributors may wish to provide a per-application option to disable
|
||||
certificate verification in selected applications that run on or embed CPython
|
||||
without needing to modify the application itself.
|
||||
This change is proposed for inclusion in CPython 2.7.12 and later CPython 2.7.x
|
||||
releases. It consists of a new ``PYTHONHTTPSVERIFY`` environment variable that
|
||||
allows the default verification to be disabled without modifying the
|
||||
application source code (which may not even be available in cases of
|
||||
bytecode-only application distribution)
|
||||
|
||||
In these cases, a configuration mechanism is needed that provides:
|
||||
It is not proposed to forward port this change to Python 3, so Python 3
|
||||
applications that need to support skipping certificate verification will still
|
||||
need to define their own suitable security context.
|
||||
|
||||
* an opt-out model that allows certificate verification to be selectively
|
||||
turned off for particular applications after upgrading to a version of
|
||||
Python that verifies certificates by default
|
||||
* the ability for all users to configure this setting on a per-application
|
||||
basis, rather than on a per-system, or per-Python-installation basis
|
||||
Feature detection
|
||||
-----------------
|
||||
|
||||
This approach may be used for any redistributor provided version of Python 2.7,
|
||||
including those that advertise themselves as providing Python 2.7.9 or later.
|
||||
The marker attribute on the ``ssl`` module related to this feature is:
|
||||
|
||||
|
||||
Required marker attribute
|
||||
-------------------------
|
||||
|
||||
The required marker attribute on the ``ssl`` module when implementing this
|
||||
recommendation is::
|
||||
|
||||
_https_verify_envvar = 'PYTHONHTTPSVERIFY'
|
||||
* the ``ssl._https_verify_envvar`` attribute, giving the name of environment
|
||||
variable affecting the default behaviour
|
||||
|
||||
This not only makes it straightforward to detect the presence (or absence) of
|
||||
the capability, it also makes it possible to programmatically determine the
|
||||
relevant environment variable name.
|
||||
|
||||
Specification
|
||||
-------------
|
||||
|
||||
Recommended modifications to the Python standard library
|
||||
--------------------------------------------------------
|
||||
Rather than always defaulting to the use of ``ssl.create_default_context``,
|
||||
the ``ssl`` module will be modified to:
|
||||
|
||||
The recommended approach to providing a per-application configuration setting
|
||||
for HTTPS certificate verification that doesn't require modifications to the
|
||||
application itself is to:
|
||||
|
||||
* modify the ``ssl`` module to read the ``PYTHONHTTPSVERIFY`` environment
|
||||
variable when the module is first imported into a Python process
|
||||
* read the ``PYTHONHTTPSVERIFY`` environment variable when the module is first
|
||||
imported into a Python process
|
||||
* set the ``ssl._create_default_https_context`` function to be an alias for
|
||||
``ssl._create_unverified_context`` if this environment variable is present
|
||||
and set to ``'0'``
|
||||
* otherwise, set the ``ssl._create_default_https_context`` function to be an
|
||||
alias for ``ssl.create_default_context`` as usual
|
||||
|
||||
|
||||
Example implementation
|
||||
----------------------
|
||||
|
||||
|
|
@ -175,14 +230,13 @@ Example implementation
|
|||
|
||||
_create_default_https_context = _get_https_context_factory()
|
||||
|
||||
|
||||
Security Considerations
|
||||
-----------------------
|
||||
|
||||
Relative to an unmodified version of CPython 2.7.9 or later, this approach
|
||||
does introduce a new downgrade attack against the default security settings
|
||||
that potentially allows a sufficiently determined attacker to revert Python
|
||||
to the vulnerable configuration used in CPython 2.7.8 and earlier releases.
|
||||
Relative to the behaviour in Python 3.4.3+ and Python 2.7.9->2.7.11, this
|
||||
approach does introduce a new downgrade attack against the default security
|
||||
settings that potentially allows a sufficiently determined attacker to revert
|
||||
Python to the default behaviour used in CPython 2.7.8 and earlier releases.
|
||||
However, such an attack requires the ability to modify the execution
|
||||
environment of a Python process prior to the import of the ``ssl`` module,
|
||||
and any attacker with such access would already be able to modify the
|
||||
|
|
@ -191,13 +245,13 @@ behaviour of the underlying OpenSSL implementation.
|
|||
Interaction with Python virtual environments
|
||||
--------------------------------------------
|
||||
|
||||
This setting is read directly from the process environment, and hence works
|
||||
the same way regardless of whether or not the interpreter is being run inside
|
||||
an activated Python virtual environment.
|
||||
The default setting is read directly from the process environment, and hence
|
||||
works the same way regardless of whether or not the interpreter is being run
|
||||
inside an activated Python virtual environment.
|
||||
|
||||
|
||||
Recommendation for backporting to earlier Python versions
|
||||
=========================================================
|
||||
Backporting PEP 476 to earlier Python versions
|
||||
==============================================
|
||||
|
||||
Some redistributors, most notably Linux distributions, may choose to backport
|
||||
the PEP 476 HTTPS verification changes to modified Python versions based on
|
||||
|
|
@ -214,17 +268,21 @@ mechanism is needed that provides:
|
|||
installations that have been explicitly configured to skip verifying HTTPS
|
||||
certificates by default
|
||||
|
||||
This approach should not be used for any Python installation that advertises
|
||||
As it only affects backports to earlier releases of Python 2.7, this change is
|
||||
not proposed for inclusion in upstream CPython, but rather is offered as
|
||||
guidance to redistributors to reduce the likelihood of multiple mutually
|
||||
incompatible approaches to backporting being adopted.
|
||||
|
||||
This approach SHOULD NOT be used for any Python installation that advertises
|
||||
itself as providing Python 2.7.9 or later, as most Python users will have the
|
||||
reasonable expectation that all such environments will validate HTTPS
|
||||
certificates by default.
|
||||
|
||||
|
||||
Required marker attribute
|
||||
-------------------------
|
||||
Feature detection
|
||||
-----------------
|
||||
|
||||
The required marker attribute on the ``ssl`` module when implementing this
|
||||
recommendation is::
|
||||
The marker attribute on the ``ssl`` module related to this feature is::
|
||||
|
||||
_cert_verification_config = '<path to configuration file>'
|
||||
|
||||
|
|
@ -320,7 +378,7 @@ Example implementation
|
|||
Security Considerations
|
||||
-----------------------
|
||||
|
||||
The specific recommendations for the backporting case are designed to work for
|
||||
The specific recommendations for this backporting case are designed to work for
|
||||
privileged, security sensitive processes, even those being run in the following
|
||||
locked down configuration:
|
||||
|
||||
|
|
@ -356,19 +414,108 @@ This setting is scoped by the interpreter installation and affects all Python
|
|||
processes using that interpreter, regardless of whether or not the interpreter
|
||||
is being run inside an activated Python virtual environment.
|
||||
|
||||
Origins of this recommendation
|
||||
------------------------------
|
||||
|
||||
Combining the recommendations
|
||||
=============================
|
||||
This recommendation is based on the backporting approach adopted for Red Hat
|
||||
Enterprise Linux 7.2, as published in the original July 2015 draft of this PEP
|
||||
and described in detail in `this KnowledgeBase article
|
||||
<https://access.redhat.com/articles/2039753>`__. Red Hat's patches implementing
|
||||
this backport for Python 2.7.5 can be found in the `CentOS git repository
|
||||
<https://git.centos.org/commit/rpms!python.git/refs!heads!c7>`__.
|
||||
|
||||
If a redistributor chooses to implement both recommendations, then the
|
||||
environment variable should take precedence over the system-wide configuration
|
||||
setting. This allows the setting to be changed for a given user, virtual
|
||||
environment or application, regardless of the system-wide default behaviour.
|
||||
|
||||
Backporting this PEP to earlier Python versions
|
||||
===============================================
|
||||
|
||||
The configuration file based backport described above is designed to cover
|
||||
backporting the PEP 476 changes to default certificate handling without the
|
||||
additional configuration mechanisms defined in this PEP.
|
||||
|
||||
If this PEP is accepted, then an additional backporting option becomes
|
||||
available, which is to backport the per-process configuration mechanisms
|
||||
defined in this PEP, without backporting the ability to change the default behaviour of the overall Python installation.
|
||||
|
||||
Such a backport would differ from the mechanism proposed in this PEP solely in
|
||||
the default behaviour when ``PYTHONHTTPSVERIFY`` was not set at all: it would
|
||||
continue to default to skipping certificate validation.
|
||||
|
||||
In this case, if the ``PYTHONHTTPSVERIFY`` environment variable is defined, and
|
||||
set to anything *other* than ``'0'``, then HTTPS certificate verification
|
||||
should be enabled.
|
||||
|
||||
|
||||
Feature detection
|
||||
-----------------
|
||||
|
||||
There's no specific attribute indicating that this situation applies. Rather,
|
||||
it is indicated by the ``ssl._verify_https_certificates`` and
|
||||
``ssl._https_verify_envvar`` attributes being present in a Python version that
|
||||
is nominally older than Python 2.7.9.
|
||||
|
||||
Specification
|
||||
-------------
|
||||
|
||||
Implementing this backport involves backporting the changes in PEP 466, 476 and
|
||||
this PEP, with the following change to the handling of the
|
||||
``PYTHONHTTPSVERIFY`` environment variable in the ``ssl`` module:
|
||||
|
||||
* read the ``PYTHONHTTPSVERIFY`` environment variable when the module is first
|
||||
imported into a Python process
|
||||
* set the ``ssl._create_default_https_context`` function to be an alias for
|
||||
``ssl.create_default_context`` if this environment variable is present
|
||||
and set to any value other than ``'0'``
|
||||
* otherwise, set the ``ssl._create_default_https_context`` function to be an
|
||||
alias for ``ssl._create_unverified_context``
|
||||
|
||||
Example implementation
|
||||
----------------------
|
||||
|
||||
::
|
||||
|
||||
_https_verify_envvar = 'PYTHONHTTPSVERIFY'
|
||||
|
||||
def _get_https_context_factory():
|
||||
if not sys.flags.ignore_environment:
|
||||
config_setting = os.environ.get(_https_verify_envvar)
|
||||
if config_setting != '0':
|
||||
return create_default_context
|
||||
return _create_unverified_context
|
||||
|
||||
_create_default_https_context = _get_https_context_factory()
|
||||
|
||||
def _disable_https_default_verification():
|
||||
"""Skip verification of HTTPS certificates by default"""
|
||||
global _create_default_https_context
|
||||
_create_default_https_context = _create_unverified_context
|
||||
|
||||
Security Considerations
|
||||
-----------------------
|
||||
|
||||
This change would be a strict security upgrade for any Python version that
|
||||
currently defaults to skipping certificate validation in standard library
|
||||
HTTPS clients. The technical trade-offs to be taken into account relate largely
|
||||
to the magnitude of the PEP 466 backport also required rather than to anything
|
||||
security related.
|
||||
|
||||
Interaction with Python virtual environments
|
||||
--------------------------------------------
|
||||
|
||||
The default setting is read directly from the process environment, and hence
|
||||
works the same way regardless of whether or not the interpreter is being run
|
||||
inside an activated Python virtual environment.
|
||||
|
||||
|
||||
Recommendation for combined feature backports
|
||||
=============================================
|
||||
|
||||
If a redistributor chooses to backport the environment variable based
|
||||
configuration setting from this PEP to a modified Python version that also
|
||||
implements the configuration file based PEP 476 , then the environment
|
||||
variable should take precedence over the system-wide configuration setting.
|
||||
This allows the setting to be changed for a given user or application,
|
||||
regardless of the installation-wide default behaviour.
|
||||
|
||||
Example implementation
|
||||
----------------------
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue