iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

PEP 466: switch to more selective backports

This commit is contained in:
Nick Coghlan 2014-03-26 21:46:42 +10:00
parent 68ce00d52e
commit 9ea9830f61
1 changed files with 125 additions and 143 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

268
pep-0466.txt
View File

@ -7,7 +7,7 @@ Status: Draft
Type: Informational Type: Informational
Content-Type: text/x-rst Content-Type: text/x-rst
Created: 23-Mar-2014 Created: 23-Mar-2014
Post-History: 23-Mar-2014 Post-History: 23-Mar-2014, 24-Mar-2014, 25-Mar-2014, 26-Mar-2014
Abstract Abstract
@ -22,8 +22,8 @@ This cadence works reasonably well during Python's normal 18-24 month
feature release cycle, which is still applicable to the Python 3 series. feature release cycle, which is still applicable to the Python 3 series.
However, the age of the standard library in Python 2 has now reached a point However, the age of the standard library in Python 2 has now reached a point
where it is sufficiently far behind the state of the art in network security where it is sufficiently far behind the state of the art in network security
protocols for it to be causing real problems in commercial use cases protocols for it to be causing real problems in use cases where upgrading to
where upgrading to Python 3 in the near term may not be practical. Python 3 in the near term may not be feasible.
In recognition of the additional practical considerations that have arisen In recognition of the additional practical considerations that have arisen
during the 4+ year maintenance cycle for Python 2.7, this PEP allows during the 4+ year maintenance cycle for Python 2.7, this PEP allows
@ -31,91 +31,74 @@ Python 2.7 standard library components that have implications for the
overall security of the internet to be updated in line with the overall security of the internet to be updated in line with the
corresponding Python 3 feature releases. corresponding Python 3 feature releases.
Specifically, the exception will apply to: Specifically, the exception allows a critical set of network security
related features to be backported from Python 3.4 to the upcoming Python
* the ``ssl`` module 2.7.7 maintenance release.
* the ``hashlib`` module
* the ``hmac`` module
* the components of the ``random`` and ``os`` modules that the above
modules rely on for cryptographic randomness
* the version of OpenSSL bundled with the binary installers for Windows
and Mac OS X
Changes to these modules will still need to undergo normal backwards
compatibility assessments to ensure their default behaviour remains
consistent with earlier Python 2.7 releases, but otherwise they will be
kept entirely aligned with the latest feature release of their Python 3
counterparts. This is designed to make it easier to implement secure
networked software in Python, even for software that currently needs to
remain compatible with the Python 2 series (which includes a lot of
network infrastructure software).
While this PEP does not make any changes to the core development team's While this PEP does not make any changes to the core development team's
handling of security-fix-only branches that are no longer in active handling of security-fix-only branches that are no longer in active
maintenance, it *does* recommend that commercial redistributors providing maintenance, it *does* recommend that commercial redistributors providing
extended support periods for the Python standard library either adopt a extended support periods for the Python standard library either backport
similar approach to ensuring that the secure networking infrastructure these features to their supported versions, or else explicitly disclaim
keeps pace with the evolution of the internet, or else explicitly support for the use of older versions in roles that involve connecting
disclaim support for the use of older versions in roles that involve directly to the public internet.
connecting directly to the public internet.
Exemption Policy Exemption Policy
================ ================
Under this policy, the following network security related modules are Under this policy, the following features SHOULD be backported from Python
granted a blanket exemption to the normal restriction against adding new 3.4 to the upcoming Python 2.7.7 maintenance release:
features in Python 2.7 maintenance releases, for the purpose of keeping
their APIs aligned with their counterparts in the latest feature release
of Python 3:
* the ``ssl`` module * in the ``os`` module:
* the ``hashlib`` module
* the ``hmac`` module
Under this exemption, these modules are updated to provide identical * persistent file descriptor for ``os.urandom()``.
functionality to their Python 3 counterparts after every new Python 3
feature release. The default behaviour of the backported modules will be * in the ``hmac`` module:
adjusted as appropriate to meet the backwards compatibility requirements
of a Python 2.7 maintenance release. * constant time comparison function (``hmac.compare_digest()``).
* in the ``hashlib`` module:
* password hashing function (``hashlib.pbkdf2_hmac()``).
* details of hash algorithm availability (``hashlib.algorithms_guaranteed``
and ``hashlib.algorithms_guaranteed``).
* in the ``ssl`` module:
* this module is almost entirely synchronised with its Python 3
counterpart, bringing TLSv2, SSLContext manipulation, Server Name
Identification, access to platform certificate stores, standard
library support for peer hostname validation and more to the Python 2
series.
* the only ``ssl`` module features *not* backported under this policy are
the ``ssl.RAND_*`` functions that provide access to OpenSSL's random
number generation capabilities - use ``os.urandom()`` instead.
As part of this policy, permission is also granted to upgrade to newer As part of this policy, permission is also granted to upgrade to newer
feature releases of OpenSSL when preparing the binary installers feature releases of OpenSSL when preparing the binary installers
for new maintenance releases of Python 2.7. for new maintenance releases of Python 2.7.
Note that the ``sha`` and ``md5`` modules are not covered by this policy,
but have been deprecated in favour of ``hashlib`` since Python 2.5 and have
been removed entirely in the Python 3 series.
In addition to the above blanket exemption, a conditional exemption is
granted for these modules that may include some network security related
features:
* the ``os`` module (specifically ``os.urandom``)
* the ``random`` module
This more limited exemption for these modules requires that the *specific*
enhancement being proposed for backporting needs to be justified as being
network security related. This is generally restricted to cases where the
feature in question is needed by an update to one of the modules covered
by the blanket exemption.
Backwards Compatibility Considerations Backwards Compatibility Considerations
====================================== ======================================
This PEP does *not* grant Python 2.7 any general exemptions to the usual As in the Python 3 series, the backported ``ssl.create_default_context()``
backwards compatibility policy for maintenance releases. Instead, by API is granted a backwards compatibility exemption that permits the
explicitly encouraging the use of feature based checks and explicitly protocol, options, cipher and other settings of the created SSL context to
opting in to less secure configurations, it is designed to make it easier be made
This PEP does *not* grant any exemptions to the usual backwards
compatibility policy for maintenance releases. Instead, by explicitly
encouraging the use of feature based checks, it is designed to make it easier
to write more secure cross-version compatible Python software, while still to write more secure cross-version compatible Python software, while still
limiting the risk of breaking currently working software when upgrading to limiting the risk of breaking currently working software when upgrading to
a new Python 2.7 maintenance release. a new Python 2.7 maintenance release.
In *all* cases where this policy is applied to backport enhancements to In all cases where this policy allows new features to be backported to
Python 2.7 maintenance releases, it MUST be possible to write cross-version the Python 2.7 release series, it is possible to write cross-version
compatible code that operates by "feature detection" (for example, checking compatible code that operates by "feature detection" (for example, checking
for particular attributes in the module), without needing to explicitly check for particular attributes in a module), without needing to explicitly check
the Python version. the Python version.
It is then up to library and framework code to provide an appropriate warning It is then up to library and framework code to provide an appropriate warning
@ -124,16 +107,17 @@ some especially security sensitive software MAY fail outright if a desired
security feature is unavailable, most software SHOULD instead emit a warning security feature is unavailable, most software SHOULD instead emit a warning
and continue operating using a slightly degraded security configuration. and continue operating using a slightly degraded security configuration.
Affected APIs SHOULD be designed to allow library and application code to The backported APIs allow library and application code to perform the
perform the following actions after detecting the presence of a relevant following actions after detecting the presence of a relevant
network security related feature: network security related feature:
* explicitly opt in to more secure settings (to allow the use of enhanced * explicitly opt in to more secure settings (to allow the use of enhanced
security features in older maintenance releases of Python) security features in older maintenance releases of Python with less
secure default behaviour)
* explicitly opt in to less secure settings (to allow the use of newer Python * explicitly opt in to less secure settings (to allow the use of newer Python
feature releases in lower security environments) feature releases in lower security environments)
* determine the default setting for the feature (this MAY require explicit * determine the default setting for the feature (this MAY require explicit
Python version checks to determine the Python feature release, but MUST Python version checks to determine the Python feature release, but DOES
NOT require checking for a specific maintenance release) NOT require checking for a specific maintenance release)
Security related changes to other modules (such as higher level networking Security related changes to other modules (such as higher level networking
@ -187,12 +171,12 @@ load.
Backporting security related fixes and enhancements to earlier versions is Backporting security related fixes and enhancements to earlier versions is
a common service for commercial redistributors to offer to their customers. a common service for commercial redistributors to offer to their customers.
This policy represents an explicit invitation to contribute some of those This policy represents an explicit invitation to implement those changes
changes back to the upstream Python community in cases where they are likely in the core development tree in cases where they are likely to have a broad
to have a broad impact that helps improve the security of the internet as a impact that helps improve the security of the internet as a whole, with the
whole, with the assurance that the existing core development team not only assurance that the existing core development team not only won't object to
won't object to such contributions, but will actively encourage their such contributions, but will actively encourage their incorporation into
incorporation into the next Python 2.7 maintenance release. the next Python 2.7 maintenance release.
Documentation Documentation
@ -201,7 +185,7 @@ Documentation
All modules covered by this policy MUST include a "Security Considerations" All modules covered by this policy MUST include a "Security Considerations"
section in their documentation in order to take advantage of this policy. section in their documentation in order to take advantage of this policy.
In addition to any other module specific contents, this section MUST In addition to any other module specific contents, this section SHOULD
enumerate key security enhancements and fixes (with CVE identifiers where enumerate key security enhancements and fixes (with CVE identifiers where
applicable), along with the feature and maintenance releases that first applicable), along with the feature and maintenance releases that first
included them. included them.
@ -232,8 +216,8 @@ to test against specific Python 2.7 maintenance releases, to ensure that
libraries, frameworks and applications can still test their handling of the libraries, frameworks and applications can still test their handling of the
legacy security infrastructure correctly (either failing or degrading legacy security infrastructure correctly (either failing or degrading
gracefully, depending on the security sensitivity of the software), even gracefully, depending on the security sensitivity of the software), even
after the latest Python 2.7 maintenance release has been synchronised with after the features covered in this policy have been backported to the
a new Python 3 feature release for the modules covered by this policy. Python 2.7 series.
Handling lower security environments with low risk tolerance Handling lower security environments with low risk tolerance
@ -258,16 +242,16 @@ scenario.
Evolution of this Policy Evolution of this Policy
======================== ========================
The key requirement for a module to be considered for inclusion in this The key requirement for a feature to be considered for inclusion in this
policy (whether under a blanket or conditional exemption) is that it must policy is that it must have security implications *beyond* the specific
have security implications *beyond* the specific application that is written application that is written in Python and the system that application is
in Python and the system that application is running on. Thus the focus on running on. Thus the focus on network security protocols, password storage
network security protocols and related cryptographic infrastructure - Python and related cryptographic infrastructure - Python is a popular choice for
is a popular choice for the development of web services and clients, and the development of web services and clients, and thus the capabilities of
thus the capabilities of widely used Python versions have implications for widely used Python versions have implications for the security design of
the security design of other services that may themselves be using newer other services that may themselves be using newer versions of Python or
versions of Python or other development languages, but need to interoperate other development languages, but need to interoperate with clients or
with clients or servers written using older versions of Python. servers written using older versions of Python.
The intent behind this requirement is to minimise any impact that the The intent behind this requirement is to minimise any impact that the
introduction of this policy may have on the stability and compatibility of introduction of this policy may have on the stability and compatibility of
@ -280,15 +264,6 @@ same release series.
Motivation and Rationale Motivation and Rationale
======================== ========================
This PEP can be seen as a more targeted version of the "faster standard
library release cycle" proposals discussed in PEP 407 and PEP 413,
focusing specifically on those areas which have implications beyond the
Python community.
Background
----------
The creation of this PEP was prompted primarily by the aging SSL support in The creation of this PEP was prompted primarily by the aging SSL support in
the Python 2 series. As of March 2014, the Python 2.7 SSL module is the Python 2 series. As of March 2014, the Python 2.7 SSL module is
approaching four years of age, and the SSL support in the still popular approaching four years of age, and the SSL support in the still popular
@ -315,8 +290,8 @@ currently limited to OpenSSL maintenance releases for the version initially
shipped with the corresponding Python feature release. shipped with the corresponding Python feature release.
With increased popularity comes increased responsibility, and this policy With increased popularity comes increased responsibility, and this policy
aims to acknowledge the fact that Python's popularity and adoption has now aims to acknowledge the fact that Python's popularity and adoption is at a
reached a level where some of our design and policy decisions have sufficiently high level that some of our design and policy decisions have
significant implications beyond the Python development community. significant implications beyond the Python development community.
As one example, the Python 2 ``ssl`` module does not support the Server As one example, the Python 2 ``ssl`` module does not support the Server
@ -371,11 +346,10 @@ itself that receives 10 years worth of support, but the overall RHEL 6
*series*. The individual RHEL 6.x point releases within the series then *series*. The individual RHEL 6.x point releases within the series then
receive a wide variety of new features, including security enhancements, receive a wide variety of new features, including security enhancements,
all while meeting strict backwards compatibility guarantees for existing all while meeting strict backwards compatibility guarantees for existing
software. Subscribers *are* able to continue using a given point release software. The policy described in this PEP brings our approach to long term
after the next point release has become available, but a corresponding maintenance more into line with this precedent - we retain our strict
add-on subscription for `Extended Update Support backwards compatibility requirements, but slightly relax the restrictions
<https://access.redhat.com/site/support/policy/updates/errata/#Extended_Update_Support>`__ against adding new features.
is needed to cover the additional backporting work involved.
To date, downstream redistributors have respected our upstream policy of To date, downstream redistributors have respected our upstream policy of
"no new features in Python maintenance releases". This PEP explicitly "no new features in Python maintenance releases". This PEP explicitly
@ -385,8 +359,8 @@ designed such that it at least has some chance of being applied to Red Hat
Enterprise Linux and its downstream derivatives. Enterprise Linux and its downstream derivatives.
Alternative: advise developers of networked software to migrate to Python 3 Rejected alternative: just advise developers to migrate to Python 3
--------------------------------------------------------------------------- -------------------------------------------------------------------
This alternative represents the status quo. Unfortunately, it has proven This alternative represents the status quo. Unfortunately, it has proven
to be unworkable in practice, as the backwards compatibility implications to be unworkable in practice, as the backwards compatibility implications
@ -428,7 +402,7 @@ that runs in the integrated system Python by necessity.
The OpenStack migration to Python 3 is also still in its infancy, and even The OpenStack migration to Python 3 is also still in its infancy, and even
though that's a project with an extensive and relatively robust automated though that's a project with an extensive and relatively robust automated
test suite, it's still large enough that it is going to take quite some time test suite, it's still large enough that it is going to take quite some time
to migrate. to migrate fully to a Python 2/3 compatible code base.
And that's just three of the highest profile open source projects that And that's just three of the highest profile open source projects that
make heavy use of Python. Given the likely existence of large amounts of make heavy use of Python. Given the likely existence of large amounts of
@ -455,8 +429,8 @@ The problem is real, so *something* needs to change, and this PEP describes
my preferred approach to addressing the situation. my preferred approach to addressing the situation.
Alternative: create and release Python 2.8 Rejected alternative: create and release Python 2.8
------------------------------------------ ---------------------------------------------------
With sufficient corporate support, it likely *would* be possible to create With sufficient corporate support, it likely *would* be possible to create
and release Python 2.8 (it's highly unlikely such a project would garner and release Python 2.8 (it's highly unlikely such a project would garner
@ -485,8 +459,8 @@ of a Python 2.8 being of any use to me in even attempting to get it
addressed. addressed.
Alternative: distribute the security enhancements via PyPI Rejected alternative: distribute the security enhancements via PyPI
---------------------------------------------------------- -------------------------------------------------------------------
While this initially appears to be an attractive and easier to manage While this initially appears to be an attractive and easier to manage
approach, it actually suffers from several significant problems. approach, it actually suffers from several significant problems.
@ -537,8 +511,8 @@ security enhancements by deliberately downgrading the provided network
security infrastructure, which most of them are unlikely to do. security infrastructure, which most of them are unlikely to do.
Alternative: provide a "legacy SSL infrastructure" branch Rejected variant: provide a "legacy SSL infrastructure" branch
--------------------------------------------------------- --------------------------------------------------------------
Earlier versions of this PEP included the concept of a ``2.7-legacy-ssl`` Earlier versions of this PEP included the concept of a ``2.7-legacy-ssl``
branch that preserved the exact feature set of the Python 2.7.6 network branch that preserved the exact feature set of the Python 2.7.6 network
@ -559,28 +533,14 @@ infrastructure updated to match Python 3.4, it would also be appropriate to
refer to Python 2.7.6 and earlier releases as "Python 2.7 with Legacy SSL". refer to Python 2.7.6 and earlier releases as "Python 2.7 with Legacy SSL".
Alternative: selectively backport particular APIs Rejected variant: synchronise particular modules entirely with Python 3
------------------------------------------------- -----------------------------------------------------------------------
An instinctively minimalist reaction to this proposal is to only backport Earlier versions of this PEP suggested synchronising the ``hmac``,
particular APIs in the affected modules that are judged to be "security ``hashlib`` and ``ssl`` modules entirely with their Python 3 counterparts.
critical". However, this ends up providing a worse end user experience,
as well as a worse developer experience.
For end users, the selective backporting approach means learning not only This approach proved too vague to build a compelling case for the exception,
the legacy Python 2.7 API and the current Python 3 APIs, but also the and has thus been replaced by the current more explicit proposal.
hybrid API created by the selective backporting process. It is much
simpler to just learn the APIs of the original Python 2.7 feature release
and the relevant Python 3 features releases, without trying to define a new
hybrid API that only exists in later Python 2.7 maintenance branches.
For developers, the main benefit of the "just align the available
functionality with Python 3" approach is that it reduces the amount of
design work needing when updating Python 2.7 with new security features.
The "feature or fix?" and "security related or not?" debates are both
handled in advance by this policy, leaving only the matter of ensuring
that backwards compatibility is preserved as needed by adjusting the
default behaviour in the Python 2.7 backport when appropriate.
Open Questions Open Questions
@ -601,10 +561,8 @@ Open Questions
as support for handling any more specific security issues affecting these as support for handling any more specific security issues affecting these
modules. modules.
* I believe I've addressed all the technical and scope questions I had, or * Did I miss anything important in the switch to a more restrictive
others raised. That just leaves the question of "Do we agree this plan proposal?
actually makes sense, given the constraints on downstream redistributors
that would also like to see this problem solved?" :)
Disclosure of Interest Disclosure of Interest
@ -620,7 +578,7 @@ and cannot make any commitments on Red Hat's behalf.
Acknowledgements Acknowledgements
================ ================
Thanks to Christian Heimes for his recent efforts on greatly improving Thanks to Christian Heimes and other for their efforts in greatly improving
Python's SSL support in the Python 3 series, and a variety of members of Python's SSL support in the Python 3 series, and a variety of members of
the Python community for helping me to better understand the implications the Python community for helping me to better understand the implications
of the default settings we provide in our SSL modules, and the impact that of the default settings we provide in our SSL modules, and the impact that
@ -628,20 +586,44 @@ tolerating the use of SSL infrastructure that was defined in 2010
(Python 2.7) or even 2008 (Python 2.6) potentially has for the security (Python 2.7) or even 2008 (Python 2.6) potentially has for the security
of the web as a whole. of the web as a whole.
Christian and Donald Stufft also provided valuable feedback on a preliminary Thanks to Donald Stufft and Alex Gaynor for identifying a more limited set
of essential security features that allowed the proposal to be made more
fine-grained than backporting entire modules from Python 3.4 [7,8]_.
Christian and Donald also provided valuable feedback on a preliminary
draft of this proposal. draft of this proposal.
Thanks also to participants in the python-dev mailing list threads [1,2,5]_ Thanks also to participants in the python-dev mailing list threads
[1,2,5,6]_
References References
========== ==========
.. [1] https://mail.python.org/pipermail/python-dev/2014-March/133334.html .. [1] PEP 466 discussion (round 1)
.. [2] https://mail.python.org/pipermail/python-dev/2014-March/133389.html (https://mail.python.org/pipermail/python-dev/2014-March/133334.html)
.. [3] https://mail.python.org/pipermail/python-dev/2014-March/133438.html
.. [4] https://mail.python.org/pipermail/python-dev/2014-March/133347.html .. [2] PEP 466 discussion (round 2)
.. [5] https://mail.python.org/pipermail/python-dev/2014-March/133442.html (https://mail.python.org/pipermail/python-dev/2014-March/133389.html)
.. [3] Marc-Andre Lemburg's OpenSSL feedback for Windows
(https://mail.python.org/pipermail/python-dev/2014-March/133438.html)
.. [4] Ned Deily's OpenSSL feedback for Mac OS X
(https://mail.python.org/pipermail/python-dev/2014-March/133347.html)
.. [5] PEP 466 discussion (round 3)
(https://mail.python.org/pipermail/python-dev/2014-March/133442.html)
.. [6] PEP 466 discussion (round 4)
(https://mail.python.org/pipermail/python-dev/2014-March/133472.html)
.. [7] Donald Stufft's recommended set of backported features
(https://mail.python.org/pipermail/python-dev/2014-March/133500.html)
.. [8] Alex Gaynor's recommended set of backported features
(https://mail.python.org/pipermail/python-dev/2014-March/133503.html)
Copyright Copyright