PEP 466: further simplify the proposal
- dropped the "legacy ssl" branch idea again - explained why selective backports are a bad idea - better explained how long term support differs from short term - better explained some downstream constraints - explained that downstream redistributors have been respecting our upstream policies, potentially against their own better judgement - used Red Hat's support cycle as a reference model - removed the snarky comments that hit unintended targets - note the need for Windows specific contributions as an open question - not the general need for contributions of time as an open question - last open question is whether or not we're happy this is the best available solution given downstream constraints
This commit is contained in:
Nick Coghlan
parent
a0068e1e6a
commit
8978bad5e8
267
pep-0466.txt
267
pep-0466.txt
|
|
@ -25,10 +25,13 @@ where it is sufficiently far behind the state of the art in network security
|
|||
protocols for it to be causing real problems in commercial use cases
|
||||
where upgrading to Python 3 in the near term may not be practical.
|
||||
|
||||
Accordingly, this PEP relaxes the normal restrictions by allowing
|
||||
enhancements to be applied in Python 2.7 maintenance releases for standard
|
||||
library components that have implications for the overall security of the
|
||||
internet. In particular, the exception will apply to:
|
||||
In recognition of the additional practical considerations that have arisen
|
||||
during the 4+ year maintenance cycle for Python 2.7, this PEP allows
|
||||
Python 2.7 standard library components that have implications for the
|
||||
overall security of the internet to be updated in line with the
|
||||
corresponding Python 3 feature releases.
|
||||
|
||||
Specifically, the exception will apply to:
|
||||
|
||||
* the ``ssl`` module
|
||||
* the ``hashlib`` module
|
||||
|
|
@ -39,52 +42,46 @@ internet. In particular, the exception will apply to:
|
|||
and Mac OS X
|
||||
|
||||
Changes to these modules will still need to undergo normal backwards
|
||||
compatibility assessments, but otherwise they will be kept entirely aligned
|
||||
with the latest feature release of their Python 3 counterparts. This is
|
||||
designed to make it easier to implement secure networked software in
|
||||
Python, even for software that currently needs to remain compatible with
|
||||
the Python 2 series (which includes a lot of network infrastructure
|
||||
software).
|
||||
|
||||
The development branches will be arranged in such a way that even though
|
||||
any further Python 2.7 releases published by the core development team
|
||||
provide updated network security infrastructure, it will remain possible
|
||||
for downstream redistributors to publish "Python 2.7 with legacy SSL
|
||||
infrastructure" if they choose to do so.
|
||||
compatibility assessments to ensure their default behaviour remains
|
||||
consistent with earlier Python 2.7 releases, but otherwise they will be
|
||||
kept entirely aligned with the latest feature release of their Python 3
|
||||
counterparts. This is designed to make it easier to implement secure
|
||||
networked software in Python, even for software that currently needs to
|
||||
remain compatible with the Python 2 series (which includes a lot of
|
||||
network infrastructure software).
|
||||
|
||||
While this PEP does not make any changes to the core development team's
|
||||
handling of security-fix-only branches that are no longer in active
|
||||
maintenance, it *does* recommend that commercial redistributors providing
|
||||
extended support periods for the Python standard library either adopt a
|
||||
similar approach to ensuring that the secure networking infrastructure
|
||||
keeps pace with the evolution of the internet, or else disclaim support
|
||||
for the use of older versions in roles that involving connecting
|
||||
directly to the public internet.
|
||||
keeps pace with the evolution of the internet, or else explicitly
|
||||
disclaim support for the use of older versions in roles that involve
|
||||
connecting directly to the public internet.
|
||||
|
||||
|
||||
Exemption Policy
|
||||
================
|
||||
|
||||
Under this policy, the following network security related modules are
|
||||
granted a blanket exemption to the restriction against adding new features
|
||||
in maintenance releases, for the purpose of keeping their APIs aligned with
|
||||
their counterparts in the latest feature release of Python 3:
|
||||
granted a blanket exemption to the normal restriction against adding new
|
||||
features in Python 2.7 maintenance releases, for the purpose of keeping
|
||||
their APIs aligned with their counterparts in the latest feature release
|
||||
of Python 3:
|
||||
|
||||
* the ``ssl`` module
|
||||
* the ``hashlib`` module
|
||||
* the ``hmac`` module
|
||||
|
||||
This exemption applies to *all* proposals to backport backwards compatible
|
||||
changes in these modules to Python 2.7 maintenance releases. This choice is
|
||||
made deliberately to ensure that the "feature or fix?" argument isn't simply
|
||||
replaced by a "security related or not?" argument. These particular modules
|
||||
are inherently security related, and all enhancements to them improve
|
||||
Python's capabilities as a platform for development of secure networked
|
||||
software.
|
||||
Under this exemption, these modules are updated to provide identical
|
||||
functionality to their Python 3 counterparts after every new Python 3
|
||||
feature release. The default behaviour of the backported modules will be
|
||||
adjusted as appropriate to meet the backwards compatibility requirements
|
||||
of a Python 2.7 maintenance release.
|
||||
|
||||
As part of this policy, permission is also granted to upgrade to newer
|
||||
feature releases of OpenSSL when preparing the binary installers
|
||||
for new maintenance releases of CPython.
|
||||
for new maintenance releases of Python 2.7.
|
||||
|
||||
Note that the ``sha`` and ``md5`` modules are not covered by this policy,
|
||||
but have been deprecated in favour of ``hashlib`` since Python 2.5 and have
|
||||
|
|
@ -107,18 +104,18 @@ by the blanket exemption.
|
|||
Backwards Compatibility Considerations
|
||||
======================================
|
||||
|
||||
This PEP does *not* grant any general exemptions to the usual backwards
|
||||
compatibility policy for maintenance releases. Instead, by explicitly
|
||||
encouraging the use of feature based checks and explicitly opting in to
|
||||
less secure configurations, it is designed to make it easier to provide
|
||||
more "secure by default" behaviour in future feature releases, while still
|
||||
This PEP does *not* grant Python 2.7 any general exemptions to the usual
|
||||
backwards compatibility policy for maintenance releases. Instead, by
|
||||
explicitly encouraging the use of feature based checks and explicitly
|
||||
opting in to less secure configurations, it is designed to make it easier
|
||||
to write more secure cross-version compatible Python software, while still
|
||||
limiting the risk of breaking currently working software when upgrading to
|
||||
a new maintenance release.
|
||||
a new Python 2.7 maintenance release.
|
||||
|
||||
In *all* cases where this policy is applied to backport enhancements to
|
||||
maintenance releases, it MUST be possible to write cross-version compatible
|
||||
code that operates by "feature detection" (for example, checking for
|
||||
particular attributes in the module), without needing to explicitly check
|
||||
Python 2.7 maintenance releases, it MUST be possible to write cross-version
|
||||
compatible code that operates by "feature detection" (for example, checking
|
||||
for particular attributes in the module), without needing to explicitly check
|
||||
the Python version.
|
||||
|
||||
It is then up to library and framework code to provide an appropriate warning
|
||||
|
|
@ -155,7 +152,7 @@ OpenSSL compatibility
|
|||
|
||||
Under this policy, OpenSSL may be upgraded to more recent feature releases
|
||||
in Python 2.7 maintenance releases. On Linux and most other POSIX systems,
|
||||
the specific version of OpenSSL used already varies, as Python dynamically
|
||||
the specific version of OpenSSL used already varies, as CPython dynamically
|
||||
links to the system provided OpenSSL library by default.
|
||||
|
||||
For the Windows binary installers, the ``_ssl`` and ``_hashlib`` modules are
|
||||
|
|
@ -191,17 +188,18 @@ load.
|
|||
Backporting security related fixes and enhancements to earlier versions is
|
||||
a common service for commercial redistributors to offer to their customers.
|
||||
This policy represents an explicit invitation to contribute some of those
|
||||
changes back to the upstream community in cases where they are likely to
|
||||
have a broad impact that helps improve the security of the internet as a
|
||||
whole, rather than sitting back and waiting for unpaid volunteers to do it
|
||||
for them.
|
||||
changes back to the upstream Python community in cases where they are likely
|
||||
to have a broad impact that helps improve the security of the internet as a
|
||||
whole, with the assurance that the existing core development team not only
|
||||
won't object to such contributions, but will actively encourage their
|
||||
incorporation into the next Python 2.7 maintenance release.
|
||||
|
||||
|
||||
Documentation
|
||||
-------------
|
||||
|
||||
All modules covered by this policy MUST include a "Security Considerations"
|
||||
section in their documentation.
|
||||
section in their documentation in order to take advantage of this policy.
|
||||
|
||||
In addition to any other module specific contents, this section MUST
|
||||
enumerate key security enhancements and fixes (with CVE identifiers where
|
||||
|
|
@ -230,36 +228,31 @@ Integration testing
|
|||
-------------------
|
||||
|
||||
Third party integration testing services should offer users the ability
|
||||
to test against both the latest Python 2.7 maintenance release and the
|
||||
latest "Python 2.7 with legacy SSL infrastructure" release, to ensure that
|
||||
libraries, frameworks and applications handle the legacy security
|
||||
infrastructure correctly (either failing or degrading gracefully, depending
|
||||
on the security sensitivity of the software).
|
||||
to test against specific Python 2.7 maintenance releases, to ensure that
|
||||
libraries, frameworks and applications can still test their handling of the
|
||||
legacy security infrastructure correctly (either failing or degrading
|
||||
gracefully, depending on the security sensitivity of the software), even
|
||||
after the latest Python 2.7 maintenance release has been synchronised with
|
||||
a new Python 3 feature release for the modules covered by this policy.
|
||||
|
||||
|
||||
Handling lower security environments with low risk tolerance
|
||||
------------------------------------------------------------
|
||||
|
||||
For better or for worse (mostly worse), there are some environments where
|
||||
the risk of latent security defects is more tolerated than the risk of
|
||||
regressions in maintenance releases. This policy largely excludes these
|
||||
environments from consideration where the modules covered by the exemption
|
||||
are concerned.
|
||||
the risk of latent security defects is more tolerated than even a slightly
|
||||
increased risk of regressions in maintenance releases. This policy largely
|
||||
excludes these environments from consideration where the modules covered by
|
||||
the exemption are concerned - this approach is entirely inappropriate for
|
||||
software connected to the public internet, and defence in depth security
|
||||
principles suggest that it is not appropriate for most private networks
|
||||
either.
|
||||
|
||||
However, one concession is made for the benefit of such environments: while
|
||||
the main ``2.7`` branch in Mercurial will include the updated network
|
||||
security infrastructure, the development process will be updated to include
|
||||
a ``2.7-legacy-ssl`` branch with the more limited feature set of the
|
||||
original 2.7 network security infrastructure, allowing downstream
|
||||
redistributors to continue to provide Python 2.7 with the legacy SSL
|
||||
infrastructure if they choose to do so. This branch will be tested
|
||||
on the CPython continuous integration infrastructure, but no releases will
|
||||
be made from it by the core development team.
|
||||
|
||||
As noted above, corporate redistributors and users are expected to provide
|
||||
the additional development effort needed to make this practical. It is not
|
||||
acceptable to expect volunteer contributors to resolve a problem created
|
||||
largely by conservative corporate development practices.
|
||||
Downstream redistributors may still choose to cater to such environments,
|
||||
but they will need to handle the process of downgrading the security
|
||||
related modules and doing the associated regression testing themselves.
|
||||
The main CPython continuous integration infrastructure will not cover this
|
||||
scenario.
|
||||
|
||||
|
||||
Evolution of this Policy
|
||||
|
|
@ -279,8 +272,9 @@ with clients or servers written using older versions of Python.
|
|||
The intent behind this requirement is to minimise any impact that the
|
||||
introduction of this policy may have on the stability and compatibility of
|
||||
maintenance releases. It would be thoroughly counterproductive if end
|
||||
users became as cautious about updating to new Python maintenance releases
|
||||
as they are about updating to new feature releases.
|
||||
users became as cautious about updating to new Python 2.7 maintenance
|
||||
releases as they are about updating to new feature releases within the
|
||||
same release series.
|
||||
|
||||
|
||||
Motivation and Rationale
|
||||
|
|
@ -312,12 +306,13 @@ infrastructure for users that, for whatever reason, are not currently in
|
|||
a position to migrate to Python 3.
|
||||
|
||||
While the use of the system OpenSSL installation addresses many of these
|
||||
concerns on Linux platforms, it doesn't address all of them, and in the
|
||||
case of the binary installers for Windows and Mac OS X that are published
|
||||
on python.org, the version of OpenSSL used is entirely within the control
|
||||
of the Python core development team, and currently limited to OpenSSL
|
||||
maintenance releases for the version initially shipped with the corresponding
|
||||
Python feature release.
|
||||
concerns on Linux platforms, it doesn't address all of them (in particular,
|
||||
it is still difficult for sotware to explicitly require some higher level
|
||||
security settings). In the case of the binary installers for Windows and
|
||||
Mac OS X that are published on python.org, the version of OpenSSL used is
|
||||
entirely within the control of the Python core development team, but is
|
||||
currently limited to OpenSSL maintenance releases for the version initially
|
||||
shipped with the corresponding Python feature release.
|
||||
|
||||
With increased popularity comes increased responsibility, and this policy
|
||||
aims to acknowledge the fact that Python's popularity and adoption has now
|
||||
|
|
@ -342,7 +337,7 @@ The Python 2 series also remains more vulnerable to remote timing attacks
|
|||
on security sensitive comparisons than the Python 3 series, as it lacks a
|
||||
standard library equivalent to the timing attack resistant
|
||||
``hmac.compare_digest()`` function. While appropriate secure comparison
|
||||
functions can be implemented in third party extensions, may users don't
|
||||
functions can be implemented in third party extensions, many users don't
|
||||
even consider the issue and use ordinary equality comparisons instead
|
||||
- while a standard library solution doesn't automatically fix that problem,
|
||||
it *does* make the barrier to resolution much lower once the problem is
|
||||
|
|
@ -361,6 +356,34 @@ will be considered too difficult to develop truly secure modern networked
|
|||
software using the Python 2 series (some developers would argue that we
|
||||
have already reached that point).
|
||||
|
||||
Python 2.7 represents the only long term maintenance release the core
|
||||
development team has provided, and it is natural that there will be things
|
||||
that worked over a historically shorter maintenance lifespan that don't work
|
||||
over this longer support period. In the specific case of the problem
|
||||
described in this PEP, the simplest available solution is to acknowledge
|
||||
that long term maintenance of network security related modules *requires*
|
||||
the ability to add new features, even while retaining backwards compatibility
|
||||
for existing interfaces.
|
||||
|
||||
It is worth comparing the approach described in this PEP with Red Hat's
|
||||
handling of its long term support commitments: it isn't the RHEL 6.0 release
|
||||
itself that receives 10 years worth of support, but the overall RHEL 6
|
||||
*series*. The individual RHEL 6.x point releases within the series then
|
||||
receive a wide variety of new features, including security enhancements,
|
||||
all while meeting strict backwards compatibility guarantees for existing
|
||||
software. Subscribers *are* able to continue using a given point release
|
||||
after the next point release has become available, but a corresponding
|
||||
add-on subscription for `Extended Update Support
|
||||
<https://access.redhat.com/site/support/policy/updates/errata/#Extended_Update_Support>`__
|
||||
is needed to cover the additional backporting work involved.
|
||||
|
||||
To date, downstream redistributors have respected our upstream policy of
|
||||
"no new features in Python maintenance releases". This PEP explicitly
|
||||
accepts that a more nuanced policy is appropriate in the case of network
|
||||
security related features, and the specific one it describes is deliberately
|
||||
designed such that it at least has some chance of being applied to Red Hat
|
||||
Enterprise Linux and its downstream derivatives.
|
||||
|
||||
|
||||
Alternative: advise developers of networked software to migrate to Python 3
|
||||
---------------------------------------------------------------------------
|
||||
|
|
@ -429,7 +452,7 @@ approach taken to address the situation and to help ensure some consistency
|
|||
across redistributors.
|
||||
|
||||
The problem is real, so *something* needs to change, and this PEP describes
|
||||
my currently preferred approach to addressing the situation.
|
||||
my preferred approach to addressing the situation.
|
||||
|
||||
|
||||
Alternative: create and release Python 2.8
|
||||
|
|
@ -440,10 +463,11 @@ and release Python 2.8 (it's highly unlikely such a project would garner
|
|||
enough interest to be achievable with only volunteers). However, this
|
||||
wouldn't actually solve the problem, as the aim is to provide a *relatively
|
||||
low impact* way to incorporate enhanced security features into integrated
|
||||
products and deployments that make use of Python 2. Upgrading to a new
|
||||
Python feature release would mean both more work for the core development
|
||||
team, as well as a more disruptive update that most potential end users
|
||||
would likely just skip entirely.
|
||||
products and deployments that make use of Python 2.
|
||||
|
||||
Upgrading to a new Python feature release would mean both more work for the
|
||||
core development team, as well as a more disruptive update that most
|
||||
potential end users would likely just skip entirely.
|
||||
|
||||
Attempting to create a Python 2.8 release would also bring in suggestions
|
||||
to backport many additional features from Python 3 (such as ``tracemalloc``
|
||||
|
|
@ -455,6 +479,11 @@ additional work for a result that is actually less effective in achieving
|
|||
the original aim (which is to eliminate the current widespread use of the
|
||||
aging network security infrastructure in the Python 2 series).
|
||||
|
||||
Furthermore, while I can't make any commitments to actually addressing
|
||||
this issue on Red Hat platforms, I *can* categorically rule out the idea
|
||||
of a Python 2.8 being of any use to me in even attempting to get it
|
||||
addressed.
|
||||
|
||||
|
||||
Alternative: distribute the security enhancements via PyPI
|
||||
----------------------------------------------------------
|
||||
|
|
@ -504,17 +533,78 @@ packages added to the pre-existing set *can* be done, but means approaching
|
|||
each and every redistributor and asking them to update their
|
||||
repackaging process accordingly. By contrast, the approach described in
|
||||
this PEP would require redistributors to deliberately *opt out* of the
|
||||
security enhancements (by switching to redistributing directly from the
|
||||
``2.7-legacy-ssl`` branch rather than the main ``2.7`` branch), which most
|
||||
of them are unlikely to do.
|
||||
security enhancements by deliberately downgrading the provided network
|
||||
security infrastructure, which most of them are unlikely to do.
|
||||
|
||||
|
||||
Alternative: provide a "legacy SSL infrastructure" branch
|
||||
---------------------------------------------------------
|
||||
|
||||
Earlier versions of this PEP included the concept of a ``2.7-legacy-ssl``
|
||||
branch that preserved the exact feature set of the Python 2.7.6 network
|
||||
security infrastructure.
|
||||
|
||||
It is the opinion of the PEP author that anyone that actually wants this is
|
||||
almost certainly making a mistake, and if they insist they really do want
|
||||
it in their specific situation, they're welcome to either make it themselves
|
||||
or arrange for a downstream redistributor to make it for them.
|
||||
|
||||
If they are made publicly available, any such rebuilds should be referred to
|
||||
as "Python 2.7 with Legacy SSL" to clearly distinguish them from the official
|
||||
Python 2.7 releases that include more up to date network security
|
||||
infrastructure.
|
||||
|
||||
After the first Python 2.7 maintenance release that has the security
|
||||
infrastructure updated to match Python 3.4, it would also be appropriate to
|
||||
refer to Python 2.7.6 and earlier releases as "Python 2.7 with Legacy SSL".
|
||||
|
||||
|
||||
Alternative: selectively backport particular APIs
|
||||
-------------------------------------------------
|
||||
|
||||
An instinctively minimalist reaction to this proposal is to only backport
|
||||
particular APIs in the affected modules that are judged to be "security
|
||||
critical". However, this ends up providing a worse end user experience,
|
||||
as well as a worse developer experience.
|
||||
|
||||
For end users, the selective backporting approach means learning not only
|
||||
the legacy Python 2.7 API and the current Python 3 APIs, but also the
|
||||
hybrid API created by the selective backporting process. It is much
|
||||
simpler to just learn the APIs of the original Python 2.7 feature release
|
||||
and the relevant Python 3 features releases, without trying to define a new
|
||||
hybrid API that only exists in later Python 2.7 maintenance branches.
|
||||
|
||||
For developers, the main benefit of the "just align the available
|
||||
functionality with Python 3" approach is that it reduces the amount of
|
||||
design work needing when updating Python 2.7 with new security features.
|
||||
The "feature or fix?" and "security related or not?" debates are both
|
||||
handled in advance by this policy, leaving only the matter of ensuring
|
||||
that backwards compatibility is preserved as needed by adjusting the
|
||||
default behaviour in the Python 2.7 backport when appropriate.
|
||||
|
||||
|
||||
Open Questions
|
||||
==============
|
||||
|
||||
* MvL has indicated he is not prepared to tackle the task of trying to
|
||||
integrate a newer OpenSSL into the also aging Python 2.7 build
|
||||
infrastructure on Windows (unfortunately, we've looked into upgrading
|
||||
that build infrastructure, and the backwards compatibility issues
|
||||
appear to be effectively insurmountable). We would require a commitment
|
||||
from another trusted contributor to handle at least this task, and
|
||||
potentially also taking over the task of creating the official
|
||||
Python 2.7 Windows installers for the remaining Python 2.7 maintenance
|
||||
releases.
|
||||
|
||||
* We would need commitments to create and review full backports of the
|
||||
components covered by this policy from Python 3.4 to Python 2.7, as well
|
||||
as support for handling any more specific security issues affecting these
|
||||
modules.
|
||||
|
||||
* I believe I've addressed all the technical and scope questions I had, or
|
||||
others raised. That just leaves the question of "If we agree to this plan,
|
||||
who is actually going to handle all the extra work involved?" :)
|
||||
others raised. That just leaves the question of "Do we agree this plan
|
||||
actually makes sense, given the constraints on downstream redistributors
|
||||
that would also like to see this problem solved?" :)
|
||||
|
||||
|
||||
Disclosure of Interest
|
||||
|
|
@ -541,7 +631,7 @@ of the web as a whole.
|
|||
Christian and Donald Stufft also provided valuable feedback on a preliminary
|
||||
draft of this proposal.
|
||||
|
||||
Thanks also to participants in the python-dev mailing list threads [1,2]_
|
||||
Thanks also to participants in the python-dev mailing list threads [1,2,5]_
|
||||
|
||||
|
||||
References
|
||||
|
|
@ -551,6 +641,7 @@ References
|
|||
.. [2] https://mail.python.org/pipermail/python-dev/2014-March/133389.html
|
||||
.. [3] https://mail.python.org/pipermail/python-dev/2014-March/133438.html
|
||||
.. [4] https://mail.python.org/pipermail/python-dev/2014-March/133347.html
|
||||
.. [5] https://mail.python.org/pipermail/python-dev/2014-March/133442.html
|
||||
|
||||
|
||||
Copyright
|
||||
|
|
|
|||
Loading…
Reference in New Issue