PEP 466: further simplify the proposal

- dropped the "legacy ssl" branch idea again
- explained why selective backports are a bad idea
- better explained how long term support differs from short term
- better explained some downstream constraints
- explained that downstream redistributors have been respecting
  our upstream policies, potentially against their own better
  judgement
- used Red Hat's support cycle as a reference model
- removed the snarky comments that hit unintended targets
- note the need for Windows specific contributions as an open question
- not the general need for contributions of time as an open question
- last open question is whether or not we're happy this is the best
  available solution given downstream constraints

pep-0466.txt

@@ -25,10 +25,13 @@ where it is sufficiently far behind the state of the art in network security
 protocols for it to be causing real problems in commercial use cases
 where upgrading to Python 3 in the near term may not be practical.
 
-Accordingly, this PEP relaxes the normal restrictions by allowing
+In recognition of the additional practical considerations that have arisen
-enhancements to be applied in Python 2.7 maintenance releases for standard
+during the 4+ year maintenance cycle for Python 2.7, this PEP allows
-library components that have implications for the overall security of the
+Python 2.7 standard library components that have implications for the
-internet. In particular, the exception will apply to:
+overall security of the internet to be updated in line with the
+corresponding Python 3 feature releases.
+
+Specifically, the exception will apply to:
 
 * the ``ssl`` module
 * the ``hashlib`` module
@@ -39,52 +42,46 @@ internet. In particular, the exception will apply to:
   and Mac OS X
 
 Changes to these modules will still need to undergo normal backwards
-compatibility assessments, but otherwise they will be kept entirely aligned
+compatibility assessments to ensure their default behaviour remains
-with the latest feature release of their Python 3 counterparts. This is
+consistent with earlier Python 2.7 releases, but otherwise they will be
-designed to make it easier to implement secure networked software in
+kept entirely aligned with the latest feature release of their Python 3
-Python, even for software that currently needs to remain compatible with
+counterparts. This is designed to make it easier to implement secure
-the Python 2 series (which includes a lot of network infrastructure
+networked software in Python, even for software that currently needs to
-software).
+remain compatible with the Python 2 series (which includes a lot of
-
+network infrastructure software).
-The development branches will be arranged in such a way that even though
-any further Python 2.7 releases published by the core development team
-provide updated network security infrastructure, it will remain possible
-for downstream redistributors to publish "Python 2.7 with legacy SSL
-infrastructure" if they choose to do so.
 
 While this PEP does not make any changes to the core development team's
 handling of security-fix-only branches that are no longer in active
 maintenance, it *does* recommend that commercial redistributors providing
 extended support periods for the Python standard library either adopt a
 similar approach to ensuring that the secure networking infrastructure
-keeps pace with the evolution of the internet, or else disclaim support
+keeps pace with the evolution of the internet, or else explicitly
-for the use of older versions in roles that involving connecting
+disclaim support for the use of older versions in roles that involve
-directly to the public internet.
+connecting directly to the public internet.
 
 
 Exemption Policy
 ================
 
 Under this policy, the following network security related modules are
-granted a blanket exemption to the restriction against adding new features
+granted a blanket exemption to the normal restriction against adding new
-in maintenance releases, for the purpose of keeping their APIs aligned with
+features in Python 2.7 maintenance releases, for the purpose of keeping
-their counterparts in the latest feature release of Python 3:
+their APIs aligned with their counterparts in the latest feature release
+of Python 3:
 
 * the ``ssl`` module
 * the ``hashlib`` module
 * the ``hmac`` module
 
-This exemption applies to *all* proposals to backport backwards compatible
+Under this exemption, these modules are updated to provide identical
-changes in these modules to Python 2.7 maintenance releases. This choice is
+functionality to their Python 3 counterparts after every new Python 3
-made deliberately to ensure that the "feature or fix?" argument isn't simply
+feature release. The default behaviour of the backported modules will be
-replaced by a "security related or not?" argument. These particular modules
+adjusted as appropriate to meet the backwards compatibility requirements
-are inherently security related, and all enhancements to them improve
+of a Python 2.7 maintenance release.
-Python's capabilities as a platform for development of secure networked
-software.
 
 As part of this policy, permission is also granted to upgrade to newer
 feature releases of OpenSSL when preparing the binary installers
-for new maintenance releases of CPython.
+for new maintenance releases of Python 2.7.
 
 Note that the ``sha`` and ``md5`` modules are not covered by this policy,
 but have been deprecated in favour of ``hashlib`` since Python 2.5 and have
@@ -107,18 +104,18 @@ by the blanket exemption.
 Backwards Compatibility Considerations
 ======================================
 
-This PEP does *not* grant any general exemptions to the usual backwards
+This PEP does *not* grant Python 2.7 any general exemptions to the usual
-compatibility policy for maintenance releases. Instead, by explicitly
+backwards compatibility policy for maintenance releases. Instead, by
-encouraging the use of feature based checks and explicitly opting in to
+explicitly encouraging the use of feature based checks and explicitly
-less secure configurations, it is designed to make it easier to provide
+opting in to less secure configurations, it is designed to make it easier
-more "secure by default" behaviour in future feature releases, while still
+to write more secure cross-version compatible Python software, while still
 limiting the risk of breaking currently working software when upgrading to
-a new maintenance release.
+a new Python 2.7 maintenance release.
 
 In *all* cases where this policy is applied to backport enhancements to
-maintenance releases, it MUST be possible to write cross-version compatible
+Python 2.7 maintenance releases, it MUST be possible to write cross-version
-code that operates by "feature detection" (for example, checking for
+compatible code that operates by "feature detection" (for example, checking
-particular attributes in the module), without needing to explicitly check
+for particular attributes in the module), without needing to explicitly check
 the Python version.
 
 It is then up to library and framework code to provide an appropriate warning
@@ -155,7 +152,7 @@ OpenSSL compatibility
 
 Under this policy, OpenSSL may be upgraded to more recent feature releases
 in Python 2.7 maintenance releases. On Linux and most other POSIX systems,
-the specific version of OpenSSL used already varies, as Python dynamically
+the specific version of OpenSSL used already varies, as CPython dynamically
 links to the system provided OpenSSL library by default.
 
 For the Windows binary installers, the ``_ssl`` and ``_hashlib`` modules are
@@ -191,17 +188,18 @@ load.
 Backporting security related fixes and enhancements to earlier versions is
 a common service for commercial redistributors to offer to their customers.
 This policy represents an explicit invitation to contribute some of those
-changes back to the upstream community in cases where they are likely to
+changes back to the upstream Python community in cases where they are likely
-have a broad impact that helps improve the security of the internet as a
+to have a broad impact that helps improve the security of the internet as a
-whole, rather than sitting back and waiting for unpaid volunteers to do it
+whole, with the assurance that the existing core development team not only
-for them.
+won't object to such contributions, but will actively encourage their
+incorporation into the next Python 2.7 maintenance release.
 
 
 Documentation
 -------------
 
 All modules covered by this policy MUST include a "Security Considerations"
-section in their documentation.
+section in their documentation in order to take advantage of this policy.
 
 In addition to any other module specific contents, this section MUST
 enumerate key security enhancements and fixes (with CVE identifiers where
@@ -230,36 +228,31 @@ Integration testing
 -------------------
 
 Third party integration testing services should offer users the ability
-to test against both the latest Python 2.7 maintenance release and the
+to test against specific Python 2.7 maintenance releases, to ensure that
-latest "Python 2.7 with legacy SSL infrastructure" release, to ensure that
+libraries, frameworks and applications can still test their handling of the
-libraries, frameworks and applications handle the legacy security
+legacy security infrastructure correctly (either failing or degrading
-infrastructure correctly (either failing or degrading gracefully, depending
+gracefully, depending on the security sensitivity of the software), even
-on the security sensitivity of the software).
+after the latest Python 2.7 maintenance release has been synchronised with
+a new Python 3 feature release for the modules covered by this policy.
 
 
 Handling lower security environments with low risk tolerance
 ------------------------------------------------------------
 
 For better or for worse (mostly worse), there are some environments where
-the risk of latent security defects is more tolerated than the risk of
+the risk of latent security defects is more tolerated than even a slightly
-regressions in maintenance releases. This policy largely excludes these
+increased risk of regressions in maintenance releases. This policy largely
-environments from consideration where the modules covered by the exemption
+excludes these environments from consideration where the modules covered by
-are concerned.
+the exemption are concerned - this approach is entirely inappropriate for
+software connected to the public internet, and defence in depth security
+principles suggest that it is not appropriate for most private networks
+either.
 
-However, one concession is made for the benefit of such environments: while
+Downstream redistributors may still choose to cater to such environments,
-the main ``2.7`` branch in Mercurial will include the updated network
+but they will need to handle the process of downgrading the security
-security infrastructure, the development process will be updated to include
+related modules and doing the associated regression testing themselves.
-a ``2.7-legacy-ssl`` branch with the more limited feature set of the
+The main CPython continuous integration infrastructure will not cover this
-original 2.7 network security infrastructure, allowing downstream
+scenario.
-redistributors to continue to provide Python 2.7 with the legacy SSL
-infrastructure if they choose to do so. This branch will be tested
-on the CPython continuous integration infrastructure, but no releases will
-be made from it by the core development team.
-
-As noted above, corporate redistributors and users are expected to provide
-the additional development effort needed to make this practical. It is not
-acceptable to expect volunteer contributors to resolve a problem created
-largely by conservative corporate development practices.
 
 
 Evolution of this Policy
@@ -279,8 +272,9 @@ with clients or servers written using older versions of Python.
 The intent behind this requirement is to minimise any impact that the
 introduction of this policy may have on the stability and compatibility of
 maintenance releases. It would be thoroughly counterproductive if end
-users became as cautious about updating to new Python maintenance releases
+users became as cautious about updating to new Python 2.7 maintenance
-as they are about updating to new feature releases.
+releases as they are about updating to new feature releases within the
+same release series.
 
 
 Motivation and Rationale
@@ -312,12 +306,13 @@ infrastructure for users that, for whatever reason, are not currently in
 a position to migrate to Python 3.
 
 While the use of the system OpenSSL installation addresses many of these
-concerns on Linux platforms, it doesn't address all of them, and in the
+concerns on Linux platforms, it doesn't address all of them (in particular,
-case of the binary installers for Windows and Mac OS X that are published
+it is still difficult for sotware to explicitly require some higher level
-on python.org, the version of OpenSSL used is entirely within the control
+security settings). In the case of the binary installers for Windows and
-of the Python core development team, and currently limited to OpenSSL
+Mac OS X that are published on python.org, the version of OpenSSL used is
-maintenance releases for the version initially shipped with the corresponding
+entirely within the control of the Python core development team, but is
-Python feature release.
+currently limited to OpenSSL maintenance releases for the version initially
+shipped with the corresponding Python feature release.
 
 With increased popularity comes increased responsibility, and this policy
 aims to acknowledge the fact that Python's popularity and adoption has now
@@ -342,7 +337,7 @@ The Python 2 series also remains more vulnerable to remote timing attacks
 on security sensitive comparisons than the Python 3 series, as it lacks a
 standard library equivalent to the timing attack resistant
 ``hmac.compare_digest()`` function. While appropriate secure comparison
-functions can be implemented in third party extensions, may users don't
+functions can be implemented in third party extensions, many users don't
 even consider the issue and use ordinary equality comparisons instead
 - while a standard library solution doesn't automatically fix that problem,
 it *does* make the barrier to resolution much lower once the problem is
@@ -361,6 +356,34 @@ will be considered too difficult to develop truly secure modern networked
 software using the Python 2 series (some developers would argue that we
 have already reached that point).
 
+Python 2.7 represents the only long term maintenance release the core
+development team has provided, and it is natural that there will be things
+that worked over a historically shorter maintenance lifespan that don't work
+over this longer support period. In the specific case of the problem
+described in this PEP, the simplest available solution is to acknowledge
+that long term maintenance of network security related modules *requires*
+the ability to add new features, even while retaining backwards compatibility
+for existing interfaces.
+
+It is worth comparing the approach described in this PEP with Red Hat's
+handling of its long term support commitments: it isn't the RHEL 6.0 release
+itself that receives 10 years worth of support, but the overall RHEL 6
+*series*. The individual RHEL 6.x point releases within the series then
+receive a wide variety of new features, including security enhancements,
+all while meeting strict backwards compatibility guarantees for existing
+software. Subscribers *are* able to continue using a given point release
+after the next point release has become available, but a corresponding
+add-on subscription for `Extended Update Support
+<https://access.redhat.com/site/support/policy/updates/errata/#Extended_Update_Support>`__
+is needed to cover the additional backporting work involved.
+
+To date, downstream redistributors have respected our upstream policy of
+"no new features in Python maintenance releases". This PEP explicitly
+accepts that a more nuanced policy is appropriate in the case of network
+security related features, and the specific one it describes is deliberately
+designed such that it at least has some chance of being applied to Red Hat
+Enterprise Linux and its downstream derivatives.
+
 
 Alternative: advise developers of networked software to migrate to Python 3
 ---------------------------------------------------------------------------
@@ -429,7 +452,7 @@ approach taken to address the situation and to help ensure some consistency
 across redistributors.
 
 The problem is real, so *something* needs to change, and this PEP describes
-my currently preferred approach to addressing the situation.
+my preferred approach to addressing the situation.
 
 
 Alternative: create and release Python 2.8
@@ -440,10 +463,11 @@ and release Python 2.8 (it's highly unlikely such a project would garner
 enough interest to be achievable with only volunteers). However, this
 wouldn't actually solve the problem, as the aim is to provide a *relatively
 low impact* way to incorporate enhanced security features into integrated
-products and deployments that make use of Python 2. Upgrading to a new
+products and deployments that make use of Python 2.
-Python feature release would mean both more work for the core development
+
-team, as well as a more disruptive update that most potential end users
+Upgrading to a new Python feature release would mean both more work for the
-would likely just skip entirely.
+core development team, as well as a more disruptive update that most
+potential end users would likely just skip entirely.
 
 Attempting to create a Python 2.8 release would also bring in suggestions
 to backport many additional features from Python 3 (such as ``tracemalloc``
@@ -455,6 +479,11 @@ additional work for a result that is actually less effective in achieving
 the original aim (which is to eliminate the current widespread use of the
 aging network security infrastructure in the Python 2 series).
 
+Furthermore, while I can't make any commitments to actually addressing
+this issue on Red Hat platforms, I *can* categorically rule out the idea
+of a Python 2.8 being of any use to me in even attempting to get it
+addressed.
+
 
 Alternative: distribute the security enhancements via PyPI
 ----------------------------------------------------------
@@ -504,17 +533,78 @@ packages added to the pre-existing set *can* be done, but means approaching
 each and every redistributor and asking them to update their
 repackaging process accordingly. By contrast, the approach described in
 this PEP would require redistributors to deliberately *opt out* of the
-security enhancements (by switching to redistributing directly from the
+security enhancements by deliberately downgrading the provided network
-``2.7-legacy-ssl`` branch rather than the main ``2.7`` branch), which most
+security infrastructure, which most of them are unlikely to do.
-of them are unlikely to do.
+
+
+Alternative: provide a "legacy SSL infrastructure" branch
+---------------------------------------------------------
+
+Earlier versions of this PEP included the concept of a ``2.7-legacy-ssl``
+branch that preserved the exact feature set of the Python 2.7.6 network
+security infrastructure.
+
+It is the opinion of the PEP author that anyone that actually wants this is
+almost certainly making a mistake, and if they insist they really do want
+it in their specific situation, they're welcome to either make it themselves
+or arrange for a downstream redistributor to make it for them.
+
+If they are made publicly available, any such rebuilds should be referred to
+as "Python 2.7 with Legacy SSL" to clearly distinguish them from the official
+Python 2.7 releases that include more up to date network security
+infrastructure.
+
+After the first Python 2.7 maintenance release that has the security
+infrastructure updated to match Python 3.4, it would also be appropriate to
+refer to Python 2.7.6 and earlier releases as "Python 2.7 with Legacy SSL".
+
+
+Alternative: selectively backport particular APIs
+-------------------------------------------------
+
+An instinctively minimalist reaction to this proposal is to only backport
+particular APIs in the affected modules that are judged to be "security
+critical". However, this ends up providing a worse end user experience,
+as well as a worse developer experience.
+
+For end users, the selective backporting approach means learning not only
+the legacy Python 2.7 API and the current Python 3 APIs, but also the
+hybrid API created by the selective backporting process. It is much
+simpler to just learn the APIs of the original Python 2.7 feature release
+and the relevant Python 3 features releases, without trying to define a new
+hybrid API that only exists in later Python 2.7 maintenance branches.
+
+For developers, the main benefit of the "just align the available
+functionality with Python 3" approach is that it reduces the amount of
+design work needing when updating Python 2.7 with new security features.
+The "feature or fix?" and "security related or not?" debates are both
+handled in advance by this policy, leaving only the matter of ensuring
+that backwards compatibility is preserved as needed by adjusting the
+default behaviour in the Python 2.7 backport when appropriate.
 
 
 Open Questions
 ==============
 
+* MvL has indicated he is not prepared to tackle the task of trying to
+  integrate a newer OpenSSL into the also aging Python 2.7 build
+  infrastructure on Windows (unfortunately, we've looked into upgrading
+  that build infrastructure, and the backwards compatibility issues
+  appear to be effectively insurmountable). We would require a commitment
+  from another trusted contributor to handle at least this task, and
+  potentially also taking over the task of creating the official
+  Python 2.7 Windows installers for the remaining Python 2.7 maintenance
+  releases.
+
+* We would need commitments to create and review full backports of the
+  components covered by this policy from Python 3.4 to Python 2.7, as well
+  as support for handling any more specific security issues affecting these
+  modules.
+
 * I believe I've addressed all the technical and scope questions I had, or
-  others raised. That just leaves the question of "If we agree to this plan,
+  others raised. That just leaves the question of "Do we agree this plan
-  who is actually going to handle all the extra work involved?" :)
+  actually makes sense, given the constraints on downstream redistributors
+  that would also like to see this problem solved?" :)
 
 
 Disclosure of Interest
@@ -541,7 +631,7 @@ of the web as a whole.
 Christian and Donald Stufft also provided valuable feedback on a preliminary
 draft of this proposal.
 
-Thanks also to participants in the python-dev mailing list threads [1,2]_
+Thanks also to participants in the python-dev mailing list threads [1,2,5]_
 
 
 References
@@ -551,6 +641,7 @@ References
 .. [2] https://mail.python.org/pipermail/python-dev/2014-March/133389.html
 .. [3] https://mail.python.org/pipermail/python-dev/2014-March/133438.html
 .. [4] https://mail.python.org/pipermail/python-dev/2014-March/133347.html
+.. [5] https://mail.python.org/pipermail/python-dev/2014-March/133442.html
 
 
 Copyright