PEP 466: make proposal even more specific

pep-0466.txt

@@ -1,5 +1,5 @@
 PEP: 466
-Title: Network Security Enhancement Exception for Python 2.7
+Title: Network Security Enhancements for Python 2.7.7
 Version: $Revision$
 Last-Modified: $Date$
 Author: Nick Coghlan <ncoghlan@gmail.com>,
@@ -7,7 +7,7 @@ Status: Draft
 Type: Informational
 Content-Type: text/x-rst
 Created: 23-Mar-2014
-Post-History: 23-Mar-2014, 24-Mar-2014, 25-Mar-2014, 26-Mar-2014
+Post-History: 23-Mar-2014, 24-Mar-2014, 25-Mar-2014, 26-Mar-2014, 16-Apr-2014
 
 
 Abstract
@@ -26,14 +26,9 @@ protocols for it to be causing real problems in use cases where upgrading to
 Python 3 in the near term may not be feasible.
 
 In recognition of the additional practical considerations that have arisen
-during the 4+ year maintenance cycle for Python 2.7, this PEP allows
+during the 4+ year maintenance cycle for Python 2.7, this PEP allows a
-Python 2.7 standard library components that have implications for the
+critical set of network security related features to be backported from
-overall security of the internet to be updated in line with the
+Python 3.4 to the upcoming Python 2.7.7 maintenance release.
-corresponding Python 3 feature releases.
-
-Specifically, the exception allows a critical set of network security
-related features to be backported from Python 3.4 to the upcoming Python
-2.7.7 maintenance release.
 
 While this PEP does not make any changes to the core development team's
 handling of security-fix-only branches that are no longer in active
@@ -44,10 +39,10 @@ support for the use of older versions in roles that involve connecting
 directly to the public internet.
 
 
-Exemption Policy
+New security related features in Python 2.7.7
-================
+=============================================
 
-Under this policy, the following features SHOULD be backported from Python
+Under this proposal, the following features will be backported from Python
 3.4 to the upcoming Python 2.7.7 maintenance release:
 
 * in the ``os`` module:
@@ -67,20 +62,26 @@ Under this policy, the following features SHOULD be backported from Python
 * in the ``ssl`` module:
 
   * this module is almost entirely synchronised with its Python 3
-    counterpart, bringing TLSv1.2, SSLContext manipulation, Server Name
+    counterpart, bringing TLSv1.x settings, SSLContext manipulation, Server
-    Identification, access to platform certificate stores, standard
+    Name Indication, access to platform certificate stores, standard
     library support for peer hostname validation and more to the Python 2
     series.
   * the only ``ssl`` module features *not* backported under this policy are
     the ``ssl.RAND_*`` functions that provide access to OpenSSL's random
     number generation capabilities - use ``os.urandom()`` instead.
 
-As part of this policy, permission is also granted to upgrade to newer
+As a general change in maintenance policy, permission is also granted to
-feature releases of OpenSSL when preparing the binary installers
+upgrade to newer feature releases of OpenSSL when preparing the binary
-for new maintenance releases of Python 2.7.
+installers for new maintenance releases of Python 2.7.
+
+This PEP does NOT propose a general exception for backporting new features
+to Python 2.7 - every new feature proposed for backporting will still need
+to be justified independently. In particular, it will need to be explained
+why relying on and independently updated backport on the Python Package Index
+instead is not an acceptable solution.
 
 
-Backwards Compatibility Considerations
+Backwards compatibility considerations
 ======================================
 
 As in the Python 3 series, the backported ``ssl.create_default_context()``
@@ -98,7 +99,7 @@ to write more secure cross-version compatible Python software, while still
 limiting the risk of breaking currently working software when upgrading to
 a new Python 2.7 maintenance release.
 
-In all cases where this policy allows new features to be backported to
+In all cases where this proposal allows new features to be backported to
 the Python 2.7 release series, it is possible to write cross-version
 compatible code that operates by "feature detection" (for example, checking
 for particular attributes in a module), without needing to explicitly check
@@ -137,7 +138,7 @@ consideration.
 OpenSSL compatibility
 ---------------------
 
-Under this policy, OpenSSL may be upgraded to more recent feature releases
+Under this proposal, OpenSSL may be upgraded to more recent feature releases
 in Python 2.7 maintenance releases. On Linux and most other POSIX systems,
 the specific version of OpenSSL used already varies, as CPython dynamically
 links to the system provided OpenSSL library by default.
@@ -169,6 +170,10 @@ expressed interest in carrying out the feature backports covered by this
 policy, and assisting with any additional maintenance burdens that arise
 in the Python 2 series as a result.
 
+Steve Dower and Brian Curtin have offered to help with the creation of the
+Windows installers, allowing Martin von Löwis the opportunity to step back
+from the task of maintaining the 2.7 Windows installer.
+
 This PEP is primarily about establishing the consensus needed to allow them
 to carry out this work. For other core developers, this policy change
 shouldn't impose any additional effort beyond potentially reviewing the
@@ -176,18 +181,6 @@ resulting patches for those developers specifically interested in the
 affected modules.
 
 
-Documentation
--------------
-
-All modules covered by this policy MUST include a "Security Considerations"
-section in their documentation in order to take advantage of this policy.
-
-In addition to any other module specific contents, this section SHOULD
-enumerate key security enhancements and fixes (with CVE identifiers where
-applicable), along with the feature and maintenance releases that first
-included them.
-
-
 Security releases
 -----------------
 
@@ -197,8 +190,8 @@ include only critical security fixes.
 
 However, the recommendations for library and application developers are
 deliberately designed to accommodate commercial redistributors that choose
-to apply this policy to additional Python release series that are either in
+to apply these changes to additional Python release series that are either
-security fix only mode, or have been declared "end of life" by the core
+in security fix only mode, or have been declared "end of life" by the core
 development team.
 
 Whether or not redistributors choose to exercise that option will be up
@@ -209,12 +202,12 @@ Integration testing
 -------------------
 
 Third party integration testing services should offer users the ability
-to test against specific Python 2.7 maintenance releases, to ensure that
+to test against multiple Python 2.7 maintenance releases (at least 2.7.6
-libraries, frameworks and applications can still test their handling of the
+and 2.7.7+), to ensure that libraries, frameworks and applications can still
-legacy security infrastructure correctly (either failing or degrading
+test their handling of the legacy security infrastructure correctly (either
-gracefully, depending on the security sensitivity of the software), even
+failing or degrading gracefully, depending on the security sensitivity of
-after the features covered in this policy have been backported to the
+the software), even after the features covered in this proposal have been
-Python 2.7 series.
+backported to the Python 2.7 series.
 
 
 Handling lower security environments with low risk tolerance
@@ -222,7 +215,7 @@ Handling lower security environments with low risk tolerance
 
 For better or for worse (mostly worse), there are some environments where
 the risk of latent security defects is more tolerated than even a slightly
-increased risk of regressions in maintenance releases. This policy largely
+increased risk of regressions in maintenance releases. This proposal largely
 excludes these environments from consideration where the modules covered by
 the exemption are concerned - this approach is entirely inappropriate for
 software connected to the public internet, and defence in depth security
@@ -236,28 +229,6 @@ The main CPython continuous integration infrastructure will not cover this
 scenario.
 
 
-Evolution of this Policy
-========================
-
-The key requirement for a feature to be considered for inclusion in this
-policy is that it must have security implications *beyond* the specific
-application that is written in Python and the system that application is
-running on. Thus the focus on network security protocols, password storage
-and related cryptographic infrastructure - Python is a popular choice for
-the development of web services and clients, and thus the capabilities of
-widely used Python versions have implications for the security design of
-other services that may themselves be using newer versions of Python or
-other development languages, but need to interoperate with clients or
-servers written using older versions of Python.
-
-The intent behind this requirement is to minimise any impact that the
-introduction of this policy may have on the stability and compatibility of
-maintenance releases. It would be thoroughly counterproductive if end
-users became as cautious about updating to new Python 2.7 maintenance
-releases as they are about updating to new feature releases within the
-same release series.
-
-
 Motivation and Rationale
 ========================
 
@@ -280,19 +251,26 @@ a position to migrate to Python 3.
 While the use of the system OpenSSL installation addresses many of these
 concerns on Linux platforms, it doesn't address all of them (in particular,
 it is still difficult for sotware to explicitly require some higher level
-security settings). In the case of the binary installers for Windows and
+security settings). The standard library support can be bypassed by using a
-Mac OS X that are published on python.org, the version of OpenSSL used is
+third party library like PyOpenSSL or Pycurl, but this still results in a
-entirely within the control of the Python core development team, but is
+security problem, as these can be difficult dependencies to deploy, and many
-currently limited to OpenSSL maintenance releases for the version initially
+users will remain unaware that they might want them. Rather than explaining
-shipped with the corresponding Python feature release.
+to potentially naive users how to obtain and use these libraries, it seems
+better to just fix the included batteries.
 
-With increased popularity comes increased responsibility, and this policy
+In the case of the binary installers for Windows and Mac OS X that are
+published on python.org, the version of OpenSSL used is entirely within
+the control of the Python core development team, but is currently limited
+to OpenSSL maintenance releases for the version initially shipped with the
+corresponding Python feature release.
+
+With increased popularity comes increased responsibility, and this proposal
 aims to acknowledge the fact that Python's popularity and adoption is at a
 sufficiently high level that some of our design and policy decisions have
 significant implications beyond the Python development community.
 
 As one example, the Python 2 ``ssl`` module does not support the Server
-Name Identification standard. While it is possible to obtain SNI support
+Name Indication standard. While it is possible to obtain SNI support
 by using the third party ``requests`` client library, actually doing so
 currently requires using not only ``requests`` and its embedded dependencies,
 but also half a dozen or more additional libraries. The lack of support
@@ -315,19 +293,6 @@ even consider the issue and use ordinary equality comparisons instead
 it *does* make the barrier to resolution much lower once the problem is
 pointed out.
 
-My position on the ongoing transition from Python 2 to Python 3 has long
-been that Python 2 remains a supported platform for the core development
-team, and that commercial support will remain available well after upstream
-maintenance ends. However, in the absence of this network security
-enhancement policy, that position is difficult to justify when it comes to
-software that operates over the public internet. Just as many developers
-consider it too difficult to develop truly secure modern networked software
-in C/C++ (largely due to the challenges associated with manual
-memory management), I anticipate that in the not too distant future, it
-will be considered too difficult to develop truly secure modern networked
-software using the Python 2 series (some developers would argue that we
-have already reached that point).
-
 Python 2.7 represents the only long term maintenance release the core
 development team has provided, and it is natural that there will be things
 that worked over a historically shorter maintenance lifespan that don't work
@@ -337,22 +302,23 @@ that long term maintenance of network security related modules *requires*
 the ability to add new features, even while retaining backwards compatibility
 for existing interfaces.
 
-It is worth comparing the approach described in this PEP with Red Hat's
+For those familiar with it, it is worth comparing the approach described in
-handling of its long term support commitments: it isn't the RHEL 6.0 release
+this PEP with Red Hat's handling of its long term open source support
-itself that receives 10 years worth of support, but the overall RHEL 6
+commitments: it isn't the RHEL 6.0 release itself that receives 10 years
-*series*. The individual RHEL 6.x point releases within the series then
+worth of support, but the overall RHEL 6 *series*. The individual RHEL 6.x
-receive a wide variety of new features, including security enhancements,
+point releases within the series then receive a wide variety of new
-all while meeting strict backwards compatibility guarantees for existing
+features, including security enhancements, all while meeting strict
-software. The policy described in this PEP brings our approach to long term
+backwards compatibility guarantees for existing software. The proposal
-maintenance more into line with this precedent - we retain our strict
+covered in this PEP brings our approach to long term maintenance more into
-backwards compatibility requirements, but slightly relax the restrictions
+line with this precedent - we retain our strict backwards compatibility
-against adding new features.
+requirements, but make an exception to the restriction against adding new
+features.
 
 To date, downstream redistributors have respected our upstream policy of
 "no new features in Python maintenance releases". This PEP explicitly
 accepts that a more nuanced policy is appropriate in the case of network
-security related features, and the specific one it describes is deliberately
+security related features, and the specific change it describes is
-designed such that it at least has some chance of being applied to Red Hat
+deliberately designed such that it is potentially suitable for Red Hat
 Enterprise Linux and its downstream derivatives.
 
 
@@ -515,19 +481,19 @@ Earlier versions of this PEP included the concept of a ``2.7-legacy-ssl``
 branch that preserved the exact feature set of the Python 2.7.6 network
 security infrastructure.
 
-It is the opinion of the PEP author that anyone that actually wants this is
+In my opinion, anyone that actually wants this is almost certainly making a
-almost certainly making a mistake, and if they insist they really do want
+mistake, and if they insist they really do want it in their specific
-it in their specific situation, they're welcome to either make it themselves
+situation, they're welcome to either make it themselves or arrange for a
-or arrange for a downstream redistributor to make it for them.
+downstream redistributor to make it for them.
 
 If they are made publicly available, any such rebuilds should be referred to
 as "Python 2.7 with Legacy SSL" to clearly distinguish them from the official
 Python 2.7 releases that include more up to date network security
 infrastructure.
 
-After the first Python 2.7 maintenance release that has the security
+After the first Python 2.7 maintenance release that implements this PEP, it
-infrastructure updated to match Python 3.4, it would also be appropriate to
+would also be appropriate to refer to Python 2.7.6 and earlier releases as
-refer to Python 2.7.6 and earlier releases as "Python 2.7 with Legacy SSL".
+"Python 2.7 with Legacy SSL".
 
 
 Rejected variant: synchronise particular modules entirely with Python 3
@@ -540,26 +506,18 @@ This approach proved too vague to build a compelling case for the exception,
 and has thus been replaced by the current more explicit proposal.
 
 
-Open Questions
+Rejected variant: open ended backport policy
-==============
+--------------------------------------------
 
-* MvL has indicated he is not prepared to tackle the task of trying to
+Earlier versions of this PEP suggested a general policy change related to
-  integrate a newer OpenSSL into the also aging Python 2.7 build
+future Python 3 enhancements that impact the general security of the
-  infrastructure on Windows (unfortunately, we've looked into upgrading
+internet.
-  that build infrastructure, and the backwards compatibility issues
-  appear to be effectively insurmountable). We would require a commitment
-  from another trusted contributor to handle at least this task, and
-  potentially also taking over the task of creating the official
-  Python 2.7 Windows installers for the remaining Python 2.7 maintenance
-  releases.
 
-* We would need commitments to create and review full backports of the
+That approach created unnecessary uncertainty, so it has been simplified to
-  components covered by this policy from Python 3.4 to Python 2.7, as well
+propose backport a specific concrete set of changes. Future feature
-  as support for handling any more specific security issues affecting these
+backport proposals can refer back to this PEP as precedent, but it will
-  modules.
+still be necessary to make a specific case for each feature addition to
-
+the Python 2.7 long term support release.
-* Did I miss anything important in the switch to a more restrictive
-  proposal?
 
 
 Disclosure of Interest
@@ -591,7 +549,8 @@ Christian and Donald also provided valuable feedback on a preliminary
 draft of this proposal.
 
 Thanks also to participants in the python-dev mailing list threads
-[1,2,5,6]_
+[1,2,5,6]_, as well as the various folks I discussed this issue with at
+PyCon 2014 in Montreal.
 
 
 References