diff --git a/pep-0594.rst b/pep-0594.rst index c6321c9fe..ad9795ce5 100644 --- a/pep-0594.rst +++ b/pep-0594.rst @@ -5,7 +5,7 @@ Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 20-May-2019 -Post-History: +Post-History: 21-May-2019 Abstract @@ -96,7 +96,8 @@ features should be added. Bug should still be fixed. --- Starting with Python 3.9, deprecated modules will start issuing -`DeprecationWarning`. +`DeprecationWarning`. The `parser`_ module is removed and potentially +replaced with a new module. 3.10 @@ -116,7 +117,9 @@ analogous Python's two release deprecation process. The first *provisionally accepted* phase targets Python 3.8.0b1. In the first phase no code is changes or removed. Modules are only documented as -deprecated. +deprecated. The only exception is the `parser`_ module. It has been +documented as deprecated since Python 2.5 and is scheduled for removal for +3.9 to make place for a more advanced parser. The final decision, which modules will be removed and how the removed code is preserved, can be delayed for another year. @@ -133,6 +136,7 @@ audio processing. .. csv-table:: Table 1: Proposed modules deprecations :header: "Module", "Deprecated in", "To be removed", "Replacement" + :widths: 1, 1, 1, 2 aifc,3.8,3.10,\- asynchat,3.8,3.10,asyncio @@ -142,13 +146,13 @@ audio processing. cgi,3.8,3.10,\- cgitb,3.8,3.10,\- chunk,3.8,3.10,\- - colorsys,**3.8?**,**3.10?**,\- - crypt,3.8,3.10,\- - fileinput,3.8,3.10,argparse + colorsys,3.8,3.10,"colormath, colour, colorspacious, Pillow" + crypt,3.8,3.10,"bcrypt, argon2cffi, hashlib, passlib" + fileinput,\-,**keep**,argparse formatter,3.4,3.10,\- fpectl,**3.7**,**3.7**,\- getopt,**3.2**,**keep**,"argparse, optparse" - imghdr,3.8,3.10,\- + imghdr,3.8,3.10,"filetype, puremagic, python-magic" imp,**3.4**,3.10,importlib lib2to3,\-,**keep**, macpath,**3.7**,**3.8**,\- @@ -157,10 +161,11 @@ audio processing. nis,3.8,3.10,\- optparse,\-,**keep**,argparse ossaudiodev,3.8,3.10,\- + parser,**2.5**,**3.9**,"ast, lib2to3.pgen2" pipes,3.8,3.10,subprocess - smtpd,**3.7**,3.10,aiosmtpd - sndhdr,3.8,3.10,\- - spwd,3.8,3.10,\- + smtpd,"**3.4.7**, **3.5.4**",3.10,aiosmtpd + sndhdr,3.8,3.10,"filetype, puremagic, python-magic" + spwd,3.8,3.10,"python-pam, simplepam" sunau,3.8,3.10,\- uu,3.8,3.10,\- wave,\-,**keep**, @@ -250,7 +255,8 @@ The `audioop `_ module contains helper functions to manipulate raw audio data and adaptive differential pulse-code modulated audio data. The module is implemented in C without any additional dependencies. The `aifc`_, `sunau`_, and `wave`_ -module depend on `audioop`_ for some operations. +module depend on `audioop`_ for some operations. The byteswap operation in +the `wave`_ module can be substituted with little work. Module type C extension @@ -266,8 +272,11 @@ colorsys The `colorsys `_ module defines color conversion functions between RGB, YIQ, HSL, and HSV coordinate -systems. The Pillow library provides much faster conversation between -color systems. +systems. + +The PyPI packages *colormath*, *colour*, and *colorspacious* provide more and +advanced features. The Pillow library is better suited to transform images +between color systems. Module type pure Python @@ -276,8 +285,10 @@ Deprecated in To be removed in 3.10 Substitute - `Pillow `_, - `colorspacious `_ + `colormath `_, + `colour `_ + `colorspacious `_, + `Pillow `_ chunk ~~~~~ @@ -311,7 +322,9 @@ Deprecated in To be removed in 3.10 Substitute - *n/a* + `puremagic `_, + `filetype `_, + `python-magic `_ ossaudiodev ~~~~~~~~~~~ @@ -348,7 +361,9 @@ Deprecated in To be removed in 3.10 Substitute - *n/a* + `puremagic `_, + `filetype `_, + `python-magic `_ sunau ~~~~~ @@ -439,6 +454,10 @@ cgitb The `cgitb `_ module is a helper for the cgi module for configurable tracebacks. +The ``cgitb`` module is not used by any major Python web framework (Django, +Pyramid, Plone, Flask, CherryPy, or Bottle). Only Paste uses it in an +optional debugging middleware. + Module type pure Python Deprecated in @@ -453,7 +472,8 @@ smtpd The `smtpd `_ module provides a simple implementation of a SMTP mail server. The module documentation -recommends ``aiosmtpd``. +marks the module as deprecated and recommends ``aiosmtpd`` instead. The +deprecation message was added in releases 3.4.7, 3.5.4, and 3.6.1. Module type pure Python @@ -471,11 +491,15 @@ The `nntplib `_ module implements the client side of the Network News Transfer Protocol (nntp). News groups used to be a dominant platform for online discussions. Over the last two decades, news has been slowly but steadily replaced with mailing lists -and web-based discussion platforms. +and web-based discussion platforms. Twisted is also +`planning `_ to deprecate NNTP +support. The ``nntplib`` tests have been the cause of additional work in the recent -past. Python only contains client side of NNTP. The test cases depend on -external news server. These servers were unstable in the past. +past. Python only contains client side of NNTP. The tests connect to +external news server. The servers are sometimes unavailble, too slow, or do +not work correctly over IPv6. The situation causes flaky test runs on +buildbots. Module type pure Python @@ -508,6 +532,10 @@ quality and insecure. Users are discouraged to use them. commonly available on Linux. * Depending on the platform, the ``crypt`` module is not thread safe. Only implementations with ``crypt_r(3)`` are thread safe. +* The module was never useful to interact with system user and password + databases. On BSD, macOS, and Linux, all user authentication and + password modification operations must go through PAM (pluggable + authentication module), see `spwd`_ deprecation. Module type C extension + Python module @@ -561,9 +589,18 @@ spwd The `spwd `_ module provides direct access to Unix shadow password database using non-standard APIs. + In general it's a bad idea to use the spwd. The spwd circumvents system -security policies, it does not use the PAM stack, and is -only compatible with local user accounts. +security policies, it does not use the PAM stack, and is only compatible +with local user accounts, because it ignores NSS. The use of the ``spwd`` +module for access control must be consider a *security bug*, as it bypasses +PAM's access control. + +Further more the ``spwd`` module uses the +`shadow(3) `_ APIs. +Functions like ``getspnam(3)`` access the ``/etc/shadow`` file directly. This +is dangerous and even forbidden for confined services on systems with a +security engine like SELinux or AppArmor. Module type C extension @@ -572,28 +609,12 @@ Deprecated in To be removed in 3.10 Substitute - **none** + `python-pam `_, + `simpleplam `_ Misc modules ------------ -fileinput -~~~~~~~~~ - -The `fileinput `_ module -implements a helpers to iterate over a list of files from ``sys.argv``. The -module predates the optparser and argparser module. The same functionality -can be implemented with the argparser module. - -Module type - pure Python -Deprecated in - 3.8 -To be removed in - 3.10 -Substitute - argparse - formatter ~~~~~~~~~ @@ -649,6 +670,35 @@ To be removed in Substitute **none** +parser +~~~~~~ + +The `parser `_ module provides +an interface to Python’s internal parser and byte-code compiler. The stdlib +has superior ways to interact with the parse tree. From Python 2.5 onward, +it's much more convenient to cut in at the Abstract Syntax Tree (AST) +generation and compilation stage. + +The ``parser`` module causes additional work. It's C code that must be +kept in sync with any change to Python's grammar and internal parser. +Pablo wants to remove the parser module and promote lib2to3's pgen2 instead +[6]_. + +Most importantly the presence of the ``parser`` module makes it harder to +switch to something more powerful than a LL(1) parser [7]_. Since the +``parser`` module is documented as deprecated since Python 2.5 and a new +parsing technology is planned for 3.9, the ``parser`` module is scheduled for +removal in 3.9. + +Module type + C extension +Deprecated in + 3.8, documented as deprecated since **2.5** +To be removed in + **3.9** +Substitute + ast, lib2to3.pgen2 + pipes ~~~~~ @@ -693,6 +743,20 @@ Modules to keep Some modules were originally proposed for deprecation. +fileinput +--------- + +The `fileinput `_ module +implements a helpers to iterate over a list of files from ``sys.argv``. The +module predates the optparser and argparser module. The same functionality +can be implemented with the argparser module. + +Several core developers expressed their interest to keep the module in the +standard library, as it is handy for quick scripts. + +Module type + pure Python + lib2to3 ------- @@ -798,8 +862,26 @@ Discussions * Multiple people (Gregory P. Smith, David Beazley, Nick Coghlan, ...) convinced me to keep the `wave`_ module. [4]_ * Gregory P. Smith proposed to deprecate `nntplib`_. [4]_ +* Andrew Svetlov mentioned the ``socketserver`` module is questionable. + However it's used to implement ``http.server`` and ``xmlrpc.server``. The + stdlib doesn't have a replacement for the servers, yet. +Update history +============== + +Update 1 +-------- + +* Deprecate `parser`_ module +* Keep `fileinput`_ module +* Elaborate why `crypt`_ and `spwd`_ are dangerous and bad +* Improve sections for `cgitb`_, `colorsys`_, `nntplib`_, and `smtpd`_ modules +* The `colorsys`_, `crypt`_, `imghdr`_, `sndhdr`_, and `spwd`_ sections now + list suitable substitutions. +* Mention that ``socketserver`` is going to stay for ``http.server`` and + ``xmlrpc.server`` + References ========== @@ -808,6 +890,8 @@ References .. [3] https://blogs.msmvps.com/installsite/blog/2015/05/03/the-future-of-windows-installer-msi-in-the-light-of-windows-10-and-the-universal-windows-platform/ .. [4] https://twitter.com/ChristianHeimes/status/1130257799475335169 .. [5] https://twitter.com/dabeaz/status/1130278844479545351 +.. [6] https://mail.python.org/pipermail/python-dev/2019-May/157464.html +.. [7] https://discuss.python.org/t/switch-pythons-parsing-tech-to-something-more-powerful-than-ll-1/379 Copyright