iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

PEP 597: Add EncodingWarning (#1788)

This commit is contained in:
Inada Naoki 2021-01-30 18:18:19 +09:00 committed by GitHub
parent e13d3c1998
commit b2ce9f86f5
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 79 additions and 94 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

173
pep-0597.rst
View File

@ -1,6 +1,6 @@
PEP: 597 PEP: 597
Title: Soft deprecation of default encoding Title: Add optional EncodingWarning
Last-Modified: 23-Jun-2020 Last-Modified: 30-Jan-2021
Author: Inada Naoki <songofacandy@gmail.com> Author: Inada Naoki <songofacandy@gmail.com>
Discussions-To: https://discuss.python.org/t/3880 Discussions-To: https://discuss.python.org/t/3880
Status: Draft Status: Draft
@ -13,16 +13,13 @@ Python-Version: 3.10
Abstract Abstract
======== ========
This PEP proposes: Add a new warning category ``EncodingWarning``. It is emitted when
``encoding`` option is ommitted and the default encoding is locale
encoding.
* ``TextIOWrapper`` raises a ``PendingDeprecationWarning`` when the The warning is disabled by default. New ``-X warn_encoding``
``encoding`` option is not specified and dev mode is enabled. commandline option and ``PYTHONWARNENCODING`` environment variable
are used to enable the warnings.
* Add ``encoding="locale"`` option to ``TextIOWrapper``. It behaves
like ``encoding=None`` but don't raise a warning.
* Add ``io.LOCALE_ENCODING = "locale"`` constant to avoid confusing
``LookupError``.
Motivation Motivation
@ -67,108 +64,123 @@ But this change will affect many applications and libraries.
Many ``DeprecationWarning`` will be raised if we start raising Many ``DeprecationWarning`` will be raised if we start raising
the warning by default. It will be too noisy. the warning by default. It will be too noisy.
While this PEP doesn't cover the change, this PEP will help to reduce While this PEP doesn't cover the change, this PEP will help to
the number of ``DeprecationWarning`` in the future. reduce the number of ``DeprecationWarning`` in the future.
Specification Specification
============= =============
Raising a PendingDeprecationWarning ``EncodingWarning``
--------------------------------------- --------------------
``TextIOWrapper`` raises the ``PendingDeprecationWarning`` when the Add new ``EncodingWarning`` warning class which is a subclass of
``encoding`` option is omitted and dev mode is enabled. ``Warning``. It is used to warn when ``encoding`` option is omitted
and the default encoding is locale-specific.
Options to enable the warning
------------------------------
``-X warn_encoding`` option and the ``PYTHONWARNENCODING``
environment variable are added. They are used to enable the
``EncodingWarning``.
``sys.flags.encoding_warning`` is also added. It is a boolean flag
represents ``EncodingWarning`` is enabled.
When the option is enabled, ``io.TextIOWrapper()``, ``open()``, and
other modules emit ``EncodingWarning`` when ``encoding`` is omitted.
``encoding="locale"`` option ``encoding="locale"`` option
---------------------------- ----------------------------
When ``encoding="locale"`` is specified to the ``TextIOWrapper``, it ``io.TextIOWrapper`` accepts ``encoding="locale"`` option. It means
behaves same to ``encoding=None`` except it doesn't raise warning. same to current ``encoding=None``. But ``io.TextIOWrapper`` doesn't
In detail, the encoding is chosen by this order: emit ``EncodingWarning`` when ``encoding="locale"`` is specified.
1. ``os.device_encoding(buffer.fileno())`` Add ``io.LOCALE_ENCODING = "locale"`` constant too. This constant can
2. ``locale.getpreferredencoding(False)`` be used to avoid confusing ``LookupError: unknown encoding: locale``
error when the code is run in old Python accidentally.
This option can be used to use the locale encoding explicitly and The constant can be used to test that ``encoding="locale"`` option is
suppress the ``PendingDeprecationWarning``. supported too. For example,
.. code-block::
``io.LOCALE_ENCODING`` # Want to suppress an EncodingWarning but still need support
----------------------
``io`` module has ``io.LOCALE_ENCODING = "locale"`` constant. This
constant can be used to avoid confusing ``LookupError: unknown
encoding: locale`` error when the code is run in Python older than
3.10 accidentally.
The constant can be used to test that ``encoding="locale"`` option
is supported too.
::
# Want to suppress the Warning in dev mode but still need support
# old Python versions. # old Python versions.
locale_encoding = getattr(io, "LOCALE_ENCODING", None) locale_encoding = getattr(io, "LOCALE_ENCODING", None)
with open(filename, encoding=locale_encoding) as f: with open(filename, encoding=locale_encoding) as f:
... ...
``io.text_encoding`` ``io.text_encoding()``
-------------------- -----------------------
``TextIOWrapper`` is used indirectly in most cases. For example, ``io.text_encoding()`` is a helper function for functions having
``open``, and ``pathlib.Path.read_text()`` use it. Warning to these ``encoding=None`` option and pass it to ``io.TextIOWrapper()`` or
functions doesn't make sense. Callers of these functions should be ``open()``.
warned instead.
``io.text_encoding(encoding, stacklevel=1)`` is a helper function for Pure Python implementation will be like this::
it. Pure Python implementation will be like this::
def text_encoding(encoding, stacklevel=1): def text_encoding(encoding, stacklevel=1):
""" """Helper function to choose the text encoding.
Helper function to choose the text encoding.
When encoding is not None, just return it. When *encoding* is not None, just return it.
Otherwise, return the default text encoding ("locale" for now), Otherwise, return the default text encoding (i.e., "locale").
and raise a PendingDeprecationWarning in dev mode.
This function can be used in APIs having encoding=None option. This function emits EncodingWarning if *encoding* is None and
But please consider encoding="utf-8" for new APIs. sys.flags.encoding_warning is true.
This function can be used in APIs having encoding=None option
and pass it to TextIOWrapper or open.
But please consider using encoding="utf-8" for new APIs.
""" """
if encoding is None: if encoding is None:
if sys.flags.dev_mode: if sys.flags.encoding_warning:
import warnings import warnings
warnings.warn( warnings.warn("'encoding' option is omitted",
"'encoding' option is not specified. The default encoding " EncodingWarning, stacklevel + 2)
"might be changed to 'utf-8' in the future",
PendingDeprecationWarning, stacklevel + 2)
encoding = LOCALE_ENCODING encoding = LOCALE_ENCODING
return encoding return encoding
``pathlib.Path.read_text()`` can use this function like this:: For example, ``pathlib.Path.read_text()`` can use the function like:
.. code-block::
def read_text(self, encoding=None, errors=None): def read_text(self, encoding=None, errors=None):
"""
Open the file in text mode, read it, and close the file.
"""
encoding = io.text_encoding(encoding) encoding = io.text_encoding(encoding)
with self.open(mode='r', encoding=encoding, errors=errors) as f: with self.open(mode='r', encoding=encoding, errors=errors) as f:
return f.read() return f.read()
subprocess module doesn't warn ``subprocess`` module
------------------------------ ----------------------
While the subprocess module uses TextIOWrapper, it doesn't raise The default encoding for pipe in the subprocess module is changed
``PendingDeprecationWarning``. It uses the ``io.LOCALE_ENCODING`` to ``io.LOCALE_ENCODING``. In other words, subprocess module doesn't
by default. emit the ``EncodingWarning``.
The default encoding for PIPE is relating to the encoding of the
stdio than the default encoding of ``TextIOWrapper``. So this PEP
doesn't propose to emit the warning for pipes.
Rationale Rationale
========= =========
Opt-in warning
---------------
Although ``DeprecationWarning`` is supressed by default, emitting
``DeprecationWarning`` alwasy when ``encoding`` option is omitted
would be too noisy.
Noisy warnings may leads developers to dismiss the ``DeprecationWarning``.
"locale" is not a codec alias "locale" is not a codec alias
----------------------------- -----------------------------
@ -180,33 +192,6 @@ when ``encoding=None``. This behavior can not be implemented in
the codec. the codec.
Use a PendingDeprecationWarning
-------------------------------
This PEP doesn't cover changing the default encoding to UTF-8.
So we use ``PendingDeprecationWarning`` instead of
``DeprecationWarning`` for now.
Raise warning only in dev mode
------------------------------
This PEP will produce a huge amount of ``PendingDeprecationWarning``.
It will be too noisy for most Python developers.
We need to fix all warnings in the standard library. We need to wait
pip and major dev tools like ``pytest`` fix warnings before raising
this warning by default.
subprocess module doesn't warn
------------------------------
The default encoding for PIPE is relating to the encoding of the
stdio than the default encoding of ``TextIOWrapper``. So this PEP
doesn't propose to raise warning from the subprocess module.
Reference Implementation Reference Implementation
======================== ========================