PEP 597: Add EncodingWarning (#1788)
This commit is contained in:
Inada Naoki
GitHub
parent
e13d3c1998
commit
b2ce9f86f5
173
pep-0597.rst
173
pep-0597.rst
|
|
@ -1,6 +1,6 @@
|
|||
PEP: 597
|
||||
Title: Soft deprecation of default encoding
|
||||
Last-Modified: 23-Jun-2020
|
||||
Title: Add optional EncodingWarning
|
||||
Last-Modified: 30-Jan-2021
|
||||
Author: Inada Naoki <songofacandy@gmail.com>
|
||||
Discussions-To: https://discuss.python.org/t/3880
|
||||
Status: Draft
|
||||
|
|
@ -13,16 +13,13 @@ Python-Version: 3.10
|
|||
Abstract
|
||||
========
|
||||
|
||||
This PEP proposes:
|
||||
Add a new warning category ``EncodingWarning``. It is emitted when
|
||||
``encoding`` option is ommitted and the default encoding is locale
|
||||
encoding.
|
||||
|
||||
* ``TextIOWrapper`` raises a ``PendingDeprecationWarning`` when the
|
||||
``encoding`` option is not specified and dev mode is enabled.
|
||||
|
||||
* Add ``encoding="locale"`` option to ``TextIOWrapper``. It behaves
|
||||
like ``encoding=None`` but don't raise a warning.
|
||||
|
||||
* Add ``io.LOCALE_ENCODING = "locale"`` constant to avoid confusing
|
||||
``LookupError``.
|
||||
The warning is disabled by default. New ``-X warn_encoding``
|
||||
commandline option and ``PYTHONWARNENCODING`` environment variable
|
||||
are used to enable the warnings.
|
||||
|
||||
|
||||
Motivation
|
||||
|
|
@ -67,108 +64,123 @@ But this change will affect many applications and libraries.
|
|||
Many ``DeprecationWarning`` will be raised if we start raising
|
||||
the warning by default. It will be too noisy.
|
||||
|
||||
While this PEP doesn't cover the change, this PEP will help to reduce
|
||||
the number of ``DeprecationWarning`` in the future.
|
||||
While this PEP doesn't cover the change, this PEP will help to
|
||||
reduce the number of ``DeprecationWarning`` in the future.
|
||||
|
||||
|
||||
Specification
|
||||
=============
|
||||
|
||||
Raising a PendingDeprecationWarning
|
||||
---------------------------------------
|
||||
``EncodingWarning``
|
||||
--------------------
|
||||
|
||||
``TextIOWrapper`` raises the ``PendingDeprecationWarning`` when the
|
||||
``encoding`` option is omitted and dev mode is enabled.
|
||||
Add new ``EncodingWarning`` warning class which is a subclass of
|
||||
``Warning``. It is used to warn when ``encoding`` option is omitted
|
||||
and the default encoding is locale-specific.
|
||||
|
||||
|
||||
Options to enable the warning
|
||||
------------------------------
|
||||
|
||||
``-X warn_encoding`` option and the ``PYTHONWARNENCODING``
|
||||
environment variable are added. They are used to enable the
|
||||
``EncodingWarning``.
|
||||
|
||||
``sys.flags.encoding_warning`` is also added. It is a boolean flag
|
||||
represents ``EncodingWarning`` is enabled.
|
||||
|
||||
When the option is enabled, ``io.TextIOWrapper()``, ``open()``, and
|
||||
other modules emit ``EncodingWarning`` when ``encoding`` is omitted.
|
||||
|
||||
|
||||
``encoding="locale"`` option
|
||||
----------------------------
|
||||
|
||||
When ``encoding="locale"`` is specified to the ``TextIOWrapper``, it
|
||||
behaves same to ``encoding=None`` except it doesn't raise warning.
|
||||
In detail, the encoding is chosen by this order:
|
||||
``io.TextIOWrapper`` accepts ``encoding="locale"`` option. It means
|
||||
same to current ``encoding=None``. But ``io.TextIOWrapper`` doesn't
|
||||
emit ``EncodingWarning`` when ``encoding="locale"`` is specified.
|
||||
|
||||
1. ``os.device_encoding(buffer.fileno())``
|
||||
2. ``locale.getpreferredencoding(False)``
|
||||
Add ``io.LOCALE_ENCODING = "locale"`` constant too. This constant can
|
||||
be used to avoid confusing ``LookupError: unknown encoding: locale``
|
||||
error when the code is run in old Python accidentally.
|
||||
|
||||
This option can be used to use the locale encoding explicitly and
|
||||
suppress the ``PendingDeprecationWarning``.
|
||||
The constant can be used to test that ``encoding="locale"`` option is
|
||||
supported too. For example,
|
||||
|
||||
.. code-block::
|
||||
|
||||
``io.LOCALE_ENCODING``
|
||||
----------------------
|
||||
|
||||
``io`` module has ``io.LOCALE_ENCODING = "locale"`` constant. This
|
||||
constant can be used to avoid confusing ``LookupError: unknown
|
||||
encoding: locale`` error when the code is run in Python older than
|
||||
3.10 accidentally.
|
||||
|
||||
The constant can be used to test that ``encoding="locale"`` option
|
||||
is supported too.
|
||||
|
||||
::
|
||||
|
||||
# Want to suppress the Warning in dev mode but still need support
|
||||
# Want to suppress an EncodingWarning but still need support
|
||||
# old Python versions.
|
||||
locale_encoding = getattr(io, "LOCALE_ENCODING", None)
|
||||
with open(filename, encoding=locale_encoding) as f:
|
||||
...
|
||||
|
||||
|
||||
``io.text_encoding``
|
||||
--------------------
|
||||
``io.text_encoding()``
|
||||
-----------------------
|
||||
|
||||
``TextIOWrapper`` is used indirectly in most cases. For example,
|
||||
``open``, and ``pathlib.Path.read_text()`` use it. Warning to these
|
||||
functions doesn't make sense. Callers of these functions should be
|
||||
warned instead.
|
||||
``io.text_encoding()`` is a helper function for functions having
|
||||
``encoding=None`` option and pass it to ``io.TextIOWrapper()`` or
|
||||
``open()``.
|
||||
|
||||
``io.text_encoding(encoding, stacklevel=1)`` is a helper function for
|
||||
it. Pure Python implementation will be like this::
|
||||
Pure Python implementation will be like this::
|
||||
|
||||
def text_encoding(encoding, stacklevel=1):
|
||||
"""
|
||||
Helper function to choose the text encoding.
|
||||
"""Helper function to choose the text encoding.
|
||||
|
||||
When encoding is not None, just return it.
|
||||
Otherwise, return the default text encoding ("locale" for now),
|
||||
and raise a PendingDeprecationWarning in dev mode.
|
||||
When *encoding* is not None, just return it.
|
||||
Otherwise, return the default text encoding (i.e., "locale").
|
||||
|
||||
This function can be used in APIs having encoding=None option.
|
||||
But please consider encoding="utf-8" for new APIs.
|
||||
This function emits EncodingWarning if *encoding* is None and
|
||||
sys.flags.encoding_warning is true.
|
||||
|
||||
This function can be used in APIs having encoding=None option
|
||||
and pass it to TextIOWrapper or open.
|
||||
But please consider using encoding="utf-8" for new APIs.
|
||||
"""
|
||||
if encoding is None:
|
||||
if sys.flags.dev_mode:
|
||||
if sys.flags.encoding_warning:
|
||||
import warnings
|
||||
warnings.warn(
|
||||
"'encoding' option is not specified. The default encoding "
|
||||
"might be changed to 'utf-8' in the future",
|
||||
PendingDeprecationWarning, stacklevel + 2)
|
||||
warnings.warn("'encoding' option is omitted",
|
||||
EncodingWarning, stacklevel + 2)
|
||||
encoding = LOCALE_ENCODING
|
||||
return encoding
|
||||
|
||||
``pathlib.Path.read_text()`` can use this function like this::
|
||||
For example, ``pathlib.Path.read_text()`` can use the function like:
|
||||
|
||||
.. code-block::
|
||||
|
||||
def read_text(self, encoding=None, errors=None):
|
||||
"""
|
||||
Open the file in text mode, read it, and close the file.
|
||||
"""
|
||||
encoding = io.text_encoding(encoding)
|
||||
with self.open(mode='r', encoding=encoding, errors=errors) as f:
|
||||
return f.read()
|
||||
|
||||
|
||||
subprocess module doesn't warn
|
||||
------------------------------
|
||||
``subprocess`` module
|
||||
----------------------
|
||||
|
||||
While the subprocess module uses TextIOWrapper, it doesn't raise
|
||||
``PendingDeprecationWarning``. It uses the ``io.LOCALE_ENCODING``
|
||||
by default.
|
||||
The default encoding for pipe in the subprocess module is changed
|
||||
to ``io.LOCALE_ENCODING``. In other words, subprocess module doesn't
|
||||
emit the ``EncodingWarning``.
|
||||
|
||||
The default encoding for PIPE is relating to the encoding of the
|
||||
stdio than the default encoding of ``TextIOWrapper``. So this PEP
|
||||
doesn't propose to emit the warning for pipes.
|
||||
|
||||
|
||||
Rationale
|
||||
=========
|
||||
|
||||
Opt-in warning
|
||||
---------------
|
||||
|
||||
Although ``DeprecationWarning`` is supressed by default, emitting
|
||||
``DeprecationWarning`` alwasy when ``encoding`` option is omitted
|
||||
would be too noisy.
|
||||
|
||||
Noisy warnings may leads developers to dismiss the ``DeprecationWarning``.
|
||||
|
||||
|
||||
"locale" is not a codec alias
|
||||
-----------------------------
|
||||
|
||||
|
|
@ -180,33 +192,6 @@ when ``encoding=None``. This behavior can not be implemented in
|
|||
the codec.
|
||||
|
||||
|
||||
Use a PendingDeprecationWarning
|
||||
-------------------------------
|
||||
|
||||
This PEP doesn't cover changing the default encoding to UTF-8.
|
||||
So we use ``PendingDeprecationWarning`` instead of
|
||||
``DeprecationWarning`` for now.
|
||||
|
||||
|
||||
Raise warning only in dev mode
|
||||
------------------------------
|
||||
|
||||
This PEP will produce a huge amount of ``PendingDeprecationWarning``.
|
||||
It will be too noisy for most Python developers.
|
||||
|
||||
We need to fix all warnings in the standard library. We need to wait
|
||||
pip and major dev tools like ``pytest`` fix warnings before raising
|
||||
this warning by default.
|
||||
|
||||
|
||||
subprocess module doesn't warn
|
||||
------------------------------
|
||||
|
||||
The default encoding for PIPE is relating to the encoding of the
|
||||
stdio than the default encoding of ``TextIOWrapper``. So this PEP
|
||||
doesn't propose to raise warning from the subprocess module.
|
||||
|
||||
|
||||
Reference Implementation
|
||||
========================
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue