PEP 538 is likely to be read by folks trying to understand the
implications of locale coercion in Python 3.7.
While I don't want to lose the history that we originally approved
and implemented "warn by default" behaviour, the examples are
easier to read if the warnings are omitted.
I also added inline notes for the sections affected by the change
in how the warning notifications are handled, since the implementation
note at the top is easy to miss when following a direct link to a specific
section.
It looks like setting LANG may have undesirable
side effects in some cases, and all the issues
the PEP aims to handle are resolved by setting
LC_CTYPE.
The proposal and implementation have thus been
updated to only set LC_CTYPE, even when the
target coercion locale is a full locale.
* Tidy up the abstract and emphasise the equivalence between
this proposal and long supported configuration settings
* Don't set LC_ALL (set LC_CTYPE instead)
* Add a rationale for that change
* Use GNU readline misbehaviour as a specific example of the
benefits of reconfiguring the locale
* Clarify rationale for enabling the changes by default on all
autotools-using platforms
* Mention the possibility of exposing a public API for use by
embedding platforms
The previous wording suggested that all three standard streams
were affected by the proposed change to the default error handler.
Only stdin and stdout are affected - the default error handler for
stderr continues to be "backslashreplace"
* default standard stream error handler is always "surrogateescape"
for the potential coercion target locales
* PEP 540 is now a purely optional follow-on PEP that improves the
handling of cases where none of these locales are available,
but doesn't require revisiting the changes made for this PEP
* the locale coercion and warning behaviours are now enabled by
default for all \*nix platforms, even Mac OS X
* covered the Android-specific changes to the use of `setlocale`
* state explicitly that we're aware this makes the behaviour
of standalone CPython and embedded CPython diverge, we just think
the potential benefits are sufficient to accept that downside
* note the reference implementation has yet to be updated with
these changes
- using PYTHONIOENCODING poses a compatibility problem for
Python 2 subprocesses, so use Py_SetStandardStreamEncoding
instead
- note that components checking for "no output on stderr
means success" will either need to avoid the warning or
switch to checking return codes instead
- Docker, Inc. ends with a full stop, not a comma (noted by
Jan Pokorný)
- explicitly acknowledge Charalampos Stratakis's work on the
Fedora 26 backport
- reference implementation should align with the PEP now, aside from
excluding Mac OS X from the changes
- Fedora's initial 3.6 release is going to be 3.6.1, so reword the
backport section accordingly
- PYTHONCOERCECLOCALE=0 now also disables the library warning
- PEP just refers to locale-aware/locale-independent components,
without specifically limiting that to C/C++ components
- updates reference implementation to use PYTHONCOERCECLOCALE
- removes hard dependency on PEP 540
- still notes PEP 540 covers case where no relevant C-with-UTF-8
locale is available
- clarifies that these settings are still recommended over the
legacy C locale settings for older Python 3 versions, even if
we don't recommend backporting the automatic coercion
- relies entirely on PEP 540 when no appropriate locale
is available
- uses surrogateescape on standard streams by default
- accounts for BSD-style UTF-8 locales
- avoids any reliance on the en_US-UTF-8 locale
- makes note of related GNU readline issue on Android
- PYTHONALLOWCLOCALE=1 -> PYTHONCOERCECLOCALE=0
- reword the proposed library warning
- try all of C.UTF-8, c.utf8 and en_US.UTF-8
- compare and contrast with PEP 540
- new Motivation section showing specific Docker problems
- discuss implications of "strict" error handling
- define configure options to turn the new behaviour off
Hugo van Kemenade
Adam Turner
Min ho Kim
Serhiy Storchaka
Nick Coghlan
Mariatta
Huang Huang
INADA Naoki
Barry Warsaw