iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

PEP 539: Update for draft implementation feedback 

* Don't use `bool` types in the implementation, per discussion at https://github.com/python/cpython/pull/1362#discussion_r136357583
* Update for removal of --without-threads
* miscellaneous other clarifications and updates
This commit is contained in:
Erik Bray 2017-09-08 00:12:06 +02:00 committed by Nick Coghlan
parent e9cf15666f
commit 86ca2506f3
1 changed files with 73 additions and 70 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

143
pep-0539.txt
View File

@ -16,15 +16,15 @@ Abstract
The proposal is to add a new Thread Local Storage (TLS) API to CPython which
would supersede use of the existing TLS API within the CPython interpreter,
while deprecating the existing API. The new API is named "Thread Specific
Storage (TSS) API" (see `Rationale for Proposed Solution`_ for the origin of
the name).
while deprecating the existing API. The new API is named the "Thread
Specific Storage (TSS) API" (see `Rationale for Proposed Solution`_ for the
origin of the name).
Because the existing TLS API is only used internally (it is not mentioned in
the documentation, and the header that defines it, ``pythread.h``, is not
included in ``Python.h`` either directly or indirectly), this proposal probably
only affects CPython, but might also affect other interpreter implementations
(PyPy?) that implement parts of the CPython API.
included in ``Python.h`` either directly or indirectly), this proposal
probably only affects CPython, but might also affect other interpreter
implementations (PyPy?) that implement parts of the CPython API.
This is motivated primarily by the fact that the old API uses ``int`` to
represent TLS keys across all platforms, which is neither POSIX-compliant,
@ -63,7 +63,7 @@ The specification also adds a few new features:
depend on the underlying TLS implementation. It is defined::
typedef struct {
bool _is_initialized;
int _is_initialized;
NATIVE_TSS_KEY_T _key;
} Py_tss_t;
@ -77,25 +77,26 @@ The specification also adds a few new features:
PyAPI_FUNC(Py_tss_t *) PyThread_tss_alloc(void)
PyAPI_FUNC(void) PyThread_tss_free(Py_tss_t *key)
PyAPI_FUNC(bool) PyThread_tss_is_created(Py_tss_t *key)
PyAPI_FUNC(int) PyThread_tss_is_created(Py_tss_t *key)
The first two are needed for dynamic (de-)allocation of a ``Py_tss_t``,
particularly in extension modules built with ``Py_LIMITED_API``, where
static allocation of this type is not possible due to its implementation
being opaque at build time. A value returned by ``PyThread_tss_alloc``
is the same state as initialized by ``Py_tss_NEEDS_INIT``, or ``NULL`` for
dynamic allocation failure. The behavior of ``PyThread_tss_free`` involves
calling ``PyThread_tss_delete`` preventively, or is no-op if the value
pointed to by the ``key`` argument is ``NULL``. ``PyThread_tss_is_created``
returns ``true`` if the given ``Py_tss_t`` has been initialized (i.e. by
``PyThread_tss_create``).
being opaque at build time. A value returned by ``PyThread_tss_alloc`` is
in the same state as a value initialized with ``Py_tss_NEEDS_INIT``, or
``NULL`` in the case of dynamic allocation failure. The behavior of
``PyThread_tss_free`` involves calling ``PyThread_tss_delete``
preventively, or is a no-op if the value pointed to by the ``key``
argument is ``NULL``. ``PyThread_tss_is_created`` returns non-zero if the
given ``Py_tss_t`` has been initialized (i.e. by ``PyThread_tss_create``).
The new TSS API does not provide functions which correspond to
``PyThread_delete_key_value`` and ``PyThread_ReInitTLS``, because these
functions are for the removed CPython's own TLS implementation, that is the
existing API behavior has become as follows: ``PyThread_delete_key_value(key)``
is equal to ``PyThread_set_key_value(key, NULL)``, and ``PyThread_ReInitTLS()``
is no-op [8]_.
functions were needed only for CPython's now defunct built-in TLS
implementation; that is the existing behavior of these functions is treated
as follows: ``PyThread_delete_key_value(key)`` is equalivalent to
``PyThread_set_key_value(key, NULL)``, and ``PyThread_ReInitTLS()`` is a
no-op [8]_.
The new ``PyThread_tss_`` functions are almost exactly analogous to their
original counterparts with a few minor differences: Whereas
@ -116,17 +117,17 @@ any size. This is especially necessary for extension modules built with
value pointed to by the ``key`` argument is ``NULL``.
Moreover, because of the use of ``Py_tss_t`` instead of ``int``, there are
additional behaviors which the existing API design would be carried over into
new API: The TSS key creation and deletion are parts of "do-if-needed" flow
and these features are silently skipped if already done--Calling
``PyThread_tss_create`` with an initialized key does nothing and returns
success soon. This is also the case of calling ``PyThread_tss_delete`` with
an uninitialized key.
behaviors in the new API which differ from the existing API with regard to
key creation and deletion. ``PyThread_tss_create`` can be called repeatedly
on the same key--calling it on an already initialized key is a no-op and
immediately returns success. Similarly for calling ``PyThread_tss_delete``
with an uninitialized key.
The behavior of ``PyThread_tss_delete`` is defined to change the key's
initialization state to "uninitialized" in order to restart the CPython
interpreter without terminating the process (e.g. embedding Python in an
application) [12]_.
initialization state to "uninitialized"--this allows, for example,
statically allocated keys to be reset to a sensible state when restarting
the CPython interpreter without terminating the process (e.g. embedding
Python in an application) [12]_.
The old ``PyThread_*_key*`` functions will be marked as deprecated in the
documentation, but will not generate runtime deprecation warnings.
@ -150,20 +151,20 @@ Features - create key - create key
- set value - set value
- get value - get value
- delete value - (set ``NULL`` instead) [8]_
- reinitialize keys (for - (unnecessary) [8]_
after fork)
- reinitialize keys (after - (unnecessary) [8]_
fork)
- dynamically (de-)allocate
key
- check key's initialization
state
Default Value (``-1`` as key creation ``Py_tss_NEEDS_INIT``
failure)
Requirement native thread native thread
Requirement native threads native threads
(since CPython 3.7 [9]_)
Restriction Not support platform where Unable to statically allocate
native TLS key is defined in key when ``Py_LIMITED_API``
a way that cannot be safely is defined.
cast to ``int``.
Restriction No support for platforms Unable to statically allocate
where native TLS key is keys when ``Py_LIMITED_API``
defined in a way that cannot is defined.
be safely cast to ``int``.
================= ============================= =============================
Example
@ -209,10 +210,10 @@ Platform Support Changes
A new "Native Thread Implementation" section will be added to PEP 11 that
states:
* As of CPython 3.7, in the case of enabling thread support, all platforms are
required to provide at least one of native thread implementation (as of
pthreads or Windows) to implement TSS API. Any TSS API problems that occur
in the implementation without native thread will be closed as "won't fix".
* As of CPython 3.7, all platforms are required to provide a native thread
implementation (such as pthreads or Windows) to implement the TSS
API. Any TSS API problems that occur in an implementation without native
threads will be closed as "won't fix".
Motivation
@ -223,11 +224,11 @@ TLS values, as defined by the original PyThread TLS API.
The original TLS API was added to Python by GvR back in 1997, and at the
time the key used to represent a TLS value was an ``int``, and so it has
been to the time of writing. This used CPython's own TLS implementation,
but the current generation of which hasn't been used, largely unchanged, in
Python/thread.c. Support for implementation of the API on top of native
thread implementations (pthreads and Windows) was added much later, and the
own implementation has been no longer necessary and removed [9]_.
been to the time of writing. This used CPython's own TLS implementation
which long remained unused, largely unchanged, in Python/thread.c. Support
for implementation of the API on top of native thread implementations
(pthreads and Windows) was added much later, and the built-in implementation
has been deemed no longer necessary and has since been removed [9]_.
The problem with the choice of ``int`` to represent a TLS key, is that while
it was fine for CPython's own TLS implementation, and happens to be
@ -265,16 +266,17 @@ Rationale for Proposed Solution
===============================
The use of an opaque type (``Py_tss_t``) to key TLS values allows the API to
be compatible, with all present (POSIX and Windows) and future (C11?)
native TLS implementations supported by CPython, as it allows the definition
of ``Py_tss_t`` to depend on the underlying implementation.
be compatible with all present (POSIX and Windows) and future (C11?) native
TLS implementations supported by CPython, as it allows the definition of
``Py_tss_t`` to depend on the underlying implementation.
Since the existing TLS API has been available in *the limited API* [13]_ for
some platforms (e.g. Linux), CPython makes an effort to provide the new TSS API
at that level likewise. Note, however, that ``Py_tss_t`` definition becomes to
be an opaque struct when ``Py_LIMITED_API`` is defined, because exposing
``NATIVE_TSS_KEY_T`` as part of the limited API would prevent us from switching
native thread implementation without rebuilding extension module.
some platforms (e.g. Linux), CPython makes an effort to provide the new TSS
API at that level likewise. Note, however, that the ``Py_tss_t`` definition
becomes to be an opaque struct when ``Py_LIMITED_API`` is defined, because
exposing ``NATIVE_TSS_KEY_T`` as part of the limited API would prevent us
from switching native thread implementation without rebuilding extension
modules.
A new API must be introduced, rather than changing the function signatures of
the current API, in order to maintain backwards compatibility. The new API
@ -316,29 +318,29 @@ Rejected Ideas
wishing to be supported by CPython should follow the requirements of
PEP-11. As explained above, while this would be a fair argument if
CPython were being to asked to make changes to support particular quirks
or features of a specific platform, in this case it is quirk of CPython
or features of a specific platform, in this case it is a quirk of CPython
that prevents it from being used to its full potential on otherwise
POSIX-compliant platforms. The fact that the current implementation
happens to work on Linux is a happy accident, and there's no guarantee
that this will never change.
* Affected platforms should just configure Python ``--without-threads``:
This is a possible temporary workaround to the issue, but only that.
Python should not be hobbled on affected platforms despite them being
otherwise perfectly capable of running multi-threaded Python.
this is no longer an option as the ``--without-threads`` option has
been removed for Python 3.7 [16]_.
* Affected platforms should use CPython's own TLS implementation instead of
native TLS implementation: This is a more acceptable alternative to the
previous idea, and in fact there had been a patch to do just that [4]_.
However, the own implementation being "slower and clunkier" in general
than native implementations still needlessly hobbles performance on affected
platforms. At least one other module (``tracemalloc``) is also broken if
Python is built without native implementation. And this idea cannot be
adopted because the own implementation was removed.
* Affected platforms should use CPython's built-in TLS implementation
instead of a native TLS implementation: This is a more acceptable
alternative to the previous idea, and in fact there had been a patch to do
just that [4]_. However, the built-in implementation being "slower and
clunkier" in general than native implementations still needlessly hobbles
performance on affected platforms. At least one other module
(``tracemalloc``) is also broken if Python is built without a native TLS
implementation. This idea also cannot be adopted because the built-in
implementation has since been removed.
* Keep the existing API, but work around the issue by providing a mapping from
``pthread_key_t`` values to ``int`` values. A couple attempts were made at
this ([5]_, [6]_), but this only injects needless complexity and overhead
this ([5]_, [6]_), but this injects needless complexity and overhead
into performance-critical code on platforms that are not currently affected
by this issue (such as Linux). Even if use of this workaround were made
conditional on platform compatibility, it introduces platform-specific code
@ -350,13 +352,13 @@ Implementation
==============
An initial version of a patch [7]_ is available on the bug tracker for this
issue. Since the migration to Github, it's being developed in the
issue. Since the migration to GitHub, its development has continued in the
``pep539-tss-api`` feature branch [10]_ in Masayuki Yamamoto's fork of the
CPython repository on Github. A work-in-progress PR is available at [11]_.
CPython repository on GitHub. A work-in-progress PR is available at [11]_.
This reference implementation covers not only the enhancement request in API
features, but also the client codes fix needed to replace the existing TLS API
with the new TSS API.
This reference implementation covers not only the new API implementation
features, but also the client code updates needed to replace the existing
TLS API with the new TSS API.
Copyright
@ -384,3 +386,4 @@ References and Footnotes
(https://www.python.org/dev/peps/pep-0384/)
.. [14] http://pubs.opengroup.org/onlinepubs/009695399/functions/pthread_key_create.html
.. [15] http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf#page=404
.. [16] https://bugs.python.org/issue31370