From 351a920ce34c9753989b9872539d2d20a7ce9c09 Mon Sep 17 00:00:00 2001 From: David Goodger Date: Fri, 6 Jun 2008 02:20:47 +0000 Subject: [PATCH] update from PEP authors --- pep-0371.txt | 92 +++++++++++++++++++++++++++++++++------------------- 1 file changed, 59 insertions(+), 33 deletions(-) diff --git a/pep-0371.txt b/pep-0371.txt index 03159785b..4ed890c42 100644 --- a/pep-0371.txt +++ b/pep-0371.txt @@ -18,9 +18,9 @@ Abstract into the Python standard library, renamed to "multiprocessing". The processing package mimics the standard library threading - module and API to provide a process-based approach to "threaded - programming" allowing end-users to dispatch multiple tasks that - effectively side-step the global interpreter lock. + module functionality to provide a process-based approach to + threaded programming allowing end-users to dispatch multiple + tasks that effectively side-step the global interpreter lock. The package also provides server and client functionality (processing.Manager) to provide remote sharing and management of @@ -45,23 +45,23 @@ Rationale The GIL itself prevents more than a single thread from running within the interpreter at any given point in time, effectively removing Python's ability to take advantage of multi-processor - systems. While I/O bound applications do not suffer the same - slow-down when using threading, they do suffer some performance - cost due to the GIL. + systems. - The pyProcessing package offers a method to side-step the GIL + The pyprocessing package offers a method to side-step the GIL allowing applications within CPython to take advantage of multi-core architectures without asking users to completely change their programming paradigm (i.e.: dropping threaded programming - for another "concurrent" approach - Twisted, etc). + for another "concurrent" approach - Twisted, Actors, etc). + + The Processing package offers CPython a "known API" which mirrors + albeit in a PEP 8 compliant manner, that of the threading API, + with known semantics and easy scalability. - The Processing package offers CPython users a known API (that of - the threading module), with known semantics and easy-scalability. In the future, the package might not be as relevant should the CPython interpreter enable "true" threading, however for some applications, forking an OS process may sometimes be more desirable than using lightweight threads, especially on those - platforms where process creation is fast/optimized. + platforms where process creation is fast and optimized. For example, a simple threaded application: @@ -74,13 +74,17 @@ Rationale t.start() t.join() - The pyprocessing package mirrors the API so well, that with a + The pyprocessing package mirrored the API so well, that with a simple change of the import to: - from processing import Process as worker + from processing import process as worker - The code now executes through the processing.Process class. This - type of compatibility means that, with a minor (in most cases) + The code would now execute through the processing.process class. + Obviously, with the renaming of the API to PEP 8 compliance there + would be additional renaming which would need to occur within + user applications, however minor. + + This type of compatibility means that, with a minor (in most cases) change in code, users' applications will be able to leverage all cores and processors on a given machine for parallel execution. In many cases the pyprocessing package is even faster than the @@ -318,15 +322,25 @@ Maintenance API Naming - While the aim of the package's API is designed to closely mimic that of - the threading and Queue modules, those modules are not PEP 8 compliant. - It has been decided that instead of adding the package as-is and - therefore perpetuating the non-PEP 8 compliant naming, we will rename - all APIs, classes, etc to be fully PEP 8 compliant. + While the aim of the package's API is designed to closely mimic that of + the threading and Queue modules as of python 2.x, those modules are not + PEP 8 compliant. It has been decided that instead of adding the package + "as is" and therefore perpetuating the non-PEP 8 compliant naming, we + will rename all APIs, classes, etc to be fully PEP 8 compliant. This change does affect the ease-of-drop in replacement for those using the threading module, but that is an acceptable side-effect in the view - of the authors. + of the authors, especially given that the threading module's own API + will change. + + Issue 3042 in the tracker proposes that for Python 2.6 there will be + two APIs for the threading module - the current one, and the PEP 8 + compliant one. Warnings about the upcoming removal of the original + java-style API will be issues when -3 is invoked. + + In Python 3000, the threading API will become PEP 8 compliant, which + means that the multiprocessing module and the threading module will + again have matching APIs. Timing/Schedule @@ -345,28 +359,24 @@ Open Issues * All existing tests for the package should be converted to UnitTest format. + * Existing documentation has to be moved to ReST formatting. + * Verify code coverage percentage of existing test suite. - * Identify any requirements to achieve a 1.0 milestone if - required. + * Verify current source tree conforms to standard library practices. - * Rename top-level package from "pyprocessing" to - "multiprocessing". + * Confirm no "default" remote connection capabilities, if needed enable the remote security mechanisms by default for those classes which offer remote capabilities. + * Some of the API (Queue methods qsize(), task_done() and join()) either need to be added, or the reason for their exclusion needs to be identified and documented clearly. - * Add in "multiprocessing.setExecutable()" method to override the - default behavior of the package to spawn processes using the - current executable name rather than the Python interpreter. Note - that Mark Hammond has suggested a factory-style interface for - this[7]. - * Also note that the default behavior of process spawning does - not make it compatible with use within IDLE as-is, this will - be examined as a bug-fix or "setExecutable" enhancement. + + * The PyGILState bug patch submitted in issue 1683 by roudkerk + must be applied for the package unit tests to work. Closed Issues @@ -375,6 +385,19 @@ Closed Issues ctypes is not supported. This is not a restriction of this package, but rather of ctypes. + * DONE: Rename top-level package from "pyprocessing" to + "multiprocessing". + + * DONE: Also note that the default behavior of process spawning + does not make it compatible with use within IDLE as-is, this + will be examined as a bug-fix or "setExecutable" enhancement. + + * DONE: Add in "multiprocessing.setExecutable()" method to override the + default behavior of the package to spawn processes using the + current executable name rather than the Python interpreter. Note + that Mark Hammond has suggested a factory-style interface for + this[7]. + References [1] PyProcessing home page @@ -398,6 +421,9 @@ References [7] http://groups.google.com/group/python-dev2/msg/54cf06d15cbcbc34 + [8] Addition Python-Dev discussion + http://mail.python.org/pipermail/python-dev/2008-June/080011.html + Copyright This document has been placed in the public domain.