From f05753e8a3b37b9f8c7e7f451e678c6762740cdd Mon Sep 17 00:00:00 2001 From: Victor Stinner Date: Sun, 15 Apr 2012 17:06:07 +0200 Subject: [PATCH] PEP 418 * time.perf_counter() is not necessary monotonic; format code * Replace system clock by system time to reuse the expression of the glossary * If CLOCK_HIGHRES is available, time.monotonic() only uses it and does not fallback to CLOCK_MONOTONIC because CLOCK_MONOTONIC may not be available at the same time * Rewrite time.perf_counter(), time.process_time() and time.time() doc * Reorder "fallback to system time" alternatives --- pep-0418.txt | 312 ++++++++++++++++++++++++++------------------------- 1 file changed, 160 insertions(+), 152 deletions(-) diff --git a/pep-0418.txt b/pep-0418.txt index 957550435..f366c07c8 100644 --- a/pep-0418.txt +++ b/pep-0418.txt @@ -21,19 +21,20 @@ This PEP proposes to add ``time.get_clock_info(name)``, Rationale ========= -If a program uses the system clock to schedule events or to implement +If a program uses the system time to schedule events or to implement a timeout, it will not run events at the right moment or stop the -timeout too early or too late when the system clock is set manually or +timeout too early or too late when the system time is set manually or adjusted automatically by NTP. A monotonic clock should be used -instead to not be affected by system clock updates. +instead to not be affected by system time updates: +``time.monotonic()``. To measure the performance of a function, ``time.clock()`` can be used but it is very different on Windows and on Unix. On Windows, ``time.clock()`` includes time elapsed during sleep, whereas it does not on Unix. ``time.clock()`` precision is very good on Windows, but -very bad on Unix. The new ``time.perf_counter()`` function can be +very bad on Unix. The new ``time.perf_counter()`` function should be used instead to always get the most precise performance counter with a -portable behaviour. +portable behaviour (ex: include time spend during sleep). To measure CPU time, Python does not provide directly a portable function. ``time.clock()`` can be used on Unix, but it has a bad @@ -60,18 +61,18 @@ New functions: Users of new functions: - * time.monotonic(): concurrent.futures, multiprocessing, queue, subprocess, - telnet and threading modules to implement timeout - * time.perf_counter(): trace and timeit modules, pybench program - * time.process_time(): profile module - * time.get_clock_info(): pybench program to display information about the - timer like the precision or the resolution +* time.monotonic(): concurrent.futures, multiprocessing, queue, subprocess, + telnet and threading modules to implement timeout +* time.perf_counter(): trace and timeit modules, pybench program +* time.process_time(): profile module +* time.get_clock_info(): pybench program to display information about the + timer like the precision or the resolution -The ``time.clock()`` function is deprecated by this PEP because it is not +The ``time.clock()`` function is deprecated because it is not portable: it behaves differently depending on the operating system. -``time.perf_counter()`` or ``time.process_time()`` should be used instead, -depending on your requirements. ``time.clock()`` is marked as deprecated but -is not planned for removal. +``time.perf_counter()`` or ``time.process_time()`` should be used +instead, depending on your requirements. ``time.clock()`` is marked as +deprecated but is not planned for removal. Python functions @@ -83,47 +84,51 @@ New Functions time.get_clock_info(name) ^^^^^^^^^^^^^^^^^^^^^^^^^ -Get information on the specified clock. Supported clocks: +Get information on the specified clock. Supported clock names: -* "clock": time.clock() -* "monotonic": time.monotonic() -* "perf_counter": time.perf_counter() -* "process_time": time.process_time() -* "time": time.time() +* ``"clock"``: ``time.clock()`` +* ``"monotonic"``: ``time.monotonic()`` +* ``"perf_counter"``: ``time.perf_counter()`` +* ``"process_time"``: ``time.process_time()`` +* ``"time"``: ``time.time()`` Return a dictionary with the following keys: * Mandatory keys: - * "implementation" (str): name of the underlying operating system - function. Examples: "QueryPerformanceCounter()", - "clock_gettime(CLOCK_REALTIME)". - * "resolution" (float): resolution in seconds of the clock - * "is_monotonic" (bool): True if the clock cannot go backward + * ``"implementation"`` (str): name of the underlying operating system + function. Examples: ``"QueryPerformanceCounter()"``, + ``"clock_gettime(CLOCK_REALTIME)"``. + * ``"resolution"`` (float): resolution in seconds of the clock. + * ``"is_monotonic"`` (bool): True if the clock cannot go backward. * Optional keys: - * "precision" (float): precision in seconds of the clock - * "is_adjusted" (bool): True if the clock can be adjusted (e.g. by a - NTP daemon) + * ``"precision"`` (float): precision in seconds of the clock + reported by the operating system. + * ``"is_adjusted"`` (bool): True if the clock is adjusted (e.g. by a + NTP daemon). time.monotonic() ^^^^^^^^^^^^^^^^ -Monotonic clock, cannot go backward. It is not affected by system clock -updates. The reference point of the returned value is undefined, so that only -the difference between the results of consecutive calls is valid. +Monotonic clock, i.e. cannot go backward. It is not affected by system +clock updates. The reference point of the returned value is +undefined, so that only the difference between the results of +consecutive calls is valid and is a number of seconds. -On Windows older than Vista, ``time.monotonic()`` detects ``GetTickCount()`` -integer overflow (32 bits, roll-over after 49.7 days): it increases a delta by -2\ :sup:`32` each time than an overflow is detected. The delta is stored in -the process-local state and so the value of ``time.monotonic()`` may be -different in two Python processes running for more than 49 days. On more recent -versions of Windows and on other operating systems, ``time.monotonic()`` is +On Windows versions older than Vista, ``time.monotonic()`` detects +``GetTickCount()`` integer overflow (32 bits, roll-over after 49.7 +days): it increases a delta by 2\ :sup:`32` each time than an overflow +is detected. The delta is stored in the process-local state and so +the value of ``time.monotonic()`` may be different in two Python +processes running for more than 49 days. On more recent versions of +Windows and on other operating systems, ``time.monotonic()`` is system-wide. -Availability: Windows, Mac OS X, Unix, Solaris. Not available on GNU/Hurd. +Availability: Windows, Mac OS X, Unix, Solaris. Not available on +GNU/Hurd. Pseudo-code [#pseudo]_:: @@ -151,16 +156,13 @@ Pseudo-code [#pseudo]_:: return _time.mach_absolute_time() * monotonic.factor monotonic.factor = None - elif hasattr(time, "clock_gettime"): + elif hasattr(time, "clock_gettime") and hasattr(time, "CLOCK_HIGHRES"): + def monotonic(): + return time.clock_gettime(time.CLOCK_HIGHRES) + + elif hasattr(time, "clock_gettime") and hasattr(time, "CLOCK_MONOTONIC"): def monotonic(): - if monotonic.use_clock_highres: - try: - time.clock_gettime(time.CLOCK_HIGHRES) - except OSError: - monotonic.use_clock_highres = False return time.clock_gettime(time.CLOCK_MONOTONIC) - monotonic.use_clock_highres = (hasattr(time, 'clock_gettime') - and hasattr(time, 'CLOCK_HIGHRES')) On Windows, ``QueryPerformanceCounter()`` is not used even though it @@ -171,29 +173,30 @@ and has too many issues. time.perf_counter() ^^^^^^^^^^^^^^^^^^^ -Performance counter used for benchmarking. It is monotonic, i.e. cannot go -backward. It does include time elapsed during sleep and is system-wide. The -reference point of the returned value is undefined, so that only the difference -between the results of consecutive calls is valid and is number of seconds. +Performance counter with the highest available precision to measure a +duration. It does include time elapsed during sleep and is +system-wide. The reference point of the returned value is undefined, +so that only the difference between the results of consecutive calls +is valid and is a number of seconds. Pseudo-code:: def perf_counter(): if perf_counter.use_performance_counter: - if perf_counter.perf_frequency is None: + if perf_counter.performance_frequency is None: try: - perf_counter.perf_frequency = float(_time.QueryPerformanceFrequency()) + perf_counter.performance_frequency = _time.QueryPerformanceFrequency() except OSError: # QueryPerformanceFrequency() fails if the installed # hardware does not support a high-resolution performance # counter perf_counter.use_performance_counter = False else: - return _time.QueryPerformanceCounter() / perf_counter.perf_frequency + return _time.QueryPerformanceCounter() / perf_counter.performance_frequency else: - return _time.QueryPerformanceCounter() / perf_counter.perf_frequency + return _time.QueryPerformanceCounter() / perf_counter.performance_frequency if perf_counter.use_monotonic: - # Monotonic clock is preferred over system clock + # The monotonic clock is preferred over the system time try: return time.monotonic() except OSError: @@ -201,17 +204,20 @@ Pseudo-code:: return time.time() perf_counter.use_performance_counter = (os.name == 'nt') if perf_counter.use_performance_counter: - perf_counter.perf_frequency = None + perf_counter.performance_frequency = None perf_counter.use_monotonic = hasattr(time, 'monotonic') time.process_time() ^^^^^^^^^^^^^^^^^^^ -Process time used for profiling: sum of the kernel and user-space CPU time. It -does not include time elapsed during sleep. It is process-wide by definition. -The reference point of the returned value is undefined, so that only the -difference between the results of consecutive calls is valid. +Sum of the system and user CPU time of the current process. It does +not include time elapsed during sleep. It is process-wide by +definition. The reference point of the returned value is undefined, +so that only the difference between the results of consecutive calls +is valid. + +It is available on all platforms. Pseudo-code [#pseudo]_:: @@ -262,9 +268,9 @@ Existing Functions time.time() ^^^^^^^^^^^ -The system time is the "wall clock". It can be set manually by the -system administrator or automatically by a NTP daemon. It can jump -backward and forward. It is system-wide but not monotonic. +The system time which is usually the civil time. It is system-wide by +definition. It can be set manually by the system administrator or +automatically by a NTP daemon. It is available on all platforms and cannot fail. @@ -347,10 +353,11 @@ time.clock() ^^^^^^^^^^^^ On Unix, return the current processor time as a floating point number -expressed in seconds. It is process-wide by definition. The precision, and in -fact the very definition of the meaning of "processor time", depends on that of -the C function of the same name, but in any case, this is the function to use -for benchmarking Python or timing algorithms. +expressed in seconds. It is process-wide by definition. The precision, +and in fact the very definition of the meaning of "processor time", +depends on that of the C function of the same name, but in any case, +this is the function to use for benchmarking Python or timing +algorithms. On Windows, this function returns wall-clock seconds elapsed since the first call to this function, as a floating point number, based on the @@ -385,35 +392,33 @@ Pseudo-code [#pseudo]_:: Alternatives: API design ======================== -Other names for new functions ------------------------------ - -time.monotonic(): +Other names for time.monotonic() +-------------------------------- * time.counter() * time.metronomic() * time.seconds() -* time.steady() +* time.steady(): "steady" is ambiguous: it means different things to + different people. For example, on Linux, CLOCK_MONOTONIC is + adjusted. If we uses the real time as the reference clock, we may + say that CLOCK_MONOTONIC is steady. But CLOCK_MONOTONIC gets + suspended on system suspend, whereas real time includes any time + spent in suspend. * time.timeout_clock() -* time.wallclock(): it is not the system time aka the "wall clock", - but a monotonic clock with an unspecified starting point +* time.wallclock(): time.monotonic() is not the system time aka the + "wall clock", but a monotonic clock with an unspecified starting + point. -"steady" is ambiguous: it means different things to different people. For -example, on Linux, CLOCK_MONOTONIC is adjusted. If we uses the real time as the -reference clock, we may say that CLOCK_MONOTONIC is steady. But -CLOCK_MONOTONIC gets suspended on system suspend, whereas real time includes -any time spent in suspend. +The name "time.try_monotonic()" was also proposed for an older +proposition of time.monotonic() which was falling back to the system +time when no monotonic clock was available. -The name "time.try_monotonic()" was also proposed when -time.monotonic() was falling back to the system clock when no -monotonic clock was available. - -time.perf_counter(): - - * time.hires() - * time.highres() - * time.timer() +Other names for time.perf_counter() +----------------------------------- +* time.hires() +* time.highres() +* time.timer() Only expose operating system clocks ----------------------------------- @@ -424,20 +429,48 @@ time.clock_gettime() and related clock identifiers were already added to Python 3.3 for example. -Fallback to system clock ------------------------- +time.monotonic(): Fallback to system time +----------------------------------------- If no monotonic clock is available, time.monotonic() falls back to the -system clock. +system time. Issues: * It is hard to define correctly such function in the documentation: - is it monotonic? is it steady? is it adjusted? + is it monotonic? Is it steady? Is it adjusted? * Some user want to decide what to do when no monotonic clock is available: use another clock, display an error, or do something else? +Different APIs were proposed to define such function. + +One function with a flag: time.monotonic(fallback=True) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +* time.monotonic(fallback=True) falls back to the system time if no + monotonic clock is available or if the monotonic clock failed. +* time.monotonic(fallback=False) raises OSError if monotonic clock + fails and NotImplementedError if the system does not provide a + monotonic clock + +A keyword argument that gets passed as a constant in the caller is +usually poor API. + +Raising NotImplementedError for a function is something uncommon in +Python and should be avoided. + + +One time.monotonic() function, no flag +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +time.monotonic() returns (time: float, is_monotonic: bool). + +An alternative is to use a function attribute: +time.monotonic.is_monotonic. The attribute value would be None before +the first call to time.monotonic(). + + Choosing the clock from a list of constraints --------------------------------------------- @@ -503,32 +536,6 @@ on their metadata. Example partial implementation: `clockutils.py `_. -One function with a flag: time.monotonic(fallback=True) -------------------------------------------------------- - -* time.monotonic(fallback=True) falls back to the system clock if no - monotonic clock is available or if the monotonic clock failed. -* time.monotonic(fallback=False) raises OSError if monotonic clock - fails and NotImplementedError if the system does not provide a - monotonic clock - -"A keyword argument that gets passed as a constant in the caller is -usually poor API." - -Raising NotImplementedError for a function is something uncommon in -Python and should be avoided. - - -One function, no flag ---------------------- - -time.monotonic() returns (time: float, is_monotonic: bool). - -An alternative is to use a function attribute: -time.monotonic.is_monotonic. The attribute value would be None before -the first call to time.monotonic(). - - Working around operating system bugs? ------------------------------------- @@ -1127,8 +1134,8 @@ Read the `gethrtime() manual page of Solaris 11 On Solaris, gethrtime() is the same as clock_gettime(CLOCK_MONOTONIC). -System clocks -------------- +System Time +----------- ========================= =============== ============= =============== Name Resolution Include sleep Include suspend @@ -1159,7 +1166,7 @@ Windows: GetSystemTimeAsFileTime ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The system time can be read using GetSystemTimeAsFileTime(), ftime() and -time(). The precision of the system clock can be read using +time(). The precision of the system time can be read using GetSystemTimeAdjustment(). Read the `GetSystemTimeAsFileTime() documentation @@ -1171,7 +1178,7 @@ System time on UNIX ^^^^^^^^^^^^^^^^^^^ gettimeofday(), ftime(), time() and clock_gettime(CLOCK_REALTIME) return -the system clock. The precision of CLOCK_REALTIME can be read using +the system time. The precision of CLOCK_REALTIME can be read using clock_getres(). The system time can be set using settimeofday() or @@ -1224,8 +1231,9 @@ clock() SunOS 5.11 1 µs 10 ms Functions ^^^^^^^^^ -* Windows: GetProcessTimes(). The precision can be read using - GetSystemTimeAdjustment(). +* Windows: `GetProcessTimes() + `_. + The precision can be read using GetSystemTimeAdjustment(). * clock_gettime(CLOCK_PROCESS_CPUTIME_ID): High-resolution per-process timer from the CPU. The precision can be read using clock_getres(). * clock(). The precision is 1 / CLOCKS_PER_SEC. @@ -1280,9 +1288,9 @@ seconds. Functions ^^^^^^^^^ -* Windows: GetThreadTimes(). The precision can be read using - GetSystemTimeAdjustment(). `MSDN documentation of GetThreadTimes() +* Windows: `GetThreadTimes() `_. + The precision can be read using GetSystemTimeAdjustment(). * clock_gettime(CLOCK_THREAD_CPUTIME_ID): Thread-specific CPU-time clock. The precision can be read using of clock_getres(). @@ -1312,11 +1320,11 @@ See also `QueryIdleProcessorCycleTime() function Sleep ----- -Suspend execution of the process for the given number of seconds. Sleep is not -affected by system clock update. Sleep is paused during system suspend. For -example, if a process sleeps for 60 seconds and the system is suspended for 30 -seconds in the middle of the sleep, the sleep duration is 90 seconds in the -real time. +Suspend execution of the process for the given number of seconds. +Sleep is not affected by system time updates. Sleep is paused during +system suspend. For example, if a process sleeps for 60 seconds and +the system is suspended for 30 seconds in the middle of the sleep, the +sleep duration is 90 seconds in the real time. Sleep can be interrupted by a signal: the function fails with EINTR. @@ -1348,12 +1356,12 @@ WaitForSingleObject() 1 ms Functions ^^^^^^^^^ - * sleep(seconds) - * usleep(microseconds) - * nanosleep(nanoseconds, remaining): - `Linux manpage of nanosleep() - `_ - * delay(milliseconds) +* sleep(seconds) +* usleep(microseconds) +* nanosleep(nanoseconds, remaining): + `Linux manpage of nanosleep() + `_ +* delay(milliseconds) clock_nanosleep @@ -1390,19 +1398,19 @@ high-) resolution timeouts `_ and Other functions ^^^^^^^^^^^^^^^ - * poll(), epoll() - * sigtimedwait(). POSIX: "If the Monotonic Clock option is supported, - the CLOCK_MONOTONIC clock shall be used to measure the time - interval specified by the timeout argument." - * pthread_cond_timedwait(), pthread_condattr_setclock(). "The default - value of the clock attribute shall refer to the system clock." - * sem_timedwait(): "If the Timers option is supported, the timeout - shall be based on the CLOCK_REALTIME clock. If the Timers option is - not supported, the timeout shall be based on the system clock as - returned by the time() function. The precision of the timeout - shall be the precision of the clock on which it is based." - * WaitForSingleObject(): use the same timer than GetTickCount() with - the same precision. +* poll(), epoll() +* sigtimedwait(). POSIX: "If the Monotonic Clock option is supported, + the CLOCK_MONOTONIC clock shall be used to measure the time + interval specified by the timeout argument." +* pthread_cond_timedwait(), pthread_condattr_setclock(). "The default + value of the clock attribute shall refer to the system time." +* sem_timedwait(): "If the Timers option is supported, the timeout + shall be based on the CLOCK_REALTIME clock. If the Timers option is + not supported, the timeout shall be based on the system time as + returned by the time() function. The precision of the timeout + shall be the precision of the clock on which it is based." +* WaitForSingleObject(): use the same timer than GetTickCount() with + the same precision. System Standby @@ -1506,7 +1514,7 @@ Time: for Windows `_ * `clockspeed `_ uses a hardware tick - counter to compensate for a persistently fast or slow system clock + counter to compensate for a persistently fast or slow system time * `Retrieving system time `_ lists hardware clocks and time functions with their resolution and