python-peps/pep-0418.txt

PEP: 418
Title: Add monotonic clock
Version: $Revision$
Last-Modified: $Date$
Author: Victor Stinner <victor.stinner@gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 26-March-2012
Python-Version: 3.3


Abstract
========

Add time.monotonic() and time.steady() functions to Python 3.3.


Rationale
=========

Use cases:

 * Display the current time to a human: use system clock. time.time() or
   datetime.datetime.now()
 * Implement a timeout: use monotonic clock, or fallback to the clock with
   the highest resolution. time.steady()
 * Benchmark and profiling: time.steady()
 * Truly monotonic clock: ?

time.steady() tries to use a monotonic clock, but it falls back to a
non-monotonic clock if no monotonic clock is available or if reading a
monotonic clock failed.

The pybench benchmark tool supports the following clocks: time.clock(),
time.time() and systimes.processtime(). By default, it uses time.clock() on
Windows, and time.time() otherwise. It is possible to choose the clock using
the --timer command line option.

Which clock?

 * System time: pthread_mutex_timedlock(), pthread_cond_wait() (CLOCK_REALTIME)
 * Monotonic: clock_nanosleep() (CLOCK_MONOTONIC)

Timeouts:

 * threading.Lock.wait()
 * select.select()
 * time.sleep()
 * subprocess.Popen.communicate()
 * etc.


Clocks
======

Monotonic
---------

 * mach_absolute_time(): Mac OS X provides a monotonic clock:
   mach_absolute_time(). mach_timebase_info() provides a fraction to convert it
   to a number of nanoseconds. According to the documentation,
   mach_timebase_info() is always equals to one and does never fail, even if
   the function may fail according to its prototype.
 * clock_gettime(CLOCK_MONOTONIC): Clock that cannot be set and represents
   monotonic time since some unspecified starting point.
 * clock_gettime(CLOCK_MONOTONIC_RAW), since Linux 2.6.28; Linux-specific:
   Similar to CLOCK_MONOTONIC, but provides access to a raw hardware-based time
   that is not subject to NTP adjustments.
 * Windows: GetTickCount(), GetTickCount64().

CLOCK_MONOTONIC and CLOCK_MONOTONIC_RAW clocks cannot be set.

On Linux, NTP may adjust CLOCK_MONOTONIC rate, but not jump backward.  If
available, CLOCK_MONOTONIC_RAW should be used instead of CLOCK_MONOTONIC to
avoid the NTP adjustement. CLOCK_MONOTONIC stops while the machine is
suspended.

Resolution:

 * mach_absolute_time(): 1 nanosecond
 * CLOCK_MONOTONIC, CLOCK_MONOTONIC_RAW: be read using clock_getres().
   1 nanosecond on Linux for example.
 * GetTickCount(), GetTickCount64(): 1 millisecond

May fail?

 * mach_absolute_time() cannot fail. According to the documentation,
   mach_timebase_info() does never fail, even if the function has a return
   value.
 * clock_gettime() can fail if the system does not support the specified clock,
   whereas the standard C library supports it. For example, CLOCK_MONOTONIC_RAW
   requires a kernel version 2.6.28 or later.
 * GetTickCount() and GetTickCount64() cannot fail

Note: clock_gettime() requires to link the program with the realtime ("rt") library.

Note: GetTickCount64() was added to Windows Vista and Windows Server 2008.


System time
-----------

The system time can be set manually by the system administrator or
automatically by a NTP daemon.  It can jump backward and forward, and is not
monotonic.

System time on Windows
^^^^^^^^^^^^^^^^^^^^^^

The system time can be read using GetSystemTimeAsFileTime(), ftime() and
time().

The system time resolution can be read using GetSystemTimeAdjustment(). The
accurary is usually between 0.5 millisecond and 15 milliseconds. Resolution:

 * GetSystemTimeAsFileTime(): 100 nanoseconds
 * ftime(): 1 millisecond
 * time(): 1 second

The system time can be set using SetSystemTime().

System time on UNIX
^^^^^^^^^^^^^^^^^^^

gettimeofday(), ftime(), time() and clock_settime(CLOCK_REALTIME) return the
system clock.

Resolution:

 * clock_settime(): clock_getres(CLOCK_REALTIME), 1 nanosecond on Linux
 * gettimeofday(): 1 microsecond
 * ftime(): 1 millisecond
 * time(): 1 second

The system time can be set using settimeofday() or clock_settime(CLOCK_REALTIME).


Process and thread time
-----------------------

The process and thread time cannot be set. They are not monotonic: the clocks
stop while the process/thread is idle.

Process
^^^^^^^

 * Windows: GetProcessTimes()
 * clock_gettime(CLOCK_PROCESS_CPUTIME_ID): High-resolution per-process timer from the CPU.
 * clock():

   * Windows:  The elapsed wall-clock time since the start of the process
     (elapsed time in seconds times CLOCKS_PER_SEC). It can fail.
   * UNIX: returns an approximation of processor time used by the program.

 * times()
 * getrusage(): ru_utime and ru_stime fields

Resolution:

 * clock() rate is CLOCKS_PER_SEC. It was called CLK_TCK in Microsoft C before
   6.0. On Linux 3, clock() has a resolution of 1 microsecond
 * The clock resolution can be read using clock_getres().
   clock_getres(CLOCK_REALTIME) is 1 nanosecond on Linux
 * GetProcessTimes(): call GetSystemTimeAdjustment()

Thread
^^^^^^

 * Windows: GetThreadTimes()
 * clock_gettime(CLOCK_THREAD_CPUTIME_ID): Thread-specific CPU-time clock.

Resolution:

 * CLOCK_THREAD_CPUTIME_ID: call clock_getres(). 1 nanosecond on Linux.
 * GetThreadTimes(): call GetSystemTimeAdjustment()

See also pthread_getcpuclockid().


QueryPerformanceCounter
-----------------------

Windows provides a high-resolution performance counter:
QueryPerformanceCounter(). Its frequency can be retrieved using
QueryPerformanceFrequency().

On Windows XP, QueryPerformanceFrequency() is the processor frequency and
QueryPerformanceCounter() is the TSC of the current processor. Windows XP
had a bug (see `KB896256 <http://support.microsoft.com/?id=896256>`_): on a
multiprocessor computer, QueryPerformanceCounter() returned a different value
for each processor.

QueryPerformanceCounter() is not monotonic.

QueryPerformanceFrequency() fails if the installed hardware does not support a
high-resolution performance counter.

QueryPerformanceFrequency() should only be called once:  the frequency will not
change while the system is running.


QueryUnbiasedInterruptTime
--------------------------

Gets the current unbiased interrupt time from the biased interrupt time and the
current sleep bias amount. This time is not affected by power management sleep
transitions.

Is it monotonic?

QueryUnbiasedInterruptTime() was introduced in Windows 7.



API design
==========

Two functions: time.monotonic(), time.steady()
----------------------------------------------

 * time.steady() falls back to another clock if no monotonic clock is not
   available or does not work, but it does never fail.
 * time.monotonic() is not always available and may raise OSError.

One function with a flag: time.steady(strict=False)
---------------------------------------------------

 * time.steady(strict=False) falls back to another clock if no monotonic clock
   is not available or does not work, but it does never fail.
 * time.steady(strict=True) raises OSError if monotonic clock fails or
   NotImplementedError if the system does not provide a monotonic clock

"A keyword argument that gets passed as a constant in the caller is usually
poor API."

One function, no flag
---------------------

time.steady() returns (time: float, is_monotonic: bool).

An alternative is to use a function attribute: time.steady.monotonic. The
attribute value would be None before the first call to time.steady().


Workaround operating system bugs?
=================================

Should Python ensure manually that a monotonic clock is truly monotonic by
computing the maximum with the clock value and the previous value?

 * Virtual machines provide less reliable clocks.
 * QueryPerformanceCounter() had a bug in 2006 on multiprocessor computers


Links
=====

 * `Windows: Game Timing and Multicore Processors
   <http://msdn.microsoft.com/en-us/library/ee417693.aspx>`_