python-peps/pep-0418.txt

PEP: 418
Title: Add monotonic and high-resolution time functions
Version: $Revision$
Last-Modified: $Date$
Author: Victor Stinner <victor.stinner@gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 26-March-2012
Python-Version: 3.3


Abstract
========

Add time.monotonic() and time.hires() functions to Python 3.3.


Rationale
=========

Use cases:

 * Display the current time to a human (e.g. display a calendar or draw a wall
   clock): use system clock. time.time() or datetime.datetime.now()
 * Benchmark, profiling, timeout: time.hires()
 * Event scheduler: time.monotonic()


Functions
=========

time.time()
-----------

The system time is the "wall clock". It can be set manually by the system
administrator or automatically by a NTP daemon.  It can jump backward and
forward. It is not monotonic.

It is available on all platforms and cannot fail.

Pseudo-code [#pseudo]_: ::

    if os.name == "nt":
        def time():
            return _time.GetSystemTimeAsFileTime()
    else:
        def time():
            if hasattr(time, "clock_gettime"):
                try:
                    # resolution = 1 nanosecond
                    return time.clock_gettime(time.CLOCK_REALTIME)
                except OSError:
                    # CLOCK_REALTIME is not supported (unlikely)
                    pass
            if hasattr(_time, "gettimeofday"):
                try:
                    # resolution = 1 microsecond
                    return _time.gettimeofday()
                except OSError:
                    # gettimeofday() should not fail
                    pass
            if hasattr(_time, "ftime"):
                # resolution = 1 millisecond
                return _time.ftime()
            else:
                # resolution = 1 second
                return _time.time()


time.monotonic()
----------------

Clock advancing at a monotonic rate relative to real time. It cannot go
backward. Its rate may be adjusted by NTP. The reference point of the returned
value is undefined so only the difference of consecutive calls is valid.

It is not available on all platforms and may raise an OSError.

The monotonic clock may stop while the system is suspended.

Pseudo-code [#pseudo]_: ::

    if os.name == 'nt':
        # GetTickCount64() requires Windows Vista, Server 2008 or later
        if hasattr(time, '_GetTickCount64'):
            _get_tick_count = _time.GetTickCount64
        else:
            def _get_tick_count():
                ticks = _time.GetTickCount()
                if ticks < _get_tick_count.last:
                    # Integer overflow detected
                    _get_tick_count.delta += 2**32
                _get_tick_count.last = ticks
                return ticks + _get_tick_count.delta
            _get_tick_count.last = 0
            _get_tick_count.delta = 0

        def monotonic():
            if monotonic.use_performance_counter:
                try:
                    return _time.QueryPerformanceCounter()
                except OSError:
                    # QueryPerformanceFrequency() may fail, if the installed
                    # hardware does not support a high-resolution performance
                    # counter for example
                    monotonic.use_performance_counter = False
            # Fallback to GetTickCount/GetTickCount64 which has
            # a lower resolution
            return _get_tick_count()
        monotonic.use_performance_counter = True

    elif os.name == 'mac':
        def monotonic():
            if monotonic.factor is None:
                factor = _time.mach_timebase_info()
                monotonic.factor = timebase[0] / timebase[1]
            return _time.mach_absolute_time() * monotonic.factor
        monotonic.factor = None

    elif hasattr(time, "clock_gettime"):
        def monotonic():
            if monotonic.use_monotonic_raw:
                try:
                    return time.clock_gettime(time.CLOCK_MONOTONIC_RAW)
                except OSError:
                    # CLOCK_MONOTONIC_RAW requires a Linux kernel >= 2.6.28
                    monotonic.use_monotonic_raw = False
            return time.clock_gettime(time.CLOCK_MONOTONIC)
        monotonic.use_monotonic_raw = hasattr(time, "CLOCK_MONOTONIC_RAW")

time.hires()
------------

High-resolution clock: use a monotonic clock if available, or fallback to the
system time.

It is available on all platforms and cannot fail.

Pseudo-code::

    def hires():
        if hires.use_monotonic:
            try:
                return time.monotonic()
            except OSError:
                hires.use_monotonic = False
        return time.time()
    hires.use_monotonic = hasattr(time, 'monotonic')


Clocks
======

Monotonic
---------

mach_absolute_time
^^^^^^^^^^^^^^^^^^

Mac OS X provides a monotonic clock: mach_absolute_time(). It is based on
absolute elapsed time delta since system boot. It is not adjusted and cannot be
set.

mach_timebase_info() gives a fraction to convert the clock value to a number of
nanoseconds. According to the documentation (`Technical Q&A QA1398
<https://developer.apple.com/library/mac/#qa/qa1398/>`_), mach_timebase_info()
is always equals to one and does never fail, even if the function may fail
according to its prototype.

mach_absolute_time() has a resolution of 1 nanosecond.

CLOCK_MONOTONIC, CLOCK_MONOTONIC_RAW
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

CLOCK_MONOTONIC and CLOCK_MONOTONIC_RAW represents monotonic time since some
unspecified starting point. They cannot be set.

CLOCK_MONOTONIC_RAW is specific to Linux. It is similar to CLOCK_MONOTONIC, but
provides access to a raw hardware-based time that is not subject to NTP
adjustments. CLOCK_MONOTONIC_RAW requires Linux 2.6.28 or later.

On Linux, NTP may adjust CLOCK_MONOTONIC rate, but not jump backward.  If
available, CLOCK_MONOTONIC_RAW should be used instead of CLOCK_MONOTONIC to
avoid the NTP adjustement.

CLOCK_MONOTONIC stops while the machine is suspended.

clock_gettime() fails if the system does not support the specified clock,
whereas the standard C library supports it. For example, CLOCK_MONOTONIC_RAW
requires a kernel version 2.6.28 or later.

clock_getres() gives the clock resolution. It is 1 nanosecond on Linux.

.. note::
   clock_gettime() requires to link the program to the rt (real-time) library.

QueryPerformanceCounter
^^^^^^^^^^^^^^^^^^^^^^^

High-resolution performance counter. It is monotonic.
QueryPerformanceFrequency() gives its frequency.

On Windows XP, QueryPerformanceFrequency() is the processor frequency and
QueryPerformanceCounter() is the TSC of the current processor. Windows XP
had a bug (see `KB896256 <http://support.microsoft.com/?id=896256>`_): on a
multiprocessor computer, QueryPerformanceCounter() returned a different value
for each processor.

QueryPerformanceFrequency() should only be called once:  the frequency will not
change while the system is running. It fails if the installed hardware does not
support a high-resolution performance counter.


GetTickCount(), GetTickCount64()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

GetTickCount() and GetTickCount64() are monotonic and cannot fail.

GetTickCount64() was added to Windows Vista and Windows Server 2008.

The clock resolution is 1 millisecond.


System time
-----------

Windows: GetSystemTimeAsFileTime
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The system time can be read using GetSystemTimeAsFileTime(), ftime() and
time().

The system time resolution can be read using GetSystemTimeAdjustment(). The
accurary is usually between 0.5 millisecond and 15 milliseconds. Resolution:

 * GetSystemTimeAsFileTime(): 100 nanoseconds
 * ftime(): 1 millisecond
 * time(): 1 second

The system time can be set using SetSystemTime().

System time on UNIX
^^^^^^^^^^^^^^^^^^^

gettimeofday(), ftime(), time() and clock_gettime(CLOCK_REALTIME) return the
system clock.

Resolution:

 * clock_gettime(): clock_getres(CLOCK_REALTIME), 1 nanosecond on Linux
 * gettimeofday(): 1 microsecond
 * ftime(): 1 millisecond
 * time(): 1 second

The system time can be set using settimeofday() or clock_settime(CLOCK_REALTIME).


Process and thread time
-----------------------

The process and thread time cannot be set. They are not monotonic: the clocks
stop while the process/thread is idle.

Process
^^^^^^^

 * Windows: GetProcessTimes()
 * clock_gettime(CLOCK_PROCESS_CPUTIME_ID): High-resolution per-process timer
   from the CPU.
 * clock():

   * Windows:  The elapsed wall-clock time since the start of the process
     (elapsed time in seconds times CLOCKS_PER_SEC). It can fail.
   * UNIX: returns an approximation of processor time used by the program.

 * times()
 * getrusage(): ru_utime and ru_stime fields

Resolution:

 * clock() rate is CLOCKS_PER_SEC. It was called CLK_TCK in Microsoft C before
   6.0. On Linux 3, clock() has a resolution of 1 microsecond
 * The clock resolution can be read using clock_getres().
   clock_getres(CLOCK_REALTIME) is 1 nanosecond on Linux
 * GetProcessTimes(): call GetSystemTimeAdjustment()

Thread
^^^^^^

 * Windows: GetThreadTimes()
 * clock_gettime(CLOCK_THREAD_CPUTIME_ID): Thread-specific CPU-time clock.

Resolution:

 * CLOCK_THREAD_CPUTIME_ID: call clock_getres(). 1 nanosecond on Linux.
 * GetThreadTimes(): call GetSystemTimeAdjustment()

See also pthread_getcpuclockid().


QueryUnbiasedInterruptTime
--------------------------

Gets the current unbiased interrupt time from the biased interrupt time and the
current sleep bias amount. This time is not affected by power management sleep
transitions.

Is it monotonic?

QueryUnbiasedInterruptTime() was introduced in Windows 7.



Alternatives: API design
========================

One function with a flag: time.monotonic(strict=False)
----------------------------------------------------------

 * time.monotonic(strict=False) falls back to the system clock if no monotonic
   clock is available or if the monotonic clock failed.
 * time.monotonic(strict=True) raises OSError if monotonic clock fails and
   NotImplementedError if the system does not provide a monotonic clock

"A keyword argument that gets passed as a constant in the caller is usually
poor API."

Raising NotImplementedError for a function is something uncommon in Python and
should be avoided.


One function, no flag
---------------------

time.monotonic() returns (time: float, is_monotonic: bool).

An alternative is to use a function attribute: time.monotonic.is_monotonic. The
attribute value would be None before the first call to time.monotonic().


Working around operating system bugs?
=====================================

Should Python ensure manually that a monotonic clock is truly monotonic by
computing the maximum with the clock value and the previous value?

 * Virtual machines provide less reliable clocks.
 * QueryPerformanceCounter() had a bug in 2006 on multiprocessor computers


Footnotes
=========

.. [#pseudo] _time is an hypothetical module used for the example. In practice,
             functions will be implemented in C and so don't need a module.


Links
=====

 * `Issue #12822: NewGIL should use CLOCK_MONOTONIC if possible.
   <http://bugs.python.org/issue12822>`_
 * `Issue #14222: Use time.steady() to implement timeout
   <http://bugs.python.org/issue14222>`_
 * `Issue #14397: Use GetTickCount/GetTickCount64 instead of QueryPerformanceCounter for monotonic clock
   <http://bugs.python.org/issue14397>`_
 * `python-monotonic-time
   <http://code.google.com/p/python-monotonic-time/>`_
   (`github <https://github.com/gavinbeatty/python-monotonic-time>`_)
 * `Qt library: QElapsedTimer
   <http://qt-project.org/doc/qt-4.8/qelapsedtimer.html>`_
 * `Windows: Game Timing and Multicore Processors
   <http://msdn.microsoft.com/en-us/library/ee417693.aspx>`_
 * `Implement a Continuously Updating, High-Resolution Time Provider for Windows
   <http://msdn.microsoft.com/en-us/magazine/cc163996.aspx>`_
 * `Perl: Time::HiRes
   <http://perldoc.perl.org/Time/HiRes.html>`_