iSharkFly-Open
/
python-peps
mirror of https://github.com/python/peps.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
python-peps/pep-0418.txt

1418 lines
51 KiB
Plaintext
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

PEP: 418
Title: Add a monotonic time functions
Version: $Revision$
Last-Modified: $Date$
Author: Jim Jewett <jimjjewett@gmail.com>, Victor Stinner <victor.stinner@gmail.com>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 26-March-2012
Python-Version: 3.3
Abstract
========
Add time.monotonic() and time.get_clock_info(name) functions to Python 3.3.
Rationale
=========
If a program uses the system clock to schedule events or to implement a
timeout, it will not run events at the right moment or stop the timeout too
early or too late when the system clock is set manually or adjusted
automatically by NTP. A monotonic clock should be used instead to not be
affected by system clock updates.
Clocks:
* time.time(): system clock
* time.monotonic(): monotonic clock
Functions
=========
To fulfill the use cases, the functions' properties are:
* time.time(): system clock, "wall clock".
* time.monotonic(): monotonic clock
* time.get_clock_info(name): get information on the specified time function
time.time()
-----------
The system time is the "wall clock". It can be set manually by the
system administrator or automatically by a NTP daemon. It can jump
backward and forward. It is not monotonic.
It is available on all platforms and cannot fail.
Pseudo-code [#pseudo]_::
if os.name == "nt":
def time():
return _time.GetSystemTimeAsFileTime()
else:
def time():
if hasattr(time, "clock_gettime"):
try:
# resolution = 1 nanosecond
return time.clock_gettime(time.CLOCK_REALTIME)
except OSError:
# CLOCK_REALTIME is not supported (unlikely)
pass
if hasattr(_time, "gettimeofday"):
try:
# resolution = 1 microsecond
return _time.gettimeofday()
except OSError:
# gettimeofday() should not fail
pass
if hasattr(_time, "ftime"):
# resolution = 1 millisecond
return _time.ftime()
else:
# resolution = 1 second
return _time.time()
time.monotonic()
----------------
Monotonic clock, cannot go backward. It is not affected by system clock
updates. The reference point of the returned value is undefined so only the
difference between consecutive calls is valid.
Availability: Windows, Mac OS X, Unix.
Pseudo-code [#pseudo]_::
if os.name == 'nt':
# GetTickCount64() requires Windows Vista, Server 2008 or later
if hasattr(time, '_GetTickCount64'):
def monotonic():
return _time.GetTickCount64() * 1e-3
else:
def monotonic():
ticks = _time.GetTickCount()
if ticks < monotonic.last:
# Integer overflow detected
monotonic.delta += 2**32
monotonic.last = ticks
return (ticks + monotonic.delta) * 1e-3
monotonic.last = 0
monotonic.delta = 0
elif os.name == 'mac':
def monotonic():
if monotonic.factor is None:
factor = _time.mach_timebase_info()
monotonic.factor = timebase[0] / timebase[1]
return _time.mach_absolute_time() * monotonic.factor
monotonic.factor = None
elif hasattr(time, "clock_gettime"):
def monotonic():
if monotonic.use_clock_highres:
try:
time.clock_gettime(time.CLOCK_HIGHRES)
except OSError:
monotonic.use_clock_highres = False
return time.clock_gettime(time.CLOCK_MONOTONIC)
monotonic.use_clock_highres = (hasattr(time, 'clock_gettime')
and hasattr(time, 'CLOCK_HIGHRES'))
On Windows, QueryPerformanceCounter() is not used even though it has a better
precision than GetTickCount(). It is not reliable and has too many issues.
.. note::
time.monotonic() detects GetTickCount() integer overflow (32 bits, roll-over
after 49.7 days): it increases a delta by 2\ :sup:`32` each time than an
overflow is detected. The delta is stored in the process-local state and so
the value of time.monotonic() may be different in two Python processes
running for more than 49 days.
time.sleep()
------------
Suspend execution for the given number of seconds. The actual suspension time
may be less than that requested because any caught signal will terminate the
time.sleep() following execution of that signal's catching routine. Also, the
suspension time may be longer than requested by an arbitrary amount because of
the scheduling of other activity in the system.
Pseudo-code [#pseudo]_::
try:
import select
except ImportError:
has_select = False
else:
has_select = hasattr(select, "select")
if has_select:
def sleep(seconds):
return select.select([], [], [], seconds)
elif hasattr(_time, "delay"):
def sleep(seconds):
milliseconds = int(seconds * 1000)
_time.delay(milliseconds)
elif os.name == "nt":
def sleep(seconds):
milliseconds = int(seconds * 1000)
win32api.ResetEvent(hInterruptEvent);
win32api.WaitForSingleObject(sleep.sigint_event, milliseconds)
sleep.sigint_event = win32api.CreateEvent(NULL, TRUE, FALSE, FALSE)
# SetEvent(sleep.sigint_event) will be called by the signal handler of SIGINT
elif os.name == "os2":
def sleep(seconds):
milliseconds = int(seconds * 1000)
DosSleep(milliseconds)
else:
def sleep(seconds):
seconds = int(seconds)
_time.sleep(seconds)
time.clock()
------------
On Unix, return the current processor time as a floating point number expressed
in seconds. The precision, and in fact the very definition of the meaning of
"processor time", depends on that of the C function of the same name, but in any
case, this is the function to use for benchmarking Python or timing algorithms.
On Windows, this function returns wall-clock seconds elapsed since the first
call to this function, as a floating point number, based on the Win32 function
``QueryPerformanceCounter()``. The resolution is typically better than one
microsecond.
Pseudo-code [#pseudo]_::
if os.name == 'nt':
def clock():
if clock.use_performance_counter:
if clock.perf_frequency is None:
try:
clock.perf_frequency = float(_time.QueryPerformanceFrequency())
except OSError:
# QueryPerformanceFrequency() fails if the installed
# hardware does not support a high-resolution performance
# counter
clock.use_performance_counter = False
else:
return _time.QueryPerformanceCounter() / clock.perf_frequency
else:
return _time.QueryPerformanceCounter() / clock.perf_frequency
return _time.clock()
clock.use_performance_counter = True
clock.perf_frequency = None
else:
clock = _time.clock
time.get_clock_info(name)
-------------------------
Get information on the specified clock. Supported clocks:
* "clock": time.clock()
* "monotonic": time.monotonic()
* "time": time.time()
Return a dictionary with the following keys:
* Mandatory keys:
* "function" (str): name of the underlying operating system function.
Examples: "QueryPerformanceCounter()", "clock_gettime(CLOCK_REALTIME)".
* "resolution" (float): resolution in seconds of the clock
* "is_monotonic" (bool): True if the clock cannot go backward
* Optional keys:
* "precision" (float): precision in seconds of the clock
* "is_adjusted" (bool): True if the clock can be adjusted (e.g. by a NTP
daemon)
Glossary
========
:Accuracy:
Is the answer correct? Any clock will eventually <drift>; if a clock is
intended to match <Civil Time>, it will need to be <adjusted> back to the
"true" time.
:Adjusted:
Resetting a clock to the correct time. This may be done either with a
<Step> or by <Slewing>.
:Civil Time:
Time of day; external to the system. 10:45:13am is a Civil time; 45 seconds
is not. Provided by existing function time.localtime() and time.gmtime().
Not changed by this PEP.
:Clock:
An instrument for measuring time. Different clocks have different
characteristics; for example, a clock with <nanonsecond> <precision> may
start to <drift> after a few minutes, while a less precise clock remained
accurate for days. This PEP is primarily concerned with clocks which use a
unit of seconds.
:Counter:
A clock which increments each time a certain event occurs. A counter is
<strictly monotonic>, but not <clock_monotonic>. It can be used to generate
a unique (and ordered) timestamp, but these timestamps cannot be mapped to
<civil time>; tick creation may well be bursty, with several advances in the
same millisecond followed by several days without any advance.
:CPU Time:
A measure of how much CPU effort has been spent on a certain task. CPU
seconds are often normalized (so that a variable number can occur in the
same actual second). CPU seconds can be important when profiling, but they
do not map directly to user response time, nor are they directly comparable
to (real time) seconds.
:Duration:
Elapsed time. The difference between the starting and ending times. A
defined <epoch> creates an implicit (and usually large) duration. More
precision can generally be provided for a relatively small <duration>.
:Drift:
The accumulated error against "true" time, as defined externally to the
system.
:Epoch:
The reference point of a clock. For clocks providing <civil time>, this is
often midnight as the day (and year) rolled over to January 1, 1970. For a
<clock_monotonic> clock, the epoch may be undefined (represented as None).
:Latency:
Delay. By the time a clock call returns, the <real time> has advanced,
possibly by more than the precision of the clock.
:Monotonic:
The characteristics expected of a monotonic clock in practice. Moving in at
most one direction; for clocks, that direction is forward. The <clock>
should also be <steady>, and should be convertible to a unit of seconds.
The tradeoffs often include lack of a defined <epoch> or mapping to <Civil
Time>, and being more expensive (in <latency>, power usage, or <duration>
spent within calls to the clock itself) to use. For example, the clock may
represent (a constant multiplied by) ticks of a specific quartz timer on a
specific CPU core, and calls would therefore require synchronization between
cores.
:Precision:
Significant Digits. What is the smallest duration that the clock can
distinguish? This differs from <resolution> in that a difference greater
than the minimum precision is actually meaningful.
:Process Time:
Time elapsed since the process began. It is typically measured in <CPU
time> rather than <real time>, and typically does not advance while the
process is suspended.
:Real Time:
Time in the real world. This differs from <Civil time> in that it is not
<adjusted>, but they should otherwise advance in lockstep. It is not
related to the "real time" of "Real Time [Operating] Systems". It is
sometimes called "wall clock time" to avoid that ambiguity; unfortunately,
that introduces different ambiguities.
:Resolution:
Represented Digits. Note that many clocks will have a resolution greater
than their actual <precision>.
:Slew:
A slight change to a clock's speed, usually intended to correct <drift> with
respect to an external authority.
:Stability:
Persistence of accuracy. A measure of expected <drift>.
:Steady:
A clock with high <stability> and relatively high <accuracy> and
<precision>. In practice, it is often used to indicate a <clock_monotonic>
clock, but places greater emphasis on the consistency of the duration
between subsequent ticks.
:Step:
An instantaneous change in the represented time. Instead of speeding or
slowing the clock (<slew>), a single offset is permanently added.
:System Time:
Time as represented by the Operating System.
:Thread Time:
Time elapsed since the thread began. It is typically measured in <CPU time>
rather than <real time>, and typically does not advance while the thread is
idle.
:Wallclock:
What the clock on the wall says. This is typically used as a synonym for
<real time>; unfortunately, wall time is itself ambiguous.
Hardware clocks
===============
List of hardware clocks
-----------------------
* HPET: An High Precision Event Timer (HPET) chip consists of a 64-bit up-counter (main counter)
counting at least at 10 MHz and a set of up to 256 comparators (at
least 3). Each HPET can have up to 32 timers. HPET can cause around 3 seconds of drift per day.
* TSC (Time Stamp Counter): Historically, the TSC increased with every internal
processor clock cycle, but now the rate is usually constant (even if the
processor changes frequency) and usually equals the maximum processor
frequency. Multiple cores having different TSC values. Hibernation of system
will reset TSC value. The RDTSC instruction can be used to read this counter.
CPU frequency scaling for power saving.
* ACPI Power Management Timer: ACPI 24-bit timer with a frequency
of 3.5 MHz (3,579,545 Hz).
* Cyclone: The Cyclone timer uses a 32-bit counter on IBM Extended
X-Architecture (EXA) chipsets which include computers that use the
IBM "Summit" series chipsets (ex: x440). This is available in IA32
and IA64 architectures.
* PIT (programmable interrupt timer): Intel 8253/8254 chipsets with a
configurable frequency in range 18.2 Hz - 1.2 MHz. It is a 16-bit counter.
* RTC (Real-time clock). Most RTCs use a crystal oscillator with a frequency of
32,768 Hz
Linux clocksource
-----------------
There were 4 implementations of the time in the Linux kernel: UTIME
(1996), timer wheel (1997), HRT (2001) and hrtimers (2007). The
latter is the result of the "high-res-timers" project started by
George Anzinger in 2001, with contributions by Thomas Gleixner and
Douglas Niehaus. hrtimers implementation was merged into Linux
2.6.21, released in 2007.
hrtimers supports various clock sources. It sets a priority to each
source to decide which one will be used. Linux supports the following
clock sources:
* tsc
* hpet
* pit
* pmtmr: ACPI Power Management Timer
* cyclone
High-resolution timers are not supported on all hardware
architectures. They are at least provided on x86/x86_64, ARM and
PowerPC.
The /sys/devices/system/clocksource/clocksource0 directory contains two useful files:
* available_clocksource: list of available clock sources
* current_clocksource: clock source currently used. It is possible to change
the current clocksource by writing the name of a clocksource into this
file.
/proc/timer_list contains the list of all hardware timers.
Read also the `time(7) manual page
<http://www.kernel.org/doc/man-pages/online/pages/man7/time.7.html>`_:
"overview of time and timers".
FreeBSD timecounter
-------------------
kern.timecounter.choice list available hardward clocks with their priority.
The sysctl program can be used to change the timecounter. Example::
# dmesg|grep Timecounter
Timecounter "i8254" frequency 1193182 Hz quality 0
Timecounter "ACPI-safe" frequency 3579545 Hz quality 850
Timecounter "HPET" frequency 100000000 Hz quality 900
Timecounter "TSC" frequency 3411154800 Hz quality 800
Timecounters tick every 10.000 msec
# sysctl kern.timecounter.choice
kern.timecounter.choice: TSC(800) HPET(900) ACPI-safe(850) i8254(0) dummy(-1000000)
# sysctl kern.timecounter.hardware="ACPI-fast"
kern.timecounter.hardware: HPET -> ACPI-fast
Available clocks:
* "TSC": Time Stamp Counter of the procesor
* "HPET": High Precision Event Timer
* "ACPI-fast": ACPI Power Management timer (fast mode)
* "ACPI-safe": ACPI Power Management timer (safe mode)
* "i8254": PIT with Intel 8254 chipset
The `commit 222222
<http://svnweb.freebsd.org/base?view=revision&revision=222222>`_ (May 2011)
decreased ACPI-fast timecounter quality to 900 and increased HPET timecounter
quality to 950: "HPET on modern platforms usually have better resolution and
lower latency than ACPI timer".
Read `Timecounters: Efficient and precise timekeeping in SMP kernels
<http://phk.freebsd.dk/pubs/timecounter.pdf>`_ by Poul-Henning Kamp (2002) for
the FreeBSD Project.
Performances
------------
Reading an hardware clock has a cost. The following table compares the
performance of different hardware clocks on Linux 3.3 with Intel Core i7-2600
at 3.40GHz (8 cores).
======================== ====== ======= ======
Function TSC ACPI PM HPET
======================== ====== ======= ======
time() 2 ns 2 ns 2 ns
CLOCK_REALTIME_COARSE 10 ns 10 ns 10 ns
CLOCK_MONOTONIC_COARSE 12 ns 13 ns 12 ns
CLOCK_THREAD_CPUTIME_ID 134 ns 135 ns 135 ns
CLOCK_PROCESS_CPUTIME_ID 127 ns 129 ns 129 ns
clock() 146 ns 146 ns 143 ns
gettimeofday() 23 ns 726 ns 637 ns
CLOCK_MONOTONIC_RAW 31 ns 716 ns 607 ns
CLOCK_REALTIME 27 ns 707 ns 629 ns
CLOCK_MONOTONIC 27 ns 723 ns 635 ns
======================== ====== ======= ======
Each function was called 10,000,000 times and CLOCK_MONOTONIC was used to get
the time before and after. The benchmark was run 5 times to keep the minimum
time.
FreeBSD 8.0 in kvm with hardware virtualization:
======================== ====== ========= ======= =======
Function TSC ACPI-Safe HPET i8254
======================== ====== ========= ======= =======
time() 191 ns 188 ns 189 ns 188 ns
CLOCK_SECOND 187 ns 184 ns 187 ns 183 ns
CLOCK_REALTIME_FAST 189 ns 180 ns 187 ns 190 ns
CLOCK_UPTIME_FAST 191 ns 185 ns 186 ns 196 ns
CLOCK_MONOTONIC_FAST 188 ns 187 ns 188 ns 189 ns
CLOCK_THREAD_CPUTIME_ID 208 ns 206 ns 207 ns 220 ns
CLOCK_VIRTUAL 280 ns 279 ns 283 ns 296 ns
CLOCK_PROF 289 ns 280 ns 282 ns 286 ns
clock() 342 ns 340 ns 337 ns 344 ns
CLOCK_UPTIME_PRECISE 197 ns 10380 ns 4402 ns 4097 ns
CLOCK_REALTIME 196 ns 10376 ns 4337 ns 4054 ns
CLOCK_MONOTONIC_PRECISE 198 ns 10493 ns 4413 ns 3958 ns
CLOCK_UPTIME 197 ns 10523 ns 4458 ns 4058 ns
gettimeofday() 202 ns 10524 ns 4186 ns 3962 ns
CLOCK_REALTIME_PRECISE 197 ns 10599 ns 4394 ns 4060 ns
CLOCK_MONOTONIC 201 ns 10766 ns 4498 ns 3943 ns
======================== ====== ========= ======= =======
Each function was called 100,000 times and CLOCK_MONOTONIC was used to get
the time before and after. The benchmark was run 5 times to keep the minimum
time.
NTP adjustment
==============
NTP has diffent methods to adjust a clock:
* "slewing": change the clock frequency to be slightly faster or slower
(which is done with adjtime()). Since the slew rate is limited to 0.5 ms/s,
each second of adjustment requires an amortization interval of 2000 s. Thus,
an adjustment of many seconds can take hours or days to amortize.
* "stepping": jump by a large amount in a single discrete step (which is done
with settimeofday())
By default, the time is slewed if the offset is less than 128 ms, or stepped
otherwise.
Slewing is generally desirable (i.e. we should use CLOCK_MONOTONIC, not
CLOCK_MONOTONIC_RAW) if one wishes to measure "real" time (and not a time-like
object like CPU cycles). This is because the clock on the other end of the NTP
connection from you is probably better at keeping time: hopefully that thirty
five thousand dollars of Cesium timekeeping goodness is doing something better
than your PC's $3 quartz crystal, after all.
Get more detail in the `documentation of the NTP daemon
<http://doc.ntp.org/4.1.2/ntpd.htm>`_.
Operating system clocks
=======================
Monotonic clocks
----------------
========================= ========== =============== ============= ===============
Name Resolution Adjusted Include sleep Include suspend
========================= ========== =============== ============= ===============
gethrtime 1 ns No Yes Yes
CLOCK_HIGHRES 1 ns No Yes ?
CLOCK_MONOTONIC 1 ns Slewed on Linux Yes No
CLOCK_MONOTONIC_RAW 1 ns No Yes No
mach_absolute_time() 1 ns No Yes No
QueryPerformanceCounter() \- No Yes ?
GetTickCount[64]() 1 ms No Yes Yes
timeGetTime() 1 ms No Yes ?
========================= ========== =============== ============= ===============
Examples of clock precision on x86_64:
========================= ================ ===============
Name Operating system Precision
========================= ================ ===============
CLOCK_MONOTONIC_RAW Linux 3.2 1 ns
CLOCK_MONOTONIC Linux 3.2 1 ns
CLOCK_HIGHRES SunOS 5.11 2 ns
CLOCK_MONOTONIC SunOS 5.11 2 ns
QueryPerformanceCounter Windows Seven 10 ns
CLOCK_MONOTONIC FreeBSD 8.2 11 ns
CLOCK_MONOTONIC OpenBSD 5.0 10 ms
GetTickCount Windows Seven 15.6 ms
========================= ================ ===============
For CLOCK_MONOTONIC and CLOCK_MONOTONIC_RAW, the precision of this table is the
result of clock_getres(). It looks like Linux does not implement
clock_getres() and always return 1 nanosecond.
mach_absolute_time
^^^^^^^^^^^^^^^^^^
Mac OS X provides a monotonic clock: mach_absolute_time(). It is
based on absolute elapsed time delta since system boot. It is not
adjusted and cannot be set.
mach_timebase_info() gives a fraction to convert the clock value to a
number of nanoseconds. According to the documentation (`Technical Q&A
QA1398 <https://developer.apple.com/library/mac/#qa/qa1398/>`_),
mach_timebase_info() is always equal to one and never fails, even
if the function may fail according to its prototype.
mach_absolute_time() stops during a sleep on a PowerPC CPU, but not on
an Intel CPU: `Different behaviour of mach_absolute_time() on i386/ppc
<http://lists.apple.com/archives/PerfOptimization-dev/2006/Jul/msg00024.html>`_.
mach_absolute_time() has a resolution of 1 nanosecond.
CLOCK_MONOTONIC, CLOCK_MONOTONIC_RAW
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
CLOCK_MONOTONIC and CLOCK_MONOTONIC_RAW represent monotonic time since
some unspecified starting point. They cannot be set.
Documentation: refer to the manual page of your operating system.
Examples:
* `FreeBSD clock_gettime() manual page
<http://www.freebsd.org/cgi/man.cgi?query=clock_gettime>`_
* `Linux clock_gettime() manual page
<http://linux.die.net/man/3/clock_gettime>`_
CLOCK_MONOTONIC is available at least on the following operating systems:
* DragonFly BSD, FreeBSD >= 5.0, OpenBSD, NetBSD
* Linux
* Solaris
The following operating systems don't support CLOCK_MONOTONIC:
* GNU/Hurd (see `open issues/ clock_gettime
<http://www.gnu.org/software/hurd/open_issues/clock_gettime.html>`_)
* Mac OS X
* Windows
On Linux, NTP may adjust the CLOCK_MONOTONIC rate (slewed), but it cannot
jump backward.
CLOCK_MONOTONIC_RAW is specific to Linux. It is similar to CLOCK_MONOTONIC, but
provides access to a raw hardware-based time that is not subject to NTP
adjustments. CLOCK_MONOTONIC_RAW requires Linux 2.6.28 or later.
CLOCK_MONOTONIC stops while the machine is suspended.
clock_gettime() fails if the system does not support the specified
clock, even if the standard C library supports it. For example,
CLOCK_MONOTONIC_RAW requires a kernel version 2.6.28 or later.
clock_getres() gives the clock resolution. It is 1 nanosecond on
Linux.
.. note::
clock_gettime() requires to link the program against the rt
(real-time) library.
.. note::
Linux provides also CLOCK_MONOTONIC_COARSE since Linux 2.6.32 which has less
accurate than CLOCK_MONOTONIC but is faster.
Windows: QueryPerformanceCounter
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
High-resolution performance counter. It is monotonic.
QueryPerformanceFrequency() gives its frequency.
It has a much higher resolution, but has lower long term precision than
GetTickCount() and timeGetTime() clocks. For example, it will drift
compared to the low precision clocks.
Documentation:
* `MSDN: QueryPerformanceCounter() documentation
<http://msdn.microsoft.com/en-us/library/windows/desktop/ms644904%28v=vs.85%29.aspx>`_
* `MSDN: QueryPerformanceFrequency() documentation
<http://msdn.microsoft.com/en-us/library/windows/desktop/ms644905%28v=vs.85%29.aspx>`_
Hardware clocks used by QueryPerformanceCounter:
* Windows XP: RDTSC instruction of Intel processors, the clock
frequency is the frequency of the processor (between 200 MHz and 3
GHz, usually greater than 1 GHz nowadays).
* Windows 2000: ACPI power management timer, frequency = 3,549,545 Hz.
It can be forced through the "/usepmtimer" flag in boot.ini.
.. * Windows 95/98: 8245 PIT chipset, frequency = 1,193,181 Hz
QueryPerformanceFrequency() should only be called once: the frequency
will not change while the system is running. It fails if the
installed hardware does not support a high-resolution performance
counter.
QueryPerformanceCounter() cannot be adjusted:
`SetSystemTimeAdjustment()
<http://msdn.microsoft.com/en-us/library/windows/desktop/ms724943(v=vs.85).aspx>`_
only adjusts the system time.
Bugs:
* The performance counter value may unexpectedly leap forward because
of a hardware bug, see `KB274323`_.
* On VirtualBox, QueryPerformanceCounter() does not increment the high
part every time the low part overflows, see `Monotonic timers
<http://code-factor.blogspot.fr/2009/11/monotonic-timers.html>`_
(2009).
* VirtualBox had a bug in its HPET virtualized device:
QueryPerformanceCounter() did jump forward by approx. 42 seconds (`issue
#8707 <https://www.virtualbox.org/ticket/8707>`_).
* Windows XP had a bug (see `KB896256`_): on a multiprocessor
computer, QueryPerformanceCounter() returned a different value for
each processor. The bug was fixed in Windows XP SP2.
* Issues with processor with variable frequency: the frequency is changed
depending on the workload to reduce memory consumption.
* Chromium don't use QueryPerformanceCounter() on Athlon X2 CPUs (model 15)
because "QueryPerformanceCounter is unreliable" (see base/time_win.cc in
Chromium source code)
.. _KB896256: http://support.microsoft.com/?id=896256
.. _KB274323: http://support.microsoft.com/?id=274323
Windows: GetTickCount(), GetTickCount64()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
GetTickCount() and GetTickCount64() are monotonic, cannot fail and are
not adjusted by SetSystemTimeAdjustment(). MSDN documentation:
`GetTickCount()
<http://msdn.microsoft.com/en-us/library/windows/desktop/ms724408(v=vs.85).aspx>`_,
`GetTickCount64()
<http://msdn.microsoft.com/en-us/library/windows/desktop/ms724411(v=vs.85).aspx>`_.
The elapsed time retrieved by GetTickCount() or GetTickCount64()
includes time the system spends in sleep or hibernation.
GetTickCount64() was added to Windows Vista and Windows Server 2008.
The clock resolution is 1 millisecond. Its precision is usually around
15 ms. It is possible to improve the precision using the `undocumented
NtSetTimerResolution() function
<http://undocumented.ntinternals.net/UserMode/Undocumented%20Functions/Time/NtSetTimerResolution.html>`_.
There are applications using this undocumented function, example:
`Timer Resolution <http://www.lucashale.com/timer-resolution/>`_.
WaitForSingleObject() use the same timer than GetTickCount() with the same
resolution.
GetTickCount() has an precision of 55 ms on Windows 9x.
Windows: timeGetTime
^^^^^^^^^^^^^^^^^^^^
The timeGetTime function retrieves the system time, in milliseconds.
The system time is the time elapsed since Windows was started. Read
the `timeGetTime() documentation
<http://msdn.microsoft.com/en-us/library/windows/desktop/dd757629(v=vs.85).aspx>`_.
The return type of timeGetTime() is a 32-bit unsigned integer. As
GetTickCount(), timeGetTime() rolls over after 2^32 milliseconds (49.7
days).
The elapsed time retrieved by timeGetTime() includes time the system spends in
sleep.
The default precision of the timeGetTime function can be five
milliseconds or more, depending on the machine.
timeBeginPeriod() can be used to increase the precision of timeGetTime() up to
1 millisecond, but it negatively affects power consumption. Calling
timeBeginPeriod() also affects the granularity of some other timing calls, such
as CreateWaitableTimer(), WaitForSingleObject() and Sleep().
.. note::
timeGetTime() and timeBeginPeriod() are part the Windows multimedia
library and so require to link the program against winmm or to
dynamically load the library.
Solaris: CLOCK_HIGHRES
^^^^^^^^^^^^^^^^^^^^^^
The Solaris OS has a CLOCK_HIGHRES timer that attempts to use an
optimal hardware source, and may give close to nanosecond resolution.
CLOCK_HIGHRES is the nonadjustable, high-resolution clock. For timers
created with a clockid_t value of CLOCK_HIGHRES, the system will
attempt to use an optimal hardware source.
Solaris: gethrtime
^^^^^^^^^^^^^^^^^^
The gethrtime() function returns the current high-resolution real
time. Time is expressed as nanoseconds since some arbitrary time in
the past; it is not correlated in any way to the time of day, and thus
is not subject to resetting or drifting by way of adjtime() or
settimeofday(). The hires timer is ideally suited to performance
measurement tasks, where cheap, accurate interval timing is required.
The linearity of gethrtime() is not preserved accross cpr
suspend-resume cycle (`Bug 4272663
<http://wesunsolve.net/bugid/id/4272663>`_).
Read the `gethrtime() manual page of Solaris 11
<http://docs.oracle.com/cd/E23824_01/html/821-1465/gethrtime-3c.html#scrolltoc>`_.
On Solaris, gethrtime() is the same as clock_gettime(CLOCK_MONOTONIC).
System time clocks
------------------
========================= ===============
Name Resolution
========================= ===============
CLOCK_REALTIME 1 ns
GetSystemTimeAsFileTime 100 ns
gettimeofday() 1 µs
ftime() 1 ms
time() 1 sec
========================= ===============
Examples of clock precision on x86_64:
========================= ================ ===============
Name Operating system Precision
========================= ================ ===============
CLOCK_REALTIME Linux 3.2 1 ns
CLOCK_REALTIME FreeBSD 8.2 11 ns
CLOCK_REALTIME SunOS 5.11 10 ms
CLOCK_REALTIME OpenBSD 5.0 10 ms
GetSystemTimeAsFileTime Windows Seven 15.6 ms
========================= ================ ===============
For CLOCK_REALTIME, the precision of this table is the result of clock_getres().
It looks like Linux does not implement clock_getres() and always return 1
nanosecond.
.. note::
Linux provides also CLOCK_REALTIME_COARSE since Linux 2.6.32 which has less
accurate than CLOCK_REALTIME but is faster.
Windows: GetSystemTimeAsFileTime
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The system time can be read using GetSystemTimeAsFileTime(), ftime()
and time().
The system time resolution can be read using
GetSystemTimeAdjustment(). The precision is usually between 1
millisecond and 15 milliseconds. Resolution:
* GetSystemTimeAsFileTime(): 100 nanoseconds
* ftime(): 1 millisecond
* time(): 1 second
Read the `GetSystemTimeAsFileTime() documentation
<http://msdn.microsoft.com/en-us/library/windows/desktop/ms724397(v=vs.85).aspx>`_.
The system time can be set using SetSystemTime().
System time on UNIX
^^^^^^^^^^^^^^^^^^^
gettimeofday(), ftime(), time() and clock_gettime(CLOCK_REALTIME)
return the system clock.
Resolution:
* clock_gettime(): clock_getres(CLOCK_REALTIME)
* gettimeofday(): 1 microsecond
* ftime(): 1 millisecond
* time(): 1 second
The system time can be set using settimeofday() or
clock_settime(CLOCK_REALTIME).
Process time
------------
The process time cannot be set. It is not monotonic: the clocks stop while the
process is idle.
========================= ===============
Name Resolution
========================= ===============
GetProcessTimes() 100 ns
CLOCK_PROCESS_CPUTIME_ID 1 ns
clock() \-
========================= ===============
Examples of clock precision on x86_64:
========================= ================ ===============
Name Operating system Precision
========================= ================ ===============
CLOCK_PROCESS_CPUTIME_ID Linux 3.2 1 ns
clock() Linux 3.2 1 µs
clock() SunOS 5.11 1 µs
clock() FreeBSD 8.2 7.8 ms
clock() OpenBSD 5.0 10 ms
GetProcessTimes() Windows Seven 15.6 ms
========================= ================ ===============
The precision of clock() in this table is the result of 1 / CLOCKS_PER_SEC.
For CLOCK_PROCESS_CPUTIME_ID, the precision of this table is the result of
clock_getres(). It looks like Linux does not implement clock_getres() and
always return 1 nanosecond. For GetProcessTimes(), the precision is read using
GetSystemTimeAdjustment().
Functions
^^^^^^^^^
* Windows: GetProcessTimes()
* clock_gettime(CLOCK_PROCESS_CPUTIME_ID): High-resolution per-process
timer from the CPU.
* clock():
* Windows: The elapsed wall-clock time since the start of the
process (elapsed time in seconds times CLOCKS_PER_SEC). It can
fail.
* UNIX: returns an approximation of processor time used by the
program.
* times()
* getrusage(): ru_utime and ru_stime fields
Resolution:
* clock() rate is CLOCKS_PER_SEC. It was called CLK_TCK in Microsoft
C before 6.0.
* The clock resolution can be read using clock_getres().
* GetProcessTimes(): call GetSystemTimeAdjustment().
Thread time
-----------
The thread time cannot be set. It is not monotonic: the clocks stop while the
thread is idle.
========================= ===============
Name Resolution
========================= ===============
GetThreadTimes() 100 ns
CLOCK_THREAD_CPUTIME_ID 1 ns
========================= ===============
Examples of clock precision on x86_64:
========================= ================ ===============
Name Operating system Precision
========================= ================ ===============
CLOCK_THREAD_CPUTIME_ID Linux 3.2 1 ns
CLOCK_THREAD_CPUTIME_ID FreeBSD 8.2 1 µs
GetThreadTimes() Windows Seven 15.6 ms
========================= ================ ===============
For CLOCK_THREAD_CPUTIME_ID, the precision of this table is the result of
clock_getres(). It looks like Linux does not implement clock_getres() and
always return 1 nanosecond. For GetThreadTimes(), the precision is read using
GetSystemTimeAdjustment().
Functions
^^^^^^^^^
* Windows: GetThreadTimes()
* clock_gettime(CLOCK_THREAD_CPUTIME_ID): Thread-specific CPU-time
clock.
Resolution:
* CLOCK_THREAD_CPUTIME_ID: call clock_getres().
* GetThreadTimes(): call GetSystemTimeAdjustment()
See also pthread_getcpuclockid().
Windows: QueryUnbiasedInterruptTime
-----------------------------------
Gets the current unbiased interrupt time from the biased interrupt
time and the current sleep bias amount. This time is not affected by
power management sleep transitions.
The elapsed time retrieved by the QueryUnbiasedInterruptTime function
includes only time that the system spends in the working state.
QueryUnbiasedInterruptTime() is not monotonic.
QueryUnbiasedInterruptTime() was introduced in Windows 7.
Sleep, suspend and monotonic time
=================================
Linux
-----
sleep() is not affected by system clock update.
On Linux, CLOCK_MONOTONIC and CLOCK_MONOTONIC_RAW includes time the system
spends in sleep; but it does not include time spent in hibernation (ACPI S3
mode). If the system clock jumps backward, CLOCK_MONOTONIC and
CLOCK_MONOTONIC_RAW are not affected.
Linux 2.6.39 and glibc 2.14 introduces a new clock: CLOCK_BOOTTIME.
CLOCK_BOOTTIME is idential to CLOCK_MONOTONIC, except it also includes any time
spent in suspend. Read also `Waking systems from suspend
<http://lwn.net/Articles/429925/>`_ (March, 2011).
Other operating systems
-----------------------
On Windows, GetTickCount() and GetTickCount64() include time the system
spends in sleep; sleep() is not affected by system clock update.
On FreeBSD 8, CLOCK_MONOTONIC include time the system spends in sleep; sleep()
is not affected by system clock update.
On OpenIndiana, CLOCK_MONOTONIC include time the system spends in sleep;
sleep() is not affected by system clock update.
On Mac OS X, mach_absolute_time() include time the system spends in sleep;
sleep() is not affected by system clock update. Sleep is interrupted during
suspend.
Sleeping
========
Sleep can be interrupted by a signal: the function fails with EINTR.
======================== ==========
Name Resolution
======================== ==========
nanosleep() 1 ns
clock_nanosleep() 1 ns
usleep() 1 µs
delay() 1 µs
sleep() 1 sec
======================== ==========
Other functions:
======================== ==========
Name Resolution
======================== ==========
sigtimedwait() 1 ns
pthread_cond_timedwait() 1 ns
sem_timedwait() 1 ns
select() 1 µs
epoll() 1 ms
poll() 1 ms
WaitForSingleObject() 1 ms
======================== ==========
Classic functions
-----------------
* sleep(seconds)
* usleep(microseconds)
* nanosleep(nanoseconds, remaining):
`Linux manpage of nanosleep()
<http://www.kernel.org/doc/man-pages/online/pages/man2/nanosleep.2.html>`_
* delay(milliseconds)
clock_nanosleep
---------------
clock_nanosleep(clock_id, flags, nanoseconds, remaining):
`Linux manpage of clock_nanosleep()
<http://www.kernel.org/doc/man-pages/online/pages/man2/clock_nanosleep.2.html>`_.
If flags is TIMER_ABSTIME, then request is interpreted as an absolute
time as measured by the clock, clock_id. If request is less than or
equal to the current value of the clock, then clock_nanosleep() returns
immediately without suspending the calling thread.
POSIX.1 specifies that changing the value of the CLOCK_REALTIME clock via
clock_settime(2) shall have no effect on a thread that is blocked on a relative
clock_nanosleep().
select()
--------
select(nfds, readfds, writefds, exceptfs, timeout).
Since Linux 2.6.28, select() uses high-resolution timers to handle the timeout.
A process has a "slack" attribute to configure the precision of the timeout, the
default slack is 50 microseconds. Before Linux 2.6.28, timeouts for select()
were handled by the main timing subsystem at a jiffy-level resolution. Read
also `High- (but not too high-) resolution timeouts
<http://lwn.net/Articles/296578/>`_ and `Timer slack
<http://lwn.net/Articles/369549/>`_.
Other functions
---------------
* poll(), epoll()
* sigtimedwait(). POSIX: "If the Monotonic Clock option is supported, the
CLOCK_MONOTONIC clock shall be used to measure the time interval specified
by the timeout argument."
* pthread_cond_timedwait(), pthread_condattr_setclock(). "The default value of
the clock attribute shall refer to the system clock."
* sem_timedwait(): "If the Timers option is supported, the timeout shall be
based on the CLOCK_REALTIME clock. If the Timers option is not supported,
the timeout shall be based on the system clock as returned by the time()
function. The resolution of the timeout shall be the resolution of the clock
on which it is based."
* WaitForSingleObject(): use the same timer than GetTickCount() with the same
resolution.
Alternatives: API design
========================
Other names for time.monotonic()
--------------------------------
* time.counter()
* time.seconds()
* time.steady()
* time.timeout_clock()
* time.wallclock(): it is not the system time aka the "wall clock", but
a monotonic clock with an unspecified starting point
The name "time.try_monotonic()" was also proposed when time.monotonic() was
falling back to the system clock when no monotonic clock was available.
Only expose operating system clocks
-----------------------------------
To not have to define high-level clocks, which is a difficult task, a simpler
approach is to only expose operating system clocks. time.clock_gettime() and
related clock identifiers were already added to Python 3.3 for example.
Fallback to system clock
------------------------
If no monotonic clock is available, time.monotonic() falls back to the system
clock.
Issues:
* It is hard to define correctly such function in the documentation: is it
monotonic? is it steady? is it adjusted?
* Some user want to decide what to do when no monotonic clock is available:
use another clock, display an error, or do something else
One function choosing the clock from a list of constraints
----------------------------------------------------------
``time.get_clock(*flags)`` with the following flags:
* time.MONOTONIC: clock cannot go backard
* time.STEADY: clock rate is steady and the clock is not adjusted
* time.HIGHRES: clock with the highest resolutions
time.get_clock() returns None if the clock is found and so calls can be chained
using the or operator. Example::
get_time = time.get_clock(time.MONOTONIC) or time.get_clock(time.STEADY) or time.time()
t = get_time()
Example of flags of system clocks:
* QueryPerformanceCounter: MONOTONIC | HIGHRES
* GetTickCount: MONOTONIC | STEADY
* CLOCK_MONOTONIC: MONOTONIC | STEADY (or only MONOTONIC on Linux)
* CLOCK_MONOTONIC_RAW: MONOTONIC | STEADY
* gettimeofday(): (no flag)
One function with a flag: time.monotonic(fallback=True)
-------------------------------------------------------
* time.monotonic(fallback=True) falls back to the system clock if no monotonic
clock is available or if the monotonic clock failed.
* time.monotonic(fallback=False) raises OSError if monotonic clock fails and
NotImplementedError if the system does not provide a monotonic clock
"A keyword argument that gets passed as a constant in the caller is usually
poor API."
Raising NotImplementedError for a function is something uncommon in Python and
should be avoided.
One function, no flag
---------------------
time.monotonic() returns (time: float, is_monotonic: bool).
An alternative is to use a function attribute:
time.monotonic.is_monotonic. The attribute value would be None before
the first call to time.monotonic().
Working around operating system bugs?
=====================================
Should Python ensure manually that a monotonic clock is truly
monotonic by computing the maximum with the clock value and the
previous value?
Since it's relatively straightforward to cache the last value returned
using a static variable, it might be interesting to use this to make
sure that the values returned are indeed monotonic.
* Virtual machines provide less reliable clocks.
* QueryPerformanceCounter() has known bugs (only one is not fixed yet)
Python may only work around a specific known operating system bug:
`KB274323`_ contains a code example to workaround the bug (use
GetTickCount() to detect QueryPerformanceCounter() leap).
Issues of a hacked monotonic function:
* if the clock is accidentally set forward by an hour and then back
again, you wouldn't have a useful clock for an hour
* the cache is not shared between processes so different processes wouldn't
see the same clock value
Deferred API: time.perf_counter()
=================================
Performance counter used for benchmarking and profiling. The reference point of
the returned value is undefined so only the difference between consecutive calls is
valid and is number of seconds.
Pseudo-code::
def perf_counter():
if perf_counter.use_performance_counter:
if perf_counter.perf_frequency is None:
try:
perf_counter.perf_frequency = float(_time.QueryPerformanceFrequency())
except OSError:
# QueryPerformanceFrequency() fails if the installed
# hardware does not support a high-resolution performance
# counter
perf_counter.use_performance_counter = False
else:
return _time.QueryPerformanceCounter() / perf_counter.perf_frequency
else:
return _time.QueryPerformanceCounter() / perf_counter.perf_frequency
if perf_counter.use_monotonic:
# Monotonic clock is preferred over system clock
try:
return time.monotonic()
except OSError:
perf_counter.use_monotonic = False
return time.time()
perf_counter.use_performance_counter = (os.name == 'nt')
if perf_counter.use_performance_counter:
perf_counter.perf_frequency = None
perf_counter.use_monotonic = hasattr(time, 'monotonic')
Other names proposed for time.perf_counter():
* time.hires()
* time.highres()
* time.timer()
Python source code includes a portable library to get the process time (CPU
time): `Tools/pybench/systimes.py
<http://hg.python.org/cpython/file/tip/Tools/pybench/systimes.py>`_.
Footnotes
=========
.. [#pseudo] "_time" is an hypothetical module only used for the example.
The time module is implemented in C and so there is no need for
such module.
Links
=====
Related Python issues:
* `Issue #12822: NewGIL should use CLOCK_MONOTONIC if possible.
<http://bugs.python.org/issue12822>`_
* `Issue #14222: Use time.steady() to implement timeout
<http://bugs.python.org/issue14222>`_
* `Issue #14309: Deprecate time.clock()
<http://bugs.python.org/issue14309>`_
* `Issue #14397: Use GetTickCount/GetTickCount64 instead of
QueryPerformanceCounter for monotonic clock
<http://bugs.python.org/issue14397>`_
* `Issue #14428: Implementation of the PEP 418
<http://bugs.python.org/issue14428>`_
* `Issue #14555: clock_gettime/settime/getres: Add more clock identifiers
<http://bugs.python.org/issue14555>`_
Libraries exposing monotonic clocks:
* `Java: System.nanoTime
<http://docs.oracle.com/javase/1.5.0/docs/api/java/lang/System.html#nanoTime()>`_
* `Qt library: QElapsedTimer
<http://qt-project.org/doc/qt-4.8/qelapsedtimer.html>`_
* `glib library: g_get_monotonic_time ()
<http://developer.gnome.org/glib/2.30/glib-Date-and-Time-Functions.html#g-get-monotonic-time>`_
uses GetTickCount64()/GetTickCount() on Windows,
clock_gettime(CLOCK_MONOTONIC) on UNIX or falls back to the system
clock
* `python-monotonic-time
<http://code.google.com/p/python-monotonic-time/>`_ (`github
<https://github.com/gavinbeatty/python-monotonic-time>`_)
* `Monoclock.nano_count()
<https://github.com/ludios/Monoclock>`_ uses clock_gettime(CLOCK_MONOTONIC)
and returns a number of nanoseconds
* `monotonic_clock <https://github.com/ThomasHabets/monotonic_clock>`_
* `Perl: Time::HiRes <http://perldoc.perl.org/Time/HiRes.html>`_
exposes clock_gettime(CLOCK_MONOTONIC)
* `Ruby: AbsoluteTime.now
<https://github.com/bwbuchanan/absolute_time/>`_: use
clock_gettime(CLOCK_MONOTONIC), mach_absolute_time() or
gettimeofday(). "AbsoluteTime.monotonic?" method indicates if
AbsoluteTime.now is monotonic or not.
* `libpthread
<http://code.google.com/p/libpthread/>`_: POSIX thread library for Windows
(`clock.c <http://code.google.com/p/libpthread/source/browse/src/clock.c>`_)
* `Boost.Chrono
<http://www.boost.org/doc/libs/1_49_0/doc/html/chrono.html>`_ uses:
* system_clock:
* mac = gettimeofday()
* posix = clock_gettime(CLOCK_REALTIME)
* win = GetSystemTimeAsFileTime()
* steady_clock:
* mac = mach_absolute_time()
* posix = clock_gettime(CLOCK_MONOTONIC)
* win = QueryPerformanceCounter()
* high_resolution_clock:
* steady_clock, if available system_clock, otherwise
Time:
* `hrtimers - subsystem for high-resolution kernel timers
<http://www.kernel.org/doc/Documentation/timers/hrtimers.txt>`_
* `C++ Timeout Specification
<http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2010/n3128.html>`_
* `Windows: Game Timing and Multicore Processors
<http://msdn.microsoft.com/en-us/library/ee417693.aspx>`_
* `Implement a Continuously Updating, High-Resolution Time Provider
for Windows
<http://msdn.microsoft.com/en-us/magazine/cc163996.aspx>`_
* `clockspeed <http://cr.yp.to/clockspeed.html>`_ uses a hardware tick
counter to compensate for a persistently fast or slow system clock
* `Retrieving system time
<http://en.wikipedia.org/wiki/System_time#Retrieving_system_time>`_
lists hardware clocks and time functions with their resolution and
epoch or range
* On Windows, the JavaScript runtime of Firefox interpolates
GetSystemTimeAsFileTime() with QueryPerformanceCounter() to get an
higher resolution. See the `Bug 363258 - bad millisecond resolution
for (new Date).getTime() / Date.now() on Windows
<https://bugzilla.mozilla.org/show_bug.cgi?id=363258>`_.
* `When microseconds matter
<http://www.ibm.com/developerworks/library/i-seconds/>`_: How the
IBM High Resolution Time Stamp Facility accurately measures itty
bits of time
* `Win32 Performance Measurement Options
<http://drdobbs.com/windows/184416651>`_ by Matthew Wilson, May 01, 2003
* `Counter Availability and Characteristics for Feed-forward Based Synchronization
<http://www.cubinlab.ee.unimelb.edu.au/~jrid/Publications/ridoux_ispcs09.pdf>`_
by Timothy Broomhead, Julien Ridoux, Darryl Veitch (2009)
* System Management Interrupt (SMI) issues:
* `System Management Interrupt Free Hardware
<http://linuxplumbersconf.org/2009/slides/Keith-Mannthey-SMI-plumers-2009.pdf>`_
(Keith Mannthey, 2009)
* `IBM Real-Time "SMI Free" mode driver
<http://lwn.net/Articles/318725/>`_
* `Fixing Realtime problems caused by SMI on Ubuntu
<http://wiki.linuxcnc.org/cgi-bin/wiki.pl?FixingSMIIssues>`_
* `[RFC] simple SMI detector
<http://lwn.net/Articles/316622/>`_
* `[PATCH 2.6.34-rc3] A nonintrusive SMI sniffer for x86.
<http://marc.info/?l=linux-kernel&m=127058720921201&w=1>`_
Copyright
=========
This document has been placed in the public domain.
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:
Reference in New Issue View Git Blame Copy Permalink