New PEP 564: Add new time functions with nanosecond resolution
This commit is contained in:
Victor Stinner
parent
3859df85e7
commit
8a44cd0f3c
|
|
@ -0,0 +1,302 @@
|
|||
PEP: 564
|
||||
Title: Add new time functions with nanosecond resolution
|
||||
Version: $Revision$
|
||||
Last-Modified: $Date$
|
||||
Author: Victor Stinner <victor.stinner@gmail.com>
|
||||
Status: Draft
|
||||
Type: Standards Track
|
||||
Content-Type: text/x-rst
|
||||
Created: 16-October-2017
|
||||
Python-Version: 3.7
|
||||
|
||||
|
||||
Abstract
|
||||
========
|
||||
|
||||
Add five new functions to the ``time`` module: ``time_ns()``,
|
||||
``perf_counter_ns()``, ``monotonic_ns()``, ``clock_gettime_ns()`` and
|
||||
``clock_settime_ns()``. They are similar to the function without the
|
||||
``_ns`` suffix, but have nanosecond resolution: use a number of
|
||||
nanoseconds as a Python int.
|
||||
|
||||
The best ``time.time_ns()`` resolution measured in Python is 3 times
|
||||
better then ``time.time()`` resolution on Linux and Windows.
|
||||
|
||||
|
||||
Rationale
|
||||
=========
|
||||
|
||||
Float type limited to 104 days
|
||||
------------------------------
|
||||
|
||||
The clocks resolution of desktop and latop computers is getting closer
|
||||
to nanosecond resolution. More and more clocks have a frequency in MHz,
|
||||
up to GHz for the CPU TSC clock.
|
||||
|
||||
The Python ``time.time()`` function returns the current time as a
|
||||
floatting point number which is usually a 64-bit binary floatting number
|
||||
(in the IEEE 754 format).
|
||||
|
||||
The problem is that the float type starts to lose nanoseconds after 104
|
||||
days. Conversion from nanoseconds (``int``) to seconds (``float``) and
|
||||
then back to nanoseconds (``int``) to check if conversions lose
|
||||
precision::
|
||||
|
||||
# no precision loss
|
||||
>>> x = 2 ** 52 + 1; int(float(x * 1e-9) * 1e9) - x
|
||||
0
|
||||
# precision loss! (1 nanosecond)
|
||||
>>> x = 2 ** 53 + 1; int(float(x * 1e-9) * 1e9) - x
|
||||
-1
|
||||
>>> print(datetime.timedelta(seconds=2 ** 53 / 1e9))
|
||||
104 days, 5:59:59.254741
|
||||
|
||||
``time.time()`` returns seconds elapsed since the UNIX epoch: January
|
||||
1st, 1970. This function loses precision since May 1970 (47 years ago)::
|
||||
|
||||
>>> import datetime
|
||||
>>> unix_epoch = datetime.datetime(1970, 1, 1)
|
||||
>>> print(unix_epoch + datetime.timedelta(seconds=2**53 / 1e9))
|
||||
1970-04-15 05:59:59.254741
|
||||
|
||||
|
||||
Previous rejected PEP
|
||||
---------------------
|
||||
|
||||
Five years ago, the PEP 410 proposed a large and complex change in all
|
||||
Python functions returning time to support nanosecond resolution using
|
||||
the ``decimal.Decimal`` type.
|
||||
|
||||
The PEP was rejected for different reasons:
|
||||
|
||||
* The idea of adding a new optional parameter to change the result type
|
||||
was rejected. It's an uncommon (and bad?) programming practice in
|
||||
Python.
|
||||
|
||||
* It was not clear if hardware clocks really had a resolution of 1
|
||||
nanosecond, especially at the Python level.
|
||||
|
||||
* The ``decimal.Decimal`` type is uncommon in Python and so requires
|
||||
to adapt code to handle it.
|
||||
|
||||
|
||||
CPython enhancements of the last 5 years
|
||||
----------------------------------------
|
||||
|
||||
Since the PEP 410 was rejected:
|
||||
|
||||
* The ``os.stat_result`` structure got 3 new fields for timestamps as
|
||||
nanoseconds (Python ``int``): ``st_atime_ns``, ``st_ctime_ns``
|
||||
and ``st_mtime_ns``.
|
||||
|
||||
* The PEP 418 was accepted, Python 3.3 got 3 new clocks:
|
||||
``time.monotonic()``, ``time.perf_counter()`` and
|
||||
``time.process_time()``.
|
||||
|
||||
* The CPython private "pytime" C API handling time now uses a new
|
||||
``_PyTime_t`` type: simple 64-bit signed integer (C ``int64_t``).
|
||||
The ``_PyTime_t`` unit is an implementation detail and not part of the
|
||||
API. The unit is currently ``1 nanosecond``.
|
||||
|
||||
Existing Python APIs using nanoseconds as int
|
||||
---------------------------------------------
|
||||
|
||||
The ``os.stat_result`` structure has 3 fields for timestamps as
|
||||
nanoseconds (``int``): ``st_atime_ns``, ``st_ctime_ns`` and
|
||||
``st_mtime_ns``.
|
||||
|
||||
The ``ns`` parameter of the ``os.utime()`` function accepts a
|
||||
``(atime_ns: int, mtime_ns: int)`` tuple: nanoseconds.
|
||||
|
||||
|
||||
Changes
|
||||
=======
|
||||
|
||||
New functions
|
||||
-------------
|
||||
|
||||
This PEP adds five new functions to the ``time`` module:
|
||||
|
||||
* ``time.clock_gettime_ns(clock_id)``
|
||||
* ``time.clock_settime_ns(clock_id, time: int)``
|
||||
* ``time.perf_counter_ns()``
|
||||
* ``time.monotonic_ns()``
|
||||
* ``time.time_ns()``
|
||||
|
||||
These functions are similar to the version without the ``_ns`` suffix,
|
||||
but use nanoseconds as Python ``int``.
|
||||
|
||||
For example, ``time.monotonic_ns() == int(time.monotonic() * 1e9)`` if
|
||||
``monotonic()`` value is small enough to not lose precision.
|
||||
|
||||
Unchanged functions
|
||||
-------------------
|
||||
|
||||
This PEP only proposed to add new functions getting or setting clocks
|
||||
with nanosecond resolution. Clocks are likely to lose precision,
|
||||
especially when their reference is the UNIX epoch.
|
||||
|
||||
Python has other functions handling time (get time, timeout, etc.), but
|
||||
no nanosecond variant is proposed for them since they are less likely to
|
||||
lose precision.
|
||||
|
||||
Example of unchanged functions:
|
||||
|
||||
* ``os`` module: ``sched_rr_get_interval()``, ``times()``, ``wait3()``
|
||||
and ``wait4()``
|
||||
|
||||
* ``resource`` module: ``ru_utime`` and ``ru_stime`` fields of
|
||||
``getrusage()``
|
||||
|
||||
* ``signal`` module: ``getitimer()``, ``setitimer()``
|
||||
|
||||
* ``time`` module: ``clock_getres()``
|
||||
|
||||
Since the ``time.clock()`` function was deprecated in Python 3.3, no
|
||||
``time.clock_ns()`` is added.
|
||||
|
||||
|
||||
Alternatives and discussion
|
||||
===========================
|
||||
|
||||
Sub-nanosecond resolution
|
||||
-------------------------
|
||||
|
||||
``time.time_ns()`` API is not "future-proof": if clocks resolutions
|
||||
increase, new Python functions may be needed.
|
||||
|
||||
In practive, the resolution of 1 nanosecond is currently enough for all
|
||||
structures used by all operating systems functions.
|
||||
|
||||
Hardware clock with a resolution better than 1 nanosecond already
|
||||
exists. For example, the frequency of a CPU TSC clock is the CPU base
|
||||
frequency: the resolution is around 0.3 ns for a CPU running at 3
|
||||
GHz. Users who have access to such hardware and really need
|
||||
sub-nanosecond resolution can easyly extend Python for their needs.
|
||||
Such rare use case don't justify to design the Python standard library
|
||||
to support sub-nanosecond resolution.
|
||||
|
||||
For the CPython implementation, nanosecond resolution is convenient: the
|
||||
standard and well supported ``int64_t`` type can be used to store time.
|
||||
It supports a time delta between -292 years and 292 years. Using the
|
||||
UNIX epoch as reference, this type supports time since year 1677 to year
|
||||
2262::
|
||||
|
||||
>>> 1970 - 2 ** 63 / (10 ** 9 * 3600 * 24 * 365.25)
|
||||
1677.728976954687
|
||||
>>> 1970 + 2 ** 63 / (10 ** 9 * 3600 * 24 * 365.25)
|
||||
2262.271023045313
|
||||
|
||||
Different types
|
||||
---------------
|
||||
|
||||
It was proposed to modify ``time.time()`` to use float type with better
|
||||
precision. The PEP 410 proposed to use ``decimal.Decimal``, but it was
|
||||
rejected. Apart ``decimal.Decimal``, no portable ``float`` type with
|
||||
better precision is currently available in Python. Changing the builtin
|
||||
Python ``float`` type is out of the scope of this PEP.
|
||||
|
||||
Other ideas of new types were proposed to support larger or arbitrary
|
||||
precision: fractions, structures or 2-tuple using integers,
|
||||
fixed-precision floating point number, etc.
|
||||
|
||||
See also the PEP 410 for a previous long discussion on other types.
|
||||
|
||||
Adding a new type requires more effort to support it, than reusing
|
||||
``int``. The standard library, third party code and applications would
|
||||
have to be modified to support it.
|
||||
|
||||
The Python ``int`` type is well known, well supported, ease to
|
||||
manipulate, and supports all arithmetic operations like:
|
||||
``dt = t2 - t1``.
|
||||
|
||||
Moreover, using nanoseconds as integer is not new in Python, it's
|
||||
already used for ``os.stat_result`` and
|
||||
``os.utime(ns=(atime_ns, mtime_ns))``.
|
||||
|
||||
.. note::
|
||||
If the Python ``float`` type becomes larger (ex: decimal128 or
|
||||
float128), the ``time.time()`` precision will increase as well.
|
||||
|
||||
Different API
|
||||
-------------
|
||||
|
||||
The ``time.time(ns=False)`` API was proposed to avoid adding new
|
||||
functions. It's an uncommon (and bad?) programming practice in Python to
|
||||
change the result type depending on a parameter.
|
||||
|
||||
Different options were proposed to allow the user to choose the time
|
||||
resolution. If each Python module uses a different resolution, it can
|
||||
become difficult to handle different resolutions, instead of just
|
||||
seconds (``time.time()`` returning ``float``) and nanoseconds
|
||||
(``time.time_ns()`` returning ``int``). Moreover, as written above,
|
||||
there is no need for resolution better than 1 nanosecond in practive in
|
||||
the Python standard library.
|
||||
|
||||
|
||||
Annex: Clocks Resolution in Python
|
||||
==================================
|
||||
|
||||
Script ot measure the smallest difference between two ``time.time()`` and
|
||||
``time.time_ns()`` reads ignoring differences of zero::
|
||||
|
||||
import math
|
||||
import time
|
||||
|
||||
LOOPS = 10 ** 6
|
||||
|
||||
print("time.time_ns(): %s" % time.time_ns())
|
||||
print("time.time(): %s" % time.time())
|
||||
|
||||
min_dt = [abs(time.time_ns() - time.time_ns())
|
||||
for _ in range(LOOPS)]
|
||||
min_dt = min(filter(bool, min_dt))
|
||||
print("min time_ns() delta: %s ns" % min_dt)
|
||||
|
||||
min_dt = [abs(time.time() - time.time())
|
||||
for _ in range(LOOPS)]
|
||||
min_dt = min(filter(bool, min_dt))
|
||||
print("min time() delta: %s ns" % math.ceil(min_dt * 1e9))
|
||||
|
||||
Results of time(), perf_counter() and monotonic().
|
||||
|
||||
Linux (kernel 4.12 on Fedora 26):
|
||||
|
||||
* time_ns(): **84 ns**
|
||||
* time(): **239 ns**
|
||||
* perf_counter_ns(): 84 ns
|
||||
* perf_counter(): 82 ns
|
||||
* monotonic_ns(): 84 ns
|
||||
* monotonic(): 81 ns
|
||||
|
||||
Windows 8.1:
|
||||
|
||||
* time_ns(): **318000 ns**
|
||||
* time(): **894070 ns**
|
||||
* perf_counter_ns(): 100 ns
|
||||
* perf_counter(): 100 ns
|
||||
* monotonic_ns(): 15000000 ns
|
||||
* monotonic(): 15000000 ns
|
||||
|
||||
The difference on ``time.time()`` is significant: **84 ns (2.8x better)
|
||||
vs 239 ns on Linux and 318 us (2.8x better) vs 894 us on Windows**. The
|
||||
difference (presion loss) will be larger next years since every day adds
|
||||
864,00,000,000,000 nanoseconds to the system clock.
|
||||
|
||||
The difference on ``time.perf_counter()`` and ``time.monotonic clock()``
|
||||
is not visible in this quick script since the script runs less than 1
|
||||
minute, and the uptime of the computer used to run the script was
|
||||
smaller than 1 week. A significant difference should be seen with an
|
||||
uptime of 104 days or greater.
|
||||
|
||||
.. note::
|
||||
Internally, Python starts ``monotonic()`` and ``perf_counter()``
|
||||
clocks at zero on some platforms which indirectly reduce the
|
||||
precision loss.
|
||||
|
||||
|
||||
|
||||
Copyright
|
||||
=========
|
||||
|
||||
This document has been placed in the public domain.
|
||||
Loading…
Reference in New Issue