New PEP 564: Add new time functions with nanosecond resolution

pep-0564.rst

@@ -0,0 +1,302 @@
+PEP: 564
+Title: Add new time functions with nanosecond resolution
+Version: $Revision$
+Last-Modified: $Date$
+Author: Victor Stinner <victor.stinner@gmail.com>
+Status: Draft
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 16-October-2017
+Python-Version: 3.7
+
+
+Abstract
+========
+
+Add five new functions to the ``time`` module: ``time_ns()``,
+``perf_counter_ns()``, ``monotonic_ns()``, ``clock_gettime_ns()`` and
+``clock_settime_ns()``. They are similar to the function without the
+``_ns`` suffix, but have nanosecond resolution: use a number of
+nanoseconds as a Python int.
+
+The best ``time.time_ns()`` resolution measured in Python is 3 times
+better then ``time.time()`` resolution on Linux and Windows.
+
+
+Rationale
+=========
+
+Float type limited to 104 days
+------------------------------
+
+The clocks resolution of desktop and latop computers is getting closer
+to nanosecond resolution. More and more clocks have a frequency in MHz,
+up to GHz for the CPU TSC clock.
+
+The Python ``time.time()`` function returns the current time as a
+floatting point number which is usually a 64-bit binary floatting number
+(in the IEEE 754 format).
+
+The problem is that the float type starts to lose nanoseconds after 104
+days.  Conversion from nanoseconds (``int``) to seconds (``float``) and
+then back to nanoseconds (``int``) to check if conversions lose
+precision::
+
+    # no precision loss
+    >>> x = 2 ** 52 + 1; int(float(x * 1e-9) * 1e9) - x
+    0
+    # precision loss! (1 nanosecond)
+    >>> x = 2 ** 53 + 1; int(float(x * 1e-9) * 1e9) - x
+    -1
+    >>> print(datetime.timedelta(seconds=2 ** 53 / 1e9))
+    104 days, 5:59:59.254741
+
+``time.time()`` returns seconds elapsed since the UNIX epoch: January
+1st, 1970. This function loses precision since May 1970 (47 years ago)::
+
+    >>> import datetime
+    >>> unix_epoch = datetime.datetime(1970, 1, 1)
+    >>> print(unix_epoch + datetime.timedelta(seconds=2**53 / 1e9))
+    1970-04-15 05:59:59.254741
+
+
+Previous rejected PEP
+---------------------
+
+Five years ago, the PEP 410 proposed a large and complex change in all
+Python functions returning time to support nanosecond resolution using
+the ``decimal.Decimal`` type.
+
+The PEP was rejected for different reasons:
+
+* The idea of adding a new optional parameter to change the result type
+  was rejected. It's an uncommon (and bad?) programming practice in
+  Python.
+
+* It was not clear if hardware clocks really had a resolution of 1
+  nanosecond, especially at the Python level.
+
+* The ``decimal.Decimal`` type is uncommon in Python and so requires
+  to adapt code to handle it.
+
+
+CPython enhancements of the last 5 years
+----------------------------------------
+
+Since the PEP 410 was rejected:
+
+* The ``os.stat_result`` structure got 3 new fields for timestamps as
+  nanoseconds (Python ``int``): ``st_atime_ns``, ``st_ctime_ns``
+  and ``st_mtime_ns``.
+
+* The PEP 418 was accepted, Python 3.3 got 3 new clocks:
+  ``time.monotonic()``, ``time.perf_counter()`` and
+  ``time.process_time()``.
+
+* The CPython private "pytime" C API handling time now uses a new
+  ``_PyTime_t`` type: simple 64-bit signed integer (C ``int64_t``).
+  The ``_PyTime_t`` unit is an implementation detail and not part of the
+  API. The unit is currently ``1 nanosecond``.
+
+Existing Python APIs using nanoseconds as int
+---------------------------------------------
+
+The ``os.stat_result`` structure has 3 fields for timestamps as
+nanoseconds (``int``): ``st_atime_ns``, ``st_ctime_ns`` and
+``st_mtime_ns``.
+
+The ``ns`` parameter of the ``os.utime()`` function accepts a
+``(atime_ns: int, mtime_ns: int)`` tuple: nanoseconds.
+
+
+Changes
+=======
+
+New functions
+-------------
+
+This PEP adds five new functions to the ``time`` module:
+
+* ``time.clock_gettime_ns(clock_id)``
+* ``time.clock_settime_ns(clock_id, time: int)``
+* ``time.perf_counter_ns()``
+* ``time.monotonic_ns()``
+* ``time.time_ns()``
+
+These functions are similar to the version without the ``_ns`` suffix,
+but use nanoseconds as Python ``int``.
+
+For example, ``time.monotonic_ns() == int(time.monotonic() * 1e9)`` if
+``monotonic()`` value is small enough to not lose precision.
+
+Unchanged functions
+-------------------
+
+This PEP only proposed to add new functions getting or setting clocks
+with nanosecond resolution. Clocks are likely to lose precision,
+especially when their reference is the UNIX epoch.
+
+Python has other functions handling time (get time, timeout, etc.), but
+no nanosecond variant is proposed for them since they are less likely to
+lose precision.
+
+Example of unchanged functions:
+
+* ``os`` module: ``sched_rr_get_interval()``, ``times()``, ``wait3()``
+  and ``wait4()``
+
+* ``resource`` module: ``ru_utime`` and ``ru_stime`` fields of
+  ``getrusage()``
+
+* ``signal`` module: ``getitimer()``, ``setitimer()``
+
+* ``time`` module: ``clock_getres()``
+
+Since the ``time.clock()`` function was deprecated in Python 3.3, no
+``time.clock_ns()`` is added.
+
+
+Alternatives and discussion
+===========================
+
+Sub-nanosecond resolution
+-------------------------
+
+``time.time_ns()`` API is not "future-proof": if clocks resolutions
+increase, new Python functions may be needed.
+
+In practive, the resolution of 1 nanosecond is currently enough for all
+structures used by all operating systems functions.
+
+Hardware clock with a resolution better than 1 nanosecond already
+exists. For example, the frequency of a CPU TSC clock is the CPU base
+frequency: the resolution is around 0.3 ns for a CPU running at 3
+GHz. Users who have access to such hardware and really need
+sub-nanosecond resolution can easyly extend Python for their needs.
+Such rare use case don't justify to design the Python standard library
+to support sub-nanosecond resolution.
+
+For the CPython implementation, nanosecond resolution is convenient: the
+standard and well supported ``int64_t`` type can be used to store time.
+It supports a time delta between -292 years and 292 years. Using the
+UNIX epoch as reference, this type supports time since year 1677 to year
+2262::
+
+    >>> 1970 - 2 ** 63 / (10 ** 9 * 3600 * 24 * 365.25)
+    1677.728976954687
+    >>> 1970 + 2 ** 63 / (10 ** 9 * 3600 * 24 * 365.25)
+    2262.271023045313
+
+Different types
+---------------
+
+It was proposed to modify ``time.time()`` to use float type with better
+precision. The PEP 410 proposed to use ``decimal.Decimal``, but it was
+rejected. Apart ``decimal.Decimal``, no portable ``float`` type with
+better precision is currently available in Python. Changing the builtin
+Python ``float`` type is out of the scope of this PEP.
+
+Other ideas of new types were proposed to support larger or arbitrary
+precision: fractions, structures or 2-tuple using integers,
+fixed-precision floating point number, etc.
+
+See also the PEP 410 for a previous long discussion on other types.
+
+Adding a new type requires more effort to support it, than reusing
+``int``. The standard library, third party code and applications would
+have to be modified to support it.
+
+The Python ``int`` type is well known, well supported, ease to
+manipulate, and supports all arithmetic operations like:
+``dt = t2 - t1``.
+
+Moreover, using nanoseconds as integer is not new in Python, it's
+already used for ``os.stat_result`` and
+``os.utime(ns=(atime_ns, mtime_ns))``.
+
+.. note::
+   If the Python ``float`` type becomes larger (ex: decimal128 or
+   float128), the ``time.time()`` precision will increase as well.
+
+Different API
+-------------
+
+The ``time.time(ns=False)`` API was proposed to avoid adding new
+functions. It's an uncommon (and bad?) programming practice in Python to
+change the result type depending on a parameter.
+
+Different options were proposed to allow the user to choose the time
+resolution. If each Python module uses a different resolution, it can
+become difficult to handle different resolutions, instead of just
+seconds (``time.time()`` returning ``float``) and nanoseconds
+(``time.time_ns()`` returning ``int``). Moreover, as written above,
+there is no need for resolution better than 1 nanosecond in practive in
+the Python standard library.
+
+
+Annex: Clocks Resolution in Python
+==================================
+
+Script ot measure the smallest difference between two ``time.time()`` and
+``time.time_ns()`` reads ignoring differences of zero::
+
+    import math
+    import time
+
+    LOOPS = 10 ** 6
+
+    print("time.time_ns(): %s" % time.time_ns())
+    print("time.time(): %s" % time.time())
+
+    min_dt = [abs(time.time_ns() - time.time_ns())
+              for _ in range(LOOPS)]
+    min_dt = min(filter(bool, min_dt))
+    print("min time_ns() delta: %s ns" % min_dt)
+
+    min_dt = [abs(time.time() - time.time())
+              for _ in range(LOOPS)]
+    min_dt = min(filter(bool, min_dt))
+    print("min time() delta: %s ns" % math.ceil(min_dt * 1e9))
+
+Results of time(), perf_counter() and monotonic().
+
+Linux (kernel 4.12 on Fedora 26):
+
+* time_ns(): **84 ns**
+* time(): **239 ns**
+* perf_counter_ns(): 84 ns
+* perf_counter(): 82 ns
+* monotonic_ns(): 84 ns
+* monotonic(): 81 ns
+
+Windows 8.1:
+
+* time_ns(): **318000 ns**
+* time(): **894070 ns**
+* perf_counter_ns(): 100 ns
+* perf_counter(): 100 ns
+* monotonic_ns(): 15000000 ns
+* monotonic(): 15000000 ns
+
+The difference on ``time.time()`` is significant: **84 ns (2.8x better)
+vs 239 ns on Linux and 318 us (2.8x better) vs 894 us on Windows**. The
+difference (presion loss) will be larger next years since every day adds
+864,00,000,000,000 nanoseconds to the system clock.
+
+The difference on ``time.perf_counter()`` and ``time.monotonic clock()``
+is not visible in this quick script since the script runs less than 1
+minute, and the uptime of the computer used to run the script was
+smaller than 1 week. A significant difference should be seen with an
+uptime of 104 days or greater.
+
+.. note::
+   Internally, Python starts ``monotonic()`` and ``perf_counter()``
+   clocks at zero on some platforms which indirectly reduce the
+   precision loss.
+
+
+
+Copyright
+=========
+
+This document has been placed in the public domain.