From 6c5d998ce0e85a67df5615a168081ce76498b662 Mon Sep 17 00:00:00 2001 From: Chris Angelico Date: Thu, 22 Jan 2015 12:15:51 +1100 Subject: [PATCH] First public draft of new PEP 485 for an is_close_to() function --- pep-0485.txt | 311 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 311 insertions(+) create mode 100644 pep-0485.txt diff --git a/pep-0485.txt b/pep-0485.txt new file mode 100644 index 000000000..f1fa0c2db --- /dev/null +++ b/pep-0485.txt @@ -0,0 +1,311 @@ +PEP: 485 +Title: A Function for testing approximate equality +Version: $Revision$ +Last-Modified: $Date$ +Author: Christopher Barker +Status: Draft +Type: Standards Track +Content-Type: text/x-rst +Created: 20-Jan-2015 +Python-Version: 3.5 +Post-History: + + +Abstract +======== + +This PEP proposes the addition of a function to the standard library +that determines whether one value is approximately equal or "close" +to another value. + + +Rationale +========= + +Floating point values contain limited precision, which results in +their being unable to exactly represent some values, and for error to +accumulate with repeated computation. As a result, it is common +advice to only use an equality comparison in very specific situations. +Often a inequality comparison fits the bill, but there are times +(often in testing) where the programmer wants to determine whether a +computed value is "close" to an expected value, without requiring them +to be exactly equal. This is common enough, particularly in testing, +and not always obvious how to do it, so it would be useful addition to +the standard library. + + +Existing Implementations +------------------------ + +The standard library includes the +``unittest.TestCase.assertAlmostEqual`` method, but it: + +* Is buried in the unittest.TestCase class + +* Is an assertion, so you can't use it as a general test (easily) + +* Uses number of decimal digits or an absolute delta, which are + particular use cases that don't provide a general relative error. + +The numpy package has the ``allclose()`` and ``isclose()`` functions. + +The statistics package tests include an implementation, used for its +unit tests. + +One can also find discussion and sample implementations on Stack +Overflow, and other help sites. + +These existing implementations indicate that this is a common need, +and not trivial to write oneself, making it a candidate for the +standard library. + + +Proposed Implementation +======================= + +NOTE: this PEP is the result of an extended discussion on the +python-ideas list [1]_. + +The new function will have the following signature:: + + is_close_to(actual, expected, tol=1e-8, abs_tol=0.0) + +``actual``: is the value that has been computed, measured, etc. + +``expected``: is the "known" value. + +``tol``: is the relative tolerance -- it is the amount of error +allowed, relative to the magnitude of the expected value. + +``abs_tol``: is an minimum absolute tolerance level -- useful for +comparisons near zero. + +Modulo error checking, etc, the function will return the result of:: + + abs(expected-actual) <= max(tol*actual, abs_tol) + + +Handling of non-finite numbers +------------------------------ + +The IEEE 754 special values of NaN, inf, and -inf will be handled +according to IEEE rules. Specifically, NaN is not considered close to +any other value, including NaN. inf and -inf are only considered close +to themselves. + + +Non-float types +--------------- + +The primary use-case is expected to be floating point numbers. +However, users may want to compare other numeric types similarly. In +theory, it should work for any type that supports ``abs()``, +comparisons, and subtraction. The code will be written and tested to +accommodate these types: + + * ``Decimal`` + + * ``int`` + + * ``Fraction`` + + * ``complex``: for complex, ``abs(z)`` will be used for scaling and + comparison. + + +Behavior near zero +------------------ + +Relative comparison is problematic if either value is zero. In this +case, the difference is relative to zero, and thus will always be +smaller than the prescribed tolerance. To handle this case, an +optional parameter, ``abs_tol`` (default 0.0) can be used to set a +minimum tolerance to be used in the case of very small relative +tolerance. That is, the values will be considered close if:: + + abs(a-b) <= abs(tol*actual) or abs(a-b) <= abs_tol + +If the user sets the rel_tol parameter to 0.0, then only the absolute +tolerance will effect the result, so this function provides an +absolute tolerance check as well. + + +Relative Difference +=================== + +There are essentially two ways to think about how close two numbers +are to each-other: absolute difference: simple ``abs(a-b)``, and +relative difference: ``abs(a-b)/scale_factor`` [2]_. The absolute +difference is trivial enough that this proposal focuses on the +relative difference. + +Usually, the scale factor is some function of the values under +consideration, for instance: + + 1) The absolute value of one of the input values + + 2) The maximum absolute value of the two + + 3) The minimum absolute value of the two. + + 4) The arithmetic mean of the two + + +Symmetry +-------- + +A relative comparison can be either symmetric or non-symmetric. For a +symmetric algorithm: + +``is_close_to(a,b)`` is always equal to ``is_close_to(b,a)`` + +This is an appealing consistency -- it mirrors the symmetry of +equality, and is less likely to confuse people. However, often the +question at hand is: + +"Is this computed or measured value within some tolerance of a known +value?" + +In this case, the user wants the relative tolerance to be specifically +scaled against the known value. It is also easier for the user to +reason about. + +This proposal uses this asymmetric test to allow this specific +definition of relative tolerance. + +Example: + +For the question: "Is the value of a within x% of b?", Using b to +scale the percent error clearly defines the result. + +However, as this approach is not symmetric, a may be within 10% of b, +but b is not within x% of a. Consider the case:: + + a = 9.0 + b = 10.0 + +The difference between a and b is 1.0. 10% of a is 0.9, so b is not +within 10% of a. But 10% of b is 1.0, so a is within 10% of b. + +Casual users might reasonably expect that if a is close to b, then b +would also be close to a. However, in the common cases, the tolerance +is quite small and often poorly defined, i.e. 1e-8, defined to only +one significant figure, so the result will be very similar regardless +of the order of the values. And if the user does care about the +precise result, s/he can take care to always pass in the two +parameters in sorted order. + +This proposed implementation uses asymmetric criteria with the scaling +value clearly identified. + + +Expected Uses +============= + +The primary expected use case is various forms of testing -- "are the +results computed near what I expect as a result?" This sort of test +may or may not be part of a formal unit testing suite. + +The function might be used also to determine if a measured value is +within an expected value. + + +Inappropriate uses +------------------ + +One use case for floating point comparison is testing the accuracy of +a numerical algorithm. However, in this case, the numerical analyst +ideally would be doing careful error propagation analysis, and should +understand exactly what to test for. It is also likely that ULP (Unit +in the Last Place) comparison may be called for. While this function +may prove useful in such situations, It is not intended to be used in +that way. + + +Other Approaches +================ + +``unittest.TestCase.assertAlmostEqual`` +--------------------------------------- + +(https://docs.python.org/3/library/unittest.html#unittest.TestCase.assertAlmostEqual) + +Tests that values are approximately (or not approximately) equal by +computing the difference, rounding to the given number of decimal +places (default 7), and comparing to zero. + +This method was not selected for this proposal, as the use of decimal +digits is a specific, not generally useful or flexible test. + +numpy ``is_close()`` +-------------------- + +http://docs.scipy.org/doc/numpy-dev/reference/generated/numpy.isclose.html + +The numpy package provides the vectorized functions is_close() and +all_close, for similar use cases as this proposal: + +``isclose(a, b, rtol=1e-05, atol=1e-08, equal_nan=False)`` + + Returns a boolean array where two arrays are element-wise equal + within a tolerance. + + The tolerance values are positive, typically very small numbers. + The relative difference (rtol * abs(b)) and the absolute + difference atol are added together to compare against the + absolute difference between a and b + +In this approach, the absolute and relative tolerance are added +together, rather than the ``or`` method used in this proposal. This is +computationally more simple, and if relative tolerance is larger than +the absolute tolerance, then the addition will have no effect. But if +the absolute and relative tolerances are of similar magnitude, then +the allowed difference will be about twice as large as expected. + +Also, if the value passed in are small compared to the absolute +tolerance, then the relative tolerance will be completely swamped, +perhaps unexpectedly. + +This is why, in this proposal, the absolute tolerance defaults to zero +-- the user will be required to choose a value appropriate for the +values at hand. + + +Boost floating-point comparison +------------------------------- + +The Boost project ( [3]_ ) provides a floating point comparison +function. Is is a symetric approach, with both "weak" (larger of the +two relative errors) and "strong" (smaller of the two relative errors) +options. + +It was decided that a method that clearly defined which value was used +to scale the relative error would be more appropriate for the standard +library. + +References +========== + +.. [1] Python-ideas list discussion thread + (https://mail.python.org/pipermail/python-ideas/2015-January/030947.html) + +.. [2] Wikipedaia page on relative difference + (http://en.wikipedia.org/wiki/Relative_change_and_difference) + +.. [3] Boost project floating-point comparison algorithms + (http://www.boost.org/doc/libs/1_35_0/libs/test/doc/components/test_tools/floating_point_comparison.html) + + +Copyright +========= + +This document has been placed in the public domain. + + +.. + Local Variables: + mode: indented-text + indent-tabs-mode: nil + sentence-end-double-space: t + fill-column: 70 + coding: utf-8