PEP 653: Change semantics to be closer to PEP 634, while keeping the important features of PEP 653. (#1868)

* Change name of `MATCH_CLASS` to `MATCH_POSITIONAL`.
* Drop `__attributes__` and use `__match_args__` instead.
  * For `MATCH_DEFAULT`, `__match_args__` acts a map from positions to names.
   * For `MATCH_POSITIONAL`, `__match_args__` acts a map from names to positions.
* Change semantics of `MATCH_DEFAULT` when match class patterns to those of PEP 634.
* Update translations.
* Further expand examples

pep-0653.rst

@@ -14,14 +14,16 @@ Abstract
 This PEP proposes a semantics for pattern matching that respects the general concept of PEP 634,
 but is more precise, easier to reason about, and should be faster.
 
-The object model will be extended with three special (dunder) attributes to support pattern matching:
+The object model will be extended with two special (dunder) attributes,
+in addition to the ``__match_args__`` attribute from PEP 634, to support pattern matching.
 
 * A ``__match_kind__`` attribute. Must be an integer.
-* An ``__attributes__`` attribute. Only needed for those classes wanting to customize matching the class pattern.
+* A ``__match_args__`` attribute. Only needed for those classes wanting to customize matching the class pattern.
   If present, it must be a tuple of strings.
-* A ``__deconstruct__()`` method. Only needed if ``__attributes__`` is present.
+* A ``__deconstruct__()`` method. Only needed for customizing matching of class patterns with positional arguments.
   Returns an iterable over the components of the deconstructed object.
 
+
 With this PEP:
 
 * The semantics of pattern matching will be clearer, so that patterns are easier to reason about.
@@ -107,7 +109,7 @@ This allows the kind of a value to be determined once and in a efficient fashion
 To deconstruct an object, pre-existing special methods can be used for sequence and mapping patterns, but something new is needed for class patterns.
 PEP 634 proposes using ad-hoc attribute access, disregarding the possibility of side-effects.
 This could be problematic should the attributes of the object be dynamically created or consume resources.
-By adding the ``__attributes__`` attribute and ``__deconstruct__()`` method, objects can control how they are deconstructed,
+By adding the ``__deconstruct__()`` method, objects can control how they are deconstructed,
 and patterns with a different set of attributes can be efficiently rejected.
 Should deconstruction of an object make no sense, then classes can define ``__match_kind__`` to reject class patterns completely.
 
@@ -131,7 +133,7 @@ bitwise ``or``\ ed with exactly one of these::
 
   0
   MATCH_DEFAULT
-  MATCH_CLASS
+  MATCH_POSITIONAL
   MATCH_SELF
 
 .. note::
@@ -139,19 +141,20 @@ bitwise ``or``\ ed with exactly one of these::
     Symbolic constants will be provided both for Python and C, and once defined they will
     never be changed.
 
-Classes inheriting from ``object`` will inherit ``__match_kind__ = MATCH_DEFAULT``.
+Classes inheriting from ``object`` will inherit ``__match_kind__ = MATCH_DEFAULT`` and ``__match_args__ = ()``
 
-Classes which define ``__match_kind__ & MATCH_CLASS`` to be non-zero must
+Classes which define ``__match_kind__ & MATCH_POSITIONAL`` to be non-zero must
-implement one additional special attribute, and one special method:
+implement ``__deconstruct__()`` and should consider redefining ``__match_args__``.
 
-* ``__attributes__``: should hold a tuple of strings indicating the names of attributes that are to be considered for matching; it may be empty for postional-only matches.
+* ``__match_args__``: should hold a tuple of strings indicating the names of attributes that are to be considered for matching; it may be empty for positional-only matches.
 * ``__deconstruct__()``: should return a sequence which contains the parts of the deconstructed object.
 
 .. note::
-    ``__attributes__`` and ``__deconstruct__`` will be automatically generated for dataclasses and named tuples.
+    ``__match_args__`` will be automatically generated for dataclasses, as specified in PEP 634.
+    ``__match_args__`` and ``__deconstruct__`` will be automatically generated for named tuples.
 
-The pattern matching implementation is *not* required to check that ``__attributes__`` and ``__deconstruct__`` behave as specified.
+The pattern matching implementation is *not* required to check that ``__match_args__`` and ``__deconstruct__`` behave as specified.
-If the value of ``__attributes__`` or the result of ``__deconstruct__()`` is not as specified, then
+If the value of ``__match_args__`` or the result of ``__deconstruct__()`` is not as specified, then
 the implementation may raise any exception, or match the wrong pattern.
 Of course, implementations are free to check these properties and provide meaningful error messages if they can do so efficiently.
 
@@ -339,8 +342,6 @@ Class pattern with no arguments::
 
 translates to::
 
-    if $kind & (MATCH_CLASS | MATCH_DEFAULT) == 0:
-        FAIL
     if not isinstance($value, ClsName):
         FAIL
 
@@ -364,46 +365,34 @@ Positional-only class pattern::
 
 translates to::
 
-    if $kind & MATCH_CLASS == 0:
-        FAIL
     if not isinstance($value, ClsName):
         FAIL
-    if $items is None:
+    if $kind & MATCH_POSITIONAL:
-        $items = type($value).__deconstruct__($value)
+        if $items is None:
-    # $VARS is a meta-variable.
+            $items = type($value).__deconstruct__($value)
-    if len($items) != len($VARS):
+        # $VARS is a meta-variable.
+        if len($items) < len($VARS):
+            FAIL
+        $VARS = $items
+    elif $kind & MATCH_DEFAULT:
+        if $attrs is None:
+            $attrs = type($value).__match_args__
+        if len($attr) < len($VARS):
+            FAIL
+        try:
+            for i, $VAR in enumerate($VARS):
+                $VAR = getattr($value, $attrs[i])
+        except AttributeError:
+            FAIL
+    else:
         FAIL
-    $VARS = $items
 
+Example: [6]_
 
 .. note::
 
-    ``__attributes__`` is not checked when matching positional-only class patterns,
+    ``__match_args__`` is not checked when matching positional-only class patterns,
-    this allows classes to match only positional-only patterns by setting ``__attributes__`` to ``()``.
+    this allows classes to match only positional-only patterns by leaving ``__match_args__`` set to the default value of ``()``.
-
-Class patterns with keyword patterns::
-
-    case ClsName($VARS, $KEYWORD_PATTERNS):
-
-translates to::
-
-    if $kind & MATCH_CLASS == 0:
-        FAIL
-    if not isinstance($value, ClsName):
-        FAIL
-    if $attrs is None:
-        $attrs = type($value).__attributes__
-    if $items is None:
-        $items = type($value).__deconstruct__($value)
-    $right_attrs = attrs[len($VARS):]
-    if not set($right_attrs) >= set($KEYWORD_PATTERNS):
-        FAIL
-    $VARS = items[:len($VARS)]
-    for $KEYWORD in $KEYWORD_PATTERNS:
-        $index = $attrs.index(QUOTE($KEYWORD))
-        $KEYWORD_PATTERNS[$KEYWORD] = $items[$index]
-
-Example: [6]_
 
 Class patterns with all keyword patterns::
 
@@ -411,22 +400,89 @@ Class patterns with all keyword patterns::
 
 translates to::
 
-    if $kind & MATCH_CLASS:
+    if not isinstance($value, ClsName):
-        As above with $VARS == ()
+        FAIL
+    if $kind & MATCH_POSITIONAL:
+        if $items is None:
+            $items = type($value).__deconstruct__($value)
+        if $attrs is None:
+            $attrs = type($value).__match_args__
+        $kwname_tuple = tuple(QUOTE($KEYWORD) for $KEYWORD in $KEYWORD_PATTERNS)
+        $indices = multi_index($attrs, $kwname_tuple, 0)
+        if $indices is None:
+            raise TypeError(...)
+        try:
+            for $KEYWORD, $index in zip($KEYWORD_PATTERNS, indices):
+                $KEYWORD_PATTERNS[$KEYWORD] = $items[$index]
+        except IndexError:
+            raise TypeError(...)
     elif $kind & MATCH_DEFAULT:
-        if not isinstance($value, ClsName):
+        try:
+            for $KEYWORD in $KEYWORD_PATTERNS:
+                $tmp = getattr($value, QUOTE($KEYWORD))
+                $KEYWORD_PATTERNS[$KEYWORD] = $tmp
+        except AttributeError:
             FAIL
-        if not hasattr($value, "__dict__"):
-            FAIL
-        if not $value.__dict__.keys() >= set($KEYWORD_PATTERNS):
-            FAIL
-        for $KEYWORD in $KEYWORD_PATTERNS:
-            $KEYWORD_PATTERNS[$KEYWORD] = $value.__dict__[QUOTE($KEYWORD)]
     else:
         FAIL
 
+Where the helper function ``multi_index(t, values, min)`` returns a tuple of indices of ``values`` into ``t``,
+or ``None`` if any value is not present in ``t`` or the index of the value is less than ``min``.
+
+Examples::
+
+    multi_index(("a", "b", "c"), ("a", "c"), 0) == (0,2)
+    multi_index(("a", "b", "c"), ("a", "c"), 1) is None
+    multi_index(("a", "b", "c"), ("a", "d"), 0) is None
+
 Example: [7]_
 
+Class patterns with positional and keyword patterns::
+
+    case ClsName($VARS, $KEYWORD_PATTERNS):
+
+translates to::
+
+    if not isinstance($value, ClsName):
+        FAIL
+    if $kind & MATCH_POSITIONAL:
+        if $items is None:
+            $items = type($value).__deconstruct__($value)
+        if $attrs is None:
+            $attrs = type($value).__match_args__
+        if len($items) < len($VARS):
+            FAIL
+        $VARS = $items[:len($VARS)]
+        $kwname_tuple = tuple(QUOTE($KEYWORD) for $KEYWORD in $KEYWORD_PATTERNS)
+        $indices = multi_index($attrs, $kwname_tuple, len($VARS))
+        if $indices is None:
+            raise TypeError(...)
+        try:
+            for $KEYWORD, $index in zip($KEYWORD_PATTERNS, indices):
+                $KEYWORD_PATTERNS[$KEYWORD] = $items[$index]
+        except IndexError:
+            raise TypeError(...)
+    elif $kind & MATCH_DEFAULT:
+        if $attrs is None:
+            $attrs = type($value).__match_args__
+        if len($attr) < len($VARS):
+            raise TypeError(...)
+        $positional_names = $attrs[:len($VARS)]
+        try:
+            for i, $VAR in enumerate($VARS):
+                $VAR = getattr($value, $attrs[i])
+            for $KEYWORD in $KEYWORD_PATTERNS:
+                $name = QUOTE($KEYWORD)
+                if $name in $positional_names:
+                    raise TypeError(...)
+                $KEYWORD_PATTERNS[$KEYWORD] = getattr($value, $name)
+        except AttributeError:
+            FAIL
+    else:
+        FAIL
+
+Example: [8]_
+
 
 Nested patterns
 '''''''''''''''
@@ -448,14 +504,8 @@ translates to::
         FAIL
     $value_0, $value_1 = $list
     #Now match on temporary values
-    $kind_0 = type($value_0).__match_kind__
-    if $kind_0 & (MATCH_CLASS | MATCH_DEFAULT) == 0:
-        FAIL
     if not isinstance($value_0, int):
         FAIL
-    $kind_1 = type($value_1).__match_kind__
-    if $kind_1 & (MATCH_CLASS | MATCH_DEFAULT) == 0:
-        FAIL
     if not isinstance($value_1, str):
         FAIL
 
@@ -480,12 +530,12 @@ All classes should ensure that the the value of ``__match_kind__`` follows the s
 Therefore, implementations can assume, without checking, that all the following are true::
 
     (__match_kind__ & (MATCH_SEQUENCE | MATCH_MAPPING)) != (MATCH_SEQUENCE | MATCH_MAPPING)
-    (__match_kind__ & (MATCH_SELF | MATCH_CLASS)) != (MATCH_SELF | MATCH_CLASS)
+    (__match_kind__ & (MATCH_SELF | MATCH_POSITIONAL)) != (MATCH_SELF | MATCH_POSITIONAL)
     (__match_kind__ & (MATCH_SELF | MATCH_DEFAULT)) != (MATCH_SELF | MATCH_DEFAULT)
-    (__match_kind__ & (MATCH_DEFAULT | MATCH_CLASS)) != (MATCH_DEFAULT | MATCH_CLASS)
+    (__match_kind__ & (MATCH_DEFAULT | MATCH_POSITIONAL)) != (MATCH_DEFAULT | MATCH_POSITIONAL)
 
 Thus, implementations can assume that ``__match_kind__ & MATCH_SEQUENCE`` implies ``(__match_kind__ & MATCH_MAPPING) == 0``, and vice-versa.
-Likewise for ``MATCH_SELF``, ``MATCH_CLASS`` and ``MATCH_DEFAULT``.
+Likewise for ``MATCH_SELF``, ``MATCH_POSITIONAL`` and ``MATCH_DEFAULT``.
 
 If ``__match_kind__`` does not follow the specification,
 then implementations may treat any of the expressions of the form ``$kind & MATCH_...`` above as having any value.
@@ -509,7 +559,7 @@ For common builtin classes ``__match_kind__`` will be:
 * ``tuple``: ``MATCH_SEQUENCE | MATCH_SELF``
 * ``dict``: ``MATCH_MAPPING | MATCH_SELF``
 
-Named tuples will have ``__match_kind__`` set to ``MATCH_SEQUENCE | MATCH_CLASS``.
+Named tuples will have ``__match_kind__`` set to ``MATCH_SEQUENCE | MATCH_POSITIONAL``.
 
 * All other standard library classes for which ``issubclass(cls, collections.abc.Mapping)`` is true will have ``__match_kind__`` set to ``MATCH_MAPPING``.
 * All other standard library classes for which ``issubclass(cls, collections.abc.Sequence)`` is true will have ``__match_kind__`` set to ``MATCH_SEQUENCE``.
@@ -530,7 +580,8 @@ to treat the following functions and methods as pure:
 * ``dict.__contains__()``
 * ``dict.__getitem__()``
 
-Implementations are also allowed to freely replace ``isinstance(obj, cls)`` with ``issubclass(type(obj), cls)`` and vice-versa.
+Implementations are allowed to freely replace ``isinstance(obj, cls)`` with ``issubclass(type(obj), cls)`` and vice-versa.
+Implementations are also allowed to elide repeated tests of ``isinstance(obj, cls)``.
 
 Security Implications
 =====================
@@ -662,19 +713,33 @@ The changes to the semantics can be summarized as:
 * Selecting the kind of pattern uses ``cls.__match_kind__`` instead of
   ``issubclass(cls, collections.abc.Mapping)`` and ``issubclass(cls, collections.abc.Sequence)``
   and allows classes control over which kinds of pattern they match.
-* Class matching is via the ``__attributes__`` attribute and ``__deconstruct__`` method,
+* Class matching is controlled by the ``__match_kind__`` attribute,
-  rather than the ``__match_args__`` method, and allows classes more control over how
+  and the ``__deconstruct__`` method allows classes more control over how they are deconstructed.
-  they are deconstructed.
+* The default behavior when matching a class pattern with keyword patterns is more precisely defined,
-* The default behavior when matching a class pattern with keyword patterns is changed.
+  but is broadly unchanged.
-  Only the instance dictionary is used. This is to avoid unintended capture of bound-methods.
 
-There are no changes to syntax.
+There are no changes to syntax. All examples given in the PEP 636 tutorial should continue to work as they do now.
 
 Rejected Ideas
 ==============
 
-None, as yet.
+An earlier version of this PEP only used attributes from the instance's dictionary when matching a class pattern with ``__match_kind__ == MATCH_DEFAULT``.
+The intent was to avoid capturing bound-methods and other synthetic attributes. However, this also mean that properties were ignored.
 
+For the class::
+
+    class C:
+        def __init__(self):
+            self.a = "a"
+        @property
+        def p(self):
+            ...
+        def m(self):
+            ...
+
+Ideally we would match the attributes "a" and "p", but not "m".
+However, there is no general way to do that, so this PEP now follows the semantics of PEP 634 for ``MATCH_DEFAULT``.
+Classes may override this behavior if needed by using ``__match_kind__ == MATCH_POSITIONAL`` or ``__match_args__``.
 
 Open Issues
 ===========
@@ -696,8 +761,7 @@ Code examples
 ::
 
     class Basic:
-        __match_kind__ = MATCH_CLASS
+        __match_kind__ = MATCH_POSITIONAL
-        __attributes__ = ()
         def __deconstruct__(self):
             return self._args
 
@@ -777,24 +841,30 @@ translates to::
 
 This::
 
-    match ClsName(x, a=y):
+    match ClsName(x, y):
 
 translates to::
 
-    if $kind & MATCH_CLASS == 0:
-        FAIL
     if not isinstance($value, ClsName):
         FAIL
-    if $attrs is None:
+    if $kind & MATCH_POSITIONAL:
-        $attrs = type($value).__attributes__
+        if $items is None:
-    if $items is None:
+            $items = type($value).__deconstruct__($value)
-        $items = type($value).__deconstruct__($value)
+        if len($items) < 2:
-    $right_attrs = $attrs[1:]
+            FAIL
-    if not "a" in $right_attrs:
+        x, y = $items
+    elif $kind & MATCH_DEFAULT:
+        if $attrs is None:
+            $attrs = type($value).__match_args__
+        if len($attr) < 2:
+            FAIL
+        try:
+            x = getattr($value, $attrs[0])
+            y = getattr($value, $attrs[1])
+        except AttributeError:
+            FAIL
+    else:
         FAIL
-    $y_index = $attrs.index("a")
-    x = $items[0]
-    y = $items[$y_index]
 
 .. [7]
 
@@ -804,36 +874,72 @@ This::
 
 translates to::
 
-    if $kind & MATCH_CLASS:
+    if not isinstance($value, ClsName):
-        if not isinstance($value, ClsName):
+        FAIL
-            FAIL
+    if $kind & MATCH_POSITIONAL:
-        if $attrs is None:
-            $attrs = type($value).__attributes__
         if $items is None:
             $items = type($value).__deconstruct__($value)
-        if not "a" in $attrs:
+        if $attrs is None:
-            FAIL
+            $attrs = type($value).__match_args__
-        if not "b" in $attrs:
+        $indices = multi_index($attrs, ("a", "b"), 0)
-            FAIL
+        if $indices is None:
-        $x_index = $attrs.index("a")
+            raise TypeError(...)
-        x = $items[$x_index]
+        try:
-        $y_index = $attrs.index("b")
+            x = $items[$indices[0]]
-        y = $items[$y_index]
+            y = $items[$indices[1]]
+        except IndexError:
+            raise TypeError(...)
     elif $kind & MATCH_DEFAULT:
-        if not isinstance($value, ClsName):
+        try:
+            x = $value.a
+            y = $value.b
+        except AttributeError:
             FAIL
-        if not hasattr($value, "__dict__"):
-            FAIL
-        $obj_dict = $value.__dict__
-        if not "a" in $attrs:
-            FAIL
-        if not "b" in $attrs:
-            FAIL
-        x = $obj_dict["a"]
-        y = $obj_dict["b"]
     else:
         FAIL
 
+.. [8]
+
+This::
+
+    match ClsName(x, a=y):
+
+translates to::
+
+
+    if not isinstance($value, ClsName):
+        FAIL
+    if $kind & MATCH_POSITIONAL:
+        if $items is None:
+            $items = type($value).__deconstruct__($value)
+        if $attrs is None:
+            $attrs = type($value).__match_args__
+        if len($items) < 1:
+            FAIL
+        x = $items[0]
+        $indices = multi_index($attrs, ("a",), 1)
+        if $indices is None:
+            raise TypeError(...)
+        $index = $indices[0]
+        try:
+            y = $items[$index]
+        except IndexError:
+            raise TypeError(...)
+    elif $kind & MATCH_DEFAULT:
+        if $attrs is None:
+            $attrs = type($value).__match_args__
+        if len($attr) < 1:
+            raise TypeError(...)
+        $positional_names = $attrs[:1]
+        try:
+            x = getattr($value, $attrs[0])
+            if "a" in $positional_names:
+                raise TypeError(...)
+            y = $value.a
+        except AttributeError:
+            FAIL
+    else:
+        FAIL
 
 Copyright
 =========