PEP 698: Add `__override__` flag to runtime behavior (#2855)

We decided to add this for two reasons:

- It was specifically requested by the author of the `overrides`
  library, because there there are some handy runtime uses
  of override information (such as propagating docstrings, which
  we highlight as a concrete example)
- We realized that we actually added `__final__` to
  `@typing.final` in spite of it not being specified in PEP 591,
  which we felt strongly suggests that even if we omit it we
  would later change our minds.

This runtime behavior is currently implemented in typing_extensions
(see https://github.com/python/typing_extensions/pull/86) as well
as pyre_extensions.

pep-0698.rst

@@ -308,10 +308,45 @@ use it will type-check as before, without the additional type safety.
 Runtime Behavior
 ================
 
-At runtime, ``@typing.override`` will do nothing but return its argument.
+Set ``__override__ = True`` when possible
+-----------------------------------------
 
-We considered other options but rejected them because the downsides seemed to
-outweigh the benefits, see the Rejected Alternatives section.
+At runtime, ``@typing.override`` will make a best-effort attempt to add an
+attribute ``__override__`` with value ``True`` to its argument. By "best-effort"
+we mean that we will try adding the attribute, but if that fails (for example
+because the input is a descriptor type with fixed slots) we will silently
+return the argument as-is.
+
+This is exactly what the ``@typing.final`` decorator does, and the motivation
+is similar - it gives runtime libraries the ability to use ``@override``. As a
+concrete example, a runtime library could check ``__override__`` in order
+to automatically populate the ``__doc__`` attribute of child class methods
+using the parent method docstring.
+
+Limitations of setting ``__override__``
+---------------------------------------
+
+As described above, adding ``__override__`` may fail at runtime, in which
+case we will simply return the argument as-is.
+
+In addition, even in cases where it does work it may be difficult for users
+to correctly work with multiple decorators, because getting the ``__override__``
+field to exist on the final output requires understanding the implementation
+of each decorator:
+
+- The ``@override`` decorator needs to execute *after* ordinary decorators
+  like ``@functools.lru_cache`` that use wrapper functions, since we want to
+  set ``__override__`` on the outermost wrapper. This means it needs to
+  go *above* all these other decorators.
+- But ``@override`` needs to execute *before* many special descriptor-based
+  decorators like ``@property``, ``@staticmethod``, and ``@classmethod``.
+- As discussed above, in some cases (for example a descriptor with fixed
+  slots or a descriptor that also wraps) it may be impossible to get the
+  ``__override__`` attribute at all.
+
+As a result, runtime support for setting ``__override__`` is best effort
+only, and we do not expect type checkers to validate the ordering of
+decorators.
 
 
 Rejected Alternatives
@@ -360,53 +395,6 @@ We rejected this for three reasons:
   does) or to use a metaclass-based approach. Neither approach seems ideal.
 
 
-Marking overrides at runtime with an ``__override__`` attribute
----------------------------------------------------------------
-
-The ``@overrides.overrides`` decorator marks methods it decorates with an
-``__override__`` attribute.
-
-We considered having ``@typing.override`` do the same, since many typing
-features are made available at runtime for runtime libraries to use them. We
-decided against this because again the downsides seem to outweigh the benefits:
-
-Setting an attribute significantly complicates correct use of the decorator
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-If we have any runtime behavior at all in our decorator, we have to worry about
-the order of decorators.
-
-A decorator usually wraps a function in another function, and ``@override``
-would behave correctly if it were placed above all such decorators.
-
-But some decorators instead define descriptors - for example ``@classmethod``,
-``@staticmethod``, and ``@property`` all use descriptors. In these cases,
-placing ``@override`` below these decorators would work, but it would be
-possible for libraries to define decorators in ways where even that would not
-work.
-
-Moreover, we believe that it would be bad for most users - many of whom may not
-even understand descriptors - to be faced with a feature where correct use of
-``@override`` depends on placing it in between decorators that are implemented
-as wrapped functions and those that are implemented as
-
-We prefer to have no runtime behavior, which allows us to not care about the
-ordering and recommend, for style reasons, that ``@override`` always comes
-first.
-
-Lack of any clear benefit
-~~~~~~~~~~~~~~~~~~~~~~~~~
-
-We are not aware of any use for explicit marking of overrides other than the
-extra type safety it provides. This is in contrast to other typing features such
-as type annotations, which have important runtime uses such as metaprogramming
-and runtime type checking.
-
-In light of the downsides described above, we decided the benefits are
-insufficient to justify runtime behavior.
-
-
-
 Mark a base class to force explicit overrides on subclasses
 -----------------------------------------------------------