PEP: 245 Title: Python Interfaces Version: $Revision$ Author: michel@digicool.com (Michel Pelletier) Discussions-To: http://www.zope.org/Wikis/Interfaces Status: Draft Type: Standards Track Created: 11-Jan-2001 Python-Version: 2.2 Post-History: 21-Mar-2001 Introduction This PEP describes a Python interface model and a proposed syntax for creating interface objects in Python. Background In addition to thinking about adding a static type system to Python, the Types-SIG was also charged to devise an interface system for Python. In December of 1998, Jim Fulton released a prototype interfaces system based on discussions from the SIG. Many of the issues and background information on this discussion and prototype can be found in the SIG archives[1]. Around the end of 2000, Digital Creations began thinking about better component model designs for Zope[2]. Zope's future component model relies heavily on interface objects. This led to further development of Jim's "Scarecrow" interfaces prototype. Starting with version 2.3, Zope comes with an Interface package as standard software. Zope's Interface package is used as the reference implementation for this PEP. Overview This PEP proposes two additions to the Python language: - An interface model - A syntax for creating interfaces The extended syntax proposed by this PEP relies on syntax enhancements describe in PEP 232 [3] and describes an underlying framework which PEP 233 [4] could be based upon. There is some work being done with regard to interface objects and Proxy objects, so for those optional parts of this PEP you may want to see[5]. The Problem Interfaces are important because they solve a number of problems that arise while developing large systems with lots of developers: - Developers waste a lot of time looking at the source code of your system to figure out how objects work. - Developers who are new to your system may misunderstand how your objects work, causing, and possibly propagating, usage errors. - Because a lack of interfaces means usage is inferred from the source, developers may end up using methods and attributes that are meant for "internal use only". - Code inspection can be hard, and very discouraging to novice programmers trying to properly understand code written by gurus. - A lot of time is wasted when many people try very hard to understand obscurity (like undocumented software). Effort spend up front documenting interfaces will save much of this time in the end. Interfaces try to solve these problems by providing a way for you to describe how to use an object, a built-in mechanism for discovering that description, and a framework for defining well known Python interfaces. Python has very useful introspection features. It is well known that this makes exploring concepts in the interactive interpreter easier, because Python gives you the ability to look at all kinds of information about the objects: the type, doc strings, instance dictionaries, base classes, unbound methods and more. Many of these features are oriented toward using and changing implementations of software, and one of them ("doc strings") is oriented toward providing documentation. This proposal describes an extension to this natural introspection framework that describes an object's interface. Backward Compatibility The proposed interface model does not introduce any backward compatibility issues in Python. The proposed syntax, however, does. Any existing code that uses `interface' as an identifier will break. There may be other kinds of backwards incompatibility that defining `interface' as a new keyword will introduce. This extension to Python's syntax does not change any existing syntax in any backward incompatible way. The new `from __future__' Python syntax[6], and the new warning framework [7] is ideal for resolving this backward incompatibility. To use interface syntax now, a developer could use the statement: from __future__ import interfaces In addition, any code that uses the keyword `interface' as an identifier will be issued a warning from Python. After the appropriate period of time, the interface syntax would become standard, the above import assertion would do nothing, and any identifiers named `interface' would raise an exception. This period of time is proposed to be 24 months. Overview of the Interface Model Python Interfaces are objects that denote, describe, and document the behavior of an object, and can be associated with that object. An object that is associated with an interface is said to "implement" that interface. Since there is no real way, short of a unit test (and then some), to prove that an implementation is absolutely correct, the object is trusted to not be lying about what interfaces it implements. This trust, or contract, between the object and the developer is the interface. Whether or not an interface enforcement mechanism is in effect is discussed below. Interface objects describe the behavior of an object by containing useful information about the object. This information includes: - Prose documentation about the object. In Python terms, this is called the "doc string" of the interface. - Descriptions of attributes including the name of the attribute and prose documentation describing the attributes usage. - Descriptions of methods, including a name and prose documentation. They also include a description of the methods parameters. - Descriptions of parameters including the name and prose documentation. They can also describe properties about the parameter like its position, name, and default value (if any). - Optional tagged data. The four type of objects proposed in the interface model, Interface, Attribute, Method and Parameter, are referred to as "interface elements". Overview of the Interface Syntax This PEP proposes just one possible syntax for spelling interfaces in the Python language. This syntax will be used throughout the PEP to describe examples of using interfaces. For the most part, the syntax of interfaces is very much like the syntax of classes, but future needs, or needs brought up in discussion, may define new possibilities for interface syntax. The syntax proposed does not effect the proposed model. Here is an example of two different interfaces created with the proposed syntax: interface CountFishInterface: "Fish counting methods" def oneFish(): "Increments the fish count by one" def twoFish(): "Increments the fish count by two" def getFishCount(): "Returns the fish count" interface ColorFishInterface: "Fish coloring methods" def redFish(): "Sets the current fish color to red" def blueFish(): "Sets the current fish color to blue" def getFishColor(): "This returns the current fish color" This code, when evaluated, will create two interfaces called `CountFishInterface' and `ColorFishInterface'. These interfaces are defined by the `interface' statement. The prose documentation for the interfaces and their methods come from doc strings. The method signature information comes from the signatures of the `def' statements. Notice how there is no body for the def statements. The interface does not implement a service to anything; it merely describes one. You can also create interfaces that "extend" other interfaces. Here, you can see a new type of Interface that extends the CountFishInterface and ColorFishInterface: interface FishMarketInterface(CountFishInterface, ColorFishInterface): "This is the documentation for the FishMarketInterface" def getFishMonger(): "Returns the fish monger you can interact with" def hireNewFishMonger(name): "Hire a new fish monger" def buySomeFish(quantity=1): "Buy some fish at the market" The FishMarketInteface extends upon the CountFishInterface and ColorfishInterface. The Interface Model The interface for Interface objects can be expressed as an interface. For example, all Interface objects implement the `InterfaceBaseInterface': interface InterfaceBaseInterface: """ A base interface that defines a common Interface interface. """ def getName(): """ Returns the name of the current interface object. """ def getDoc(): """ Returns the documentation for the current interface object. """ def setTaggedValue(tag, value): """ Associates `value' to the object with `tag'. """ def getTaggedValue(tag): """ Returns the value associated with `tag'. """ def getTaggedValueTags(): """ Returns a list of all tags associated with this object. """ With this interface, you can ask an interface for its name or documentation and a way to tag the interface with special data (see the section Tagging Attributes for descriptions on using the Tag methods): >>> FishMarketInterface.getName() 'FishMarketInterface' >>> FishMarketInterface.getDoc() 'This is the documentation for the FishMarketInterface' >>> Note that Methods, Attributes and Parameters also implement the InterfaceBaseInterface, and their interfaces are described in a later section of this PEP. Interfaces can be asked about the interfaces they extend and what Methods and Attributes they define by using the `InterfaceInterface': interface InterfaceInterface(InterfaceBaseInterface): """ This is the interface to interface objects that are described in this document. """ def getBases(): """ Returns a sequence of base interfaces this interface extends. """ def extends(other): """ Does this interface extend the `other' interface? """ def isImplementedBy(object): """ Does the given object implement the interface? """ def isImplementedByInstancesOf(klass): """ Do instances of the given class implement this interface? """ def names(): """ Return the attribute names defined by the interface """ def namesAndDescriptions(): """ Return the attribute names and description objects defined by the interface """ def getDescriptionFor(name, default=None): """ Return the attribute description object for the given name """ def deferred(): """ Return a deferred class corresponding to the interface. A deferred class is a class built like the interface that can be subclassed by other classes. Calling any methods on a deferred class will raise a `NotImplemented' error. Sub-classes of deferred classes are therefor obliged to override deferred class methods. """ The InterfaceInterface provides a way for discovering methods. Here is an example of inspecting the methods of ColorFishInterface: >>> ColorFishInteface.names() ['redFish', 'blueFish', 'getFishColor'] >>> ColorFishInterface.namesAndDescriptions() [('redFish', ), ('blueFish', ), ('getFishColor', )] >>> ColorFishInteface.getDescriptionFor('getFishColor') >>> print ColorFishInterface.getDescriptionFor('getFishColor').getDoc() This returns the current fish color >>> Other method provide ways to determine base interfaces: >>> FishMarketInterface.getBases() (, ) >>> FishMarketInterface.extends(ColorFishInterface) 1 >>> FishMarketInterface.extends(CountFishInterface) 1 It's important to point out that interface definitions may look a lot like class definitions, but their semantics have a couple different behaviors. For example, when a class definition uses the following code: class Foo: def bar(self): pass "Foo.bar" returns you an unbound Python method. However, the similar looking interface construct: interface FooInterface: def bar(): pass FooInterface does not have the method attribute `bar'. Instead, bar is described by using the interface interface to find out the various attributes of the interface, like:: >>> FooInterface.names() ['bar'] >>> This is an intentional feature of the design. Sharing names with inheritance is sharing implementation. Interfaces can extend other interfaces, but they do not share the contractual responsibility of the interfaces they extend. Classes and Interfaces The example interfaces above do not describe any kind of behavior for their methods, they just describe an interface that a typical FishMarket object would realize. You may notice a similarity between interfaces extending from other interfaces and classes sub-classing from other classes. This is a similar concept. However it is important to note that interfaces extend interfaces and classes subclass classes. You cannot extend a class or subclass an interface. Classes and interfaces are separate. The purpose of a class is to share the implementation of how an object works. The purpose of an interface is to document how to work with an object, not how the object is implemented. It is possible to have several different classes with very different implementations realize the same interface. It's also possible to implement one interface with many classes that mix in pieces the functionality of the interface or, conversely, it's possible to have one class implement many interfaces. Because of this, interfaces and classes should not be confused or intermingled. Interface Assertion The next step is to put classes and interfaces together by creating a concrete Python class that asserts that it implements an interface. Here is an example FishMarket component that might do this: class FishError(Error): pass class FishMarket implements FishMarketInterface: number = 0 color = None monger_name = 'Crusty Barnacles' def __init__(self, number, color): self.number = number self.color = color def oneFish(self): self.number += 1 def twoFish(self): self.number += 2 def redFish(self): self.color = 'red' def blueFish(self): self.color = 'blue' def getFishCount(self): return self.number def getFishColor(self): return self.color def getFishMonger(self): return self.monger_name def hireNewFishMonger(self, name): self.monger_name = name def buySomeFish(self, quantity=1): if quantity > self.count: raise FishError("There's not enough fish") self.count -= quantity return quantity This new class, FishMarket defines a concrete class which implements the FishMarketInterface. The object following the `implements' statement is called an "interface assertion". An interface assertion can be either an interface object, or tuple of interface assertions. The interface assertion provided in a `class' statement like this is stored in the class's `__implements__' class attribute. After interpreting the above example, you would have a class statement that can be examined like this: >>> FishMarket >>> FishMarket.__implements__ (,) >>> f = FishMarket(6, 'red') >>> implements(f, FishMarketInterface) 1 >>> A class can realize more than one interface. For example, say you had an interface called `ItemInterface' that described how an object worked as an item in a container object. If you wanted to assert that FishMarket instances realized the ItemInterface interface as well as the FishMarketInterface, you can provide an interface assertion that contained a tuple of interface objects to the FishMarket class: class FishMarket implements FishMarketInterface, ItemInterface: # ... Interface assertions can also be used if you want to assert that one class implements an interface, and all of the interfaces that another class implements: class MyFishMarket implements FishMarketInterface, ItemInterface: # ... class YourFishMarket implements FooInterface, MyFishMarket.__implements__: # ... This new class YourFishMarket, asserts that it implements the FooInterface, as well as the interfaces implemented by the MyFishMarket class. It's worth going into a little bit more detail about interface assertions. An interface assertion is either an interface object, or a tuple of interface assertions. For example: FooInterface FooInterface, (BarInteface, BobInterface) FooInterface, (BarInterface, (BobInterface, MyClass.__implements__)) Are all valid interface assertions. When two interfaces define the same attributes, the order in which information is preferred in the assertion is from top-to-bottom, left-to-right. There are other interface proposals that, in the need for simplicity, have combined the notion of class and interface to provide simple interface enforcement. Interface objects have a `deferred' method that returns a deferred class that implements this behavior:: >>> FM = FishMarketInterface.deferred() >>> class MyFM(FM): pass >>> f = MyFM() >>> f.getFishMonger() Traceback (innermost last): File "", line 1, in ? Interface.Exceptions.BrokenImplementation: An object has failed to implement interface FishMarketInterface The getFishMonger attribute was not provided. >>> This provides for a bit of passive interface enforcement by telling you what you forgot to do to implement that interface. Tagging Interface Elements Interface elements can be tagged with optional data for application specific reasons. For example, in Zope an interface defines not only documentation and descriptions for methods, it also provides security assertions that associate methods with Zope's permission based security model. In Zope, you could define permissions on an interface by using the new syntax proposed by PEP-232 (Function Attributes [3]) and implemented in Python 2.1: from Zope.Security.Permissions import View interface MyZopeInterface: def foo(): """ This is the foo method """ foo.permission = View This is a Zope specific example, but it shows off the syntax. For your application, you many find other kinds of information to associate with interface attributes. Some possible use cases we came up with for tagged data are: - type - pre/post-conditions - unit tests - security assertions - examples - exception descriptions These are some suggestions; this proposal does not specify what tagged data can be used for. Interface-aware built-ins A useful extension to Python's list of built-in functions in the light of interface objects would be `implements()'. This builtin would expect two arguments, an object and an interface, and return a true value if the object implements the interface, false otherwise. For example: >>> interface FooInterface: pass >>> class Foo implements FooInterface: pass >>> f = Foo() >>> implements(f, FooInterface) 1 Currently, this functionality exists in the reference implementation as functions in the `Interface' package, requiring an "import Interface" to use it. Its existence as a built-in would be purely for a convenience, and not necessary for using interfaces, and analogous to `isinstance()' for classes. Interface Enforcement and Type Checking Interface enforcement could be useful for many different purposes. For example, during the building/testing phase of software development, enforcement could be turned on during unit testing to ensure that interfaces are being used properly. When the software is ready to ship in "production", this enforcement could be turned off. We have implemented a simple interface enforcement system with mxProxy [5] that can be turned on and off to debug implementations. Itamar Shtull-Trauring has also implemented a precondition/postcondition enforcer based on Zope's ExtensionClass and Interfaces. This package can be found along with the reference implementation. This package allows you to turn on debugging assertions when methods on an object are called. Here's is an example from Itamar's implementation: from Interface import Base, EnforcedInterface class IDummy(Base): """ Dummy interface """ def foo(self, n, s): """ Return n * s, where n is an integer, and s is a string """ def bar(self, x): """ Return 1 """ EnforcedInterface.setPrecondition( "foo", IDummy, "n > 0 and type(s) == type('string')") EnforcedInterface.setPostcondition( "foo", IDummy, "type(result) == type('string')") EnforcedInterface.setPostcondition("bar", IDummy, "result == 1") This is Itamar's prototype implementation based on Zope's ExtentionClass and Interface package. This code illustrates how it is easy to associate a type qualifying assertion as a precondition or post condition on an interface object. Using the new 2.1 function attribute syntax, this same concept can be expressed as an interface:: interface IDummy: """ Dummy interface """ def foo(n, s): """ Return n * s, where n is an integer, and s is a string """ foo.precondition = "n > 0 and type(s) == type('string')" foo.postcondition = "type(result) == type('string')" def bar(x): """ Return 1 """ bar.postcondition = "result == 1" Now, this interface is tagged with data that a class wrapper based on Extension Class might want to play with, like Itamar's implementation. Standard Python Protocols There is another possible use for interfaces with type checking: often, a programmer does not want to specify that a parameter must be of a certain type, but that the parameter must implement a specific interface. There are a number of standard, but informal, interfaces in Python, often referred to as protocols. For example, it is possible to think of "sequence" and "mapping" protocols as interfaces, and do type checking with them:: interface MappingInterface: """Anything supporting __getitem__.""" def __getitem__(key): """Return an item from the mapping.""" interface SequenceInterface(MappingInterface): """Keys must be integers in a sequence starting at 0.""" This gives the programmer similar flexibility in defining expected types as that which is proposed by other type checking proposals. The current reference implementation defines the following Python protocol definitions as interfaces: interface MappingInterface: def __getitem__(key): """ Get the item that maps to the key """ interface SequentialInterface: """ Keys must be used in order """ interface SizedInterface: def __len__(self): """ Returns the length of the object """ interface MutableInterface: """ Object can be changed in place """ interface ComparableInterface: """ Objects that can be tested for equality """ interface HashableInterface: """ Objects that support hash """ interface SequenceInterface(MappingInterface, SequentialInterface): """Keys must be integers in a sequence starting at 0.""" interface OrderableInterface(ComparableInterface): """Objects that can be compared with < > == <= >= !=""" interface HashKeyInterface(ComparableInterface, HashableInterface): """Objects that are immutable with respect to state that influences comparisons or hash values""" There are many Python protocols to define in Python; this is by no means an exhaustive list. The effort of defining protocol interfaces for the entire Python language is not too extensive. But the work load to define and provide interface assertions for the entire standard library distribution would be a fairly enormous task. Type Assertion This PEP so far only explains how to associate interfaces with classes and their instances, but not with Python types. Since types cannot be assigned attributes, the Interface package maintains a registry of types and the interfaces they assert to implement. This is done with a type assertion: from Interface import Attribute, assertTypeImplements import types interface FunctionInterface: __doc__ = Attribute("documentation string") __name__ = Attribute("name with which this function was defined") func_code = Attribute("Code object containing compiled function bytecode") func_defaults = Attribute("tuple of any default values for arguments") func_doc = Attribute("same as __doc__") func_globals = Attribute("global namespace in which this function was defined") func_name = Attribute("same as __name__") assertTypeImplements(types.FunctionType, FunctionInterface) The Interface package makes a number of assertions like this, based on the properties of Python types defined in Ka-Ping Yee's `inspect.py' module. The Interface package defines and asserts the following type interfaces (for complete descriptions, including Attribute definitions, see the reference implementation): - DocumentedInterface (Abstract interface that provides __doc__) - NamedInterface (Abstract interface that provides __name___ - FunctionInterface - TracebackInterface - FrameInterface - LambdaInterface - CodeInterface - ModuleInterface(DocumentedInterface) - ClassInterface(DocumentedInterface) - MethodInterface(DocumentedInterface, NamedInterface) - BuiltinInterface(DocumentedInterface, NamedInterface) - RoutineInterface(FunctionInterface, LambdaInterface, MethodInterface, BuiltinInterface) The Interface package maintains a registry of common Python types and the above interfaces that they "assert". The framework also provides an API function for registering new types called `assertTypeImplements'. There are other Python types that come with Python that are not defined yet, like file and other C written file like types. Creating Interfaces Using the `interface' and `implements' syntax allows you to create complex interface objects. The `Interface' module provides you with some features that the Python syntax does not let you create, and provides some standard interface tools to make using interfaces more convenient. Interfaces can be created by calling Interface.new(). New supports the following interface: def __init__(name, bases=(), attrs=None, doc=None): """ Create a new interface object. `Name' is the name of the interface and the only required attribute. `bases' is either an interface or a tuple. A sequence of pre-built attribute objects can be passed to `attrs'. This could be useful for building interface objects from modeling tools or other generation tools. Last, documentation the interface in general can be passed to `doc'. A `__doc__' object can also be provided in the `attrs' sequence. """ Creating Attributes Attributes can be created with Interface.Attribute(). The Attribute constructor implements the following interface: interface AttributeInterface(InterfaceBaseInterface): def __init__(name=None, doc=None): """ Create a new attribute object. The object can be named and described with name and doc. """ Creating Methods Interface Methods can be created by calling Interface.Method(). Method objects have the same constructor arguments as Attribute objects. In addition, they provide methods for return parameter descriptions and other information: interface MethodInterface(InterfaceBaseInterface, AttributeInterface): def fromConcreteMethod(m): """ Introspects the python method `m' and creates a Method object based on it. It can be either a bound method, an unbound method, or a function """ def getParameters(): """ Returns a sequence of parameter objects that describe the individual parameters to a method call """ def getSignatureString(): """ Returns a valid signature string that describes, in python, the parameter interface to a method """ Parameter Objects Parameter objects can be created with Interface.Parameter(): interface ParameterInterface(InterfaceBaseInterface): def __init__(name, type=None, default=None, __doc__=None): """ Build a Parameter object with `name'. A `type' can be provided and must be an interface, an interface tuple, a python type, a python class, or None. If `default' is provided, the default value is remembered and the method is flagged as optional. Documentation can be provided with `__doc__'. """ def default(): """ Default value. """ def isOptional(): """ a flag indicating if it is optional """ Reference Implementation Zope's prototype system is distributed with Zope versions 2.3a1 and higher. The project Wiki also contains a stand-alone distribution at[8]. At the object level, there are very few differences between this PEP and the software distributed with Zope. With the current model, Interfaces are declared with the following syntax: import Interface class CountFishInterface(Interface.Base): "Fish counting methods" def oneFish(self): "Increments the fish count by one" def twoFish(self): "Increments the fish count by two" def getFishCount(self): "Returns the fish count" This creates an Interface object called `CountFishInteface' by overloading the Python `class' statement. This PEP proposes adding a new keyword `interface' which allows the following, equivalent declaration in Python:: interface CountFishInterface: "Fish counting methods" def oneFish(): "Increments the fish count by one" def twoFish(): "Increments the fish count by two" def getFishCount(): "Returns the fish count" This syntax is cleaner and the intent is more clear. This syntax has the added benefit of not overloading the concepts behind `class'. Notice that interface methods do not define a `self' parameter. This proposal also suggests a way to associate Python classes with the interfaces they implement by adding the `implements' keyword, like this: class Foo implements CountFishInterface: """ The foo interface """ In the current model, this is done by assigning a class attribute `__implements__': class Foo: """ The foo interface """ __implements__ = CountFishInterface Because this is done with a class attribute, inheritance rules can end up asserting interfaces for classes that may not want to be signed up for those contracts. Use Cases A complete set of use cases is collected in the Wiki[9]. Additional use cases should be suggested to the author or entered into the Wiki. Other Uses Component Model The next generation of Zope will use interfaces heavily in its component model. Zope's definition of a component is, in fact, "an object with an interface". The Zope Component model defines a base set of interfaces (including a component interface). These base interfaces define how you work with Zope components, how Zope components are accessed by other components, and how components are access from the web and other mediums. Online Help System Zope's online help system uses interface objects to derive the API and prose online help information from objects. Other Python applications, like text editors (think IDLE) could be extended to introspect interfaces and provide easy context sensitive help. Summary of Proposed Changes to Python Adding new `interface' keyword and extending class syntax with `implements'. Turning existing Interfaces Python package into a standard Python type that provides objects that implement the InterfaceInterface, described in this PEP. Extending class interface to include __implements__. Risks This PEP proposes adding one new keyword to the Python language, `interface'. This will break code. Open Issues Goals Syntax Architecture Dissenting Opinion This PEP has not yet been discussed on python-dev. References [1] http://mail.python.org/pipermail/types-sig/1998-December/date.html [2] http://www.zope.org [3] PEP 232, Function Attributes, Warsaw http://python.sourceforge.net/peps/pep-0232.html [4] PEP 233, Python Online Help, Prescod http://python.sourceforge.net/peps/pep-0233.html [5] http://www.lemburg.com/files/python/mxProxy.html [6] PEP 236, Back to the __future__, Peters http://python.sourceforge.net/peps/pep-0236.html [7] PEP 230, Warning Framework, van Rossum http://python.sourceforge.net/peps/pep-0236.html [8] http://www.zope.org/Wikis/Interfaces [9] http://www.zope.org/Wikis/Interfaces/InterfacesUseCases Copyright This document has been placed in the public domain. Local Variables: mode: indented-text indent-tabs-mode: nil End: