reSTify PEP 205 (#369)
This commit is contained in:
parent
a49551d35b
commit
d80d5be223
683
pep-0205.txt
683
pep-0205.txt
|
@ -5,444 +5,461 @@ Last-Modified: $Date$
|
||||||
Author: Fred L. Drake, Jr. <fdrake@acm.org>
|
Author: Fred L. Drake, Jr. <fdrake@acm.org>
|
||||||
Status: Final
|
Status: Final
|
||||||
Type: Standards Track
|
Type: Standards Track
|
||||||
|
Content-Type: text/x-rst
|
||||||
Created:
|
Created:
|
||||||
Python-Version: 2.1
|
Python-Version: 2.1
|
||||||
Post-History: 11-Jan-2001
|
Post-History: 11-Jan-2001
|
||||||
|
|
||||||
Motivation
|
Motivation
|
||||||
|
==========
|
||||||
|
|
||||||
There are two basic applications for weak references which have
|
There are two basic applications for weak references which have
|
||||||
been noted by Python programmers: object caches and reduction of
|
been noted by Python programmers: object caches and reduction of
|
||||||
pain from circular references.
|
pain from circular references.
|
||||||
|
|
||||||
Caches (weak dictionaries)
|
Caches (weak dictionaries)
|
||||||
|
--------------------------
|
||||||
|
|
||||||
There is a need to allow objects to be maintained that represent
|
There is a need to allow objects to be maintained that represent
|
||||||
external state, mapping a single instance to the external
|
external state, mapping a single instance to the external
|
||||||
reality, where allowing multiple instances to be mapped to the
|
reality, where allowing multiple instances to be mapped to the
|
||||||
same external resource would create unnecessary difficulty
|
same external resource would create unnecessary difficulty
|
||||||
maintaining synchronization among instances. In these cases,
|
maintaining synchronization among instances. In these cases,
|
||||||
a common idiom is to support a cache of instances; a factory
|
a common idiom is to support a cache of instances; a factory
|
||||||
function is used to return either a new or existing instance.
|
function is used to return either a new or existing instance.
|
||||||
|
|
||||||
The difficulty in this approach is that one of two things must
|
The difficulty in this approach is that one of two things must
|
||||||
be tolerated: either the cache grows without bound, or there
|
be tolerated: either the cache grows without bound, or there
|
||||||
needs to be explicit management of the cache elsewhere in the
|
needs to be explicit management of the cache elsewhere in the
|
||||||
application. The later can be very tedious and leads to more
|
application. The later can be very tedious and leads to more
|
||||||
code than is really necessary to solve the problem at hand,
|
code than is really necessary to solve the problem at hand,
|
||||||
and the former can be unacceptable for long-running processes
|
and the former can be unacceptable for long-running processes
|
||||||
or even relatively short processes with substantial memory
|
or even relatively short processes with substantial memory
|
||||||
requirements.
|
requirements.
|
||||||
|
|
||||||
- External objects that need to be represented by a single
|
- External objects that need to be represented by a single
|
||||||
instance, no matter how many internal users there are. This
|
instance, no matter how many internal users there are. This
|
||||||
can be useful for representing files that need to be written
|
can be useful for representing files that need to be written
|
||||||
back to disk in whole rather than locked & modified for
|
back to disk in whole rather than locked & modified for
|
||||||
every use.
|
every use.
|
||||||
|
|
||||||
- Objects that are expensive to create, but may be needed by
|
- Objects that are expensive to create, but may be needed by
|
||||||
multiple internal consumers. Similar to the first case, but
|
multiple internal consumers. Similar to the first case, but
|
||||||
not necessarily bound to external resources, and possibly
|
not necessarily bound to external resources, and possibly
|
||||||
not an issue for shared state. Weak references are only
|
not an issue for shared state. Weak references are only
|
||||||
useful in this case if there is some flavor of "soft"
|
useful in this case if there is some flavor of "soft"
|
||||||
references or if there is a high likelihood that users of
|
references or if there is a high likelihood that users of
|
||||||
individual objects will overlap in lifespan.
|
individual objects will overlap in lifespan.
|
||||||
|
|
||||||
Circular references
|
Circular references
|
||||||
|
-------------------
|
||||||
|
|
||||||
- DOMs require a huge amount of circular (to parent & document
|
- DOMs require a huge amount of circular (to parent & document
|
||||||
nodes) references, but these could be eliminated using a weak
|
nodes) references, but these could be eliminated using a weak
|
||||||
dictionary mapping from each node to its parent. This
|
dictionary mapping from each node to its parent. This
|
||||||
might be especially useful in the context of something like
|
might be especially useful in the context of something like
|
||||||
xml.dom.pulldom, allowing the .unlink() operation to become
|
``xml.dom.pulldom``, allowing the ``.unlink()`` operation to become
|
||||||
a no-op.
|
a no-op.
|
||||||
|
|
||||||
This proposal is divided into the following sections:
|
This proposal is divided into the following sections:
|
||||||
|
|
||||||
- Proposed Solution
|
- Proposed Solution
|
||||||
- Implementation Strategy
|
- Implementation Strategy
|
||||||
- Possible Applications
|
- Possible Applications
|
||||||
- Previous Weak Reference Work in Python
|
- Previous Weak Reference Work in Python
|
||||||
- Weak References in Java
|
- Weak References in Java
|
||||||
|
|
||||||
The full text of one early proposal is included as an appendix
|
The full text of one early proposal is included as an appendix
|
||||||
since it does not appear to be available on the net.
|
since it does not appear to be available on the net.
|
||||||
|
|
||||||
|
|
||||||
Aspects of the Solution Space
|
Aspects of the Solution Space
|
||||||
|
=============================
|
||||||
|
|
||||||
There are two distinct aspects to the weak references problem:
|
There are two distinct aspects to the weak references problem:
|
||||||
|
|
||||||
- Invalidation of weak references
|
- Invalidation of weak references
|
||||||
- Presentation of weak references to Python code
|
- Presentation of weak references to Python code
|
||||||
|
|
||||||
Invalidation:
|
Invalidation
|
||||||
|
------------
|
||||||
|
|
||||||
Past approaches to weak reference invalidation have often hinged
|
Past approaches to weak reference invalidation have often hinged
|
||||||
on storing a strong reference and being able to examine all the
|
on storing a strong reference and being able to examine all the
|
||||||
instances of weak reference objects, and invalidating them when
|
instances of weak reference objects, and invalidating them when
|
||||||
the reference count of their referent goes to one (indicating that
|
the reference count of their referent goes to one (indicating that
|
||||||
the reference stored by the weak reference is the last remaining
|
the reference stored by the weak reference is the last remaining
|
||||||
reference). This has the advantage that the memory management
|
reference). This has the advantage that the memory management
|
||||||
machinery in Python need not change, and that any type can be
|
machinery in Python need not change, and that any type can be
|
||||||
weakly referenced.
|
weakly referenced.
|
||||||
|
|
||||||
The disadvantage of this approach to invalidation is that it
|
The disadvantage of this approach to invalidation is that it
|
||||||
assumes that the management of the weak references is called
|
assumes that the management of the weak references is called
|
||||||
sufficiently frequently that weakly-referenced objects are noticed
|
sufficiently frequently that weakly-referenced objects are noticed
|
||||||
within a reasonably short time frame; since this means a scan over
|
within a reasonably short time frame; since this means a scan over
|
||||||
some data structure to invalidate references, an operation which
|
some data structure to invalidate references, an operation which
|
||||||
is O(N) on the number of weakly referenced objects, this is not
|
is O(N) on the number of weakly referenced objects, this is not
|
||||||
effectively amortized for any single object which is weakly
|
effectively amortized for any single object which is weakly
|
||||||
referenced. This also assumes that the application is calling
|
referenced. This also assumes that the application is calling
|
||||||
into code which handles weakly-referenced objects with some
|
into code which handles weakly-referenced objects with some
|
||||||
frequency, which makes weak-references less attractive for library
|
frequency, which makes weak-references less attractive for library
|
||||||
code.
|
code.
|
||||||
|
|
||||||
An alternate approach to invalidation is that the de-allocation
|
An alternate approach to invalidation is that the de-allocation
|
||||||
code to be aware of the possibility of weak references and make a
|
code to be aware of the possibility of weak references and make a
|
||||||
specific call into the weak-reference management code to all
|
specific call into the weak-reference management code to all
|
||||||
invalidation whenever an object is deallocated. This requires a
|
invalidation whenever an object is deallocated. This requires a
|
||||||
change in the tp_dealloc handler for weakly-referencable objects;
|
change in the tp_dealloc handler for weakly-referencable objects;
|
||||||
an additional call is needed at the "top" of the handler for
|
an additional call is needed at the "top" of the handler for
|
||||||
objects which support weak-referencing, and an efficient way to
|
objects which support weak-referencing, and an efficient way to
|
||||||
map from an object to a chain of weak references for that object
|
map from an object to a chain of weak references for that object
|
||||||
is needed as well.
|
is needed as well.
|
||||||
|
|
||||||
Presentation:
|
Presentation
|
||||||
|
------------
|
||||||
|
|
||||||
Two ways that weak references are presented to the Python layer
|
Two ways that weak references are presented to the Python layer
|
||||||
have been as explicit reference objects upon which some operation
|
have been as explicit reference objects upon which some operation
|
||||||
is required in order to retrieve a usable reference to the
|
is required in order to retrieve a usable reference to the
|
||||||
underlying object, and proxy objects which masquerade as the
|
underlying object, and proxy objects which masquerade as the
|
||||||
original objects as much as possible.
|
original objects as much as possible.
|
||||||
|
|
||||||
Reference objects are easy to work with when some additional layer
|
Reference objects are easy to work with when some additional layer
|
||||||
of object managemenet is being added in Python; references can be
|
of object managemenet is being added in Python; references can be
|
||||||
checked for liveness explicitly, without having to invoke
|
checked for liveness explicitly, without having to invoke
|
||||||
operations on the referents and catching some special exception
|
operations on the referents and catching some special exception
|
||||||
raised when an invalid weak reference is used.
|
raised when an invalid weak reference is used.
|
||||||
|
|
||||||
However, a number of users favor the proxy approach simply because
|
However, a number of users favor the proxy approach simply because
|
||||||
the weak reference looks so much like the original object.
|
the weak reference looks so much like the original object.
|
||||||
|
|
||||||
|
|
||||||
Proposed Solution
|
Proposed Solution
|
||||||
|
=================
|
||||||
|
|
||||||
Weak references should be able to point to any Python object that
|
Weak references should be able to point to any Python object that
|
||||||
may have substantial memory size (directly or indirectly), or hold
|
may have substantial memory size (directly or indirectly), or hold
|
||||||
references to external resources (database connections, open
|
references to external resources (database connections, open
|
||||||
files, etc.).
|
files, etc.).
|
||||||
|
|
||||||
A new module, weakref, will contain new functions used to create
|
A new module, weakref, will contain new functions used to create
|
||||||
weak references. weakref.ref() will create a "weak reference
|
weak references. ``weakref.ref()`` will create a "weak reference
|
||||||
object" and optionally attach a callback which will be called when
|
object" and optionally attach a callback which will be called when
|
||||||
the object is about to be finalized. weakref.mapping() will
|
the object is about to be finalized. ``weakref.mapping()`` will
|
||||||
create a "weak dictionary". A third function, weakref.proxy(),
|
create a "weak dictionary". A third function, ``weakref.proxy()``,
|
||||||
will create a proxy object that behaves somewhat like the original
|
will create a proxy object that behaves somewhat like the original
|
||||||
object.
|
object.
|
||||||
|
|
||||||
A weak reference object will allow access to the referenced object
|
A weak reference object will allow access to the referenced object
|
||||||
if it hasn't been collected and to determine if the object still
|
if it hasn't been collected and to determine if the object still
|
||||||
exists in memory. Retrieving the referent is done by calling the
|
exists in memory. Retrieving the referent is done by calling the
|
||||||
reference object. If the referent is no longer alive, this will
|
reference object. If the referent is no longer alive, this will
|
||||||
return None instead.
|
return None instead.
|
||||||
|
|
||||||
A weak dictionary maps arbitrary keys to values, but does not own
|
A weak dictionary maps arbitrary keys to values, but does not own
|
||||||
a reference to the values. When the values are finalized, the
|
a reference to the values. When the values are finalized, the
|
||||||
(key, value) pairs for which it is a value are removed from all
|
(key, value) pairs for which it is a value are removed from all
|
||||||
the mappings containing such pairs. Like dictionaries, weak
|
the mappings containing such pairs. Like dictionaries, weak
|
||||||
dictionaries are not hashable.
|
dictionaries are not hashable.
|
||||||
|
|
||||||
Proxy objects are weak references that attempt to behave like the
|
Proxy objects are weak references that attempt to behave like the
|
||||||
object they proxy, as much as they can. Regardless of the
|
object they proxy, as much as they can. Regardless of the
|
||||||
underlying type, proxies are not hashable since their ability to
|
underlying type, proxies are not hashable since their ability to
|
||||||
act as a weak reference relies on a fundamental mutability that
|
act as a weak reference relies on a fundamental mutability that
|
||||||
will cause failures when used as dictionary keys -- even if the
|
will cause failures when used as dictionary keys -- even if the
|
||||||
proper hash value is computed before the referent dies, the
|
proper hash value is computed before the referent dies, the
|
||||||
resulting proxy cannot be used as a dictionary key since it cannot
|
resulting proxy cannot be used as a dictionary key since it cannot
|
||||||
be compared once the referent has expired, and comparability is
|
be compared once the referent has expired, and comparability is
|
||||||
necessary for dictionary keys. Operations on proxy objects after
|
necessary for dictionary keys. Operations on proxy objects after
|
||||||
the referent dies cause weakref.ReferenceError to be raised in
|
the referent dies cause weakref.ReferenceError to be raised in
|
||||||
most cases. "is" comparisons, type(), and id() will continue to
|
most cases. "is" comparisons, ``type()``, and ``id()`` will continue to
|
||||||
work, but always refer to the proxy and not the referent.
|
work, but always refer to the proxy and not the referent.
|
||||||
|
|
||||||
The callbacks registered with weak references must accept a single
|
The callbacks registered with weak references must accept a single
|
||||||
parameter, which will be the weak reference or proxy object
|
parameter, which will be the weak reference or proxy object
|
||||||
itself. The object cannot be accessed or resurrected in the
|
itself. The object cannot be accessed or resurrected in the
|
||||||
callback.
|
callback.
|
||||||
|
|
||||||
|
|
||||||
Implementation Strategy
|
Implementation Strategy
|
||||||
|
=======================
|
||||||
|
|
||||||
The implementation of weak references will include a list of
|
The implementation of weak references will include a list of
|
||||||
reference containers that must be cleared for each weakly-
|
reference containers that must be cleared for each weakly-
|
||||||
referencable object. If the reference is from a weak dictionary,
|
referencable object. If the reference is from a weak dictionary,
|
||||||
the dictionary entry is cleared first. Then, any associated
|
the dictionary entry is cleared first. Then, any associated
|
||||||
callback is called with the object passed as a parameter. Once
|
callback is called with the object passed as a parameter. Once
|
||||||
all callbacks have been called, the object is finalized and
|
all callbacks have been called, the object is finalized and
|
||||||
deallocated.
|
deallocated.
|
||||||
|
|
||||||
Many built-in types will participate in the weak-reference
|
Many built-in types will participate in the weak-reference
|
||||||
management, and any extension type can elect to do so. The type
|
management, and any extension type can elect to do so. The type
|
||||||
structure will contain an additional field which provides an
|
structure will contain an additional field which provides an
|
||||||
offset into the instance structure which contains a list of weak
|
offset into the instance structure which contains a list of weak
|
||||||
reference structures. If the value of the field is <= 0, the
|
reference structures. If the value of the field is <= 0, the
|
||||||
object does not participate. In this case, weakref.ref(),
|
object does not participate. In this case, ``weakref.ref()``,
|
||||||
<weakdict>.__setitem__() and .setdefault(), and item assignment will
|
``<weakdict>.__setitem__()`` and ``.setdefault()``, and item assignment will
|
||||||
raise TypeError. If the value of the field is > 0, a new weak
|
raise ``TypeError``. If the value of the field is > 0, a new weak
|
||||||
reference can be generated and added to the list.
|
reference can be generated and added to the list.
|
||||||
|
|
||||||
This approach is taken to allow arbitrary extension types to
|
This approach is taken to allow arbitrary extension types to
|
||||||
participate, without taking a memory hit for numbers or other
|
participate, without taking a memory hit for numbers or other
|
||||||
small types.
|
small types.
|
||||||
|
|
||||||
Standard types which support weak references include instances,
|
Standard types which support weak references include instances,
|
||||||
functions, and bound & unbound methods. With the addition of
|
functions, and bound & unbound methods. With the addition of
|
||||||
class types ("new-style classes") in Python 2.2, types grew
|
class types ("new-style classes") in Python 2.2, types grew
|
||||||
support for weak references. Instances of class types are weakly
|
support for weak references. Instances of class types are weakly
|
||||||
referencable if they have a base type which is weakly referencable,
|
referencable if they have a base type which is weakly referencable,
|
||||||
the class not specify __slots__, or a slot is named __weakref__.
|
the class not specify ``__slots__``, or a slot is named ``__weakref__``.
|
||||||
Generators also support weak references.
|
Generators also support weak references.
|
||||||
|
|
||||||
|
|
||||||
Possible Applications
|
Possible Applications
|
||||||
|
=====================
|
||||||
|
|
||||||
PyGTK+ bindings?
|
PyGTK+ bindings?
|
||||||
|
|
||||||
Tkinter -- could avoid circular references by using weak
|
Tkinter -- could avoid circular references by using weak
|
||||||
references from widgets to their parents. Objects won't be
|
references from widgets to their parents. Objects won't be
|
||||||
discarded any sooner in the typical case, but there won't be so
|
discarded any sooner in the typical case, but there won't be so
|
||||||
much dependence on the programmer calling .destroy() before
|
much dependence on the programmer calling ``.destroy()`` before
|
||||||
releasing a reference. This would mostly benefit long-running
|
releasing a reference. This would mostly benefit long-running
|
||||||
applications.
|
applications.
|
||||||
|
|
||||||
DOM trees.
|
DOM trees.
|
||||||
|
|
||||||
|
|
||||||
Previous Weak Reference Work in Python
|
Previous Weak Reference Work in Python
|
||||||
|
======================================
|
||||||
|
|
||||||
Dianne Hackborn has proposed something called "virtual references".
|
Dianne Hackborn has proposed something called "virtual references".
|
||||||
'vref' objects are very similar to java.lang.ref.WeakReference
|
'vref' objects are very similar to java.lang.ref.WeakReference
|
||||||
objects, except there is no equivalent to the invalidation
|
objects, except there is no equivalent to the invalidation
|
||||||
queues. Implementing a "weak dictionary" would be just as
|
queues. Implementing a "weak dictionary" would be just as
|
||||||
difficult as using only weak references (without the invalidation
|
difficult as using only weak references (without the invalidation
|
||||||
queue) in Java. Information on this has disappeared from the Web,
|
queue) in Java. Information on this has disappeared from the Web,
|
||||||
but is included below as an Appendix.
|
but is included below as an Appendix.
|
||||||
|
|
||||||
Marc-André Lemburg's mx.Proxy package:
|
Marc-André Lemburg's mx.Proxy package:
|
||||||
|
|
||||||
http://www.lemburg.com/files/python/mxProxy.html
|
http://www.lemburg.com/files/python/mxProxy.html
|
||||||
|
|
||||||
The weakdict module by Dieter Maurer is implemented in C and
|
The weakdict module by Dieter Maurer is implemented in C and
|
||||||
Python. It appears that the Web pages have not been updated since
|
Python. It appears that the Web pages have not been updated since
|
||||||
Python 1.5.2a, so I'm not yet sure if the implementation is
|
Python 1.5.2a, so I'm not yet sure if the implementation is
|
||||||
compatible with Python 2.0.
|
compatible with Python 2.0.
|
||||||
|
|
||||||
http://www.handshake.de/~dieter/weakdict.html
|
http://www.handshake.de/~dieter/weakdict.html
|
||||||
|
|
||||||
PyWeakReference by Alex Shindich:
|
PyWeakReference by Alex Shindich:
|
||||||
|
|
||||||
http://sourceforge.net/projects/pyweakreference/
|
http://sourceforge.net/projects/pyweakreference/
|
||||||
|
|
||||||
Eric Tiedemann has a weak dictionary implementation:
|
Eric Tiedemann has a weak dictionary implementation:
|
||||||
|
|
||||||
http://www.hyperreal.org/~est/python/weak/
|
http://www.hyperreal.org/~est/python/weak/
|
||||||
|
|
||||||
|
|
||||||
Weak References in Java
|
Weak References in Java
|
||||||
|
=======================
|
||||||
|
|
||||||
http://java.sun.com/j2se/1.3/docs/api/java/lang/ref/package-summary.html
|
http://java.sun.com/j2se/1.3/docs/api/java/lang/ref/package-summary.html
|
||||||
|
|
||||||
Java provides three forms of weak references, and one interesting
|
Java provides three forms of weak references, and one interesting
|
||||||
helper class. The three forms are called "weak", "soft", and
|
helper class. The three forms are called "weak", "soft", and
|
||||||
"phantom" references. The relevant classes are defined in the
|
"phantom" references. The relevant classes are defined in the
|
||||||
java.lang.ref package.
|
java.lang.ref package.
|
||||||
|
|
||||||
For each of the reference types, there is an option to add the
|
For each of the reference types, there is an option to add the
|
||||||
reference to a queue when it is invalidated by the memory
|
reference to a queue when it is invalidated by the memory
|
||||||
allocator. The primary purpose of this facility seems to be that
|
allocator. The primary purpose of this facility seems to be that
|
||||||
it allows larger structures to be composed to incorporate
|
it allows larger structures to be composed to incorporate
|
||||||
weak-reference semantics without having to impose substantial
|
weak-reference semantics without having to impose substantial
|
||||||
additional locking requirements. For instance, it would not be
|
additional locking requirements. For instance, it would not be
|
||||||
difficult to use this facility to create a "weak" hash table which
|
difficult to use this facility to create a "weak" hash table which
|
||||||
removes keys and referents when a reference is no longer used
|
removes keys and referents when a reference is no longer used
|
||||||
elsewhere. Using weak references for the objects without some
|
elsewhere. Using weak references for the objects without some
|
||||||
sort of notification queue for invalidations leads to much more
|
sort of notification queue for invalidations leads to much more
|
||||||
tedious implementation of the various operations required on hash
|
tedious implementation of the various operations required on hash
|
||||||
tables. This can be a performance bottleneck if deallocations of
|
tables. This can be a performance bottleneck if deallocations of
|
||||||
the stored objects are infrequent.
|
the stored objects are infrequent.
|
||||||
|
|
||||||
Java's "weak" references are most like Dianne Hackborn's old vref
|
Java's "weak" references are most like Dianne Hackborn's old vref
|
||||||
proposal: a reference object refers to a single Python object,
|
proposal: a reference object refers to a single Python object,
|
||||||
but does not own a reference to that object. When that object is
|
but does not own a reference to that object. When that object is
|
||||||
deallocated, the reference object is invalidated. Users of the
|
deallocated, the reference object is invalidated. Users of the
|
||||||
reference object can easily determine that the reference has been
|
reference object can easily determine that the reference has been
|
||||||
invalidated, or a NullObjectDereferenceError can be raised when
|
invalidated, or a NullObjectDereferenceError can be raised when
|
||||||
an attempt is made to use the referred-to object.
|
an attempt is made to use the referred-to object.
|
||||||
|
|
||||||
The "soft" references are similar, but are not invalidated as soon
|
The "soft" references are similar, but are not invalidated as soon
|
||||||
as all other references to the referred-to object have been
|
as all other references to the referred-to object have been
|
||||||
released. The "soft" reference does own a reference, but allows
|
released. The "soft" reference does own a reference, but allows
|
||||||
the memory allocator to free the referent if the memory is needed
|
the memory allocator to free the referent if the memory is needed
|
||||||
elsewhere. It is not clear whether this means soft references are
|
elsewhere. It is not clear whether this means soft references are
|
||||||
released before the malloc() implementation calls sbrk() or its
|
released before the ``malloc()`` implementation calls ``sbrk()`` or its
|
||||||
equivalent, or if soft references are only cleared when malloc()
|
equivalent, or if soft references are only cleared when ``malloc()``
|
||||||
returns NULL.
|
returns ``NULL``.
|
||||||
|
|
||||||
"Phantom" references are a little different; unlike weak and soft
|
"Phantom" references are a little different; unlike weak and soft
|
||||||
references, the referent is not cleared when the reference is
|
references, the referent is not cleared when the reference is
|
||||||
added to its queue. When all phantom references for an object
|
added to its queue. When all phantom references for an object
|
||||||
are dequeued, the object is cleared. This can be used to keep an
|
are dequeued, the object is cleared. This can be used to keep an
|
||||||
object alive until some additional cleanup is performed which
|
object alive until some additional cleanup is performed which
|
||||||
needs to happen before the objects .finalize() method is called.
|
needs to happen before the objects ``.finalize()`` method is called.
|
||||||
|
|
||||||
Unlike the other two reference types, "phantom" references must be
|
Unlike the other two reference types, "phantom" references must be
|
||||||
associated with an invalidation queue.
|
associated with an invalidation queue.
|
||||||
|
|
||||||
|
|
||||||
Appendix -- Dianne Hackborn's vref proposal (1995)
|
Appendix -- Dianne Hackborn's vref proposal (1995)
|
||||||
|
==================================================
|
||||||
|
|
||||||
[This has been indented and paragraphs reflowed, but there have be
|
[This has been indented and paragraphs reflowed, but there have be
|
||||||
no content changes. --Fred]
|
no content changes. --Fred]
|
||||||
|
|
||||||
Proposal: Virtual References
|
Proposal: Virtual References
|
||||||
|
----------------------------
|
||||||
|
|
||||||
In an attempt to partly address the recurring discussion
|
In an attempt to partly address the recurring discussion
|
||||||
concerning reference counting vs. garbage collection, I would like
|
concerning reference counting vs. garbage collection, I would like
|
||||||
to propose an extension to Python which should help in the
|
to propose an extension to Python which should help in the
|
||||||
creation of "well structured" cyclic graphs. In particular, it
|
creation of "well structured" cyclic graphs. In particular, it
|
||||||
should allow at least trees with parent back-pointers and
|
should allow at least trees with parent back-pointers and
|
||||||
doubly-linked lists to be created without worry about cycles.
|
doubly-linked lists to be created without worry about cycles.
|
||||||
|
|
||||||
The basic mechanism I'd like to propose is that of a "virtual
|
The basic mechanism I'd like to propose is that of a "virtual
|
||||||
reference," or a "vref" from here on out. A vref is essentially a
|
reference," or a "vref" from here on out. A vref is essentially a
|
||||||
handle on an object that does not increment the object's reference
|
handle on an object that does not increment the object's reference
|
||||||
count. This means that holding a vref on an object will not keep
|
count. This means that holding a vref on an object will not keep
|
||||||
the object from being destroyed. This would allow the Python
|
the object from being destroyed. This would allow the Python
|
||||||
programmer, for example, to create the aforementioned tree
|
programmer, for example, to create the aforementioned tree
|
||||||
structure tree structure, which is automatically destroyed when it
|
structure tree structure, which is automatically destroyed when it
|
||||||
is no longer in use -- by making all of the parent back-references
|
is no longer in use -- by making all of the parent back-references
|
||||||
into vrefs, they no longer create reference cycles which keep the
|
into vrefs, they no longer create reference cycles which keep the
|
||||||
tree from being destroyed.
|
tree from being destroyed.
|
||||||
|
|
||||||
In order to implement this mechanism, the Python core must ensure
|
In order to implement this mechanism, the Python core must ensure
|
||||||
that no -real- pointers are ever left referencing objects that no
|
that no -real- pointers are ever left referencing objects that no
|
||||||
longer exist. The implementation I would like to propose involves
|
longer exist. The implementation I would like to propose involves
|
||||||
two basic additions to the current Python system:
|
two basic additions to the current Python system:
|
||||||
|
|
||||||
1. A new "vref" type, through which the Python programmer creates
|
1. A new "vref" type, through which the Python programmer creates
|
||||||
and manipulates virtual references. Internally, it is
|
and manipulates virtual references. Internally, it is
|
||||||
basically a C-level Python object with a pointer to the Python
|
basically a C-level Python object with a pointer to the Python
|
||||||
object it is a reference to. Unlike all other Python code,
|
object it is a reference to. Unlike all other Python code,
|
||||||
however, it does not change the reference count of this object.
|
however, it does not change the reference count of this object.
|
||||||
In addition, it includes two pointers to implement a
|
In addition, it includes two pointers to implement a
|
||||||
doubly-linked list, which is used below.
|
doubly-linked list, which is used below.
|
||||||
|
|
||||||
2. The addition of a new field to the basic Python object
|
2. The addition of a new field to the basic Python object
|
||||||
[PyObject_Head in object.h], which is either NULL, or points to
|
[``PyObject_Head`` in object.h], which is either ``NULL``, or points to
|
||||||
the head of a list of all vref objects that reference it. When
|
the head of a list of all vref objects that reference it. When
|
||||||
a vref object attaches itself to another object, it adds itself
|
a vref object attaches itself to another object, it adds itself
|
||||||
to this linked list. Then, if an object with any vrefs on it
|
to this linked list. Then, if an object with any vrefs on it
|
||||||
is deallocated, it may walk this list and ensure that all of
|
is deallocated, it may walk this list and ensure that all of
|
||||||
the vrefs on it point to some safe value, e.g. Nothing.
|
the vrefs on it point to some safe value, e.g. Nothing.
|
||||||
|
|
||||||
|
|
||||||
This implementation should hopefully have a minimal impact on the
|
This implementation should hopefully have a minimal impact on the
|
||||||
current Python core -- when no vrefs exist, it should only add one
|
current Python core -- when no vrefs exist, it should only add one
|
||||||
pointer to all objects, and a check for a NULL pointer every time
|
pointer to all objects, and a check for a ``NULL`` pointer every time
|
||||||
an object is deallocated.
|
an object is deallocated.
|
||||||
|
|
||||||
Back at the Python language level, I have considered two possible
|
Back at the Python language level, I have considered two possible
|
||||||
semantics for the vref object --
|
semantics for the vref object --
|
||||||
|
|
||||||
==> Pointer semantics:
|
Pointer semantics
|
||||||
|
-----------------
|
||||||
|
|
||||||
In this model, a vref behaves essentially like a Python-level
|
In this model, a vref behaves essentially like a Python-level
|
||||||
pointer; the Python program must explicitly dereference the vref
|
pointer; the Python program must explicitly dereference the vref
|
||||||
to manipulate the actual object it references.
|
to manipulate the actual object it references.
|
||||||
|
|
||||||
An example vref module using this model could include the
|
An example vref module using this model could include the
|
||||||
function "new"; When used as 'MyVref = vref.new(MyObject)', it
|
function "new"; When used as 'MyVref = vref.new(MyObject)', it
|
||||||
returns a new vref object such that MyVref.object ==
|
returns a new vref object such that ``MyVref.object == MyObject``.
|
||||||
MyObject. MyVref.object would then change to Nothing if
|
``MyVref.object`` would then change to Nothing if
|
||||||
MyObject is ever deallocated.
|
``MyObject`` is ever deallocated.
|
||||||
|
|
||||||
For a concrete example, we may introduce some new C-style syntax:
|
For a concrete example, we may introduce some new C-style syntax:
|
||||||
|
|
||||||
& -- unary operator, creates a vref on an object, same as vref.new().
|
* ``&`` -- unary operator, creates a vref on an object, same as ``vref.new()``.
|
||||||
* -- unary operator, dereference a vref, same as VrefObject.object.
|
* ``*`` -- unary operator, dereference a vref, same as ``VrefObject.object``.
|
||||||
|
|
||||||
We can then define:
|
We can then define::
|
||||||
|
|
||||||
1. type(&MyObject) == vref.VrefType
|
1. type(&MyObject) == vref.VrefType
|
||||||
2. *(&MyObject) == MyObject
|
2. *(&MyObject) == MyObject
|
||||||
3. (*(&MyObject)).attr == MyObject.attr
|
3. (*(&MyObject)).attr == MyObject.attr
|
||||||
4. &&MyObject == Nothing
|
4. &&MyObject == Nothing
|
||||||
5. *MyObject -> exception
|
5. *MyObject -> exception
|
||||||
|
|
||||||
Rule #4 is subtle, but comes about because we have made a vref
|
Rule #4 is subtle, but comes about because we have made a vref
|
||||||
to (a vref with no real references). Thus the outer vref is
|
to (a vref with no real references). Thus the outer vref is
|
||||||
cleared to Nothing when the inner one inevitably disappears.
|
cleared to Nothing when the inner one inevitably disappears.
|
||||||
|
|
||||||
==> Proxy semantics:
|
Proxy semantics
|
||||||
|
----------------
|
||||||
|
|
||||||
In this model, the Python programmer manipulates vref objects
|
In this model, the Python programmer manipulates vref objects
|
||||||
just as if she were manipulating the object it is a reference
|
just as if she were manipulating the object it is a reference
|
||||||
of. This is accomplished by implementing the vref so that all
|
of. This is accomplished by implementing the vref so that all
|
||||||
operations on it are redirected to its referenced object. With
|
operations on it are redirected to its referenced object. With
|
||||||
this model, the dereference operator (*) no longer makes sense;
|
this model, the dereference operator (*) no longer makes sense;
|
||||||
instead, we have only the reference operator (&), and define:
|
instead, we have only the reference operator (&), and define::
|
||||||
|
|
||||||
1. type(&MyObject) == type(MyObject)
|
1. type(&MyObject) == type(MyObject)
|
||||||
2. &MyObject == MyObject
|
2. &MyObject == MyObject
|
||||||
3. (&MyObject).attr == MyObject.attr
|
3. (&MyObject).attr == MyObject.attr
|
||||||
4. &&MyObject == MyObject
|
4. &&MyObject == MyObject
|
||||||
|
|
||||||
Again, rule #4 is important -- here, the outer vref is in fact a
|
Again, rule #4 is important -- here, the outer vref is in fact a
|
||||||
reference to the original object, and -not- the inner vref.
|
reference to the original object, and -not- the inner vref.
|
||||||
This is because all operations applied to a vref actually apply
|
This is because all operations applied to a vref actually apply
|
||||||
to its object, so that creating a vref of a vref actually
|
to its object, so that creating a vref of a vref actually
|
||||||
results in creating a vref of the latter's object.
|
results in creating a vref of the latter's object.
|
||||||
|
|
||||||
The first, pointer semantics, has the advantage that it would be
|
The first, pointer semantics, has the advantage that it would be
|
||||||
very easy to implement; the vref type is extremely simple,
|
very easy to implement; the vref type is extremely simple,
|
||||||
requiring at minimum a single attribute, object, and a function to
|
requiring at minimum a single attribute, object, and a function to
|
||||||
create a reference.
|
create a reference.
|
||||||
|
|
||||||
However, I really like the proxy semantics. Not only does it put
|
However, I really like the proxy semantics. Not only does it put
|
||||||
less of a burden on the Python programmer, but it allows you to do
|
less of a burden on the Python programmer, but it allows you to do
|
||||||
nice things like use a vref anywhere you would use the actual
|
nice things like use a vref anywhere you would use the actual
|
||||||
object. Unfortunately, it would probably an extreme pain, if not
|
object. Unfortunately, it would probably an extreme pain, if not
|
||||||
practically impossible, to implement in the current Python
|
practically impossible, to implement in the current Python
|
||||||
implementation. I do have some thoughts, though, on how to do
|
implementation. I do have some thoughts, though, on how to do
|
||||||
this, if it seems interesting; one possibility is to introduce new
|
this, if it seems interesting; one possibility is to introduce new
|
||||||
type-checking functions which handle the vref. This would
|
type-checking functions which handle the vref. This would
|
||||||
hopefully older C modules which don't expect vrefs to simply
|
hopefully older C modules which don't expect vrefs to simply
|
||||||
return a type error, until they can be fixed.
|
return a type error, until they can be fixed.
|
||||||
|
|
||||||
Finally, there are some other additional capabilities that this
|
Finally, there are some other additional capabilities that this
|
||||||
system could provide. One that seems particularly interesting to
|
system could provide. One that seems particularly interesting to
|
||||||
me involves allowing the Python programmer to add "destructor"
|
me involves allowing the Python programmer to add "destructor"
|
||||||
function to a vref -- this Python function would be called
|
function to a vref -- this Python function would be called
|
||||||
immediately prior to the referenced object being deallocated,
|
immediately prior to the referenced object being deallocated,
|
||||||
allowing a Python program to invisibly attach itself to another
|
allowing a Python program to invisibly attach itself to another
|
||||||
object and watch for it to disappear. This seems neat, though I
|
object and watch for it to disappear. This seems neat, though I
|
||||||
haven't actually come up with any practical uses for it, yet... :)
|
haven't actually come up with any practical uses for it, yet... :)
|
||||||
|
|
||||||
-- Dianne
|
-- Dianne
|
||||||
|
|
||||||
|
|
||||||
Copyright
|
Copyright
|
||||||
|
=========
|
||||||
|
|
||||||
This document has been placed in the public domain.
|
This document has been placed in the public domain.
|
||||||
|
|
||||||
|
|
||||||
|
..
|
||||||
Local Variables:
|
Local Variables:
|
||||||
mode: indented-text
|
mode: indented-text
|
||||||
indent-tabs-mode: nil
|
indent-tabs-mode: nil
|
||||||
sentence-end-double-space: t
|
sentence-end-double-space: t
|
||||||
fill-column: 70
|
fill-column: 70
|
||||||
coding: utf-8
|
coding: utf-8
|
||||||
End:
|
End:
|
||||||
|
|
Loading…
Reference in New Issue