From 86fe83480d414397ddc480ac3983f5e137ef04a9 Mon Sep 17 00:00:00 2001 From: Neal Norwitz Date: Tue, 15 Aug 2006 06:27:32 +0000 Subject: [PATCH] Updates from Travis. I modified his version based on a followup mail that stated: As long as the object returned is a proper sub-class then it should work fine. The Exact checks should be removed from both the code and the PEP. --- pep-0357.txt | 90 +++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 71 insertions(+), 19 deletions(-) diff --git a/pep-0357.txt b/pep-0357.txt index ec17e624d..a017368b6 100644 --- a/pep-0357.txt +++ b/pep-0357.txt @@ -45,31 +45,68 @@ Rationale Proposal Add an nb_index slot to PyNumberMethods, and a corresponding - __index__ special method. Objects could define a function to place - in the nb_index slot that returns an appropriate C-integer (Py_ssize_t - after PEP 353). This C-integer will be used whenever Python needs - one such as in PySequence_GetSlice, PySequence_SetSlice, and - PySequence_DelSlice. + __index__ special method. Objects could define a function to + place in the nb_index slot that returns a Python integer + (either an int or a long). This integer can + then be appropriately converted to a Py_ssize_t value whenever + Python needs one such as in PySequence_GetSlice, + PySequence_SetSlice, and PySequence_DelSlice. Specification: - 1) The nb_index slot will have the signature + 1) The nb_index slot will have the following signature - Py_ssize_t index_func (PyObject *self) + PyObject *index_func (PyObject *self) + + The returned object must be a Python IntType or + Python LongType. NULL should be returned on + error with an appropriate error set. 2) The __index__ special method will have the signature def __index__(self): return obj - where obj must be either an int or a long. + where obj must be either an int or a long. - 3) A new C-API function PyNumber_Index will be added with signature + 3) 3 new abstract C-API functions will be added - Py_ssize_t PyNumber_Index (PyObject *obj) + a) The first checks to see if the object supports the index + slot and if it is filled in. - which will return obj->ob_type->tp_as_number->nb_index(obj) if it is available. - A -1 will be returned and an exception set on an error. + int PyIndex_Check(obj) + + This will return true if the object defines the nb_index + slot. + + b) The second is a simple wrapper around the nb_index call that + raises PyExc_TypeError if the call is not available or if it + doesn't return an int or long. Because the + PyIndex_Check is performed inside the PyNumber_Index call + you can call it directly and manage any error rather than + check for compatibility first. + + PyObject *PyNumber_Index (PyObject *obj) + + c) The third call helps deal with the common situation of + actually needing a Py_ssize_t value from the object to use for + indexing or other needs. + + Py_ssize_t PyNumber_AsSsize_t(PyObject *obj, PyObject *exc) + + The function calls the nb_index slot of obj if it is + available and then converts the returned Python integer into + a Py_ssize_t value. If this goes well, then the value is + returned. The second argument allows control over what + happens if the integer returned from nb_index cannot fit + into a Py_ssize_t value. + + If exc is NULL, then the returnd value will be clipped to + PY_SSIZE_T_MAX or PY_SSIZE_T_MIN depending on whether the + nb_index slot of obj returned a positive or negative + integer. If exc is non-NULL, then it is the error object + that will be set to replace the PyExc_OverflowError that was + raised when the Python integer or long was converted to Py_ssize_t. 4) A new operator.index(obj) function will be added that calls equivalent of obj.__index__() and raises an error if obj does not implement @@ -90,13 +127,19 @@ Implementation Plan slots for subscript access and use a special-check for integers to check for the slot as well. - 5) Add the nb_index slot to integers and long_integers. + 5) Add the nb_index slot to integers and long_integers + (which just return themselves) 6) Add PyNumber_Index C-API to return an integer from any Python Object that has the nb_index slot. 7) Add the operator.index(x) function. + 8) Alter arrayobject.c and mmapmodule.c to use the new C-API for their + sub-scripting and other needs. + + 9) Add unit-tests + Discussion Questions @@ -129,13 +172,22 @@ Discussion Questions for examples of names that were suggested such as "__discrete__" and "__ordinal__". - Why return Py_ssize_t from nb_index? + Why return PyObject * from nb_index? - The nb_index slot is primarily intended to return an integer - needed by the sequence interface. In Python 2.5 this is - Py_ssize_t. As this is the primary purpose of the slot, it - makes sense to return the C-integer directly and not wrapped - in a Python int object. + Intially Py_ssize_t was selected as the return type for the + nb_index slot. However, this led to an inability to track and + distinguish overflow and underflow errors without ugly and brittle + hacks. As the nb_index slot is used in at least 3 different ways + in the Python core (to get an integer, to get a slice end-point, + and to get a sequence index), there is quite a bit of flexibility + needed to handle all these cases. The importance of having the + necessary flexibility to handle all the use cases is critical. + For example, the initial implementation that returned Py_ssize_t for + nb_index led to the discovery that on a 32-bit machine with >=2GB of RAM + s = 'x' * (2**100) works but len(s) was clipped at 2147483647. + Several fixes were suggested but eventually it was decided that + nb_index needed to return a Python Object similar to the nb_int + and nb_long slots in order to handle overflow correctly. Why can't __index__ return any object with the nb_index method?