Advanced
Mailing List Archive

Mailing List Archive

  1. Home >
  2. Python>
  3. Dev
1 2  View All
Arbitrary attributes on funcs and methods
bwarsaw at python
Apr 10, 2000, 8:52 AM
Post #1 of 42 (2142 views)
Permalink
A number of people have played FAST and loose with function and method
docstrings, including John Aycock[1], Zope's ORB[2]. Docstrings are
handy because they are the one attribute on funcs and methods that are
easily writable. But as more people overload the semantics for
docstrings, we'll get collisions. I've had a number of discussions
with folks about adding attribute dictionaries to functions and
methods so that you can essentially add any attribute. Namespaces are
one honking great idea -- let's do more of those!

Below is a very raw set of patches to add an attribute dictionary to
funcs and methods. It's only been minimally tested, but if y'all like
the idea, I'll clean it up, sanity check the memory management, and
post the changes to patches@python.org. Here's some things you can
do:

-------------------- snip snip --------------------
Python 1.6a2 (#10, Apr 10 2000, 11:27:59) [GCC 2.8.1] on sunos5
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> def a(): pass
...
>>> a.publish = 1
>>> a.publish
1
>>> a.__doc__
>>> a.__doc__ = 'a doc string'
>>> a.__doc__
'a doc string'
>>> a.magic_string = a.__doc__
>>> a.magic_string
'a doc string'
>>> dir(a)
['__doc__', '__name__', 'func_code', 'func_defaults', 'func_dict', 'func_doc', 'func_globals', 'func_name', 'magic_string', 'publish']
>>> class F:
... def a(self): pass
...
>>> f = F()
>>> f.a.publish
Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: publish
>>> f.a.publish = 1
>>> f.a.publish
1
>>> f.a.__doc__
>>> f.a.__doc__ = 'another doc string'
>>> f.a.__doc__
'another doc string'
>>> f.a.magic_string = f.a.__doc__
>>> f.a.magic_string
'another doc string'
>>> dir(f.a)
['__dict__', '__doc__', '__name__', 'im_class', 'im_func', 'im_self', 'magic_string', 'publish']
>>>
-------------------- snip snip --------------------

-Barry

[1] Aycock, "Compiling Little Languages in Python", http://www.foretec.com/python/workshops/1998-11/proceedings/papers/aycock-little/aycock-little.html

[2] http://classic.zope.org:8080/Documentation/Reference/ORB


P.S. I promised to add a little note about setattr and getattr
vs. setattro and getattro. There's very little documentation about
the differences, and searching on python.org doesn't seem to turn up
anything. The differences are simple. setattr/getattr take a char*
argument naming the attribute to change, while setattro/getattro take
a PyObject* (hence the trailing `o' -- for Object). This stuff should
get documented in the C API, but at least now, it'll turn up in a SIG
search. :)

-------------------- snip snip --------------------
Index: funcobject.h
===================================================================
RCS file: /projects/cvsroot/python/dist/src/Include/funcobject.h,v
retrieving revision 2.16
diff -c -r2.16 funcobject.h
*** funcobject.h 1998/12/04 18:48:02 2.16
--- funcobject.h 2000/04/07 21:30:40
***************
*** 44,49 ****
--- 44,50 ----
PyObject *func_defaults;
PyObject *func_doc;
PyObject *func_name;
+ PyObject *func_dict;
} PyFunctionObject;

extern DL_IMPORT(PyTypeObject) PyFunction_Type;
Index: classobject.c
===================================================================
RCS file: /projects/cvsroot/python/dist/src/Objects/classobject.c,v
retrieving revision 2.84
diff -c -r2.84 classobject.c
*** classobject.c 2000/04/10 13:03:19 2.84
--- classobject.c 2000/04/10 15:27:15
***************
*** 1550,1577 ****
/* Dummies that are not handled by getattr() except for __members__ */
{"__doc__", T_INT, 0},
{"__name__", T_INT, 0},
{NULL} /* Sentinel */
};

static PyObject *
instancemethod_getattr(im, name)
register PyMethodObject *im;
! PyObject *name;
{
! char *sname = PyString_AsString(name);
! if (sname[0] == '_') {
/* Inherit __name__ and __doc__ from the callable object
implementing the method */
! if (strcmp(sname, "__name__") == 0 ||
! strcmp(sname, "__doc__") == 0)
! return PyObject_GetAttr(im->im_func, name);
}
if (PyEval_GetRestricted()) {
PyErr_SetString(PyExc_RuntimeError,
"instance-method attributes not accessible in restricted mode");
return NULL;
}
! return PyMember_Get((char *)im, instancemethod_memberlist, sname);
}

static void
--- 1550,1608 ----
/* Dummies that are not handled by getattr() except for __members__ */
{"__doc__", T_INT, 0},
{"__name__", T_INT, 0},
+ {"__dict__", T_INT, 0},
{NULL} /* Sentinel */
};

+ static int
+ instancemethod_setattr(im, name, v)
+ register PyMethodObject *im;
+ char *name;
+ PyObject *v;
+ {
+ int rtn;
+
+ if (PyEval_GetRestricted() ||
+ strcmp(name, "im_func") == 0 ||
+ strcmp(name, "im_self") == 0 ||
+ strcmp(name, "im_class") == 0)
+ {
+ PyErr_Format(PyExc_TypeError, "read-only attribute: %s", name);
+ return -1;
+ }
+ return PyObject_SetAttrString(im->im_func, name, v);
+ }
+
+
static PyObject *
instancemethod_getattr(im, name)
register PyMethodObject *im;
! char *name;
{
! PyObject *rtn;
!
! if (strcmp(name, "__name__") == 0 ||
! strcmp(name, "__doc__") == 0) {
/* Inherit __name__ and __doc__ from the callable object
implementing the method */
! return PyObject_GetAttrString(im->im_func, name);
}
if (PyEval_GetRestricted()) {
PyErr_SetString(PyExc_RuntimeError,
"instance-method attributes not accessible in restricted mode");
return NULL;
+ }
+ if (strcmp(name, "__dict__") == 0)
+ return PyObject_GetAttrString(im->im_func, name);
+
+ rtn = PyMember_Get((char *)im, instancemethod_memberlist, name);
+ if (rtn == NULL) {
+ PyErr_Clear();
+ rtn = PyObject_GetAttrString(im->im_func, name);
+ if (rtn == NULL)
+ PyErr_SetString(PyExc_AttributeError, name);
}
! return rtn;
}

static void
***************
*** 1662,1669 ****
0,
(destructor)instancemethod_dealloc, /*tp_dealloc*/
0, /*tp_print*/
! 0, /*tp_getattr*/
! 0, /*tp_setattr*/
(cmpfunc)instancemethod_compare, /*tp_compare*/
(reprfunc)instancemethod_repr, /*tp_repr*/
0, /*tp_as_number*/
--- 1693,1700 ----
0,
(destructor)instancemethod_dealloc, /*tp_dealloc*/
0, /*tp_print*/
! (getattrfunc)instancemethod_getattr, /*tp_getattr*/
! (setattrfunc)instancemethod_setattr, /*tp_setattr*/
(cmpfunc)instancemethod_compare, /*tp_compare*/
(reprfunc)instancemethod_repr, /*tp_repr*/
0, /*tp_as_number*/
***************
*** 1672,1678 ****
(hashfunc)instancemethod_hash, /*tp_hash*/
0, /*tp_call*/
0, /*tp_str*/
! (getattrofunc)instancemethod_getattr, /*tp_getattro*/
0, /*tp_setattro*/
};

--- 1703,1709 ----
(hashfunc)instancemethod_hash, /*tp_hash*/
0, /*tp_call*/
0, /*tp_str*/
! 0, /*tp_getattro*/
0, /*tp_setattro*/
};

Index: funcobject.c
===================================================================
RCS file: /projects/cvsroot/python/dist/src/Objects/funcobject.c,v
retrieving revision 2.18
diff -c -r2.18 funcobject.c
*** funcobject.c 1998/05/22 00:55:34 2.18
--- funcobject.c 2000/04/07 22:15:33
***************
*** 62,67 ****
--- 62,68 ----
doc = Py_None;
Py_INCREF(doc);
op->func_doc = doc;
+ op->func_dict = PyDict_New();
}
return (PyObject *)op;
}
***************
*** 133,138 ****
--- 134,140 ----
{"__name__", T_OBJECT, OFF(func_name), READONLY},
{"func_defaults",T_OBJECT, OFF(func_defaults)},
{"func_doc", T_OBJECT, OFF(func_doc)},
+ {"func_dict", T_OBJECT, OFF(func_dict)},
{"__doc__", T_OBJECT, OFF(func_doc)},
{NULL} /* Sentinel */
};
***************
*** 142,153 ****
PyFunctionObject *op;
char *name;
{
if (name[0] != '_' && PyEval_GetRestricted()) {
PyErr_SetString(PyExc_RuntimeError,
"function attributes not accessible in restricted mode");
return NULL;
}
! return PyMember_Get((char *)op, func_memberlist, name);
}

static int
--- 144,167 ----
PyFunctionObject *op;
char *name;
{
+ PyObject* rtn;
+
if (name[0] != '_' && PyEval_GetRestricted()) {
PyErr_SetString(PyExc_RuntimeError,
"function attributes not accessible in restricted mode");
return NULL;
+ }
+ if (strcmp(name, "__dict__") == 0)
+ return op->func_dict;
+
+ rtn = PyMember_Get((char *)op, func_memberlist, name);
+ if (rtn == NULL) {
+ PyErr_Clear();
+ rtn = PyDict_GetItemString(op->func_dict, name);
+ if (rtn == NULL)
+ PyErr_SetString(PyExc_AttributeError, name);
}
! return rtn;
}

static int
***************
*** 156,161 ****
--- 170,177 ----
char *name;
PyObject *value;
{
+ int rtn;
+
if (PyEval_GetRestricted()) {
PyErr_SetString(PyExc_RuntimeError,
"function attributes not settable in restricted mode");
***************
*** 178,185 ****
}
if (value == Py_None)
value = NULL;
}
! return PyMember_Set((char *)op, func_memberlist, name, value);
}

static void
--- 194,214 ----
}
if (value == Py_None)
value = NULL;
+ }
+ else if (strcmp(name, "func_dict") == 0) {
+ if (value == NULL || !PyDict_Check(value)) {
+ PyErr_SetString(
+ PyExc_TypeError,
+ "func_dict must be set to a dict object");
+ return -1;
+ }
+ }
+ rtn = PyMember_Set((char *)op, func_memberlist, name, value);
+ if (rtn < 0) {
+ PyErr_Clear();
+ rtn = PyDict_SetItemString(op->func_dict, name, value);
}
! return rtn;
}

static void
***************
*** 191,196 ****
--- 220,226 ----
Py_DECREF(op->func_name);
Py_XDECREF(op->func_defaults);
Py_XDECREF(op->func_doc);
+ Py_XDECREF(op->func_dict);
PyMem_DEL(op);
}
Re: Arbitrary attributes on funcs and methods [ In reply to ]
guido at python
Apr 10, 2000, 9:02 AM
Post #2 of 42 (2125 views)
Permalink
> A number of people have played FAST and loose with function and method
> docstrings, including John Aycock[1], Zope's ORB[2]. Docstrings are
> handy because they are the one attribute on funcs and methods that are
> easily writable. But as more people overload the semantics for
> docstrings, we'll get collisions. I've had a number of discussions
> with folks about adding attribute dictionaries to functions and
> methods so that you can essentially add any attribute. Namespaces are
> one honking great idea -- let's do more of those!
>
> Below is a very raw set of patches to add an attribute dictionary to
> funcs and methods. It's only been minimally tested, but if y'all like
> the idea, I'll clean it up, sanity check the memory management, and
> post the changes to patches@python.org. Here's some things you can
> do:
>
> -------------------- snip snip --------------------
> Python 1.6a2 (#10, Apr 10 2000, 11:27:59) [GCC 2.8.1] on sunos5
> Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
> >>> def a(): pass
> ...
> >>> a.publish = 1
> >>> a.publish
> 1
> >>> a.__doc__
> >>> a.__doc__ = 'a doc string'
> >>> a.__doc__
> 'a doc string'
> >>> a.magic_string = a.__doc__
> >>> a.magic_string
> 'a doc string'
> >>> dir(a)
> ['__doc__', '__name__', 'func_code', 'func_defaults', 'func_dict', 'func_doc', 'func_globals', 'func_name', 'magic_string', 'publish']
> >>> class F:
> ... def a(self): pass
> ...
> >>> f = F()
> >>> f.a.publish
> Traceback (most recent call last):
> File "<stdin>", line 1, in ?
> AttributeError: publish
> >>> f.a.publish = 1
> >>> f.a.publish
> 1

Here I have a question. Should this really change F.a, or should it
change the method bound to f only? You implement the former, but I'm
not sure if those semantics are right -- if I have two instances, f1
and f2, and you change f2.a.spam, I'd be surprised if f1.a.spam got
changed as well (since f1.a and f2.a are *not* the same thing -- they
are not shared. f1.a.im_func and f2.a.im_func are the same thing, but
f1.a and f2.a are distinct!

I would suggest that you only allow setting attributes via the class
or via a function. (This means that you must still implement the
pass-through on method objects, but reject it if the method is bound
to an instance.)

--Guido van Rossum (home page: http://www.python.org/~guido/)
Re: Arbitrary attributes on funcs and methods [ In reply to ]
gstein at lyra
Apr 10, 2000, 11:13 AM
Post #3 of 42 (2127 views)
Permalink
On Mon, 10 Apr 2000, Barry Warsaw wrote:
>...
> Below is a very raw set of patches to add an attribute dictionary to
> funcs and methods. It's only been minimally tested, but if y'all like
> the idea,

+1 on concept, -1 on the patch :-)

>...
> P.S. I promised to add a little note about setattr and getattr
> vs. setattro and getattro. There's very little documentation about
> the differences, and searching on python.org doesn't seem to turn up
> anything. The differences are simple. setattr/getattr take a char*
> argument naming the attribute to change, while setattro/getattro take
> a PyObject* (hence the trailing `o' -- for Object). This stuff should
> get documented in the C API, but at least now, it'll turn up in a SIG
> search. :)

And note that the getattro/setattro is preferred. It is easy to extract
the char* from them; the other direction requires construction of an
object.

>...
> + static int
> + instancemethod_setattr(im, name, v)
> + register PyMethodObject *im;
> + char *name;
> + PyObject *v;

IMO, this should be instancemethod_setattro() and take a PyObject *name.
In the function, you can extract the string for comparison.

>...
> + {
> + int rtn;

This variable isn't used.

>...
> static PyObject *
> instancemethod_getattr(im, name)
> register PyMethodObject *im;
> ! char *name;

IMO, this should remain a getattro function. (and fix the name)

In your update, note how many GetAttrString calls there are. The plain
GetAttr is typically faster.

>...
> + rtn = PyMember_Get((char *)im, instancemethod_memberlist, name);
> + if (rtn == NULL) {
> + PyErr_Clear();
> + rtn = PyObject_GetAttrString(im->im_func, name);
> + if (rtn == NULL)
> + PyErr_SetString(PyExc_AttributeError, name);

Why do you mask this second error with the AttributeError? Seems that you
should just leave whatever is there (typically an AttributeError, but
maybe not!).

>...
> --- 144,167 ----
> PyFunctionObject *op;
> char *name;
> {
> + PyObject* rtn;
> +
> if (name[0] != '_' && PyEval_GetRestricted()) {
> PyErr_SetString(PyExc_RuntimeError,
> "function attributes not accessible in restricted mode");
> return NULL;
> + }
> + if (strcmp(name, "__dict__") == 0)
> + return op->func_dict;

This is superfluous. The PyMember_Get will do this.

> + rtn = PyMember_Get((char *)op, func_memberlist, name);
> + if (rtn == NULL) {
> + PyErr_Clear();
> + rtn = PyDict_GetItemString(op->func_dict, name);
> + if (rtn == NULL)
> + PyErr_SetString(PyExc_AttributeError, name);

Again, with the masking...

>...
> + else if (strcmp(name, "func_dict") == 0) {
> + if (value == NULL || !PyDict_Check(value)) {
> + PyErr_SetString(
> + PyExc_TypeError,
> + "func_dict must be set to a dict object");

This raises an interesting thought. Why not just require the mapping
protocol?

Cheers,
-g

--
Greg Stein, http://www.lyra.org/
Re: Arbitrary attributes on funcs and methods [ In reply to ]
bwarsaw at python
Apr 10, 2000, 4:48 PM
Post #4 of 42 (2125 views)
Permalink
> Below is a very raw set of patches to add an attribute dictionary to
> funcs and methods. It's only been minimally tested, but if y'all like
> the idea,

>>>>> "GS" == Greg Stein <gstein@lyra.org> writes:


Well, that's good, because I /knew/ the patch was a quick hack (which
is why I posted it to python-dev and not patches :). Since there's
been generally positive feedback on the idea, I think I'll flesh it
out a bit.


Good point.

>...
> + rtn = PyMember_Get((char *)im, instancemethod_memberlist, name);
> + if (rtn == NULL) {
> + PyErr_Clear();
> + rtn = PyObject_GetAttrString(im->im_func, name);
> + if (rtn == NULL)
> + PyErr_SetString(PyExc_AttributeError, name);


Good point here, but...

> + rtn = PyMember_Get((char *)op, func_memberlist, name);
> + if (rtn == NULL) {
> + PyErr_Clear();
> + rtn = PyDict_GetItemString(op->func_dict, name);
> + if (rtn == NULL)
> + PyErr_SetString(PyExc_AttributeError, name);


...here I don't want the KeyError to leak through the getattr() call.
If you do "print func.non_existent_attr" wouldn't you want an
AttributeError instead of a KeyError? Maybe it should explicitly test
for KeyError rather than masking any error coming back from
PyDict_GetItemString()? Or better yet (based on your suggestion
below), it should do a PyMapping_HasKey() test, raise an
AttributeError if not, then just return PyMapping_GetItemString().

>...
> + else if (strcmp(name, "func_dict") == 0) {
> + if (value == NULL || !PyDict_Check(value)) {
> + PyErr_SetString(
> + PyExc_TypeError,
> + "func_dict must be set to a dict object");

GS> This raises an interesting thought. Why not just require the
GS> mapping protocol?

Good point again.

-Barry
Re: Arbitrary attributes on funcs and methods [ In reply to ]
gstein at lyra
Apr 10, 2000, 6:12 PM
Post #5 of 42 (2131 views)
Permalink
On Mon, 10 Apr 2000 bwarsaw@python.org wrote:
>...
> >...
> > + rtn = PyMember_Get((char *)im, instancemethod_memberlist, name);
> > + if (rtn == NULL) {
> > + PyErr_Clear();
> > + rtn = PyObject_GetAttrString(im->im_func, name);
> > + if (rtn == NULL)
> > + PyErr_SetString(PyExc_AttributeError, name);
>
> GS> Why do you mask this second error with the AttributeError?
> GS> Seems that you should just leave whatever is there (typically
> GS> an AttributeError, but maybe not!).
>
> Good point here, but...
>
> > + rtn = PyMember_Get((char *)op, func_memberlist, name);
> > + if (rtn == NULL) {
> > + PyErr_Clear();
> > + rtn = PyDict_GetItemString(op->func_dict, name);
> > + if (rtn == NULL)
> > + PyErr_SetString(PyExc_AttributeError, name);
>
> GS> Again, with the masking...
>
> ...here I don't want the KeyError to leak through the getattr() call.

Ah! Subtle difference in the code there :-)

I agree with you, on remapping the second one. I don't think the first
needs to be remapped, however.

> If you do "print func.non_existent_attr" wouldn't you want an
> AttributeError instead of a KeyError? Maybe it should explicitly test
> for KeyError rather than masking any error coming back from
> PyDict_GetItemString()? Or better yet (based on your suggestion
> below), it should do a PyMapping_HasKey() test, raise an
> AttributeError if not, then just return PyMapping_GetItemString().

Seems that you could just do the PyMapping_GetItemString() and remap the
error *if* it occurs. Presumably, the exception is the infrequent case and
can stand to be a bit slower.

Cheers,
-g

--
Greg Stein, http://www.lyra.org/
Re: Arbitrary attributes on funcs and methods [ In reply to ]
gstein at lyra
Apr 10, 2000, 6:37 PM
Post #6 of 42 (2128 views)
Permalink
On Mon, 10 Apr 2000 bwarsaw@python.org wrote:
>...
> GS> And note that the getattro/setattro is preferred. It is easy
> GS> to extract the char* from them; the other direction requires
> GS> construction of an object.
>
> Good point.

Oh. Also, I noticed that you removed a handy optimization from the getattr
function. Testing a character for '_' *before* calling strcmp() will save
a good chunk of time, especially considering how often this function is
used. Basically, review whether a quick test can save a strmp() call (and
can be easily integrated).

Cheers,
-g

--
Greg Stein, http://www.lyra.org/
Re: Arbitrary attributes on funcs and methods [ In reply to ]
bwarsaw at python
Apr 10, 2000, 10:08 PM
Post #7 of 42 (2126 views)
Permalink
>>>>> "GvR" == Guido van Rossum <guido@python.org> writes:

GvR> Here I have a question. Should this really change F.a, or
GvR> should it change the method bound to f only? You implement
GvR> the former, but I'm not sure if those semantics are right --
GvR> if I have two instances, f1 and f2, and you change f2.a.spam,
GvR> I'd be surprised if f1.a.spam got changed as well (since f1.a
GvR> and f2.a are *not* the same thing -- they are not shared.
GvR> f1.a.im_func and f2.a.im_func are the same thing, but f1.a
GvR> and f2.a are distinct!

As are f1.a and f1.a! :)

GvR> I would suggest that you only allow setting attributes via
GvR> the class or via a function. (This means that you must still
GvR> implement the pass-through on method objects, but reject it
GvR> if the method is bound to an instance.)

Given that, Python should probably raise a TypeError if an attempt is
made to set an attribute on a bound method object. However, it should
definitely succeed to /get/ an attribute on a bound method object.
I'm not 100% sure that setting bound-method-attributes should be
illegal, but we can be strict about it now and see if it makes sense
to loosen the restriction later.

Here's a candidate for Lib/test/test_methattr.py which should print a
bunch of `1's. I'll post the revised diffs (taking into account GvR's
and GS's suggestions) tomorrow after I've had a night to sleep on it.

-Barry

-------------------- snip snip --------------------
from test_support import verbose

class F:
def a(self):
pass

def b():
pass

# setting attributes on functions
try:
b.blah
except AttributeError:
pass
else:
print 'did not get expected AttributeError'

b.blah = 1
print b.blah == 1
print 'blah' in dir(b)

# setting attributes on unbound methods
try:
F.a.blah
except AttributeError:
pass
else:
print 'did not get expected AttributeError'

F.a.blah = 1
print F.a.blah == 1
print 'blah' in dir(F.a)

# setting attributes on bound methods is illegal
f1 = F()
try:
f1.a.snerp = 1
except TypeError:
pass
else:
print 'did not get expected TypeError'

# but accessing attributes on bound methods is fine
print f1.a.blah
print 'blah' in dir(f1.a)

f2 = F()
print f1.a.blah == f2.a.blah

F.a.wazoo = F
f1.a.wazoo is f2.a.wazoo

# try setting __dict__ illegally
try:
F.a.__dict__ = (1, 2, 3)
except TypeError:
pass
else:
print 'did not get expected TypeError'

F.a.__dict__ = {'one': 111, 'two': 222, 'three': 333}
print f1.a.two == 222

from UserDict import UserDict
d = UserDict({'four': 444, 'five': 555})
F.a.__dict__ = d

try:
f2.a.two
except AttributeError:
pass
else:
print 'did not get expected AttributeError'

print f2.a.four is f1.a.four is F.a.four
Re: Arbitrary attributes on funcs and methods [ In reply to ]
Vladimir.Marangozov at inrialpes
Apr 12, 2000, 2:47 AM
Post #8 of 42 (2131 views)
Permalink
Barry Warsaw wrote:
>
> A number of people have played FAST and loose with function and method
> docstrings, including John Aycock[1], Zope's ORB[2]. Docstrings are
> handy because they are the one attribute on funcs and methods that are
> easily writable. But as more people overload the semantics for
> docstrings, we'll get collisions. I've had a number of discussions
> with folks about adding attribute dictionaries to functions and
> methods so that you can essentially add any attribute. Namespaces are
> one honking great idea -- let's do more of those!

Barry, I wonder what for...

Just because there's a Python entity implemented as a C structure
in which we can easily include a dict + access functions?

I don't see the purpose of attaching state (vars) to an algorithm
(i.e. a function). What are the benefits compared to class instances?

And these special assignment rules because of the real overlap with
real instances... Grrr, all this is pretty dark, conceptually.

Okay, I inderstood: modules become classes, functions become instances,
module variables are class variables, and classes become ... 2-nd order
instances of modules. The only missing piece of the puzzle is a legal
way to instantiate modules for obtaining functions and classes dynamically,
because using eval(), the `new' module or The Hook is perceived as very
hackish and definitely not OO.

Once the puzzle would be solved, we'll discover that there would be only
one additional little step towards inheritance for modules. How weird!
Sounds like we're going to metaclass again...

-1 until P3K.

This is no so cute as it is dangerous. It opens the way to mind abuse.

--
Vladimir MARANGOZOV | Vladimir.Marangozov@inrialpes.fr
http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252
Re: Arbitrary attributes on funcs and methods [ In reply to ]
fredrik at pythonware
Apr 12, 2000, 3:04 AM
Post #9 of 42 (2127 views)
Permalink
Vladimir Marangozov wrote:
> I don't see the purpose of attaching state (vars) to an algorithm
> (i.e. a function). What are the benefits compared to class instances?
>
> And these special assignment rules because of the real overlap with
> real instances... Grrr, all this is pretty dark, conceptually.
>
> -1 until P3K.

I agree.

</F>
Re: Arbitrary attributes on funcs and methods [ In reply to ]
paul at prescod
Apr 12, 2000, 5:20 AM
Post #10 of 42 (2125 views)
Permalink
Vladimir Marangozov wrote:
>
> ...
>
> I don't see the purpose of attaching state (vars) to an algorithm
> (i.e. a function).

A function is also an object.

> What are the benefits compared to class instances?

If I follow you, you are saying that whenever you need to associate
information with a function, you should wrap up the function and object
into a class. But the end result of this transformation could be a
program in which every single function is a class. That would be
incredibly annoying, especially with Python's scoping rules. In general,
it may not even be possible.

Consider the following cases:

* I need to associate a Java-style type declaration with a method so
that it can be recognized based on its type during Java method dispatch.
How would you do that with instances?

* I need to associate a "grammar rule" with a Python method so that the
method is invoked when the parser recognizes a syntactic construct in
the input data.

* I need to associate an IDL declaration with a method so that a COM
interface definition can be generated from the source file.

* I need to associate an XPath "pattern string" with a Python method so
that the method can be invoked when a tree walker discovers a particular
pattern in an XML DOM.

* I need to associate multiple forms of documentation with a method.
They are optimized for different IDEs, environments or languages.

> And these special assignment rules because of the real overlap with
> real instances... Grrr, all this is pretty dark, conceptually.

I don't understand what you are saying here.

> Once the puzzle would be solved, we'll discover that there would be only
> one additional little step towards inheritance for modules. How weird!
> Sounds like we're going to metaclass again...

I don't see what any of this has to do with Barry's extremely simple
idea. Functions *are objects* in Python. It's too late to change that.
Objects can have properties. Barry is just allowing arbitrary properties
to be associated with functions. I don't see where there is anything
mysterious here.

--
Paul Prescod - ISOGEN Consulting Engineer speaking for himself
"I and my companions suffer from a disease of the heart that can only
be cured with gold", Hernan Cortes
Re: Arbitrary attributes on funcs and methods [ In reply to ]
akuchlin at mems-exchange
Apr 12, 2000, 5:22 AM
Post #11 of 42 (2127 views)
Permalink
Vladimir Marangozov writes:
>Barry, I wonder what for...

In the two quoted examples, docstrings are used to store additional
info about a function. SPARK uses them to contain grammar rules and
the regular expressions for matching tokens. The object publisher in
Zope uses the presence of a docstring to indicate whether a function
or method is publicly accessible. As a third example, the optional
type info being thrashed over in the Types-SIG would be another
annotation for a function (though doing def f(): ... f.type = 'void'
would be really clunky.

>Once the puzzle would be solved, we'll discover that there would be only
>one additional little step towards inheritance for modules. How weird!
>Sounds like we're going to metaclass again...

No, that isn't why Barry is experimenting with this -- instead, it's
simply because annotating functions seems useful, but everyone uses
the docstring because it's the only option.

--amk
Re: Arbitrary attributes on funcs and methods [ In reply to ]
fredrik at pythonware
Apr 12, 2000, 5:50 AM
Post #12 of 42 (2132 views)
Permalink
Paul Prescod wrote:
> * I need to associate a Java-style type declaration with a method so
> that it can be recognized based on its type during Java method dispatch.

class foo:
typemap = {}
def myfunc(self):
pass
typemap[myfunc] = typeinfo

> * I need to associate a "grammar rule" with a Python method so that the
> method is invoked when the parser recognizes a syntactic construct in
> the input data.

class foo:
rules = []
def myfunc(self):
pass
rules.append(pattern, myfunc)

> * I need to associate an IDL declaration with a method so that a COM
> interface definition can be generated from the source file.

class foo:
idl = {}
def myfunc(self):
pass
idl[myfunc] = "declaration"

> * I need to associate an XPath "pattern string" with a Python method so
> that the method can be invoked when a tree walker discovers a particular
> pattern in an XML DOM.

class foo:
xpath = []
def myfunc(self):
pass
xpath.append("pattern", myfunc)

</F>
Re: Arbitrary attributes on funcs and methods [ In reply to ]
tismer at tismer
Apr 12, 2000, 7:43 AM
Post #13 of 42 (2130 views)
Permalink
Fredrik Lundh wrote:
>
> Paul Prescod wrote:
> > * I need to associate a Java-style type declaration with a method so
> > that it can be recognized based on its type during Java method dispatch.
>
> class foo:
> typemap = {}
> def myfunc(self):
> pass
> typemap[myfunc] = typeinfo

Yes, I know that nearly everything is possible to be emulated
via classes. But what is so bad about an arbitrary function
attribute?

ciao - chris

p.s.: Paul, did you know that you can use *anything* for __doc__?
You could use a class instance instead which still serves
as a __doc__ but has your attributes and more.
Yes I know this is ugly :-))

--
Christian Tismer :^) <mailto:tismer@appliedbiometrics.com>
Applied Biometrics GmbH : Have a break! Take a ride on Python's
Kaunstr. 26 : *Starship* http://starship.python.net
14163 Berlin : PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF
where do you want to jump today? http://www.stackless.com
Re: Arbitrary attributes on funcs and methods [ In reply to ]
bwarsaw at cnri
Apr 12, 2000, 8:37 AM
Post #14 of 42 (2128 views)
Permalink
Functions and methods are first class objects, and they already have
attributes, some of which are writable. Why should __doc__ be
special? Just because it was the first such attribute to have
syntactic support for easily defining? Think about my proposal this
way: it actual removes a restriction.

What I don't like about /F's approach is that if you were building a
framework, you'd now have two conventions you'd have to describe:
where to find the mapping, and what keys to use in that mapping. With
attributes, you've already got the former: getattr(). Plus, let's say
you're handed a method object `x', would you rather do:

if x.im_class.typemap[x.im_func] == 'int':
...

or

if x.__type__ == 'int':
...

And what about function objects (as opposed to unbound methods).
Where do you stash the typemap? In the module, I supposed. And if
you can be passed either type of object, do you now have to do this?

if hasattr(x, 'im_class'):
if hasattr(x.im_class, 'typemap'):
if x.im_class.typemap[x.im_func] == 'int':
...
elif hasattr(x, 'func_globals'):
if x.func_globals.has_key('typemap'):
if x.func_globals['typemap'][x] == 'int':
...

instead of the polymorphic elegance of

if x.__type__ == 'int':
...

Finally, think of this proposal as an evolutionary step toward
enabling all kinds of future frameworks. At some point, there may be
some kind of optional static type system. There will likely be some
syntactic support for easily specifying the contents of the __type__
attribute. With the addition of func/meth attrs now, we can start to
play with prototypes of this system, define conventions and standards,
and then later when there is compiler support, simplify the
definitions, but not have to change code that uses them.

-Barry
Re: Arbitrary attributes on funcs and methods [ In reply to ]
jeremy at cnri
Apr 12, 2000, 9:37 AM
Post #15 of 42 (2127 views)
Permalink
>>>>> "BAW" == Barry A Warsaw <bwarsaw@cnri.reston.va.us> writes:

BAW> Functions and methods are first class objects, and they already
BAW> have attributes, some of which are writable. Why should
BAW> __doc__ be special? Just because it was the first such
BAW> attribute to have syntactic support for easily defining?

I don't have a principled argument about why doc strings should be
special, but I think that they should be. I think it's weird that you
can change __doc__ at runtime; I would prefer that it be constant.

BAW> Think about my proposal this way: it actually removes a
BAW> restriction.

I think this is really the crux of the matter! The proposal removes
a useful restriction.

The alternatives /F suggested seem clearer to me that sticking new
attributes on functions and methods. Three things I like about the
approach: It affords an opportunity to be very clear about how the
attributes are intended to be used. I suspect it would be easier to
describe with a static type system. It prevents confusion and errors
that might result from unprincipled use of function attributes.

Jeremy
Re: Arbitrary attributes on funcs and methods [ In reply to ]
gmcm at hypernet
Apr 12, 2000, 9:56 AM
Post #16 of 42 (2127 views)
Permalink
Jeremy Hylton wrote:

> BAW> Think about my proposal this way: it actually removes a
> BAW> restriction.
>
> I think this is really the crux of the matter! The proposal removes
> a useful restriction.
>
> The alternatives /F suggested seem clearer to me that sticking new
> attributes on functions and methods. Three things I like about the
> approach: It affords an opportunity to be very clear about how the
> attributes are intended to be used. I suspect it would be easier to
> describe with a static type system.

Having to be explicit about the method <-> regex / rule would
severely damage SPARK's elegance. It would make Tim's
doctest useless.

> It prevents confusion and errors
> that might result from unprincipled use of function attributes.

While I'm sure I will be properly shocked and horrified when
you come up with an example, in my naivety, I can't imagine
what it will look like ;-).

- Gordon
Re: Arbitrary attributes on funcs and methods [ In reply to ]
skip at mojam
Apr 12, 2000, 10:28 AM
Post #17 of 42 (2128 views)
Permalink
BAW> Functions and methods are first class objects, and they already
BAW> have attributes, some of which are writable.

(Trying to read Fredrik's mind...)

By extension, we should allow writable attributes to work for other objects.
To pollute this discussion with an example from another one:

i = 3.1416
i.__precision__ = 4

I haven't actually got anything against adding attributes to functions (or
numbers, if it's appropriate). Just wondering out loud and playing a bit of
a devil's advocate.

Skip
Re: Arbitrary attributes on funcs and methods [ In reply to ]
moshez at math
Apr 12, 2000, 10:36 AM
Post #18 of 42 (2126 views)
Permalink
On Wed, 12 Apr 2000, Skip Montanaro wrote:

> To pollute this discussion with an example from another one:
>
> i = 3.1416
> i.__precision__ = 4
>

And voila! Numbers are no longer immutable. Using any numbers as keys in
dicts?
Moshe Zadka <mzadka@geocities.com>.
http://www.oreilly.com/news/prescod_0300.html
http://www.linux.org.il -- we put the penguin in .com
Re: Arbitrary attributes on funcs and methods [ In reply to ]
tismer at tismer
Apr 12, 2000, 10:39 AM
Post #19 of 42 (2127 views)
Permalink
Skip Montanaro wrote:
>
> BAW> Functions and methods are first class objects, and they already
> BAW> have attributes, some of which are writable.
>
> (Trying to read Fredrik's mind...)

takes too long since it isn't countable infinite...

> By extension, we should allow writable attributes to work for other objects.
> To pollute this discussion with an example from another one:
>
> i = 3.1416
> i.__precision__ = 4
>
> I haven't actually got anything against adding attributes to functions (or
> numbers, if it's appropriate). Just wondering out loud and playing a bit of
> a devil's advocate.

please let me join your hexensabbat

(de|en)lighted-ly -y'rs - rapunzel

--
Christian Tismer :^) <mailto:tismer@appliedbiometrics.com>
Applied Biometrics GmbH : Have a break! Take a ride on Python's
Kaunstr. 26 : *Starship* http://starship.python.net
14163 Berlin : PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF
where do you want to jump today? http://www.stackless.com
Re: Arbitrary attributes on funcs and methods [ In reply to ]
effbot at telia
Apr 12, 2000, 10:42 AM
Post #20 of 42 (2130 views)
Permalink
Moshe Zadka wrote:
> > To pollute this discussion with an example from another one:
> >
> > i = 3.1416
> > i.__precision__ = 4
>
> And voila! Numbers are no longer immutable. Using any
> numbers as keys in dicts?

so? you can use methods as keys today, you know...

</F>
Re: Arbitrary attributes on funcs and methods [ In reply to ]
moshez at math
Apr 12, 2000, 10:44 AM
Post #21 of 42 (2128 views)
Permalink
On Wed, 12 Apr 2000, Fredrik Lundh wrote:

> so? you can use methods as keys today, you know...

Actually, I didn't know. What hapens if you use a method as a key, and
then change it's doc string?
--
Moshe Zadka <mzadka@geocities.com>.
http://www.oreilly.com/news/prescod_0300.html
http://www.linux.org.il -- we put the penguin in .com
Re: Arbitrary attributes on funcs and methods [ In reply to ]
skip at mojam
Apr 12, 2000, 10:47 AM
Post #22 of 42 (2128 views)
Permalink
Moshe> On Wed, 12 Apr 2000, Skip Montanaro wrote:
>> To pollute this discussion with an example from another one:
>>
>> i = 3.1416
>> i.__precision__ = 4
>>

Moshe> And voila! Numbers are no longer immutable. Using any numbers as
Moshe> keys in dicts?

Yes, and I use functions on occasion as dict keys as well.

>>> def foo(): pass
...
>>> d = {foo: 1}
>>> print d[foo]
1

I suspect adding methods to functions won't invalidate their use in that
context, nor would adding attributes to numbers. At any rate, it was just
an example.

Skip
Re: Arbitrary attributes on funcs and methods [ In reply to ]
jeremy at cnri
Apr 12, 2000, 10:51 AM
Post #23 of 42 (2124 views)
Permalink
>>>>> "GMcM" == Gordon McMillan <gmcm@hypernet.com> writes:
[please imagine that the c is raised]

BAW> Think about my proposal this way: it actually removes a
BAW> restriction.
[Jeremy Hylton wrote:]
>> I think this is really the crux of the matter! The proposal
>> removes a useful restriction.
>>
>> The alternatives /F suggested seem clearer to me that sticking
>> new attributes on functions and methods. Three things I like
>> about the approach: It affords an opportunity to be very clear
>> about how the attributes are intended to be used. I suspect it
>> would be easier to describe with a static type system.

GMcM> Having to be explicit about the method <-> regex / rule would
GMcM> severely damage SPARK's elegance. It would make Tim's doctest
GMcM> useless.

Do either of these examples modify the __doc__ attribute? I am happy
to think of both of them as elegant abuses of the doc string. (Not
sure what semantics I mean for "elegant abuse" but not pejorative.)
I'm not arguing that we should change the language to prevent them
from using doc strings.

Fred and I were just talking, and he observed that a variant of Python
that included a syntactic mechanism to specify more than one attribute
(effectively, a multiple doc string syntax) might be less
objectionable than setting arbitrary attributes at runtime. Neither
of us could imagine just what that syntax would be.

>> It prevents confusion and errors that might result from
>> unprincipled use of function attributes.

GMcM> While I'm sure I will be properly shocked and horrified when
GMcM> you come up with an example, in my naivety, I can't imagine
GMcM> what it will look like ;-).

It would look really, really bad ;-). I couldn't think of a good
example, so I guess this is a FUD argument. A rough sketch, though,
would be a program that assigned attribute X to all functions that
were to be used in a certain way. If the assignment is a runtime
operation, rather than a syntactic construct that defines a static
attribute, it would be possible to accidentally assign attribute X to
a function that was not intended to be used that way. This connection
between a group of functions and a particular behavior would depend
entirely on some runtime magic with settable attributes.

Jeremy
Re: Arbitrary attributes on funcs and methods [ In reply to ]
mal at lemburg
Apr 12, 2000, 10:55 AM
Post #24 of 42 (2126 views)
Permalink
Jeremy Hylton wrote:
> BAW> Think about my proposal this way: it actually removes a
> BAW> restriction.
>
> I think this is really the crux of the matter! The proposal removes
> a useful restriction.

Not sure... I wouldn't mind having the ability to add attributes
to all Python objects at my own liking. Ok, maybe a bit far
fetched, but the idea would certainly be useful in some cases,
e.g. to add new methods to built-in types or to add encoding
name information to strings...

> The alternatives /F suggested seem clearer to me that sticking new
> attributes on functions and methods. Three things I like about the
> approach: It affords an opportunity to be very clear about how the
> attributes are intended to be used. I suspect it would be easier to
> describe with a static type system. It prevents confusion and errors
> that might result from unprincipled use of function attributes.

The nice side-effect of having these function/method instance
dictionaries is that they follow class inheritance. Something
which is hard to do right with Fredrik's approach.

I suspect that in Py3K we'll only have one type of class
system: everything inherits from one global base class -- seen
in that light, method attributes are really nothing unusual,
since all instances would have instance dictionaries anyway
(well maybe only on demand, but that's another story).

Anyway, more power to you Barry :-)

--
Marc-Andre Lemburg
______________________________________________________________________
Business: http://www.lemburg.com/
Python Pages: http://www.lemburg.com/python/
Re: Arbitrary attributes on funcs and methods [ In reply to ]
moshez at math
Apr 12, 2000, 10:55 AM
Post #25 of 42 (2129 views)
Permalink
On Wed, 12 Apr 2000, Gordon McMillan wrote:

> Bounded, however. And therefore, um, dense <duck>...

I sorta imagined it more like the Cantor set. Nowhere dense, but perfect
<wink>

sorry-but-he-started-with-the-maths-ly y'rs, Z.
--
Moshe Zadka <mzadka@geocities.com>.
http://www.oreilly.com/news/prescod_0300.html
http://www.linux.org.il -- we put the penguin in .com
1 2  View All

©2024 GT.net. A Gossamer Threads company.
5th Floor, 455 Granville St., Vancouver, BC V6C 1T1, Canada | Legal