Mailing List Archive

On Thu, Aug 28, 2008 at 8:54 PM, Phillip J. Eby <pje@telecommunity.com> wrote:
> I created a "universal metaclass" in
> DecoratorTools whose sole function is to delegate metaclass __new__,
> __init__, and __call__ to class-level methods (e.g. __class_new__,
> __class_call__, etc.), thereby eliminating the need to have custom
> metaclasses for most use cases in the first place. Now, wherever possible,
> I use that single metaclass in my frameworks, so that there's no need to mix
> them.

easy_installed DecoratorTools and found it: classy_class.
>From the point of view of the code, this is a beautiful and elegant
snippet. However, suppose that from tomorrow everybody starts
using it. Since metaclasses would become so easy to use, possibly a
lot of people would take advantage of them. Then we would have
potentially complex (multiple) inheritance hierarchies with
chains of methods (_class__new__/_class__init__) calling
themselves cooperatively in the MRO. Would the resulting
code be readable? How easy would be for an average framework user
to understand what is happening to his class?
I think class decorators would be a much better solution than
classy_class for most use cases and we should push that way,
not the cooperative inheritance way.

Generally speaking I like
more solutions bases on functional composition (as in WSGI
that you know very well) than on method cooperation. Rather than
improve the support for inheritance, I would like (in an ideal
world) to reduce it, to make easier the choice for people between
inheritance and alternatives (object composition, delegation, functional
composition). In the real world, I am content in documenting the
pitfalls of super, warn people about the dangers of complex
design involving multiple inheritance and cooperation, and suggest
alternatives.

Michele Simionato
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

At 06:07 AM 8/29/2008 +0200, Michele Simionato wrote:
>On Thu, Aug 28, 2008 at 8:54 PM, Phillip J. Eby <pje@telecommunity.com> wrote:
> > I created a "universal metaclass" in
> > DecoratorTools whose sole function is to delegate metaclass __new__,
> > __init__, and __call__ to class-level methods (e.g. __class_new__,
> > __class_call__, etc.), thereby eliminating the need to have custom
> > metaclasses for most use cases in the first place. Now, wherever possible,
> > I use that single metaclass in my frameworks, so that there's no
> need to mix
> > them.
>
>easy_installed DecoratorTools and found it: classy_class.
> >From the point of view of the code, this is a beautiful and elegant
>snippet. However, suppose that from tomorrow everybody starts
>using it. Since metaclasses would become so easy to use, possibly a
>lot of people would take advantage of them.

That was sort of the idea. ;-)

> Then we would have
>potentially complex (multiple) inheritance hierarchies with
>chains of methods (_class__new__/_class__init__) calling
>themselves cooperatively in the MRO. Would the resulting
>code be readable?

Readability's orthogonal. Some of them might be readable, some
not. Depends on who's writing them. :)


>How easy would be for an average framework user
>to understand what is happening to his class?

You're right, let's abolish inheritance, too, because then you might
have to read more than one class to see what's happening.


>I think class decorators would be a much better solution than
>classy_class for most use cases

Obviously, I disagree. :) You'll notice that DecoratorTools
supports class decorators for Python 2.3 and up (actually, I think
that particular bit worked in 2.2 as well). So, it's not the absence
of class decorators that motivated the 'classy' mixin.


>Generally speaking I like
>more solutions bases on functional composition (as in WSGI
>that you know very well) than on method cooperation. Rather than
>improve the support for inheritance, I would like (in an ideal
>world) to reduce it, to make easier the choice for people between
>inheritance and alternatives (object composition, delegation, functional
>composition). In the real world, I am content in documenting the
>pitfalls of super, warn people about the dangers of complex
>design involving multiple inheritance and cooperation, and suggest
>alternatives.

Naturally, if you can design a system to use delegates instead of
class hierarchy to represent a chain of responsibility, it might well
be an improvement. But there are tradeoffs, and no matter what you
are going to end up coding chains of responsibility. Co-operative
inheritance is a nice solution for chains of responsibility that can
be expressed in a class hierarchy, and are no more "dangerous" than
any other sort of chain of responsibility. In fact, they are in some
ways less so since the patterns are likely to be better documented
than anything you come up with on your own.

_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On Fri, Aug 29, 2008 at 6:22 AM, Phillip J. Eby <pje@telecommunity.com> wrote:
> You're right, let's abolish inheritance, too, because then you might have to
> read more than one class to see what's happening.

You are joking, but I actually took this idea quite seriously. Once
(four years ago or so) I did implement an object system from scratch
in Scheme, completely without inheritance, to see how far it would
go. It didn't go far, of course (nor I did expect it to go very far) but
at least I learned exactly what (single) inheritance was good for.
OTOH, for what concerns multiple inheritance, I am still not
convinced it is really worth it. I mean, the MRO is beautiful,
elegant and all that on paper, but on real-life code things as different,
especially from the side of the users of frameworks heavily
based on inheritance.

> Naturally, if you can design a system to use delegates instead of class
> hierarchy to represent a chain of responsibility, it might well be an
> improvement. But there are tradeoffs, and no matter what you are going to
> end up coding chains of responsibility.

Agreed, it is all about tradeoffs. We have a different opinion on what
a good tradeoff is in this case, but that's fine. I guess it depends
on personal experience and the kind of code one has to work with.
For instance I never had to integrated different frameworks
using different metaclasses in my daily work, so I don't see
a very strong case for classy_class over class decorators,
but I could change my mind in the future, who knows?

Anyway, It would be nice to have a good simple *real life*
use case of cooperative inheritance not involving metaclasses,
suitable for a beginners' tutorial about super, but I haven't
found one yet :-(

M.S.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

Michele Simionato wrote:
> OTOH, for what concerns multiple inheritance, I am still not
> convinced it is really worth it. I mean, the MRO is beautiful,
> elegant and all that on paper, but on real-life code things as different,
> especially from the side of the users of frameworks heavily
> based on inheritance.

The mixin methods in the ABC machinery would be a lot less useful
without multiple inheritance (and the collections ABCs would be a whole
lot harder to define and to write).

So if you're looking for use cases for multiple inheritance, I'd suggest
starting with the Python 2.6 collections module and seeing how you would
go about rewriting it using only single inheritance. I believe the new
io module is also fairly dependent on multiple inheritance.

Cheers,
Nick.

--
Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
---------------------------------------------------------------
http://www.boredomandlaziness.org
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On Fri, Aug 29, 2008 at 6:15 PM, Nick Coghlan <ncoghlan@gmail.com> wrote:
> The mixin methods in the ABC machinery would be a lot less useful
> without multiple inheritance (and the collections ABCs would be a whole
> lot harder to define and to write).
>
> So if you're looking for use cases for multiple inheritance, I'd suggest
> starting with the Python 2.6 collections module and seeing how you would
> go about rewriting it using only single inheritance. I believe the new
> io module is also fairly dependent on multiple inheritance.

I am very well aware of the collection module and the ABC mechanism.
However, you are missing that mixins can be implemented in a single-inheritance
world without the complications of the MRO. See my answer to Alex
Martelli in this same thread.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On Aug 29, 2008, at 11:46 AM, Michele Simionato wrote:

> On Fri, Aug 29, 2008 at 6:15 PM, Nick Coghlan <ncoghlan@gmail.com>
> wrote:
>> The mixin methods in the ABC machinery would be a lot less useful
>> without multiple inheritance (and the collections ABCs would be a
>> whole
>> lot harder to define and to write).
>>
>> So if you're looking for use cases for multiple inheritance, I'd
>> suggest
>> starting with the Python 2.6 collections module and seeing how you
>> would
>> go about rewriting it using only single inheritance. I believe the
>> new
>> io module is also fairly dependent on multiple inheritance.
>
> I am very well aware of the collection module and the ABC mechanism.
> However, you are missing that mixins can be implemented in a single-
> inheritance
> world without the complications of the MRO. See my answer to Alex
> Martelli in this same thread.

As interesting as this conversation is at a meta-level, I'm not sure
how much more can be accomplished here by debating the merits of
multiple vs. single inheritance. Unfortunately I think this is a case
where there is not just one good way to do it in all cases, especially
given the subjective nature of "good" in this context.

This is what I take away from this:

- super() is tricky to use at best, and its documentation is
inaccurate and incomplete. I think it should also be made more clear
that super() is really mostly useful for framework developers,
including users extending frameworks. Unfortunately many frameworks
require you to extend them in order to write useful applications in my
experience, so it trickles down to the app developer at times. In
short, more correct documentation == good.

- The difficulties of super() are really symptomatic of the
difficulties of multiple inheritance. I think it's clear that multiple
inheritance is here to stay in Python, and it solves certain classes
of problems quite well. But, it requires careful consideration, and
it's easy to get carried away and create a huge mess (ala Zope2, which
I am all too familiar), and it can hinder code clarity as much as it
facilitates reuse.

- There are good alternatives to multiple inheritance for many cases,
but there are cases where multiple inheritance is arguably best.
Traits are a possible alternative that deserve further study. I think
that study would be greatly aided by a 3rd-party library implementing
traits for Python. If traits are to gain any traction or ever be
considered for inclusion into the language such a library would need
to exist first and demonstrate its utility.

I know I'm probably just stating the obvious here, but I found it
therapeutic ;^)

-Casey
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On 06:33 pm, casey@pandora.com wrote:
>On Aug 29, 2008, at 11:46 AM, Michele Simionato wrote:
>>On Fri, Aug 29, 2008 at 6:15 PM, Nick Coghlan <ncoghlan@gmail.com>
>>wrote:

>>I am very well aware of the collection module and the ABC mechanism.
>>However, you are missing that mixins can be implemented in a single-
>>inheritance
>>world without the complications of the MRO. See my answer to Alex
>>Martelli in this same thread.

>- super() is tricky to use at best, and its documentation is
>inaccurate and incomplete. I think it should also be made more clear
>that super() is really mostly useful for framework developers,
>including users extending frameworks. Unfortunately many frameworks
>require you to extend them in order to write useful applications in my
>experience, so it trickles down to the app developer at times. In
>short, more correct documentation == good.

>I know I'm probably just stating the obvious here, but I found it
>therapeutic ;^)

I think this is a problem with this topic. Everyone writing about
super() seems to be not just clearing up the documentation issues that
surround it, but venting from personal frustrations with using it as
well. I confess that I have done the same - if not in widely-publicized
articles, at least on IRC and mailing list posts.

I think it would benefit everyone if this discussion would end up with
some patches to the library documentation that documented the semantics
of super() more completely in the reference documentation and the
"multiple inheritance" area of the tutorial, so that when people *do*
run in to difficulties there is a very clear, central explanation of
what it's supposed to do.

Personally I think the thing that really should be pointed out is that
it may behave in a confusing manner if the signature of the method being
invoked is not the same on all classes in the same inheritance
hierarchy. Theoretical problems aside, 99% of the trouble I've had with
super() has to do with __init__ methods.

I'll try my hand at such a patch over the weekend, but I'd be grateful
for some pointers on a "quick start" for that. I am a complete newb at
modifying the official documentation, and I seem to recall from a prior
(failed) attempt that the tools are a bit difficult to work with.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

From: <glyph@divmod.com>
> I think it would benefit everyone if this discussion would end up with
> some patches to the library documentation that documented the semantics
> of super() more completely in the reference documentation and the
> "multiple inheritance" area of the tutorial, so that when people *do*
> run in to difficulties there is a very clear, central explanation of
> what it's supposed to do.

Am working on a doc patch, will post this weekend.


Raymond
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On Fri, Aug 29, 2008 at 8:33 PM, Casey Duncan <casey@pandora.com> wrote:
> - There are good alternatives to multiple inheritance for many cases, but
> there are cases where multiple inheritance is arguably best.

Maybe, but I am still biased in the opposite direction ;)

>Traits are a
> possible alternative that deserve further study. I think that study would be
> greatly aided by a 3rd-party library implementing traits for Python. If
> traits are to gain any traction or ever be considered for inclusion into the
> language such a library would need to exist first and demonstrate its
> utility.

I wrote a trait library which I plan to release soon or later. However
it is intended as a proof of concept, not as a candidate for inclusion
in the standard library. As of now, I don't think we should have
a different way of doing mixins in the standard library. There should
be only one obvious way and the obvious way in current Python
is multiple inheritance as it is now. The proof of concept is
important for educational purposes, to open the mind to
alternatives, to give inspiration to the creators of new languages:
it is not intended to add complication (whenever small) to
current Python. Having said that, maybe once I release the
library people will start using it in production and will ask
for inclusion for the standard library, but this is not my goal
now. This happened for my decorator module years ago:
when I wrote it I did not expect people to use it, I saw it
as a temporary hack until we got a better support for
fiddling with function signatures in the standard library.
Nevertheless now a lot of people are using it and I am
not even sure it is a good thing (I have seen many decorator
abuses out there). This the strange thing that happens when
you release a library: you will never now what people will
end up using it for ;)

Michele Simionato
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

On Sat, Aug 30, 2008 at 6:16 AM, Michele Simionato
> I wrote a trait library which I plan to release soon or later.

Ok, just for the people here that cannot wait I have prepared a pre-alpha
snapshot and uploaded it to my site:

http://www.phyast.pitt.edu/~micheles/python/strait.html

At some moment I want to release it officially, but as of now
I do not feel I have nailed out all the details and there may be
difficulties I have not foreseen. If it is so, I am sure Phillip
will find out all the loopholes ;)

Nevertheless, I feel that I have covered out a lot of use
cases that I cared to cover, and that there is something good
in there, so have fun with this foolish attempt of putting multiple
inheritance straight! ;-)

Michele Simionato
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

It seems that the frustration with super revolves around how Python
currently conflates (as well as many users) two very different types
of inheritance, both "is-a" and "has-a" (or compositional)
inheritance. Unfortunately, Python assists this confusion because the
language doesn't provide a distinct enough way to differentiate
between them.

For the former (i.e. is-a inheritance), users should not have to
explicitly make a call to the super class (via super() or
otherwise)--this should be automatic for every method held in common
so as to guarantee the invariants of the simpler class. For this type
of inheritance it makes more sense to be explicit about *bypassing* a
call to the parent class--IF necessary (perhaps to take advantage of
an optimization available only in the subclass)--rather than explicit
about *making* one. This would remove many of the bugs, misuses, and
frustrations with super(). No more ambiguity and confusion because
the language now guarantees certain behavior. Super() would then be
limited to cooperative mixin classes; for all others, explicit naming
of the class would be required. But perhaps that there is some
language abstraction that has yet to be fully invented that would tie
it all together nicely.

By the way, regarding your trait implementation, it may be worthwhile
revisiting the discussion surrounding the Prothon language discussed a
few years ago on comp.lang.python which got rid of classes and
metaclasses and replaced them with prototypes.

Regards,

zipher
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com

average wrote:
> It seems that the frustration with super revolves around how Python
> currently conflates (as well as many users) two very different types
> of inheritance, both "is-a" and "has-a" (or compositional)
> inheritance. Unfortunately, Python assists this confusion because the
> language doesn't provide a distinct enough way to differentiate
> between them.

has-a should be modelling with attributes, not inheritance. The latter
relationship should always mean is-a (even for mixins).

Cheers,
Nick.

--
Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
---------------------------------------------------------------
http://www.boredomandlaziness.org
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/list-python-dev%40lists.gossamer-threads.com