Mailing List Archive

[garbage collection]

gvwilson@nevex.com wrote:

> I agree on all counts except garbage collection --- I'm half-way through
> the second day of the Python class I teach at Los Alamos (the people who
> are funding the Python tool design competition), and it's come up a couple
> of times. People want to be able to prototype meshes, throw callbacks
> around without worrying about circularity, and some other things that I
> don't really understand yet. There's also a couple of smart guys in the
> class who are wondering about CPython vs. JPython ("So this'll be safe in
> one version of the language, but not in the other?"), and about marketing
> ("Help me win a feature comparison against Java in my group...").

Guido once posted some proposal of a hybrid system *with* refcounts
and some additional garbage collection scheme to match circular
things. I believe this is a much better approach than what Java
and therefor also JPython does at the moment.

Although people might argue differently, I'm pretty sure that
reference counting is the stronger concept. By reference counting,
the idea of object ownership can be made explicit. This plays a
central role in the Corba specification for instance, and I made
the same observation when implementing continuations for Stackless
Python. Refcounts are no burden but a virtue.
Even better: Refcounting can lead to many new optimizations if
we pay the cost to make INCREF/DECREF into methods. It has its cost
(about 10 percent less pystones), but massive long-term benefits.

I'm currently in charge to develop a custom version of Python's
builtin types where this concept is used. Everything is refcounted,
but without storing the refcounts in the objects. This is possible
(proven) and will be shown in my next paper.

Conclusion: I vote for a kind of GC that does just what refcounts
cannot do, but please keep with the refcounts.

cheers - chris

--
Christian Tismer :^) <mailto:tismer@appliedbiometrics.com>
Virtual Photonics GmbH : Have a break! Take a ride on Python's
Carnotstr. 6 : *Starship* http://starship.python.net
10587 Berlin : PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF
we're tired of banana software - shipped green, ripens at home

> Conclusion: I vote for a kind of GC that does just what refcounts
> cannot do, but please keep with the refcounts.

The patch that I received and that has Tim's <0.5 blessing> does just
that. I haven't had the time to understand why it doesn't have his
<1.0 blessing>.

--Guido van Rossum (home page: http://www.python.org/~guido/)

[Christian]
> Conclusion: I vote for a kind of GC that does just
> what refcounts cannot do, but please keep with the
> refcounts.

[Guido]
> The patch that I received and that has Tim's <0.5
> blessing> does just that. I haven't had the time to
> understand why it doesn't have his <1.0 blessing>.

Primarily because it doesn't reclaim the most common cycles; e.g., cycles
among class instances aren't touched. This seems easily repairable, but at
an unknown cost (it needs to do the "reachability" transitive closure
business from the set of all "suspicious" objects, and instances are never
considered suspicious now; adding them will certainly cause a lot more
pointer chasing).

Apart from that, the code appears unreasonably expensive as written today,
using e.g. splay trees instead of hash tables to keep track of objects.

The author hasn't said anything more in a bit over two weeks, so I suspect
he's off on other things now. The technical approach is sound, but even its
inventor (Rafael Lins; Toby Kelsey may have reinvented it on his own,
though) stresses that getting it to run fast is difficult.

needs-work!-ly y'rs - tim, who hasn't the time to do it

Daddy Warbucks (uh, Greg Wilson =):

> There's also been questions about tighter integration of NumPy (e.g.
> overloading operators rather than calling 'greater()' to do comparison),
> but I think that's a separate discussion...

That's the rich comparison proposal which Guido mentioned.

--david

[Greg Wilson]
> There's also been questions about tighter integration
> of NumPy (e.g. overloading operators rather than
> calling 'greater()' to do comparison), but I think
? that's a separate discussion...

[David Ascher]
> That's the rich comparison proposal which Guido mentioned.

But there's also been talk about moving (at least) the basic NumPy array
type into the core. This would be a Good Thing. Speaking for my employer,
however, only Unicode is an Important Thing <wink>.

As a developer, I have railed against schedule-driven release cycles.
Python tends toward the opposite end of that spectrum, driven by features no
matter how bloody long they take. Add Unicode to what's already waiting to
go, and that's *good enough* reason for a major release; heck, it's been 9
months & we haven't even had a 1.5.2 bugfix patch.

BTW, do the Perl-Porters have real jobs?

pay-me-to-do-python-releases-and-you'll-get-a-major-new-
release-every-three-days<wink>-ly y'rs - tim

Tim Peters

> But there's also been talk about moving (at least) the basic NumPy array
> type into the core. This would be a Good Thing.

IMNSHO, moving the current NumPy array into the core would be a Bad Thing.
Moving a new similar object with cleaned up semantics and better
implementation in would be a Good Thing. But it won't happen until 1.7 at
the earliest, as the semantics haven't even been agreed on, let alone the
code written.

--david

[Greg Wilson]
> ...
> There's also a couple of smart guys in the class who are
> wondering about CPython vs. JPython ("So this'll be safe in
> one version of the language, but not in the other?"),

Greg, people who have been exposed to Fortran (this is LANL, right <wink>?)
can't possibly have a problem with the concept of "not defined by the
standard". Don't sell these as different *versions* of the language, but as
different implementations. That's what they are. The Python *language*
doesn't define anything about the lifetime of objects. Even when CPython
grows "real GC", thanks to refcounting too you'll still be able to *rely* on
behaviors in CPython you'll see only accidentally in JPython. You do so at
your own risk, same as e.g. you rely on floating point Fortran x+y+z getting
evaluated "left to right" at your own risk (BTW, x+y+z *is* "left to right"
in Python -- maybe they'll trade that for the lack of GC promises <wink>).

[hristian Tismer]
> ...
> Even better: Refcounting can lead to many new optimizations
> if we pay the cost to make INCREF/DECREF into methods. It
> has its cost (about 10 percent less pystones), but massive
> long-term benefits.

The GC patch Guido forced <wink> me to look at is based on the observation
that it's impossible to create cyclic trash unless a decref leaves a nonzero
refcount. So the patch adds a function call to the DECREF macro (if the new
refcount is > 0, the object must be added to the set of "suspicious"
objects; else the object must be removed from that set).

So it roughly adds the cost of a method call to each decref anyway. You
would think it adds less <wink>, but "the set" now is represented as a splay
tree, so *gobs* of hairy code get executed in either case (splay trees do
mutating rotations even on a lookup).

> ...
> Conclusion: I vote for a kind of GC that does just what
> refcounts cannot do, but please keep with the refcounts.

I like 'em too!

BTW, Toby posted (at least an earlier version of) the patch to c.l.py, if
anyone else wants to dig into it (I may have mentioned before that I'm short
on time <wink>).