Mailing List Archive

On 10 Jun 1999 09:43:07 +0900, Hisao Suzuki <suzuki611@okisoft.co.jp> wrote:
>In article <87pv384koy.fsf@ev.netlab.co.jp>,
>Yukihiro Matsumoto <matz@netlab.co.jp> wrote:
>| I claimed in summary:
>| * I don't like mere ref counting; it cannot reclaim cyclic data, and
>| easy to forget INCREF/DECREF.
>| * Many among Python users misunderstood about real GC.
>| (slow, hangs, non portable, etc.)
>| * ref counting with GC may be good for Python 2.0.
>| Did I do any advocacy on Ruby? I was trying constructive discussion.
>I'm sorry to say, but your argument is too trivial (or rather
>naive) to consider as a constructive discussion. Having almost
>nothing to do with real problems on Python, your referrings to
>your Ruby (surely in triumphant tones) sound evangelic. I'd be
>glad if you would present a non-trivial viewpoint or propose a
>practical solution _relevant_ to Python actually.

I think this is a little harsh. I've been reading this entire thread
(No, I don't know why), and Matz has seemed to me to be remarkably
restrained in his evangelical attempts. Whenever he mentions Ruby, he
makes the disclaimer that it's not particularly portable, and only seems
to use it as a proof-of-concept, or demonstration that he has some skill
in writing GCs.

Ref counting with GC sounds to me like a good idea for Python 2.0, as it
will eliminate a class of memory leaks that Python can currently fall
prey to. As long as you don't move stuff around in memory, and only
collect things where all the refcounts are accounted for, and can't be
reached, you should be safe. Granted, this means that the people
writing the extensions will have to be careful to INCREF/DECREF where
they're supposed to, but I don't see that as a bad thing since they
should already be careful about that sort of stuff.

Later,
Blake.

--
One Will. One Dream. One Truth. One Destiny. One Love.

On Wed, 9 Jun 1999, Tim Peters <tim_one@email.msn.com> wrote:
>[Graham Matthews]
>> There are two obvious answers to almost all the above problems in
>> Python
>> a) don't call finalisers for circularly referenced collected objects.
>> Instead just collect their memory. This is not perfect but better
>> than current Python which doesn't run finalisers on circularly
>> referenced objects nor collect their memory.
>Solve the problem by ignoring it? Guido would find that more
>attractive than I would <wink>.

If so, then he would have given up on this thread a long time ago...

Later,
Blake.

--
One Will. One Dream. One Truth. One Destiny. One Love.

On Thu, 10 Jun 1999, Blake Winton wrote:

> >Solve the problem by ignoring it? Guido would find that more
> >attractive than I would <wink>.
>
> If so, then he would have given up on this thread a long time ago...

Why? He seems to ignore it just fine!

quoting-out-of-context-ly y'rs, Z.
--
Moshe Zadka <mzadka@geocities.com>.
QOTD: My own exit is more likely to be horizontal then perpendicular.

In article <000c01beb2e8$0b825440$af9e2299@tim>,
"Tim Peters" <tim_one@email.msn.com> wrote:
| [Hisao Suzuki]
| > ...
| > It would be an issue of distributed garbage collection. And it
| > is a difficulty for automatic GC with finalization.
| Yes, there are real problems here, and Python certainly isn't the first to
| face them.

Certainly. And many languages either give up them to
programmers entirely (---"Please make your own finalization
mechanism from scratch" from ???), or provide rather
insufficient ones (---"However, it is not a good idea to rely on
finalization for the timely freeing of resources." from Java
Programmers FAQ).

| > (The solution is another story. It is not so obvious for
| > Python.)
| > # Certainly the current reference counting system of Python
| > # reduces the issues to fairly (though _not_ fully) tolerable
| > # ones...
| It automatically deals with the order-of-finalization problem in a correct
| way, by following a topological sort of the "references" graph. Which says
| almost nothing, since that's *natural* for RC. In return it can't deal with
| cycles at all, and indeed it's still easy to write __del__ methods that
| trigger mysterious exceptions at system shutdown time, when Python *does*
| clean up *some* kinds of cyclic trash (namely the cycles the interpreter
| itself creates for its own nefarious purposes).

Surely the shutdown time is the most critical for the current
Python and it is the main reason why I wrote "though _not_
fully". You might know that

in article <u36756gcoc.fsf@ares.sys1.ngo.okisoft.co.jp>,
I wrote:
< As for the issue of circular references, in my opinion, the
< future Python should use non-compacting GC at the end of the
< last thread and destroy the remained objects by invoking their
< __del__ methods --- it is THE time to destroy all the objects
< remained, so this is conceptually clean (thus very Pythonic :-),
< though it is the way of memory management used in the present
< Perl in fact...

It is only a casual idea and I am not sure what sort of
non-compacting garbage collector (or garbage detector, in this
case?) is implementable and acceptable as part of Python system.

For example, I have once thought what if each instance has two
additional pointers, which make up the object list, i.e. oblist.
(PyInstance_New and others will need modifying then.) After
folding up the stack(s), Python checks the referred instances
from, say sys.modules (see my article
<u31zfszath.fsf@ares.sys1.ngo.okisoft.co.jp> for a half-baked,
or rather a-tenth-baked quick-and-incomplete prototype), detects
possible circular references from the oblist, invokes their
__del__s first and then shuts down the same way as the current
Python does. Well, it may correct some faults, but I know it is
neither perfect nor efficient anyway.

--===-----========------------- Sana esprimo naskas sanan ideon.
SUZUKI Hisao suzuki611@okisoft.co.jp, suzuki@acm.org.

adjih@technologist.com writes:

| Yukihiro Matsumoto <matz@netlab.co.jp> wrote:
|> You don't have to know stack address to implement mark-sweep register,
|> unless you want to make it conservative. The mark-sweep GC can be
|> written in ANSI-C portable.
|
|But then this can break Python C modules, because references are held
|from the C stack.

In case you abandon ref counting, yes. But with the method like which
described by Guido in <5laeu8zmza.fsf@eric.cnri.reston.va.us>, you
still don't have to peep in C stack, I think.

matz.

[Graham Matthews]
> a) don't call finalisers for circularly referenced collected objects.
> Instead just collect their memory. This is not perfect but better
> than current Python which doesn't run finalisers on circularly
> referenced objects nor collect their memory.
Tim Peters (tim_one@email.msn.com) wrote:
: Solve the problem by ignoring it? Guido would find that more attractive
: than I would <wink>.

Even if you do this you still end up in a better situation than current
Python, since you can at least reclaim the memory of circularly reffed
objects. Moreover for builtin objects (like files) you can do better
and also reclaim their resources. So it is not perfect but still an
improvement on current Python.

[Graham Matthews]
> b) better still use a two pass collector. If you don't know what that is
> just think about what you need to do (or not do) in pass 1 to ensure
> that all the objects needed for a __del__ method to execute in a known
> environment are there. <wink>
Tim Peters (tim_one@email.msn.com) wrote:
: Java does this. Start the thread from its beginning again <0.5 wink>.
:
I am sure you can come up with something better than this Tim! <wink>

graham
--
As you grow up and leave the playground
where you kissed your prince and found your frog
Remember the jester that showed you tears
the script for tears

[Hisao Suzuki]
> Me too. And I am afraid that some evangelists would do nothing
> but _only_ point out Python's some so-called fault in order to
> drive us into convert to their favorite or invented language...
[Yukihiro Matsumoto]
> Sorry for bothering the newsgroup.
Tim Peters (tim_one@email.msn.com) wrote:
: Hmm. The level of paranoia in this thread is quite remarkable! I don't
: think Suzuki was talking about you (or Ruby) here, and your contributions to
: the thread have been civil and helpful.

Well my reading of Hisao's posts (especially the most recent ones) is
indeed that he was talking about Yukihiro.

Now two data points.

graham
--
As you grow up and leave the playground
where you kissed your prince and found your frog
Remember the jester that showed you tears
the script for tears

Blake Winton (bwinton@tor.dhs.org) wrote:
: Ref counting with GC sounds to me like a good idea for Python 2.0, as it
: will eliminate a class of memory leaks that Python can currently fall
: prey to. As long as you don't move stuff around in memory, and only
: collect things where all the refcounts are accounted for, and can't be
: reached, you should be safe. Granted, this means that the people
: writing the extensions will have to be careful to INCREF/DECREF where
: they're supposed to, but I don't see that as a bad thing since they
: should already be careful about that sort of stuff.

Actually Blake I have a feeling that I have read a paper somewhere that
says that a mark sweep collector with refcounts can in fact patch up
refcount errors, so it won't be so critical that extension writers get
their INCREF/DECREFs just right. I am not 100% sure of this but at least
on the surface it seems plausible (even if it can only give a warning
about such errors). Anyone know more about this?
graham

--
As you grow up and leave the playground
where you kissed your prince and found your frog
Remember the jester that showed you tears
the script for tears

Graham Matthews wrote:
>
> Actually Blake I have a feeling that I have read a paper somewhere that
> says that a mark sweep collector with refcounts can in fact patch up
> refcount errors,

Obviously this is true if the M&S can be sure of finding
all the roots, since the refcount information is then
redundant.

But if there are roots that the M&S can miss (e.g. on
the C stack or in data structures managed by an extension
module) then it's still just as important as ever to keep
the refcounts correct, since the M&S needs them to make
sure it doesn't make a mistake.

Greg

Tim Peters wrote:
>
> "Therefore, we recommend that the design of finalize methods be kept simple
> and that they be programmed defensively, so that they will work in all
> cases."

In my experience, this advice applies just as much to Python
__del__ methods, their alleged greater predictability
notwithstanding.

The only times I can recall using __del__
methods were in shadow classes which needed to free resources
in an extension module. I found that I had to program them
very defensively, because when the interpreter shut down,
it would dismantle the contents of my modules in a random
order, causing my __del__ methods to crash if I wasn't
careful.

So I regard the environment of a __del__ method as
pretty much unknowable as well!

Greg

[Tim, quoting the Java Language Spec]
> "Therefore, we recommend that the design of finalize methods be
> kept simple and that they be programmed defensively, so that they
> will work in all cases."

[Greg Ewing]
> In my experience, this advice applies just as much to Python
> __del__ methods, their alleged greater predictability
> notwithstanding.

I'll repeat my sig, which you cut off <wink>:

good-advice-in-any-language-ly y'rs - tim

Greg Ewing <greg.ewing@compaq.com> writes:

|Obviously this is true if the M&S can be sure of finding
|all the roots, since the refcount information is then
|redundant.

M&S without knowing all roots is not only imperfect, but also hell. It
may free living objects. On the other hand, mark and DETECT method to
report cyclic reference would have no problem, though not amusing
really.

With proper M&S, refcount infomation appears redundant. But M&S makes
reclamation and finalization behavior incompatible, so I doubt we can
remove refcounting from future Python.

matz.

In article <u37lpdssbq.fsf@ares.sys1.ngo.okisoft.co.jp>,
Hisao Suzuki <suzuki611@okisoft.co.jp> wrote:
> | >[Salvador]
> | > I have been using Java since it was invented and I haven't
programed a
> | > finaliser ever. They should run under so unknown and inconsistent
> | > environment that they are not usable at all (except for freeing
external
> | > resources).
>
> I have not ever programmed any finalizer, too, while I have
> programmed a lot of codes in Java.

I know this is getting off-topic, but just for general info, one example
of where a finalizer is useful in Java is when using JNI (the Java/C
interface) - it is often useful for a Java object to mirror a C
structure, and when the Java object is GCed you need to free() the C
stuff (which is an example of "freeing external resources", I guess).

Andrew


Sent via Deja.com http://www.deja.com/
Share what you know. Learn what you don't.