Mailing List Archive

On Tue, 8 Jun 1999, Guido van Rossum wrote:

> We should structurally split the Python Virtual Machine and related
> code up into different parts -- both at the source code level and at
> the runtime level. The core PVM becomes a replaceable component, and
> so do a few other parts like the parser, the bytecode compiler, the
> import code, and the interactive read-eval-print loop.

> The goal of course, is to create a market for 3rd party components
> here, e.g. Chris' flat PVM, or Skip's bytecode optimizer, or Greg's
> importer, and so on.
>
> Thoughts?

If I understand it correctly, it means that I can fit in a third-party
read-eval-print loop, which is my biggest area of frustration with the
current internal structure. Sounds like a plan to me, and one which (lucky
for me) I'm not qualified for!

--david

Guido van Rossum wrote:
>
> Tim wrote:
>
> > Sorry, Chris! Just a case of "no time" here. Of *course* you
> > should continue, and Guido should pop in with an encouraging word
> > too -- or a "forget it". I think this design opens the doors to a
> > world of interesting ideas, but that's based on informed prejudice
> > rather than careful study of your code. Cheer up: if everyone
> > thought you were a lame ass, we all would have studied your code
> > intensely by now <wink>.
>
> No time here either...
>
> I did try to have a quick peek and my first impression is that it's
> *very* tricky code! You know what I think of that...

Thanks for looking into it, thanks for saying it's tricky.
Since I failed to supply proper documentation yet, this
impression must come up.

But it is really not true. The code is not tricky
but just straightforward and consequent, after one has understood
what it means to work without a stack, under the precondition
to avoid too much changes. I didn't want to rewrite
the world, and I just added the tiny missing bits.

I will write up my documentation now, and you will
understand what the difficulties were. These will not
vanish, "stackless" is a brainteaser. My problem was not how
to change the code, but finally it was how to change
my brain. Now everything is just obvious.

> Here's what I think we should do first (I've mentioned this before but
> nobody cheered me on :-). I'd like to see this as the basis for 1.6.
>
> We should structurally split the Python Virtual Machine and related
> code up into different parts -- both at the source code level and at
> the runtime level. The core PVM becomes a replaceable component, and
> so do a few other parts like the parser, the bytecode compiler, the
> import code, and the interactive read-eval-print loop. Most object
> implementations are shared between all -- or at least the interfaces
> are interchangeable. Clearly, a few object types are specific to one
> or another PVM (e.g. frames). The collection of builtins is also a
> separate component (though some builtins may again be specific to a
> PVM -- details, details!).

Good idea, and a lot of work.
Having different frames for different PVM's was too much for
me. Instead, I tried to adjust frames in a way where a lot
of machines can work with.

I tried to show the concept of having different VM's by
implementing a stackless map. Stackless map is a very tiny one
which uses frames again (and yes, this was really hacked).
Well, different frame flavors would make sense, perhaps.

But I have a central routine which handles all calls to frames,
and this is what I think is needed. I already *have*
pluggable interpreters here, since a function can produce
a frame which is bound to an interpreter, and push it
to the frame stack.

> The goal of course, is to create a market for 3rd party components
> here, e.g. Chris' flat PVM, or Skip's bytecode optimizer, or Greg's
> importer, and so on.

I'm with that component goal, of course. Much work,
not for one persone, but great.

While I don't think it makes sense to make a flat PVM
pluggable. I would start with a flat PVM, since that opens
a world of possibilities. You can hardly plug flatness in
after you started with a wrong stack layout. Vice versa,
plugging the old machine would be possible.

later - chris

--
Christian Tismer :^) <mailto:tismer@appliedbiometrics.com>
Applied Biometrics GmbH : Have a break! Take a ride on Python's
Kaiserin-Augusta-Allee 101 : *Starship* http://starship.python.net
10553 Berlin : PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF
we're tired of banana software - shipped green, ripens at home

[Christian Tismer, replying to Guido's enthusiasm <wink>]
> Thanks for looking into it, thanks for saying it's tricky.
> Since I failed to supply proper documentation yet, this
> impression must come up.
>
> But it is really not true. The code is not tricky
> but just straightforward and consequent, after one has understood
> what it means to work without a stack, under the precondition
> to avoid too much changes. I didn't want to rewrite
> the world, and I just added the tiny missing bits.
>
> I will write up my documentation now, and you will
> understand what the difficulties were. These will not
> vanish, "stackless" is a brainteaser. My problem was not how
> to change the code, but finally it was how to change
> my brain. Now everything is just obvious.

FWIW, I believe you! There's something *inherently* tricky about
maintaining the effect of a stack without using the stack C supplies
implicitly, and from all you've said and what I've learned of your code, it
really isn't the code that's tricky here. You're making formerly-hidden
connections explicit, which means more stuff is visible, but also means more
power and flexibility *because* "more stuff is visible".

Agree too that this clearly <wink> moves in the direction of making the VM
pluggable.

> ...
> I *will* continue, regardless what anybody says.

Ah, if that's how this works, then STOP! Immediately! Don't you dare waste
more of our time with this crap <wink>.

want-some-money?-ly y'rs - tim

[Guido van Rossum]
> ...
> Here's what I think we should do first (I've mentioned this before but
> nobody cheered me on :-). I'd like to see this as the basis for 1.6.
>
> We should structurally split the Python Virtual Machine and related
> code up into different parts -- both at the source code level and at
> the runtime level. The core PVM becomes a replaceable component, and
> so do a few other parts like the parser, the bytecode compiler, the
> import code, and the interactive read-eval-print loop. Most object
> implementations are shared between all -- or at least the interfaces
> are interchangeable. Clearly, a few object types are specific to one
> or another PVM (e.g. frames). The collection of builtins is also a
> separate component (though some builtins may again be specific to a
> PVM -- details, details!).
>
> The goal of course, is to create a market for 3rd party components
> here, e.g. Chris' flat PVM, or Skip's bytecode optimizer, or Greg's
> importer, and so on.
>
> Thoughts?

The idea of major subsystems getting reworked to conform to well-defined and
well-controlled interfaces is certainly appealing.

I'm just more comfortable squeezing another 1.7% out of list.sort() <0.9
wink>.

trying-to-reduce-my-ambitions-to-match-my-time-ly y'rs - tim

> Here's what I think we should do first (I've mentioned this before but
> nobody cheered me on :-).

Go, Guido, GO!!!!

What I'd like in the split you propose is to see which of the items would be
implementable in Python, and try to do the split in such a way that such a
Python implementation isn't ruled out.

Am I correct in guessing that after factoring out the components you mention
the only things that aren't in a "replaceable component" are the builtin
objects, and a little runtime glue (malloc and such)?
--
Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm

Tim Peters wrote:
>
> [Christian Tismer, replying to Guido's enthusiasm <wink>]
...
> > I will write up my documentation now, and you will
still under some work :)
> > understand what the difficulties were. These will not
> > vanish, "stackless" is a brainteaser. My problem was not how
> > to change the code, but finally it was how to change
> > my brain. Now everything is just obvious.
>
> FWIW, I believe you! There's something *inherently* tricky about
> maintaining the effect of a stack without using the stack C supplies
> implicitly, and from all you've said and what I've learned of your code, it
> really isn't the code that's tricky here. You're making formerly-hidden
> connections explicit, which means more stuff is visible, but also means more
> power and flexibility *because* "more stuff is visible".

I knew you would understand me. Feeling much, much better now :-))

After this is finalized, restartable exceptions
might be interesting to explore. No, Chris, do the doco...

> > I *will* continue, regardless what anybody says.
>
> Ah, if that's how this works, then STOP! Immediately! Don't you dare waste
> more of our time with this crap <wink>.

Thanks, you fired me a continuation.

Here the way to get me into an endless loop:
Give me an unsolvable problem and claim I can't do that. :)

(just realized that I'm just another pluggable interpreter)

> want-some-money?-ly y'rs - tim

No, but meet you at least once in my life.

--
Christian Tismer :^) <mailto:tismer@appliedbiometrics.com>
Applied Biometrics GmbH : Have a break! Take a ride on Python's
Kaiserin-Augusta-Allee 101 : *Starship* http://starship.python.net
10553 Berlin : PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF
we're tired of banana software - shipped green, ripens at home

[me]
> > Here's what I think we should do first (I've mentioned this before but
> > nobody cheered me on :-).

[Jack]
> Go, Guido, GO!!!!
>
> What I'd like in the split you propose is to see which of the items would be
> implementable in Python, and try to do the split in such a way that such a
> Python implementation isn't ruled out.

Indeed. The importing code and the read-eval-print loop are obvious
candidates (in fact IDLE shows how the latter can be done today). I'm
not sure if it makes sense to have a parser/compiler or the VM written
in Python, because of the expected slowdown (plus, the VM would
present a chicken-egg problem :-) although for certain purposes one
might want to do this. An optimizing pass would certainly be a good
candidate.

> Am I correct in guessing that after factoring out the components you mention
> the only things that aren't in a "replaceable component" are the builtin
> objects, and a little runtime glue (malloc and such)?

I guess (although how much exactly will only become clear when it's
done). I guess that things like thread-safety and GC policy are also
pervasive.

--Guido van Rossum (home page: http://www.python.org/~guido/)