> I think the main point is how to change code when a Python frame already
> references it. I dont think the structure of the frames is as important as
> the general concept. But while we were talking frame-fiddling it seemed a
> good point to try and hijack it a little :-)
>
> Would it be possible to recompile just a block of code (eg, just the
> current function or method) and patch it back in such a way that the
> current frame continues execution of the new code?
This topic sounds mostly unrelated to the stackless discussion -- in
either case you need to be able to fiddle the contents of the frame
and the bytecode pointer to reflect the changed function.
Some issues:
- The slots containing local variables may be renumbered after
recompilation; fortunately we know the name--number mapping so we can
move them to their new location. But it is still tricky.
- Should you be able to edit functions that are present on the call
stack below the top? Suppose we have two functions:
def f():
return 1 + g()
def g():
return 0
Suppose set a break in g(), and then edit the source of f(). We can
do all sorts of evil to f(): e.g. we could change it to
return g() + 2
which affects the contents of the value stack when g() returns
(originally, the value stack contained the value 1, now it is empty).
Or we could even change f() to
return 3
thereby eliminating the call to g() altogether!
What kind of limitations do other systems that support modifying a
"live" program being debugged impose? Only allowing modification of
the function at the top of the stack might eliminate some problems,
although there are still ways to mess up. The value stack is not
always empty even when we only stop at statement boundaries -- e.g. it
contains 'for' loop indices, and there's also the 'block' stack, which
contains try-except information. E.g. what should happen if we change
def f():
for i in range(10):
print 1
stopped at the 'print 1' into
def f():
print 1
???
(Ditto for removing or adding a try/except block.)
> I feel this is somewhat related to the inability to change class
> implementation for an existing instance. I know there have been hacks
> around this before but they arent completly reliable and IMO it would be
> nice if the core Python made it easier to change already running code -
> whether that code is in an existing stack frame, or just in an already
> created instance, it is very difficult to do.
I've been thinking a bit about this. Function objects now have
mutable func_code attributes (and also func_defaults), I think we can
use this.
The hard part is to do the analysis needed to decide which functions
to recompile! Ideally, we would simply edit a file and tell the
programming environment "recompile this". The programming environment
would compare the changed file with the old version that it had saved
for this purpose, and notice (for example) that we changed two methods
of class C. It would then recompile those methods only and stuff the
new code objects in the corresponding function objects.
But what would it do when we changed a global variable? Say a module
originally contains a statement "x = 0". Now we change the source
code to say "x = 100". Should we change the variable x? Suppose that
x is modified by some of the computations in the module, and the that,
after some computations, the actual value of x was 50. Should the
"recompile" reset x to 100 or leave it alone?
One option would be to actually change the semantics of the class and
def statements so that they modify an existing class or function
rather than using assignment. Effectively, this proposal would change
the semantics of
class A:
...some code...
class A:
...some more code...
to be the same as
class A:
...more code...
...some more code...
This is somewhat similar to the way the module or package commands in
some other dynamic languages work, I think; and I don't think this
would break too much existing code.
The proposal would also change
def f():
...some code...
def f():
...other code...
but here the equivalence is not so easy to express, since I want
different semantics (I don't want the second f's code to be tacked
onto the end of the first f's code).
If we understand that def f(): ... really does the following:
f = NewFunctionObject()
f.func_code = ...code object...
then the construct above (def f():... def f(): ...) would do this:
f = NewFunctionObject()
f.func_code = ...some code...
f.func_code = ...other code...
i.e. there is no assignment of a new function object for the second
def.
Of course if there is a variable f but it is not a function, it would
have to be assigned a new function object first.
But in the case of def, this *does* break existing code. E.g.
# module A
from B import f
.
.
.
if ...some test...:
def f(): ...some code...
This idiom conditionally redefines a function that was also imported
from some other module. The proposed new semantics would change B.f
in place!
So perhaps these new semantics should only be invoked when a special
"reload-compile" is asked for... Or perhaps the programming
environment could do this through source parsing as I proposed
before...
> This has come to try and deflect some conversation away from changing
> Python as such towards an attempt at enhancing its _environment_. To
> paraphrase many people before me, even if we completely froze the language
> now there would still plenty of work ahead of us :-)
Please, no more posts about Scheme. Each new post mentioning call/cc
makes it *less* likely that something like that will ever be part of
Python. "What if Guido's brain exploded?" :-)
--Guido van Rossum (home page:
http://www.python.org/~guido/)