Mailing List Archive

[issue45256] Remove the usage of the C stack in Python to Python calls
Change by STINNER Victor <vstinner@python.org>:


----------
title: Remove the usage of the cstack in Python to Python calls -> Remove the usage of the C stack in Python to Python calls

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Christian Tismer <tismer@stackless.com> added the comment:

Hey guys, you know that you are about to implement the core idea of Stackless Python, right? :-D

----------
nosy: +Christian.Tismer

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Mark Shannon <mark@hotpy.org> added the comment:

I've trying to do this since about 2011 :)

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Steve Dower <steve.dower@python.org> added the comment:

I fully support implementing the core idea of Stackless Python :)

I spent a whole EuroPython a couple of years back discussing the idea with (apparently) everyone except Mark.

Though I wouldn't like to lose the ability to extract the Python stack by inspecting native memory alone. That is very useful for debugging, particularly when you've only got a crash dump. So provided all the code objects are only an indirection or two away from a local (or better yet, parameter) value, it should be fine.

----------
nosy: +steve.dower

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Pablo Galindo Salgado <pablogsal@gmail.com> added the comment:

> Though I wouldn't like to lose the ability to extract the Python stack by inspecting native memory alone.

Don't worry about it, I am personally making sure that keeps being possible. it will need some changes in the tools, but not any more that any other changes between minor versions of the interpreter

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Christian Tismer <tismer@stackless.com> added the comment:

FYI., in Stackless Python I built a chain of frames by a double linked list. This was the replacement for the current frame/stack mix.
Debugging was still quite easy, following this frame chain.

Note that it is a rather easy step to add the capability to squirrel the whole current chain away and replace it by another one. Add some header to such a chain, and you can call it "Tasklet".

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Mark Shannon <mark@hotpy.org> added the comment:

PR 28488 has no NEWS entry, or What's New entry.

However, adding multiple entries will be confusing, so that's best left until all calls to Python functions and method don't use the C stack.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Christian Tismer <tismer@stackless.com> added the comment:

FWIW, getting all function to avoid the C stack will most probably take a long time, if it happens at all. Especially functions which might call into Python multiple times must be re-designed heavily, turned into multiple pieces to be in tail position. Been there, long ago... :)

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Pablo Galindo Salgado <pablogsal@gmail.com> added the comment:

We are starting to optimize first the easy cases of CALL_FUNCTION and move slowly to add more and more calls into it. This approach has the advantage that allow us to take it in small increments.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Christian Tismer <tismer@stackless.com> added the comment:

Very much appreciated approach.
Too bad that things stop when people are writing extensions as usual. Or do you think we can teach them how to avoid the C stack? Anyway, good luck!

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Steve Dower <steve.dower@python.org> added the comment:

The goal is reduced stack depth, not reframing the entire call model around not having a C stack.

We can't even reasonably rewrite getattr() without supporting callbacks from C into Python, so further generalisation is very unlikely.

But if you inspect the native stack of most Python programs, you'll see that it's mostly taken up with calls within Python code. Compressing all of those is a significant advantage, akin to inlining the Python code at compile time, even if it doesn't see "through" native calls.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Pablo Galindo Salgado <pablogsal@gmail.com> added the comment:


New changeset b4903afd4debbbd71dc49a2c8fefa74a3b6c6832 by Pablo Galindo Salgado in branch 'main':
bpo-45256: Remove the usage of the C stack in Python to Python calls (GH-28488)
https://github.com/python/cpython/commit/b4903afd4debbbd71dc49a2c8fefa74a3b6c6832


----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Change by Pablo Galindo Salgado <pablogsal@gmail.com>:


----------
pull_requests: +27152
pull_request: https://github.com/python/cpython/pull/28836

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Pablo Galindo Salgado <pablogsal@gmail.com> added the comment:


New changeset 543acbce5a1e23633379a853f38dc55b12f6d931 by Pablo Galindo Salgado in branch 'main':
bpo-45256: Small cleanups for the code that inlines Python-to-Python calls in ceval.c (GH-28836)
https://github.com/python/cpython/commit/543acbce5a1e23633379a853f38dc55b12f6d931


----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Change by Pablo Galindo Salgado <pablogsal@gmail.com>:


----------
pull_requests: +27196
pull_request: https://github.com/python/cpython/pull/28905

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Mark Shannon <mark@hotpy.org> added the comment:


New changeset 3901c081143ef29624f9c1cb49cc70a70321d139 by Pablo Galindo Salgado in branch 'main':
bpo-45256: Fix cleanup of stolen locals for Python-to-Python calls (GH-28905)
https://github.com/python/cpython/commit/3901c081143ef29624f9c1cb49cc70a70321d139


----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Change by Mark Shannon <mark@hotpy.org>:


----------
pull_requests: +27226
pull_request: https://github.com/python/cpython/pull/28937

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Mark Shannon <mark@hotpy.org> added the comment:


New changeset 70945d57e775b335eb58b734d82e68484063e835 by Mark Shannon in branch 'main':
bpo-45256: Avoid C calls for most Python to Python calls. (GH-28937)
https://github.com/python/cpython/commit/70945d57e775b335eb58b734d82e68484063e835


----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Change by Mark Shannon <mark@hotpy.org>:


----------
pull_requests: +27498
pull_request: https://github.com/python/cpython/pull/29235

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Change by Mark Shannon <mark@hotpy.org>:


----------
pull_requests: +27501
pull_request: https://github.com/python/cpython/pull/29238

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Pablo Galindo Salgado <pablogsal@gmail.com> added the comment:

Unfortunately, seems that https://github.com/python/cpython/pull/28937 has broken the AMD64 FreeBSD Shared 3.x buildbot:

https://buildbot.python.org/all/#/builders/483/builds/1003/steps/5/logs/stdio

The buildbot was green until we merged this

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Mark Shannon <mark@hotpy.org> added the comment:


New changeset 7f61d9d84843e3445f62eb00c47902f0daa30a72 by Mark Shannon in branch 'main':
bpo-45256: Rationalize code around Python-to-Python calls a bit. (GH-29235)
https://github.com/python/cpython/commit/7f61d9d84843e3445f62eb00c47902f0daa30a72


----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[issue45256] Remove the usage of the C stack in Python to Python calls [ In reply to ]
Change by Sam James <sam@cmpct.info>:


----------
nosy: +thesamesam

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45256>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com