Mailing List Archive: Why does IDLE use a subprocess?

Why does IDLE use a subprocess?

May 28, 2023, 1:10 PM

Post #1 of 7 (247 views)

Originally posted to idle-dev, but thought this might be a better place. Let me know if it isn't.

Hi,

I was curious about the internals of IDLE, and noticed that IDLE uses executes user code in a "subprocess" that's separate from the Python interpreter that is running IDLE itself (which does tasks such as making the window and coloring the text).

As far as I understand, IDLE runs a modified version of code.InteractiveInterpreter by sending user code through a socket. Even the IDLE documentation says that without a subprocess, "user code is not isolated from IDLE itself." However, some minimal testing of InteractiveInterpreter leads me to believe that the Interpreter object has its own view of local/global variables and therefore shouldn't be able to affect the calling interpreter (please correct me if I'm wrong).

So my question is a combination of "Why does IDLE use a subprocess?" and "Why is InteractiveInterpreter not secure enough?" What possible security vulnerabilities exist if one uses IDLE without the subprocess? If anyone knows (or could point me to information on) why IDLE is designed this way, I'd really appreciate it. Thank you!

Jim
--
https://mail.python.org/mailman/listinfo/python-list

Re: Why does IDLE use a subprocess? [ In reply to ]

barry at barrys-emacs

May 30, 2023, 3:15 PM

Post #2 of 7 (247 views)

Permalink

> On 30 May 2023, at 21:10, James Schaffler via Python-list <python-list@python.org> wrote:
>
> ?Originally posted to idle-dev, but thought this might be a better place. Let me know if it isn't.
>
> Hi,
>
> I was curious about the internals of IDLE, and noticed that IDLE uses executes user code in a "subprocess" that's separate from the Python interpreter that is running IDLE itself (which does tasks such as making the window and coloring the text).
>
> As far as I understand, IDLE runs a modified version of code.InteractiveInterpreter by sending user code through a socket. Even the IDLE documentation says that without a subprocess, "user code is not isolated from IDLE itself." However, some minimal testing of InteractiveInterpreter leads me to believe that the Interpreter object has its own view of local/global variables and therefore shouldn't be able to affect the calling interpreter (please correct me if I'm wrong).
>
> So my question is a combination of "Why does IDLE use a subprocess?" and "Why is InteractiveInterpreter not secureuldenough?" What possible security vulnerabilities exist if one uses IDLE without the subprocess? If anyone knows (or could point me to information on) why IDLE is designed this way, I'd really appreciate it. Thank you!

I don’t think it security but robustness that needs the subprocess.

You can crash idle with bugs in the code that you are developing.
By running your code in a subprocess idle protects itself, and your edits from bugs in your code.

Also if your code use tk then it would conflict with idle’s use of tk.

That is my assumption on why the subprocess is required.

Barry

>
> Jim
> --
> https://mail.python.org/mailman/listinfo/python-list
>

--
https://mail.python.org/mailman/listinfo/python-list

Re: Why does IDLE use a subprocess? [ In reply to ]

rosuav at gmail

May 30, 2023, 3:32 PM

Post #3 of 7 (247 views)

Permalink

On Wed, 31 May 2023 at 08:16, Barry <barry@barrys-emacs.org> wrote:
> I don’t think it security but robustness that needs the subprocess.
>
> Also if your code use tk then it would conflict with idle’s use of tk.
>

From my memory, it's precisely this - it's much MUCH easier to allow
you to use Tk in your own program without conflicting with Idle's own
use of it. I'm sure it would be possible to make everything work in
one process, but the cost of doing so would be greater than the costs
of juggling two processes.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Why does IDLE use a subprocess? [ In reply to ]

python-list at python

May 30, 2023, 6:14 PM

Post #4 of 7 (247 views)

Permalink

On 29/05/23 8:10 am, James Schaffler wrote:
> However, some minimal testing of InteractiveInterpreter leads me to believe that the Interpreter object has its own view of local/global variables and therefore shouldn't be able to affect the calling interpreter

Globals you create by executing code in the REPL have their own
namespace. But everything else is shared -- builtins, imported
Python modules, imported C extension modules, etc. etc.

There's a long-running project to make it possible to have
multiple fully-isolated Python interpreters in one process, but
the way CPython is structured makes that very difficult to achieve,
and as far as I know it's not there yet.

In the case of IDLE, there's really no reason not to use a
subprocess[1]. It's easy and guarantees 100% isolation.

[1] Well, mostly. There used to be a small hitch on Windows with
the default firewall settings not letting you connect to a local
socket (nice one, Microsoft). I don't know whether that's still
an issue.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Why does IDLE use a subprocess? [ In reply to ]

python-list at python

May 30, 2023, 7:01 PM

Post #5 of 7 (247 views)

Permalink

On Tuesday, May 30th, 2023 at 9:14 PM, Greg Ewing wrote:
> Globals you create by executing code in the REPL have their own
> namespace. But everything else is shared -- builtins, imported
> Python modules, imported C extension modules, etc. etc.

Thanks for the explanation. Could you elaborate on precisely how "everything else is shared"? As far as I understand, if you run the following code:

from code import InteractiveInterpreter
interp = InteractiveInterpreter()
import numpy as np
interp.runcode("np.__name__")

this will result in the error
Traceback (most recent call last):
File "<string>", line 1, in <module>
NameError: name 'np' is not defined

which seems to imply that imports in the parent shell are not shared with interpreters and vice versa (if you swap the places of the import and the __name__ call).

Thanks,
Jim
--
https://mail.python.org/mailman/listinfo/python-list

Re: Why does IDLE use a subprocess? [ In reply to ]

rosuav at gmail

May 30, 2023, 7:18 PM

Post #6 of 7 (247 views)

Permalink

On Wed, 31 May 2023 at 12:03, James Schaffler via Python-list
<python-list@python.org> wrote:
>
> On Tuesday, May 30th, 2023 at 9:14 PM, Greg Ewing wrote:
> > Globals you create by executing code in the REPL have their own
> > namespace. But everything else is shared -- builtins, imported
> > Python modules, imported C extension modules, etc. etc.
>
> Thanks for the explanation. Could you elaborate on precisely how "everything else is shared"? As far as I understand, if you run the following code:
>
> from code import InteractiveInterpreter
> interp = InteractiveInterpreter()
> import numpy as np
> interp.runcode("np.__name__")
>
> this will result in the error
> Traceback (most recent call last):
> File "<string>", line 1, in <module>
> NameError: name 'np' is not defined
>
> which seems to imply that imports in the parent shell are not shared with interpreters and vice versa (if you swap the places of the import and the __name__ call).
>

Yep, what you're seeing there is the namespace and nothing else. But
if you mess with an actual builtin object, it'll be changed for the
other interpreter too.

>>> import ctypes
>>> ctypes.cast(id(42), ctypes.POINTER(ctypes.c_int))[6] = 43
>>> 41+1
43
>>> from code import InteractiveInterpreter
>>> interp = InteractiveInterpreter()
>>> interp.runcode("print(41+1)")
43

(Note that this only works in CPython and only with integers small
enough to be in the cache, meaning that there is only one such object
representing that integer.)

The same is true of C extensions, which often have their own internal
state, and that state isn't isolated to a single interpreter.

Better isolation is coming with PEP 554
https://peps.python.org/pep-0554/ which also has some great
information about what currently is NOT isolated. (Also, even then,
some things won't be fully isolated; I think that the ctypes trick
above might still affect a subinterpreter even in a post-PEP554
world.)

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Why does IDLE use a subprocess? [ In reply to ]

python-list at python

May 31, 2023, 7:08 PM

Post #7 of 7 (245 views)

Permalink

On Tuesday, May 30th, 2023 at 10:18 PM, Chris Angelico wrote:
> Yep, what you're seeing there is the namespace and nothing else. But
> if you mess with an actual builtin object, it'll be changed for the
> other interpreter too.
>
> > > > import ctypes
> > > > ctypes.cast(id(42), ctypes.POINTER(ctypes.c_int))[6] = 43
> > > > 41+1
>
> 43
>
> > > > from code import InteractiveInterpreter
> > > > interp = InteractiveInterpreter()
> > > > interp.runcode("print(41+1)")
>
> 43
>
> (Note that this only works in CPython and only with integers small
> enough to be in the cache, meaning that there is only one such object
> representing that integer.)
>
> The same is true of C extensions, which often have their own internal
> state, and that state isn't isolated to a single interpreter.
>
> Better isolation is coming with PEP 554
> https://peps.python.org/pep-0554/ which also has some great
> information about what currently is NOT isolated. (Also, even then,
> some things won't be fully isolated; I think that the ctypes trick
> above might still affect a subinterpreter even in a post-PEP554
> world.)

Amazing example! Thank you everyone for the detailed responses - will be sure to check out the PEP as well.

Jim
--
https://mail.python.org/mailman/listinfo/python-list