Mailing List Archive

Interrupting a thread
Ive struck this a number of times, and the simple question is "can we make
it possible to interrupt a thread without the thread's knowledge" or
otherwise stated "how can we asynchronously raise an exception in another
thread?"

The specific issue is that quite often, I find it necessary to interrupt
one thread from another. One example is Pythonwin - rather than use the
debugger hooks as IDLE does, I use a secondary thread. But how can I use
that thread to interrupt the code executing in the first? (With magic that
only works sometimes is how :-)

Another example came up on the newsgroup recently - discussion about making
Medusa a true Windows NT Service. A trivial solution would be to have a
"service thread", that simply runs Medusa's loop in a seperate thread.
When the "service thread" recieves a shut-down request from NT, how can it
interrupt Medusa?

I probably should not have started with a Medusa example - it may have a
solution. Pretend I said "any arbitary script written to run similarly to
a Unix daemon". There are one or 2 other cases where I have wanted to
execute existing code that assumes it runs stand-alone, and can really only
be stopped with a KeyboardInterrupt. I can't see a decent way to do this.

[.I guess this ties into the "signals and threads" limitations - I believe
you cant direct signals at threads either?]

Is it desirable? Unfortunately, I can see that it might be hard :-(

But-sounds-pretty-easy-under-those-fake-threads<wink>-ly,

Mark.
RE: Interrupting a thread [ In reply to ]
[Mark Hammond]
> Ive struck this a number of times, and the simple question is "can we
> make it possible to interrupt a thread without the thread's knowledge"
> or otherwise stated "how can we asynchronously raise an exception in
> another thread?"

I don't think there's any portable way to do this. Even restricting the
scope to Windows, forget Python for a moment: can you do this reliably with
NT threads from C, availing yourself of every trick in the SDK? Not that I
know of; not without crafting a new protocol that the targeted threads agree
to in advance.

> ...
> But-sounds-pretty-easy-under-those-fake-threads<wink>-ly,

Yes, piece o' cake! Fake threads can do anything, because unless we write
every stick of their implementation they can't do anything at all <wink>.

odd-how-solutions-create-more-problems-than-they-solve-ly y'rs - tim
RE: Interrupting a thread [ In reply to ]
> I don't think there's any portable way to do this. Even
> restricting the
> scope to Windows, forget Python for a moment: can you do
> this reliably with
> NT threads from C, availing yourself of every trick in the
> SDK? Not that I

Nope - not if I forget Python. However, when I restrict myself _to_
Python, I find this nice little ceval.c loop and nice little per-thread
structures - even with nice-looking exception place-holders ;-)

Something tells me that it wont be quite as easy as filling these in (while
you have the lock, of course!), but it certainly seems far more plausible
than if we consider it a C problem :-)

> odd-how-solutions-create-more-problems-than-they-solve-ly y'rs - tim

Only because they often open your eyes to a whole new class of problem
<wink>. Continuations/generators/co-routines (even threads themselves!)
would appear to be a good example - for all their power, I shudder to think
at the number of questions they will generate! If I understand correctly,
it is a recognised deficiency WRT signals and threads - so its all Guido's
fault for adding these damn threads in the first place :-)

just-more-proof-there-is-no-such-thing-as-a-free-lunch-ly,

Mark.
Re: Interrupting a thread [ In reply to ]
Mark Hammond wrote:
...

> Another example came up on the newsgroup recently - discussion about making
> Medusa a true Windows NT Service. A trivial solution would be to have a
> "service thread", that simply runs Medusa's loop in a seperate thread.

Ah, thanks, that was what I'd like to know :-)

> When the "service thread" recieves a shut-down request from NT, how can it
> interrupt Medusa?

Very simple. I do this shutdown stuff already, at a user request.
Medusa has its polling loop which is so simple (wait until a timeout,
then run again) that I pulled it out of Medusa, and added a polling
function. I have even simulated timer objects by this, which do
certain tasks from time to time (at the granularity of the loop of
course). One of these looks if there is a global object in module
__main__ with a special name which is executable. This happens to
be the shutdown, which may be injected by another thread as well.
I can send you an example.

> I probably should not have started with a Medusa example - it may have a
> solution. Pretend I said "any arbitary script written to run similarly to
> a Unix daemon". There are one or 2 other cases where I have wanted to
> execute existing code that assumes it runs stand-alone, and can really only
> be stopped with a KeyboardInterrupt. I can't see a decent way to do this.

Well, yes, I would want to have this too, and see also no way.

> [.I guess this ties into the "signals and threads" limitations - I believe
> you cant direct signals at threads either?]
>
> Is it desirable? Unfortunately, I can see that it might be hard :-(
>
> But-sounds-pretty-easy-under-those-fake-threads<wink>-ly,

You mean you would catch every signal in the one thread, and
redirect it to the right fake thread.
Given exactly two real threads, one always sitting waiting in a
multiple select, the other running any number of fake threads.
Would this be enough to do everything which is done with threads
today?

maybe-almost-ly y'rs - chris

--
Christian Tismer :^) <mailto:tismer@appliedbiometrics.com>
Applied Biometrics GmbH : Have a break! Take a ride on Python's
Kaiserin-Augusta-Allee 101 : *Starship* http://starship.python.net
10553 Berlin : PGP key -> http://wwwkeys.pgp.net
PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF
we're tired of banana software - shipped green, ripens at home
Re: Interrupting a thread [ In reply to ]
> Ive struck this a number of times, and the simple question is "can we make
> it possible to interrupt a thread without the thread's knowledge" or
> otherwise stated "how can we asynchronously raise an exception in another
> thread?"
>
> The specific issue is that quite often, I find it necessary to interrupt
> one thread from another. One example is Pythonwin - rather than use the
> debugger hooks as IDLE does, I use a secondary thread. But how can I use
> that thread to interrupt the code executing in the first? (With magic that
> only works sometimes is how :-)
>
> Another example came up on the newsgroup recently - discussion about making
> Medusa a true Windows NT Service. A trivial solution would be to have a
> "service thread", that simply runs Medusa's loop in a seperate thread.
> When the "service thread" recieves a shut-down request from NT, how can it
> interrupt Medusa?
>
> I probably should not have started with a Medusa example - it may have a
> solution. Pretend I said "any arbitary script written to run similarly to
> a Unix daemon". There are one or 2 other cases where I have wanted to
> execute existing code that assumes it runs stand-alone, and can really only
> be stopped with a KeyboardInterrupt. I can't see a decent way to do this.
>
> [.I guess this ties into the "signals and threads" limitations - I believe
> you cant direct signals at threads either?]
>
> Is it desirable? Unfortunately, I can see that it might be hard :-(
>
> But-sounds-pretty-easy-under-those-fake-threads<wink>-ly,

Hmm... Forget about signals -- they're twisted Unixisms (even if they
are nominally supported on NT).

The interesting thing is that you can interrupt the "main" thread
easily (from C) using Py_AddPendingCall() -- this registers a function
that will be invoked by the main thread the next time it gets to the
top of the VM loop. But the mechanism here was designed with a
specific purpose in mind, and it doesn't allow you to aim at a
specific thread -- it only works for the main thread. It might be
possible to add an API that allows you to specify a thread id
though...

Of course if the thread to be interrupted is blocked waiting for I/O,
this is not going to interrupt the I/O. (On Unix, that's what signals
do; is there an equivalent on NT? I don't think so.)

Why do you say that your magic only works sometimes? You mailed me
your code once and the Python side of it looks okay to me: it calls
PyErr_SetInterrupt(), which calls Py_AddPendingCall(), which is
threadsafe. Of course it only works if the thread you try to
interrupt is recognized by Python as the main thread -- perhaps this
is not always under your control, e.g. when COM interferes?

Where is this going? Is the answer "provide a C-level API like
Py_AddPendingCall() that takes a thread ID" good enough?

Note that for IDLE, I have another problem -- how to catch the ^C
event when Tk is processing events?

--Guido van Rossum (home page: http://www.python.org/~guido/)
RE: Interrupting a thread [ In reply to ]
[.Tim sez there's no portable way to violate another thread "even in C"]

[Mark Hammond]
> Nope - not if I forget Python. However, when I restrict myself _to_
> Python, I find this nice little ceval.c loop and nice little per-thread
> structures - even with nice-looking exception place-holders ;-)

Good point! Python does have its own notion of threads.

> Something tells me that it wont be quite as easy as filling these
> in (while you have the lock, of course!), but it certainly seems far
> more plausible than if we consider it a C problem :-)

Adding a scheme that builds on the global lock and Python-controlled thread
switches may not be prudent if your life's goal is to make Python
free-threaded <wink>.

But if "if you can't beat 'em, join 'em" rules the day, making
Py_AddPendingCall thread safe, adding a target thread argument, and fleshing
out the

XXX Darn! With the advent of thread state, we should have an array
of pending calls per thread in the thread state! Later...

comment before it, could go a long way toward facilitating groping in the
back seat of dad's car <wink>.

cheaper-than-renting-a-motel-room-for-sure-ly y'rs - tim
RE: Interrupting a thread [ In reply to ]
On Wed, 14 Jul 1999, Gordon McMillan wrote:

a reply to the python-dev thread on python-list.

You didn't really intend to do that, did you Gordon? =)

--david