Mailing List Archive: cross platform alternative for signal.SIGALRM?

cross platform alternative for signal.SIGALRM?

framstag at rus

Nov 11, 2015, 8:16 AM

Post #1 of 16 (4523 views)

I am rewriting a Perl program into Python (2.7).
It must run on Linux and Windows.
With Linux I have no problems, but Windows... :-(

The current show stopper is signal.SIGALRM which is not available on
Windows:

File "fexit.py", line 674, in formdata_post
signal.signal(signal.SIGALRM,timeout_handler)
AttributeError: 'module' object has no attribute 'SIGALRM'

https://docs.python.org/2/library/signal.html

signal.alarm(time) (...) Availability: Unix.

Perl for Windows has had SIGALRM support (or some kind of emulation).

Ok, I have to redesign this part of my code:

def timeout_handler(sig,frame):
raise ValueError("timeout!")

signal.signal(signal.SIGALRM,timeout_handler)

while True:
chunk = fileo.read(bs)
sock.sendall(chunk)
(...)

What is the best practise for a cross platform timeout handler?

--
Ullrich Horlacher Server und Virtualisierung
Rechenzentrum IZUS/TIK E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart Tel: ++49-711-68565868
Allmandring 30a Fax: ++49-711-682357
70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/
--
https://mail.python.org/mailman/listinfo/python-list

Re: cross platform alternative for signal.SIGALRM? [ In reply to ]

marko at pacujo

Nov 11, 2015, 8:30 AM

Post #2 of 16 (4493 views)

Ulli Horlacher <framstag@rus.uni-stuttgart.de>:

> What is the best practise for a cross platform timeout handler?

Here's the simplest answer:

https://docs.python.org/3/library/threading.html#threading.Timer

(Also available in Python 2.)

Marko
--
https://mail.python.org/mailman/listinfo/python-list

Re: cross platform alternative for signal.SIGALRM? [ In reply to ]

framstag at rus

Nov 11, 2015, 9:06 AM

Post #3 of 16 (4493 views)

Marko Rauhamaa <marko@pacujo.net> wrote:
> Ulli Horlacher <framstag@rus.uni-stuttgart.de>:
>
> > What is the best practise for a cross platform timeout handler?
>
> Here's the simplest answer:
>
> https://docs.python.org/3/library/threading.html#threading.Timer
>
> (Also available in Python 2.)

Hmmm... not so simple for me. My test code:

from time import *
import threading
import sys

def hello():
raise ValueError("hello!!!")

t = threading.Timer(3.0,hello)
t.start()
try:
print "start"
sleep(5)
print "end"
except ValueError as e:
print e.args[0]
sys.exit(1)

gives:

start
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
self.run()
File "/usr/lib/python2.7/threading.py", line 759, in run
self.function(*self.args, **self.kwargs)
File "x.py", line 7, in hello
def hello(): raise ValueError("hello!!!")
ValueError: hello!!!

end

--
Ullrich Horlacher Server und Virtualisierung
Rechenzentrum IZUS/TIK E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart Tel: ++49-711-68565868
Allmandring 30a Fax: ++49-711-682357
70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/
--
https://mail.python.org/mailman/listinfo/python-list

Re: cross platform alternative for signal.SIGALRM? [ In reply to ]

marko at pacujo

Nov 11, 2015, 10:03 AM

Post #4 of 16 (4496 views)

Ulli Horlacher <framstag@rus.uni-stuttgart.de>:

> Hmmm... not so simple for me. My test code:
>
> from time import *
> import threading
> import sys
>
> def hello():
> raise ValueError("hello!!!")
>
> t = threading.Timer(3.0,hello)
> t.start()
> try:
> print "start"
> sleep(5)
> print "end"
> except ValueError as e:
> print e.args[0]
> sys.exit(1)

Correct. The timer callback function (hello) would be called in a
separate thread. An exception raised in one thread cannot be caught in
the main thread. In general, there is no way for a thread to interrupt a
sibling thread that is in a blocking function call.

Marko
--
https://mail.python.org/mailman/listinfo/python-list

Re: cross platform alternative for signal.SIGALRM? [ In reply to ]

framstag at rus

Nov 11, 2015, 2:42 PM

Post #5 of 16 (4493 views)

Marko Rauhamaa <marko@pacujo.net> wrote:

> Correct. The timer callback function (hello) would be called in a
> separate thread. An exception raised in one thread cannot be caught in
> the main thread. In general, there is no way for a thread to interrupt a
> sibling thread that is in a blocking function call.

Then threading.Timer is not a solution for my problem.

--
Ullrich Horlacher Server und Virtualisierung
Rechenzentrum IZUS/TIK E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart Tel: ++49-711-68565868
Allmandring 30a Fax: ++49-711-682357
70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/
--
https://mail.python.org/mailman/listinfo/python-list

Re: cross platform alternative for signal.SIGALRM? [ In reply to ]

tjreedy at udel

Nov 11, 2015, 5:37 PM

Post #6 of 16 (4496 views)

On 11/11/2015 11:16 AM, Ulli Horlacher wrote:
> I am rewriting a Perl program into Python (2.7).

I recommend using 3.4+ if you possibly can.

> It must run on Linux and Windows.
> With Linux I have no problems, but Windows... :-(
>
> The current show stopper is signal.SIGALRM which is not available on
> Windows:

> Perl for Windows has had SIGALRM support (or some kind of emulation).
>
> Ok, I have to redesign this part of my code:
>
> def timeout_handler(sig,frame):
> raise ValueError("timeout!")
>
> signal.signal(signal.SIGALRM,timeout_handler)
>
> while True:
> chunk = fileo.read(bs)
> sock.sendall(chunk)
> (...)
>
> What is the best practise for a cross platform timeout handler?

The cross-platform 3.4 asyncio module has some functions with timeouts.
(3.5 has new 'async' syntac which supposedly makes it easier to use. I
have not looked at this yet.)

For instance: coroutine asyncio.wait(futures, *, loop=None,
timeout=None, return_when=ALL_COMPLETED)
Wait for the Futures and coroutine objects given by the sequence
futures to complete. Coroutines will be wrapped in Tasks. Returns two
sets of Future: (done, pending).
...
Usage:
done, pending = yield from asyncio.wait(fs)

I believe the backport on pypi.python.org, called tulip, works on 2.7.

In the example above, the read/send would be a task. Wait on the task,
and when it returns, cancel the task if in pending.

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: cross platform alternative for signal.SIGALRM? [ In reply to ]

Nov 11, 2015, 9:20 PM

Post #7 of 16 (4489 views)

On 11Nov2015 16:16, Ulli Horlacher <framstag@rus.uni-stuttgart.de> wrote:
>I am rewriting a Perl program into Python (2.7).
>It must run on Linux and Windows.
>With Linux I have no problems, but Windows... :-(
>
>The current show stopper is signal.SIGALRM which is not available on
>Windows:
>
> File "fexit.py", line 674, in formdata_post
> signal.signal(signal.SIGALRM,timeout_handler)
> AttributeError: 'module' object has no attribute 'SIGALRM'
>
> https://docs.python.org/2/library/signal.html
>
> signal.alarm(time) (...) Availability: Unix.
>
>Perl for Windows has had SIGALRM support (or some kind of emulation).
>
>Ok, I have to redesign this part of my code:
>
> def timeout_handler(sig,frame):
> raise ValueError("timeout!")
>
> signal.signal(signal.SIGALRM,timeout_handler)
>
> while True:
> chunk = fileo.read(bs)
> sock.sendall(chunk)
> (...)
>
>What is the best practise for a cross platform timeout handler?

I suggest you look at the socket.settimeout function. Avoid SIGALRM altogether.
Then (untested):

import socket
...
socket.settimeout(timeout_in_seconds)
...
while True:
...
chunk = fileo.read(bs)
try:
sock.sendall(chunk)
except socket.timeout as e:
... complain about timeout, reciting "e" in the message ...

Cheers,
Cameron Simpson <cs@zip.com.au>

I think you're confusing "recognizing" and "understanding" with "caring".
The net is cruel, sometimes, but always fair.
- Rick Gordon <rickg@crl.com>
--
https://mail.python.org/mailman/listinfo/python-list

Re: cross platform alternative for signal.SIGALRM? [ In reply to ]

marko at pacujo

Nov 11, 2015, 10:14 PM

Post #8 of 16 (4485 views)

Terry Reedy <tjreedy@udel.edu>:

> The cross-platform 3.4 asyncio module has some functions with
> timeouts.

Even that doesn't forcefully interrupt an obnoxious blocking function
call like

time.sleep(10000)

The original question claimed signal.alarm() would do the trick in
Linux. However, even that cannot be relied on as "man alarm" states:

sleep(3) may be implemented using SIGALRM; mixing calls to alarm()
and sleep(3) is a bad idea.

I'm thinking the only portable way is to run a watchdog process with
subprocess or multiprocessing.

Marko
--
https://mail.python.org/mailman/listinfo/python-list

Re: cross platform alternative for signal.SIGALRM? [ In reply to ]

auriocus at gmx

Nov 11, 2015, 10:43 PM

Post #9 of 16 (4483 views)

Am 12.11.15 um 07:14 schrieb Marko Rauhamaa:
> Terry Reedy <tjreedy@udel.edu>:
>
>> The cross-platform 3.4 asyncio module has some functions with
>> timeouts.
>
> Even that doesn't forcefully interrupt an obnoxious blocking function
> call like
>
> time.sleep(10000)

A blocking call - granted. But what happens in a blocking loop, i.e.

for i in range(10000000000000000000000000):
pass

?

My understanding of async is that it creates an event loop. In which
case the loop has no chance to run within a block of code that computes
anything, is that correct? Or does it hook into the interpreter and is
able to interrupt the program between bytecodes?

> I'm thinking the only portable way is to run a watchdog process with
> subprocess or multiprocessing.

What about a thread which calls exit() after the timeout? Does that
forcefully kill the whole process?

Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: cross platform alternative for signal.SIGALRM? [ In reply to ]

framstag at rus

Nov 11, 2015, 10:58 PM

Post #10 of 16 (4483 views)

Terry Reedy <tjreedy@udel.edu> wrote:
> On 11/11/2015 11:16 AM, Ulli Horlacher wrote:
> > I am rewriting a Perl program into Python (2.7).
>
> I recommend using 3.4+ if you possibly can.

It is not possible.
The main target platform offers only python 2.7

--
Ullrich Horlacher Server und Virtualisierung
Rechenzentrum IZUS/TIK E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart Tel: ++49-711-68565868
Allmandring 30a Fax: ++49-711-682357
70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/
--
https://mail.python.org/mailman/listinfo/python-list

Re: cross platform alternative for signal.SIGALRM? [ In reply to ]

framstag at rus

Nov 11, 2015, 11:22 PM

Post #11 of 16 (4484 views)

Marko Rauhamaa <marko@pacujo.net> wrote:

> I'm thinking the only portable way is to run a watchdog process with
> subprocess or multiprocessing.

How can a subprocess interrupt a function in another process?

For example: waiting for user input with a timeout.

raw_input("Hit ENTER to continue or wait 10 s")

--
Ullrich Horlacher Server und Virtualisierung
Rechenzentrum IZUS/TIK E-Mail: horlacher@tik.uni-stuttgart.de
Universitaet Stuttgart Tel: ++49-711-68565868
Allmandring 30a Fax: ++49-711-682357
70550 Stuttgart (Germany) WWW: http://www.tik.uni-stuttgart.de/
--
https://mail.python.org/mailman/listinfo/python-list

Re: cross platform alternative for signal.SIGALRM? [ In reply to ]

rosuav at gmail

Nov 11, 2015, 11:37 PM

Post #12 of 16 (4488 views)

On Thu, Nov 12, 2015 at 5:43 PM, Christian Gollwitzer <auriocus@gmx.de> wrote:
> My understanding of async is that it creates an event loop. In which case
> the loop has no chance to run within a block of code that computes anything,
> is that correct?

This is correct. At its simplest, asynchronous code is an abstraction
over the select() call, which basically says "Hey system, tell me when
(a) I can read from here, (b) I can write to here, or (c) I've been
waiting this long". The most common use is sockets; a web server has
its main listening socket (it becomes readable when someone connects),
any clients that haven't finished sending their requests yet (they
become readable when more data arrives), any clients that you're still
sending to (they become writeable when there's room in their output
buffers), and maybe some sort of periodic checks ("every hour, do
maintenance"). Whenever you finish a bit of processing (reading from a
client, sending to a client, whatever), you return to the "event
loop", which in this case would be select().

An async library makes all this look a lot cleaner in your code, but
ultimately, it's not preemptive. You still have to make sure the
processing doesn't take too long.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

Re: cross platform alternative for signal.SIGALRM? [ In reply to ]

marko at pacujo

Nov 12, 2015, 12:15 AM

Post #13 of 16 (4483 views)

Ulli Horlacher <framstag@rus.uni-stuttgart.de>:

> Marko Rauhamaa <marko@pacujo.net> wrote:
>
>> I'm thinking the only portable way is to run a watchdog process with
>> subprocess or multiprocessing.
>
> How can a subprocess interrupt a function in another process?
>
> For example: waiting for user input with a timeout.
>
> raw_input("Hit ENTER to continue or wait 10 s")

By sending the parent a signal with os.kill().

Now, signal handling in Python is brittle so you must be careful:

There is no way to â€œblockâ€ signals temporarily from critical sections
<URL: https://docs.python.org/2/library/signal.html>

Marko
--
https://mail.python.org/mailman/listinfo/python-list

Re: cross platform alternative for signal.SIGALRM? [ In reply to ]

tjreedy at udel

Nov 12, 2015, 2:15 AM

Post #14 of 16 (4483 views)

On 11/12/2015 2:37 AM, Chris Angelico wrote:
> On Thu, Nov 12, 2015 at 5:43 PM, Christian Gollwitzer <auriocus@gmx.de> wrote:
>> My understanding of async is that it creates an event loop. In which case
>> the loop has no chance to run within a block of code that computes anything,
>> is that correct?
>
> This is correct. At its simplest, asynchronous code is an abstraction
> over the select() call,

True on Unix-derived systems, where 'select' includes the various
derivatives. It is also an abstraction over the Windows completion
calls, which are quite different. The latter is why one must generally
use a different event loop on Windows. The point is that asyncio
provides an *abstraction* such that after choosing the event loop, the
rest of one's code is os-agnostic.

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: cross platform alternative for signal.SIGALRM? [ In reply to ]

rosuav at gmail

Nov 12, 2015, 3:38 AM

Post #15 of 16 (4481 views)

On Thu, Nov 12, 2015 at 9:15 PM, Terry Reedy <tjreedy@udel.edu> wrote:
> On 11/12/2015 2:37 AM, Chris Angelico wrote:
>>
>> On Thu, Nov 12, 2015 at 5:43 PM, Christian Gollwitzer <auriocus@gmx.de>
>> wrote:
>>>
>>> My understanding of async is that it creates an event loop. In which case
>>> the loop has no chance to run within a block of code that computes
>>> anything,
>>> is that correct?
>>
>>
>> This is correct. At its simplest, asynchronous code is an abstraction
>> over the select() call,
>
>
> True on Unix-derived systems, where 'select' includes the various
> derivatives. It is also an abstraction over the Windows completion calls,
> which are quite different. The latter is why one must generally use a
> different event loop on Windows. The point is that asyncio provides an
> *abstraction* such that after choosing the event loop, the rest of one's
> code is os-agnostic.

I've never done that kind of thing on Windows, so I'm not sure how it
works; it's still broadly based on I/O availability, right? And
ultimately, it comes down to "go back to the event loop so others can
run".

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

Re: cross platform alternative for signal.SIGALRM? [ In reply to ]

tjreedy at udel

Nov 12, 2015, 6:01 AM

Post #16 of 16 (4479 views)

On 11/12/2015 6:38 AM, Chris Angelico wrote:
> On Thu, Nov 12, 2015 at 9:15 PM, Terry Reedy <tjreedy@udel.edu> wrote:
>> On 11/12/2015 2:37 AM, Chris Angelico wrote:
>>>
>>> On Thu, Nov 12, 2015 at 5:43 PM, Christian Gollwitzer <auriocus@gmx.de>
>>> wrote:
>>>>
>>>> My understanding of async is that it creates an event loop. In which case
>>>> the loop has no chance to run within a block of code that computes
>>>> anything,
>>>> is that correct?
>>>
>>>
>>> This is correct. At its simplest, asynchronous code is an abstraction
>>> over the select() call,
>>
>>
>> True on Unix-derived systems, where 'select' includes the various
>> derivatives. It is also an abstraction over the Windows completion calls,
>> which are quite different. The latter is why one must generally use a
>> different event loop on Windows. The point is that asyncio provides an
>> *abstraction* such that after choosing the event loop, the rest of one's
>> code is os-agnostic.
>
> I've never done that kind of thing on Windows, so I'm not sure how it
> works; it's still broadly based on I/O availability, right?

That is the general abstraction. But there was something in the design
discussion about a difference between 'edge' versus 'level' triggering
that made it a challenge to get something more detailed that covers both
implementations.

> ultimately, it comes down to "go back to the event loop so others can
> run".

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list