Mailing List Archive

[ python-Bugs-1759997 ] poll() on cygwin sometimes fails [PATCH]
Bugs item #1759997, was opened at 2007-07-24 17:58
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1759997&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Brian Warner (warner)
Assigned to: Nobody/Anonymous (nobody)
Summary: poll() on cygwin sometimes fails [PATCH]

Initial Comment:

While trying to track down a problem with our application (http://allmydata.org) running under cygwin, I discovered that the select.poll() object sometimes returns completely bogus data. poll() returns a list of tuples of (fd, revent), but fds are supposed to be small integers, and revents are a bitmask of POLLIN/POLLOUT flags. In my tests, I saw poll() return a list that started out looking normal, but the last half of the list contained fds and revents values like fd=0x7672a646, revent=0xd819.

It turns out that under cygwin-1.5.24 (which I believe is a pretty recent version), the poll() call sometimes violates the POSIX specification, and provides a return value which is different than the number of pollfd structures that have non-zero .revents fields (generally larger). This causes the implementation of poll_poll() (in Modules/selectmodule.c) to read beyond the end of the pollfd array, copying random memory into the python list it is building, causing the bogus values I observed during my tests.

These bogus values were mostly ignored, because the Twisted pollreactor that I was using noticed that the fd didn't correspond to any previously-registered file descriptor. It was only when the bogus fd happened to coincide with a real one (and when that indicated that a TCP listening socket became writable, which should never happen) that an exception was raised.

The attached patch (against 2.5.1) works around the problem by manually counting the number of non-zero .revents, rather than relying upon the return value from poll(). This version passes test_poll on both linux and cygwin.

cheers,
-Brian Warner




----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1759997&group_id=5470
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[ python-Bugs-1759997 ] poll() on cygwin sometimes fails [PATCH] [ In reply to ]
Bugs item #1759997, was opened at 2007-07-24 17:58
Message generated for change (Comment added) made by nnorwitz
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1759997&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Brian Warner (warner)
Assigned to: Nobody/Anonymous (nobody)
Summary: poll() on cygwin sometimes fails [PATCH]

Initial Comment:

While trying to track down a problem with our application (http://allmydata.org) running under cygwin, I discovered that the select.poll() object sometimes returns completely bogus data. poll() returns a list of tuples of (fd, revent), but fds are supposed to be small integers, and revents are a bitmask of POLLIN/POLLOUT flags. In my tests, I saw poll() return a list that started out looking normal, but the last half of the list contained fds and revents values like fd=0x7672a646, revent=0xd819.

It turns out that under cygwin-1.5.24 (which I believe is a pretty recent version), the poll() call sometimes violates the POSIX specification, and provides a return value which is different than the number of pollfd structures that have non-zero .revents fields (generally larger). This causes the implementation of poll_poll() (in Modules/selectmodule.c) to read beyond the end of the pollfd array, copying random memory into the python list it is building, causing the bogus values I observed during my tests.

These bogus values were mostly ignored, because the Twisted pollreactor that I was using noticed that the fd didn't correspond to any previously-registered file descriptor. It was only when the bogus fd happened to coincide with a real one (and when that indicated that a TCP listening socket became writable, which should never happen) that an exception was raised.

The attached patch (against 2.5.1) works around the problem by manually counting the number of non-zero .revents, rather than relying upon the return value from poll(). This version passes test_poll on both linux and cygwin.

cheers,
-Brian Warner




----------------------------------------------------------------------

>Comment By: Neal Norwitz (nnorwitz)
Date: 2007-07-24 22:53

Message:
Logged In: YES
user_id=33168
Originator: NO

Has this problem been reported to cygwin? Have they fixed the problem?

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1759997&group_id=5470
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[ python-Bugs-1759997 ] poll() on cygwin sometimes fails [PATCH] [ In reply to ]
Bugs item #1759997, was opened at 2007-07-24 17:58
Message generated for change (Comment added) made by warner
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1759997&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Brian Warner (warner)
Assigned to: Nobody/Anonymous (nobody)
Summary: poll() on cygwin sometimes fails [PATCH]

Initial Comment:

While trying to track down a problem with our application (http://allmydata.org) running under cygwin, I discovered that the select.poll() object sometimes returns completely bogus data. poll() returns a list of tuples of (fd, revent), but fds are supposed to be small integers, and revents are a bitmask of POLLIN/POLLOUT flags. In my tests, I saw poll() return a list that started out looking normal, but the last half of the list contained fds and revents values like fd=0x7672a646, revent=0xd819.

It turns out that under cygwin-1.5.24 (which I believe is a pretty recent version), the poll() call sometimes violates the POSIX specification, and provides a return value which is different than the number of pollfd structures that have non-zero .revents fields (generally larger). This causes the implementation of poll_poll() (in Modules/selectmodule.c) to read beyond the end of the pollfd array, copying random memory into the python list it is building, causing the bogus values I observed during my tests.

These bogus values were mostly ignored, because the Twisted pollreactor that I was using noticed that the fd didn't correspond to any previously-registered file descriptor. It was only when the bogus fd happened to coincide with a real one (and when that indicated that a TCP listening socket became writable, which should never happen) that an exception was raised.

The attached patch (against 2.5.1) works around the problem by manually counting the number of non-zero .revents, rather than relying upon the return value from poll(). This version passes test_poll on both linux and cygwin.

cheers,
-Brian Warner




----------------------------------------------------------------------

>Comment By: Brian Warner (warner)
Date: 2007-07-27 16:25

Message:
Logged In: YES
user_id=24186
Originator: YES

We've begun the process: zooko is working on a patch for cygwin and is
working with them to figure out how to compile the thing. We've not yet
explained the poll() bug to them in detail (wanting to have a patch in hand
first).

I'll report back once we get some word from them about how likely it is
this problem will be fixed on the cygwin side.


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-07-24 22:53

Message:
Logged In: YES
user_id=33168
Originator: NO

Has this problem been reported to cygwin? Have they fixed the problem?

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1759997&group_id=5470
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com
[ python-Bugs-1759997 ] poll() on cygwin sometimes fails [PATCH] [ In reply to ]
Bugs item #1759997, was opened at 2007-07-25 00:58
Message generated for change (Comment added) made by zooko
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1759997&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Extension Modules
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Brian Warner (warner)
Assigned to: Nobody/Anonymous (nobody)
Summary: poll() on cygwin sometimes fails [PATCH]

Initial Comment:

While trying to track down a problem with our application (http://allmydata.org) running under cygwin, I discovered that the select.poll() object sometimes returns completely bogus data. poll() returns a list of tuples of (fd, revent), but fds are supposed to be small integers, and revents are a bitmask of POLLIN/POLLOUT flags. In my tests, I saw poll() return a list that started out looking normal, but the last half of the list contained fds and revents values like fd=0x7672a646, revent=0xd819.

It turns out that under cygwin-1.5.24 (which I believe is a pretty recent version), the poll() call sometimes violates the POSIX specification, and provides a return value which is different than the number of pollfd structures that have non-zero .revents fields (generally larger). This causes the implementation of poll_poll() (in Modules/selectmodule.c) to read beyond the end of the pollfd array, copying random memory into the python list it is building, causing the bogus values I observed during my tests.

These bogus values were mostly ignored, because the Twisted pollreactor that I was using noticed that the fd didn't correspond to any previously-registered file descriptor. It was only when the bogus fd happened to coincide with a real one (and when that indicated that a TCP listening socket became writable, which should never happen) that an exception was raised.

The attached patch (against 2.5.1) works around the problem by manually counting the number of non-zero .revents, rather than relying upon the return value from poll(). This version passes test_poll on both linux and cygwin.

cheers,
-Brian Warner




----------------------------------------------------------------------

Comment By: Zooko O'Whielacronx (zooko)
Date: 2007-08-01 23:08

Message:
Logged In: YES
user_id=52562
Originator: NO

FYI, this is the issue ticket on the allmydata.org Tahoe project:

http://allmydata.org/trac/tahoe/ticket/31

I've written a patch for cygwin poll and am now testing it before
submitting it to the cygwin developers.

----------------------------------------------------------------------

Comment By: Brian Warner (warner)
Date: 2007-07-27 23:25

Message:
Logged In: YES
user_id=24186
Originator: YES

We've begun the process: zooko is working on a patch for cygwin and is
working with them to figure out how to compile the thing. We've not yet
explained the poll() bug to them in detail (wanting to have a patch in hand
first).

I'll report back once we get some word from them about how likely it is
this problem will be fixed on the cygwin side.


----------------------------------------------------------------------

Comment By: Neal Norwitz (nnorwitz)
Date: 2007-07-25 05:53

Message:
Logged In: YES
user_id=33168
Originator: NO

Has this problem been reported to cygwin? Have they fixed the problem?

----------------------------------------------------------------------

You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1759997&group_id=5470
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com