Mailing List Archive: [issue41467] asyncio: recv_into() must not return b'' if the socket/pipe is closed

[issue41467] asyncio: recv_into() must not return b'' if the socket/pipe is closed

Aug 3, 2020, 3:19 PM

Post #1 of 11 (59 views)

New submission from STINNER Victor <vstinner@python.org>:

The proactor event loop of asyncio returns b'' on recv_into() if the socket/pipe is closed:

def recv_into(self, conn, buf, flags=0):
...
try:
...
except BrokenPipeError:
return self._result(b'')
...

But a socket recv_into() must always return a number: the number of written bytes. Example with socket.socket.recv_into() method:
https://docs.python.org/3/library/socket.html#socket.socket.recv_into

IMHO zero must be returned here. This bug may be the root cause of bpo-38912 bug.

Attached PR fix the issue.

----------
components: asyncio
messages: 374760
nosy: asvetlov, vstinner, yselivanov
priority: normal
severity: normal
status: open
title: asyncio: recv_into() must not return b'' if the socket/pipe is closed
versions: Python 3.10

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41467>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41467] asyncio: recv_into() must not return b'' if the socket/pipe is closed [ In reply to ]

Aug 3, 2020, 3:21 PM

Post #2 of 11 (59 views)

Change by STINNER Victor <vstinner@python.org>:

----------
keywords: +patch
pull_requests: +20863
stage: -> patch review
pull_request: https://github.com/python/cpython/pull/21720

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41467>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41467] asyncio: recv_into() must not return b'' if the socket/pipe is closed [ In reply to ]

Aug 3, 2020, 3:35 PM

Post #3 of 11 (59 views)

STINNER Victor <vstinner@python.org> added the comment:

The function was added in 2017 in bpo-31819, PR 4051:

commit 525f40d231aba2c004619fc7a5207171ed65b0cb
Author: Antoine Pitrou <pitrou@free.fr>
Date: Thu Oct 19 21:46:40 2017 +0200

bpo-31819: Add AbstractEventLoop.sock_recv_into() (#4051)

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41467>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41467] asyncio: recv_into() must not return b'' if the socket/pipe is closed [ In reply to ]

Aug 3, 2020, 3:44 PM

Post #4 of 11 (59 views)

STINNER Victor <vstinner@python.org> added the comment:

> This bug may be the root cause of bpo-38912 bug.

Yeah, very likely. This bug makes asyncio inconsistent. A transport is "not closed" and "closed" at the same time...

C:\vstinner\python\master\lib\asyncio\proactor_events.py:121: ResourceWarning: unclosed transport <_ProactorReadPipeTransport>
_warn(f"unclosed transport {self!r}", ResourceWarning, source=self)
ResourceWarning: Enable tracemalloc to get the object allocation traceback

Warning -- Unraisable exception
Exception ignored in: <function _ProactorBasePipeTransport.__del__ at 0x000001F30C938EB0>
Traceback (most recent call last):
File "C:\vstinner\python\master\lib\asyncio\proactor_events.py", line 122, in __del__
self.close()
File "C:\vstinner\python\master\lib\asyncio\proactor_events.py", line 114, in close
self._loop.call_soon(self._call_connection_lost, None)
File "C:\vstinner\python\master\lib\asyncio\base_events.py", line 746, in call_soon
self._check_closed()
File "C:\vstinner\python\master\lib\asyncio\base_events.py", line 510, in _check_closed
raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41467>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41467] asyncio: recv_into() must not return b'' if the socket/pipe is closed [ In reply to ]

Aug 3, 2020, 5:40 PM

Post #5 of 11 (59 views)

STINNER Victor <vstinner@python.org> added the comment:

New changeset 602a971a2af3a685d625c912c400cadd452718b1 by Victor Stinner in branch 'master':
bpo-41467: Fix asyncio recv_into() on Windows (GH-21720)
https://github.com/python/cpython/commit/602a971a2af3a685d625c912c400cadd452718b1

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41467>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41467] asyncio: recv_into() must not return b'' if the socket/pipe is closed [ In reply to ]

Aug 3, 2020, 5:40 PM

Post #6 of 11 (59 views)

Change by miss-islington <mariatta.wijaya+miss-islington@gmail.com>:

----------
nosy: +miss-islington
nosy_count: 3.0 -> 4.0
pull_requests: +20867
pull_request: https://github.com/python/cpython/pull/21724

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41467>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41467] asyncio: recv_into() must not return b'' if the socket/pipe is closed [ In reply to ]

Aug 3, 2020, 5:41 PM

Post #7 of 11 (59 views)

Change by miss-islington <mariatta.wijaya+miss-islington@gmail.com>:

----------
pull_requests: +20869
pull_request: https://github.com/python/cpython/pull/21726

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41467>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41467] asyncio: recv_into() must not return b'' if the socket/pipe is closed [ In reply to ]

Aug 3, 2020, 5:58 PM

Post #8 of 11 (59 views)

miss-islington <mariatta.wijaya+miss-islington@gmail.com> added the comment:

New changeset b934d832d1e16bf235c536dcde3006faf29757fc by Miss Islington (bot) in branch '3.8':
bpo-41467: Fix asyncio recv_into() on Windows (GH-21720)
https://github.com/python/cpython/commit/b934d832d1e16bf235c536dcde3006faf29757fc

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41467>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41467] asyncio: recv_into() must not return b'' if the socket/pipe is closed [ In reply to ]

Aug 3, 2020, 6:00 PM

Post #9 of 11 (59 views)

miss-islington <mariatta.wijaya+miss-islington@gmail.com> added the comment:

New changeset 1d16229f3f5b91f2389c7c5c6425c5524c413651 by Miss Islington (bot) in branch '3.9':
bpo-41467: Fix asyncio recv_into() on Windows (GH-21720)
https://github.com/python/cpython/commit/1d16229f3f5b91f2389c7c5c6425c5524c413651

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41467>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41467] asyncio: recv_into() must not return b'' if the socket/pipe is closed [ In reply to ]

Aug 4, 2020, 5:09 PM

Post #10 of 11 (41 views)

Change by STINNER Victor <vstinner@python.org>:

----------
resolution: -> fixed
stage: patch review -> resolved
status: open -> closed

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41467>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

[issue41467] asyncio: recv_into() must not return b'' if the socket/pipe is closed [ In reply to ]

Aug 5, 2020, 11:49 AM

Post #11 of 11 (41 views)

David Bolen <db3l.net@gmail.com> added the comment:

Just for the record, this fix also appears to have resolved the issue with the Win10 buildbot from bpo-41273, which was related to the same unraisable exceptions mentioned in bpo-38912.

----------
nosy: +db3l

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41467>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com