Mailing List Archive: [issue41406] subprocess: Calling Popen.communicate() after Popen.stdout.read() returns an empty string

STINNER Victor <vstinner@python.org> added the comment:

Calling proc.communicate() after proc.stdout.read() doesn't seem to be supported. What is your use case? Why not just calling communicate()? Why not only using stdout directly?

----------
components: -2to3 (2.x to 3.x conversion tool), IO
nosy: +gregory.p.smith
title: BufferedReader causes Popen.communicate losing the remaining output. -> subprocess: Calling Popen.communicate() after Popen.stdout.read() returns an empty string
versions: -Python 3.6, Python 3.7, Python 3.8, Python 3.9

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41406>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Gregory P. Smith <greg@krypto.org> added the comment:

A workaround should be pass bufsize=0.

There might be performance consequences. That depends on your read patterns and child process.

If this is to be supported and fixed, the selectors used in POpen._communicate on the POSIX side presumably don't bother to look at buffered IO objects buffer. https://github.com/python/cpython/blob/master/Lib/subprocess.py#L1959

manually consuming data from the stdout and stderr buffers, if any, before entering that loop is probably a fix.

Higher up the chain, should the https://docs.python.org/3/library/selectors.html be enhanced to support emptying the buffer on buffered IO objects? That sounds complicated; probably even infeasible if in text mode. In general it is understood that poll/select type APIs are meant to be used on unbuffered raw binary file objects.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue41406>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com