Mailing List Archive

Eryk Sun <eryksun@gmail.com> added the comment:

> The Windows cmd implementation clearly uses readline or
> some compatible lib under the hood

The REPL shell and input() call PyOS_Readline(). If this call isn't hooked (e.g. by the readline module), and stdin is a console file, then it reads a line from the console via ReadConsoleW(). If the console's input stream is configured in line-input mode, which we assume it is, this function provides basic readline-ish support, including a line editor that supports input history and aliases. The history is stored in an in-memory buffer in the console, which doesn't get save and reused across console sessions. There is no public API to get the history contents, and there is no support at all to set it.

The console/terminal team at Microsoft apparently don't want to do anything with the builtin readline support, which is seen as a legacy feature. To the contrary, I've even seen them suggest that they're evaluating the addition of a new client-side readline library in the native API, which I assume would be similar to PowerShell's PSReadLine (e.g. based on ReadConsoleInputW(), and supporting a classic console mode in addition to emacs and vi modes).

For now, I suggest patching pyreadline for your own use. In the reported issue, I see it's trying to use collections.Callable. Instead it should import collections.abc and use collections.abc.Callable.

----------
nosy: +eryksun

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45870>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Terry J. Reedy <tjreedy@udel.edu> added the comment:

Thank you Eryk for the info.

As a bug report, this should be closed as '3rd party'. As an enhancement request, it needs to be specified more and should perhaps be discussed on python-ideas.

----------
nosy: +terry.reedy

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45870>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

keeely <smh@sophos.com> added the comment:

You can take the view that it's not a bug (with some justification), but a few lines in the cmd docs would make all the difference in terms of wasted time.

I have now abandoned my Windows port and suggested users install WSL2 instead which is the easiest way forward for me, but it'd be nice to have known from the start that portions of cmd functionality are not available for Win32 instead of the indirect references via readline. You could throw us a bone here.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45870>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Terry J. Reedy <tjreedy@udel.edu> added the comment:

What specific sentences would you like where in which doc. (Please link as 'cmd doc' is too vague.)

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45870>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Terry J. Reedy <tjreedy@udel.edu> added the comment:

Sorry, you obviously mean
https://docs.python.org/3/library/cmd.html#module-cmd
What to add where still applies.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45870>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Terry J. Reedy <tjreedy@udel.edu> added the comment:

Guido and Raymond, you are the two active coredevs that have contributed the most lines to cmd module. What do either of you think?

----------
nosy: +gvanrossum, rhettinger

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45870>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Eryk Sun <eryksun@gmail.com> added the comment:

> You can take the view that it's not a bug (with some justification),
> but a few lines in the cmd docs would make all the difference in
> terms of wasted time.

If anything, I think the readline documentation should have a note explaining the situation in Windows. The documentation of the cmd module already makes the readline dependency clear:

If the readline module is loaded, input will automatically inherit
bash-like history-list editing (e.g. Control-P scrolls back to the
last command, Control-N forward to the next one, Control-F moves the
cursor to the right non-destructively, Control-B moves the cursor to
the left non-destructively, etc.).

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45870>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Guido van Rossum <guido@python.org> added the comment:

AFAIK the reason command history works in cmd.py on Windows is that it's built into the terminal program. Or maybe into the operating system.

Thus, the user can use line editing and history, but there is no API (in Python) to interact with these.

I'm sure Steve Dower can explain the exact situation -- it may depend on which Windows version and which terminal program you use (my only recent experience is with winterm on Windows 10).

I agree that (once we sort out what works in what versions of Windows and which terminal programs) we should clarify this in the docs.

----------
nosy: +steve.dower

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45870>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Eryk Sun <eryksun@gmail.com> added the comment:

> AFAIK the reason command history works in cmd.py on Windows is
> that it's built into the terminal program. Or maybe into the
> operating system.

As mentioned in msg406800, input editing, history (e.g. up/down arrows, F7 popup, F8 completion), and alias support is implemented by the Windows console host (conhost.exe or openconsole.exe) for ReadFile() and ReadConsole() calls when the input stream is in line-input mode. Currently, it's the same whether a classic console session or a pseudoconsole (headless) session is hosted. When Windows Terminal is used, the overall connection of components looks like Python<->ConDrv (kernel device)<->OpenConsole<->NamedPipe (kernel device)<->Windows Terminal. The headless console's use of Windows Terminal for the user interface doesn't matter to Python's ReadConsoleW() call.

A headless console session always starts with 4 history buffers (one for each attached process) that store up to 50 commands. For a classic console session, the initial number and size of history buffers can be configured in the session properties or defaults. It can always be set dynamically via SetConsoleHistoryInfo(). There's *undocumented* support to get the commands from a history buffer that's associated with the name of an attached process: GetConsoleCommandHistoryLengthW(executable_name) and GetConsoleCommandHistoryW(buffer, buffer_length, executable_name). However, the API provides no function to set the command history. I suppose one could loop over WriteConsoleInputW() and ReadConsoleW() to implement it as a kludge.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45870>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Guido van Rossum <guido@python.org> added the comment:

Thanks, Eryk, I only read the part of the issue that landed in my inbox (fhe first message and everything after Terry added me to the nosy list). Sorry.

You wrote:

> The console/terminal team at Microsoft apparently don't want to do anything with the builtin readline support, which is seen as a legacy feature.

What does "the builtin readline support" refer to here? Presumably not GNU Readline?

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45870>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Eryk Sun <eryksun@gmail.com> added the comment:

> What does "the builtin readline support" refer to here?
> Presumably not GNU Readline?

That's referring to the readline(ish) support that's built into the console host for ReadFile() and ReadConsole() calls when the input stream is in line-input mode. I've never seen the console developers speak positively of this feature on their GitHub repo. They've suggested the addition of a native readline API on the client side, like PowerShell's PSReadLine module provides. But who knows when/if that would be released.

Python has the third-party pyreadline module, but it's no longer actively developed. To bring pyreadline into the standard library would be a non-trivial task. OTOH, I assume if Microsoft provided an official readline API, which does all the heavy lifting, that Python could support it in the readline extension module, if the API is basically compatible with libreadline/libedit.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45870>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

Guido van Rossum <guido@python.org> added the comment:

Okay, so that's all hypothetical. It looks like the status quo is not
likely to change, so we should just document it. I wonder if keeely is
interested in submitting a PR for the docs?

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45870>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

keeely <smh@sophos.com> added the comment:

Regrettably I cannot submit a PR for the docs because I value my online anonymity and Python submissions require a real name (IIRC), but my suggestion would be pretty simple.

Taking as an example, for termios (https://docs.python.org/3/library/termios.html), we currently have:

This module provides an interface to the POSIX calls for tty I/O control. For a complete description of these calls, see termios(3) Unix manual page. It is only available for those Unix versions that support POSIX termios style tty I/O control configured during installation.


For readline (https://docs.python.org/3/library/readline.html#module-readline) we have:

The readline module defines a number of functions to facilitate completion and reading/writing of history files from the Python interpreter. This module can be used directly, or via the rlcompleter module, which supports completion of Python identifiers at the interactive prompt. Settings made using this module affect the behaviour of both the interpreter’s interactive prompt and the prompts offered by the built-in input() function.

In similar way to the first para of the termios description I would add the following text: “It is only available on platforms that support the readline functionality, generally POSIX”.


Then perhaps I’d also add to the cmd documentation at https://docs.python.org/3/library/cmd.html

The Cmd class provides a simple framework for writing line-oriented command interpreters. These are often useful for test harnesses, administrative tools, and prototypes that will later be wrapped in a more sophisticated interface.

I would add at the end of that first paragraph:
“Some features will be unavailable on non-POSIX platforms due to the readline requirement”.

Hope this helps, if it's not clear I can provide in diff form if you prefer.

----------

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45870>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com

keeely <smh@sophos.com> added the comment:

I'm attaching an example usage of cmd + readline to show how you can have context-specific history for sub-shells. WARNING: WRITES FILES TO CWD! In the event that someone does implement this on Windows it would be nice if this worked. That doesn't mean less-capable readline support wouldn't also be useful.

thanks.

----------
Added file: https://bugs.python.org/file50461/cmd_shell_example.py

_______________________________________
Python tracker <report@bugs.python.org>
<https://bugs.python.org/issue45870>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/list-python-bugs%40lists.gossamer-threads.com