Mailing List Archive

1 2 3  View All
RE: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument? [ In reply to ]
Although today you could say POSIX is the reason for many things including
the use of "--" I hesitate to mention I and many others used that convention
long before as a standard part of many UNIX utilities. Like many other such
things, you build things first and by the time you standardize, ...


-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com@python.org> On
Behalf Of 2QdxY4RzWzUUiLuE@potatochowder.com
Sent: Sunday, January 29, 2023 7:12 AM
To: python-list@python.org
Subject: Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string
argument?

On 2023-01-29 at 16:51:20 +1100,
Cameron Simpson <cs@cskk.id.au> wrote:

> They're unrelated. As others have mentioned, "--" is _extremely_
> common; almost _all_ UNIX command like programmes which handle -*
> style options honour the "--" convention. _argparse_ itself honours
> that convention, as does getopt etc.

And why do UNIX programs behave this way?

Because POSIX says they should:

https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html
--
https://mail.python.org/mailman/listinfo/python-list

--
https://mail.python.org/mailman/listinfo/python-list
RE: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument? [ In reply to ]
Cameron,

You are technically correct but perhaps off the mark.

Yes, a python program only sees what is handed to it by some shell if invoked a certain way.

The issue here is what you tell people using your program about what they need to type to get it to work. That means if their shell is going to make changes in what they typed, they need to know how to avoid unintended changes. As one example not mentioned, whitespace disappears if not somehow protected as in quotes.

What the OP is being told is that their Python program only controls what is fed to it. A user needs to know enough to avoid doing silly things like provide an unquoted string containing reserved symbols like a pipe symbol or odd things may happen and their program may not even be called.

So the documentation of how to use the program may need to spell some things out alongside suggesting use of "--" ...


-----Original Message-----
From: Python-list <python-list-bounces+avi.e.gross=gmail.com@python.org> On Behalf Of Cameron Simpson
Sent: Sunday, January 29, 2023 12:51 AM
To: python-list@python.org
Subject: Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument?

On 28Jan2023 18:55, Jach Feng <jfong@ms4.hinet.net> wrote:
>Mark Bourne ? 2023?1?28? ?????10:00:01 [UTC+8] ??????
>> I notice you explain the need to enclose the equation in quotes if it
>> contains spaces. That's not even a feature of your application, but
>> of the shell used to call it. So why so much objection to explaining
>> the need for "--"?
>>
>> Depending on the shell, there are other cases where quoting might be
>> needed, e.g. if the equation includes a "*" or "?" and happens to
>> look like a pattern matching files in the current directory (most
>> shells I've used pass the pattern unchanged if it doesn't match any
>> files). In bash, if a "$" is used I'd need to enclose that in 'single quotes'
>> (can't even use "double quotes" for that one). You can't really
>> expect to document all that sort of thing, because it depends on
>> which shell the user happens to run your application from - you just
>> have to trust the user to know or learn how to use their shell.
>
>Thank you for detail explanation of the role the shell is involved in this problem. I'm very appreciated!

The shell has basicly _nothing_ to do with your problems. By the time you've got sys.argv in your Python programme you will have no idea whether quotes were used with an argument. (UNIX/POSIX, not Windows, where things are ... more complex.) This means you don't know if the use
typed:

-4.5

or

"-4.5"

You'll just get a string '4.5' in your Python programme both ways.

All the quotes in the shell do is delimit what things should be kept together as a single argument versus several, or where variables should be interpolated when computing arguments etc. It's just _shell_ punctuation and the invoked programme doesn't see it.

>It seems that a CLI app may become very complex when dealing with different kind of shell, and may not be possible to solve its problem.

It doesn't matter what shell is used. The just controls what punctuation the end user may need to use to invoke your programme. You programme doesn't need to care (and can't because it doesn't get the quotes etc, only their result).

>> So why so much objection to explaining the need for "--"?
>Because of using " to enclose a space separated string is a common
>convention, and adding a "--" is not:-)

They're unrelated. As others have mentioned, "--" is _extremely_ common; almost _all_ UNIX command like programmes which handle -* style options honour the "--" convention. _argparse_ itself honours that convention, as does getopt etc.

The "--" convention has nothing to do with the shell.

Cheers,
Cameron Simpson <cs@cskk.id.au>
--
https://mail.python.org/mailman/listinfo/python-list

--
https://mail.python.org/mailman/listinfo/python-list
Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument? [ In reply to ]
On 2023-01-29 07:59:21 -0500, avi.e.gross@gmail.com wrote:
> Although today you could say POSIX is the reason for many things including
> the use of "--" I hesitate to mention I and many others used that convention
> long before as a standard part of many UNIX utilities.

I somewhat doubt that you are remembering correctly. The POSIX standard
was published in 1988. I was starting to use Unix shortly before that
and I don't think there was really a common way to indicate the end of
the options. Some utilities used "-", some "--", some just stopped at
the first argument which didn't look like an option ...

The convergence to "--" happened later. My guess is that GNU getopt and
the popularity of the GNU utilities had a lot to do with it.

> Like many other such things, you build things first and by the time
> you standardize, ...

Right. But in this case lots of people were building different things
and then *one* of them was standardized. Truth to be told, I don't know
when that happened. It might have been in the original 1988 edition of
POSIX, but I don't think so. I think it was in a later (possibly much
later) edition, wennn "--" was already really common. But POSIX did lead
the way in some cases and it might have been the case here, too.

hp

--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | hjp@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"
Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument? [ In reply to ]
On 2023-01-27 21:03:39 -0800, Jach Feng wrote:
> I have to admit that I don't know the background upon which the
> argparse was built. The good side is that I don't carry its historical
> knowledge ( or burden?), that's why I can use it in a new way

I don't see what's new about that.

> which may make someone feel uneasy.

It seems that the person who feels most uneasy about it is yourself
because it doesn't work like you want it to.


> The reason I am still keep on using argparse on this "one positional
> argument only" CLI app is that I can't see what advantage I can get
> when writing code to handling sys.argv directly for the following two
> situations,

It would almost certainly be shorter and easier to understand,
especially since you are *also* handling sys.argv directly in your code
to prevent argparse to work as designed.

But since you are probably the only person who will have to read your
code in the future that's not our problem.

hp

--
_ | Peter J. Holzer | Story must make more sense than reality.
|_|_) | |
| | | hjp@hjp.at | -- Charles Stross, "Creative writing
__/ | http://www.hjp.at/ | challenge!"
Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument? [ In reply to ]
Jach Feng wrote:
> Jach Feng ? 2023?1?22? ?????11:11:22 [UTC+8] ??????
>> Fail on command line,
>>
>> e:\Works\Python>py infix2postfix.py "-4^2+5.3*abs(-2-1)/2"
>> usage: infix2postfix.py [-h] [infix]
>> infix2postfix.py: error: unrecognized arguments: -4^2+5.3*abs(-2-1)/2
>>
>> Also fail in REPL,
>>
>> e:\Works\Python>py
>> Python 3.8.8 (tags/v3.8.8:024d805, Feb 19 2021, 13:08:11) [MSC v.1928 32 bit (Intel)] on win32
>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> import argparse
>>>>> parser = argparse.ArgumentParser(description='Convert infix notation to postfix')
>>>>> parser.parse_args("-4^2+5.3*abs(-2-1)/2")
>> usage: [-h]
>> : error: unrecognized arguments: - 4 ^ 2 + 5 . 3 * a b s ( - 2 - 1 ) / 2
>>
>> Just can't figure out where is wrong!?
>>
>> --Jach
> OK, I take a quick try to use argv directly. I write the code in a way even get rid of the -h option.
>
> import sys
> if len(sys.argv) == 1:
> ....infix = " "
> ....print("Usage: a math equation must follow!")
> else:
> ....infix = "".join(sys.argv[1:])
>
> Simple enough, right? But unfortunately it didn't survive. The ^ symbol was lost!
>
> e:\Works\Python>py infix2postfix.py
> Usage: a math equation must follow!
>
> e:\Works\Python>py infix2postfix.py -4^2 +5.3 * abs(-2-1)/2
> -42 5.3 -2 1 - abs * 2 / +
>
> Hmm...

I'm not certain, but I think that's a shell feature again. In Windows'
command prompt, "^" can be used at the end of a line to indicate that
the command continues on the next line. I'm not sure what happens if
it's in the middle of a line (and not on Windows to be able to check),
but it's quite possible that it just gets ignored.

Enclosing your argument in quotes might help. Again, this is a feature
of the shell, not your application, and other shells might behave
differently. You probably don't want to go down the line of trying to
document this kind of thing in your applications usage information,
because it won't work like that for someone using e.g. the bash shell
(which can be run on Windows).

--
Mark.
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument? [ In reply to ]
Jach Feng wrote:
> Thank you for detail explanation of the role the shell is involved in this problem. I'm very appreciated!
>
> It seems that a CLI app may become very complex when dealing with different kind of shell, and may not be possible to solve its problem. But the good thing in my app is that I need only to handle math equation:-)

If you want to try to tell the user how to deal with their shell's
requirements for quoting arguments, regardless of which shell they might
be using, yes, that explanation would become very complicated. It
doesn't affect the rest of the implementation of the application though
- the user just needs to know how to use their shell to pass the
arguments they want into the application. That's really something they
should look up in their shell's documentation, rather than something
your application should attempt to document.

Since your application requires equations to be passed in, and they're
quite likely to include characters handled specially by the shell
(space, "*" and "^" have already come up in this thread, but there may
be others), it may be worth highlighting that, but directing the user to
consult the documentation for their shell rather than assuming a
particular shell and attempting to cover all its features and limitations.

>> So why so much objection to explaining the need for "--"?
> Because of using " to enclose a space separated string is a common convention, and adding a "--" is not:-)

If you don't consider use of "--" to be a common convention, that seems
even more reason to mention it in your application's documentation.

Telling the user to use quotes around an argument containing spaces is
nothing to do with your application, and might not even be applicable if
they use a different shell to call your application. In most shells
I've come across, there are also various other characters that need
special handling (either quoting or escaping) - but exactly which
characters again depends on the shell. Yet you seem quite happy to
document that one particular case in your usage information.

Using "--" is also a common convention (as I and others have mentioned),
although perhaps not as common in Windows (where it's more common to use
"/" rather than "-" for options anyway). But more to the point, it is a
feature that is implemented under your application's control (if you
don't want this feature, don't use argparse). Use of "--" is applicable
regardless of which shell your user calls it from, and other
applications might not use that convention even if called from the same
shell, so it seems *more* in scope for your application to document than
using quotes around spaces.

--
Mark.
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument? [ In reply to ]
This thread has run its course and seems to now be generating more heat than light.

It is now closed (at least on the Python List side).

Thank you everyone for your participation and understanding.

--
~Ethan~
Moderator
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument? [ In reply to ]
On 29Jan2023 07:12, 2QdxY4RzWzUUiLuE@potatochowder.com <2QdxY4RzWzUUiLuE@potatochowder.com> wrote:
>On 2023-01-29 at 16:51:20 +1100,
>Cameron Simpson <cs@cskk.id.au> wrote:
>> They're unrelated. As others have mentioned, "--" is _extremely_
>> common;
>> almost _all_ UNIX command like programmes which handle -* style options
>> honour the "--" convention. _argparse_ itself honours that convention, as
>> does getopt etc.
>
>And why do UNIX programs behave this way?
>
>Because POSIX says they should:
>
> https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html

Aye, but POSIX says they should because POSIX formalises a lot of
existing practices so that people have a reference to let them _also_
use that practice in a consistent way.

I used to lurk on the POSIX discussion list, and the length of the
discussions around corner cases was quite wearing, even just to read.

Cheers,
Cameron Simpson <cs@cskk.id.au>
--
https://mail.python.org/mailman/listinfo/python-list
Re: How to make argparse accept "-4^2+5.3*abs(-2-1)/2" string argument? [ In reply to ]
On 29Jan2023 08:10, avi.e.gross@gmail.com <avi.e.gross@gmail.com> wrote:
>You are technically correct but perhaps off the mark.
>
>Yes, a python program only sees what is handed to it by some shell if invoked a certain way.
>
>The issue here is what you tell people using your program about what they need to type to get it to work. That means if their shell is going to make changes in what they typed, they need to know how to avoid unintended changes. As one example not mentioned, whitespace disappears if not somehow protected as in quotes.

Hmm, there is that. But the OP needs clarity on what happens in a shell
and what happens in a programme once the shell has invoked it for a
user.

Cheers,
Cameron Simpson <cs@cskk.id.au>
--
https://mail.python.org/mailman/listinfo/python-list

1 2 3  View All