Mailing List Archive

On Tue, Jul 13, 2021 at 5:22 AM lucas <lucas@bourneuf.net> wrote:
>
> Hello everyone,
>
> Let us consider this patch of code:
>
> import argparse
>
> def parse_cli() -> argparse.Namespace:
> parser = argparse.ArgumentParser()
> parser.add_argument('n', type=int)
> return parser.parse_args()
>
> args = parse_cli()
> print(args.n + ' ') # type error
>
> Running CPython on it will raise a TypeError, and running Mypy on it
> will indicate that no issues were found.
>
> I was wondering if there is any way for me to have mypy detecting the
> args.n type, based on the type keyword of the parser.add_argument function ?
>
> It appears that some type annotations were added to tierce party
> modules, provided by mypy itself. Is there a technical issue preventing
> such work to be made for argparse (or other CLI ; i didn't find anything
> for others either)
>

Seems complicated, since it depends on a lot of run-time information.
What if you flip the problem on its head? Instead of creating the
argparser and letting that govern the types, maybe create a dataclass,
and then programmatically build the parser.

from dataclasses import dataclass
import argparse

@dataclass
class Args:
n: int

def parse_cli() -> Args:
parser = argparse.ArgumentParser()
for field, typ in Args.__dataclass_fields__.items():
if hasattr(typ, "type"): typ = typ.type # Python 3.10 changed
things a bit
parser.add_argument(field, type=typ)
return Args(**vars(parser.parse_args()))

args = parse_cli()
print(args.n + ' ')


Only barely tested it and didn't try MyPy, but that's the basic idea.
You'd have to figure out the tidiest way to define all the other
attributes of your arguments (help text, etc), but ideally, all the
info should be able to be coded in the dataclass.

Incidentally, you could choose to make parse_cli into a classmethod of
Args. Might be cleaner.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

Mmmh, that may work just fine !
Thanks for that idea, ChrisA, i'm looking forward to implement that.

Best wishes,
--lucas



On 12/07/2021 21:33, Chris Angelico wrote:
> On Tue, Jul 13, 2021 at 5:22 AM lucas <lucas@bourneuf.net> wrote:
>>
>> Hello everyone,
>>
>> Let us consider this patch of code:
>>
>> import argparse
>>
>> def parse_cli() -> argparse.Namespace:
>> parser = argparse.ArgumentParser()
>> parser.add_argument('n', type=int)
>> return parser.parse_args()
>>
>> args = parse_cli()
>> print(args.n + ' ') # type error
>>
>> Running CPython on it will raise a TypeError, and running Mypy on it
>> will indicate that no issues were found.
>>
>> I was wondering if there is any way for me to have mypy detecting the
>> args.n type, based on the type keyword of the parser.add_argument function ?
>>
>> It appears that some type annotations were added to tierce party
>> modules, provided by mypy itself. Is there a technical issue preventing
>> such work to be made for argparse (or other CLI ; i didn't find anything
>> for others either)
>>
>
> Seems complicated, since it depends on a lot of run-time information.
> What if you flip the problem on its head? Instead of creating the
> argparser and letting that govern the types, maybe create a dataclass,
> and then programmatically build the parser.
>
> from dataclasses import dataclass
> import argparse
>
> @dataclass
> class Args:
> n: int
>
> def parse_cli() -> Args:
> parser = argparse.ArgumentParser()
> for field, typ in Args.__dataclass_fields__.items():
> if hasattr(typ, "type"): typ = typ.type # Python 3.10 changed
> things a bit
> parser.add_argument(field, type=typ)
> return Args(**vars(parser.parse_args()))
>
> args = parse_cli()
> print(args.n + ' ')
>
>
> Only barely tested it and didn't try MyPy, but that's the basic idea.
> You'd have to figure out the tidiest way to define all the other
> attributes of your arguments (help text, etc), but ideally, all the
> info should be able to be coded in the dataclass.
>
> Incidentally, you could choose to make parse_cli into a classmethod of
> Args. Might be cleaner.
>
> ChrisA
>
--
https://mail.python.org/mailman/listinfo/python-list

On Mon, Jul 12, 2021 at 12:34 PM Chris Angelico <rosuav@gmail.com> wrote:

> On Tue, Jul 13, 2021 at 5:22 AM lucas <lucas@bourneuf.net> wrote:
> > Running CPython on it will raise a TypeError, and running Mypy on it
> > will indicate that no issues were found.
> >
> > I was wondering if there is any way for me to have mypy detecting the
> > args.n type, based on the type keyword of the parser.add_argument
> function ?
> >
> > It appears that some type annotations were added to tierce party
> > modules, provided by mypy itself. Is there a technical issue preventing
> > such work to be made for argparse (or other CLI ; i didn't find anything
> > for others either)
> >
>
> Seems complicated, since it depends on a lot of run-time information.
> What if you flip the problem on its head? Instead of creating the
> argparser and letting that govern the types, maybe create a dataclass,
> and then programmatically build the parser.
>

This is why I eschew argparse. I instead frequently do something like:

def usage(retval):
"""Output a usage message."""
if retval:
write = sys.stderr.write
else:
write = sys.stdout.write

write(f'{sys.argv[0]} --age 50 --help\n')

sys.exit(retval)


def main():
"""Compute maximum heart rate as a function of age."""
age = -1
while sys.argv[1:]:
if sys.argv[1] == '--age':
age = float(sys.argv[2])
del sys.argv[1]
elif sys.argv[1] in ('-h', '--help'):
usage(0)
else:
sys.stderr.write(f'{sys.argv[0]}: unrecognized option:
{sys.argv[1]}\n')
usage(1)
del sys.argv[1]

if age == -1:
sys.stderr.write('--age is a required option\n')
usage(1)


It is of course a little more typing, but tools like mypy and pylint are
much more effective this way. This is a small program, but in a large one,
this approach can really help with correctness.

If someone has a mypy extension/plugin that can do argparse as well, I may
switch. Until then....
--
https://mail.python.org/mailman/listinfo/python-list

On Thu, Jul 15, 2021 at 2:57 PM Dan Stromberg <drsalists@gmail.com> wrote:
>
>
> On Mon, Jul 12, 2021 at 12:34 PM Chris Angelico <rosuav@gmail.com> wrote:
>>
>> On Tue, Jul 13, 2021 at 5:22 AM lucas <lucas@bourneuf.net> wrote:
>> > Running CPython on it will raise a TypeError, and running Mypy on it
>> > will indicate that no issues were found.
>> >
>> > I was wondering if there is any way for me to have mypy detecting the
>> > args.n type, based on the type keyword of the parser.add_argument function ?
>> >
>> > It appears that some type annotations were added to tierce party
>> > modules, provided by mypy itself. Is there a technical issue preventing
>> > such work to be made for argparse (or other CLI ; i didn't find anything
>> > for others either)
>> >
>>
>> Seems complicated, since it depends on a lot of run-time information.
>> What if you flip the problem on its head? Instead of creating the
>> argparser and letting that govern the types, maybe create a dataclass,
>> and then programmatically build the parser.
>
>
> This is why I eschew argparse. I instead frequently do something like:
>
> def usage(retval):
> """Output a usage message."""
> if retval:
> write = sys.stderr.write
> else:
> write = sys.stdout.write
>
> write(f'{sys.argv[0]} --age 50 --help\n')
>
> sys.exit(retval)
>
>
> def main():
> """Compute maximum heart rate as a function of age."""
> age = -1
> while sys.argv[1:]:
> if sys.argv[1] == '--age':
> age = float(sys.argv[2])
> del sys.argv[1]
> elif sys.argv[1] in ('-h', '--help'):
> usage(0)
> else:
> sys.stderr.write(f'{sys.argv[0]}: unrecognized option: {sys.argv[1]}\n')
> usage(1)
> del sys.argv[1]
>
> if age == -1:
> sys.stderr.write('--age is a required option\n')
> usage(1)
>
>
> It is of course a little more typing, but tools like mypy and pylint are much more effective this way. This is a small program, but in a large one, this approach can really help with correctness.
>
> If someone has a mypy extension/plugin that can do argparse as well, I may switch. Until then....
>

The more complicated the program, the more you lose by this approach.
Yes, mypy is satisfied with your code, but your argument parsing is
more rigid and you don't get all the other benefits of argparse. Plus
it's all too easy to make a copy and paste error, where you make a new
parameter that accidentally puts its value into the same variable as
an old parameter, or something.

Much better to make a wrapper around argparse. I've made good use of
clize [1], and have made ad-hoc wrappers for various purposes. Try
adding mypy support to something like that instead - it's likely to be
a lot easier. In fact, since clize primarily uses function arguments,
it'd probably Just Work™ for the most part, although I haven't tried.

[1] https://pypi.org/project/clize/

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list

>>> [1] https://pypi.org/project/clize/

I use and like docopt (https://github.com/docopt/docopt). Is clize a
better choice?
--
https://mail.python.org/mailman/listinfo/python-list

On Fri, Jul 23, 2021 at 5:34 PM Albert-Jan Roskam
<sjeik_appie@hotmail.com> wrote:
>
> >>> [1] https://pypi.org/project/clize/
>
>
> I use and like docopt (https://github.com/docopt/docopt). Is clize a better choice?
>

Not necessarily. Both are good. Explore both, see which one makes more sense.

ChrisA
--
https://mail.python.org/mailman/listinfo/python-list