Mailing List Archive: Making command-line args available to deeply-nested functions

Making command-line args available to deeply-nested functions

Aug 20, 2021, 2:53 AM

Post #1 of 7 (290 views)

Hi,

TL;DR:

If I have a command-line argument for a program, what is the best way
of making this available to a deeply-nested[1] function call without
passing the parameter through every intermediate function?

Long version:

If I have, say, a command-line program to send an email with a
personalised salutation, a naive approach to the function calls might
look like the following

create_email(..., args.salutation_server_credentials)
create_body(..., args.salutation_server_credentials)
create_salutation(..., args.salutation_server_credentials)

where args.salutation_server_credentials could be given on the
command-line or read from a configuration file by the top-level, but is
only ever actually needed by the create_salutation function.

I can see that the top-level could just create an object from a class
which encapsulates everything, but what if I want to keep the salutation
generation separate, so that I can have a separate program which just
generates the salutation and print it to the terminal?

I guess I am really asking how to avoid "passing through" arguments to
functions which only need them to call other functions, so maybe the
answer is just to avoid nesting.

Cheers,

Loris

Footnotes:

[1] Is a TL;DR allowed to have footnotes? Probably not, but just to
clarify, I would consider the third of three levels as being
already "deeply-nested".

--
This signature is currently under construction.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Making command-line args available to deeply-nested functions [ In reply to ]

loris.bennett at fu-berlin

Aug 23, 2021, 2:27 AM

Post #2 of 7 (285 views)

Permalink

George Fischhof <george@fischhof.hu> writes:

> Loris Bennett <loris.bennett@fu-berlin.de> ezt írta (id?pont: 2021. aug.
> 20., P 17:54):
>
>> Julio Di Egidio <julio@diegidio.name> writes:
>>
>> > On Friday, 20 August 2021 at 11:54:00 UTC+2, Loris Bennett wrote:
>> >> Hi,
>> >>
>> >> TL;DR:
>> >>
>> >> If I have a command-line argument for a program, what is the best way
>> >> of making this available to a deeply-nested[1] function call without
>> >> passing the parameter through every intermediate function?
>> >
>> > To not pass arguments you need shared state ("global variables"): and
>> > options in shard state, unless you have very good reasons to do
>> > otherwise, is simply a no no.
>>
>> Doesn't that slightly depend on the size of your "globe"? If a program
>> does a few, in some sense unrelated things, and, say, only runs for a
>> few minutes, could you not decide to make a particular parameter global,
>> even though only one function needs it? In general, however, I would
>> also avoid this.
>>
>> > <snip>
>> >> I can see that the top-level could just create an object from a class
>> >> which encapsulates everything, but what if I want to keep the salutation
>> >> generation separate, so that I can have a separate program which just
>> >> generates the salutation and print it to the terminal?
>> >
>> > Yes, that's basically the way to go: parse arguments into a structure (an
>> > object) that contains all options/parameters then pass that down. Next
>> level:
>> > some sections of your code may require a certain subset of those
>> options, some
>> > may require some other, so you would structure your options object in
>> > sub-objects for the various sets of correlated options, then rather pass
>> just
>> > the sub-object(s) that are relevant to the section of code you are
>> calling.
>> > Variations are of course possible, anyway that's the basic idea.
>> >
>> > Also have a look at the "argparse" library, it does all the heavy
>> lifting for
>> > the parsing and creation of those objects, definitely advised for in non
>> trivial
>> > cases: <https://docs.python.org/3/library/argparse.html>.
>>
>> I am already using 'argparse' ('configargparse' actually). What aspect
>> should I be looking at in order to produce "sub-objects"?
>>
>> >> I guess I am really asking how to avoid "passing through" arguments to
>> >> functions which only need them to call other functions, so maybe the
>> >> answer is just to avoid nesting.
>> >
>> > No, you don't get rid of code structure just not to pass arguments to
>> > a function... Code may be poorly structured, but that's another
>> > story.
>>
>> As I am writing new code it is more a question of imposing structure,
>> rather than getting rid of structure. Unwritten code for a given
>> purpose obviously has some sort of structure with regards to, say, loops
>> and conditions, but I am less sure about the implications for how the
>> code should be nested. Another argument against deeply-nested functions
>> is the increased complexity of testing.

[snip (15 lines)]>

> Hi,
>
> Also you can give a try to click and / or typer packages.
> Putting args into environment variables can be a solution too
> All of these depends on several things: personal preferences, colleagues /
> firm standards, the program, readability, variable accessibility (IDE
> support, auto completition) (env vars not supported by IDEs as they are not
> part of code)

Thanks for the pointers, although I have only just got my head around
argparse/configargparse, so click is something I might have a look at
for future project.

However, the question of how to parse the arguments is somewhat separate
from that of how to pass (or not pass) the arguments around within a
program.

Cheers,

Loris

--
This signature is currently under construction.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Making command-line args available to deeply-nested functions [ In reply to ]

george at fischhof

Aug 23, 2021, 1:08 PM

Post #3 of 7 (285 views)

Permalink

Loris Bennett <loris.bennett@fu-berlin.de> ezt írta (id?pont: 2021. aug.
23., H 19:26):

> George Fischhof <george@fischhof.hu> writes:
>
> > Loris Bennett <loris.bennett@fu-berlin.de> ezt írta (id?pont: 2021. aug.
> > 20., P 17:54):
> >
> >> Julio Di Egidio <julio@diegidio.name> writes:
> >>
> >> > On Friday, 20 August 2021 at 11:54:00 UTC+2, Loris Bennett wrote:
> >> >> Hi,
> >> >>
> >> >> TL;DR:
> >> >>
> >> >> If I have a command-line argument for a program, what is the best way
> >> >> of making this available to a deeply-nested[1] function call without
> >> >> passing the parameter through every intermediate function?
> >> >
> >> > To not pass arguments you need shared state ("global variables"): and
> >> > options in shard state, unless you have very good reasons to do
> >> > otherwise, is simply a no no.
> >>
> >> Doesn't that slightly depend on the size of your "globe"? If a program
> >> does a few, in some sense unrelated things, and, say, only runs for a
> >> few minutes, could you not decide to make a particular parameter global,
> >> even though only one function needs it? In general, however, I would
> >> also avoid this.
> >>
> >> > <snip>
> >> >> I can see that the top-level could just create an object from a class
> >> >> which encapsulates everything, but what if I want to keep the
> salutation
> >> >> generation separate, so that I can have a separate program which just
> >> >> generates the salutation and print it to the terminal?
> >> >
> >> > Yes, that's basically the way to go: parse arguments into a structure
> (an
> >> > object) that contains all options/parameters then pass that down.
> Next
> >> level:
> >> > some sections of your code may require a certain subset of those
> >> options, some
> >> > may require some other, so you would structure your options object in
> >> > sub-objects for the various sets of correlated options, then rather
> pass
> >> just
> >> > the sub-object(s) that are relevant to the section of code you are
> >> calling.
> >> > Variations are of course possible, anyway that's the basic idea.
> >> >
> >> > Also have a look at the "argparse" library, it does all the heavy
> >> lifting for
> >> > the parsing and creation of those objects, definitely advised for in
> non
> >> trivial
> >> > cases: <https://docs.python.org/3/library/argparse.html>.
> >>
> >> I am already using 'argparse' ('configargparse' actually). What aspect
> >> should I be looking at in order to produce "sub-objects"?
> >>
> >> >> I guess I am really asking how to avoid "passing through" arguments
> to
> >> >> functions which only need them to call other functions, so maybe the
> >> >> answer is just to avoid nesting.
> >> >
> >> > No, you don't get rid of code structure just not to pass arguments to
> >> > a function... Code may be poorly structured, but that's another
> >> > story.
> >>
> >> As I am writing new code it is more a question of imposing structure,
> >> rather than getting rid of structure. Unwritten code for a given
> >> purpose obviously has some sort of structure with regards to, say, loops
> >> and conditions, but I am less sure about the implications for how the
> >> code should be nested. Another argument against deeply-nested functions
> >> is the increased complexity of testing.
>
> [snip (15 lines)]>
>
> > Hi,
> >
> > Also you can give a try to click and / or typer packages.
> > Putting args into environment variables can be a solution too
> > All of these depends on several things: personal preferences, colleagues
> /
> > firm standards, the program, readability, variable accessibility (IDE
> > support, auto completition) (env vars not supported by IDEs as they are
> not
> > part of code)
>
> Thanks for the pointers, although I have only just got my head around
> argparse/configargparse, so click is something I might have a look at
> for future project.
>
> However, the question of how to parse the arguments is somewhat separate
> from that of how to pass (or not pass) the arguments around within a
> program.
>
> Cheers,
>
> Loris
>
> --
> This signature is currently under construction.
> --
> https://mail.python.org/mailman/listinfo/python-list
>

>
>
>

Hi,
I thought not just parsing, but the usage method: you add a decorator to
the function where you want to use the parameters. This way you do not have
to pass the value through the calling hierarchy.

Note: typer is a newer package, it contains click and leverages command
line parsing even more.

BR,
George
--
https://mail.python.org/mailman/listinfo/python-list

Re: Making command-line args available to deeply-nested functions [ In reply to ]

loris.bennett at fu-berlin

Aug 26, 2021, 2:51 AM

Post #4 of 7 (285 views)

Permalink

George Fischhof <george@fischhof.hu> writes:

[snip (79 lines)]

>> > Hi,
>> >
>> > Also you can give a try to click and / or typer packages.
>> > Putting args into environment variables can be a solution too
>> > All of these depends on several things: personal preferences, colleagues
>> /
>> > firm standards, the program, readability, variable accessibility (IDE
>> > support, auto completition) (env vars not supported by IDEs as they are
>> not
>> > part of code)
>>
>> Thanks for the pointers, although I have only just got my head around
>> argparse/configargparse, so click is something I might have a look at
>> for future project.
>>
>> However, the question of how to parse the arguments is somewhat separate
>> from that of how to pass (or not pass) the arguments around within a
>> program.

[snip (16 lines)]
>
> Hi,
> I thought not just parsing, but the usage method: you add a decorator to
> the function where you want to use the parameters. This way you do not have
> to pass the value through the calling hierarchy.
>
> Note: typer is a newer package, it contains click and leverages command
> line parsing even more.

Do you have an example of how this is done? From a cursory reading of
the documentation, it didn't seem obvious to me how to do this, but then
I don't have much understanding of how decorators work.

Cheers,

Loris

--
This signature is currently under construction.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Making command-line args available to deeply-nested functions [ In reply to ]

george at fischhof

Aug 29, 2021, 12:27 PM

Post #5 of 7 (283 views)

Permalink

Loris Bennett <loris.bennett@fu-berlin.de> ezt írta (id?pont: 2021. aug.
26., Cs, 16:02):

> George Fischhof <george@fischhof.hu> writes:
>
> [snip (79 lines)]
>
> >> > Hi,
> >> >
> >> > Also you can give a try to click and / or typer packages.
> >> > Putting args into environment variables can be a solution too
> >> > All of these depends on several things: personal preferences,
> colleagues
> >> /
> >> > firm standards, the program, readability, variable accessibility (IDE
> >> > support, auto completition) (env vars not supported by IDEs as they
> are
> >> not
> >> > part of code)
> >>
> >> Thanks for the pointers, although I have only just got my head around
> >> argparse/configargparse, so click is something I might have a look at
> >> for future project.
> >>
> >> However, the question of how to parse the arguments is somewhat separate
> >> from that of how to pass (or not pass) the arguments around within a
> >> program.
>
> [snip (16 lines)]
> >
> > Hi,
> > I thought not just parsing, but the usage method: you add a decorator to
> > the function where you want to use the parameters. This way you do not
> have
> > to pass the value through the calling hierarchy.
> >
> > Note: typer is a newer package, it contains click and leverages command
> > line parsing even more.
>
> Do you have an example of how this is done? From a cursory reading of
> the documentation, it didn't seem obvious to me how to do this, but then
> I don't have much understanding of how decorators work.
>
> Cheers,
>
> Loris
>
>
> --
> This signature is currently under construction.
> --
> https://mail.python.org/mailman/listinfo/python-list

Hi,

will create a sample code on Monday - Tuesday

BR,
George
--
https://mail.python.org/mailman/listinfo/python-list

Re: Making command-line args available to deeply-nested functions [ In reply to ]

george at fischhof

Aug 31, 2021, 7:35 AM

Post #6 of 7 (283 views)

Permalink

George Fischhof <george@fischhof.hu> ezt írta (id?pont: 2021. aug. 29., V,
21:27):

>
>
> Loris Bennett <loris.bennett@fu-berlin.de> ezt írta (id?pont: 2021. aug.
> 26., Cs, 16:02):
>
>> George Fischhof <george@fischhof.hu> writes:
>>
>> [snip (79 lines)]
>>
>> >> > Hi,
>> >> >
>> >> > Also you can give a try to click and / or typer packages.
>> >> > Putting args into environment variables can be a solution too
>> >> > All of these depends on several things: personal preferences,
>> colleagues
>> >> /
>> >> > firm standards, the program, readability, variable accessibility (IDE
>> >> > support, auto completition) (env vars not supported by IDEs as they
>> are
>> >> not
>> >> > part of code)
>> >>
>> >> Thanks for the pointers, although I have only just got my head around
>> >> argparse/configargparse, so click is something I might have a look at
>> >> for future project.
>> >>
>> >> However, the question of how to parse the arguments is somewhat
>> separate
>> >> from that of how to pass (or not pass) the arguments around within a
>> >> program.
>>
>> [snip (16 lines)]
>> >
>> > Hi,
>> > I thought not just parsing, but the usage method: you add a decorator to
>> > the function where you want to use the parameters. This way you do not
>> have
>> > to pass the value through the calling hierarchy.
>> >
>> > Note: typer is a newer package, it contains click and leverages command
>> > line parsing even more.
>>
>> Do you have an example of how this is done? From a cursory reading of
>> the documentation, it didn't seem obvious to me how to do this, but then
>> I don't have much understanding of how decorators work.
>>
>> Cheers,
>>
>> Loris
>>
>>
>> --
>> This signature is currently under construction.
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>
>
> Hi,
>
> will create a sample code on Monday - Tuesday
>
> BR,
> George
>

Hi,

here is the program ;-) (see below)
typer does not uses decorators, to solve this problem they advice to use
click's decorators, mixing typer and click.
Practically I prefer not to mix them, also the parts for easiest way to do
this just available in latest click, which is not supported in typer.

So I created all the stuff in click, 8.x should be used

BR,
George

import click

# read command line parameters
@click.command()
@click.option('--level_1', help='Level 1')
@click.option('--level_2', help='Level 2')
def main(level_1, level_2):
# put command line parameters into global context
ctx = click.get_current_context()
ctx.meta['level_1'] = level_1
ctx.meta['level_2'] = level_2

level_1_function()

# pass / inject level_1 parameter to this function
@click.decorators.pass_meta_key('level_1')
def level_1_function(level_1):
print(f'level 1 variable: {level_1}')
level_2_function()

# pass / inject level_2 parameter to this function
@click.decorators.pass_meta_key('level_2')
def level_2_function(level_2):
print(f'level 2 variable: {level_2}')

if __name__ == "__main__":
main()
--
https://mail.python.org/mailman/listinfo/python-list

Re: Making command-line args available to deeply-nested functions [ In reply to ]

loris.bennett at fu-berlin

Sep 10, 2021, 4:24 AM

Post #7 of 7 (280 views)

Permalink

George Fischhof <george@fischhof.hu> writes:

> George Fischhof <george@fischhof.hu> ezt írta (id?pont: 2021. aug. 29., V,
> 21:27):
>
>>
>>
>> Loris Bennett <loris.bennett@fu-berlin.de> ezt írta (id?pont: 2021. aug.
>> 26., Cs, 16:02):
>>
>>> George Fischhof <george@fischhof.hu> writes:
>>>
>>> [snip (79 lines)]
>>>
>>> >> > Hi,
>>> >> >
>>> >> > Also you can give a try to click and / or typer packages.
>>> >> > Putting args into environment variables can be a solution too
>>> >> > All of these depends on several things: personal preferences,
>>> colleagues
>>> >> /
>>> >> > firm standards, the program, readability, variable accessibility (IDE
>>> >> > support, auto completition) (env vars not supported by IDEs as they
>>> are
>>> >> not
>>> >> > part of code)
>>> >>
>>> >> Thanks for the pointers, although I have only just got my head around
>>> >> argparse/configargparse, so click is something I might have a look at
>>> >> for future project.
>>> >>
>>> >> However, the question of how to parse the arguments is somewhat
>>> separate
>>> >> from that of how to pass (or not pass) the arguments around within a
>>> >> program.
>>>
>>> [snip (16 lines)]
>>> >
>>> > Hi,
>>> > I thought not just parsing, but the usage method: you add a decorator to
>>> > the function where you want to use the parameters. This way you do not
>>> have
>>> > to pass the value through the calling hierarchy.
>>> >
>>> > Note: typer is a newer package, it contains click and leverages command
>>> > line parsing even more.
>>>
>>> Do you have an example of how this is done? From a cursory reading of
>>> the documentation, it didn't seem obvious to me how to do this, but then
>>> I don't have much understanding of how decorators work.
>>>
>>> Cheers,
>>>
>>> Loris
>>>
>>>
>>> --
>>> This signature is currently under construction.
>>> --
>>> https://mail.python.org/mailman/listinfo/python-list
>>
>>
>> Hi,
>>
>> will create a sample code on Monday - Tuesday
>>
>> BR,
>> George
>>
>
>
> Hi,
>
> here is the program ;-) (see below)
> typer does not uses decorators, to solve this problem they advice to use
> click's decorators, mixing typer and click.
> Practically I prefer not to mix them, also the parts for easiest way to do
> this just available in latest click, which is not supported in typer.
>
> So I created all the stuff in click, 8.x should be used
>
> BR,
> George
>
>
> import click
>
>
> # read command line parameters
> @click.command()
> @click.option('--level_1', help='Level 1')
> @click.option('--level_2', help='Level 2')
> def main(level_1, level_2):
> # put command line parameters into global context
> ctx = click.get_current_context()
> ctx.meta['level_1'] = level_1
> ctx.meta['level_2'] = level_2
>
> level_1_function()
>
>
> # pass / inject level_1 parameter to this function
> @click.decorators.pass_meta_key('level_1')
> def level_1_function(level_1):
> print(f'level 1 variable: {level_1}')
> level_2_function()
>
>
> # pass / inject level_2 parameter to this function
> @click.decorators.pass_meta_key('level_2')
> def level_2_function(level_2):
> print(f'level 2 variable: {level_2}')
>
>
> if __name__ == "__main__":
> main()

Thanks for the example - that's very interesting. However, after a bit
of reflection I think I am going to stick to explicit argument passing,
so that I can have more generic modules that can be used by other
programs. I'll then just encapsulate the argument parsing in a single
function corresponding to the command line tool.

Cheers,

Loris

--
This signature is currently under construction.
--
https://mail.python.org/mailman/listinfo/python-list