Thank you for carefully considering suggestions (and implications) - and
which will 'work' for you.
Further comment below (and with apologies that, unusually for me, there
are many personal opinions mixed-in):-
On 06/04/2023 01.06, Loris Bennett wrote:
> "Loris Bennett" <loris.bennett@fu-berlin.de> writes:
>> dn <PythonList@DancesWithMice.info> writes:
>>> On 01/04/2023 02.01, Loris Bennett wrote:
>>>> Hi,
>>>> In my top level program file, main.py, I have
>>>> def main_function():
>>>> parser = argparse.ArgumentParser(description="my prog")
>>>> ...
>>>> args = parser.parse_args()
>>>> config = configparser.ConfigParser()
>>>> if args.config_file is None:
>>>> config_file = DEFAULT_CONFIG_FILE
>>>> else:
>>>> config_file = args.config_file
>>>> config.read(config_file)
>>>> logging.config.fileConfig(fname=config_file)
>>>> logger = logging.getLogger(__name__)
>>>> do_some_stuff()
>>>> my_class_instance = myprog.MyClass()
>>>> def do_some_stuff():
>>>> logger.info("Doing stuff")
>>>> This does not work, because 'logger' is not known in the function
>>>> 'do_some_stuff'.
>>>> However, if in 'my_prog/my_class.py' I have
>>>> class MyClass:
>>>> def __init__(self):
>>>> logger.debug("created instance of MyClass")
>>>> this 'just works'.
>>>> I can add
>>>> logger = logging.getLogger(__name__)
>>>> to 'do_some_stuff', but why is this necessary in this case but not
>>>> in
>>>> the class?
>>>> Or should I be doing this entirely differently?
>>>
>>> Yes: differently.
>>>
>>> To complement @Peter's response, two items for consideration:
>>>
>>> 1 once main_function() has completed, have it return logger and other
>>> such values/constructs. The target-identifiers on the LHS of the
>>> function-call will thus be within the global scope.
>>>
>>> 2 if the purposes of main_function() are condensed-down to a few
>>> (English, or ..., language) phrases, the word "and" will feature, eg
>>> - configure env according to cmdLN args,
>>> - establish log(s),
>>> - do_some_stuff(), ** AND **
>>> - instantiate MyClass.
>>>
>>> If these (and do_some_stuff(), like MyClass' methods) were split into
>>> separate functions* might you find it easier to see them as separate
>>> sub-solutions? Each sub-solution would be able to contribute to the
>>> whole - the earlier ones as creating (outputting) a description,
>>> constraint, or basis; which becomes input to a later function/method.
>>
>> So if I want to modify the logging via the command line I might have the
>> following:
>>
>> ---------------------------------------------------------------------
>>
>> #!/usr/bin/env python3
>>
>> import argparse
>> import logging
>>
>>
>> def get_logger(log_level):
>> """Get global logger"""
>>
>> logger = logging.getLogger('example')
>> logger.setLevel(log_level)
>> ch = logging.StreamHandler()
>> formatter = logging.Formatter('%(levelname)s - %(message)s')
>> ch.setFormatter(formatter)
>> logger.addHandler(ch)
>>
>> return logger
>>
>>
>> def do_stuff():
>> """Do some stuff"""
>>
>> # logger.info("Doing stuff!")
>
> Looks like I just need
>
> logger = logging.getLogger('example)
> logger.info("Doing stuff!")
>
>>
>> def main():
>> """Main"""
>>
>> parser = argparse.ArgumentParser()
>> parser.add_argument("--log-level", dest="log_level", type=int)
>> args = parser.parse_args()
>>
>> print(f"log level: {args.log_level}")
>>
>> logger = get_logger(args.log_level)
>> logger.debug("Logger!")
>> do_stuff()
>>
>>
>> if __name__ == "__main__":
>> main()
>>
>> ---------------------------------------------------------------------
>>
>> How can I get logging for 'do_stuff' in this case without explicitly
>> passing 'logger' as an argument or using 'global'?
>>
>> Somehow I am failing to understand how to get 'logger' defined
>> sufficiently high up in the program that all references 'lower down' in
>> the program will be automatically resolved.
At the risk of 'heresy', IMHO the idea of main() is (almost always)
unnecessary in Python, and largely a habit carried-over from other
languages (need for an entry-/end-point).
NB be sure of the difference between a "script" and a "module"...
My script template-overview:
''' Script docstring. '''
- author, license, etc docs
global constants such as import-s
set environment
if __name__ == "__main__":
do_this()
do_that()
ie the business of the script
Despite its frequent use, I'm somewhat amused by the apparent
duplication within:
if __name__ == "__main__":
main()
ie if this .py file is being executed as a script, call main() - where
main() is the purpose of the script. Whereas if the file is an import-ed
module, do not execute any of the contained-code.
Thus, the if-statement achieves the same separation as the main()
function encapsulation. Also, the word main() conveys considerably less
meaning than (say) establish_logging().
NB others may care to offer alternate advice for your consideration...
There is a good argument for main(), in that it might enable tests to be
written which assure the integration of several tasks called by the
script. Plus, if one excludes non-TDD test-able code, such as user
interfaces, even 'fancy headings' and printing of conclusions; there's
little left. Accordingly, I don't mind the apparent duplication involved
in coding an integration test which checks that do_this() and do_that()
are playing-nicely together. YMMV!
I'm not sure why, but my 'mainlines' never seem to be very long. Once I
had been introduced to "Modular Programming" (1970s?), it seemed that
the purpose of a mainline was merely calling one do-it function after
another.
To me mainline's seem highly unlikely to be re-usable. Thus, the
possibility that main() might need to be import-able to some other
script, is beyond my experience/recollection.
Further, my .py file scripts/mainlines tend to be short and often don't
contain any of the functions or classes which actually do-the-work - all
are import-ed (and thus, they are easier to re-use!?)
Returning to 'set environment': the code-examples include argparse and
logger. Both (in my thinking) are part of creating the environment in
which the code will execute.
Logging for example, is the only choice when we need to be aware of how
a web-based system is running (or running into problems) - otherwise, as
some would argue, we might as well use print(). Similarly, argparse will
often influence the way a script executes, the data it should use, etc.
Much of such forms the (global) environment in which (this run of) the
code will execute. Hence locating such setting of priorities and
constraints, adjacent to the import statements.
These must be established before getting on with 'the business'. That
said, if someone prefers to put it under if __main__, I won't burst into
tears. (see earlier comment about the .py file as a script cf module)
You have answered your own question about logging. Well done!
The logging instance can either be explicitly passed into each
(relevant) function, or it can be treated as a global and thus available
implicitly. (see also: "Zen of Python")
I prefer your approach of get_logger() - even if that name doesn't quite
describe the establishment of a logger and its settings. All of that
part of setting the environment is collected into one place. Which
in-turn, makes it easy to work on any changes, or work-in any parameters
which may come from other environment-setting activity.
As well as 'the documentation' there is a HowTo
(
https://docs.python.org/3/howto/logging.html). Other logging-learners
have found the DigitalOcean tutorial helpful
(
https://www.digitalocean.com/community/tutorials/how-to-use-logging-in-python-3)
>>> * there is some debate amongst developers about whether "one function,
>>> one purpose" should be a rule, a convention, or tossed in the
>>> trash. YMMV!
>>>
>>> Personal view: SOLID's "Single" principle applies: there should be
>>> only one reason (hanging over the head of each method/function, like
>>> the Sword of Damocles) for it to change - or one 'user' who could
>>> demand a change to that function. In other words, an updated cmdLN
>>> option shouldn't affect a function which establishes logging, for
>>> example.
>>>
>>>
>>> Web.Refs:
>>> https://people.engr.tamu.edu/choe/choe/courses/20fall/315/lectures/slide23-solid.pdf
>>> https://www.hanselminutes.com/145/solid-principles-with-uncle-bob-robert-c-martin
>>> https://idioms.thefreedictionary.com/sword+of+Damocles
>>> https://en.wikipedia.org/wiki/Damocles
>>
>> I don't really get the "one reason" idea and the Sword of Damocles
>> analogy. The later to me is more like "there's always a downside",
>> since the perks of being king may mean someone might try to usurp the
>> throne and kill you. Where is the "single principle" aspect?
>>
>> However, the idea of "one responsibility" in the sense of "do only one
>> thing" seems relatively clear, especially if I think in terms of writing
>> unit tests.
+1
Users are notoriously unable to write clear specifications. Ultimately,
and not necessarily unkindly, they often do not know what they want -
and particularly not specified at the level of detail we need. This is
the downfall of the "waterfall" SDLC development model, and the reason
why short 'Agile' sprints are (often/in-theory) more successful. The
sooner a mistake, misunderstanding, or omission is realised, the better!
Not wanting to do more than state the above, things change, life goes
on, etc, etc. So, when one first demonstrates code to a user, it is rare
that they won't want to change something. Similarly, over time, it is
highly unlikely that someone won't dream up some 'improvement' - or
similar need to amend the code be imposed by an externality, eg
government legislation or regulation. Such changes may be trivial, eg
cosmetic; others may be more far-reaching, eg imposition of a tax where
previously there was none (or v-v).
Accordingly, adding the fourth dimension to one's code, and
program[me]-design - and the advice about being kind to those who will
maintain the code after you or your six-months-time-self.
So, the 'sword of Damocles' is knowing that our code should not be
considered secure (or static). That change is inevitable. We don't need
to be worrying about danger afflicting us when we least expect it - your
users are unlikely to actually kill you. However, it is a good idea to
be aware that change may be required, and that it could come from any
random direction or "stakeholder".
Perhaps worse, change implies risk. We make a change to suit one aspect,
and something else 'breaks'. That is the reason many programmers
actively resist 'change'. That is also one of the promises of TDD - if
we can quickly run tests which assure (if not "prove") code-correctness,
then the risks of change decrease. Whereas, if there are no tests to
ensure 'life goes on' after a change, well, unhappiness reigns!
Hence the allusion.
(apologies if it was too oblique)
Accordingly, the idea that if a function does one job (and does it
well), should you decide/be required to change that function, then the
impacts, any side-effects, etc, of such will only affect that one
function (and its tests), and whatever is 'downstream' (integrating)
from there.
Which also reduces the chance that some change 'over here', will have
some unforeseen impact 'over there'. The structure and the flow work
together to achieve both a degree of separation and a bringing-together
or unity. The tension of team-work if you like, cf the dysfunction of
disorganisation.
NB not that a 'small change' means running only that one test in TDD -
still run the entire suite of tests (auto-magically) to ensure that
there are no surprises...
Change is inevitable. Thus, consider that your throne as code-author is
illusory - the user/client is able/likely to want change.
Then, if the code is organised according to functionality, the logging
environment (for example) can be changed without any need to re-code
somewhere else - and/or that tinkering with cmdLN arguments and such
code-changes can be done quite separately from the establishment of logging.
Ultimately, the smaller the function (think of establishing logging),
the more likely it will be able to be re-used in the next assignment
which requires a log! (whereas if it is mixed-in with argparse, re-use
is unlikely because the cmdLN args will be different)
There's plenty of references about such on the web. Will be happy to
discuss whichever sub-topics might be of-interest, further...
--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list