Mailing List Archive

Mark Hammond wrote:
>
> A little thought I have had a few times now, so it is time to pass it
> on for comment.
>
> Basically _every_ time I use getopt, I write code like this:
>
> import getopt
> try:
> opts, args = getopt.getopt(sys.argv[1:], "slc:d:e:")
> except getopt.error, why:
> print why
> print usage % (os.path.basename(sys.argv[0],))
> sys.exit(1)
>
> Every single time. I have never used getopt without this code.
>
> How about we put this functionality into getopt itself?
>
> It could be triggered either by adding a new "usage" param defaulting
> to None, or by adding a new entry point. ie:
>
> opts, args = getopt.getopt(sys.argv[1:], "slc:d:e:", usage=usage)
> or
> opts, args = getopt.getoptex(sys.argv[1:], "slc:d:e:", usage=usage)
>
> I know it is fairly trivial, but IMO is such a useful module and the
> pattern is used so regularly that is seems to make sense to add it.
>
> Any thoughts? If it is seen favourably, how should we spell it?

Why not just add a higher level interface ? Something
like CommandLine.py which is included in mxDateTime ?

--
Marc-Andre Lemburg
______________________________________________________________________
Y2000: 92 days left
Business: http://www.lemburg.com/
Python Pages: http://www.lemburg.com/python/

> > Basically _every_ time I use getopt, I write code like this:

> Why not just add a higher level interface ? Something
> like CommandLine.py which is included in mxDateTime ?

Because _every_ time I use getopt, I write code like that :-)

A higher level interface would maybe be handy, but it simply occurred
to me that every time I used the module I used that pattern. I just
got sick of typing it all the time and wondered if it struck a chord
with anyone else (and I dont have or use a general purpose "mhutil"
module :-) Im really just trying to save myself 10 lines of
boilerplate coding, not introduce a new standard module :-)

Mark.

Mark Hammond wrote:
>
> > > Basically _every_ time I use getopt, I write code like this:
>
> > Why not just add a higher level interface ? Something
> > like CommandLine.py which is included in mxDateTime ?
>
> Because _every_ time I use getopt, I write code like that :-)
>
> A higher level interface would maybe be handy, but it simply occurred
> to me that every time I used the module I used that pattern. I just
> got sick of typing it all the time and wondered if it struck a chord
> with anyone else (and I dont have or use a general purpose "mhutil"
> module :-) Im really just trying to save myself 10 lines of
> boilerplate coding, not introduce a new standard module :-)

Just a thought :-)

I wrote the CommandLine.py for pretty much the same reason: I
have quite a few command line apps lying in my bin/ dir and
they all did some kind of getopt/sys.argv tricks to handle
the input... way to confusing and not easy to maintain. So I
decided to take an OO-approach to have them use the same
interface with nice help output and to reduce the coding effort.

As an example taken from mxDateTime:

#!/usr/local/bin/python -u

""" Simple Forking Alarm

Sample Application for DateTime types and CommandLine. Only works
on OSes which support os.fork().

Author: Marc-Andre Lemburg, mailto:mal@lemburg.com
"""
import time,sys,os
from mx.DateTime import *
from CommandLine import Application,ArgumentOption

class Alarm(Application):

header = "Simple Forking Alarm"
options = [.ArgumentOption('-s',
'set the alarm to now + arg seconds'),
ArgumentOption('-m',
'set the alarm to now + arg minutes'),
ArgumentOption('-a',
'set the alarm to ring at arg (hh:mm)'),
]
version = '0.1'

def main(self):

atime = now() + (self.values['-s'] or
self.values['-m'] * 60 or
self.values['-h'] * 3600) * oneSecond
abs = self.values['-a']
if abs:
atime = strptime(abs,'%H:%M',today(second=0))
if atime < now():
print 'Alarm time has expired...'
return
print 'Alarm will ring at',atime
if not os.fork():
time.sleep((atime - now()).seconds)
alarm()
os._exit(0)

def alarm():

""" Ring alarm
"""
for i in range(10):
sys.stdout.write('\007')
sys.stdout.flush()
time.sleep(0.2)

if __name__ == '__main__':
Alarm()

Here's the help output this produces:

/home/lemburg> alarm -h
------------------------------------------------------------------------
Simple Forking Alarm
------------------------------------------------------------------------

Synopsis:
alarm [option] files...

Options and default settings:
-s arg set the alarm to now + arg seconds
-m arg set the alarm to now + arg minutes
-a arg set the alarm to ring at arg (hh:mm)
-h show this help text
--help show this help text
--copyright show copyright
--examples show examples of usage

Version:
0.1

--
Marc-Andre Lemburg
______________________________________________________________________
Y2000: 92 days left
Business: http://www.lemburg.com/
Python Pages: http://www.lemburg.com/python/

[Mark]
> > > > Basically _every_ time I use getopt, I write code like this:

[Marc-Andre]
> > > Why not just add a higher level interface ? Something
> > > like CommandLine.py which is included in mxDateTime ?

[Mark]
> > Because _every_ time I use getopt, I write code like that :-)

[Marc-Andre]
> I wrote the CommandLine.py for pretty much the same reason:

Marc-Andre, you're not hearing what Mark is saying. He wants a change
to the standard library, and he knows that small additions to existing
modules there stand a better chance of adoption than new modules.

I personally liked the idea of getoptex() best, except I would call it
getopt_or_die(). If the usage message is omitted it can synthesize
one from the (short and long) options arguments and sys.argv[0] (the
latter being a bit controversial, but it's just a default).

Hmm... Perhaps getopt_or_die() shouldn't take the args argument but
extract sys.argv[1:] itself?

--Guido van Rossum (home page: http://www.python.org/~guido/)

On 30 September 1999, Guido van Rossum said:
> Marc-Andre, you're not hearing what Mark is saying. He wants a change
> to the standard library, and he knows that small additions to existing
> modules there stand a better chance of adoption than new modules.
>
> I personally liked the idea of getoptex() best, except I would call it
> getopt_or_die().

Gasp! Guido hath drunk from the poisoned well of the Great Camel and it
doth infest his thinking! "or die" indeed -- not an idiom I've seen
since the last time I wrote some Perl code (umm, last week?).

> If the usage message is omitted it can synthesize
> one from the (short and long) options arguments and sys.argv[0] (the
> latter being a bit controversial, but it's just a default).
>
> Hmm... Perhaps getopt_or_die() shouldn't take the args argument but
> extract sys.argv[1:] itself?

Whatever we call it, it should be able to take an explicit argument
list, and only default to sys.argv[1:]. What if I want to parse options
from an environment variable or a config file? I also like the "don't
clobber sys.argv, but return the modified version instead" model -- it's
nice to keep a pristine copy of the original argument list!

Another problem with getopt is that it doesn't correlate long and short
options. I wrote distutils.fancy_getopt (download your copy today!
hurry, don't delay -- at this price, it WON'T LAST LONG!) to address
this, and to maybe someday do something with help text.

On the other hand, don't listen to me -- I tend to write mammoth,
bloated, all-singing, all-dancing command-line parsing modules for every
new language I encounter. They get more insane with each iteration. I
have yet to top my Getopt::Tabular for Perl, though; see

http://search.cpan.org/doc/GWARD/Getopt-Tabular-0.3/Tabular.pod

if you've ever wondered how far this sort of thing can be taken in a
high-level, dynamically typed language.

Greg
--
Greg Ward - software developer gward@cnri.reston.va.us
Corporation for National Research Initiatives
1895 Preston White Drive voice: +1-703-620-8990
Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913

> I personally liked the idea of getoptex() best, except I would call it
> getopt_or_die(). If the usage message is omitted it can synthesize
> one from the (short and long) options arguments and sys.argv[0] (the
> latter being a bit controversial, but it's just a default).
>
> Hmm... Perhaps getopt_or_die() shouldn't take the args argument but
> extract sys.argv[1:] itself?

sounds reasonable. what are you waiting for ;-)

</F>

Guido van Rossum wrote:
>
> [Mark]
> > > > > Basically _every_ time I use getopt, I write code like this:
>
> [Marc-Andre]
> > > > Why not just add a higher level interface ? Something
> > > > like CommandLine.py which is included in mxDateTime ?
>
> [Mark]
> > > Because _every_ time I use getopt, I write code like that :-)
>
> [Marc-Andre]
> > I wrote the CommandLine.py for pretty much the same reason:
>
> Marc-Andre, you're not hearing what Mark is saying. He wants a change
> to the standard library, and he knows that small additions to existing
> modules there stand a better chance of adoption than new modules.

Oh, I did get the idea... just wanted to plug my module here in a
take-it-or-leave-it way ;-) I usually put such things into
my lib/ dir for Python to find -- no need to make them a standard.

> I personally liked the idea of getoptex() best, except I would call it
> getopt_or_die(). If the usage message is omitted it can synthesize
> one from the (short and long) options arguments and sys.argv[0] (the
> latter being a bit controversial, but it's just a default).
>
> Hmm... Perhaps getopt_or_die() shouldn't take the args argument but
> extract sys.argv[1:] itself?

Better not: it's sometimes very useful to call the main(args) function
of a script in interactive mode which then passes the args list to
getopt(). How about adding something like:

def getoptex(...,args=None,helptext='Read the source, Luke ;-)'):
if args is None: args = sys.argv[1:]
...

--
Marc-Andre Lemburg
______________________________________________________________________
Y2000: 92 days left
Business: http://www.lemburg.com/
Python Pages: http://www.lemburg.com/python/

I can sympathize with Mark, I have nearly the same code in every
script I write. I once wrote a nice (IMO) class for doing all the
common things I do with c.l. args, but I can't seem to dig it up at
the moment.

The idea was basically to have a base class that had all the
machinery, while derived classes included specially named methods that
did the app-specific option handling. It knew whether the method took
1 or zero arguments (not including self), glommed up the shortarg
string and longarg list (with appropriate `:' and `=' thrown in), then
parsed the args, dispatching to the handlers, e.g.:

class MyOptions(getopt.Options):
def handle_a(self):
self.alpha = 1
handle_alpha = handle_a

def handle_b(self, arg):
self.beta = arg
handle_beta = handle_b

def handle_i(self, arg):
try:
self.integer = int(arg)
except ValueError:
self.usage()
handle_integer = handle_i


and could be used like so:

#! /usr/bin/env python
#
# ...

opts = MyOptions(sys.argv[1:], usage=__doc__ % globals())
if opts.alpha:
do_my_alpha_thang()
if opts.integer = 1:
do_something_else()
for file in opts.leftoverargs:
process_file(file)

While I liked this a lot, I seem to remember showing it to Guido,
receiving the expected scoffing. So now, I just cut and paste the opt
parsing stuff into every script I write :) I think it might be nice to
add such a base class to the standard getopt module.

-Barry

OK - general agreement. Wheee :-)

How about this:

def getopt_or_die(opts,
long_opts=[],
usage=None,
prog_name=None,
args=None,
exit_code = 1):

if usage is None, we build a very simple usage string from
opts/long_opts.

If prog_name is None, we use os.path.basename(sys.argv[0]). This is
printed just before the usage message.

If args is None, we use sys.argv[1:]

exit_code specifies the param to sys.exit() on invalid options.

I have based the ordering on my guess at the most likely to be used -
but maybe "prog_name" and "args" should be reversed? This should
_not_ be all-singing, all-dancing, as it is simple to use the existing
getopt.getopt() directly for more esoteric requirements; the old 80-20
rule applies here :-)

If the general agreement continues, I will then knock together an
implementation.

Mark.

On 01 October 1999, Mark Hammond said:
> OK - general agreement. Wheee :-)
>
> How about this:
>
> def getopt_or_die(opts,
> long_opts=[],
> usage=None,
> prog_name=None,
> args=None,
> exit_code = 1):
>
> if usage is None, we build a very simple usage string from
> opts/long_opts.

I still think it would be very desirable to tie the short and long
options together. Eg.

options = [('verbose', 'v'),
('quiet', 'q'),
('thingy', None),
(None, 'x')
('output=', 'o:')]
opts, args = getopt_or_die (options, usage, ...)

Then opts would have possible keys 'verbose', 'quiet', 'thingy', 'x',
and 'value' -- never 'v', 'q', or 'o'

(Look, I restrained my tendency to invent type systems and auto-generate
help text. There may be hope for me yet.)

Greg

Greg writes:
> I still think it would be very desirable to tie the short and long
> options together. Eg.
>
> options = [.('verbose', 'v'),
> ('quiet', 'q'),
> ('thingy', None),
> (None, 'x')
> ('output=', 'o:')]
> opts, args = getopt_or_die (options, usage, ...)
>

Im not convinced this is worth it. I only use "long options" when I
have too many, or a few obscure ones. I have never have "-v"
synonymous for "--verbose" - why bother? I know I would never type
the later:-)

The existing mechanism still handles this quite well - the standard
"if opt==blah:" simply becomes "if opt in [...]:" - no real drag.

Plus its less change for reasonable reward - handy enough I may
actually add command-line handling as I create each little test/util
script :-)

What say anyone else? Go with my "little change", Gregs "only
slightly more change" or "don't worry about it"?

Mark.

Mark> Im not convinced this [pairing up long and short options] is worth
Mark> it.

The primary reason where it would be helpful is to generate clearer default
usage strings. Aside from that, I'm not sure the extra structure would be
worth it.

Skip