Mailing List Archive

> I'd like to propose a third alternative. How about if the string
> interpolation function accepted a tuple of dictionaries directly:
>
> s = format % (d1, d2)
>
> It would only be used when named interpolation was expected. I don't think
> there would be any conflict with current % operator semantics.

Gut feeling: it's dangerous to fill up every possible dark corner with
specific semantics that are occasionally useful, because if you go too
far you lose useful redundancy, and you end up with Perl.

Not sure whether this particular suggestion is "going too far."

I think it depends on to what extent this is a common, useful idiom.
Do you have evidence of that? Examples?

--Guido van Rossum (home page: http://www.python.org/~guido/)

>> I'd like to propose a third alternative. How about if the string
>> interpolation function accepted a tuple of dictionaries directly:
>>
>> s = format % (d1, d2)

Guido> Gut feeling: it's dangerous to fill up every possible dark corner
Guido> with specific semantics that are occasionally useful, because if
Guido> you go too far you lose useful redundancy, and you end up with
Guido> Perl.

Yeah, I am kind of taking advantage of the fact that the format operator
doesn't happen to use tuples of dicts already, though this seems like a
natural extension of the current semantics. Currently, you can have

Style of Simple form Complex form
Interpolation
------- ----------- ------------
sprintf string tuple of strings
named dict tuple of dicts

It does complicate the decision making process in the string format routine
a bit.

Guido> I think it depends on to what extent this is a common, useful
Guido> idiom. Do you have evidence of that? Examples?

Well, the first place I ran into it was in DocumentTemplates a few years
ago. They used an idiom heavily which may have now been replaced by
acquisition where you'd effectively push and pop value dicts onto a stack as
you entered and exited nested blocks of DTML code. Their solution was a
special dict-like object.

The example that made the light click for me this evening was having an
object whose class dict stores constants and whose instant dict stores
varying values. It seemed cleaner to me to do

obj = Class()
...
s = format % (Class.__dict__, obj.__dict__)

than go through the intermediate step of building a separate dict which
would just get discarded as soon as this bit of string building was
complete. (I will perform this once for each of several thousand
instances.)

It's not a big deal. If it seems too obscure the other obvious solutions
are not gruesome.

Skip

Oh crap... Of course, the table should have been

Style of Simple form Complex form
Interpolation
------- ----------- ------------
sprintf string tuple of strings
named dict

(empty fourth cell...)

Skip

On Tue, 25 Jan 2000, Skip Montanaro wrote:

> Guido> Skip:
> >> I'd like to propose a third alternative. How about if the string
> >> interpolation function accepted a tuple of dictionaries directly:
> >>
> >> s = format % (d1, d2)
> [...]
> Guido> I think it depends on to what extent this is a common, useful
> Guido> idiom. Do you have evidence of that? Examples?
>
> Well, the first place I ran into it was in DocumentTemplates a few years
> ago. They used an idiom heavily which may have now been replaced by
> acquisition where you'd effectively push and pop value dicts onto a stack as
> you entered and exited nested blocks of DTML code. Their solution was a
> special dict-like object.

Implementation of acquisition basically uses a MultiDict underneath.
Consider acquisition as a cumulative context composed from the containers
of the target of a web request. (Actually, a distinction is made between
the object containment hierarchy of the target and the successive
components of the path along which the target is reached by the request,
with the containment contexts taking precedence - but that's probably not
important here:-) Add in incidental things like the server environment
(from which you can get HTTP_REFERER and cookies and so forth). Each of
the components can be a dictionary or a MultiDict (or a sequence of pairs,
i think), and they're ultimately composed in a MultiDict.

I think another place in zope where multidicts play prominently is in the
security mechanism, where any object can have local roles, and the
ultimate role of a user within a context is composed from the union across
the containment hierarchy. There probably are lots of other places where
multidicts are used.

Suffice to say that there's a lot of composing of contexts that goes on in
Zope in general, acquistion being a prime example but not the only one,
and multidicts play heavily in many. I would be surprised if this need to
combine contexts is peculiar to web server, or general server
applications.

> [...]
> It's not a big deal. If it seems too obscure the other obvious solutions
> are not gruesome.

I suppose we'd be pretty happy to have something like MultiDict as part of
python...

Ken
klm@digicool.com

(Who's the only one left in fredericksburg to speak up, at the moment:-)

[.Skip, wants to interpolate multiple dicts via "%",
suggests passing a tuple of dicts: format % (d1, d2, ...)]

[Guido]
> ...
> I think it depends on to what extent this is a common, useful
> idiom. Do you have evidence of that? Examples?

You yourself raised one last century <wink>: simply wanting to interpolate
from both locals() and globals(). At the time, the idea of a new dict-like
mapping object (capturing Python's lookup rules) appealed to you. I still
like that, and note that the apparent need becomes more acute if "deep
nesting" is ever added.

I wasn't aware of the MultiDict approach Skip mentioned, but thought it
looked spot on for the general case! Skip, is the long-windedness of

dict = MultiDict()
dict.append(d1)
dict.append(d2)
...
s = format % dict

the part you didn't like about that? If so, how about changing the
constructor to

def __init__(self, *dicts):
...

instead so you could use it as a one-liner

format % MultiDict(d1, d2, ...)

? That's exactly the same as the tuple idea, except there's a nice
descriptive word in the middle of it <wink>.

> the part you didn't like about that? If so, how about changing the
> constructor to
>
> def __init__(self, *dicts):
> ...
>
> instead so you could use it as a one-liner
>
> format % MultiDict(d1, d2, ...)
>
> ? That's exactly the same as the tuple idea, except there's a nice
> descriptive word in the middle of it <wink>.

I've always wonderer why dict+dict isn't supported (or possibly dict|dict, if
the key-collision semantics of + on dict are seen as a problem). Is there a
good reason for this, or is it just that there are other more important things
to implement?

This wouldn't be a replacement for all uses of MultiDict, as it would probably
have to create a new dict to keep semantics in line with those of list+list
--
Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++
Jack.Jansen@oratrix.com | ++++ if you agree copy these lines to your sig ++++
www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm

[Tim]
> > format % MultiDict(d1, d2, ...)
> >
> > ? That's exactly the same as the tuple idea, except there's a nice
> > descriptive word in the middle of it <wink>.

Nice.

[Jack]
> I've always wonderer why dict+dict isn't supported (or possibly
> dict|dict, if the key-collision semantics of + on dict are seen as a
> problem). Is there a good reason for this, or is it just that there
> are other more important things to implement?

The reason is that + (or |) looks symmetrical, but for the key
collisions, one of them has to lose. We now have dict1.update(dict2),
which is a bit more cumbersome, but makes it much clearer who is the
loser.

--Guido van Rossum (home page: http://www.python.org/~guido/)

Tim> Skip, is the long-windedness of

Tim> dict = MultiDict()
Tim> dict.append(d1)
Tim> dict.append(d2)
Tim> ...
Tim> s = format % dict

Tim> the part you didn't like about that? If so, how about changing the
Tim> constructor to

Tim> def __init__(self, *dicts):
Tim> ...

Tim> instead so you could use it as a one-liner

Tim> format % MultiDict(d1, d2, ...)

Tim> ? That's exactly the same as the tuple idea, except there's a nice
Tim> descriptive word in the middle of it <wink>.

The long-windedness was part of it. The performance hit of composing
dictionaries thousands of times to perform a single format operation was
also a consideration.

Okay, side excursion into the Zope source tree...

What I was calling MultiDict is actually MultiMapping (written in C, BTW).
As a side effect of my Zope install here, I even already have it in sys.path
(go figure!). And it turns out to work just as Tim surmised:

>>> d1 = {"a": 1}
>>> d2 = {"b": 2}
>>> d = MultiMapping.MultiMapping(d1, d2)
>>> d["b"]
2
>>> d["a"]
1

Dang! Turns out Jim Fulton has a time machine also.

I guess the next question is to extend Ken's comment about getting it into
the Python core. Would that be something possible for 1.6? I used a Python
version of MultiMapping in an ancient version of DocumentTemplate. I'm sure
the C version has been around for at least two or three years and would
appear pretty darn stable, since it seems to be at the core of a lot of
Zope's coolness.

Skip

On Tue, 25 Jan 2000, Skip Montanaro wrote:
>
> Guido> I think it depends on to what extent this is a common, useful
> Guido> idiom. Do you have evidence of that? Examples?
>
> Well, the first place I ran into it was in DocumentTemplates a few years
> ago. They used an idiom heavily which may have now been replaced by
> acquisition where you'd effectively push and pop value dicts onto a stack as
> you entered and exited nested blocks of DTML code. Their solution was a
> special dict-like object.

That's really interesting. I wrote a bunch of Python wrappers for
a UI toolkit a little while ago, and there were two main pieces of
machinery i built to make the toolkit pleasant to use:

1. a reasonably nice C++ extension class kit (this is where i
tried overloading operator new for the first time, so it
would allocate memory that could be freed by PyMem_DEL --
i don't know if CXX uses the same approach)

2. a second layer of Python wrapper classes for the extension
classes that implements extra methods in Python, and maintains
a hierarchy of default keyword argument values along the
inheritance hierarchy of widgets

The second of these things involved implementing exactly the kind
of dictionary stack that you mentioned.

The programming idiom for building widgets goes something like:

defaultstack = {} # maps class object to list of defaults dictionaries

class Label(Component):
defaults = _dict(text="Label", align=LEFT)

class Button(Label):
defaults = _dict(text="Button", align=CENTER, shadow=2)

...

w = Window(...)

Button.push(fg="white", bg="red",
font="-*-courier-bold-r-normal--14-*-*-*-*-*-*-*")
a = Button(w, text="one")
b = Button(w, text="two")
c = Button(w, text="three")
Button.pop()

This way you can install some options for a while and make a bunch
of widgets with your defaults, then pop the options and go back to
the previous state.

The layers of dictionaries that get composed in every widget's
__init__ function are (in order of priority):

1. any non-None keyword arguments to __init__
2. any non-None values in class.defaults
3. any non-None values in class.__bases__[0].defaults
4. any non-None values in class.__bases__[0].__bases__[0].defaults
etc.

When a new set of defaults is push()ed, the class's current
defaults are saved on the defaultstack for the class, and restored
when pop() gets called.

I don't *know* if this is really the best way to do things, but it
has seemed to work out pretty well in practice. It makes it more
convenient to deal with all the options you have to throw around
especially when doing UI stuff.

Anyway, i noticed the same sort of thing today while wondering about
using keyword arguments for HTML tag attributes. (Perhaps HTMLgen
does this sort of thing already -- sorry i haven't looked at it.)
Anyway, the following sort of helper might be useful in general:

class Default:
def __init__(self, cl, **defaults):
self.cl = cl
self.defaults = defaults

def __call__(self, *args, **kw):
for key, value in self.defaults:
if not kw.has_key(key): kw[key] = value
return apply(self.cl, args, kw)

Then you could do the same thing as above with:

MyButton = Default(Button, fg="white", bg="red",
font="-*-courier-bold-r-normal--14-*-*-*-*-*-*-*")
a = MyButton(w, text="one")
b = MyButton(w, text="two")
c = MyButton(w, text="three")

This is probably a cleaner way to do things. I haven't tried it,
but it might be a nice thing to have in Tkinter.



-- ?!ng

Skip Montanaro wrote:
>
> Every once in awhile I want to perform string interpolation using more than
> one dictionary. One way is to build a dictionary that's a union of multiple
> dictionaries:
>
> dict = {}
> dict.update(d1)
> dict.update(d2)
> ...
> s = format % dict
>
> Another way is the MultiDict approach that Digital Creations (used to?) use
> in their DocumentTemplate module (I can't remember the exact usage any
> more):
>
> dict = MultiDict()
> dict.append(d1)
> dict.append(d2)

Actually, push (and pop). The namspaces are managed as a stack.

> ...
> s = format % dict
>
> A MultiDict object maintains a list of the dicts it's been fed and searches
> them in order when __getitem__ is called.
>
> I'd like to propose a third alternative. How about if the string
> interpolation function accepted a tuple of dictionaries directly:
>
> s = format % (d1, d2)
>
> It would only be used when named interpolation was expected. I don't think
> there would be any conflict with current % operator semantics.

Yes. In the current semantics, you output the two dictionaries.

Try:

'%s %s' % ({'hello':'skip'},{})

Jim

--
Jim Fulton mailto:jim@digicool.com Python Powered!
Technical Director (888) 344-4332 http://www.python.org
Digital Creations http://www.digicool.com http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission. Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

Ken Manheimer wrote:
>
> On Tue, 25 Jan 2000, Skip Montanaro wrote:
>
> > Guido> Skip:
> > >> I'd like to propose a third alternative. How about if the string
> > >> interpolation function accepted a tuple of dictionaries directly:
> > >>
> > >> s = format % (d1, d2)
> > [...]
> > Guido> I think it depends on to what extent this is a common, useful
> > Guido> idiom. Do you have evidence of that? Examples?
> >
> > Well, the first place I ran into it was in DocumentTemplates a few years
> > ago. They used an idiom heavily which may have now been replaced by
> > acquisition where you'd effectively push and pop value dicts onto a stack as
> > you entered and exited nested blocks of DTML code. Their solution was a
> > special dict-like object.
>
> Implementation of acquisition basically uses a MultiDict underneath.

No it doesn't. Acquisition achieves combination of multiple
namespaces in much the same way that inheritence does (through
delagation).

> Consider acquisition as a cumulative context composed from the containers
> of the target of a web request. (Actually, a distinction is made between
> the object containment hierarchy of the target and the successive
> components of the path along which the target is reached by the request,
> with the containment contexts taking precedence - but that's probably not
> important here:-) Add in incidental things like the server environment
> (from which you can get HTTP_REFERER and cookies and so forth). Each of
> the components can be a dictionary or a MultiDict (or a sequence of pairs,
> i think), and they're ultimately composed in a MultiDict.
>
> I think another place in zope where multidicts play prominently is in the
> security mechanism, where any object can have local roles, and the
> ultimate role of a user within a context is composed from the union across
> the containment hierarchy. There probably are lots of other places where
> multidicts are used.

Acquisition plays a role in security, but MultiDicts-like things are
not used.

> Suffice to say that there's a lot of composing of contexts that goes on in
> Zope in general, acquistion being a prime example but not the only one,
> and multidicts play heavily in many. I would be surprised if this need to
> combine contexts is peculiar to web server, or general server
> applications.
>
> > [...]
> > It's not a big deal. If it seems too obscure the other obvious solutions
> > are not gruesome.
>
> I suppose we'd be pretty happy to have something like MultiDict as part of
> python...

Note that Zope actually uses two separate flavors. The one used most
in Zope (in DocumentTemplate) has very specific hooks to work with
Zope's security machinery.

Jim

--
Jim Fulton mailto:jim@digicool.com Python Powered!
Technical Director (888) 344-4332 http://www.python.org
Digital Creations http://www.digicool.com http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission. Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

Tim Peters wrote:
>
(snip)
> I wasn't aware of the MultiDict approach Skip mentioned,

See the MultiMapping module in ExtensionClass. You can get the
latest flavor of this in the latest Zope release.

> but thought it
> looked spot on for the general case! Skip, is the long-windedness of
>
> dict = MultiDict()
> dict.append(d1)
> dict.append(d2)
> ...
> s = format % dict

Note the rather important *stack* sematics of
MultiMappings. We often push and pop namespaces
on and off of MultiMappings in use.

> the part you didn't like about that? If so, how about changing the
> constructor to
>
> def __init__(self, *dicts):
> ...
>
> instead so you could use it as a one-liner
>
> format % MultiDict(d1, d2, ...)
>
> ? That's exactly the same as the tuple idea, except there's a nice
> descriptive word in the middle of it <wink>.

This is exactly what the current MultiMapping "class" does.

Jim

--
Jim Fulton mailto:jim@digicool.com Python Powered!
Technical Director (888) 344-4332 http://www.python.org
Digital Creations http://www.digicool.com http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission. Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.

Skip Montanaro wrote:
>
(snip)
> I guess the next question is to extend Ken's comment about getting it into
> the Python core. Would that be something possible for 1.6? I used a Python
> version of MultiMapping in an ancient version of DocumentTemplate. I'm sure
> the C version has been around for at least two or three years and would
> appear pretty darn stable,

Yes. I'm sure you'd have to de-ExtensionClass-ify it to get it
into the core. :)

> since it seems to be at the core of a lot of
> Zope's coolness.

Actually, it isn't, as there is a separate similar thing
(TemplateDict) used in DocumentTemplate.

Jim

--
Jim Fulton mailto:jim@digicool.com Python Powered!
Technical Director (888) 344-4332 http://www.python.org
Digital Creations http://www.digicool.com http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission. Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.