Mailing List Archive

>That cuts both ways, of course, as does any difference. That is, I once got
>miffed at Perl because my named, nested functions weren't working. "man
>pages" or not <wink>, it wasn't until I studied the source that I figured
>out Perl effectively moves all named functions to the top level, and if you
>have two nested functions in a package that happen to have the same name,
>the first gets tossed into the bit bucket:

You didn't localize your function! Remember, in Perl things
are only local if you say they are.

>$x = 5;

>sub outer1 {
> my $x = 50;
> sub inner {
> print "in the first inner, and here's x: $x\n";
> }
> &inner();
>}

>sub outer2 {
> my $x = 500;
> sub inner {
> print "in the second inner, and here's x: $x\n";
> }
> &inner();
>}

>&outer1();

That could have been like this:

sub outer1 {
my $x = 50;
local *inner = sub {
print "in the first inner, and here's x: $x\n";
};
&inner();
}

sub outer2 {
my $x = 500;
local *inner = sub {
print "in the second inner, and here's x: $x\n";
};
&inner();
}

&outer1();

And run just fine. BTW, you should have gotten a bucket of frightened
warnings out of the compiler about that. Syntax checking the compile,
with warnings (somewhat like gcc -Wall -pedantic, but without *.o
generation):

% perl -wc /tmp/timstuff
Variable "$x" will not stay shared at /tmp/timstuff line 6.
Variable "$x" will not stay shared at /tmp/timstuff line 14.
Subroutine inner redefined at /tmp/timstuff line 13.
Name "main::x" used only once: possible typo at /tmp/timstuff line 1.
/tmp/timstuff syntax OK

If that is mysterious, bump up the verbosity:

% perl -Mdiagnostics -wc /tmp/timstuff

Variable "$x" will not stay shared at /tmp/timstuff line 6 (#1)

(W) An inner (nested) named subroutine is referencing a lexical
variable defined in an outer subroutine.

When the inner subroutine is called, it will probably see the
value of the outer subroutine's variable as it was before and
during the *first* call to the outer subroutine; in this case,
after the first call to the outer subroutine is complete, the
inner and outer subroutines will no longer share a common value
for the variable. In other words, the variable will no longer
be shared.

Furthermore, if the outer subroutine is anonymous and references
a lexical variable outside itself, then the outer and inner
subroutines will never share the given variable.

This problem can usually be solved by making the inner subroutine
anonymous, using the sub {} syntax. When inner anonymous subs
that reference variables in outer subroutines are called or
referenced, they are automatically rebound to the current values
of such variables.

Variable "$x" will not stay shared at /tmp/timstuff line 14 (#1)

Subroutine inner redefined at /tmp/timstuff line 13 (#2)

(W) You redefined a subroutine. To suppress this warning, say

{
local $^W = 0;
eval "sub name { ... }";
}

Name "main::x" used only once: possible typo at /tmp/timstuff line 1 (#3)

(W) Typographical errors often show up as unique variable names.
If you had a good reason for having a unique name, then just
mention it again somehow to suppress the message. The use vars
pragma is provided for just this purpose.

/tmp/timstuff syntax OK

Yes, this local function thing is in the Perl Cookbook in the subroutines
chapter. Larry said that the next release of Perl *shall support

sub outer1 {
my $x = 50;
my sub inner {
print "in the first inner, and here's x: $x\n";
}
&inner();
}

sub outer2 {
my $x = 500;
my sub inner {
print "in the second inner, and here's x: $x\n";
}
&inner();
}

&outer1();

But this is lexical scoping, not dynamic scoping. With the local
sub inner, inner could call fred, and then fred could still
see inner to call it again in indirect recursion. That couldn't
happen with my sub inner, which is lexically scoped.

BTW, see what I mean about the importance of good error messages?
I'm used to something more than Ken Thompson's "?" retort. Everytime I
get NameError from Python, I just want to kick it.

And yes, those were compile-time warnigns. In the developer
release of Perl, you can promote some or all of those into
fatals if you want, which will make Guido carp less.

use warnings qw/FATAL uninitialized numeric/;

is more in the "python" style. And you can always say

use warnings qw/FATAL all/;

if you feel all warnings should kill you. Note that that pragma, like use
strict, is lexically scoped to the surrounding nested scope. That way
one block can selectively enable classes of warnings. Warning classes
are:

all ambiguous closed closure default deprecated exec io misc newline
numeric octal once parenthesis pipe precedence printf recursion
redefine reserved semicolon signal substr syntax taint uninitialized
unopened unsafe untie utf8 void

And you can do

use warnings;
no warnings qw/once exec ambiguous/;

or whatever.

You know, it occurs to me that it might be nice to be able to selectively
promote only compile-time warnings to fatals. Perl does a lot of
compile-time stuff (including constant expression folding, inlining of
some subroutines, and folding of those results!).

What does Python do at compile-time?

--tom

[posted & mailed]

[.tchrist, discovers that Python "def" is executable, and notes that
this is a gotcha for Perl programmers]

[.Tim, notes that Perl's "sub" is equally a gotcha for Python programmers]

[back to tchrist]
> You didn't localize your function! Remember, in Perl things
> are only local if you say they are.

But it doesn't matter, Tom <0.5 wink>: the "gotchas" you identified in
Python are things that would throw a Perl programmer if they hadn't read and
understood the Python docs first. Perl is at least equally as surprising to
the Python programmer who hasn't read and understood the Perl docs first (as
the self-proclaimed Pythoneers who flame on c.l.p.m. prove more than often
enough).

[.and where I had
sub inner sub { ... },
do instead]
> ...
> That could have been like this:
>
> sub outer1 {
> my $x = 50;
> local *inner = sub {
> print "in the first inner, and here's x: $x\n";
> };
> &inner();
> }

No problem, but that's (symbol table globbing) a *real* stretch for someone
just learning the language. There are real barriers for native speakers of
either language in picking up the other.

I ended up doing something like this instead:

sub makefac {
my $fac;
$fac = sub {
my $n = shift;
return 1 if $n < 2;
return $n * &$fac($n - 1);
};
return $fac;
}

my $i;
for ($i = 0; $i < 1000000; ++$i) {
my $fac = makefac;
my $n = &$fac(3);
print "$n ";
}

This one is more interesting than it looks <wink>: in the last version of
Perl I tried it on, it leaked memory at a prodigious rate. The reason it's
interesting is because it's faking pretty much what Python would have to do
to support lexical closures, given that a nested "def" acts exactly like an
assignment statement binding a name local to its containing function (and
here an anonymous sub is assigned to a lexical local of *its* containing
function).

In Python and in Perl (at least in the Perl I last tried that under) that
creates cyclic garbage: makefac's lexicals have to contain a binding for
fac, and fac has to point back to makefac's lexicals in order to resolve
up-level references. And that's (unrecoverable trash) the reason Python
doesn't have lexical closures today. If a nested def were-- as my original
nested "sub inner" is in Perl --treated as global, that problem would go
away, and for the same reason it does in Perl. But it wouldn't be Python
anymore either <wink>.

> ...
> BTW, you should have gotten a bucket of frightened warnings out of
> the compiler about that.

I got strong indications of trouble with -w, yes.

[.examples of great msgs produced by
perl -wc
and
perl -Mdiagnostics -wc
]

> ...
> BTW, see what I mean about the importance of good error messages?

No argument from me that Perl's msgs are often very helpful, much more often
pinpointing a problem than Python's (the "runaway multi-line string starting
at line nnn" is one Python should steal!). On the other hand, I've written
a few Klines of production Perl too, and my experience is that Perl has so
many more syntactic and semantic gotchas than Python that the language would
be nearly unusable without great msgs. Python's NameError may drive you
nuts, but it's the same as Perl's runtime "use of uninitialized value" --
and Perl is no more help in tracking down the *cause* of those than is
Python in cracking a NameError. At least Perl's -w warns about names that
appear only once! You have to grab a copy of PyLint for that (but then you
are a fan of separate tools, right <wink>?).

[various ways to promote warning into errors, "if you feel all
warnings should kill you"]

Whereas Python has no warnings of any kind -- it it's irked enough at me to
complain at all, it's irked enough to kill me. In that it's a faithful
reflection of Guido's personality <wink>. Seriously, I do prefer that.

> ...
> You know, it occurs to me that it might be nice to be able to
> selectively promote only compile-time warnings to fatals.

As a native speaker of Python, -w and "use strict" decorate all my Perl, and
in the few Official Perl projects I've been on "NO WARNINGS!" was an
absolute rule. The more fatals the better.

> Perl does a lot of compile-time stuff (including constant
> expression folding, inlining of some subroutines, and folding
> of those results!).
>
> What does Python do at compile-time?

It's too busy desperately scrounging for excuses to kill your program with a
bogus SyntaxError to do anything useful <wink>. About the only compile-time
optimization is that function locals (in the absence of "exec" and "import
*" statements) are mapped to consecutive indices into a contiguous runtime
vector.

"exec" and "import *" can alter the *set* of local names, so their presence
inhibits that optimization. That's why "exec" is a stmt rather than a
function: since no builtin names are resolved at compile-time, and any name
may be bound to any object at runtime, if exec were a function Python
couldn't know that

exec(s)

does refer to the builtin "exec", or that

f(s)

*doesn't*, so the local-var optimization would be crippled almost always.
So "exec" is a keyword, the only kind of word whose meaning is known for
sure at compile-time.

Python's default arguments are sometimes used to fake lexical closures, but
they're also used to fake the effect of compile-time binding; e.g., because
def *is* executable, when Python (at runtime) gets to

def somefunc(x, int=int):
... = int(...)

the local name "int" gets bound to whatever-the-heck the global name "int"
is bound to at the time the def is executed. Of course it *is* bound to the
builtin int function, but Python can't know that at compile-time. This
trick lets the programmer force the issue, and within somefunc the "int"
lookup is very fast (just a C indexed array ref); without this trick each
dynamic instance of "int" in somefunc does a failing lookup in the global
dict first, and then a successful lookup in the builtin dict.

So, no matter what anyone here tells you, Perl doesn't have a monopoly on
being cute & repulsive at the same time <0.8 wink>.

takes-one-to-know-one-ly y'rs - tim

On 20 Aug 1999, Tom Christiansen wrote:
> In comp.lang.python,
> "Tim Peters" <tim_one@email.msn.com> writes:
> :In Python and in Perl (at least in the Perl I last tried that under) that
> :creates cyclic garbage: makefac's lexicals have to contain a binding for
> :fac, and fac has to point back to makefac's lexicals in order to resolve
> :up-level references. And that's (unrecoverable trash) the reason Python
> :doesn't have lexical closures today.
>
> That's quite right. Doesn't python have some similar issue related
> to the use of bound methods?

Not in the simple case, no.

I take it you're thinking a bound method has a reference to the instance
which has a reference to the method? That's not the case, for, if you have
a class like this:

class Account:
def __init__(self,amount):
self.amount = amount
def withdraw(self,amount):
self.amount = self.amount - amount

then executing

W=Account(50).withdraw

gets you a bound method which has a im_self member which refers to an
Account object which has a __class__ member which refers to the Account
class which has a __dict__ member which contains a function "withdraw" (as
in plain Python function).

(with apologies to the authors of sicp for using that example yet again,
and to all concerned for the length of that previous sentence).

>>> Account(50).withdraw.im_self.__class__.__dict__["withdraw"]
<function withdraw at 87cd30>

No cycles, see?

internally-y'rs
Michael

[courtesy cc of this posting mailed to cited author]

In comp.lang.python,
"Tim Peters" <tim_one@email.msn.com> writes:
:There are real barriers for native speakers of
:either language in picking up the other.

What do you think falls into this category? What thing
about Python makes it hard for a native speaker thereof
to learn Perl, and vice versa?

[massive closure generator deleted]

:This one is more interesting than it looks <wink>: in the last version of
:Perl I tried it on, it leaked memory at a prodigious rate.

Absolutely. You've a circular reference there.

:The reason it's
:interesting is because it's faking pretty much what Python would have to do
:to support lexical closures,

I'm not sure why you say "faking".

:In Python and in Perl (at least in the Perl I last tried that under) that
:creates cyclic garbage: makefac's lexicals have to contain a binding for
:fac, and fac has to point back to makefac's lexicals in order to resolve
:up-level references. And that's (unrecoverable trash) the reason Python
:doesn't have lexical closures today.

That's quite right. Doesn't python have some similar issue related
to the use of bound methods?

:Whereas Python has no warnings of any kind -- it it's irked enough at me to
:complain at all, it's irked enough to kill me. In that it's a faithful
:reflection of Guido's personality <wink>. Seriously, I do prefer that.

perl -w is "pardon me, but are you really sure you know what you're
doing there?", it's not "die die die, you ignorant script kiddie!".

:As a native speaker of Python, -w and "use strict" decorate all my Perl, and
:in the few Official Perl projects I've been on "NO WARNINGS!" was an
:absolute rule. The more fatals the better.

That's how I virtually always programmed it. Sometimes I don't bother
on little one-liners that turn Perl into Super Sed, like

perl -i.orig -pe 's/^(\d+)/$1 + $./e' file1 file2 file3
# add current line number to each number that occurs at
# the front of a line, and put changes back in place,
# backups into file.orig

But real programs get -w and use strict always. After all,
just as with /* Lint Happiness */ of days of old, I know
how to temporarily rescind them if I really am convinced
that I really do know what I'm doing.

:> What does Python do at compile-time?
:It's too busy desperately scrounging for excuses to kill your program with a
:bogus SyntaxError to do anything useful <wink>. About the only compile-time
:optimization is that function locals (in the absence of "exec" and "import
:*" statements) are mapped to consecutive indices into a contiguous runtime
:vector.

You, Tim, might someday look at the interesting tricks that the Perl
compiler and interpreter go through about storage, speed, and tacit reuse
of my() variables (lexically scoped locals). Remember that my() in Perl
is one of Perl's four declarations, something then that the *compiler*
knows about, and can take suitable actions based up static analysis.

(And I nearly never mean the (C or byte)-code-generating backends when
I say "compiler" in a Perl context. I just the guy who does the parsing
in the previous paragraph.)

I remember my frutration as a flaming C advocate (you didn't really
think I started with Perl, did you? :-) getting a job at a company
that made US$75k Fortran compilers being told that Fortran would always
be faster than C because the fc compiler could know more things about
your code than the cc compiler could, and thus make better compile-time
optimization to speed run-time. I wonder whether Python simply hasn't
matured as far as Perl has along those lines, or whether in fact there
are inherent barriers, as there are between Fortran and C. What do you
think? I also notice that Python is bereft of compiler pragmata.

:"exec" and "import *" can alter the *set* of local names, so their presence
:inhibits that optimization.

Aren't you glad you don't have Tcl's "upscope" naughtiness?
People sometimes ask for this in Perl, to be able pop up a level
into your dynamic caller's private lexical scope? I'd rather have
pass-by-name. :-)

:So, no matter what anyone here tells you, Perl doesn't have a monopoly on
:being cute & repulsive at the same time <0.8 wink>.

Oh, I know. It's the new converts who are scariest. The adage that
one should "beware the man of one book" becomes "beware the programmer
of one language" in our world, or should that really be "the zeal of
fresh converts burns annoyingly"? :-)

--tom
--
Timesharing: the use of several people by the computer

On Fri, Aug 20, 1999 at 05:39:30AM -0400, Tim Peters wrote:

[Tim about Python's compiler & interpreter]
> It's too busy desperately scrounging for excuses to kill your program with a
> bogus SyntaxError to do anything useful <wink>. About the only compile-time
> optimization is that function locals (in the absence of "exec" and "import
> *" statements) are mapped to consecutive indices into a contiguous runtime
> vector.
>
> "exec" and "import *" can alter the *set* of local names, so their presence
> inhibits that optimization. That's why "exec" is a stmt rather than a

[..]

Out of curiosity, is the compiler smart enough to see that exec statements
like

exec "a = 1 * 10" in {}

or

exec "foo = bar * zed" in {'bar': 5, 'zed', 'z'}

are not able to modify any namespace ? I can understand that something like

exec "foo = bar * zed" in myenv

is too indirect for the compiler to worry about, but the upper two examples
do not use any type of object that isn't 100% certain at compile time... Or
is it possible to redefine builtins like ints, dicts and strings ? ;) (Or
isn't the compiler aware of this ?)

--
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

[Thomas Wouters]
> Out of curiosity, is the compiler smart enough to see that
> exec statements like
>
> exec "a = 1 * 10" in {}
>
> or
>
> exec "foo = bar * zed" in {'bar': 5, 'zed', 'z'}
> ...

No. It would first have to parse the exec'ed string at compile-time, but
that's delayed to run-time, too late to help the local/global decisions made at
compile-time. In real execs, one or more of the arguments are unknown at
compile-time anyway (nobody would write the above in a real program, but if
you're the exception feel free to submit a tedious patch to look for stuff like
that <wink>).

most-optimizations-aren't-ly y'rs - tim

On Mon, Aug 23, 1999 at 01:58:23AM -0400, Tim Peters wrote:

> > Out of curiosity, is the compiler smart enough to see that
> > exec statements like
> > exec "a = 1 * 10" in {}

> No. It would first have to parse the exec'ed string at compile-time, but
> that's delayed to run-time, too late to help the local/global decisions made at
> compile-time. In real execs, one or more of the arguments are unknown at
> compile-time anyway (nobody would write the above in a real program, but if
> you're the exception feel free to submit a tedious patch to look for stuff like
> that <wink>).

Heh, like i said, it was curiosity. I can imagine code doing the above to
provide simple syntax checking before writing the exec'd code to a .py file
for use later. However, I dont plan on writing anything that does that
without using 'exec' in other places, and cares about speed. I'll use the
extra runtime to make loving python-remarks to the perl-loving colleague who
sits across from me ;-)

Nevertheless, compiler and language internals fascinate me. As do computer
internals in general. Which reminds me... I hardly see any
'development-traffic' on this list, other than some sideways references to a
CVS tree... Is there a secret society or do you just have meetings in
guido's favorite pub ?

Noisily-and-nosey-y'rs, Thomas

--
Thomas Wouters <thomas@xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

Tom Christiansen <tchrist@mox.perl.com> wrote:
> I remember my frutration as a flaming C advocate (you didn't really
> think I started with Perl, did you? :-) getting a job at a company
> that made US$75k Fortran compilers being told that Fortran would always
> be faster than C because the fc compiler could know more things about
> your code than the cc compiler could, and thus make better compile-time
> optimization to speed run-time. I wonder whether Python simply hasn't
> matured as far as Perl has along those lines, or whether in fact there
> are inherent barriers, as there are between Fortran and C. What do you
> think? I also notice that Python is bereft of compiler pragmata.

In part it's inherent problems. Python is way dynamic and therefore hard
to speed up.

But, Pythoneers also know it can be done, by looking at other languages
that are dynamic, such as Smalltalk and Lisp, which have managed to gain some
quite massive improvements in speed.

Pythoneers are actively looking at ways to speed up Python code, so in part
it's also simply maturing. I heard Python *has* speeded up a lot since
earlier versions, in fact.

Regards,

Martijn

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

>>>>> "Tim" == Tim Peters <tim_one@email.msn.com> writes:

[SNIPPED lots of interesting stuff]

Tim> You have to grab a copy of PyLint for that (but then you are
Tim> a fan of separate tools, right <wink>?).

And could you tell us where this mythical PyLint is? More advtising
couldn't hurt (I have browsed python.org a coupl times, but have seen
no mention... though I haven't looked for PyLint (didn't know it
existed))

[SNIPPED remainder of mail]

Bye, J

PS: Or is this really "mythical" in that it only exists in Tim's
imagination? ;-)

- --
Jürgen A. Erhard eMail: jae@ilk.de phone: (GERMANY) 0721 27326
My WebHome: http://members.tripod.com/~Juergen_Erhard
GTK - Free X Toolkit (http://www.gtk.org)
"Windows NT" is an acronym for "Windows? No thanks." -- Russ McManus
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v0.9.10 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iEYEARECAAYFAjfBk5YACgkQN0B+CS56qs0ehACggQNyv4ZRq+j/WPvMXm0gh6bc
AZcAni5OB0bqX3yUbQaYx/DWuvkU+qZw
=NaQV
-----END PGP SIGNATURE-----

"Juergen A. Erhard" wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> >>>>> "Tim" == Tim Peters <tim_one@email.msn.com> writes:
>
> [SNIPPED lots of interesting stuff]
>
> Tim> You have to grab a copy of PyLint for that (but then you are
> Tim> a fan of separate tools, right <wink>?).
>
> And could you tell us where this mythical PyLint is? More advtising
> couldn't hurt (I have browsed python.org a coupl times, but have seen
> no mention... though I haven't looked for PyLint (didn't know it
> existed))
>
> [SNIPPED remainder of mail]
>
> Bye, J
>
> PS: Or is this really "mythical" in that it only exists in Tim's
> imagination? ;-)

Nope, it's a creation of the venerable Andrew Kuchling as part of the
kwParsing package (a parser generator package), which also includes the
gadfly relational database - way cool.

http://www.chordate.com/kwParsing/index.html
--
import string,sys;t=string;f=t.index;j=t.join;print(lambda
s:j(map(lambda
x,s=s:x[:f(x,':')+2]+j(map(lambda y:'%02X'%ord(y),x[f(x,':')+2:]),'
'),map
(lambda i,s=s:("%04X:
"%i)+s[i:i+16],range(0,len(s),16))),'\n'))(open(sys.
argv[1]).read())

On Mon, 23 Aug 1999 01:58:23 -0400, Tim Peters wrote:
>[Thomas Wouters]
>> exec "a = 1 * 10" in {}
>> or
>> exec "foo = bar * zed" in {'bar': 5, 'zed', 'z'}
>No. It would first have to parse the exec'ed string at compile-time,
>but that's delayed to run-time, too late to help the local/global
>decisions made at compile-time. In real execs, one or more of the
>arguments are unknown at compile-time anyway (nobody would write the
>above in a real program, but if you're the exception feel free to
>submit a tedious patch to look for stuff like that <wink>).
>most-optimizations-aren't-ly y'rs - tim

That almost looks like a place bytecodehacks could help...

Unfortunately, my net connection seems a little flaky today, so I can't
test it out in any greater detail...

Later,
Blake.

--
One Will. One Dream. One Truth. One Destiny. One Love.

[Thomas Wouters]
> ...
> Nevertheless, compiler and language internals fascinate me.

With luck you'll grow out of it <wink>.

> ...
> Which reminds me... I hardly see any 'development-traffic' on this
> list, other than some sideways references to a CVS tree... Is there
> a secret society or do you just have meetings in guido's favorite
> pub ?

There are many secret societies; most development takes place in Guido's head,
while most of the rest occurs under the auspices of one of the Secret Insider
Groups (SIG, for short):

http://www.python.org/sigs/

Python's source code is exceptionally easy to understand, so pick something
that interests you from

http://www.python.org/cgi-bin/todo.py

and have at it.

we'll-beat-the-curiosity-out-of-you-yet<wink>-ly y'rs - tim