Mailing List Archive

Re: [Idle-dev] Forward progress with full backward compatibility
Hi!

David Scherer on idle-dev@python.org:
[...]
> in the interpreter* is fast. In principle, one could put THREE operators in
> the language: one with the new "float division" semantics, one that divided
> only integers, and a "backward compatibility" operator with EXACTLY the old
> semantics:
[...]
> An outline of what I did:
[...]

Yes, this really clever. I like the ideas.

[me]:
> > 2. What should the new Interpreter do, if he sees a source file without a
> > pragma defining the language level? There are two possibilities:
[...]
> > 2. Assume, it is a new source file and apply language level 2 to it.
> > This has the disadvantage, that it will break any existing code.

> I think the answer is 2. A high-quality script for adding the pragma to
> existing files, with CLI and GUI interfaces, should be packaged with Python.
> Running it on your existing modules would be part of the installation
> process.

Okay. But what is with the Python packages available on
the Internet? May be the upcoming dist-utils should handle this?
Or should the Python core distribution contain a clever installer
program, which handles this?

> Long-lived modules should always have a language level, since it makes them
> more robust against changes and also serves as documentation. A version
> statement could be encouraged at the top of any nontrivial script, e.g:
>
> python 1.6
[...]

global python_1_5 #implies global old_division
or
global python_1_6 #implies global old_division
or
global python_1_7 #may be implies global new_division

may be we can solve another issue just discussed on python_dev with

global source_iso8859_1
or
global source_utf_8

Cute idea... but we should keep the list of such pragmas short.

> Personally, I think that it makes more sense to talk about ways to
> gracefully migrate individual changes into the language than to put off
> every backward-incompatible change to a giant future "flag day" that will
> break all existing scripts. Versioning of some sort should be encouraged
> starting *now*, and incorporated into 1.6 before it goes final.

Yes.

> Indeed, but Guido has spoken:
>
> > Great ideas there, Bruce! I hope you will post these to an
> > appropriate mailing list (perhaps idle-dev, as there's no official SIG
> > to discuss the Python 3000 transition yet, and python-dev is closed).

May be someone can invite you into 'python-dev'? However the archives
are open to anyone and writing to the list is also open to anybody.
Only subscription is closed. I don't know why.

Regards, Peter
P.S.: Redirected Reply-To: to David and python-dev@python.org !
--
Peter Funk, Oldenburger Str.86, D-27777 Ganderkesee, Germany, Fax:+49 4222950260
office: +49 421 20419-0 (ArtCom GmbH, Grazer Str.8, D-28359 Bremen)
RE: Re: [Idle-dev] Forward progress with full backward compatibility [ In reply to ]
[Peter Funk]
> ...
> May be someone can invite you into 'python-dev'? However the archives
> are open to anyone and writing to the list is also open to anybody.
> Only subscription is closed. I don't know why.

The explanation is to be found at the very start of the list -- before it
became public <wink>. The idea was to have a much smaller group than
c.l.py, and composed of people who had contributed non-trivial stuff to
Python's implementation. Also a group that felt comfortable arguing with
each other (any heat you may perceive on this list is purely illusory
<wink -- but the mutual respect level here *is* sky high, and we count on
Guido's "subscription veto" to keep it that way>).

So the idea was definitely to discourage participation(!), but never to do
things in secret. Keeping subscription closed has served its purposes
pretty well, despite that the only mechanism enforcing civility here is the
lack of an invitation. Elitist social manipulation at its finest <wink>.
RE: [Idle-dev] Forward progress with full backward compatibility [ In reply to ]
[Peter Funk]
> ...
> 2. What should the new Interpreter do, if he sees a source file without a
> pragma defining the language level?

Python has tens of thousands of users now -- if it doesn't default to
"Python 1.5.2" (however that's spelled), approximately 79.681% of them will
scream. Had the language done this earlier, it would have been much more
sellable to default to the current version.

However, a default is *just* "a default", and platform-appropriate config
mechanism (from Windows registry to cmdline flag) could be introduced to
change the default.

That is, 1.7 comes out and all my code runs fine without changing a thing.
Then I set a global config option to "pretend every module that doesn't say
otherwise has a 1.7 pragma in it", and run things again to see what breaks.
As part of the process of editing the files that need to be fixed, I can
take that natural opportunity to dump in a 1.7 pragma in the modules I've
changed, or a 1.6 pragma in the broken modules I can't (for whatever reason)
alter just yet.

Two pleasant minutes <wink> later, I'll have 6,834 .py files all saying
"1.7" at the top. Hmm! So when 1.8 comes out, not a one of them will use
any incompatible 1.8 features. So I'll also need a global config option
that says "pretend every module has a 1.8 pragma in it, *regardless* of
whether it has some other pragma in it already". But that will also screw
up the one .py file I forgot that had a 1.5.2 pragma in it.

Iterate this process a half dozen times, and I'm afraid the end result is
intractable.

Seems it would be much more tractable over the long haul to default to the
current version. Then every incompatible change will require changing every
file that relied on the old behavior (to dump in a "no, I can't use the
current version's semantics" pragma) -- but that's the situation today too.
The difference is that the minimal change required to get unstuck would be
trivial.

A nice user (like me <wink>) would devote their life to keeping up with
incompatible changes, so would never ever have a version pragma in any file.

So I vote "default to current version" -- but, *wow*, that's going to be
hard to sell.

Tech note: Python's front end is not structured today in such a way that
it's feasible to have the parser deal with a change in the set of keywords
keying off a pragma -- any given identifier today is either always or never
a keyword, and that choice is hardwired into the generated parse tables.
Not a reason to avoid starting this process with 1.6, just a reason to avoid
adding new keywords in 1.6 (it will take some real work to overcome the
front end's limitations here).

go-for-it!-ly y'rs - tim
Re: RE: [Idle-dev] Forward progress with full backward compatibility [ In reply to ]
>>>>> "GL" == Glyph Lefkowitz <glyph@twistedmatrix.com> writes:

GL> As long as we're talking about versioning issues, could we
GL> perhaps introduce a slightly more robust introspective
GL> facility than

GL> assert(sys.version[:3])=='1.5'

sys.hexversion?

Python 1.6a2 (#26, Apr 12 2000, 13:53:57) [GCC 2.8.1] on sunos5
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
>>> import sys
>>> sys.hexversion
17170593
>>> hex(sys.hexversion)
'0x10600a1'
Re: RE: [Idle-dev] Forward progress with full backward compatibility [ In reply to ]
>>>>> "GL" == Glyph Lefkowitz <glyph@twistedmatrix.com> writes:

BAW> sys.hexversion?

GL> Thank you!

GL> I stand corrected (and embarrassed) but perhaps this could be
GL> a bit better documented? a search of Google comes up with
GL> only one hit for this on the entire web:
GL> http://www.python.org/1.5/NEWS-152b2.txt ...

Yup, it looks like it's missing from Fred's 1.6 doc tree too.

Do python-devers think we also need to make the other patchlevel.h
constants available through sys?

If so, and because sys.hexversion is currently undocumented, I'd
propose making sys.hexversion a tuple of

(PY_VERSION_HEX, PY_MAJOR_VERSION, PY_MINOR_VERSION,
PY_MICRO_VERSION, PY_RELEASE_LEVEL, PY_RELEASE_SERIAL)

or leaving sys.hexversion as is and crafting a new sys variable which
is the [1:] of the tuple above.

Prolly need to expose PY_RELEASE_LEVEL_{ALPHA,BETA,GAMMA,FINAL} as
constants too.

-Barry
Re: RE: [Idle-dev] Forward progress with full backward compatibility [ In reply to ]
> sys.hexversion?
>
> Python 1.6a2 (#26, Apr 12 2000, 13:53:57) [GCC 2.8.1] on sunos5
> Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
> >>> import sys
> >>> sys.hexversion
> 17170593
> >>> hex(sys.hexversion)
> '0x10600a1'

bitmasks!?

(ouch. python is definitely not what it used to be. wonder
if the right answer to this is "wouldn't a tuple be much more
python-like?" or "I'm outta here...")

</F>
Re: RE: [Idle-dev] Forward progress with full backward compatibility [ In reply to ]
>>>>> "FL" == Fredrik Lundh <effbot@telia.com> writes:

FL> (ouch. python is definitely not what it used to be. wonder
FL> if the right answer to this is "wouldn't a tuple be much more
FL> python-like?" or "I'm outta here...")

Yeah, pulling the micro version number out of sys.hexversion is ugly
and undocumented, hence my subsequent message.

The basically idea is pretty cool though, and I've adopted it to
Mailman. It allows me to do this:

previous_version = last_hex_version()
this_version = mm_cfg.HEX_VERSION

if previous_version < this_version:
# I'm upgrading

-Barry
Re: RE: [Idle-dev] Forward progress with full backwardcompatibility [ In reply to ]
> If so, and because sys.hexversion is currently undocumented, I'd
> propose making sys.hexversion a tuple of
>
> (PY_VERSION_HEX, PY_MAJOR_VERSION, PY_MINOR_VERSION,
> PY_MICRO_VERSION, PY_RELEASE_LEVEL, PY_RELEASE_SERIAL)

thanks. I feel better now ;-)

but wouldn't something like (1, 6, 0, "a1") be easier
to understand and use?

</F>
Re: RE: [Idle-dev] Forward progress with full backwardcompatibility [ In reply to ]
Fredrik Lundh writes:
> but wouldn't something like (1, 6, 0, "a1") be easier
> to understand and use?

Yes! (But you knew that....)


-Fred

--
Fred L. Drake, Jr. <fdrake at acm.org>
Corporation for National Research Initiatives
RE: [Idle-dev] Forward progress with full backward compatibility [ In reply to ]
Language pragmas are all fine and good, BUT ...

Here in the Real World(TM) we have to deal with version in compatibilities
out the wazoo. I am currently writing a java application that has to run
on JDK 1.1, and 1.2, and microsoft's half-way JDK 1.1+1/2 thingy. Python
comes installed on many major linux distributions, and the installed base
is likely to be even larger than it is now by the time Python 1.6 is ready
for the big time. I'd like to tell people who still have RedHat 6.2
installed in six months that they can just download a 40k script and not a
5M interpreter source tarball (which will be incompatible with their
previous python installation, which they need for other stuff) when I
deploy an end-user application. (Sorry, I can't think of another way to
say that, I'm still recovering from java-isms...) :-)

What I'm saying is that it would be great if you could write an app that
would still function with existing versions of the interpreter, but would
be missing certain features that were easier to implement with the new
language symantics or required a new core library feature. Backward
compatibility is as important to me as forward compatibility, and I'd
prefer not to achieve it by writing exclusively to python 1.5.2 for the
rest of my life.

The way I would like to see this happen is NOT with language pragmas
('global' strikes me as particularly inappropriate, since that already
means something else...) but with file-extensions. For example, if you
have a python file which uses 1.6 features, I call it 'foo.1_6.py'. I
also have a version that will work with 1.5, albeit slightly slower/less
featureful: so I call it 'foo.py'. 'import foo' will work correctly.
Or, if I only have 'foo.1_6.py' it will break, which I gather would be the
desired behavior.

As long as we're talking about versioning issues, could we perhaps
introduce a slightly more robust introspective facility than

assert(sys.version[:3])=='1.5'

?

And finally, I appreciate that some physics students may find it confusing
that 1/2 yeilds 0 instead of 0.5, but I think it would be easier to just
teach them to do 1./2 rather than changing the symantics of integer
constants completely ... I use python to do a lot of GUI work right now
(and it's BEAUTIFUL for interfacing with Gtk/Tk/Qt, so I'm looking forward
to doing more of it) and when I divide *1* by *2*, that's what I mean. I
want integers, because I'm talking about pixels.

It would be a real drag to go through all of my code and insert int(1/2)
because there's no way to do integer math in python anymore... (Besides,
what about 100000000000000000000L/200000000000000000000L, which I believe
will shortly be lacking the Ls...?)

Maybe language features that are like this could be handled by a
pseudo-module? I.E.

import syntax
syntax.floating_point_division()

or somesuch ...

I'm not sure how you'd implement this so it would be automatic in certain
contexts (merging it into your 'site' module maybe? that has obvious
problems though), but considering that such features may be NOT the
behavior desired by everyone, it seems strange to move the language in
that direction unilaterally.

______ __ __ _____ _ _
| ____ | \_/ |_____] |_____|
|_____| |_____ | | | |
@ t w i s t e d m a t r i x . c o m
http://www.twistedmatrix.com/~glyph/
Re: RE: [Idle-dev] Forward progress with full backwardcompatibility [ In reply to ]
>>>>> "FL" == Fredrik Lundh <effbot@telia.com> writes:

FL> but wouldn't something like (1, 6, 0, "a1") be easier
FL> to understand and use?

I wasn't planning on splitting PY_VERSION, just in exposing the other
#define ints in patchlevel.h

-Barry
Re: RE: [Idle-dev] Forward progress with full backward compatibility [ In reply to ]
On Wed, 12 Apr 2000, Barry A. Warsaw wrote:

> sys.hexversion?

Thank you!

I stand corrected (and embarrassed) but perhaps this could be a bit better
documented? a search of Google comes up with only one hit for this on the
entire web: http://www.python.org/1.5/NEWS-152b2.txt ...
Re: RE: [Idle-dev] Forward progress with full backwardcompatibility [ In reply to ]
Barry wrote:
> >>>>> "FL" == Fredrik Lundh <effbot@telia.com> writes:
>
> FL> but wouldn't something like (1, 6, 0, "a1") be easier
> FL> to understand and use?
>
> I wasn't planning on splitting PY_VERSION, just in exposing the other
> #define ints in patchlevel.h

neither was I. I just want Python to return those values in a
form suitable for a Python programmer, not a C preprocessor.

in other words:

char release[2+1];
sprintf(release, "%c%c", PY_RELEASE_LEVEL - 0x0A + 'a',
PY_RELEASE_SERIAL + '0');
sys.longversion = BuildTuple("iiis", PY_MAJOR_VERSION,
PY_MINOR_VERSION, PY_MICRO_VERSION, release)

(this assumes that the release serial will never exceed 9, but
I think that's a reasonable restriction...)

</F>
RE: RE: [Idle-dev] Forward progress with full backward compatibility [ In reply to ]
> The basically idea is pretty cool though, and I've adopted it to
> Mailman. It allows me to do this:
>
> previous_version = last_hex_version()
> this_version = mm_cfg.HEX_VERSION
>
> if previous_version < this_version:
> # I'm upgrading

Why can't you do that with tuples?

--david
RE: RE: [Idle-dev] Forward progress with full backward compatibility [ In reply to ]
>>>>> "DA" == David Ascher <DavidA@ActiveState.com> writes:

>> The basically idea is pretty cool though, and I've adopted it
>> to Mailman. It allows me to do this: previous_version =
>> last_hex_version() this_version = mm_cfg.HEX_VERSION if
>> previous_version < this_version: # I'm upgrading

DA> Why can't you do that with tuples?

How do you know they aren't tuples? :)

(no, Moshe, you do not need to answer :)

-Barry
RE: RE: [Idle-dev] Forward progress with full backwardcompatibility [ In reply to ]
> Do python-devers think we also need to make the other patchlevel.h
> constants available through sys?

Can't see why, but also can't see why not!

> If so, and because sys.hexversion is currently undocumented,

Since when has that ever stopped anyone :-)


> I'd
> propose making sys.hexversion a tuple of
>
> (PY_VERSION_HEX, PY_MAJOR_VERSION, PY_MINOR_VERSION,
> PY_MICRO_VERSION, PY_RELEASE_LEVEL, PY_RELEASE_SERIAL)
>
> or leaving sys.hexversion as is and crafting a new sys
> variable which
> is the [1:] of the tuple above.

My code already uses sys.hexversion to differentiate between 1.5 and
1.6, so if we do anything I would vote for a new name.

Personally however, I think the hexversion gives all the information
you need - ie, you either want a printable version - sys.version -
or a machine comparable version - sys.hexversion. Can't really
think of a reason you would want the other attributes...

Mark.
RE: RE: [Idle-dev] Forward progress with full backward compatibility [ In reply to ]
> > >>> hex(sys.hexversion)
> > '0x10600a1'
>
> bitmasks!?

Nah - a comparable number :-)

if sys.hexversion >= 0x01060100: # Require Python 1.6 or later!

Seems perfectly reasonable and understandable to me. And much
cleaner than a tuple:

if tuple_version[0] > 1 or
tuple_version[0] == 1 and tuple_version[6] >= 1:

etc

Unless Im missing the point - but I can't see any case other than
version comparisons in which hexversion is useful - so it seems
perfect to me.

> (ouch. python is definitely not what it used to be. wonder
> if the right answer to this is "wouldn't a tuple be much more
> python-like?" or "I'm outta here...")

Be sure to let us know.

Mark.
RE: RE: [Idle-dev] Forward progress with full backwardcompatibility [ In reply to ]
Mark Hammond quoted Barry Warsaw:
>> I'd
>> propose making sys.hexversion a tuple of
>> (PY_VERSION_HEX, PY_MAJOR_VERSION, PY_MINOR_VERSION,
>> PY_MICRO_VERSION, PY_RELEASE_LEVEL, PY_RELEASE_SERIAL)

If it's a tuple, the name "hexversion" makes absolutely no sense.
Call it version_tuple or something like that.

--amk
Re: RE: [Idle-dev] Forward progress with full backward compatibility [ In reply to ]
Mark Hammond wrote:
> Nah - a comparable number :-)

tuples can also be compared.

> if sys.hexversion >= 0x01060100: # Require Python 1.6 or later!

if sys.versiontuple >= (1, 6, 1):
...

</F>