Mailing List Archive

Notice of intent: rich comparisons
Now that 1.5.2 is finalized, I'll be starting to prepare a patch for the
rich comparisons.

Guido, how do you envision python-dev working? Do you want to hand out a
TODO list for 1.6?

--david
Re: Notice of intent: rich comparisons [ In reply to ]
> Now that 1.5.2 is finalized, I'll be starting to prepare a patch for the
> rich comparisons.

Cool!

> Guido, how do you envision python-dev working?

I though we'd just crack Monty Python jokes while pretending to work :-)

> Do you want to hand out a TODO list for 1.6?

Well, I'm not saying that this is the definitive list, but certainly
there are a bunch of things that definitely need to be incorporated.
Here's an excerpt of a list that I keep for myself, in no particular
order, with brief annotations (I have a meeting in 15 minutes :):

Redesign thread/init APIs

See the recent discussion in the thread-sig. There should be
some kind of micro-init that initializes essential components
(including the thread lock!) but doesn't create an interpreter
or a threadstate object. There should also be a notion of a
default interpreter.

Break up into VM, parser, import, mainloop, and other components?

The porting efforts to seriously small machines (Win/CE,
Pilot, etc.) all seem to need a Python VM that doesn't
automatically pull in the parser and interactive main loop.
There are other reasons as well to isolate various parts
of the functionality (including all numeric data types except
ints).

This could include: restructuring of the parser so codeop.py
can be simplified; making the interactive main loop a script.

String methods

Barry has his patches ready.

Unicode

What is the status of /F's latest attempts?

Rich comparisons

This is Dave's plan.

Coercions

Marc-Andre Lemburg has some good suggestions here:
http://starship.python.net/~lemburg/coercion-0.6.zip

Import revamp

It should be much easier to add hooks to allow importing
from zip files, from a URL, etc., or (different) hooks to
allow looking for different extensions (e.g. to automatically
generate ILU stubs). This should help the small platforms
too, since they often don't have a filesystem, so their
imports will have to come from some other place.

ANSI C

I'm tired of supporting K&R C. All code should be converted
to using prototypes. The Py_PROTO macros should go. We can
add const-correctness to all code. Too bad for platforms
without decent compilers (let them get GCC).

Buffer object API extensions

Marc-Andre Lemburg has a patch

Distutil

Should we start integrating some of the results of the
distutil SIG? (What's the status? I haven't been paying
attention.)

Misc

Jeremy Hylton's urllib2.py

Greg Stein's new httplib.py (or Jeremy's?)

Andrew Kuchling's new PCRE release?

The IPv6 patches?

Gotta run!

--Guido van Rossum (home page: http://www.python.org/~guido/)
Re: Notice of intent: rich comparisons [ In reply to ]
On 21 April 1999, Guido van Rossum said:
> Distutil
>
> Should we start integrating some of the results of the
> distutil SIG? (What's the status? I haven't been paying
> attention.)

It's in a state that other people can look at the code, but (apparently)
not yet good enough for floods of patches to come in. Right now,
Distutils has *just* enough marbles to "build" and install itself on
Unix systems, and someone has contributed a patch so it works on NT. No
support yet for compiling extensions, which of course is the important
thing.

Greg
--
Greg Ward - software developer gward@cnri.reston.va.us
Corporation for National Research Initiatives
1895 Preston White Drive voice: +1-703-620-8990
Reston, Virginia, USA 20191-5434 fax: +1-703-620-0913
Re: Notice of intent: rich comparisons [ In reply to ]
On Wed, 21 Apr 1999, Guido van Rossum wrote:

> > Guido, how do you envision python-dev working?
>
> I though we'd just crack Monty Python jokes while pretending to work :-)

That'd be better than what I've read on the perl5-porters list in the last
couple of days (which I'm reading because of the ad hoopla). They seem to
spend their time arguing about whether Perl codes are programs or scripts.
Viciously, too. =)

More seriously, some of the discussions they're having a worth keeping an
eye on, just as a form of legal industrial spying.

> Import revamp

There was an interesting bit from Greg, Jack, and MarkH on this topic
recently in the distutil-sig. Maybe one of them can bring the rest of us
up to date on the effort.

--david
Re: Notice of intent: rich comparisons [ In reply to ]
Guido wrote:

> String methods
>
> Barry has his patches ready.
>
> Unicode
>
> What is the status of /F's latest attempts?

most of the code is written and tested; the API still
needs some cleaning up. I also have a few patches
and other contributions in my inbox that I gotta
look into...

btw, the unicode module can be compiled for 8-bit
characters too. we could merge it with the string
module (one place to fix bugs...)

> Import revamp
>
> It should be much easier to add hooks to allow importing
> from zip files, from a URL, etc., or (different) hooks to
> allow looking for different extensions (e.g. to automatically
> generate ILU stubs). This should help the small platforms
> too, since they often don't have a filesystem, so their
> imports will have to come from some other place.

something similar to greg's imputil.py should definitely
be in the core...

> ANSI C
>
> I'm tired of supporting K&R C. All code should be converted
> to using prototypes. The Py_PROTO macros should go. We can
> add const-correctness to all code. Too bad for platforms
> without decent compilers (let them get GCC).

or ansi2knr.c (available from http://www.cs.wisc.edu/~ghost/).
otoh, it only converts function headers, so I'm not sure it's
worth the effort...

</F>
Re: Notice of intent: rich comparisons [ In reply to ]
Guido van Rossum writes:
>
> ANSI C
>
> I'm tired of supporting K&R C. All code should be converted
> to using prototypes. The Py_PROTO macros should go. We can
> add const-correctness to all code. Too bad for platforms
> without decent compilers (let them get GCC).
>

Maybe I'm just lucky, but I've written nothing but ANSI C for the last
8 years and have never had a problem compiling it on any machine. I
don't see this as being a huge issue.

>
> Misc
>

Depending on how deep I want to dig, I may have some patches to
ParseTuple() that would make life easier for some of us extension
writing fans. As it is right now, my experimental version of Swig
is using it's own version of ParseTuple (for various reasons).
Also, out of curiousity, is anyone making heavy use of the CObject
type right now?

Cheers,

Dave
Re: Notice of intent: rich comparisons [ In reply to ]
David Beazley wrote:
>
> Guido van Rossum writes:
> >
> > ANSI C
> >
> > I'm tired of supporting K&R C. All code should be converted
> > to using prototypes. The Py_PROTO macros should go. We can
> > add const-correctness to all code. Too bad for platforms
> > without decent compilers (let them get GCC).
> >
>
> Maybe I'm just lucky, but I've written nothing but ANSI C for the last
> 8 years and have never had a problem compiling it on any machine. I
> don't see this as being a huge issue.

I've had the same experience with things like prototypes.
The recent introduction of indented preprocessor instructions
caused me significant pain on some platforms. Fortunately, I
haven't had to deal with those platforms recently, still, the elegence
of indented preporicessor instructions wasn't worth the pain it caused.

As far as choice of compilers goes, sometimes gcc isn't an option.
I've had cases where I *had* to use the native compiler in order to
link to third-party libraries.

I support moving to ANSI C, for example, wrt prototypes, but
lets still be somewhat sensative to portability and not abuse
bad compilers without good reason.


>
> >
> > Misc
> >
>
> Depending on how deep I want to dig, I may have some patches to
> ParseTuple() that would make life easier for some of us extension
> writing fans. As it is right now, my experimental version of Swig
> is using it's own version of ParseTuple (for various reasons).
> Also, out of curiousity, is anyone making heavy use of the CObject
> type right now?

Yes.

Jim

--
Jim Fulton mailto:jim@digicool.com Python Powered!
Technical Director (888) 344-4332 http://www.python.org
Digital Creations http://www.digicool.com http://www.zope.org

Under US Code Title 47, Sec.227(b)(1)(C), Sec.227(a)(2)(B) This email
address may not be added to any commercial mail list with out my
permission. Violation of my privacy with advertising or SPAM will
result in a suit for a MINIMUM of $500 damages/incident, $1500 for
repeats.
Re: Notice of intent: rich comparisons [ In reply to ]
>>>>> "DB" == David Beazley <beazley@cs.uchicago.edu> writes:

DB> various reasons). Also, out of curiousity, is anyone making
DB> heavy use of the CObject type right now?

I used it a little, but only in a modified Python backend for SWIG.

Jeremy
Re: Notice of intent: rich comparisons [ In reply to ]
Guido van Rossum wrote:
>
> Break up into VM, parser, import, mainloop, and other components?
>
> The porting efforts to seriously small machines (Win/CE,
> Pilot, etc.) all seem to need a Python VM that doesn't
> automatically pull in the parser and interactive main loop.
> There are other reasons as well to isolate various parts
> of the functionality (including all numeric data types except
> ints).
>
> This could include: restructuring of the parser so codeop.py
> can be simplified; making the interactive main loop a script.

There's some good work out there by Skip Montanaro to revamp the
VM into a combined register/stack machine. He calls it rattlesnake.
More infos can be found in the list archive (it's currently sleeping):

http://www.egroups.com/folders/rattlesnake

I would like to see the VM/compiler pluggable so that experiments
like rattlesnake become easier to implement (extension modules vs.
patches to the core interpreter).

> Coercions
>
> Marc-Andre Lemburg has some good suggestions here:
> http://starship.python.net/~lemburg/coercion-0.6.zip

I will continue to work on these patches now that 1.5.2 is out.

There is a (more or less) detailed explanation of the patch set
at:

http://starship.skyport.net/~lemburg/CoercionProposal.html

Instead of using of using the PY_NEWSTYLENUMBER approach I'll
turn to the new type flags that were introduced in 1.5.2. The
Py_NotImplemented singleton return value will stay, because
performance tests have shown that this method does not cause
any significant hit where as the exception raise and catch
method does introduce a significant slow-down.

> Import revamp

Greg's imputil.py (it's in the distutils package) should provide
a good start.

> Buffer object API extensions

I'll put the complete patch up on starship later this week...
here are the basic prototypes:

DL_IMPORT(int) PyObject_AsCharBuffer Py_PROTO((PyObject *obj,
const char **buffer,
int *buffer_len));

/* Takes an arbitrary object which must support the (character,
single segment) buffer interface and returns a pointer to a
read-only memory location useable as character based input for
subsequent processing.

buffer and buffer_len are only set in case no error
occurrs. Otherwise, -1 is returned and an exception set.

*/

DL_IMPORT(int) PyObject_AsReadBuffer Py_PROTO((PyObject *obj,
const void **buffer,
int *buffer_len));

/* Same as PyObject_AsCharBuffer() except that this API expects
(readable, single segment) buffer interface and returns a
pointer to a read-only memory location which can contain
arbitrary data.

buffer and buffer_len are only set in case no error
occurrs. Otherwise, -1 is returned and an exception set.

*/

DL_IMPORT(int) PyObject_AsWriteBuffer Py_PROTO((PyObject *obj,
void **buffer,
int *buffer_len));

/* Takes an arbitrary object which must support the (writeable,
single segment) buffer interface and returns a pointer to a
writeable memory location in buffer of size buffer_len.

buffer and buffer_len are only set in case no error
occurrs. Otherwise, -1 is returned and an exception set.

*/

> Distutil
>
> Should we start integrating some of the results of the
> distutil SIG? (What's the status? I haven't been paying
> attention.)

With all the different modules and packages available for Python
starting to slowly introduce dependencies, I think the main
objective should be introducing a well organized installation
info system, e.g. a registry where modules and packages can
query and save version and dependency information. Someone on
the main list mentioned that JPython has something like this...

> Misc
>
> Jeremy Hylton's urllib2.py
>
> Greg Stein's new httplib.py (or Jeremy's?)
>
> Andrew Kuchling's new PCRE release?
>
> The IPv6 patches?

These are on my wish list, so I'll simply add them here:

· APIs to faciliate creation of extensions classes (using Python
classes and C functions as methods)

The main open question here is where and how to store C data in
the instances. This could also provide a way to migrate to
all Python classes for some future version.

· Restructure the calling mechanism used in ceval.c to make
it clearer, more flexible and faster (of course, :-). Also,
inline calling of C functions/methods (this produces a noticable
speedup).

I have a patch for the restructuring and an old one (against 1.5)
for the C function call inlining.

· A "fastpath" hook made available through the sys module.

This should hook into the module/package loader and should be
used whenever set to find a module (reverting to the standard
lookup method in case it can't find it). I have an old patch
that does this. It uses a simple Python function to redirect the
lookup to a precalculated dictionary of installed modules/packages.
This results in faster startup of the Python interpreter (by saving
a few hundred stat() calls).

--
Marc-Andre Lemburg Y2000: 254 days left
---------------------------------------------------------------------
: Python Pages >>> http://starship.skyport.net/~lemburg/ :
---------------------------------------------------------
Re: Notice of intent: rich comparisons [ In reply to ]
David Beazley wrote:
>
> >
> > Misc
> >
>
> Depending on how deep I want to dig, I may have some patches to
> ParseTuple() that would make life easier for some of us extension
> writing fans. As it is right now, my experimental version of Swig
> is using it's own version of ParseTuple (for various reasons).
> Also, out of curiousity, is anyone making heavy use of the CObject
> type right now?

You mean the one in Objects/cobject.c ? Sure, all my type extensions
export their C API that way... and it works just great (no more
linker problems, nice error messages when extensions are not found,
etc.).

--
Marc-Andre Lemburg Y2000: 254 days left
---------------------------------------------------------------------
: Python Pages >>> http://starship.skyport.net/~lemburg/ :
---------------------------------------------------------
Re: Notice of intent: rich comparisons [ In reply to ]
David Beazley wrote:

> Also, out of curiousity, is anyone making heavy use of the CObject
> type right now?

NumPy does.

--da
RE: Notice of intent: rich comparisons [ In reply to ]
[Guido and /F]
> > Unicode
> >
> > What is the status of /F's latest attempts?
>
> most of the code is written and tested; the API still
> needs some cleaning up. I also have a few patches
> and other contributions in my inbox that I gotta
> look into...

Im one of those contributors :-) I have made changes to the Win32
extensions that allows it to use /F's type instead of its own built-in
Unicode type. Further, the Windows CE port of these extensions actually
enables this - so we have some code actively using it.

However, this still doesnt give me anything more than I had before. We
need at least _some_ level of integration of this type into Python. The
key areas to me are:

* PyArg_ParseTuple and Py_BuildValue - Seems OK we use the "u" format
character analogous to "s". We havent thought of a "z" equivilent. This
is fairly easy.

* Some way for these functions to auto-convert. Eg, when the user passes
an 8bit string object, but the C side of the world wants a Unicode string.
This is harder, as an extra memory buffer for the conversion is needed.

This second problem is, to me, the next stumbling block. When we sort
these out, I feel we would have the start of a good platform to start
experimenting.

Mark.
Re: Notice of intent: rich comparisons [ In reply to ]
David Beazley writes:
> Maybe I'm just lucky, but I've written nothing but ANSI C for the last
> 8 years and have never had a problem compiling it on any machine. I
> don't see this as being a huge issue.

Wasn't SunOS 4.x the problem child here? There are still a lot of
people using these systems, for reasons that aren't clear to me. (Did
support for some older hardware get dropped? Sounds like the most
reasonable reason to use the old OS is legacy hardware.)


-Fred

--
Fred L. Drake, Jr. <fdrake@acm.org>
Corporation for National Research Initiatives
Re: Notice of intent: rich comparisons [ In reply to ]
>>>>> "Fred" == Fred L Drake <fdrake@cnri.reston.va.us> writes:

Fred> Wasn't SunOS 4.x the problem child here? There are still
Fred> a lot of people using these systems, for reasons that aren't
Fred> clear to me. (Did support for some older hardware get
Fred> dropped? Sounds like the most reasonable reason to use the
Fred> old OS is legacy hardware.)

There might be, but the bundled compiler in SunOS 4 was never intended
to be used for anything more than recompiling the kernel. Years ago,
I had conversations with Sun engineers who claimed that no bug in the
bundled compiler was ever fixed unless it directly impacted on kernel
rebuilds. Even back then they recommended using something other than
the bundled crufty old K&R compiler for building all other stuff, and
this was when there was that huge hoopla over the unbundling of the
compiler.

SunOS 4 users can probably be mostly satisfied with gcc.

-Barry
Re: Notice of intent: rich comparisons [ In reply to ]
Barry A. Warsaw writes:
> There might be, but the bundled compiler in SunOS 4 was never intended
> to be used for anything more than recompiling the kernel. Years ago,
...
> SunOS 4 users can probably be mostly satisfied with gcc.

Barry,
This is good enough for me. I'd certainly rather avoid K&R C!


-Fred

--
Fred L. Drake, Jr. <fdrake@acm.org>
Corporation for National Research Initiatives
Re: Notice of intent: rich comparisons [ In reply to ]
Barry> SunOS 4 users can probably be mostly satisfied with gcc.

Fred> This is good enough for me. I'd certainly rather avoid K&R C!

Besides, aren't there tools available that can strip most of the ANSI-ness
out of ANSI C? (Un)Protoize? For those (few?) people who are really still
stuck using a K&R compiler I would think those tools, perhaps supplemented
with some patch files maintained outside the main Python source tree, would
be sufficient.

Skip Montanaro | Mojam: "Uniting the World of Music" http://www.mojam.com/
skip@mojam.com | Musi-Cal: http://www.musi-cal.com/
518-372-5583
Re: Notice of intent: rich comparisons [ In reply to ]
>>>>> "skip" == <skip@mojam.com> writes:

skip> Besides, aren't there tools available that can strip most of
skip> the ANSI-ness out of ANSI C? (Un)Protoize?

I think so. Seems I have unprotoize already; probably got it for free
with gcc, or some other GNU tool.

-B
Re: Notice of intent: rich comparisons [ In reply to ]
mal> There's some good work out there by Skip Montanaro to revamp the VM
mal> into a combined register/stack machine. He calls it rattlesnake.
mal> More infos can be found in the list archive (it's currently
mal> sleeping):

Thanks to Marc-Andre for mentioning this. Rattlesnake is indeed sleeping at
the moment. If anyone is interested in the idea I'd be happy to awaken it.

The main observation was that much of the virtual machine's activity is
pushing/popping objects to/from the stack. By treating a chunk of storage
as a register file much of that churn can be avoided.

Skip Montanaro | Mojam: "Uniting the World of Music" http://www.mojam.com/
skip@mojam.com | Musi-Cal: http://www.musi-cal.com/
518-372-5583
Re: Notice of intent: rich comparisons [ In reply to ]
Guido van Rossum wrote:
>...
> Import revamp
>
> It should be much easier to add hooks to allow importing
> from zip files, from a URL, etc., or (different) hooks to
> allow looking for different extensions (e.g. to automatically
> generate ILU stubs). This should help the small platforms
> too, since they often don't have a filesystem, so their
> imports will have to come from some other place.

I would recommend that Python's import mechanism be changed to use some
of the concepts from my imputil.py module and in a couple of my posts to
distutils-sig:

http://www.python.org/pipermail/distutils-sig/1999-January/000142.html
http://www.python.org/pipermail/distutils-sig/1998-December/000077.html

> ANSI C
>
> I'm tired of supporting K&R C. All code should be converted
> to using prototypes. The Py_PROTO macros should go. We can
> add const-correctness to all code. Too bad for platforms
> without decent compilers (let them get GCC).

Or those platforms can stick with Python 1.5.2

> Misc
>
> Jeremy Hylton's urllib2.py
>
> Greg Stein's new httplib.py (or Jeremy's?)

Mine is a drop-in replacement for the current httplib.py. httplib.HTTP
is compatible with the current version and does HTTP/1.0.
httplib.HTTPConnection is similar to the other (but (IMO) cleans up a
few elements of the API); it does HTTP/1.1, including persistent
connections. This module follows the design of the original httplib.py
much more closely. Jeremy's module uses a very different design approach
(no value judgement here; it's just different); also, it simply copies
over the old HTTP class, while mine derives the compatible version from
HTTPConnection. I'm not sure how much testing Jeremy's has had, but I
use my module for all Python-based http work now. Some number of other
people have downloaded and used it, too.

I would recommend my new module, given the two options.

I'll eventually have a davlib.py to add to the distribution, but it
needs more work. I'll contribute that later.

---

I'd also like to see some cleanup of the build process. For example,
(Guido:) I sent in a different way to handle the signal stuff (shifting
files around and stuff) as opposed to the "hassignal" thing.

About the build process: it may be nice to move to a single makefile.
Here is an interesting paper about this:

http://www.canb.auug.org.au/~millerp/rmch/recu-make-cons-harm.html

Is it reasonable to split out the Demo, Doc, Misc, and Tools subdirs
since those never really end up as part of an installed Python? e.g.
into a python-extras distribution? (which would allow doc and tools
updates at arbitrary times)

Does it make sense to remove platform-specific modules? (e.g. the SGI
modules)

While distutils doesn't have anything *yet*, there are changes that can
be made in the build process to improve the distutils process. I know
that Greg Ward wants distutils to work on previous versions of Python,
but (IMO) things would be simpler all around if distutils-based packages
were simply designed for 1.6 systems as a basic dependency. However,
that choice is quite separate from the decision to generate distutils
support during the build.

Oh: there were some things floating around at one point about using
Python itself within the build process (e.g. for modules). Is any of
that work useful?

Cheers,
-g

--
Greg Stein, http://www.lyra.org/
Re: Notice of intent: rich comparisons [ In reply to ]
Greg Stein writes:
> Is it reasonable to split out the Demo, Doc, Misc, and Tools subdirs
> since those never really end up as part of an installed Python? e.g.
> into a python-extras distribution? (which would allow doc and tools
> updates at arbitrary times)

The Doc/ stuff is already in a separate distribution; this has been
in place for about a year (if I recall correctly). I try to make doc
releases shortly after Python releases (I think the 1.5.2 doc release
is the longest I've waited past the Python release so far, and it may
end up being next week), and I have made interim releases as the
documentation has grown.
I certainly think that doing something similar for the Demo/ and
Tools/ directories would be reasonable; they could share a single
distribution.


-Fred

--
Fred L. Drake, Jr. <fdrake@acm.org>
Corporation for National Research Initiatives
Re: Notice of intent: rich comparisons [ In reply to ]
Fred> I certainly think that doing something similar for the Demo/ and
Fred> Tools/ directories would be reasonable; they could share a single
Fred> distribution.

We need to make sure none of the tools are needed in the install
process. h2py comes to mind.

Skip Montanaro | Mojam: "Uniting the World of Music" http://www.mojam.com/
skip@mojam.com | Musi-Cal: http://www.musi-cal.com/
518-372-5583
Re: Notice of intent: rich comparisons [ In reply to ]
>>>>> "GS" == Greg Stein <gstein@lyra.org> writes:

>> Greg Stein's new httplib.py (or Jeremy's?)

GS> Mine is a drop-in replacement for the current
GS> httplib.py. httplib.HTTP is compatible with the current version
GS> and does HTTP/1.0. httplib.HTTPConnection is similar to the
GS> other (but (IMO) cleans up a few elements of the API); it does
GS> HTTP/1.1, including persistent connections. This module follows
GS> the design of the original httplib.py much more
GS> closely. Jeremy's module uses a very different design approach
GS> (no value judgement here; it's just different); also, it simply
GS> copies over the old HTTP class, while mine derives the
GS> compatible version from HTTPConnection. I'm not sure how much
GS> testing Jeremy's has had, but I use my module for all
GS> Python-based http work now. Some number of other people have
GS> downloaded and used it, too.

GS> I would recommend my new module, given the two options.

I think I agree. I stopped working on my version quite some time ago,
before I made enough progress to make it useful. Thus, it makes a lot
of sense to use Greg's working code.

I'd be curious to see how to use HTTPConnection to do persistent
connections and pipelined requests. It wasn't entirely clear from the
code what I would need to do to keep the connection open. Do you just
re-use the HTTPConnection object -- call putrequest again after
getreply?

I like the revised API that Greg's code uses. The request method on
HTTPConnection objects cleans the API up in exactly the right way. It
has always seemed unnecessary to have many separate calls to add
headers and then a call to endheaders after putrequest. Packaging
them all together in one method will simplify client code a lot.

GS> I'll eventually have a davlib.py to add to the distribution, but
GS> it needs more work. I'll contribute that later.

Cool!

Jeremy