Mailing List Archive: Proper application of the buffer interface

Proper application of the buffer interface

Aug 4, 1999, 7:30 PM

Post #1 of 4 (314 views)

There has been a bit of discussion lately regarding the buffer
interface to objects in the C API. Some of this has been on the main
list (comp.lang.python) and some has been on other Python-related
lists.
I have a question that I don't recall seeing answered (or asked):
What is the real intent of the buffer interface? When should a Python
C type implement it?
The context is this: the t1python package (an interface to a Type1
font renderer) has, in the past, been passing bitmaps back to Python
as string objects. In the next release, I may create a new type in
the C layer that provides all the interesting stuff, and would like to
be able to convert these "glyph" objects to PIL images and GTK+/GNOME
compatible images. Does it make the most sense for the glyph objects
to offer the buffer interface to make more conversions possible
without increasing the number of memory copies, or do I misunderstand
the application of the interface?
I'd appreciate some input on this matter. I hope to get the next
release of t1python done before too much longer, and this is probably
the biggest remaining question that I need to deal with.
(Any other feedback on t1python would be useful as well!)
Thanks!

-Fred

--
Fred L. Drake, Jr. <fdrake@acm.org>
Corporation for National Research Initiatives

Proper application of the buffer interface [ In reply to ]

skip at mojam

Aug 6, 1999, 10:07 AM

Post #2 of 4 (314 views)

Permalink

Sorry to be a numbskull, but since there is apparently little or no
documentation for the buffer interface yet, can someone give me a thumbnail
sketch of why we have it and what it's used for? I can guess that there are
some situations where using string objects as raw buffers could lead to
misinterpretation of their contents, especially since not everything is
US-ASCII anymore. Beyond that, I'm not sure why there would be a
distinction between buffers and strings.

Thx,

Skip Montanaro | http://www.mojam.com/
skip@mojam.com | http://www.musi-cal.com/~skip/
847-971-7098

Proper application of the buffer interface [ In reply to ]

da at ski

Aug 6, 1999, 10:13 AM

Post #3 of 4 (316 views)

Permalink

On Fri, 6 Aug 1999, Skip Montanaro wrote:

> Sorry to be a numbskull, but since there is apparently little or no
> documentation for the buffer interface yet, can someone give me a thumbnail
> sketch of why we have it and what it's used for? I can guess that there are
> some situations where using string objects as raw buffers could lead to
> misinterpretation of their contents, especially since not everything is
> US-ASCII anymore. Beyond that, I'm not sure why there would be a
> distinction between buffers and strings.

I know of three (there may be more):

1) read/write, not just read (I'm using them to pass data to win32 calls
which modify them, w/o new copies, etc.).

2) segmented architecture (non-contiguous strips of memory)

3) manipulating data which doesn't start its life as a PyString -- e.g. an
mmap'ed file can expose a buffer interface, an Image can expose a
buffer interface, a NumPy array can expose a buffer interface -- one
can manipulate the data in those as buffers without having to copy the
data.

--david ascher

Proper application of the buffer interface [ In reply to ]

MHammond at skippinet

Aug 12, 1999, 6:56 AM

Post #4 of 4 (316 views)

Permalink

About a week ago, I wrote in message
<006501bedeee$33ef5c20$1101a8c0@bobcat>...

>Just to extend my guesswork somewhat, there is a new built-in "buffer()"
>function. This returns a "buffer" object. I speculate this should be used
>in preference to Python strings when you have binary data. As this buffer
>object supports the buffer interfaces, they are basically as functional as
>strings for this purpose, but clearly indicate the data is not really a
>string! This appears to be more a matter of style, and also paves the road
>to Unicode - eg, it makes sense to convert any Python string object to
>Unicode, but not necessarily a binary buffer.

Just to prove it was guess-work, Bill Tutt indirectly pointed me at my error
here.

I may have given the impression that the buffer object returns a completely
new object - similar to a string. The reason I may have given that
impression is because it is exactly what I meant. However, it is wrong (and
I did know it was wrong <sigh>)

The "buffer()" built-in returns a read-only buffer for an _existing_ object.
It would only be an effective replacement for a string as a "byte buffer"
when coupled with an array object.

>>> import array
>>> a=array.array("b", range(ord('A'), ord('G')))
>>> b=buffer(a)
>>> b
<read-only buffer for fdcca0, ptr fdcc90, size 6 at fdccd0>
>>> b[:-1]
'ABCDE'
>>>

Just thought it worth a clarification.

As a side note, this also makes it far more difficult for extensions to
replace the use of strings with a different object for "byte buffers" - it
appears PyArray_New does not exist, so extension module writers are forced
to implement their own object if they wish to use the buffer interface, but
not use a PyString.

Mark.