Mailing List Archive: Bug in struct module?

Bug in struct module?

Aug 16, 1999, 1:33 PM

Post #1 of 5 (429 views)

This is a multi-part message in MIME format.
--------------25E540AB75BDF1FC1303CA24
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Hi,

>>> calcsize('hd')
16
>>> calcsize('dh')
10
That seems like a bug to me. BTW: it won't convert
>>> unpack('dh', s[-10:])
(2.47032822921e-323, 16424)
>>> unpack('hd', s[-10:])
Traceback (innermost last):
File "<stdin>", line 1, in ?
struct.error: unpack str size does not match format

Does anyone know how to unpack a bin file written in VB:
Fno = FreeFile
Open CDir & Fn & "." & cdte For Binary As #Fno
Put #Fno, , xMat

where xMat is same as x created by?

Dim x(4, 3) As Double
'This code example opens the file in Binary mode for reading; other
processes can't read file.
For r = 0 To UBound(x, 1)
For c = 0 To UBound(x, 2)
x(r, c) = r * c
Next c
Next r

--
*****************************************************************************
S. Hoon Yoon (Quant) Merrill Lynch Equity Trading,
yelled@yahoo.com hoon@bigfoot.com(w)
"Miracle is always only few standard deviations away, but so is
catastrophe."
* Expressed opinions are often my own, but NOT my employer's.
"I feel like a fugitive from the law of averages." Mauldin
*****************************************************************************
--------------25E540AB75BDF1FC1303CA24
Content-Type: text/x-vcard; charset=us-ascii; name="vcard.vcf"
Content-Transfer-Encoding: 7bit
Content-Description: Card for Hoon Yoon
Content-Disposition: attachment; filename="vcard.vcf"

begin: vcard
fn: Hoon Yoon
n: ;Hoon Yoon
email;internet: hyoon@bigfoot.com
x-mozilla-cpt: ;0
x-mozilla-html: FALSE
version: 2.1
end: vcard

--------------25E540AB75BDF1FC1303CA24--

Bug in struct module? [ In reply to ]

skip at mojam

Aug 16, 1999, 2:39 PM

Post #2 of 5 (429 views)

Permalink

>>>> calcsize('hd')
>> 16
>>>> calcsize('dh')
>> 10
>> That seems like a bug to me.

Only if the native structs in C would be represented differently, and even
perhaps not then. On my Linux box, the following short C program displays

short size: 2
double size: 8
foo size: 12
bar size: 12

when run.

main() {
struct {
double f;
short s;
} foo;

struct {
short s;
double f;
} bar;

printf("short size: %d\n", sizeof(short));
printf("double size: %d\n", sizeof(double));
printf("foo size: %d\n", sizeof(foo));
printf("bar size: %d\n", sizeof(bar));
}

It's possible that your compiler forces doubles onto eight-byte boundaries
but will truncate shorter fields at the end of structs. On my machine it
appears to to pad everything to four-byte boundaries. Compile and run the
program and see what you get. Also, note that the struct module tries to
mimic the behavior of the C compiler. From the struct module documentation.

This module performs conversions between Python values and C structs
represented as Python strings. ....

By default, C numbers are represented in the machine's native format and
byte order, and properly aligned by skipping pad bytes if necessary
(according to the rules used by the C compiler).

That may well not be how VB does it, so you can't just assume that the
spacing the struct module gives you is how the VB object will be layed out.

Skip Montanaro | http://www.mojam.com/
skip@mojam.com | http://www.musi-cal.com/~skip/
847-971-7098

Bug in struct module? [ In reply to ]

tim_one at email

Aug 16, 1999, 9:32 PM

Post #3 of 5 (426 views)

Permalink

[Hoon Yoon]
> >>> calcsize('hd')
> 16
> >>> calcsize('dh')
> 10
> That seems like a bug to me.

Assuming you're running on Win9x/NT, it's functioning as designed, although
the actual size of a double-followed-by-short struct under MSVC's default
rules is 16 instead of 10. To get the actual total size of a struct under
MSVC's defaults, you need to append "0{x}" to the format, where {x} is the
typecode of the largest of the preceding typecodes:

Python 1.5.2 (#0, Apr 13 1999, 10:51:12) [MSC 32 bit (Intel)] on win32
Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam
IDLE 0.5 -- press F1 for help
>>> import struct
>>> struct.calcsize("hd0d")
16
>>> struct.calcsize("dh0d")
16
>>>

If you'd rather get 10 for each result, use the "native byte order, standard
alignment" format code:

>>> struct.calcsize("=dh")
10
>>> struct.calcsize("=hd")
10
>>>

See the docs for details on that. If you're not using native C structs, the
"standard alignment" rules are much easier to live with (they never
introduce hidden pad bytes, while native alignment may).

but-first-figure-out-how-the-data-is-layed-out-ly y'rs - tim

Bug in struct module? [ In reply to ]

hyoon at bigfoot

Aug 17, 1999, 10:42 AM

Post #4 of 5 (426 views)

Permalink

This is a multi-part message in MIME format.
--------------9DCE25D9CC09F2C078B5E7F4
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Everyone,

Thanks that solves it. I am just beginning to dive into the binary
files. And platform was NT, I will remember to put more details in the
future (sorry).

Hoon,

--
*****************************************************************************
S. Hoon Yoon (Quant) Merrill Lynch Equity Trading,
yelled@yahoo.com hoon@bigfoot.com(w)
"Miracle is always only few standard deviations away, but so is
catastrophe."
* Expressed opinions are often my own, but NOT my employer's.
"I feel like a fugitive from the law of averages." Mauldin
*****************************************************************************
--------------9DCE25D9CC09F2C078B5E7F4
Content-Type: text/x-vcard; charset=us-ascii; name="vcard.vcf"
Content-Transfer-Encoding: 7bit
Content-Description: Card for Hoon Yoon
Content-Disposition: attachment; filename="vcard.vcf"

begin: vcard
fn: Hoon Yoon
n: ;Hoon Yoon
email;internet: hyoon@bigfoot.com
x-mozilla-cpt: ;0
x-mozilla-html: FALSE
version: 2.1
end: vcard

--------------9DCE25D9CC09F2C078B5E7F4--

Bug in struct module? [ In reply to ]

drew_csillag at geocities

Aug 17, 1999, 11:01 AM

Post #5 of 5 (430 views)

Permalink

Hoon Yoon wrote:
>
> Hi,
>
> >>> calcsize('hd')
> 16
> >>> calcsize('dh')
> 10
> That seems like a bug to me.

You forgot about structure alignment. If the machine is 1 byte-aligned,
the sizes would be the same. From the looks of things, your machine
aligns on 8 byte boundaries (Sun machine?).

> BTW: it won't convert
> >>> unpack('dh', s[-10:])
> (2.47032822921e-323, 16424)
> >>> unpack('hd', s[-10:])
> Traceback (innermost last):
> File "<stdin>", line 1, in ?
> struct.error: unpack str size does not match format

It shouldn't. Since the above sizes are what they are, you'd need a
string of 16 bytes for the second call to unpack.

If you want these to work, try the network byte order/alignment option
(prefix your format string with a '>'). On my sun box, both sizes
become 10 and the unpack should also work.
--
"Programmers are just machines that convert coffee to executable code"
-Unknown

"Drew Csillag" <drew_csillag@geocities.com>