Mailing List Archive

On 2 Oct 2023, at 16:02, Jen Kris via Python-list
<python-list@python.org> wrote:

Iwant to write a list of 64-bit integers to a binary file.  Everyexample
I have seen in my research convertsit to .txt, but I want it in binary.
 I wrote this code,based on some earlier work I have done:
buf= bytes((len(qs_array)) * 8)

buf is not writable so cannot be used by pack_into. I think you need to
use bytesarray not bytes.

foroffset in range(len(qs_array)):
item_to_write= bytes(qs_array[offset])
struct.pack_into(buf,"<Q", offset, item_to_write)

You have the parameters in the wrong order.

struct.pack_into(format, buffer, offset, v1, v2, ...)[1]¶

Pack the values v1, v2, … according to the format string format and
write the packed bytes into the writable buffer buffer starting at
position offset. Note that offset is a required argument.

ButI get the error "struct.error: embedded null character."

The nul is because you past buf of 0’s as the format.

Maybethere's a better way to do this?

Anyhelp will be very appreciated.
Thanks. 

Barry

--
https://mail.python.org/mailman/listinfo/python-list

References

Visible links
1. Permalink to this definition
https://docs.python.org/3/library/struct.html#struct.pack_into
--
https://mail.python.org/mailman/listinfo/python-list

On 2023-10-01 23:04, Jen Kris via Python-list wrote:
>
> Iwant to write a list of 64-bit integers to a binary file.
Everyexample I have seen in my research convertsit to .txt, but I want
it in binary. I wrote this code,based on some earlier work I have done:
>
> buf= bytes((len(qs_array)) * 8)
>
> foroffset in range(len(qs_array)):
>
> item_to_write= bytes(qs_array[offset])
>
> struct.pack_into(buf,"<Q", offset, item_to_write)
>
> ButI get the error "struct.error: embedded null character."
>
> Maybethere's a better way to do this?
>
You can't pack into a 'bytes' object because it's immutable.

The simplest solution I can think of is:

buf = struct.pack("<%sQ" % len(qs_array), *qs_array)
--
https://mail.python.org/mailman/listinfo/python-list

Thanks very much, MRAB.  I just tried that and it works.  What frustrated me is that every research example I found writes integers as strings.  That works -- sort of -- but it requires re-casting each string to integer when reading the file.  If I'm doing binary work I don't want the extra overhead, and it's more difficult yet if I'm using the Python integer output in a C program.  Your solution solves those problems. 



Oct 2, 2023, 17:11 by python-list@python.org:

> On 2023-10-01 23:04, Jen Kris via Python-list wrote:
>
>>
>> Iwant to write a list of 64-bit integers to a binary file. Everyexample I have seen in my research convertsit to .txt, but I want it in binary. I wrote this code,based on some earlier work I have done:
>>
>> buf= bytes((len(qs_array)) * 8)
>>
>> foroffset in range(len(qs_array)):
>>
>> item_to_write= bytes(qs_array[offset])
>>
>> struct.pack_into(buf,"<Q", offset, item_to_write)
>>
>> ButI get the error "struct.error: embedded null character."
>>
>> Maybethere's a better way to do this?
>>
> You can't pack into a 'bytes' object because it's immutable.
>
> The simplest solution I can think of is:
>
> buf = struct.pack("<%sQ" % len(qs_array), *qs_array)
> --
> https://mail.python.org/mailman/listinfo/python-list
>

--
https://mail.python.org/mailman/listinfo/python-list

Jen Kris wrote at 2023-10-2 00:04 +0200:
>Iwant to write a list of 64-bit integers to a binary file. Everyexample I have seen in my research convertsit to .txt, but I want it in binary. I wrote this code,based on some earlier work I have done:
>
>buf= bytes((len(qs_array)) * 8)
>
>for offset in range(len(qs_array)):
> item_to_write= bytes(qs_array[offset])
> struct.pack_into(buf,"<Q", offset, item_to_write)
>
>But I get the error "struct.error: embedded null character."

You made a lot of errors:

* the signature of `struct.pack_into` is
`(format, buffer, offset, v1, v2, ...)`.
Especially: `format` is the first, `buffer` the second argument

* In your code, `offset` is `0`, `1`, `2`, ...
but it should be `0 *8`, `1 * 8`, `2 * 8`, ...

* The `vi` should be something which fits with the format:
integers in your case. But you pass bytes.

Try `struct.pack_into("<Q", buf, 0, *qs_array)`
instead of your loop.


Next time: carefully read the documentation and think carefully
about the types involved.

--
https://mail.python.org/mailman/listinfo/python-list

Dieter, thanks for your comment that:

* In your code, `offset` is `0`, `1`, `2`, ...
but it should be `0 *8`, `1 * 8`, `2 * 8`, ...

But you concluded with essentially the same solution proposed by MRAB, so that would obviate the need to write item by item because it writes the whole buffer at once. 

Thanks for your help. 


Oct 2, 2023, 17:47 by dieter@handshake.de:

> Jen Kris wrote at 2023-10-2 00:04 +0200:
> >Iwant to write a list of 64-bit integers to a binary file. Everyexample I have seen in my research convertsit to .txt, but I want it in binary. I wrote this code,based on some earlier work I have done:
>
>>
>>
> >buf= bytes((len(qs_array)) * 8)
>
>>
>>
> >for offset in range(len(qs_array)):
>
>> item_to_write= bytes(qs_array[offset])
>> struct.pack_into(buf,"<Q", offset, item_to_write)
>>
> >But I get the error "struct.error: embedded null character."
>
> You made a lot of errors:
>
> * the signature of `struct.pack_into` is
> `(format, buffer, offset, v1, v2, ...)`.
> Especially: `format` is the first, `buffer` the second argument
>
> * In your code, `offset` is `0`, `1`, `2`, ...
> but it should be `0 *8`, `1 * 8`, `2 * 8`, ...
>
> * The `vi` should be something which fits with the format:
> integers in your case. But you pass bytes.
>
> Try `struct.pack_into("<Q", buf, 0, *qs_array)`
> instead of your loop.
>
>
> Next time: carefully read the documentation and think carefully
> about the types involved.
>

--
https://mail.python.org/mailman/listinfo/python-list