Quantcast

write binary with struct.pack_into

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

write binary with struct.pack_into

palmeira
This post has NOT been accepted by the mailing list yet.
Dear pythonists,

I'm having a problem with read/write binary in python.
I have a binary file that I need to read information, extract a array, modify this array and put these values into file again in same binary format.
I need to use unpack_from and pack_into because sometimes gonna need read/write in the middle of file.

Script:

import struct
bloco='>%df' %(252)  #Binary format

# READ
fa=open('testIN.bin')
my_array=struct.unpack_from(bloco,fa.read()[0*4:251*4])    # my_aray = 252 elements array
## This read is OK!

#WRITE
fb=open('testOUT.bin')
test=struct.pack_into(bloco,fb.write()[0*4:251*4])  # ERROR in this WRITE




Regards,

Ronaldo Palmeira.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

write binary with struct.pack_into

Chris Angelico
On Sat, Oct 6, 2012 at 1:27 PM, palmeira <palmeira at gmail.com> wrote:

> import struct
> bloco='>%df' %(252)  #Binary format
>
> # READ
> fa=open('testIN.bin')
> my_array=struct.unpack_from(bloco,fa.read()[0*4:251*4])    # my_aray = 252
> elements array
> ## This read is OK!
>
> #WRITE
> fb=open('testOUT.bin')
> test=struct.pack_into(bloco,fb.write()[0*4:251*4])  # ERROR in this WRITE

You have a beautiful parallel here, but I think that's where your
problem is. In the READ section, you have fa.read() which will read
the whole file, and then you slice the resulting string. That's pretty
inefficient for large files, but it'll work.

But when you write, that completely does not work. (Even assuming
you've opened read/write, per Dennis's comment.)
>>> fb.write()
Traceback (most recent call last):
  File "<pyshell#37>", line 1, in <module>
    fb.write()
TypeError: write() takes exactly 1 argument (0 given)

It needs an argument of what to write out, it doesn't return a
writable buffer suitable for pack_into.

I recommend you completely rewrite your file handling to use actual
seeks and file writes. Also, you'll want (I think) to use binary mode
on your files; character encodings don't mean much when you're working
with arrays of numbers.

Finally, when you post issues, "ERROR in this WRITE" isn't very
helpful. Please please post the full traceback (like in my trivial
example above), as problems will be much more visible.

Hope that's of some value!

ChrisA


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

write binary with struct.pack_into

Peter Otten
In reply to this post by palmeira
palmeira wrote:

> Dear pythonists,
>
> I'm having a problem with read/write binary in python.
> I have a binary file that I need to read information, extract a array,
> modify this array and put these values into file again in same binary
> format.
> I need to use unpack_from and pack_into because sometimes gonna need
> read/write in the middle of file.

Use pack/unpack and file.seek() instead.
 

> Script:
>
> import struct
> bloco='>%df' %(252)  #Binary format
>
> # READ
> fa=open('testIN.bin')
> my_array=struct.unpack_from(bloco,fa.read()[0*4:251*4])    # my_aray = 252
> elements array
> ## This read is OK!
>
> #WRITE
> fb=open('testOUT.bin')
> test=struct.pack_into(bloco,fb.write()[0*4:251*4])  # ERROR in this WRITE

However, I think you have picked the wrong API. So:

# untested

import sys
import array

offset = 0
N = 252

a = array.array("f")
with open("testIN.bin", "rb") as f:
    f.seek(offset)
    a.read(f, N)
if sys.byteorder == "little":
    a.byteswap()

# process a

if sys.byteorder == "little":
    a.byteswap()
with open("testOUT.bin", "wb") as f:
    f.seek(offset)
    a.write(f)




Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

write binary with struct.pack_into

Grant Edwards-7
In reply to this post by palmeira
On 2012-10-06, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
> On Fri, 5 Oct 2012 20:27:36 -0700 (PDT), palmeira <palmeira at gmail.com>
> declaimed the following in gmane.comp.python.general:
>
>>
>> #WRITE
>> fb=open('testOUT.bin')
>
> Unless you specify otherwise, open() defaults to read-only

It also defaults to 'text' mode which does cr/lf translaction.  That
will break both reads and writes on any binary file containing 0x0a
and 0x0d bytes.

--
Grant


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

write binary with struct.pack_into

Chris Angelico
On Sat, Oct 6, 2012 at 11:26 PM, Grant Edwards <invalid at invalid.invalid> wrote:

> On 2012-10-06, Dennis Lee Bieber <wlfraed at ix.netcom.com> wrote:
>> On Fri, 5 Oct 2012 20:27:36 -0700 (PDT), palmeira <palmeira at gmail.com>
>> declaimed the following in gmane.comp.python.general:
>>
>>>
>>> #WRITE
>>> fb=open('testOUT.bin')
>>
>>       Unless you specify otherwise, open() defaults to read-only
>
> It also defaults to 'text' mode which does cr/lf translaction.  That
> will break both reads and writes on any binary file containing 0x0a
> and 0x0d bytes.

And, in Python 3, it'll be returning Unicode characters too, unless
the decode fails. You definitely want binary mode.

ChrisA


Loading...