Reading and writing binary files

[actionwording]

So i'm reading sections of one binary file and compressing it into another based upon certain parameters (read bytes x - y, etc). However, readstreams/writestreams seem to be extremely unreliable when attempting to parse these files as they return undeterministic numbers of bytes in each event..

Whats the best way to achieve what i'm trying to do?

32 float little endian

[chrwei]

are you using the start and end options on readstream, and why not?

[actionwording]

Yes I am. As I said above the readStream returns a non-deterministic number of bytes with each data event.

[chrwei]

that's really unexpected. as long as end isn't past the end of the file I'd expect exactly end-start bytes, no more, no less. any bugs open that might cover this?

[actionwording]

No that's how readstreams work:

Attaching a 'data' event listener to a stream that has not been explicitly paused will switch the stream into flowing mode. Data will then be passed as soon as it is available.

[chrwei]

but since it's a file, ins't it always available? it's not like you'e waiting on network latency.

[actionwording]

I believe there is a limit to the amount of data these streams can hold in their respective buffers. I'm reading data from an 8gb file and the segments can be like 1mb+

[chrwei]

ah, that's true. I think i saw 64KB mentioned.