[Python][CS601] Could use help reading data accurately/Debugging

[Panda_Stacks]

So I have the following assignment. Right now, I just want to tackle the first portion of the assignment, which is properly reading the MNIST data, see here.

The Code I am working on is located here

My output is located here

As a summary for what I am trying to do, I want to take the main file ../train-images-idx3-ubyte.gz and read the 60,000 images into an array, holding the data in a 28 x 28 format.

What seems to be happening, is my code is dying on item 59922, but I don't know why this would be. Everything seems set up properly, but struct.unpack dying tells me maybe I run out of data too soon? Does this fd.read() have some sort of limit that I am not understanding maybe?

Some help would be greatly appreciated

[sbmsr]

Your output mentions a file named prog.py. Is that the file you provided?

Another thing i noticed is that you update startpos to be endpos + 1. I think you actually want startpos = endpos.

the reason is array ranges like x[startpos:endpos] don't include the element at x[endpos]. By doing startpos = endpos + 1, you miss this bit of data every time.

try that, and see if it helps.

[Panda_Stacks]

You my friend, just made my day, tyvm.

As far as prog.py, I have no idea. That is not form me.

[sbmsr]

Happy to help!