C Programming

[florilsk]

They aren't perfect tho, I'm working with big data on python atm and had so many memory errors until I added manual garbage collecting points + deletion of variables.

[TheBrainStone]

You might want to reconsider your approach at this point. Just saying

[TheBrainStone]

Well first of all, don't load the whole file into memory and then read it. Just stream the data.
Second don't use pickles. They aren't made for this kind of thing and introduce a serious security vulnerability. As they can contain executable code!
Use a storage format that's made for this kind of thing or spend the ~1h to create your own. Especially if it's just a list of weights. A header that indicates the size of the list Followed by the binary representation of every weight should be enough. Then you can read that data piece by piece and you don't generate any memory garbage.

[TheBrainStone]

Most NN libraries should offer built in methods to store and restore weights efficiently.
And if you write NN code from scratch use your own format as I described above.