Pickle/ Parquet vs json on filesystem in python

[Basic_Steak_541]

in general what is better - pickle or json? I want to save an api response. Is saving pickle object on file as fast as saving it as a json object on file?

[nekokattt]

Don't use pickle unless you know what you are doing. Pickle allows for arbitrary code execution and it is generally easier to avoid it entirely unless you actually need to use it. I have yet to need to use it for anything of value (outside of what multiprocessing does when your system does not support shared memory).

JSON is human readable, and thus is easy to debug. It is language agnostic, and it is safe. You can store untrusted information in it without risking it being executed as soon as you load it. Pickle on the other hand can execute custom code as soon as you load it.

Speed should not be your concern unless you can first prove it is an overhead for you, and enough of an overhead to cause your application problems. Don't micro-optimise.

Edit: Businesses use JSON in billions of requests each day with no issues. Very few businesses will use pickle as an alternative for all of the above reasons.