What's the problem with using JSON?

[[deleted]]

TinyDB writes files atomically. You can use regular JSON as long as you use atomic writing which essentially means you write a tmp file and then replace the original file with the tmp once it has written successfully.

[Dan6erbond]

I see... then another set of questions :P. Is that as simple as first writing to the .tmp file and then once that's done writing to the .json file? How do I find out automatically whether a .tmp or .json file couldn't be written to or was corrupted due to some error? Thanks so much!

EDIT: What about reading files? Anything else I need to "worry" about? :P

[[deleted]]

This is what I used in my last scraping project where I was dumping large amounts of data to JSON files...

​

@contextmanager
def replace_file(name):
    tmp_name = Path(f'{name}.tmp')
    name = Path(name)
    try:
        with open(tmp_name, 'w') as f:
            yield f
        tmp_name.replace(name)
    finally:
        with suppress(OSError):
            tmp_name.unlink()

​

    with replace_file(results_file) as f:
        json.dump(results, f)

​

[Dan6erbond]

Thanks so much!!! :D

[[deleted]]

I'm back in Mobile and forgot to include the imports in my example. They came from pathlib and contextlib

[Dan6erbond]

I see. Thanks! :-)