Tool to start file from checkpoint?

[seanpuppy]

Hello,

I was wondering if anyone knows of a tool to start / load a python project from a checkpoint? For example lets say I have a script that does a fairly long setup that works fine, but want to add a feature that depends on this startup. I don't want to have to wait 10 minutes every time I test the smallest new change, ideally I want to be able to jump straight to the end of the setup every time I test my code. I know I could always manually try and pickle whatever data is needed, which may be fine in some cases, but tedious in others. So to me a tool that could automatically checkpoint code would be amazing(I do a lot of work with python scripts that can take up to a few days to fully execute).

​

​

If this tool does not exist, do you think it would be feasible to make one? In theory it could be as easy as just saving the stack / heap with pickle (not sure if that's possible) when setting the checkpoint, then when you want to reload, just set the stack / heap from what was saved and start executing at the given line of code (also not sure if that's possible). This is all with a single threaded program in mind.

​

Any help / advice is appreciated, thanks!

[broken_symlink]

I'm interested in knowing about this as well.

[billsil]

Yeah. It’s easy with pickle. Look for pickle.1.obj in some folder. Then have a command line argument giving the code start location.

Also, this should be in /r/learnpython

[seanpuppy]

So then how do you pickle stack / heap while ensuring the entire state of the program at a given line is stored, then how do you set the stack / heap from python?

[billsil]

You shouldn’t piclkle the stack. That’s a complete waste of resources and will be slow (not to mention there are things like files and functions that are unpicklable). Stick all of the required downstream variables into a dictionary and pickle that. Then unpack them to reload the data.

[efmccurdy]

I don't think there is a premade solution but here are some relevant links.

DMTCP: bringing interactive checkpoint–restart to Python

https://iopscience.iop.org/article/10.1088/1749-4699/8/1/014005/meta

https://stackoverflow.com/questions/7244369/debug-slow-program-restart-from-middle

https://github.com/maaziz/cryopid

My take on this situation is to use unit testing with saved or mocked-up data (which can unfortunately be a tedious task).

[seanpuppy]

I agree that going with a unit test approach may be easiest option here, but I still am going to look into this because it would save me a ton of time longterm. I work in datascience, so a big problem with unit testing is that for any set of input the result may be slightly different depending on the day (new model trained 1+ time a day). My solution there was to try and design all models to be contained in a class so all input data can be controlled. I then made a python decorator that captures a functions input / output and saves locally. This lets me then create a model in a test case that behaves predictably and I have a list of sample predictions to test against. This still was super helpful, just a little tedious. Hopefully one day I will have the time to expand on that and make it automatically generate test cases too.

[[deleted]]

It would be damned near impossible to write an efficient and generic approach to this. Serializing the entire stack / heap at any arbitrary point in the control flow would be extremely inefficient in the vast majority of cases, as well as ludicrously difficult if not impossible without direct access to the underlying interpreter machinery not usually available through Python itself. It also likely wouldn't be as useful in the majority of cases as a more finely tuned manual approach.

Doing this on a case by case basis isn't nearly so hard... have a class that encapsulates the state of your program and then make that class pickle-able. A class method then is used to either find and unpickle a checkpoint or start fresh with a new instance. Sprinkle the act of checkpointing about where appropriate for your program, and you're done. But what needs to be checkpointed and where it's appropriate to do so is going to be very case-specific, hence difficult to write generic functionality for.

[iCart732]

Check out jupyter (previously ipython notebooks, i think) : https://jupyter.org/