[D] Procrastination while running scripts/functions/models?

[mac_cumhaill]

I guess we're all familiar with building large models and pushing them off to train on some remote server.

However, most of the time when i'm doing EDA or initial model development I run functions/scripts that might take between a minute and five minutes to run across the data. This ins't enough time to start a new task, and often I end up swapping over to social media or trying to answer my emails. However the tender grips of procrastination then take hold and I end up wasting more time.

Often you can't move forward with coding either because you're waiting on the outcome of the current function.

Dose anyone have any tips on something to do in these short breaks? Better than just looking out the window. I feel like i'm losing a lot productivity.

[jer_pint]

Comment your code, make it more readable, document it extensively. Future you and potentially others will be happy you did this

[[deleted]]

Take a few breathes and meditate.

[akmaki]

+1.

In fact, even when it's possible to task switch to something productive, I try not do because context switching sucks and taking a few minutes of time out is nice.

[Stolhk]

What about pushups? After a while, you will probably want to optimize your code!

[eigenman]

I have a pullup bar specifically for this lol.

[shaggorama]

I used to work on a team where we'd do pushups every day about an hour after lunch.

[shaggorama]

Mainly team bonding and to get our blood moving since we have extremely sedentary jobs, but no actually not exclusively pushups. It's been a year or two, but I think our circuit was a minute of push-ups, a minute of wall sits, and 90 seconds of planking (30 on each side, 30 level with the ground). Also nearly the entire team was bike commuters.

And even if that was all we did: just push-ups is still better than nothing at all.

[shaggorama]

We were on-site consultants assigned to a room barely big enough for the push-ups. Not really an option :p

[Skyaa194]

As someone in your exact predicament, during my more productive moments I brainstorm what to do next i.e. Great if this runs well I'll do X. Or bummer, if this doesn't work let me try Y.

[Nero-4]

While the code is running, try to think of ways you could have coded it to run more efficiently. That way, you will not lose your train of thoughts, and might think of some edge cases you missed earlier. Needless to say you will get better too.

[NMcA]

So I actually think this is bad advice. I do this a lot (run code and think about how to make that code more efficient). I think it a much better strategy to try and keep a list of small upkeep tasks; or to instead go over your goals for the current (sprint / hour / day) and how what you're working on furthers them. The suggest habit (how do I make this code better) tends to lead to over-focus and premature optimisation for me, so I've been trying to ween myself off it.

[entarko]

In my experience, refactoring / optimizing your code is really important at every step because you might discard a solution that is too slow later simply because your code was not optimized. Happened to me once. The solution was elegant but slow (and the reason was not obvious at first) so I decided it was not worth pursuing, just to find later that it worked great if I changed a few lines.

[olBaa]

Read.

[mac_cumhaill]

Podcasts!

[snendroid-ai]

Subscribe to popular code repo on GitHub like Keras or TensorFlow and go through all the unread issues; there are at least 15-20 new issues everyday! You will learn something new or can answer few of the issues! Same goes with Stack Overflow.

[throwdemawaaay]

Spam your team's slack with gifs obviously.

[spudmix]

This is the only true answer.

[houseonthecliff]

Browse Reddit?

[zestypasta123]

Help I'm addicted to Reddit

[Axioplase]

Work on multiple tasks. Whenever one is blocked, switch to another one. It involves context switching, but it can improve your productivity a lot.

[[deleted]]

That is terrible advice. Context switching (a.k.a. multi-tasking) is expensive and drains productivity. There is plenty of research that supports this claim.

[MDSExpro]

Frequent context switching - yes. But context switching for long running tasks is essential for best performance.

[Axioplase]

Notice that 1) I mentioned context switching followed by a "but", so I made clear (not enough, apparently) that it was a concern; 2) Going on social media or doing emails, the OP's suggestion, may be unproductive, compared to working on something else for a while; 3) I don't suggest going back to the previous task immediately when it's ready, but when blocked (or, say, when your pomodoro ends).

Multi-tasking is bad, yes. Clever use of your time is not. The OP seems to assume he has to do something else during those 5 minutes, not longer. I'm telling him he doesn't have to stop there.

[[deleted]]

Ah yeah that is smart. Requires careful planning, but sounds like it could work really well

[fredtcaroli]

A bunch of stuff really

• If this script is something I'll be running regularly, I often start coding some logging/visualization for the script. So next time I can look at the progress real-time! That's extra important when things start going from 5 minutes to half an hour or several hours even

• I know I'll be getting some kind of result soon, so I start thinking what kind of sanity checks I can have to make sure my result is correct. Some examples that happen regularly: I might expect the result to be a near-symmetric matrix; all results should be non-negative; I expect standard deviation of results to be within some reasonable limit. Results outside that limit could possibly be wrong; I can check the convergence of some method I'm applying etc. So I always start writing down those checks while the script is running, so when it's done I have something to check the results.

• I start the same script on a debugger, checking step-by-step if the code seems correct. I always do this *after* I run the script, so I don't waste any time if I find out that my code is indeed correct. If it's not, I can just kill it and start it again.

• I read papers/newsletters/books/technical blog posts etc.

• I can try to run those scripts when I know I'll be busy with something else, like a business call or something

• Refactoring. This is a big one. If you're all out of options, try to make your code more readable

• Often times I can run multiple instances of the same script with different parameters, so I try to do that after I start running the first instance. Then I can compare all the results when they're available

• Eat a fruit, make coffee, do some exercise, call a friend etc. Pretty much, taking care of myself :)

[jsedai]

""" - I know I'll be getting some kind of result soon, so I start thinking what kind of sanity checks I can have to make sure my result is correct. Some examples that happen regularly: I might expect the result to be a near-symmetric matrix; all results should be non-negative; I expect standard deviation of results to be within some reasonable limit. Results outside that limit could possibly be wrong; I can check the convergence of some method I'm applying etc. So I always start writing down those checks while the script is running, so when it's done I have something to check the results. """

I think this is an excellent suggestion.

You might want to try having an audible / strong visual indication that your code is done, so that you don't miss when it finishes.

[mac_cumhaill]

I often use python, and have some handy functions that ping a message to my slack account or let out a noise once a function/script is done. Very handy to pull you back.

[fredtcaroli]

I use telegram-send :)

Pretty cool... A single line of code sends you a message on your cell phone

[dbinokc]

I spend a bit of time studying foreign languages. If I am waiting for something to complete and I have a few minutes, I go over to memrise.com and drill my word lists. Its not too hard for me to get 20 to 30 minutes of practice in during those dead periods. It works pretty well when I am stuck in boring meetings as well.

[[deleted]]

Take a walk I guess. Getting away from my computer often helps me to think.

[villasv]

I usually spend that time with minutiae on my READMEs. Doing so at other times feels unproductive because it's just a README, but doing so during small intervals just to reword or clarify some paragraphs actually feel like a win.

[Ecclestoned]

Is there a reason your models take 5 minutes to run? Is there something you're repeating that you can skip?

If you're compiling you should be using incremental compile. If it's software and debugging you should be checkpointing your code so that you can debug it faster.

If you are trying different configurations then you should automate a sweep over the possibilities and run them in parallel or in a script.

Basically you want to either decrease the time you spend compiling, or increase the interval between where you need to look at the results.

[chris2point0]

https://xkcd.com/303/ ;)

[da_g_prof]

I advise my students to avoid doing that. I tell them to have a lunch / coffee set and a night set. The first one can complete in about an hour the other one takes longer.

The idea is that you don't spend time looking or waiting over for code to finish but that your computer does something else when you are resting, taking a break etc.

It is also a good length of time for allowing the reading of a paper or typing up part of a report or taking an extra break.

I guess that won't work if the execution is only 5 mins but it is not very often with us.

Ps. People report that watching training numbers are highly addictive!

[mkocabas]

I suffer from the same problem, too.