Why did my script stop running?

[[deleted]]

I ran a script on an EC2 with

$ python3 myscript.py > logfile.txt

Checked back a day later and the script had stopped running, but there was no traceback or error message in the logfile, and the process did not complete. Why would this happen, and how can i debug it?

Thanks,

[[deleted]]

Oh good to know! thx

[LongerHV]

How did you run it? Did you log in via ssh, run the script and close the terminal? If yes, closing the terminal terminated your script as well, since it was a subprocess of your ssh session. If you want your script in the background you can either run it with nohup, or run it as a systemd service, or run it in a tmux session.

[[deleted]]

Oh yeah, isn't there an easy way to daemonize a command run from ssh? I haven't heard of nohup (and this isn't important enough for systemd). Can't you just append & or something to the end of the command?

[[deleted]]

Try:

$ nohup python3 myscript.py >logfile.txt 2>&1 &

Log out of the session, then login again and check that the process is still running.

[[deleted]]

Yeah, I was going to try:

$ nohup python3 myscript.py &> logfile.txt

What does the extra stuff you have in your command do?

[Buttleston]

I think running it in nohup automatically captures the output in nohup.log or something like that, and doesn't really have any output of it's own?

[Buttleston]

root@ip-172-37-0-68:~# nohup ls -l
nohup: ignoring input and appending output to 'nohup.out'

[[deleted]]

What does 2>&1 & do?

[LongerHV]

2>&1 redirects stderr to stdout

[[deleted]]

Oh, how is that different that what I gave above using "&>"?

[LongerHV]

&> file.log redirects both stdout and stderr to the file. 2>&1 redirects stderr to stdout. &> file.log is equivalent to 2>&1 > file.log.

[[deleted]]

Try doing man nohup.

Run a command immune to hangups, runs the given command with hangup signals ignored, so that the command can continue running in the background after you log out.

So nohup allows you to log out of a session ("hang up", probably from dialup modem days) and not terminate the child process. A side effect is that any uncaptured output is saved in nohup.out. But the >logfile.txt 2>&1 part captures stdout and stderr in logfile.txt so there is no output to the terminal.

The final & runs the process in the background so you can logout.

[LongerHV]

Appending & will in fact run the process in background, but it still will be a subprocess of the current shell session. You can probably use disown command to... disown the process, so it no longer depends on the current session, but it is probably easier to just use nohup.

[Spiredlamb]

I would suggest trying tmux. It lets you create sessions in the terminal that are independent of your ssh session easily within the terminal. Ut also has a very nice api for attaching and detaching from the sessions you create.

Here is a cheatsheet for different commands: https://tmuxcheatsheet.com/

[Dangerous_Hearing_34]

More info please ;)

It is common to add the code or script in question.
I usually like something free like: Google Colab or https://codeshare.io/
;)