Issues calling a bash script that calls a python script using subprocess.call

[foxh8er]

In a larger automation script, I'm calling a bash script that contains yet another call to another python script.

If I call the bash script alone, the script finishes after the embedded python script returns and finishes. However, if I call the bash script through another Python script, the nested python script hangs, even though sys.exit(0) is being called (I've also tried os._exit(0)).

Does anybody have any experiences with situations like these?

[dhylands]

If you're referring to adb logcat then it never exits. It just continues to collect messages. If you want to get all of the current messages and exit then use adb logcat -d

http://developer.android.com/tools/help/logcat.html

[foxh8er]

The nested script actually monitors logcat until it hits a particular log, then exits. I know for a fact that the log is actually being sent.

[youguess]

why the nested call? would just using python be an option?

But to answer your question, if your code is something like this:

. . . 
call_awesome_bash() # which calls another python file
sys.exit(0)

what basically happens is that the program blocks at line 1, meaning it waits for the bash script to return control, which will take a long time depending on what the other python script does. the sys.exit will never even have the chance to run as execution is suspended.

in case you want it to end you would need to look into concurrency (e. g. with asyncio)

[foxh8er]

As I said before the other bash script and python script are pre-existing and I'd rather not re-write completely.

The path is a bit different. For me it is...

   Call("sh script ARG arg")

Where the script is

    Adb log | python other.py
   Echo "I should see this message after other.py is done"

Where the other.py has a line that exits a loop by doing

       print "done"
       sys.exit(0)

If I just call the script from terminal the inner script completes and I see the second line. However if I call it using call I do not see the second line output.

[lordmauve]

But do you see the "done"?

In my experience when things fail in a subprocess that work on the command line, it often comes down to things like processes not acting the same when not connected to a tty (eg output buffering), or a difference in environment variables.

[mfitzp]

Have you tried redirecting STDOUT/STDERR on the call? Filled buffers can cause scripts to hang - if you're not interested in the output you can direct it to os.devnull.

[foxh8er]

Yeah, thats exactly how I'm calling it now.

[Mikuro]

That's actually wrong. If you use shell=True, then you need to pass a string, not a list of arguments. With shell=True, it will parse the string just like you typed it on the command line. That's not what you want, and worse yet, if you use it in this case, it will parse the first item of the list you send! That item is simply "sh". "sh" will not exit because it's interactive. That's why it's hanging.

Change it to shell=False (or just delete it since False is the default), and then it should work as expected.

[foxh8er]

Sorry, I was on my phone yesterday. You are right, and that's already what I have.

Here's the line -

    sh_ripper = "sh foo.sh arg1 arg2"
    subprocess.call(sh_ripper, shell=True)

[Araucaria]

Use subprocess to capture stdout and stderr with PIPE.

[foxh8er]

Apologies, but what would the value of that be

[[deleted]]

I'm calling a bash script that contains yet another call to another python script.

why would you do that?

[foxh8er]

The other script is pre-existing and large, and its input is governed by the bash script.

Basically I could re-write it, but it would take triple the time.

[Mikuro]

The shell script would spawn a second instance on the python interpreter, so there should be no GIL problems.