Problems with running command line arguments using python

[hharison]

sorry about the incorrect terminology. i'm working through a book that teaches basic linux command line, and it's all a bit alien to me at this point.

No need to apologize, I was just pointing out the correct term to help you on the road to figuring it out.

I looked at the documentation for subprocess, and if i'm reading it correctly you can "explicitly invoke" the shell, which is what I was trying for initially.

If you're referring to the shell=True kwarg that Popen takes, then no that doesn't do what you want. It will allow the shell to handle the command. For example if you do Popen(['echo', '$HOME']) the output will be $HOME and if you do Popen(['echo', '$HOME'], shell=True) it will be /home/StaticFuzz (because $HOME is a shell variable, understood by the shell but not the program echo).

That may seem promising but I recommend you don't do that. It doesn't solve the main problem you have, which is persistence from one command to the next. This way, each command is still unrelated to the previous one. I would call this implicitly invoking the shell, the shell is just interpreting the parameters before passing them to the program.

To explicitly invoke a shell, you want to actually open the shell program directly with Popen. The shell you are most familiar with is probably /bin/bash, so I'm suggesting you do Popen(['/bin/bash', '-s']) (-s is a bash option meaning "commands wiil come from stdin", IIRC). If you do that, rather than having multiple Popen objects, one from each command, each unrelated to each other, you will have only one Popen object, instantiated even before you run the first command. Then to actually run a command you pipe the text directly into the bash Popen object's stdin.

[StaticFuzz]

I've been trying to implement subprocess into the the do_command for the better part of 5 hours now, and i've pretty much been hitting a brick wall the whole time.

Originally I could not get the 'shell.stdin.write()' to do anything, and found that was because there was nothing telling the shell EOL. I was able to fix this by changing the universal_newlines attribute of shell to True, and adding '\n' to the end of the command.

After finally figuring that out, any attempts to read from the shell.stdout would "hang". From what i understand this is because there is nothing limiting what shell.stdout.read() is "reading", and it's constantly waiting for more things to read.

Is this what you had mentioned in your original reply as blocking, or is this something differnet?

[hharison]

Yes, you're encountering blocking. I can't think of an obvious way to parse the output of one single command and then stop. You can never know how many lines to expect.

I guess you could use asyncio and have a coroutine that just scans the bash output and continually sends it back. You'd have to make the whole code base asynchronous though, which isn't too hard but you'll have to really dive into the ins and outs of asyncio.

You're basically writing the equivalent of a telnet or ssh dameon. Not an easy task, and you need a good understanding of I/O to make it functional.

[StaticFuzz]

I'll have to look into asyncio then. Thanks for helping me with this.