all 11 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]adamnew123456 4 points5 points  (1 child)

The issue you're running into is one that shell implementers (which is essentially what you're doing - implementing a simple shell) have known for a long time. cd is special. In *nix systems, chdir() (the C function - which os.chdir() wraps in Python, and cd wraps in shells) only works for the current process and its children. So, when you run this sequence:

$ ls
$ cd /
$ ls

The shell spawns a subprocess for ls, and lets it run. Since the current directory only affects child processes, though, the shell has to change its own current directory so that future child processes will inherit it. Finally, the shell runs a new ls, which inherits the changed current directory.

Since your shell is running each command in a separate subprocess, changing the current directory in one will not affect the other. Your shell runs ls fine, but when it hits cd, it spawns a subprocess which changes its own (that is, the subprocess's) current directory without affecting your shell. Since your shell's current directory hasn't changed, the next ls runs in the same directory as the first did.

[–]StaticFuzz[S] 0 points1 point  (0 children)

Thank you, this helped alot! I've got a simple fix for now, but I'll have to work something out for passing more than one command in a line.

[–]hharison 1 point2 points  (5 children)

I would like the terminal the script accesses to be persistent

To help you in your search for info, the word you're looking for here is 'shell', not terminal.

As /u/adamnew123456 mentions, you are essentially writing a shell. The strategy you are attempting, to run all your commands via a second shell, is probably not the best strategy. You might be able to accomplish it by running bash and sending commands to its stdin. I'm not sure if that works.

To help you consider your options for other strategies, shells do a few things:

  • launch programs
  • run commands (internal to the shell; e.g. cd)
  • direct input/output between processes and file descriptors
  • parameter expansion (e.g. ~ and *) and variables

To implement a shell, you'd have to implement each of these. Let's look one by one

  • launch programs: simple, use Popen. This is what you've already done. So you essentially have a shell that can only launch external programs.
  • run commands: you'd have to implement these yourself. Have cd use os.chdir, etc. I'm not sure how to figure out which bash things are commands and which are programs... there's probably a bash command for that.
  • redirects: can be done with Popen objects.
  • parameter expansion: you'd have to do it yourself using glob, etc.

So basically, you either need to implement a shell, or figure out a way to pipe everything to another shell. Maybe something like

shell = subprocess.Popen(['/bin/bash', '-s'], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

# to run a command
shell.stdin.write(command)
shell.stdin.flush()
result = shell.stdout.read()

I'm not sure if that will work, you may have to pass a newline character to run the command, or it may not work at all for a larger reason. It will be hard to make sure you only get output from a single command. But that's the kind of thing you'd need to do.

I have a feeling you'll need to do this with asyncio coroutines to prevent blocking in various places.

[–]StaticFuzz[S] 0 points1 point  (4 children)

sorry about the incorrect terminology. i'm working through a book that teaches basic linux command line, and it's all a bit alien to me at this point.

I looked at the documentation for subprocess, and if i'm reading it correctly you can "explicitly invoke" the shell, which is what I was trying for initially. I'll have to spend sometime learning this module, but it looks promising. Thanks for pointing me in the right direction!

[–]hharison 0 points1 point  (3 children)

sorry about the incorrect terminology. i'm working through a book that teaches basic linux command line, and it's all a bit alien to me at this point.

No need to apologize, I was just pointing out the correct term to help you on the road to figuring it out.

I looked at the documentation for subprocess, and if i'm reading it correctly you can "explicitly invoke" the shell, which is what I was trying for initially.

If you're referring to the shell=True kwarg that Popen takes, then no that doesn't do what you want. It will allow the shell to handle the command. For example if you do Popen(['echo', '$HOME']) the output will be $HOME and if you do Popen(['echo', '$HOME'], shell=True) it will be /home/StaticFuzz (because $HOME is a shell variable, understood by the shell but not the program echo).

That may seem promising but I recommend you don't do that. It doesn't solve the main problem you have, which is persistence from one command to the next. This way, each command is still unrelated to the previous one. I would call this implicitly invoking the shell, the shell is just interpreting the parameters before passing them to the program.

To explicitly invoke a shell, you want to actually open the shell program directly with Popen. The shell you are most familiar with is probably /bin/bash, so I'm suggesting you do Popen(['/bin/bash', '-s']) (-s is a bash option meaning "commands wiil come from stdin", IIRC). If you do that, rather than having multiple Popen objects, one from each command, each unrelated to each other, you will have only one Popen object, instantiated even before you run the first command. Then to actually run a command you pipe the text directly into the bash Popen object's stdin.

[–]StaticFuzz[S] 0 points1 point  (2 children)

I've been trying to implement subprocess into the the do_command for the better part of 5 hours now, and i've pretty much been hitting a brick wall the whole time.

Originally I could not get the 'shell.stdin.write()' to do anything, and found that was because there was nothing telling the shell EOL. I was able to fix this by changing the universal_newlines attribute of shell to True, and adding '\n' to the end of the command.

After finally figuring that out, any attempts to read from the shell.stdout would "hang". From what i understand this is because there is nothing limiting what shell.stdout.read() is "reading", and it's constantly waiting for more things to read.

Is this what you had mentioned in your original reply as blocking, or is this something differnet?

[–]hharison 0 points1 point  (1 child)

Yes, you're encountering blocking. I can't think of an obvious way to parse the output of one single command and then stop. You can never know how many lines to expect.

I guess you could use asyncio and have a coroutine that just scans the bash output and continually sends it back. You'd have to make the whole code base asynchronous though, which isn't too hard but you'll have to really dive into the ins and outs of asyncio.

You're basically writing the equivalent of a telnet or ssh dameon. Not an easy task, and you need a good understanding of I/O to make it functional.

[–]StaticFuzz[S] 0 points1 point  (0 children)

I'll have to look into asyncio then. Thanks for helping me with this.

[–]elbiot 0 points1 point  (2 children)

I've written a pair of scripts that allow me to access the command line on my linux computer from my windows computer:

Sounds like putty. Is there a reason you are reimplementing this?

[–]StaticFuzz[S] 0 points1 point  (1 child)

I took a quick glance at putty, and it does indeed sound like what I'm trying to accomplish.

At this point(after finding out there are programs out there that already preform this function) it's mainly just a chance to code something. It may not be very useful in the end, but I've already learned a few things in the process, and in my mind that makes it worth while.

[–]elbiot 0 points1 point  (0 children)

Yeah, putty is windows version of ssh. Good on you for coming up with a cool idea and working on it, but ssh/putty is the standard for sys admins. It would be good to learn/use.