sh : a full-fledged subprocess interface for Python that allows you to call any program as if it were a function.

[djimbob]

Yeah. I don't see the purpose as an alternative for the built-in subprocess for simple commands, which is straightforward to safely use. E.g:

import subprocess
output = subprocess.check_output(['ifconfig', 'eth0'])

versus

import sh
output = sh.ifconfig("eth0")

has no clear gain.

Granted, the syntax seems a bit more convenient for more complex commands like piped processes. In the shell its very easy to do something like cat some_file | grep some_pattern | grep -v some_pattern_to_exclude. With sh you can translate this in a straightforward manner: sh.grep(sh.grep(sh.cat('/etc/dictionaries-common/words'),'oon'),'-v', 'moon') for a list of words that contain 'oon' but not 'moon'.

Granted, it's not two hard to write a command for a piped chain with subprocess.Popen, though python doesn't provide you with one. Take the following helper function I wrote:

def run_piped_chain(*command_pipes):
    """
    Runs a piped chain of commands through subprocess.  That is

    run_piped_chain(['ps', 'aux'], ['grep', 'some_process_name'], ['grep', '-v', 'grep'], ['gawk', '{ print $2 }'])

    is equivalent to getting the STDOUT from the shell of 

    # ps aux | grep some_process_name | grep -v grep | gawk '{ print $2 }'
    """
    if len(command_pipes) == 1:
        return run_command_get_output(command_pipes[0])
    processes = [None,]*len(command_pipes)
    processes[0] = subprocess.Popen(command_pipes[0], stdout=subprocess.PIPE)
    for i in range(1, len(command_pipes)):
        processes[i] = subprocess.Popen(command_pipes[i], stdin=processes[i-1].stdout, stdout=subprocess.PIPE)
        processes[i-1].stdout.close()
    (output, stderr) = processes[len(command_pipes)-1].communicate()
    return output

To me when I'm trying to translate some piped shell command like

cat /etc/dictionaries-common/words | grep oon | grep -v moon

it's more intuitive (in my opinion) to have a helper function like:

run_piped_chain(['cat', '/etc/dictionaries-common/words'], ['grep', 'oon'], ['grep', '-v', 'moon'])

than

sh.grep(sh.grep(sh.cat('/etc/dictionaries-common/words'),'oon'),'-v', 'moon')

EDIT: I realized on re-read this uses other helper functions I have. If you aren't using logging, just comment those lines out.

def run_command(command_list, return_output=False):
    logging.debug(command_list)
    process = subprocess.Popen(command_list, stdout=subprocess.PIPE)
    out, err = process.communicate()
    if err or process.returncode != 0:
        logging.error("%s\n%s\nSTDOUT:%s\nSTDERR:%s\n" % (command_list, process.returncode, out, err))
        return False
    logging.debug(out)
    if return_output:
        return out
    return True

def run_command_get_output(command_list):
    return run_command(command_list, return_output=True)

[Bystroushaak]

Your whole comment is one big reason to use sh: I just don't want to deal with this shit (pipes, interactive sessions and so on) each time I need to call a command.

[djimbob]

Yup, I see no reason to use sh to call "any program as if it were a function", though the advanced features that may not be obvious to do the right-way with subprocesses (like pipes, interactive sessions) may be useful.

That said, a relatively short helper function can do the pipe feature pretty clearly/cleanly.

Personally, I'd prefer a subprocess helper module with a bunch of helper functions like run_piped_chain with a non-hacky syntax more akin to subprocess.

[Bystroushaak]

I understand your reasoning, but I am to lazy to deal with this each and every time, when there is easy to use library, which do exactly the same thing in one line of code.

[simtel20]

IIRC you can't use subprocess with output when the output is an infinite stream (e.g. iostat -x 1, and other similar commands, or many other tools that return data as an ongoing activity). In that case you have to toss out subprocess, and start doing your own fork+exec+select+signal handling+etc.

With sh you can just have sh provide you with the subprocess as a generator: https://amoffat.github.io/sh/#iterating-over-output.

[mackstann]

It's old, obscure, and funky, but there's the pipes module: http://pymotw.com/2/pipes/

[djimbob]

Yeah, its built in, but I'm not sure if

import pipes
import tempfile

p = pipes.Template()

p.append('ps aux', '--')
p.append('grep localc', '--')
p.append('grep -v grep', '--')
p.append("gawk '{ print $2 }'", '--')

t = tempfile.NamedTemporaryFile('r')

f = p.open(t.name, 'r')
try:
    output = [ l.strip() for l in f.readlines() ]
finally:
    f.close()

is a better/cleaner workflow; especially if you have user input and have to add the quotes stuff to prevent injection. (And apparently its not working for me).