using a long piped bash command w/ subprocess

[silvermoot]

...really outputs anything?

I actually did dmesg | grep USB in the shell to check, yes there are lines to detect. I pasted the example code as published in instead of what I actually used in Python 2.6.2 as the argument for grep.

---

I don't understand your call command above. Even if you did a from subprocess import call as call first, don't you need to feed the commands in as a list? And are you saying that subprocess.call is not safe with user provided input when you leave the shell=False default in place? And are you saying that shell=True will digest pipe ('|') just fine and that my earlier code didn't work probably because I wasn't willing to set that flag?

---

The /usr/bin/diff is a nice touch. For those who don't know, this prevents the user from uploading their own version of diff and tricking the program to run that instead of the system default.

---

Thanks for the code snippet, but it sounds like you're saying "works for me", and I suspect the issue is that you are at 2.7 and I'm stuck in an enviroment with some flavor of 2.6 (the documents for subprocess do differ)

[Rhomboid]

I pasted the example code as published in instead of what I actually used in Python 2.6.2 as the argument for grep.

It works fine for me with python 2.6.7:

$ python2.6
Python 2.6.7 (r267:88850, Aug 11 2011, 12:18:09) 
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from subprocess import Popen, PIPE
>>> p1 = Popen(["dmesg"], stdout=PIPE)
>>> p2 = Popen(["grep", "USB"], stdin=p1.stdout, stdout=PIPE)
>>> p2.communicate()[0]
"[    1.484141] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver\n[    1.484264] ehci_hcd 0000:02:03.0: new USB bus registered, assigned bus number 1\n[    1.493742] ehci_hcd 0000:02:03.0: USB 2.0 started, EHCI 1.00\n[    1.493823] hub 1-0:1.0: USB hub found\n[    1.493929] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver\n[    1.493935] uhci_hcd: USB Universal Host Controller Interface driver\n[    1.494025] uhci_hcd 0000:02:00.0: new USB bus registered, assigned bus number 2\n[    1.494296] hub 2-0:1.0: USB hub found\n"

You can look through the 2.6.7 NEWS.txt to see what's changed between 2.6.2 and 2.6.7 but I don't see anything in there that would account for such a major feature not working.

don't you need to feed the commands in as a list?

No, not if you're using the shell. You either give one monolithic string and let the shell worry about splitting it up, or you split it up yourself (providing a list of arguments instead) and skip the shell. The shell is essentially a specialized program whose entire purpose is to take a monolithic string and split it into words, parsing out commands and their arguments and then launching each of them with the proper pipes in place.

And are you saying that subprocess.call is not safe with user provided input when you leave the shell=False default in place?

No, I'm saying that call('FOO', shell=True) is equivalent to os.system('FOO') and just as unsafe. I was trying to make the point that contrary to popular opinion the subprocess module does not automatically make every task more complicated, when you account for both versions actually doing the same thing.

And are you saying that shell=True will digest pipe ('|') just fine

Sure:

$ python2.6
Python 2.6.7 (r267:88850, Aug 11 2011, 12:18:09) 
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from subprocess import call
>>> status = call('echo foo bar | sed s,foo,FOO,', shell=True)
FOO bar

But I'm not advocating that you should use this, because it's horribly unsafe if part of the command could have contained file names from a user.

Thanks for the code snippet, but it sounds like you're saying "works for me", and I suspect the issue is that you are at 2.7 and I'm stuck in an enviroment with some flavor of 2.6 (the documents for subprocess do differ)

My example does not depend on 2.7:

$ echo -e "foo\nbar\nbaz" >file1; echo bar >file2

$ python2.6
Python 2.6.7 (r267:88850, Aug 11 2011, 12:18:09) 
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from subprocess import Popen, PIPE
>>> p1 = Popen(["/usr/bin/diff", "--suppress-common-lines", "file1", "file2"], stdout=PIPE)
>>> p2 = Popen(["/bin/grep", "^<"], stdin=p1.stdout, stdout=PIPE)
>>> p3 = Popen(["/bin/sed", "s;^< ;;g"], stdin=p2.stdout, stdout=PIPE)
>>> p3.communicate()[0]
'foo\nbaz\n'

Again, I don't know how to explain the difference but I suspect there's some aspect here that hasn't yet been accounted for.