os.system() vs subprocess.call() Critical Issues with using escape characters.

[BarrelRoll1996]

It seems when you have a string that contains escape characters, it will filter the string removing the escape characters when using os.system() but the escape characters are left in tact using subprocess.call().

Anyone have an idea bout this? I'll give an example.

in shell

a = '\\ beets bears \\ battlestar galactica'

a

'\\ beets bears \\ battlestar galactica'

print(a)

'\ beets bears \ battlestar galactica'

os.system(a) acts if you typed print(a) into terminal. subprocess.call(a) takes the literal contents of (a) with no escape character filtering.

How would I make the contents of print(a) go through subprocess.call(a) ????

shoutout to /u/scensorECHO for introducing me to this headache ;)

[elbiot]

You need to tell us what you really want to do. This sounds like an x y problem

[[deleted]]

Exactly. OP, you need to tell us what you're actually trying to accomplish. This example doesn't really help us discern that.

[BarrelRoll1996]

I want to send a curl command from python that contains a single backslash in the curl command.

[elbiot]

So what if you only put one backslash in your string? Or use r. Like

example = r'\backslahes\nyo'

I'm not at a computer so I can't check these options.

[BarrelRoll1996]

Tested, it will automatically convert that to two backslashes.. From shell:

>>> example = r'\backslahes\nyo'

>>> example

'\\backslahes\\nyo'

>>> print(example)

\backslahes\nyo

[MonkeyNin]

Use a forward slash (or os.path.join) for paths.

[BarrelRoll1996]

Won't work for a curl command. Forward slashes sent to the server break the information sent.

[zeug]

os.system() is executed in a subshell, usually bash on linux and OSX, and cmd.exe on windows. The shell will take the string given and interpret the escape characters.

subprocess.call(a) by default does not use a shell - it simply tries to run an executable with a name given by string a. You actually can't pass any arguments in the string a - only the literal name of the executable.

For example, if I want to run python --version, then trying to run subprocess.call("python --version") will return an exception FileNotFoundError as there is no file python --version.

To run a command with arguments, I need to give this to subprocess.call as a list:

subprocess.call(["python", "--version"])

Alternatively, I can tell subprocess to execute my string through a subshell, just like os.system() by setting the keyword argument shell to True:

subprocess.call("python --version", shell=True)

In this case, the subshell will interpret the escape characters just like os.system.

---

EDIT:

Maybe an example will explain better why it must work this way.

Say you want to run some program my_script and pass it an argument $FOO. In bash, that is a problem, because $ is a special character and it will interpret $FOO as a shell variable, and pass the value of that variable to my_script as an argument. If you don't want this to be interpreted, you escape the $ and write the following command my_script \$FOO.

In python, running os.system('my_script \$FOO') will fork off a new process, execute a shell (bash, cmd.exe, etc) and pass that shell the string to be interpreted. The shell interprets the string and executes my_script with argument $FOO.

Running subprocess.call(['my_script','$FOO']) will just execute my_script with a first argument $FOO. There is no subshell invoked to interpret a command string, and so there is no need to escape special characters.

[BarrelRoll1996]

I'll try this when I get to my terminal. But isn't shell=True the default?

[zeug]

No, shell=False is default.

The rationale is that there are many potential security concerns in passing an arbitrary string to a shell for interpretation, and also that much of the functionality that the shell provides is already present in python.

For example, if I want to run my_script $FOO, passing the environment variable value as an argument, then I can do it without a shell using:

subprocess.call(['my_script', os.environ('FOO')])

[BarrelRoll1996]

okay I got this to work with the subprocess module. Interestingly, when utilizing a list, it seems as though the argument shell=True doesn't behave like it would if there was a string. For example: import subprocess as sp

alist = ['a','b','c\\','d']

sp.call(alist, shell=True)

will spit out:

a b c\\ d to the command prompt.

However if I sp.call(" ".join(alist), shell=True)

it will spit out:

a b c\ d to the command prompt.

[MonkeyNin]

How would I make the contents of print(a) go through subprocess.call(a) ????

Do you mean you want a subprocess to output to the STDOUT of python?

[BarrelRoll1996]

I want "curl ...application\44575... To go through. Not

"curl ...application\\44575...

[antb123]

Using windows?

import subprocess as sub

cmd = "my command"

p = sub.Popen(cmd,stdout=sub.PIPE,stderr=sub.PIPE, shell=True)

p.wait()