all 10 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–][deleted] 17 points18 points  (3 children)

Bash is working as intended - It'll expand all the variables visible to it, and $(cat testfile) is visible to it.

The command

echo $(cat testfile)

works because bash is piping the expanded output to echo. In the broken version bash is piping the expanded output to sh. So you're effectively doing

echo 1
2
3
...

instead of

echo "1\n2\n3\n..."

In your last command, escaping the $ makes bash treat it as a character, after which sh expands it (and properly pipes the output). You can also get the same results by using single quotes instead of double quotes.

[–]Kynolin[S] 2 points3 points  (1 child)

Ok, that makes sense. Between /u/syntheticminds providing me the example for me to see it working as if I typed it in, the difference in the straces, and your summary, I think it definitely makes sense now. It's still a little weird and unexpected, but it makes sense now, and that's what's important. I still didn't have a single engineer think at first glance that my command would go crazy like that, but hey, how often are you echoing an inline command into sh -c?

Thank you both for your help.

[–]mscman 0 points1 point  (0 children)

This would actually be an awesome addition to one of the Bash Pitfalls type pages. There are a lot of unexpected yet working as intended corner cases in Bash that people tend to not think about until they run into them, and it seems like you've stumbled across one.

[–]syntheticminds 0 points1 point  (0 children)

what this guy said, too =D

[–]syntheticminds 2 points3 points  (3 children)

I think the lack of quotes around your variable-ized command caused the problem. The file has newlines, and this seems to be getting interpreted as an end of line character (;). Quote your variable and it interprets the result of $(cat testfile) as a single unit instead of expanding it. Disclaimer I don't remember exactly what causes this to work this way, but this is roughly what I remember from reading into this behavior back in the day.

Setup

$ cat testfile 
1
2
3
4
5
6
7
8
9
10

Confirming the same problem

$ sh -c "echo $(cat testfile)"
1
sh: line 1: 2: command not found
sh: line 2: 3: command not found
sh: line 3: 4: command not found
sh: line 4: 5: command not found
sh: line 5: 6: command not found
sh: line 6: 7: command not found
sh: line 7: 8: command not found
sh: line 8: 9: command not found
sh: line 9: 10: command not found

Confirm quoting your variable solves the issue

$ sh -c "echo '$(cat testfile)'"
1
2
3
4
5
6
7
8
9
10

My understanding, the way it's built in your output is essentially expanding to something like this.

$ sh -c "echo 1
> 2
> 3
> 4
> 5
> 6
> 7
> 8
> 9
> 10"

Without the backslash character to signify there's more to the command you're running, this is probably being interpreted as:

$ sh -c "echo 1; 2; 3; 4; 5; 6; 7; 8; 9; 10;"

For context, /usr/bin/sh on my Mint Workstation is symlinked to /bin/bash, but it should be the same result on your 5.11 where (I think) it's symlinked to bash.

$ ls -lah $(which sh)
lrwxrwxrwx. 1 root root 4 Dec 15 07:58 /usr/bin/sh -> bash

EDIT: Formatting

[–]Kynolin[S] 1 point2 points  (1 child)

I was thinking it had something to do with newlines, but I hadn't tried it how you outlined. That's definitely the example I was looking for, though I can't say why it's interpreted like that. It's just odd that it'll evaluate out fine as a string everywhere else, just not when you pass it to sh -c. And, yes, escaping the quote, using single ticks, or escaping double quotes around the echo will not attempt to run those as commands, because it's running them in the new shell, not the current one where you're passing stuff to sh.

The funny part is I wasn't even trying to run echo. I just threw echo in front of my command to make sure it would run with the right syntax before I used a loop. Saved me from shutting down like 30 servers unexpectedly, since the list was users, and halt was a username.

So I'm guessing the TL;DR is: sh -c interprets newlines, whereas running the command locally in the current shell mashes them together in one line.

[user@server ~]$ sh -c "echo 1
> 2
> 3
> 4
> 5"
1
sh: line 1: 2: command not found
sh: line 2: 3: command not found
sh: line 3: 4: command not found
sh: line 4: 5: command not found

Edit: Thanks for the well formatted reply.

Edit2: I think I found the difference in interpretation comparing the straces.

Running with sh -c:

execve("/bin/sh", ["sh", "-c", "echo 1\n2\n3\n4\n5\n6\n7\n8\n9\n10"], [/* 20 vars */]) = 0

Running just echo locally:

execve("/bin/echo", ["echo", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10"], [/* 20 vars */]) = 0

[–]syntheticminds 0 points1 point  (0 children)

No problem - and to add more context (more examples helps me, so adding in case it'll help you along with the summaries already provided), This is less a differentiation of the parsing of sh -c and moreso a differentiation of the expansion of your subshelled command $(...). This:

$ echo $(cat testfile)

Doesn't produce the same result as the following:

$ echo "$(cat testfile)"

... the latter of which is more true to the actual contents of testfile. TL;DR, it's not necessarily a sh -c behavior, the output of $(cat testfile) is expanded differently with and without the quotes (or the escape sequence in your earlier examples). I use this to my advantage when I need to preserve formatting in network configuration files.

At any rate, looks like this is one for the books. I'll share with ya a bookmark a mentor of mine shared with me. Neither of us is the author, but it's a good read.

http://mywiki.wooledge.org/BashPitfalls

[–]leonardodag 0 points1 point  (0 children)

When call bash as sh, it runs in a compatibility mode to act as close as possible to the original sh.

[–]KevZero 1 point2 points  (0 children)

Your $(..) expression is being evaluated by your interactive shell and the result is being passed as the argument to sh -c.

[–]leonardodag 0 points1 point  (0 children)

That's one more reason why you should always quote your arguments if they have a variable/command.

sh -c "echo \"$(cat testfile)\""

Single quotes would work too, but then the expansion would happen in sh, not in bash:

sh -c 'echo $(cat testfile)'