unexpected behaviour in shell command substitution
Given the shell script test.sh
for var in "$@"
do
echo "$var"
done
and the invocation sh test.sh $(echo '"hello " "hi and bye"')
I would expect the output
hello
hi and bye
but instead get:
"hello
"
"hi
and
bye"
If I change to sh test.sh $(echo "hello " "hi and bye")
then I get
hello
hi
and
bye
Neither of these behaviours are desired, how can I get the desired behaviour?
What I want to understand is why sh test.sh $(echo "hello " "hi and bye")
and sh test.sh "hello " "hi and bye"
are not equivalent?
shell
add a comment |
Given the shell script test.sh
for var in "$@"
do
echo "$var"
done
and the invocation sh test.sh $(echo '"hello " "hi and bye"')
I would expect the output
hello
hi and bye
but instead get:
"hello
"
"hi
and
bye"
If I change to sh test.sh $(echo "hello " "hi and bye")
then I get
hello
hi
and
bye
Neither of these behaviours are desired, how can I get the desired behaviour?
What I want to understand is why sh test.sh $(echo "hello " "hi and bye")
and sh test.sh "hello " "hi and bye"
are not equivalent?
shell
add a comment |
Given the shell script test.sh
for var in "$@"
do
echo "$var"
done
and the invocation sh test.sh $(echo '"hello " "hi and bye"')
I would expect the output
hello
hi and bye
but instead get:
"hello
"
"hi
and
bye"
If I change to sh test.sh $(echo "hello " "hi and bye")
then I get
hello
hi
and
bye
Neither of these behaviours are desired, how can I get the desired behaviour?
What I want to understand is why sh test.sh $(echo "hello " "hi and bye")
and sh test.sh "hello " "hi and bye"
are not equivalent?
shell
Given the shell script test.sh
for var in "$@"
do
echo "$var"
done
and the invocation sh test.sh $(echo '"hello " "hi and bye"')
I would expect the output
hello
hi and bye
but instead get:
"hello
"
"hi
and
bye"
If I change to sh test.sh $(echo "hello " "hi and bye")
then I get
hello
hi
and
bye
Neither of these behaviours are desired, how can I get the desired behaviour?
What I want to understand is why sh test.sh $(echo "hello " "hi and bye")
and sh test.sh "hello " "hi and bye"
are not equivalent?
shell
shell
edited Jan 21 at 11:10
user1207217
asked Jan 21 at 10:59


user1207217user1207217
60849
60849
add a comment |
add a comment |
4 Answers
4
active
oldest
votes
Your command substitution will generate a string.
In the case of
$(echo '"hello " "hi and bye"')
this string will be "hello " "hi and bye"
.
The string is then undergoing word splitting (and filename globbing, but it doesn't affect this example). The word splitting happen on every character that is the same as one of the characters in $IFS
(by default, spaces, tabs and newlines).
The words generated by the default value of IFS
would be "hello
, "
, "hi
, and
, and bye"
.
These are then given as separate arguments to your script.
In your second command, the command substitution is
$(echo "hello " "hi and bye")
This generates the string hello hi and bye
and the word splitting would result in the four words hello
, hi
, and
, and bye
.
In your last example, you use the two arguments hello
and hi and bye
directly with your script. These won't undergo word splitting because they are quoted.
Thanks. Is the short answer to this then that you cannot (reliably) use command substitution to generate arguments to another command?
– user1207217
Jan 21 at 11:26
@user1207217 Correct, especially if the arguments contain characters from$IFS
or filename globbing characters.
– Kusalananda
Jan 21 at 11:29
@Kusalananda thanks, although thats a huge pain ....
– user1207217
Jan 21 at 11:30
1
@user1207217 You may want to ask a separate question about what it actually is that you are intending or trying to do.
– Kusalananda
Jan 21 at 11:31
Good idea, thanks. unix.stackexchange.com/questions/495769/…
– user1207217
Jan 21 at 11:40
add a comment |
TL;DR: the result of echo
is a victim of word splitting on $()
result, and sh test.sh "hello " "hi and bye"
uses a different rule.
Let's take a peak as what's actually happening, when you add set -x
for debugging:
> set -x
> ./main.sh $(echo '"hello " "hi and bye"')
++ echo '"hello " "hi and bye"'
+ ./main.sh '"hello' '"' '"hi' and 'bye"'
"hello
"
"hi
and
bye"
echo
receives "hello " "hi and bye"
as a single argument (yep, including all spaces and single quote) due to single-quotes in the command itself. But command substitution turns that into "hello " "hi and bye"
. To quote bash manual:
If the substitution appears within double quotes, word splitting and pathname expansion are not performed on the results.
However, in your case, unquoted command substitution allows exactly for word splitting to occur based on blank character (which is one of the 3 characters set in the $IFS
variable, which is what is used for word splitting). Thus, if you break the resulting string at each space, what do you have?
"hello
, then space, after which we have next token
"
- that one character by itself, isolated by spaces from both sides.
"hi
, again with space on left and rightand
bye"
all in all, you get 5 tokens broken at whitespaces. It's important to realize that word splitting, in this case, occurs exactly a result of command substitution. Later, all of these are treated as individual tokens that have been already parsed. As you see in set -x
output, the shell script is called with exactly those tokens.
As for why sh test.sh "hello " "hi and bye"
behaves differently, that's because the different rule applies. Your command-line is parsed after you enter it, and quoted strings are treated as a single unit, i.e. hello
and hi and bye
add a comment |
You could change IFS
to something other than space (since you want to protect the space in hi and bye
), and use that something to separate the two strings:
$ IFS=:
$ sh test.sh $(echo "hello ":"hi and bye")
hello
hi and bye
The order of operations (those relevant in this example, anyway) is somewhat like this:
- substitutions
- word splitting
- quote removal
Note that quotes created as a result of substitutions aren't special in word splitting or quote removal, so a quote in the output of (1) passes through (2) and (3) like any other character. So:
- In
echo "hello " "hi and bye"
, the shell removes these quotes, soecho
getshello
andhi and bye
, so outputshello hi and bye
, joining the strings with a space. - In
echo '"hello " "hi and bye"'
, the shell removes the outer'
, the echo gets"hello " "hi and bye"
and outputs that.
In
sh test.sh $(echo '"hello " "hi and bye"')
, the command substitution is replaced with"hello " "hi and bye"
, but these quotes are a result of a substitution, and so aren't involved in word splitting or quote removal.
So the shell splits those into
"hello
,"
,"hi
,and
,bye"
, hence the output you get.
- In
sh test.sh $(echo "hello " "hi and bye")
, the command substitution is replaced withhello hi and bye
, which gets split intohello
,hi
andbye
.
1
Thanks. I guess what I am not understanding is why the usual shell substitution is not occurring, i.e. why is the behaviour not the same as if I dosh test.sh "hello " "hi and bye"
?
– user1207217
Jan 21 at 11:09
@user1207217 does the edit help?
– Olorin
Jan 21 at 11:27
In$(echo "hello ":"hi and bye")
, you might as well write$(echo "hello :hi and bye")
. The colon works the same quoted or not.
– ilkkachu
Jan 21 at 11:31
add a comment |
How about
sh test.sh "hello " "hi and bye"
?
I'm aware I can do this, the shell script is a MWE of an unexpected behaviour -- I'm trying to use a command substitution to generate a long list of args (some of which contain spaces) but it doesn't work correctly because even with quoting in the command output the above is occurring, i.e. the shell is not doing its usual parsing and treating everything inside a single or double quote pair as a single arg
– user1207217
Jan 21 at 11:15
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f495755%2funexpected-behaviour-in-shell-command-substitution%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
4 Answers
4
active
oldest
votes
4 Answers
4
active
oldest
votes
active
oldest
votes
active
oldest
votes
Your command substitution will generate a string.
In the case of
$(echo '"hello " "hi and bye"')
this string will be "hello " "hi and bye"
.
The string is then undergoing word splitting (and filename globbing, but it doesn't affect this example). The word splitting happen on every character that is the same as one of the characters in $IFS
(by default, spaces, tabs and newlines).
The words generated by the default value of IFS
would be "hello
, "
, "hi
, and
, and bye"
.
These are then given as separate arguments to your script.
In your second command, the command substitution is
$(echo "hello " "hi and bye")
This generates the string hello hi and bye
and the word splitting would result in the four words hello
, hi
, and
, and bye
.
In your last example, you use the two arguments hello
and hi and bye
directly with your script. These won't undergo word splitting because they are quoted.
Thanks. Is the short answer to this then that you cannot (reliably) use command substitution to generate arguments to another command?
– user1207217
Jan 21 at 11:26
@user1207217 Correct, especially if the arguments contain characters from$IFS
or filename globbing characters.
– Kusalananda
Jan 21 at 11:29
@Kusalananda thanks, although thats a huge pain ....
– user1207217
Jan 21 at 11:30
1
@user1207217 You may want to ask a separate question about what it actually is that you are intending or trying to do.
– Kusalananda
Jan 21 at 11:31
Good idea, thanks. unix.stackexchange.com/questions/495769/…
– user1207217
Jan 21 at 11:40
add a comment |
Your command substitution will generate a string.
In the case of
$(echo '"hello " "hi and bye"')
this string will be "hello " "hi and bye"
.
The string is then undergoing word splitting (and filename globbing, but it doesn't affect this example). The word splitting happen on every character that is the same as one of the characters in $IFS
(by default, spaces, tabs and newlines).
The words generated by the default value of IFS
would be "hello
, "
, "hi
, and
, and bye"
.
These are then given as separate arguments to your script.
In your second command, the command substitution is
$(echo "hello " "hi and bye")
This generates the string hello hi and bye
and the word splitting would result in the four words hello
, hi
, and
, and bye
.
In your last example, you use the two arguments hello
and hi and bye
directly with your script. These won't undergo word splitting because they are quoted.
Thanks. Is the short answer to this then that you cannot (reliably) use command substitution to generate arguments to another command?
– user1207217
Jan 21 at 11:26
@user1207217 Correct, especially if the arguments contain characters from$IFS
or filename globbing characters.
– Kusalananda
Jan 21 at 11:29
@Kusalananda thanks, although thats a huge pain ....
– user1207217
Jan 21 at 11:30
1
@user1207217 You may want to ask a separate question about what it actually is that you are intending or trying to do.
– Kusalananda
Jan 21 at 11:31
Good idea, thanks. unix.stackexchange.com/questions/495769/…
– user1207217
Jan 21 at 11:40
add a comment |
Your command substitution will generate a string.
In the case of
$(echo '"hello " "hi and bye"')
this string will be "hello " "hi and bye"
.
The string is then undergoing word splitting (and filename globbing, but it doesn't affect this example). The word splitting happen on every character that is the same as one of the characters in $IFS
(by default, spaces, tabs and newlines).
The words generated by the default value of IFS
would be "hello
, "
, "hi
, and
, and bye"
.
These are then given as separate arguments to your script.
In your second command, the command substitution is
$(echo "hello " "hi and bye")
This generates the string hello hi and bye
and the word splitting would result in the four words hello
, hi
, and
, and bye
.
In your last example, you use the two arguments hello
and hi and bye
directly with your script. These won't undergo word splitting because they are quoted.
Your command substitution will generate a string.
In the case of
$(echo '"hello " "hi and bye"')
this string will be "hello " "hi and bye"
.
The string is then undergoing word splitting (and filename globbing, but it doesn't affect this example). The word splitting happen on every character that is the same as one of the characters in $IFS
(by default, spaces, tabs and newlines).
The words generated by the default value of IFS
would be "hello
, "
, "hi
, and
, and bye"
.
These are then given as separate arguments to your script.
In your second command, the command substitution is
$(echo "hello " "hi and bye")
This generates the string hello hi and bye
and the word splitting would result in the four words hello
, hi
, and
, and bye
.
In your last example, you use the two arguments hello
and hi and bye
directly with your script. These won't undergo word splitting because they are quoted.
edited Jan 21 at 11:30
answered Jan 21 at 11:20


KusalanandaKusalananda
134k17255418
134k17255418
Thanks. Is the short answer to this then that you cannot (reliably) use command substitution to generate arguments to another command?
– user1207217
Jan 21 at 11:26
@user1207217 Correct, especially if the arguments contain characters from$IFS
or filename globbing characters.
– Kusalananda
Jan 21 at 11:29
@Kusalananda thanks, although thats a huge pain ....
– user1207217
Jan 21 at 11:30
1
@user1207217 You may want to ask a separate question about what it actually is that you are intending or trying to do.
– Kusalananda
Jan 21 at 11:31
Good idea, thanks. unix.stackexchange.com/questions/495769/…
– user1207217
Jan 21 at 11:40
add a comment |
Thanks. Is the short answer to this then that you cannot (reliably) use command substitution to generate arguments to another command?
– user1207217
Jan 21 at 11:26
@user1207217 Correct, especially if the arguments contain characters from$IFS
or filename globbing characters.
– Kusalananda
Jan 21 at 11:29
@Kusalananda thanks, although thats a huge pain ....
– user1207217
Jan 21 at 11:30
1
@user1207217 You may want to ask a separate question about what it actually is that you are intending or trying to do.
– Kusalananda
Jan 21 at 11:31
Good idea, thanks. unix.stackexchange.com/questions/495769/…
– user1207217
Jan 21 at 11:40
Thanks. Is the short answer to this then that you cannot (reliably) use command substitution to generate arguments to another command?
– user1207217
Jan 21 at 11:26
Thanks. Is the short answer to this then that you cannot (reliably) use command substitution to generate arguments to another command?
– user1207217
Jan 21 at 11:26
@user1207217 Correct, especially if the arguments contain characters from
$IFS
or filename globbing characters.– Kusalananda
Jan 21 at 11:29
@user1207217 Correct, especially if the arguments contain characters from
$IFS
or filename globbing characters.– Kusalananda
Jan 21 at 11:29
@Kusalananda thanks, although thats a huge pain ....
– user1207217
Jan 21 at 11:30
@Kusalananda thanks, although thats a huge pain ....
– user1207217
Jan 21 at 11:30
1
1
@user1207217 You may want to ask a separate question about what it actually is that you are intending or trying to do.
– Kusalananda
Jan 21 at 11:31
@user1207217 You may want to ask a separate question about what it actually is that you are intending or trying to do.
– Kusalananda
Jan 21 at 11:31
Good idea, thanks. unix.stackexchange.com/questions/495769/…
– user1207217
Jan 21 at 11:40
Good idea, thanks. unix.stackexchange.com/questions/495769/…
– user1207217
Jan 21 at 11:40
add a comment |
TL;DR: the result of echo
is a victim of word splitting on $()
result, and sh test.sh "hello " "hi and bye"
uses a different rule.
Let's take a peak as what's actually happening, when you add set -x
for debugging:
> set -x
> ./main.sh $(echo '"hello " "hi and bye"')
++ echo '"hello " "hi and bye"'
+ ./main.sh '"hello' '"' '"hi' and 'bye"'
"hello
"
"hi
and
bye"
echo
receives "hello " "hi and bye"
as a single argument (yep, including all spaces and single quote) due to single-quotes in the command itself. But command substitution turns that into "hello " "hi and bye"
. To quote bash manual:
If the substitution appears within double quotes, word splitting and pathname expansion are not performed on the results.
However, in your case, unquoted command substitution allows exactly for word splitting to occur based on blank character (which is one of the 3 characters set in the $IFS
variable, which is what is used for word splitting). Thus, if you break the resulting string at each space, what do you have?
"hello
, then space, after which we have next token
"
- that one character by itself, isolated by spaces from both sides.
"hi
, again with space on left and rightand
bye"
all in all, you get 5 tokens broken at whitespaces. It's important to realize that word splitting, in this case, occurs exactly a result of command substitution. Later, all of these are treated as individual tokens that have been already parsed. As you see in set -x
output, the shell script is called with exactly those tokens.
As for why sh test.sh "hello " "hi and bye"
behaves differently, that's because the different rule applies. Your command-line is parsed after you enter it, and quoted strings are treated as a single unit, i.e. hello
and hi and bye
add a comment |
TL;DR: the result of echo
is a victim of word splitting on $()
result, and sh test.sh "hello " "hi and bye"
uses a different rule.
Let's take a peak as what's actually happening, when you add set -x
for debugging:
> set -x
> ./main.sh $(echo '"hello " "hi and bye"')
++ echo '"hello " "hi and bye"'
+ ./main.sh '"hello' '"' '"hi' and 'bye"'
"hello
"
"hi
and
bye"
echo
receives "hello " "hi and bye"
as a single argument (yep, including all spaces and single quote) due to single-quotes in the command itself. But command substitution turns that into "hello " "hi and bye"
. To quote bash manual:
If the substitution appears within double quotes, word splitting and pathname expansion are not performed on the results.
However, in your case, unquoted command substitution allows exactly for word splitting to occur based on blank character (which is one of the 3 characters set in the $IFS
variable, which is what is used for word splitting). Thus, if you break the resulting string at each space, what do you have?
"hello
, then space, after which we have next token
"
- that one character by itself, isolated by spaces from both sides.
"hi
, again with space on left and rightand
bye"
all in all, you get 5 tokens broken at whitespaces. It's important to realize that word splitting, in this case, occurs exactly a result of command substitution. Later, all of these are treated as individual tokens that have been already parsed. As you see in set -x
output, the shell script is called with exactly those tokens.
As for why sh test.sh "hello " "hi and bye"
behaves differently, that's because the different rule applies. Your command-line is parsed after you enter it, and quoted strings are treated as a single unit, i.e. hello
and hi and bye
add a comment |
TL;DR: the result of echo
is a victim of word splitting on $()
result, and sh test.sh "hello " "hi and bye"
uses a different rule.
Let's take a peak as what's actually happening, when you add set -x
for debugging:
> set -x
> ./main.sh $(echo '"hello " "hi and bye"')
++ echo '"hello " "hi and bye"'
+ ./main.sh '"hello' '"' '"hi' and 'bye"'
"hello
"
"hi
and
bye"
echo
receives "hello " "hi and bye"
as a single argument (yep, including all spaces and single quote) due to single-quotes in the command itself. But command substitution turns that into "hello " "hi and bye"
. To quote bash manual:
If the substitution appears within double quotes, word splitting and pathname expansion are not performed on the results.
However, in your case, unquoted command substitution allows exactly for word splitting to occur based on blank character (which is one of the 3 characters set in the $IFS
variable, which is what is used for word splitting). Thus, if you break the resulting string at each space, what do you have?
"hello
, then space, after which we have next token
"
- that one character by itself, isolated by spaces from both sides.
"hi
, again with space on left and rightand
bye"
all in all, you get 5 tokens broken at whitespaces. It's important to realize that word splitting, in this case, occurs exactly a result of command substitution. Later, all of these are treated as individual tokens that have been already parsed. As you see in set -x
output, the shell script is called with exactly those tokens.
As for why sh test.sh "hello " "hi and bye"
behaves differently, that's because the different rule applies. Your command-line is parsed after you enter it, and quoted strings are treated as a single unit, i.e. hello
and hi and bye
TL;DR: the result of echo
is a victim of word splitting on $()
result, and sh test.sh "hello " "hi and bye"
uses a different rule.
Let's take a peak as what's actually happening, when you add set -x
for debugging:
> set -x
> ./main.sh $(echo '"hello " "hi and bye"')
++ echo '"hello " "hi and bye"'
+ ./main.sh '"hello' '"' '"hi' and 'bye"'
"hello
"
"hi
and
bye"
echo
receives "hello " "hi and bye"
as a single argument (yep, including all spaces and single quote) due to single-quotes in the command itself. But command substitution turns that into "hello " "hi and bye"
. To quote bash manual:
If the substitution appears within double quotes, word splitting and pathname expansion are not performed on the results.
However, in your case, unquoted command substitution allows exactly for word splitting to occur based on blank character (which is one of the 3 characters set in the $IFS
variable, which is what is used for word splitting). Thus, if you break the resulting string at each space, what do you have?
"hello
, then space, after which we have next token
"
- that one character by itself, isolated by spaces from both sides.
"hi
, again with space on left and rightand
bye"
all in all, you get 5 tokens broken at whitespaces. It's important to realize that word splitting, in this case, occurs exactly a result of command substitution. Later, all of these are treated as individual tokens that have been already parsed. As you see in set -x
output, the shell script is called with exactly those tokens.
As for why sh test.sh "hello " "hi and bye"
behaves differently, that's because the different rule applies. Your command-line is parsed after you enter it, and quoted strings are treated as a single unit, i.e. hello
and hi and bye
edited Feb 4 at 15:01


msp9011
4,38044066
4,38044066
answered Jan 21 at 11:19


Sergiy KolodyazhnyySergiy Kolodyazhnyy
10.5k42663
10.5k42663
add a comment |
add a comment |
You could change IFS
to something other than space (since you want to protect the space in hi and bye
), and use that something to separate the two strings:
$ IFS=:
$ sh test.sh $(echo "hello ":"hi and bye")
hello
hi and bye
The order of operations (those relevant in this example, anyway) is somewhat like this:
- substitutions
- word splitting
- quote removal
Note that quotes created as a result of substitutions aren't special in word splitting or quote removal, so a quote in the output of (1) passes through (2) and (3) like any other character. So:
- In
echo "hello " "hi and bye"
, the shell removes these quotes, soecho
getshello
andhi and bye
, so outputshello hi and bye
, joining the strings with a space. - In
echo '"hello " "hi and bye"'
, the shell removes the outer'
, the echo gets"hello " "hi and bye"
and outputs that.
In
sh test.sh $(echo '"hello " "hi and bye"')
, the command substitution is replaced with"hello " "hi and bye"
, but these quotes are a result of a substitution, and so aren't involved in word splitting or quote removal.
So the shell splits those into
"hello
,"
,"hi
,and
,bye"
, hence the output you get.
- In
sh test.sh $(echo "hello " "hi and bye")
, the command substitution is replaced withhello hi and bye
, which gets split intohello
,hi
andbye
.
1
Thanks. I guess what I am not understanding is why the usual shell substitution is not occurring, i.e. why is the behaviour not the same as if I dosh test.sh "hello " "hi and bye"
?
– user1207217
Jan 21 at 11:09
@user1207217 does the edit help?
– Olorin
Jan 21 at 11:27
In$(echo "hello ":"hi and bye")
, you might as well write$(echo "hello :hi and bye")
. The colon works the same quoted or not.
– ilkkachu
Jan 21 at 11:31
add a comment |
You could change IFS
to something other than space (since you want to protect the space in hi and bye
), and use that something to separate the two strings:
$ IFS=:
$ sh test.sh $(echo "hello ":"hi and bye")
hello
hi and bye
The order of operations (those relevant in this example, anyway) is somewhat like this:
- substitutions
- word splitting
- quote removal
Note that quotes created as a result of substitutions aren't special in word splitting or quote removal, so a quote in the output of (1) passes through (2) and (3) like any other character. So:
- In
echo "hello " "hi and bye"
, the shell removes these quotes, soecho
getshello
andhi and bye
, so outputshello hi and bye
, joining the strings with a space. - In
echo '"hello " "hi and bye"'
, the shell removes the outer'
, the echo gets"hello " "hi and bye"
and outputs that.
In
sh test.sh $(echo '"hello " "hi and bye"')
, the command substitution is replaced with"hello " "hi and bye"
, but these quotes are a result of a substitution, and so aren't involved in word splitting or quote removal.
So the shell splits those into
"hello
,"
,"hi
,and
,bye"
, hence the output you get.
- In
sh test.sh $(echo "hello " "hi and bye")
, the command substitution is replaced withhello hi and bye
, which gets split intohello
,hi
andbye
.
1
Thanks. I guess what I am not understanding is why the usual shell substitution is not occurring, i.e. why is the behaviour not the same as if I dosh test.sh "hello " "hi and bye"
?
– user1207217
Jan 21 at 11:09
@user1207217 does the edit help?
– Olorin
Jan 21 at 11:27
In$(echo "hello ":"hi and bye")
, you might as well write$(echo "hello :hi and bye")
. The colon works the same quoted or not.
– ilkkachu
Jan 21 at 11:31
add a comment |
You could change IFS
to something other than space (since you want to protect the space in hi and bye
), and use that something to separate the two strings:
$ IFS=:
$ sh test.sh $(echo "hello ":"hi and bye")
hello
hi and bye
The order of operations (those relevant in this example, anyway) is somewhat like this:
- substitutions
- word splitting
- quote removal
Note that quotes created as a result of substitutions aren't special in word splitting or quote removal, so a quote in the output of (1) passes through (2) and (3) like any other character. So:
- In
echo "hello " "hi and bye"
, the shell removes these quotes, soecho
getshello
andhi and bye
, so outputshello hi and bye
, joining the strings with a space. - In
echo '"hello " "hi and bye"'
, the shell removes the outer'
, the echo gets"hello " "hi and bye"
and outputs that.
In
sh test.sh $(echo '"hello " "hi and bye"')
, the command substitution is replaced with"hello " "hi and bye"
, but these quotes are a result of a substitution, and so aren't involved in word splitting or quote removal.
So the shell splits those into
"hello
,"
,"hi
,and
,bye"
, hence the output you get.
- In
sh test.sh $(echo "hello " "hi and bye")
, the command substitution is replaced withhello hi and bye
, which gets split intohello
,hi
andbye
.
You could change IFS
to something other than space (since you want to protect the space in hi and bye
), and use that something to separate the two strings:
$ IFS=:
$ sh test.sh $(echo "hello ":"hi and bye")
hello
hi and bye
The order of operations (those relevant in this example, anyway) is somewhat like this:
- substitutions
- word splitting
- quote removal
Note that quotes created as a result of substitutions aren't special in word splitting or quote removal, so a quote in the output of (1) passes through (2) and (3) like any other character. So:
- In
echo "hello " "hi and bye"
, the shell removes these quotes, soecho
getshello
andhi and bye
, so outputshello hi and bye
, joining the strings with a space. - In
echo '"hello " "hi and bye"'
, the shell removes the outer'
, the echo gets"hello " "hi and bye"
and outputs that.
In
sh test.sh $(echo '"hello " "hi and bye"')
, the command substitution is replaced with"hello " "hi and bye"
, but these quotes are a result of a substitution, and so aren't involved in word splitting or quote removal.
So the shell splits those into
"hello
,"
,"hi
,and
,bye"
, hence the output you get.
- In
sh test.sh $(echo "hello " "hi and bye")
, the command substitution is replaced withhello hi and bye
, which gets split intohello
,hi
andbye
.
edited Jan 21 at 11:27
answered Jan 21 at 11:07


OlorinOlorin
3,8581721
3,8581721
1
Thanks. I guess what I am not understanding is why the usual shell substitution is not occurring, i.e. why is the behaviour not the same as if I dosh test.sh "hello " "hi and bye"
?
– user1207217
Jan 21 at 11:09
@user1207217 does the edit help?
– Olorin
Jan 21 at 11:27
In$(echo "hello ":"hi and bye")
, you might as well write$(echo "hello :hi and bye")
. The colon works the same quoted or not.
– ilkkachu
Jan 21 at 11:31
add a comment |
1
Thanks. I guess what I am not understanding is why the usual shell substitution is not occurring, i.e. why is the behaviour not the same as if I dosh test.sh "hello " "hi and bye"
?
– user1207217
Jan 21 at 11:09
@user1207217 does the edit help?
– Olorin
Jan 21 at 11:27
In$(echo "hello ":"hi and bye")
, you might as well write$(echo "hello :hi and bye")
. The colon works the same quoted or not.
– ilkkachu
Jan 21 at 11:31
1
1
Thanks. I guess what I am not understanding is why the usual shell substitution is not occurring, i.e. why is the behaviour not the same as if I do
sh test.sh "hello " "hi and bye"
?– user1207217
Jan 21 at 11:09
Thanks. I guess what I am not understanding is why the usual shell substitution is not occurring, i.e. why is the behaviour not the same as if I do
sh test.sh "hello " "hi and bye"
?– user1207217
Jan 21 at 11:09
@user1207217 does the edit help?
– Olorin
Jan 21 at 11:27
@user1207217 does the edit help?
– Olorin
Jan 21 at 11:27
In
$(echo "hello ":"hi and bye")
, you might as well write $(echo "hello :hi and bye")
. The colon works the same quoted or not.– ilkkachu
Jan 21 at 11:31
In
$(echo "hello ":"hi and bye")
, you might as well write $(echo "hello :hi and bye")
. The colon works the same quoted or not.– ilkkachu
Jan 21 at 11:31
add a comment |
How about
sh test.sh "hello " "hi and bye"
?
I'm aware I can do this, the shell script is a MWE of an unexpected behaviour -- I'm trying to use a command substitution to generate a long list of args (some of which contain spaces) but it doesn't work correctly because even with quoting in the command output the above is occurring, i.e. the shell is not doing its usual parsing and treating everything inside a single or double quote pair as a single arg
– user1207217
Jan 21 at 11:15
add a comment |
How about
sh test.sh "hello " "hi and bye"
?
I'm aware I can do this, the shell script is a MWE of an unexpected behaviour -- I'm trying to use a command substitution to generate a long list of args (some of which contain spaces) but it doesn't work correctly because even with quoting in the command output the above is occurring, i.e. the shell is not doing its usual parsing and treating everything inside a single or double quote pair as a single arg
– user1207217
Jan 21 at 11:15
add a comment |
How about
sh test.sh "hello " "hi and bye"
?
How about
sh test.sh "hello " "hi and bye"
?
answered Jan 21 at 11:13


gerhard d.gerhard d.
1,256311
1,256311
I'm aware I can do this, the shell script is a MWE of an unexpected behaviour -- I'm trying to use a command substitution to generate a long list of args (some of which contain spaces) but it doesn't work correctly because even with quoting in the command output the above is occurring, i.e. the shell is not doing its usual parsing and treating everything inside a single or double quote pair as a single arg
– user1207217
Jan 21 at 11:15
add a comment |
I'm aware I can do this, the shell script is a MWE of an unexpected behaviour -- I'm trying to use a command substitution to generate a long list of args (some of which contain spaces) but it doesn't work correctly because even with quoting in the command output the above is occurring, i.e. the shell is not doing its usual parsing and treating everything inside a single or double quote pair as a single arg
– user1207217
Jan 21 at 11:15
I'm aware I can do this, the shell script is a MWE of an unexpected behaviour -- I'm trying to use a command substitution to generate a long list of args (some of which contain spaces) but it doesn't work correctly because even with quoting in the command output the above is occurring, i.e. the shell is not doing its usual parsing and treating everything inside a single or double quote pair as a single arg
– user1207217
Jan 21 at 11:15
I'm aware I can do this, the shell script is a MWE of an unexpected behaviour -- I'm trying to use a command substitution to generate a long list of args (some of which contain spaces) but it doesn't work correctly because even with quoting in the command output the above is occurring, i.e. the shell is not doing its usual parsing and treating everything inside a single or double quote pair as a single arg
– user1207217
Jan 21 at 11:15
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f495755%2funexpected-behaviour-in-shell-command-substitution%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown