POSIX shell: does `$` lose its special meaning if it is the last character in a word?
On ash, dash and bash, when I run
$ echo ab$
it returns
ab$
Is this behavior specified by POSIX or is it just a common convention in POSIX-compliant shells? I couldn't find anything on the POSIX Shell Command Language page that mentions this behavior.
shell posix
add a comment |
On ash, dash and bash, when I run
$ echo ab$
it returns
ab$
Is this behavior specified by POSIX or is it just a common convention in POSIX-compliant shells? I couldn't find anything on the POSIX Shell Command Language page that mentions this behavior.
shell posix
4
The better question is "Does$
gain a special meaning if it is the last character in a word?" There is no single special meaning assigned to$
; it is used to introduce multiple, but distinct, expansions, like parameter expansion${...}
, command substitution$(...)
, and arithmetic expressions$((...))
. Some shells introduce additional contexts, likeksh
's command-substitution variantx=${ echo foo; echo bar;}
(which differs from the standard$(...)
by not executing the commands in a subshell).
– chepner
Jan 4 at 15:45
@chepner Would you care to weigh in on the difference of opinion between Issac and Michael Homer? Their answers explicitly contradict each other
– Harold Fischer
Jan 6 at 2:15
1
I agree with Michael Homer's interpretation; the shell doesn't even start to worry about expansions until after parsing is complete, so in the single wordab$
, there is no character following$
, whether it was "followed" by a null character in the original input string or a space in a case likeecho ab$ foo
; the original unquoted whitespace has been recognized and discarded after parsing.
– chepner
Jan 6 at 14:14
add a comment |
On ash, dash and bash, when I run
$ echo ab$
it returns
ab$
Is this behavior specified by POSIX or is it just a common convention in POSIX-compliant shells? I couldn't find anything on the POSIX Shell Command Language page that mentions this behavior.
shell posix
On ash, dash and bash, when I run
$ echo ab$
it returns
ab$
Is this behavior specified by POSIX or is it just a common convention in POSIX-compliant shells? I couldn't find anything on the POSIX Shell Command Language page that mentions this behavior.
shell posix
shell posix
edited Jan 4 at 8:11
Sparhawk
9,38763992
9,38763992
asked Jan 4 at 7:12
Harold FischerHarold Fischer
656315
656315
4
The better question is "Does$
gain a special meaning if it is the last character in a word?" There is no single special meaning assigned to$
; it is used to introduce multiple, but distinct, expansions, like parameter expansion${...}
, command substitution$(...)
, and arithmetic expressions$((...))
. Some shells introduce additional contexts, likeksh
's command-substitution variantx=${ echo foo; echo bar;}
(which differs from the standard$(...)
by not executing the commands in a subshell).
– chepner
Jan 4 at 15:45
@chepner Would you care to weigh in on the difference of opinion between Issac and Michael Homer? Their answers explicitly contradict each other
– Harold Fischer
Jan 6 at 2:15
1
I agree with Michael Homer's interpretation; the shell doesn't even start to worry about expansions until after parsing is complete, so in the single wordab$
, there is no character following$
, whether it was "followed" by a null character in the original input string or a space in a case likeecho ab$ foo
; the original unquoted whitespace has been recognized and discarded after parsing.
– chepner
Jan 6 at 14:14
add a comment |
4
The better question is "Does$
gain a special meaning if it is the last character in a word?" There is no single special meaning assigned to$
; it is used to introduce multiple, but distinct, expansions, like parameter expansion${...}
, command substitution$(...)
, and arithmetic expressions$((...))
. Some shells introduce additional contexts, likeksh
's command-substitution variantx=${ echo foo; echo bar;}
(which differs from the standard$(...)
by not executing the commands in a subshell).
– chepner
Jan 4 at 15:45
@chepner Would you care to weigh in on the difference of opinion between Issac and Michael Homer? Their answers explicitly contradict each other
– Harold Fischer
Jan 6 at 2:15
1
I agree with Michael Homer's interpretation; the shell doesn't even start to worry about expansions until after parsing is complete, so in the single wordab$
, there is no character following$
, whether it was "followed" by a null character in the original input string or a space in a case likeecho ab$ foo
; the original unquoted whitespace has been recognized and discarded after parsing.
– chepner
Jan 6 at 14:14
4
4
The better question is "Does
$
gain a special meaning if it is the last character in a word?" There is no single special meaning assigned to $
; it is used to introduce multiple, but distinct, expansions, like parameter expansion ${...}
, command substitution $(...)
, and arithmetic expressions $((...))
. Some shells introduce additional contexts, like ksh
's command-substitution variant x=${ echo foo; echo bar;}
(which differs from the standard $(...)
by not executing the commands in a subshell).– chepner
Jan 4 at 15:45
The better question is "Does
$
gain a special meaning if it is the last character in a word?" There is no single special meaning assigned to $
; it is used to introduce multiple, but distinct, expansions, like parameter expansion ${...}
, command substitution $(...)
, and arithmetic expressions $((...))
. Some shells introduce additional contexts, like ksh
's command-substitution variant x=${ echo foo; echo bar;}
(which differs from the standard $(...)
by not executing the commands in a subshell).– chepner
Jan 4 at 15:45
@chepner Would you care to weigh in on the difference of opinion between Issac and Michael Homer? Their answers explicitly contradict each other
– Harold Fischer
Jan 6 at 2:15
@chepner Would you care to weigh in on the difference of opinion between Issac and Michael Homer? Their answers explicitly contradict each other
– Harold Fischer
Jan 6 at 2:15
1
1
I agree with Michael Homer's interpretation; the shell doesn't even start to worry about expansions until after parsing is complete, so in the single word
ab$
, there is no character following $
, whether it was "followed" by a null character in the original input string or a space in a case like echo ab$ foo
; the original unquoted whitespace has been recognized and discarded after parsing.– chepner
Jan 6 at 14:14
I agree with Michael Homer's interpretation; the shell doesn't even start to worry about expansions until after parsing is complete, so in the single word
ab$
, there is no character following $
, whether it was "followed" by a null character in the original input string or a space in a case like echo ab$ foo
; the original unquoted whitespace has been recognized and discarded after parsing.– chepner
Jan 6 at 14:14
add a comment |
2 Answers
2
active
oldest
votes
$
does not have a special meaning by itself (try echo $
), only when combined with other character after it and forming an expansion, e.g. $var
(or ${var}
), $(util)
, $((1+2))
.
The $
gets its "special" meaning as defining an expansion in the POSIX standard under the section Token Recognition:
If the current character is an unquoted
$
or`
, the shell shall identify the start of any candidates for parameter expansion, command substitution, or arithmetic expansion from their introductory unquoted character sequences:$
or${
,$(
or`
, and$((
, respectively. The shell shall read sufficient input to determine the end of the unit to be expanded (as explained in the cited sections). While processing the characters, if instances of expansions or quoting are found nested within the substitution, the shell shall recursively process them in the manner specified for the construct that is found. The characters found from the beginning of the substitution to its end, allowing for any recursion necessary to recognize embedded constructs, shall be included unmodified in the result token, including any embedded or enclosing substitution operators or quotes. The token shall not be delimited by the end of the substitution.
So, if $
does not form an expansion, other parsing rules come into effect:
If the previous character was part of a word, the current character shall be appended to that word.
That covers your ab$
string.
In the case of a lone $
(the "new word" would be the $
by itself):
The current character is used as the start of a new word.
The meaning of the generated word containing a $
that is not a standard expansion is explicitly defined as unspecified by POSIX.
Also note that $
is the last character in $$
, but that this also happens to be the variable that holds the current shell's PID. In bash
, !$
may invoke a history expansion (the last argument af the previous command). So in general, no, $
is not without meaning at the end of an unquoted word, but at the end of a word it does at least not denote a standard expansion.
add a comment |
Depending on the exact situation, this is either explicitly unspecified (so implementations may do as they will) or required to happen as you observed. In your exact scenario echo ab$
, POSIX mandates the output "ab$" that you observed and it is not unspecified. A quick summary of all the different cases is at the end.
There are two elements: first tokenising into words, and then interpretation of those words.
Tokenisation
POSIX tokenisation requires that a $
that is not the start of a valid parameter expansion, command substitution, or arithmetic substitution to be considered a literal part of the WORD
token being constructed. This is because rule 5 ("If the current character is an unquoted $
or `
, the shell shall identify the start of any candidates for parameter expansion, command substitution, or arithmetic expansion from their introductory unquoted character sequences: $
or ${
, $(
or `
, and $((
, respectively") does not apply, as none of those expansions are viable there. Parameter expansion requires a valid name to appear there, and an empty name is not valid.
Since this rule did not apply, we continue following until we find one that does. The two candidates are #8 ("If the previous character was part of a word, the current character shall be appended to that word.") and #10 ("The current character is used as the start of a new word."), which apply to echo a$
and echo $
respectively.
There is also a third case of the form echo a$+b
which falls through the same crack, since +
is not the name of a special parameter. This one we'll return to later, since it triggers different parts of the rules.
The specification thus requires that the $
be considered a part of the word syntactically, and it can then be further processed later on.
Word expansion
After the input has been parsed in this way, with the $
included in the word, word expansions are applied to each of the words that have been read. Each word is processed individually.
It is specified that:
If an unquoted '$' is followed by a character that is not one of the following:
- A numeric character
- The name of one of the special parameters (see Special Parameters)
- A valid first character of a variable name
- A
<left-curly-bracket>
( '{' )
- A
<left-parenthesis>
the result is unspecified.
"Unspecified" is a particular term here meaning that
- A conforming shell can choose any behaviour in this case
- A conforming application cannot rely on any particular behaviour
In your example, echo ab$
, the $
is not followed by any character, so this rule does not apply and the unspecified result is not invoked. There is simply no expansion incited by the $
, so it is literally present and printed out.
Where it would apply is in our third case from above: echo a$+b
. Here $
is followed by +
, which is not a number, special parameter (@
, *
, #
, ?
, -
, $
, !
, or 0
), start of a variable name (underscore or an alphabetic from the portable character set), or one of the brackets. In this case, the behaviour is unspecified: a conforming shell is permitted to invent a special parameter called +
to expand, and a conforming application should not assume that the shell does not. The shell could do anything else it liked as well, including reporting an error.
For example, zsh, including in its POSIX mode, interprets $+b
as "is variable b
set" and substitutes either 1 or 0 in its place. It similarly has extensions for ~
and =
. This is conforming behaviour.
Another place this could happen is echo "a$ b"
. Again, the shell is permitted to do as it wishes, and you as the script author should escape the $
if you want literal output. If you don't, it may work, but you can't rely on it. This is the absolute letter of the specification, but I don't think this sort of granularity was intended or considered.
In summary
echo ab$
: literal output, fully specified
echo a$ b
: literal output, fully specified
echo a$ b$
: literal output, fully specified
echo a$b
: expansion of parameterb
, fully specified
echo a$-b
: expansion of special parameter-
, fully specified
echo a$+b
: unspecified behaviour
echo "a$ b"
: unspecified behaviour
For a $
at the end of a word, you are permitted to rely on the behaviour and it must be treated literally and passed on to the echo
command as part of its argument. That is a conformance requirement on the shell.
Comments are not for extended discussion; this conversation has been moved to chat.
– terdon♦
Jan 6 at 13:25
@MichaelHomer Wouldecho $
also be literal and fully specified?
– Harold Fischer
Jan 6 at 15:15
@HaroldFischer Yes
– Michael Homer
Jan 6 at 17:44
Where doecho "$"
andecho "a b$"
fall?
– Harold Fischer
Jan 7 at 4:20
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f492397%2fposix-shell-does-lose-its-special-meaning-if-it-is-the-last-character-in-a%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
$
does not have a special meaning by itself (try echo $
), only when combined with other character after it and forming an expansion, e.g. $var
(or ${var}
), $(util)
, $((1+2))
.
The $
gets its "special" meaning as defining an expansion in the POSIX standard under the section Token Recognition:
If the current character is an unquoted
$
or`
, the shell shall identify the start of any candidates for parameter expansion, command substitution, or arithmetic expansion from their introductory unquoted character sequences:$
or${
,$(
or`
, and$((
, respectively. The shell shall read sufficient input to determine the end of the unit to be expanded (as explained in the cited sections). While processing the characters, if instances of expansions or quoting are found nested within the substitution, the shell shall recursively process them in the manner specified for the construct that is found. The characters found from the beginning of the substitution to its end, allowing for any recursion necessary to recognize embedded constructs, shall be included unmodified in the result token, including any embedded or enclosing substitution operators or quotes. The token shall not be delimited by the end of the substitution.
So, if $
does not form an expansion, other parsing rules come into effect:
If the previous character was part of a word, the current character shall be appended to that word.
That covers your ab$
string.
In the case of a lone $
(the "new word" would be the $
by itself):
The current character is used as the start of a new word.
The meaning of the generated word containing a $
that is not a standard expansion is explicitly defined as unspecified by POSIX.
Also note that $
is the last character in $$
, but that this also happens to be the variable that holds the current shell's PID. In bash
, !$
may invoke a history expansion (the last argument af the previous command). So in general, no, $
is not without meaning at the end of an unquoted word, but at the end of a word it does at least not denote a standard expansion.
add a comment |
$
does not have a special meaning by itself (try echo $
), only when combined with other character after it and forming an expansion, e.g. $var
(or ${var}
), $(util)
, $((1+2))
.
The $
gets its "special" meaning as defining an expansion in the POSIX standard under the section Token Recognition:
If the current character is an unquoted
$
or`
, the shell shall identify the start of any candidates for parameter expansion, command substitution, or arithmetic expansion from their introductory unquoted character sequences:$
or${
,$(
or`
, and$((
, respectively. The shell shall read sufficient input to determine the end of the unit to be expanded (as explained in the cited sections). While processing the characters, if instances of expansions or quoting are found nested within the substitution, the shell shall recursively process them in the manner specified for the construct that is found. The characters found from the beginning of the substitution to its end, allowing for any recursion necessary to recognize embedded constructs, shall be included unmodified in the result token, including any embedded or enclosing substitution operators or quotes. The token shall not be delimited by the end of the substitution.
So, if $
does not form an expansion, other parsing rules come into effect:
If the previous character was part of a word, the current character shall be appended to that word.
That covers your ab$
string.
In the case of a lone $
(the "new word" would be the $
by itself):
The current character is used as the start of a new word.
The meaning of the generated word containing a $
that is not a standard expansion is explicitly defined as unspecified by POSIX.
Also note that $
is the last character in $$
, but that this also happens to be the variable that holds the current shell's PID. In bash
, !$
may invoke a history expansion (the last argument af the previous command). So in general, no, $
is not without meaning at the end of an unquoted word, but at the end of a word it does at least not denote a standard expansion.
add a comment |
$
does not have a special meaning by itself (try echo $
), only when combined with other character after it and forming an expansion, e.g. $var
(or ${var}
), $(util)
, $((1+2))
.
The $
gets its "special" meaning as defining an expansion in the POSIX standard under the section Token Recognition:
If the current character is an unquoted
$
or`
, the shell shall identify the start of any candidates for parameter expansion, command substitution, or arithmetic expansion from their introductory unquoted character sequences:$
or${
,$(
or`
, and$((
, respectively. The shell shall read sufficient input to determine the end of the unit to be expanded (as explained in the cited sections). While processing the characters, if instances of expansions or quoting are found nested within the substitution, the shell shall recursively process them in the manner specified for the construct that is found. The characters found from the beginning of the substitution to its end, allowing for any recursion necessary to recognize embedded constructs, shall be included unmodified in the result token, including any embedded or enclosing substitution operators or quotes. The token shall not be delimited by the end of the substitution.
So, if $
does not form an expansion, other parsing rules come into effect:
If the previous character was part of a word, the current character shall be appended to that word.
That covers your ab$
string.
In the case of a lone $
(the "new word" would be the $
by itself):
The current character is used as the start of a new word.
The meaning of the generated word containing a $
that is not a standard expansion is explicitly defined as unspecified by POSIX.
Also note that $
is the last character in $$
, but that this also happens to be the variable that holds the current shell's PID. In bash
, !$
may invoke a history expansion (the last argument af the previous command). So in general, no, $
is not without meaning at the end of an unquoted word, but at the end of a word it does at least not denote a standard expansion.
$
does not have a special meaning by itself (try echo $
), only when combined with other character after it and forming an expansion, e.g. $var
(or ${var}
), $(util)
, $((1+2))
.
The $
gets its "special" meaning as defining an expansion in the POSIX standard under the section Token Recognition:
If the current character is an unquoted
$
or`
, the shell shall identify the start of any candidates for parameter expansion, command substitution, or arithmetic expansion from their introductory unquoted character sequences:$
or${
,$(
or`
, and$((
, respectively. The shell shall read sufficient input to determine the end of the unit to be expanded (as explained in the cited sections). While processing the characters, if instances of expansions or quoting are found nested within the substitution, the shell shall recursively process them in the manner specified for the construct that is found. The characters found from the beginning of the substitution to its end, allowing for any recursion necessary to recognize embedded constructs, shall be included unmodified in the result token, including any embedded or enclosing substitution operators or quotes. The token shall not be delimited by the end of the substitution.
So, if $
does not form an expansion, other parsing rules come into effect:
If the previous character was part of a word, the current character shall be appended to that word.
That covers your ab$
string.
In the case of a lone $
(the "new word" would be the $
by itself):
The current character is used as the start of a new word.
The meaning of the generated word containing a $
that is not a standard expansion is explicitly defined as unspecified by POSIX.
Also note that $
is the last character in $$
, but that this also happens to be the variable that holds the current shell's PID. In bash
, !$
may invoke a history expansion (the last argument af the previous command). So in general, no, $
is not without meaning at the end of an unquoted word, but at the end of a word it does at least not denote a standard expansion.
edited Jan 6 at 21:59
Isaac
11.4k11652
11.4k11652
answered Jan 4 at 7:55
KusalanandaKusalananda
124k16234385
124k16234385
add a comment |
add a comment |
Depending on the exact situation, this is either explicitly unspecified (so implementations may do as they will) or required to happen as you observed. In your exact scenario echo ab$
, POSIX mandates the output "ab$" that you observed and it is not unspecified. A quick summary of all the different cases is at the end.
There are two elements: first tokenising into words, and then interpretation of those words.
Tokenisation
POSIX tokenisation requires that a $
that is not the start of a valid parameter expansion, command substitution, or arithmetic substitution to be considered a literal part of the WORD
token being constructed. This is because rule 5 ("If the current character is an unquoted $
or `
, the shell shall identify the start of any candidates for parameter expansion, command substitution, or arithmetic expansion from their introductory unquoted character sequences: $
or ${
, $(
or `
, and $((
, respectively") does not apply, as none of those expansions are viable there. Parameter expansion requires a valid name to appear there, and an empty name is not valid.
Since this rule did not apply, we continue following until we find one that does. The two candidates are #8 ("If the previous character was part of a word, the current character shall be appended to that word.") and #10 ("The current character is used as the start of a new word."), which apply to echo a$
and echo $
respectively.
There is also a third case of the form echo a$+b
which falls through the same crack, since +
is not the name of a special parameter. This one we'll return to later, since it triggers different parts of the rules.
The specification thus requires that the $
be considered a part of the word syntactically, and it can then be further processed later on.
Word expansion
After the input has been parsed in this way, with the $
included in the word, word expansions are applied to each of the words that have been read. Each word is processed individually.
It is specified that:
If an unquoted '$' is followed by a character that is not one of the following:
- A numeric character
- The name of one of the special parameters (see Special Parameters)
- A valid first character of a variable name
- A
<left-curly-bracket>
( '{' )
- A
<left-parenthesis>
the result is unspecified.
"Unspecified" is a particular term here meaning that
- A conforming shell can choose any behaviour in this case
- A conforming application cannot rely on any particular behaviour
In your example, echo ab$
, the $
is not followed by any character, so this rule does not apply and the unspecified result is not invoked. There is simply no expansion incited by the $
, so it is literally present and printed out.
Where it would apply is in our third case from above: echo a$+b
. Here $
is followed by +
, which is not a number, special parameter (@
, *
, #
, ?
, -
, $
, !
, or 0
), start of a variable name (underscore or an alphabetic from the portable character set), or one of the brackets. In this case, the behaviour is unspecified: a conforming shell is permitted to invent a special parameter called +
to expand, and a conforming application should not assume that the shell does not. The shell could do anything else it liked as well, including reporting an error.
For example, zsh, including in its POSIX mode, interprets $+b
as "is variable b
set" and substitutes either 1 or 0 in its place. It similarly has extensions for ~
and =
. This is conforming behaviour.
Another place this could happen is echo "a$ b"
. Again, the shell is permitted to do as it wishes, and you as the script author should escape the $
if you want literal output. If you don't, it may work, but you can't rely on it. This is the absolute letter of the specification, but I don't think this sort of granularity was intended or considered.
In summary
echo ab$
: literal output, fully specified
echo a$ b
: literal output, fully specified
echo a$ b$
: literal output, fully specified
echo a$b
: expansion of parameterb
, fully specified
echo a$-b
: expansion of special parameter-
, fully specified
echo a$+b
: unspecified behaviour
echo "a$ b"
: unspecified behaviour
For a $
at the end of a word, you are permitted to rely on the behaviour and it must be treated literally and passed on to the echo
command as part of its argument. That is a conformance requirement on the shell.
Comments are not for extended discussion; this conversation has been moved to chat.
– terdon♦
Jan 6 at 13:25
@MichaelHomer Wouldecho $
also be literal and fully specified?
– Harold Fischer
Jan 6 at 15:15
@HaroldFischer Yes
– Michael Homer
Jan 6 at 17:44
Where doecho "$"
andecho "a b$"
fall?
– Harold Fischer
Jan 7 at 4:20
add a comment |
Depending on the exact situation, this is either explicitly unspecified (so implementations may do as they will) or required to happen as you observed. In your exact scenario echo ab$
, POSIX mandates the output "ab$" that you observed and it is not unspecified. A quick summary of all the different cases is at the end.
There are two elements: first tokenising into words, and then interpretation of those words.
Tokenisation
POSIX tokenisation requires that a $
that is not the start of a valid parameter expansion, command substitution, or arithmetic substitution to be considered a literal part of the WORD
token being constructed. This is because rule 5 ("If the current character is an unquoted $
or `
, the shell shall identify the start of any candidates for parameter expansion, command substitution, or arithmetic expansion from their introductory unquoted character sequences: $
or ${
, $(
or `
, and $((
, respectively") does not apply, as none of those expansions are viable there. Parameter expansion requires a valid name to appear there, and an empty name is not valid.
Since this rule did not apply, we continue following until we find one that does. The two candidates are #8 ("If the previous character was part of a word, the current character shall be appended to that word.") and #10 ("The current character is used as the start of a new word."), which apply to echo a$
and echo $
respectively.
There is also a third case of the form echo a$+b
which falls through the same crack, since +
is not the name of a special parameter. This one we'll return to later, since it triggers different parts of the rules.
The specification thus requires that the $
be considered a part of the word syntactically, and it can then be further processed later on.
Word expansion
After the input has been parsed in this way, with the $
included in the word, word expansions are applied to each of the words that have been read. Each word is processed individually.
It is specified that:
If an unquoted '$' is followed by a character that is not one of the following:
- A numeric character
- The name of one of the special parameters (see Special Parameters)
- A valid first character of a variable name
- A
<left-curly-bracket>
( '{' )
- A
<left-parenthesis>
the result is unspecified.
"Unspecified" is a particular term here meaning that
- A conforming shell can choose any behaviour in this case
- A conforming application cannot rely on any particular behaviour
In your example, echo ab$
, the $
is not followed by any character, so this rule does not apply and the unspecified result is not invoked. There is simply no expansion incited by the $
, so it is literally present and printed out.
Where it would apply is in our third case from above: echo a$+b
. Here $
is followed by +
, which is not a number, special parameter (@
, *
, #
, ?
, -
, $
, !
, or 0
), start of a variable name (underscore or an alphabetic from the portable character set), or one of the brackets. In this case, the behaviour is unspecified: a conforming shell is permitted to invent a special parameter called +
to expand, and a conforming application should not assume that the shell does not. The shell could do anything else it liked as well, including reporting an error.
For example, zsh, including in its POSIX mode, interprets $+b
as "is variable b
set" and substitutes either 1 or 0 in its place. It similarly has extensions for ~
and =
. This is conforming behaviour.
Another place this could happen is echo "a$ b"
. Again, the shell is permitted to do as it wishes, and you as the script author should escape the $
if you want literal output. If you don't, it may work, but you can't rely on it. This is the absolute letter of the specification, but I don't think this sort of granularity was intended or considered.
In summary
echo ab$
: literal output, fully specified
echo a$ b
: literal output, fully specified
echo a$ b$
: literal output, fully specified
echo a$b
: expansion of parameterb
, fully specified
echo a$-b
: expansion of special parameter-
, fully specified
echo a$+b
: unspecified behaviour
echo "a$ b"
: unspecified behaviour
For a $
at the end of a word, you are permitted to rely on the behaviour and it must be treated literally and passed on to the echo
command as part of its argument. That is a conformance requirement on the shell.
Comments are not for extended discussion; this conversation has been moved to chat.
– terdon♦
Jan 6 at 13:25
@MichaelHomer Wouldecho $
also be literal and fully specified?
– Harold Fischer
Jan 6 at 15:15
@HaroldFischer Yes
– Michael Homer
Jan 6 at 17:44
Where doecho "$"
andecho "a b$"
fall?
– Harold Fischer
Jan 7 at 4:20
add a comment |
Depending on the exact situation, this is either explicitly unspecified (so implementations may do as they will) or required to happen as you observed. In your exact scenario echo ab$
, POSIX mandates the output "ab$" that you observed and it is not unspecified. A quick summary of all the different cases is at the end.
There are two elements: first tokenising into words, and then interpretation of those words.
Tokenisation
POSIX tokenisation requires that a $
that is not the start of a valid parameter expansion, command substitution, or arithmetic substitution to be considered a literal part of the WORD
token being constructed. This is because rule 5 ("If the current character is an unquoted $
or `
, the shell shall identify the start of any candidates for parameter expansion, command substitution, or arithmetic expansion from their introductory unquoted character sequences: $
or ${
, $(
or `
, and $((
, respectively") does not apply, as none of those expansions are viable there. Parameter expansion requires a valid name to appear there, and an empty name is not valid.
Since this rule did not apply, we continue following until we find one that does. The two candidates are #8 ("If the previous character was part of a word, the current character shall be appended to that word.") and #10 ("The current character is used as the start of a new word."), which apply to echo a$
and echo $
respectively.
There is also a third case of the form echo a$+b
which falls through the same crack, since +
is not the name of a special parameter. This one we'll return to later, since it triggers different parts of the rules.
The specification thus requires that the $
be considered a part of the word syntactically, and it can then be further processed later on.
Word expansion
After the input has been parsed in this way, with the $
included in the word, word expansions are applied to each of the words that have been read. Each word is processed individually.
It is specified that:
If an unquoted '$' is followed by a character that is not one of the following:
- A numeric character
- The name of one of the special parameters (see Special Parameters)
- A valid first character of a variable name
- A
<left-curly-bracket>
( '{' )
- A
<left-parenthesis>
the result is unspecified.
"Unspecified" is a particular term here meaning that
- A conforming shell can choose any behaviour in this case
- A conforming application cannot rely on any particular behaviour
In your example, echo ab$
, the $
is not followed by any character, so this rule does not apply and the unspecified result is not invoked. There is simply no expansion incited by the $
, so it is literally present and printed out.
Where it would apply is in our third case from above: echo a$+b
. Here $
is followed by +
, which is not a number, special parameter (@
, *
, #
, ?
, -
, $
, !
, or 0
), start of a variable name (underscore or an alphabetic from the portable character set), or one of the brackets. In this case, the behaviour is unspecified: a conforming shell is permitted to invent a special parameter called +
to expand, and a conforming application should not assume that the shell does not. The shell could do anything else it liked as well, including reporting an error.
For example, zsh, including in its POSIX mode, interprets $+b
as "is variable b
set" and substitutes either 1 or 0 in its place. It similarly has extensions for ~
and =
. This is conforming behaviour.
Another place this could happen is echo "a$ b"
. Again, the shell is permitted to do as it wishes, and you as the script author should escape the $
if you want literal output. If you don't, it may work, but you can't rely on it. This is the absolute letter of the specification, but I don't think this sort of granularity was intended or considered.
In summary
echo ab$
: literal output, fully specified
echo a$ b
: literal output, fully specified
echo a$ b$
: literal output, fully specified
echo a$b
: expansion of parameterb
, fully specified
echo a$-b
: expansion of special parameter-
, fully specified
echo a$+b
: unspecified behaviour
echo "a$ b"
: unspecified behaviour
For a $
at the end of a word, you are permitted to rely on the behaviour and it must be treated literally and passed on to the echo
command as part of its argument. That is a conformance requirement on the shell.
Depending on the exact situation, this is either explicitly unspecified (so implementations may do as they will) or required to happen as you observed. In your exact scenario echo ab$
, POSIX mandates the output "ab$" that you observed and it is not unspecified. A quick summary of all the different cases is at the end.
There are two elements: first tokenising into words, and then interpretation of those words.
Tokenisation
POSIX tokenisation requires that a $
that is not the start of a valid parameter expansion, command substitution, or arithmetic substitution to be considered a literal part of the WORD
token being constructed. This is because rule 5 ("If the current character is an unquoted $
or `
, the shell shall identify the start of any candidates for parameter expansion, command substitution, or arithmetic expansion from their introductory unquoted character sequences: $
or ${
, $(
or `
, and $((
, respectively") does not apply, as none of those expansions are viable there. Parameter expansion requires a valid name to appear there, and an empty name is not valid.
Since this rule did not apply, we continue following until we find one that does. The two candidates are #8 ("If the previous character was part of a word, the current character shall be appended to that word.") and #10 ("The current character is used as the start of a new word."), which apply to echo a$
and echo $
respectively.
There is also a third case of the form echo a$+b
which falls through the same crack, since +
is not the name of a special parameter. This one we'll return to later, since it triggers different parts of the rules.
The specification thus requires that the $
be considered a part of the word syntactically, and it can then be further processed later on.
Word expansion
After the input has been parsed in this way, with the $
included in the word, word expansions are applied to each of the words that have been read. Each word is processed individually.
It is specified that:
If an unquoted '$' is followed by a character that is not one of the following:
- A numeric character
- The name of one of the special parameters (see Special Parameters)
- A valid first character of a variable name
- A
<left-curly-bracket>
( '{' )
- A
<left-parenthesis>
the result is unspecified.
"Unspecified" is a particular term here meaning that
- A conforming shell can choose any behaviour in this case
- A conforming application cannot rely on any particular behaviour
In your example, echo ab$
, the $
is not followed by any character, so this rule does not apply and the unspecified result is not invoked. There is simply no expansion incited by the $
, so it is literally present and printed out.
Where it would apply is in our third case from above: echo a$+b
. Here $
is followed by +
, which is not a number, special parameter (@
, *
, #
, ?
, -
, $
, !
, or 0
), start of a variable name (underscore or an alphabetic from the portable character set), or one of the brackets. In this case, the behaviour is unspecified: a conforming shell is permitted to invent a special parameter called +
to expand, and a conforming application should not assume that the shell does not. The shell could do anything else it liked as well, including reporting an error.
For example, zsh, including in its POSIX mode, interprets $+b
as "is variable b
set" and substitutes either 1 or 0 in its place. It similarly has extensions for ~
and =
. This is conforming behaviour.
Another place this could happen is echo "a$ b"
. Again, the shell is permitted to do as it wishes, and you as the script author should escape the $
if you want literal output. If you don't, it may work, but you can't rely on it. This is the absolute letter of the specification, but I don't think this sort of granularity was intended or considered.
In summary
echo ab$
: literal output, fully specified
echo a$ b
: literal output, fully specified
echo a$ b$
: literal output, fully specified
echo a$b
: expansion of parameterb
, fully specified
echo a$-b
: expansion of special parameter-
, fully specified
echo a$+b
: unspecified behaviour
echo "a$ b"
: unspecified behaviour
For a $
at the end of a word, you are permitted to rely on the behaviour and it must be treated literally and passed on to the echo
command as part of its argument. That is a conformance requirement on the shell.
edited Jan 4 at 23:05
answered Jan 4 at 21:40
Michael HomerMichael Homer
46.5k8122161
46.5k8122161
Comments are not for extended discussion; this conversation has been moved to chat.
– terdon♦
Jan 6 at 13:25
@MichaelHomer Wouldecho $
also be literal and fully specified?
– Harold Fischer
Jan 6 at 15:15
@HaroldFischer Yes
– Michael Homer
Jan 6 at 17:44
Where doecho "$"
andecho "a b$"
fall?
– Harold Fischer
Jan 7 at 4:20
add a comment |
Comments are not for extended discussion; this conversation has been moved to chat.
– terdon♦
Jan 6 at 13:25
@MichaelHomer Wouldecho $
also be literal and fully specified?
– Harold Fischer
Jan 6 at 15:15
@HaroldFischer Yes
– Michael Homer
Jan 6 at 17:44
Where doecho "$"
andecho "a b$"
fall?
– Harold Fischer
Jan 7 at 4:20
Comments are not for extended discussion; this conversation has been moved to chat.
– terdon♦
Jan 6 at 13:25
Comments are not for extended discussion; this conversation has been moved to chat.
– terdon♦
Jan 6 at 13:25
@MichaelHomer Would
echo $
also be literal and fully specified?– Harold Fischer
Jan 6 at 15:15
@MichaelHomer Would
echo $
also be literal and fully specified?– Harold Fischer
Jan 6 at 15:15
@HaroldFischer Yes
– Michael Homer
Jan 6 at 17:44
@HaroldFischer Yes
– Michael Homer
Jan 6 at 17:44
Where do
echo "$"
and echo "a b$"
fall?– Harold Fischer
Jan 7 at 4:20
Where do
echo "$"
and echo "a b$"
fall?– Harold Fischer
Jan 7 at 4:20
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f492397%2fposix-shell-does-lose-its-special-meaning-if-it-is-the-last-character-in-a%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
4
The better question is "Does
$
gain a special meaning if it is the last character in a word?" There is no single special meaning assigned to$
; it is used to introduce multiple, but distinct, expansions, like parameter expansion${...}
, command substitution$(...)
, and arithmetic expressions$((...))
. Some shells introduce additional contexts, likeksh
's command-substitution variantx=${ echo foo; echo bar;}
(which differs from the standard$(...)
by not executing the commands in a subshell).– chepner
Jan 4 at 15:45
@chepner Would you care to weigh in on the difference of opinion between Issac and Michael Homer? Their answers explicitly contradict each other
– Harold Fischer
Jan 6 at 2:15
1
I agree with Michael Homer's interpretation; the shell doesn't even start to worry about expansions until after parsing is complete, so in the single word
ab$
, there is no character following$
, whether it was "followed" by a null character in the original input string or a space in a case likeecho ab$ foo
; the original unquoted whitespace has been recognized and discarded after parsing.– chepner
Jan 6 at 14:14