I just assigned a variable, but echo $variable shows something elseHow do I preserve newlines in a quoted string in Bash?grep output prints only single line in bash scriptBash: preserve string with spaces input on command line?How to store output from printf with formatting in a variable?How to preserve spaces when outputting a shell variable?echo trims multiple spaces from a Bash variableecho removes leading whitespace in a Bash variableHow to write a multiline string to a file in BashBash script echo seems to remove my new lines?In bash how to insert a variable in a grep command and affect output to another?sudo echo “something” >> /etc/privilegedFile doesn't workHow to assign a heredoc value to a variable in Bash?Assigning default values to shell variables with a single command in bashCommand not found error in Bash variable assignmentHow to assign the output of a Bash command to a variable?Read a file line by line assigning the value to a variableShell script: setting title for new gnome-terminal tabcorrect bash parsing of unquoted file arguments containing backslash spaces instead of quotesBash script: paste and execute an external bash scriptHow can you echo the 'output' of echoing a variable in bash?
Can anyone give me the reason why music is taught this way?
Avoiding dust scattering when you drill
Can I cast Death Ward on additional creatures without causing previous castings to end?
Everyone Gets a Window Seat
Can a passenger predict that an airline or a tour operator is about to go bankrupt?
Why most footers have a background color as a divider of section?
Why has Speaker Pelosi been so hesitant to impeach President Trump?
Shell Sort, Insertion Sort, Bubble Sort, Selection Sort Algorithms (Python)
How to interpret the challenge rating of creatures?
MaxCounters solution in C# from Codility
All over the place
Would an object shot from earth fall into the sun?
What does a textbook look like while you are writing it?
How dangerous is a very out-of-true disc brake wheel?
How to find places to store/land a private airplane?
How to identify whether a publisher is genuine or not?
How important is knowledge of trig identities for use in Calculus
SOQL injection vulnerability issue
French license plates
How to "Start as close to the end as possible", and why to do so?
Should I be an author on another PhD student's paper if I went to their meetings and gave advice?
Looking for circuit board material that can be dissolved
What does the triangle look like in this diagram?
A word that refers to saying something in an attempt to anger or embarrass someone into doing something that they don’t want to do?
I just assigned a variable, but echo $variable shows something else
How do I preserve newlines in a quoted string in Bash?grep output prints only single line in bash scriptBash: preserve string with spaces input on command line?How to store output from printf with formatting in a variable?How to preserve spaces when outputting a shell variable?echo trims multiple spaces from a Bash variableecho removes leading whitespace in a Bash variableHow to write a multiline string to a file in BashBash script echo seems to remove my new lines?In bash how to insert a variable in a grep command and affect output to another?sudo echo “something” >> /etc/privilegedFile doesn't workHow to assign a heredoc value to a variable in Bash?Assigning default values to shell variables with a single command in bashCommand not found error in Bash variable assignmentHow to assign the output of a Bash command to a variable?Read a file line by line assigning the value to a variableShell script: setting title for new gnome-terminal tabcorrect bash parsing of unquoted file arguments containing backslash spaces instead of quotesBash script: paste and execute an external bash scriptHow can you echo the 'output' of echoing a variable in bash?
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty
margin-bottom:0;
Here are a series of cases where echo $var
can show a different value than what was just assigned. This happens regardless of whether the assigned value was "double quoted", 'single quoted' or unquoted.
How do I get the shell to set my variable correctly?
Asterisks
The expected output is /* Foobar is free software */
, but instead I get a list of filenames:
$ var="/* Foobar is free software */"
$ echo $var
/bin /boot /dev /etc /home /initrd.img /lib /lib64 /media /mnt /opt /proc ...
Square brackets
The expected value is [a-z]
, but sometimes I get a single letter instead!
$ var=[a-z]
$ echo $var
c
Line feeds (newlines)
The expected value is a a list of separate lines, but instead all the values are on one line!
$ cat file
foo
bar
baz
$ var=$(cat file)
$ echo $var
foo bar baz
Multiple spaces
I expected a carefully aligned table header, but instead multiple spaces either disappear or are collapsed into one!
$ var=" title | count"
$ echo $var
title | count
Tabs
I expected two tab separated values, but instead I get two space separated values!
$ var=$'keytvalue'
$ echo $var
key value
bash shell sh quoting
add a comment
|
Here are a series of cases where echo $var
can show a different value than what was just assigned. This happens regardless of whether the assigned value was "double quoted", 'single quoted' or unquoted.
How do I get the shell to set my variable correctly?
Asterisks
The expected output is /* Foobar is free software */
, but instead I get a list of filenames:
$ var="/* Foobar is free software */"
$ echo $var
/bin /boot /dev /etc /home /initrd.img /lib /lib64 /media /mnt /opt /proc ...
Square brackets
The expected value is [a-z]
, but sometimes I get a single letter instead!
$ var=[a-z]
$ echo $var
c
Line feeds (newlines)
The expected value is a a list of separate lines, but instead all the values are on one line!
$ cat file
foo
bar
baz
$ var=$(cat file)
$ echo $var
foo bar baz
Multiple spaces
I expected a carefully aligned table header, but instead multiple spaces either disappear or are collapsed into one!
$ var=" title | count"
$ echo $var
title | count
Tabs
I expected two tab separated values, but instead I get two space separated values!
$ var=$'keytvalue'
$ echo $var
key value
bash shell sh quoting
2
Thanks for doing this. I encounter the line feeds one often. Sovar=$(cat file)
is fine, butecho "$var"
is needed.
– snd
Apr 1 '15 at 0:07
3
BTW, this is also BashPitfalls #14: mywiki.wooledge.org/BashPitfalls#echo_.24foo
– Charles Duffy
May 20 '15 at 16:47
See also stackoverflow.com/questions/10067266/…
– tripleee
Jul 9 '15 at 14:39
Also, see also stackoverflow.com/questions/2414150/…
– tripleee
Dec 18 '15 at 16:21
add a comment
|
Here are a series of cases where echo $var
can show a different value than what was just assigned. This happens regardless of whether the assigned value was "double quoted", 'single quoted' or unquoted.
How do I get the shell to set my variable correctly?
Asterisks
The expected output is /* Foobar is free software */
, but instead I get a list of filenames:
$ var="/* Foobar is free software */"
$ echo $var
/bin /boot /dev /etc /home /initrd.img /lib /lib64 /media /mnt /opt /proc ...
Square brackets
The expected value is [a-z]
, but sometimes I get a single letter instead!
$ var=[a-z]
$ echo $var
c
Line feeds (newlines)
The expected value is a a list of separate lines, but instead all the values are on one line!
$ cat file
foo
bar
baz
$ var=$(cat file)
$ echo $var
foo bar baz
Multiple spaces
I expected a carefully aligned table header, but instead multiple spaces either disappear or are collapsed into one!
$ var=" title | count"
$ echo $var
title | count
Tabs
I expected two tab separated values, but instead I get two space separated values!
$ var=$'keytvalue'
$ echo $var
key value
bash shell sh quoting
Here are a series of cases where echo $var
can show a different value than what was just assigned. This happens regardless of whether the assigned value was "double quoted", 'single quoted' or unquoted.
How do I get the shell to set my variable correctly?
Asterisks
The expected output is /* Foobar is free software */
, but instead I get a list of filenames:
$ var="/* Foobar is free software */"
$ echo $var
/bin /boot /dev /etc /home /initrd.img /lib /lib64 /media /mnt /opt /proc ...
Square brackets
The expected value is [a-z]
, but sometimes I get a single letter instead!
$ var=[a-z]
$ echo $var
c
Line feeds (newlines)
The expected value is a a list of separate lines, but instead all the values are on one line!
$ cat file
foo
bar
baz
$ var=$(cat file)
$ echo $var
foo bar baz
Multiple spaces
I expected a carefully aligned table header, but instead multiple spaces either disappear or are collapsed into one!
$ var=" title | count"
$ echo $var
title | count
Tabs
I expected two tab separated values, but instead I get two space separated values!
$ var=$'keytvalue'
$ echo $var
key value
bash shell sh quoting
bash shell sh quoting
edited Sep 25 '18 at 17:41
that other guy
asked Mar 31 '15 at 21:05
that other guythat other guy
80.8k9 gold badges94 silver badges133 bronze badges
80.8k9 gold badges94 silver badges133 bronze badges
2
Thanks for doing this. I encounter the line feeds one often. Sovar=$(cat file)
is fine, butecho "$var"
is needed.
– snd
Apr 1 '15 at 0:07
3
BTW, this is also BashPitfalls #14: mywiki.wooledge.org/BashPitfalls#echo_.24foo
– Charles Duffy
May 20 '15 at 16:47
See also stackoverflow.com/questions/10067266/…
– tripleee
Jul 9 '15 at 14:39
Also, see also stackoverflow.com/questions/2414150/…
– tripleee
Dec 18 '15 at 16:21
add a comment
|
2
Thanks for doing this. I encounter the line feeds one often. Sovar=$(cat file)
is fine, butecho "$var"
is needed.
– snd
Apr 1 '15 at 0:07
3
BTW, this is also BashPitfalls #14: mywiki.wooledge.org/BashPitfalls#echo_.24foo
– Charles Duffy
May 20 '15 at 16:47
See also stackoverflow.com/questions/10067266/…
– tripleee
Jul 9 '15 at 14:39
Also, see also stackoverflow.com/questions/2414150/…
– tripleee
Dec 18 '15 at 16:21
2
2
Thanks for doing this. I encounter the line feeds one often. So
var=$(cat file)
is fine, but echo "$var"
is needed.– snd
Apr 1 '15 at 0:07
Thanks for doing this. I encounter the line feeds one often. So
var=$(cat file)
is fine, but echo "$var"
is needed.– snd
Apr 1 '15 at 0:07
3
3
BTW, this is also BashPitfalls #14: mywiki.wooledge.org/BashPitfalls#echo_.24foo
– Charles Duffy
May 20 '15 at 16:47
BTW, this is also BashPitfalls #14: mywiki.wooledge.org/BashPitfalls#echo_.24foo
– Charles Duffy
May 20 '15 at 16:47
See also stackoverflow.com/questions/10067266/…
– tripleee
Jul 9 '15 at 14:39
See also stackoverflow.com/questions/10067266/…
– tripleee
Jul 9 '15 at 14:39
Also, see also stackoverflow.com/questions/2414150/…
– tripleee
Dec 18 '15 at 16:21
Also, see also stackoverflow.com/questions/2414150/…
– tripleee
Dec 18 '15 at 16:21
add a comment
|
6 Answers
6
active
oldest
votes
In all of the cases above, the variable is correctly set, but not correctly read! The right way is to use double quotes when referencing:
echo "$var"
This gives the expected value in all the examples given. Always quote variable references!
Why?
When a variable is unquoted, it will:
Undergo field splitting where the value is split into multiple words on whitespace (by default):
Before:
/* Foobar is free software */
After:
/*
,Foobar
,is
,free
,software
,*/
Each of these words will undergo pathname expansion, where patterns are expanded into matching files:
Before:
/*
After:
/bin
,/boot
,/dev
,/etc
,/home
, ...Finally, all the arguments are passed to echo, which writes them out separated by single spaces, giving
/bin /boot /dev /etc /home Foobar is free software Desktop/ Downloads/
instead of the variable's value.
When the variable is quoted it will:
- Be substituted for its value.
- There is no step 2.
This is why you should always quote all variable references, unless you specifically require word splitting and pathname expansion. Tools like shellcheck are there to help, and will warn about missing quotes in all the cases above.
it's not always working. I can give an example: paste.ubuntu.com/p/8RjR6CS668
– recolic
May 5 at 3:08
1
Yup,$(..)
strips trailing linefeeds. You can usevar=$(cat file; printf x); var="$var%x"
to work around it.
– that other guy
Jun 6 at 19:17
add a comment
|
You may want to know why this is happening. Together with the great explanation by that other guy, find a reference of Why does my shell script choke on whitespace or other special characters? written by Gilles in Unix & Linux:
Why do I need to write
"$foo"
? What happens without the quotes?
$foo
does not mean “take the value of the variablefoo
”. It means
something much more complex:
- First, take the value of the variable.
- Field splitting: treat that value as a whitespace-separated list of fields, and build the resulting list. For example, if the variable
containsfoo * bar
then the result of this step is the 3-element
listfoo
,*
,bar
.
- Filename generation: treat each field as a glob, i.e. as a wildcard pattern, and replace it by the list of file names that match this
pattern. If the pattern doesn't match any files, it is left
unmodified. In our example, this results in the list containingfoo
,
following by the list of files in the current directory, and finally
bar
. If the current directory is empty, the result isfoo
,*
,
bar
.
Note that the result is a list of strings. There are two contexts in
shell syntax: list context and string context. Field splitting and
filename generation only happen in list context, but that's most of
the time. Double quotes delimit a string context: the whole
double-quoted string is a single string, not to be split. (Exception:
"$@"
to expand to the list of positional parameters, e.g."$@"
is
equivalent to"$1" "$2" "$3"
if there are three positional
parameters. See What is the difference between $* and $@?)
The same happens to command substitution with
$(foo)
or with. On a side note, don't use
`foo``foo`
: its quoting rules are
weird and non-portable, and all modern shells support$(foo)
which
is absolutely equivalent except for having intuitive quoting rules.
The output of arithmetic substitution also undergoes the same
expansions, but that isn't normally a concern as it only contains
non-expandable characters (assumingIFS
doesn't contain digits or
-
).
See When is double-quoting necessary? for more details about the
cases when you can leave out the quotes.
Unless you mean for all this rigmarole to happen, just remember to
always use double quotes around variable and command substitutions. Do
take care: leaving out the quotes can lead not just to errors but to
security
holes.
add a comment
|
In addition to other issues caused by failing to quote, -n
and -e
can be consumed by echo
as arguments. (Only the former is legal per the POSIX spec for echo
, but several common implementations violate the spec and consume -e
as well).
To avoid this, use printf
instead of echo
when details matter.
Thus:
$ vars="-e -n -a"
$ echo $vars # breaks because -e and -n can be treated as arguments to echo
-a
$ echo "$vars"
-e -n -a
However, correct quoting won't always save you when using echo
:
$ vars="-n"
$ echo $vars
$ ## not even an empty line was printed
...whereas it will save you with printf
:
$ vars="-n"
$ printf '%sn' "$vars"
-n
Yay, we need a good dedup for this! I agree this fits the question title, but I don't think it'll get the visibility it deserves here. How about a new question à la "Why is my-e
/-n
/backslash not showing up?" We can add links from here as appropriate.
– that other guy
Sep 21 '18 at 23:24
Did you mean consume-n
as well?
– PesaThe
Jan 7 at 15:32
1
@PesaThe, no, I meant-e
. The standard forecho
does not specify output when its first argument is-n
, making any/all possible output legal in that case; there is no such provision for-e
.
– Charles Duffy
Jan 7 at 15:46
Oh...I can't read. Let's blame my English for that. Thanks for the explanation.
– PesaThe
Jan 7 at 21:03
add a comment
|
user double quote to get the exact value. like this:
echo "$var"
and it will read your value correctly.
add a comment
|
echo $var
output highly depends on the value of IFS
variable. By default it contains space, tab, and newline characters:
[ks@localhost ~]$ echo -n "$IFS" | cat -vte
^I$
This means that when shell is doing field splitting (or word splitting) it uses all these characters as word separators. This is what happens when referencing a variable without double quotes to echo it ($var
) and thus expected output is altered.
One way to prevent word splitting (besides using double quotes) is to set IFS
to null. See http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_06_05 :
If the value of IFS is null, no field splitting shall be performed.
Setting to null means setting to empty
value:
IFS=
Test:
[ks@localhost ~]$ echo -n "$IFS" | cat -vte
^I$
[ks@localhost ~]$ var=$'keynvalue'
[ks@localhost ~]$ echo $var
key value
[ks@localhost ~]$ IFS=
[ks@localhost ~]$ echo $var
key
value
[ks@localhost ~]$
2
You would also have toset -f
to prevent globbing
– that other guy
Nov 15 '15 at 18:31
@thatotherguy, is it really necessary for your 1-st example with path expansion? WithIFS
set to null,echo $var
will be expanded toecho '/* Foobar is free software */'
and path expansion is not performed inside single quoted strings.
– ks1322
Nov 16 '15 at 9:02
1
Yes. If youmkdir "/this thing called Foobar is free software etc/"
you'll see that it still expands. It's obviously more practical for the[a-z]
example.
– that other guy
Nov 16 '15 at 17:05
I see, this makes sense for[a-z]
example.
– ks1322
Nov 17 '15 at 8:42
add a comment
|
Additional to putting the variable in quotation, one could also translate the output of the variable using tr
and converting spaces to newlines.
$ echo $var | tr " " "n"
foo
bar
baz
Although this is a little more convoluted, it does add more diversity with the output as you can substitute any character as the separator between array variables.
2
But this substitutes all spaces to newlines. Quoting preserves the existing newlines and spaces.
– user000001
May 20 '15 at 18:06
True, yes. I suppose it depends on what is within the variable. I actually usetr
the other way around to create arrays from text files.
– Alek
May 20 '15 at 19:00
3
Creating a problem by not quoting the variable properly and then working around it with a hamfisted extra process is not good programming.
– tripleee
Feb 1 '16 at 6:22
@Alek, ...err, what? There's notr
needed to properly/correctly create an array from a text file -- you can specify whatever separator you want by setting IFS. For instance:IFS=$'n' read -r -d '' -a arrayname < <(cat file.txt && printf '')
works all the way back through bash 3.2 (the oldest version in wide circulation), and correctly sets exit status to false if yourcat
failed. And if you wanted, say, tabs instead newlines, you'd just replace the$'n'
with$'t'
.
– Charles Duffy
Jan 23 '17 at 21:43
1
@Alek, ...if you're doing something likearrayname=( $( cat file | tr 'n' ' ' ) )
, then that's broken on multiple layers: It's globbing your results (so a*
turns into a list of files in the current directory), and it would work just as well without thetr
(or thecat
, for that matter; one could just usearrayname=$( $(<file) )
and it would be broken in the same ways, but less inefficiently so).
– Charles Duffy
Jan 23 '17 at 21:45
|
show 1 more comment
protected by codeforester Aug 5 '18 at 17:11
Thank you for your interest in this question.
Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).
Would you like to answer one of these unanswered questions instead?
6 Answers
6
active
oldest
votes
6 Answers
6
active
oldest
votes
active
oldest
votes
active
oldest
votes
In all of the cases above, the variable is correctly set, but not correctly read! The right way is to use double quotes when referencing:
echo "$var"
This gives the expected value in all the examples given. Always quote variable references!
Why?
When a variable is unquoted, it will:
Undergo field splitting where the value is split into multiple words on whitespace (by default):
Before:
/* Foobar is free software */
After:
/*
,Foobar
,is
,free
,software
,*/
Each of these words will undergo pathname expansion, where patterns are expanded into matching files:
Before:
/*
After:
/bin
,/boot
,/dev
,/etc
,/home
, ...Finally, all the arguments are passed to echo, which writes them out separated by single spaces, giving
/bin /boot /dev /etc /home Foobar is free software Desktop/ Downloads/
instead of the variable's value.
When the variable is quoted it will:
- Be substituted for its value.
- There is no step 2.
This is why you should always quote all variable references, unless you specifically require word splitting and pathname expansion. Tools like shellcheck are there to help, and will warn about missing quotes in all the cases above.
it's not always working. I can give an example: paste.ubuntu.com/p/8RjR6CS668
– recolic
May 5 at 3:08
1
Yup,$(..)
strips trailing linefeeds. You can usevar=$(cat file; printf x); var="$var%x"
to work around it.
– that other guy
Jun 6 at 19:17
add a comment
|
In all of the cases above, the variable is correctly set, but not correctly read! The right way is to use double quotes when referencing:
echo "$var"
This gives the expected value in all the examples given. Always quote variable references!
Why?
When a variable is unquoted, it will:
Undergo field splitting where the value is split into multiple words on whitespace (by default):
Before:
/* Foobar is free software */
After:
/*
,Foobar
,is
,free
,software
,*/
Each of these words will undergo pathname expansion, where patterns are expanded into matching files:
Before:
/*
After:
/bin
,/boot
,/dev
,/etc
,/home
, ...Finally, all the arguments are passed to echo, which writes them out separated by single spaces, giving
/bin /boot /dev /etc /home Foobar is free software Desktop/ Downloads/
instead of the variable's value.
When the variable is quoted it will:
- Be substituted for its value.
- There is no step 2.
This is why you should always quote all variable references, unless you specifically require word splitting and pathname expansion. Tools like shellcheck are there to help, and will warn about missing quotes in all the cases above.
it's not always working. I can give an example: paste.ubuntu.com/p/8RjR6CS668
– recolic
May 5 at 3:08
1
Yup,$(..)
strips trailing linefeeds. You can usevar=$(cat file; printf x); var="$var%x"
to work around it.
– that other guy
Jun 6 at 19:17
add a comment
|
In all of the cases above, the variable is correctly set, but not correctly read! The right way is to use double quotes when referencing:
echo "$var"
This gives the expected value in all the examples given. Always quote variable references!
Why?
When a variable is unquoted, it will:
Undergo field splitting where the value is split into multiple words on whitespace (by default):
Before:
/* Foobar is free software */
After:
/*
,Foobar
,is
,free
,software
,*/
Each of these words will undergo pathname expansion, where patterns are expanded into matching files:
Before:
/*
After:
/bin
,/boot
,/dev
,/etc
,/home
, ...Finally, all the arguments are passed to echo, which writes them out separated by single spaces, giving
/bin /boot /dev /etc /home Foobar is free software Desktop/ Downloads/
instead of the variable's value.
When the variable is quoted it will:
- Be substituted for its value.
- There is no step 2.
This is why you should always quote all variable references, unless you specifically require word splitting and pathname expansion. Tools like shellcheck are there to help, and will warn about missing quotes in all the cases above.
In all of the cases above, the variable is correctly set, but not correctly read! The right way is to use double quotes when referencing:
echo "$var"
This gives the expected value in all the examples given. Always quote variable references!
Why?
When a variable is unquoted, it will:
Undergo field splitting where the value is split into multiple words on whitespace (by default):
Before:
/* Foobar is free software */
After:
/*
,Foobar
,is
,free
,software
,*/
Each of these words will undergo pathname expansion, where patterns are expanded into matching files:
Before:
/*
After:
/bin
,/boot
,/dev
,/etc
,/home
, ...Finally, all the arguments are passed to echo, which writes them out separated by single spaces, giving
/bin /boot /dev /etc /home Foobar is free software Desktop/ Downloads/
instead of the variable's value.
When the variable is quoted it will:
- Be substituted for its value.
- There is no step 2.
This is why you should always quote all variable references, unless you specifically require word splitting and pathname expansion. Tools like shellcheck are there to help, and will warn about missing quotes in all the cases above.
answered Mar 31 '15 at 21:05
that other guythat other guy
80.8k9 gold badges94 silver badges133 bronze badges
80.8k9 gold badges94 silver badges133 bronze badges
it's not always working. I can give an example: paste.ubuntu.com/p/8RjR6CS668
– recolic
May 5 at 3:08
1
Yup,$(..)
strips trailing linefeeds. You can usevar=$(cat file; printf x); var="$var%x"
to work around it.
– that other guy
Jun 6 at 19:17
add a comment
|
it's not always working. I can give an example: paste.ubuntu.com/p/8RjR6CS668
– recolic
May 5 at 3:08
1
Yup,$(..)
strips trailing linefeeds. You can usevar=$(cat file; printf x); var="$var%x"
to work around it.
– that other guy
Jun 6 at 19:17
it's not always working. I can give an example: paste.ubuntu.com/p/8RjR6CS668
– recolic
May 5 at 3:08
it's not always working. I can give an example: paste.ubuntu.com/p/8RjR6CS668
– recolic
May 5 at 3:08
1
1
Yup,
$(..)
strips trailing linefeeds. You can use var=$(cat file; printf x); var="$var%x"
to work around it.– that other guy
Jun 6 at 19:17
Yup,
$(..)
strips trailing linefeeds. You can use var=$(cat file; printf x); var="$var%x"
to work around it.– that other guy
Jun 6 at 19:17
add a comment
|
You may want to know why this is happening. Together with the great explanation by that other guy, find a reference of Why does my shell script choke on whitespace or other special characters? written by Gilles in Unix & Linux:
Why do I need to write
"$foo"
? What happens without the quotes?
$foo
does not mean “take the value of the variablefoo
”. It means
something much more complex:
- First, take the value of the variable.
- Field splitting: treat that value as a whitespace-separated list of fields, and build the resulting list. For example, if the variable
containsfoo * bar
then the result of this step is the 3-element
listfoo
,*
,bar
.
- Filename generation: treat each field as a glob, i.e. as a wildcard pattern, and replace it by the list of file names that match this
pattern. If the pattern doesn't match any files, it is left
unmodified. In our example, this results in the list containingfoo
,
following by the list of files in the current directory, and finally
bar
. If the current directory is empty, the result isfoo
,*
,
bar
.
Note that the result is a list of strings. There are two contexts in
shell syntax: list context and string context. Field splitting and
filename generation only happen in list context, but that's most of
the time. Double quotes delimit a string context: the whole
double-quoted string is a single string, not to be split. (Exception:
"$@"
to expand to the list of positional parameters, e.g."$@"
is
equivalent to"$1" "$2" "$3"
if there are three positional
parameters. See What is the difference between $* and $@?)
The same happens to command substitution with
$(foo)
or with. On a side note, don't use
`foo``foo`
: its quoting rules are
weird and non-portable, and all modern shells support$(foo)
which
is absolutely equivalent except for having intuitive quoting rules.
The output of arithmetic substitution also undergoes the same
expansions, but that isn't normally a concern as it only contains
non-expandable characters (assumingIFS
doesn't contain digits or
-
).
See When is double-quoting necessary? for more details about the
cases when you can leave out the quotes.
Unless you mean for all this rigmarole to happen, just remember to
always use double quotes around variable and command substitutions. Do
take care: leaving out the quotes can lead not just to errors but to
security
holes.
add a comment
|
You may want to know why this is happening. Together with the great explanation by that other guy, find a reference of Why does my shell script choke on whitespace or other special characters? written by Gilles in Unix & Linux:
Why do I need to write
"$foo"
? What happens without the quotes?
$foo
does not mean “take the value of the variablefoo
”. It means
something much more complex:
- First, take the value of the variable.
- Field splitting: treat that value as a whitespace-separated list of fields, and build the resulting list. For example, if the variable
containsfoo * bar
then the result of this step is the 3-element
listfoo
,*
,bar
.
- Filename generation: treat each field as a glob, i.e. as a wildcard pattern, and replace it by the list of file names that match this
pattern. If the pattern doesn't match any files, it is left
unmodified. In our example, this results in the list containingfoo
,
following by the list of files in the current directory, and finally
bar
. If the current directory is empty, the result isfoo
,*
,
bar
.
Note that the result is a list of strings. There are two contexts in
shell syntax: list context and string context. Field splitting and
filename generation only happen in list context, but that's most of
the time. Double quotes delimit a string context: the whole
double-quoted string is a single string, not to be split. (Exception:
"$@"
to expand to the list of positional parameters, e.g."$@"
is
equivalent to"$1" "$2" "$3"
if there are three positional
parameters. See What is the difference between $* and $@?)
The same happens to command substitution with
$(foo)
or with. On a side note, don't use
`foo``foo`
: its quoting rules are
weird and non-portable, and all modern shells support$(foo)
which
is absolutely equivalent except for having intuitive quoting rules.
The output of arithmetic substitution also undergoes the same
expansions, but that isn't normally a concern as it only contains
non-expandable characters (assumingIFS
doesn't contain digits or
-
).
See When is double-quoting necessary? for more details about the
cases when you can leave out the quotes.
Unless you mean for all this rigmarole to happen, just remember to
always use double quotes around variable and command substitutions. Do
take care: leaving out the quotes can lead not just to errors but to
security
holes.
add a comment
|
You may want to know why this is happening. Together with the great explanation by that other guy, find a reference of Why does my shell script choke on whitespace or other special characters? written by Gilles in Unix & Linux:
Why do I need to write
"$foo"
? What happens without the quotes?
$foo
does not mean “take the value of the variablefoo
”. It means
something much more complex:
- First, take the value of the variable.
- Field splitting: treat that value as a whitespace-separated list of fields, and build the resulting list. For example, if the variable
containsfoo * bar
then the result of this step is the 3-element
listfoo
,*
,bar
.
- Filename generation: treat each field as a glob, i.e. as a wildcard pattern, and replace it by the list of file names that match this
pattern. If the pattern doesn't match any files, it is left
unmodified. In our example, this results in the list containingfoo
,
following by the list of files in the current directory, and finally
bar
. If the current directory is empty, the result isfoo
,*
,
bar
.
Note that the result is a list of strings. There are two contexts in
shell syntax: list context and string context. Field splitting and
filename generation only happen in list context, but that's most of
the time. Double quotes delimit a string context: the whole
double-quoted string is a single string, not to be split. (Exception:
"$@"
to expand to the list of positional parameters, e.g."$@"
is
equivalent to"$1" "$2" "$3"
if there are three positional
parameters. See What is the difference between $* and $@?)
The same happens to command substitution with
$(foo)
or with. On a side note, don't use
`foo``foo`
: its quoting rules are
weird and non-portable, and all modern shells support$(foo)
which
is absolutely equivalent except for having intuitive quoting rules.
The output of arithmetic substitution also undergoes the same
expansions, but that isn't normally a concern as it only contains
non-expandable characters (assumingIFS
doesn't contain digits or
-
).
See When is double-quoting necessary? for more details about the
cases when you can leave out the quotes.
Unless you mean for all this rigmarole to happen, just remember to
always use double quotes around variable and command substitutions. Do
take care: leaving out the quotes can lead not just to errors but to
security
holes.
You may want to know why this is happening. Together with the great explanation by that other guy, find a reference of Why does my shell script choke on whitespace or other special characters? written by Gilles in Unix & Linux:
Why do I need to write
"$foo"
? What happens without the quotes?
$foo
does not mean “take the value of the variablefoo
”. It means
something much more complex:
- First, take the value of the variable.
- Field splitting: treat that value as a whitespace-separated list of fields, and build the resulting list. For example, if the variable
containsfoo * bar
then the result of this step is the 3-element
listfoo
,*
,bar
.
- Filename generation: treat each field as a glob, i.e. as a wildcard pattern, and replace it by the list of file names that match this
pattern. If the pattern doesn't match any files, it is left
unmodified. In our example, this results in the list containingfoo
,
following by the list of files in the current directory, and finally
bar
. If the current directory is empty, the result isfoo
,*
,
bar
.
Note that the result is a list of strings. There are two contexts in
shell syntax: list context and string context. Field splitting and
filename generation only happen in list context, but that's most of
the time. Double quotes delimit a string context: the whole
double-quoted string is a single string, not to be split. (Exception:
"$@"
to expand to the list of positional parameters, e.g."$@"
is
equivalent to"$1" "$2" "$3"
if there are three positional
parameters. See What is the difference between $* and $@?)
The same happens to command substitution with
$(foo)
or with. On a side note, don't use
`foo``foo`
: its quoting rules are
weird and non-portable, and all modern shells support$(foo)
which
is absolutely equivalent except for having intuitive quoting rules.
The output of arithmetic substitution also undergoes the same
expansions, but that isn't normally a concern as it only contains
non-expandable characters (assumingIFS
doesn't contain digits or
-
).
See When is double-quoting necessary? for more details about the
cases when you can leave out the quotes.
Unless you mean for all this rigmarole to happen, just remember to
always use double quotes around variable and command substitutions. Do
take care: leaving out the quotes can lead not just to errors but to
security
holes.
edited May 23 '17 at 12:18
Community♦
11 silver badge
11 silver badge
answered Jul 9 '15 at 14:20
fedorquifedorqui
184k58 gold badges379 silver badges421 bronze badges
184k58 gold badges379 silver badges421 bronze badges
add a comment
|
add a comment
|
In addition to other issues caused by failing to quote, -n
and -e
can be consumed by echo
as arguments. (Only the former is legal per the POSIX spec for echo
, but several common implementations violate the spec and consume -e
as well).
To avoid this, use printf
instead of echo
when details matter.
Thus:
$ vars="-e -n -a"
$ echo $vars # breaks because -e and -n can be treated as arguments to echo
-a
$ echo "$vars"
-e -n -a
However, correct quoting won't always save you when using echo
:
$ vars="-n"
$ echo $vars
$ ## not even an empty line was printed
...whereas it will save you with printf
:
$ vars="-n"
$ printf '%sn' "$vars"
-n
Yay, we need a good dedup for this! I agree this fits the question title, but I don't think it'll get the visibility it deserves here. How about a new question à la "Why is my-e
/-n
/backslash not showing up?" We can add links from here as appropriate.
– that other guy
Sep 21 '18 at 23:24
Did you mean consume-n
as well?
– PesaThe
Jan 7 at 15:32
1
@PesaThe, no, I meant-e
. The standard forecho
does not specify output when its first argument is-n
, making any/all possible output legal in that case; there is no such provision for-e
.
– Charles Duffy
Jan 7 at 15:46
Oh...I can't read. Let's blame my English for that. Thanks for the explanation.
– PesaThe
Jan 7 at 21:03
add a comment
|
In addition to other issues caused by failing to quote, -n
and -e
can be consumed by echo
as arguments. (Only the former is legal per the POSIX spec for echo
, but several common implementations violate the spec and consume -e
as well).
To avoid this, use printf
instead of echo
when details matter.
Thus:
$ vars="-e -n -a"
$ echo $vars # breaks because -e and -n can be treated as arguments to echo
-a
$ echo "$vars"
-e -n -a
However, correct quoting won't always save you when using echo
:
$ vars="-n"
$ echo $vars
$ ## not even an empty line was printed
...whereas it will save you with printf
:
$ vars="-n"
$ printf '%sn' "$vars"
-n
Yay, we need a good dedup for this! I agree this fits the question title, but I don't think it'll get the visibility it deserves here. How about a new question à la "Why is my-e
/-n
/backslash not showing up?" We can add links from here as appropriate.
– that other guy
Sep 21 '18 at 23:24
Did you mean consume-n
as well?
– PesaThe
Jan 7 at 15:32
1
@PesaThe, no, I meant-e
. The standard forecho
does not specify output when its first argument is-n
, making any/all possible output legal in that case; there is no such provision for-e
.
– Charles Duffy
Jan 7 at 15:46
Oh...I can't read. Let's blame my English for that. Thanks for the explanation.
– PesaThe
Jan 7 at 21:03
add a comment
|
In addition to other issues caused by failing to quote, -n
and -e
can be consumed by echo
as arguments. (Only the former is legal per the POSIX spec for echo
, but several common implementations violate the spec and consume -e
as well).
To avoid this, use printf
instead of echo
when details matter.
Thus:
$ vars="-e -n -a"
$ echo $vars # breaks because -e and -n can be treated as arguments to echo
-a
$ echo "$vars"
-e -n -a
However, correct quoting won't always save you when using echo
:
$ vars="-n"
$ echo $vars
$ ## not even an empty line was printed
...whereas it will save you with printf
:
$ vars="-n"
$ printf '%sn' "$vars"
-n
In addition to other issues caused by failing to quote, -n
and -e
can be consumed by echo
as arguments. (Only the former is legal per the POSIX spec for echo
, but several common implementations violate the spec and consume -e
as well).
To avoid this, use printf
instead of echo
when details matter.
Thus:
$ vars="-e -n -a"
$ echo $vars # breaks because -e and -n can be treated as arguments to echo
-a
$ echo "$vars"
-e -n -a
However, correct quoting won't always save you when using echo
:
$ vars="-n"
$ echo $vars
$ ## not even an empty line was printed
...whereas it will save you with printf
:
$ vars="-n"
$ printf '%sn' "$vars"
-n
answered Sep 21 '18 at 22:31
Charles DuffyCharles Duffy
194k29 gold badges222 silver badges279 bronze badges
194k29 gold badges222 silver badges279 bronze badges
Yay, we need a good dedup for this! I agree this fits the question title, but I don't think it'll get the visibility it deserves here. How about a new question à la "Why is my-e
/-n
/backslash not showing up?" We can add links from here as appropriate.
– that other guy
Sep 21 '18 at 23:24
Did you mean consume-n
as well?
– PesaThe
Jan 7 at 15:32
1
@PesaThe, no, I meant-e
. The standard forecho
does not specify output when its first argument is-n
, making any/all possible output legal in that case; there is no such provision for-e
.
– Charles Duffy
Jan 7 at 15:46
Oh...I can't read. Let's blame my English for that. Thanks for the explanation.
– PesaThe
Jan 7 at 21:03
add a comment
|
Yay, we need a good dedup for this! I agree this fits the question title, but I don't think it'll get the visibility it deserves here. How about a new question à la "Why is my-e
/-n
/backslash not showing up?" We can add links from here as appropriate.
– that other guy
Sep 21 '18 at 23:24
Did you mean consume-n
as well?
– PesaThe
Jan 7 at 15:32
1
@PesaThe, no, I meant-e
. The standard forecho
does not specify output when its first argument is-n
, making any/all possible output legal in that case; there is no such provision for-e
.
– Charles Duffy
Jan 7 at 15:46
Oh...I can't read. Let's blame my English for that. Thanks for the explanation.
– PesaThe
Jan 7 at 21:03
Yay, we need a good dedup for this! I agree this fits the question title, but I don't think it'll get the visibility it deserves here. How about a new question à la "Why is my
-e
/-n
/backslash not showing up?" We can add links from here as appropriate.– that other guy
Sep 21 '18 at 23:24
Yay, we need a good dedup for this! I agree this fits the question title, but I don't think it'll get the visibility it deserves here. How about a new question à la "Why is my
-e
/-n
/backslash not showing up?" We can add links from here as appropriate.– that other guy
Sep 21 '18 at 23:24
Did you mean consume
-n
as well?– PesaThe
Jan 7 at 15:32
Did you mean consume
-n
as well?– PesaThe
Jan 7 at 15:32
1
1
@PesaThe, no, I meant
-e
. The standard for echo
does not specify output when its first argument is -n
, making any/all possible output legal in that case; there is no such provision for -e
.– Charles Duffy
Jan 7 at 15:46
@PesaThe, no, I meant
-e
. The standard for echo
does not specify output when its first argument is -n
, making any/all possible output legal in that case; there is no such provision for -e
.– Charles Duffy
Jan 7 at 15:46
Oh...I can't read. Let's blame my English for that. Thanks for the explanation.
– PesaThe
Jan 7 at 21:03
Oh...I can't read. Let's blame my English for that. Thanks for the explanation.
– PesaThe
Jan 7 at 21:03
add a comment
|
user double quote to get the exact value. like this:
echo "$var"
and it will read your value correctly.
add a comment
|
user double quote to get the exact value. like this:
echo "$var"
and it will read your value correctly.
add a comment
|
user double quote to get the exact value. like this:
echo "$var"
and it will read your value correctly.
user double quote to get the exact value. like this:
echo "$var"
and it will read your value correctly.
answered Aug 3 '18 at 4:47
vanishedzhouvanishedzhou
1211 silver badge4 bronze badges
1211 silver badge4 bronze badges
add a comment
|
add a comment
|
echo $var
output highly depends on the value of IFS
variable. By default it contains space, tab, and newline characters:
[ks@localhost ~]$ echo -n "$IFS" | cat -vte
^I$
This means that when shell is doing field splitting (or word splitting) it uses all these characters as word separators. This is what happens when referencing a variable without double quotes to echo it ($var
) and thus expected output is altered.
One way to prevent word splitting (besides using double quotes) is to set IFS
to null. See http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_06_05 :
If the value of IFS is null, no field splitting shall be performed.
Setting to null means setting to empty
value:
IFS=
Test:
[ks@localhost ~]$ echo -n "$IFS" | cat -vte
^I$
[ks@localhost ~]$ var=$'keynvalue'
[ks@localhost ~]$ echo $var
key value
[ks@localhost ~]$ IFS=
[ks@localhost ~]$ echo $var
key
value
[ks@localhost ~]$
2
You would also have toset -f
to prevent globbing
– that other guy
Nov 15 '15 at 18:31
@thatotherguy, is it really necessary for your 1-st example with path expansion? WithIFS
set to null,echo $var
will be expanded toecho '/* Foobar is free software */'
and path expansion is not performed inside single quoted strings.
– ks1322
Nov 16 '15 at 9:02
1
Yes. If youmkdir "/this thing called Foobar is free software etc/"
you'll see that it still expands. It's obviously more practical for the[a-z]
example.
– that other guy
Nov 16 '15 at 17:05
I see, this makes sense for[a-z]
example.
– ks1322
Nov 17 '15 at 8:42
add a comment
|
echo $var
output highly depends on the value of IFS
variable. By default it contains space, tab, and newline characters:
[ks@localhost ~]$ echo -n "$IFS" | cat -vte
^I$
This means that when shell is doing field splitting (or word splitting) it uses all these characters as word separators. This is what happens when referencing a variable without double quotes to echo it ($var
) and thus expected output is altered.
One way to prevent word splitting (besides using double quotes) is to set IFS
to null. See http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_06_05 :
If the value of IFS is null, no field splitting shall be performed.
Setting to null means setting to empty
value:
IFS=
Test:
[ks@localhost ~]$ echo -n "$IFS" | cat -vte
^I$
[ks@localhost ~]$ var=$'keynvalue'
[ks@localhost ~]$ echo $var
key value
[ks@localhost ~]$ IFS=
[ks@localhost ~]$ echo $var
key
value
[ks@localhost ~]$
2
You would also have toset -f
to prevent globbing
– that other guy
Nov 15 '15 at 18:31
@thatotherguy, is it really necessary for your 1-st example with path expansion? WithIFS
set to null,echo $var
will be expanded toecho '/* Foobar is free software */'
and path expansion is not performed inside single quoted strings.
– ks1322
Nov 16 '15 at 9:02
1
Yes. If youmkdir "/this thing called Foobar is free software etc/"
you'll see that it still expands. It's obviously more practical for the[a-z]
example.
– that other guy
Nov 16 '15 at 17:05
I see, this makes sense for[a-z]
example.
– ks1322
Nov 17 '15 at 8:42
add a comment
|
echo $var
output highly depends on the value of IFS
variable. By default it contains space, tab, and newline characters:
[ks@localhost ~]$ echo -n "$IFS" | cat -vte
^I$
This means that when shell is doing field splitting (or word splitting) it uses all these characters as word separators. This is what happens when referencing a variable without double quotes to echo it ($var
) and thus expected output is altered.
One way to prevent word splitting (besides using double quotes) is to set IFS
to null. See http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_06_05 :
If the value of IFS is null, no field splitting shall be performed.
Setting to null means setting to empty
value:
IFS=
Test:
[ks@localhost ~]$ echo -n "$IFS" | cat -vte
^I$
[ks@localhost ~]$ var=$'keynvalue'
[ks@localhost ~]$ echo $var
key value
[ks@localhost ~]$ IFS=
[ks@localhost ~]$ echo $var
key
value
[ks@localhost ~]$
echo $var
output highly depends on the value of IFS
variable. By default it contains space, tab, and newline characters:
[ks@localhost ~]$ echo -n "$IFS" | cat -vte
^I$
This means that when shell is doing field splitting (or word splitting) it uses all these characters as word separators. This is what happens when referencing a variable without double quotes to echo it ($var
) and thus expected output is altered.
One way to prevent word splitting (besides using double quotes) is to set IFS
to null. See http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_06_05 :
If the value of IFS is null, no field splitting shall be performed.
Setting to null means setting to empty
value:
IFS=
Test:
[ks@localhost ~]$ echo -n "$IFS" | cat -vte
^I$
[ks@localhost ~]$ var=$'keynvalue'
[ks@localhost ~]$ echo $var
key value
[ks@localhost ~]$ IFS=
[ks@localhost ~]$ echo $var
key
value
[ks@localhost ~]$
answered Nov 15 '15 at 12:14
ks1322ks1322
24.1k11 gold badges72 silver badges116 bronze badges
24.1k11 gold badges72 silver badges116 bronze badges
2
You would also have toset -f
to prevent globbing
– that other guy
Nov 15 '15 at 18:31
@thatotherguy, is it really necessary for your 1-st example with path expansion? WithIFS
set to null,echo $var
will be expanded toecho '/* Foobar is free software */'
and path expansion is not performed inside single quoted strings.
– ks1322
Nov 16 '15 at 9:02
1
Yes. If youmkdir "/this thing called Foobar is free software etc/"
you'll see that it still expands. It's obviously more practical for the[a-z]
example.
– that other guy
Nov 16 '15 at 17:05
I see, this makes sense for[a-z]
example.
– ks1322
Nov 17 '15 at 8:42
add a comment
|
2
You would also have toset -f
to prevent globbing
– that other guy
Nov 15 '15 at 18:31
@thatotherguy, is it really necessary for your 1-st example with path expansion? WithIFS
set to null,echo $var
will be expanded toecho '/* Foobar is free software */'
and path expansion is not performed inside single quoted strings.
– ks1322
Nov 16 '15 at 9:02
1
Yes. If youmkdir "/this thing called Foobar is free software etc/"
you'll see that it still expands. It's obviously more practical for the[a-z]
example.
– that other guy
Nov 16 '15 at 17:05
I see, this makes sense for[a-z]
example.
– ks1322
Nov 17 '15 at 8:42
2
2
You would also have to
set -f
to prevent globbing– that other guy
Nov 15 '15 at 18:31
You would also have to
set -f
to prevent globbing– that other guy
Nov 15 '15 at 18:31
@thatotherguy, is it really necessary for your 1-st example with path expansion? With
IFS
set to null, echo $var
will be expanded to echo '/* Foobar is free software */'
and path expansion is not performed inside single quoted strings.– ks1322
Nov 16 '15 at 9:02
@thatotherguy, is it really necessary for your 1-st example with path expansion? With
IFS
set to null, echo $var
will be expanded to echo '/* Foobar is free software */'
and path expansion is not performed inside single quoted strings.– ks1322
Nov 16 '15 at 9:02
1
1
Yes. If you
mkdir "/this thing called Foobar is free software etc/"
you'll see that it still expands. It's obviously more practical for the [a-z]
example.– that other guy
Nov 16 '15 at 17:05
Yes. If you
mkdir "/this thing called Foobar is free software etc/"
you'll see that it still expands. It's obviously more practical for the [a-z]
example.– that other guy
Nov 16 '15 at 17:05
I see, this makes sense for
[a-z]
example.– ks1322
Nov 17 '15 at 8:42
I see, this makes sense for
[a-z]
example.– ks1322
Nov 17 '15 at 8:42
add a comment
|
Additional to putting the variable in quotation, one could also translate the output of the variable using tr
and converting spaces to newlines.
$ echo $var | tr " " "n"
foo
bar
baz
Although this is a little more convoluted, it does add more diversity with the output as you can substitute any character as the separator between array variables.
2
But this substitutes all spaces to newlines. Quoting preserves the existing newlines and spaces.
– user000001
May 20 '15 at 18:06
True, yes. I suppose it depends on what is within the variable. I actually usetr
the other way around to create arrays from text files.
– Alek
May 20 '15 at 19:00
3
Creating a problem by not quoting the variable properly and then working around it with a hamfisted extra process is not good programming.
– tripleee
Feb 1 '16 at 6:22
@Alek, ...err, what? There's notr
needed to properly/correctly create an array from a text file -- you can specify whatever separator you want by setting IFS. For instance:IFS=$'n' read -r -d '' -a arrayname < <(cat file.txt && printf '')
works all the way back through bash 3.2 (the oldest version in wide circulation), and correctly sets exit status to false if yourcat
failed. And if you wanted, say, tabs instead newlines, you'd just replace the$'n'
with$'t'
.
– Charles Duffy
Jan 23 '17 at 21:43
1
@Alek, ...if you're doing something likearrayname=( $( cat file | tr 'n' ' ' ) )
, then that's broken on multiple layers: It's globbing your results (so a*
turns into a list of files in the current directory), and it would work just as well without thetr
(or thecat
, for that matter; one could just usearrayname=$( $(<file) )
and it would be broken in the same ways, but less inefficiently so).
– Charles Duffy
Jan 23 '17 at 21:45
|
show 1 more comment
Additional to putting the variable in quotation, one could also translate the output of the variable using tr
and converting spaces to newlines.
$ echo $var | tr " " "n"
foo
bar
baz
Although this is a little more convoluted, it does add more diversity with the output as you can substitute any character as the separator between array variables.
2
But this substitutes all spaces to newlines. Quoting preserves the existing newlines and spaces.
– user000001
May 20 '15 at 18:06
True, yes. I suppose it depends on what is within the variable. I actually usetr
the other way around to create arrays from text files.
– Alek
May 20 '15 at 19:00
3
Creating a problem by not quoting the variable properly and then working around it with a hamfisted extra process is not good programming.
– tripleee
Feb 1 '16 at 6:22
@Alek, ...err, what? There's notr
needed to properly/correctly create an array from a text file -- you can specify whatever separator you want by setting IFS. For instance:IFS=$'n' read -r -d '' -a arrayname < <(cat file.txt && printf '')
works all the way back through bash 3.2 (the oldest version in wide circulation), and correctly sets exit status to false if yourcat
failed. And if you wanted, say, tabs instead newlines, you'd just replace the$'n'
with$'t'
.
– Charles Duffy
Jan 23 '17 at 21:43
1
@Alek, ...if you're doing something likearrayname=( $( cat file | tr 'n' ' ' ) )
, then that's broken on multiple layers: It's globbing your results (so a*
turns into a list of files in the current directory), and it would work just as well without thetr
(or thecat
, for that matter; one could just usearrayname=$( $(<file) )
and it would be broken in the same ways, but less inefficiently so).
– Charles Duffy
Jan 23 '17 at 21:45
|
show 1 more comment
Additional to putting the variable in quotation, one could also translate the output of the variable using tr
and converting spaces to newlines.
$ echo $var | tr " " "n"
foo
bar
baz
Although this is a little more convoluted, it does add more diversity with the output as you can substitute any character as the separator between array variables.
Additional to putting the variable in quotation, one could also translate the output of the variable using tr
and converting spaces to newlines.
$ echo $var | tr " " "n"
foo
bar
baz
Although this is a little more convoluted, it does add more diversity with the output as you can substitute any character as the separator between array variables.
answered May 20 '15 at 16:33
AlekAlek
747 bronze badges
747 bronze badges
2
But this substitutes all spaces to newlines. Quoting preserves the existing newlines and spaces.
– user000001
May 20 '15 at 18:06
True, yes. I suppose it depends on what is within the variable. I actually usetr
the other way around to create arrays from text files.
– Alek
May 20 '15 at 19:00
3
Creating a problem by not quoting the variable properly and then working around it with a hamfisted extra process is not good programming.
– tripleee
Feb 1 '16 at 6:22
@Alek, ...err, what? There's notr
needed to properly/correctly create an array from a text file -- you can specify whatever separator you want by setting IFS. For instance:IFS=$'n' read -r -d '' -a arrayname < <(cat file.txt && printf '')
works all the way back through bash 3.2 (the oldest version in wide circulation), and correctly sets exit status to false if yourcat
failed. And if you wanted, say, tabs instead newlines, you'd just replace the$'n'
with$'t'
.
– Charles Duffy
Jan 23 '17 at 21:43
1
@Alek, ...if you're doing something likearrayname=( $( cat file | tr 'n' ' ' ) )
, then that's broken on multiple layers: It's globbing your results (so a*
turns into a list of files in the current directory), and it would work just as well without thetr
(or thecat
, for that matter; one could just usearrayname=$( $(<file) )
and it would be broken in the same ways, but less inefficiently so).
– Charles Duffy
Jan 23 '17 at 21:45
|
show 1 more comment
2
But this substitutes all spaces to newlines. Quoting preserves the existing newlines and spaces.
– user000001
May 20 '15 at 18:06
True, yes. I suppose it depends on what is within the variable. I actually usetr
the other way around to create arrays from text files.
– Alek
May 20 '15 at 19:00
3
Creating a problem by not quoting the variable properly and then working around it with a hamfisted extra process is not good programming.
– tripleee
Feb 1 '16 at 6:22
@Alek, ...err, what? There's notr
needed to properly/correctly create an array from a text file -- you can specify whatever separator you want by setting IFS. For instance:IFS=$'n' read -r -d '' -a arrayname < <(cat file.txt && printf '')
works all the way back through bash 3.2 (the oldest version in wide circulation), and correctly sets exit status to false if yourcat
failed. And if you wanted, say, tabs instead newlines, you'd just replace the$'n'
with$'t'
.
– Charles Duffy
Jan 23 '17 at 21:43
1
@Alek, ...if you're doing something likearrayname=( $( cat file | tr 'n' ' ' ) )
, then that's broken on multiple layers: It's globbing your results (so a*
turns into a list of files in the current directory), and it would work just as well without thetr
(or thecat
, for that matter; one could just usearrayname=$( $(<file) )
and it would be broken in the same ways, but less inefficiently so).
– Charles Duffy
Jan 23 '17 at 21:45
2
2
But this substitutes all spaces to newlines. Quoting preserves the existing newlines and spaces.
– user000001
May 20 '15 at 18:06
But this substitutes all spaces to newlines. Quoting preserves the existing newlines and spaces.
– user000001
May 20 '15 at 18:06
True, yes. I suppose it depends on what is within the variable. I actually use
tr
the other way around to create arrays from text files.– Alek
May 20 '15 at 19:00
True, yes. I suppose it depends on what is within the variable. I actually use
tr
the other way around to create arrays from text files.– Alek
May 20 '15 at 19:00
3
3
Creating a problem by not quoting the variable properly and then working around it with a hamfisted extra process is not good programming.
– tripleee
Feb 1 '16 at 6:22
Creating a problem by not quoting the variable properly and then working around it with a hamfisted extra process is not good programming.
– tripleee
Feb 1 '16 at 6:22
@Alek, ...err, what? There's no
tr
needed to properly/correctly create an array from a text file -- you can specify whatever separator you want by setting IFS. For instance: IFS=$'n' read -r -d '' -a arrayname < <(cat file.txt && printf '')
works all the way back through bash 3.2 (the oldest version in wide circulation), and correctly sets exit status to false if your cat
failed. And if you wanted, say, tabs instead newlines, you'd just replace the $'n'
with $'t'
.– Charles Duffy
Jan 23 '17 at 21:43
@Alek, ...err, what? There's no
tr
needed to properly/correctly create an array from a text file -- you can specify whatever separator you want by setting IFS. For instance: IFS=$'n' read -r -d '' -a arrayname < <(cat file.txt && printf '')
works all the way back through bash 3.2 (the oldest version in wide circulation), and correctly sets exit status to false if your cat
failed. And if you wanted, say, tabs instead newlines, you'd just replace the $'n'
with $'t'
.– Charles Duffy
Jan 23 '17 at 21:43
1
1
@Alek, ...if you're doing something like
arrayname=( $( cat file | tr 'n' ' ' ) )
, then that's broken on multiple layers: It's globbing your results (so a *
turns into a list of files in the current directory), and it would work just as well without the tr
(or the cat
, for that matter; one could just use arrayname=$( $(<file) )
and it would be broken in the same ways, but less inefficiently so).– Charles Duffy
Jan 23 '17 at 21:45
@Alek, ...if you're doing something like
arrayname=( $( cat file | tr 'n' ' ' ) )
, then that's broken on multiple layers: It's globbing your results (so a *
turns into a list of files in the current directory), and it would work just as well without the tr
(or the cat
, for that matter; one could just use arrayname=$( $(<file) )
and it would be broken in the same ways, but less inefficiently so).– Charles Duffy
Jan 23 '17 at 21:45
|
show 1 more comment
protected by codeforester Aug 5 '18 at 17:11
Thank you for your interest in this question.
Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).
Would you like to answer one of these unanswered questions instead?
2
Thanks for doing this. I encounter the line feeds one often. So
var=$(cat file)
is fine, butecho "$var"
is needed.– snd
Apr 1 '15 at 0:07
3
BTW, this is also BashPitfalls #14: mywiki.wooledge.org/BashPitfalls#echo_.24foo
– Charles Duffy
May 20 '15 at 16:47
See also stackoverflow.com/questions/10067266/…
– tripleee
Jul 9 '15 at 14:39
Also, see also stackoverflow.com/questions/2414150/…
– tripleee
Dec 18 '15 at 16:21