Are shell scripts sensitive to encoding and line endings?./configure : /bin/sh^M : bad interpreter-bash: ./my_script: /bin/bash^M: bad interpreter: No such file or directory'r': command not found - .bashrc / .bash_profilerunning bash script in cygwin on windows 7Bash programmation (Cygwin): Illegal Character ^MWhy is a shell script giving syntax errors when the same code works elsewhere?Why would a correct shell script give a wrapped/truncated/corrupted error message?Why doesn't my commands work in shell scripts but in bash?expr: non-integer argument while doing Arithmetic in Shell ScriptMXPOST bash: ./mxpost: /bin/ksh^M: bad interpreter: No such file or directoryCheck if a directory exists in a shell scriptGet the source directory of a Bash script from within the script itselfHow do I prompt for Yes/No/Cancel input in a Linux shell script?How to use SSH to run a shell script on a remote machine?How to check if a program exists from a Bash script?In the shell, what does “ 2>&1 ” mean?How to count all the lines of code in a directory recursively?YYYY-MM-DD format date in shell scriptHow to declare and use boolean variables in shell script?Check existence of input argument in a Bash shell script
How to prevent bad sectors?
Why does the 6502 have the BIT instruction?
Why doesn't the Earth's acceleration towards the Moon accumulate to push the Earth off its orbit?
Does `declare -a A` create an empty array `A` in Bash?
Tic-Tac-Toe for the terminal
The deliberate use of misleading terminology
What are the problems in teaching guitar via Skype?
What is game ban VS VAC ban in steam?
What does "tea juice" mean in this context?
When a current flow in an inductor is interrupted, what limits the voltage rise?
Why do Russians call their women expensive ("дорогая")?
Is it possible to change original filename of an exe?
Where can I find the list of all tendons in the human body?
The qvolume of an integer
Boots: Does light damage affect waterproofing?
Restoring order in a deck of playing cards
Can the Help action be used to give advantage to a specific ally's attack (rather than just the next ally who attacks the target)?
Writing the notation when gates act on non successive registers
Modern approach to radio buttons
Creating Fictional Slavic Place Names
What does uniform continuity mean exactly?
Asking bank to reduce APR instead of increasing credit limit
Expenditure in Poland - Forex doesn't have Zloty
Can a wire having a 610-670 THz (frequency of blue light) AC frequency supply, generate blue light?
Are shell scripts sensitive to encoding and line endings?
./configure : /bin/sh^M : bad interpreter-bash: ./my_script: /bin/bash^M: bad interpreter: No such file or directory'r': command not found - .bashrc / .bash_profilerunning bash script in cygwin on windows 7Bash programmation (Cygwin): Illegal Character ^MWhy is a shell script giving syntax errors when the same code works elsewhere?Why would a correct shell script give a wrapped/truncated/corrupted error message?Why doesn't my commands work in shell scripts but in bash?expr: non-integer argument while doing Arithmetic in Shell ScriptMXPOST bash: ./mxpost: /bin/ksh^M: bad interpreter: No such file or directoryCheck if a directory exists in a shell scriptGet the source directory of a Bash script from within the script itselfHow do I prompt for Yes/No/Cancel input in a Linux shell script?How to use SSH to run a shell script on a remote machine?How to check if a program exists from a Bash script?In the shell, what does “ 2>&1 ” mean?How to count all the lines of code in a directory recursively?YYYY-MM-DD format date in shell scriptHow to declare and use boolean variables in shell script?Check existence of input argument in a Bash shell script
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
I am making a NW.js app on Mac, and want to run the app in dev mode by double-clicking on an icon. First step, I'm trying to make my shell script work.
Using VSCode on Windows (I wanted to gain time), I have created a run-nw
file at the root of my project, containing this:
#!/bin/bash
cd "src"
npm install
cd ..
./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &
but I get this output:
$ sh ./run-nw
: command not found
: No such file or directory
: command not found
: No such file or directory
Usage: npm <command>
where <command> is one of: (snip commands list)
(snip npm help)
npm@3.10.3 /usr/local/lib/node_modules/npm
: command not found
: No such file or directory
: command not found
I really don't understand:
- it seems that it takes empty lines as commands. In my editor (VSCode) I have tried to replace
rn
withn
(in case ther
creates problems) but it changes nothing. - it seems that it doesn't find the folders (with or without the
dirname
instruction), or maybe it doesn't know about thecd
command ? - it seems that it doesn't understand the
install
argument tonpm
- the part that really weirds me out, is that it still runs the app (if I did a
npm install
manually)...
Not able to make it work properly, and suspecting something weird with the file itself, I created a new one directly on the Mac, using vim this time. I entered the exact same instructions, and... now it works without any issue.
A diff on the two files reveals exactly zero difference.
What can be the difference? What can make the first script not work? How can I find out?
Update
Following the accepted answer's recommandations, after the wrong line endings came back, I checked multiple things. It turns out that since I copied my ~/.gitconfig
from my Windows machine, I had autocrlf=true
, so every time I modified the bash file under Windows, it re-set the line endings to rn
.
So, in addition to running dos2unix (which you will have to install using Homebrew on mac), if you're using Git, check your config.
bash shell sh
add a comment |
I am making a NW.js app on Mac, and want to run the app in dev mode by double-clicking on an icon. First step, I'm trying to make my shell script work.
Using VSCode on Windows (I wanted to gain time), I have created a run-nw
file at the root of my project, containing this:
#!/bin/bash
cd "src"
npm install
cd ..
./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &
but I get this output:
$ sh ./run-nw
: command not found
: No such file or directory
: command not found
: No such file or directory
Usage: npm <command>
where <command> is one of: (snip commands list)
(snip npm help)
npm@3.10.3 /usr/local/lib/node_modules/npm
: command not found
: No such file or directory
: command not found
I really don't understand:
- it seems that it takes empty lines as commands. In my editor (VSCode) I have tried to replace
rn
withn
(in case ther
creates problems) but it changes nothing. - it seems that it doesn't find the folders (with or without the
dirname
instruction), or maybe it doesn't know about thecd
command ? - it seems that it doesn't understand the
install
argument tonpm
- the part that really weirds me out, is that it still runs the app (if I did a
npm install
manually)...
Not able to make it work properly, and suspecting something weird with the file itself, I created a new one directly on the Mac, using vim this time. I entered the exact same instructions, and... now it works without any issue.
A diff on the two files reveals exactly zero difference.
What can be the difference? What can make the first script not work? How can I find out?
Update
Following the accepted answer's recommandations, after the wrong line endings came back, I checked multiple things. It turns out that since I copied my ~/.gitconfig
from my Windows machine, I had autocrlf=true
, so every time I modified the bash file under Windows, it re-set the line endings to rn
.
So, in addition to running dos2unix (which you will have to install using Homebrew on mac), if you're using Git, check your config.
bash shell sh
3
trydos2unix
?
– PS.
Sep 16 '16 at 9:10
2
If you run a shell script on Linux, at least all the shell implementations I have encountered so far, would get upset if they found a r somewhere. No you say that you have removed the r, and I hope you verified that they are really gone. For the safe side, you should look at your file at the hexadecimal level, to ensure that you don't have other weird characters in it. The next step would then be to execute the script withsh -x ./run-nw
, to get more information.
– user1934428
Sep 16 '16 at 9:34
Another good command to look for weird characters in a text file isLC_ALL=C cat -vet /path/to/file
. If the file is normal, it'll look normal (except for having a "$" at the end of each line). Anything abnormal should stand out fairly well. DOS/Windows files will have "^M$" at the end of lines.
– Gordon Davisson
Sep 16 '16 at 18:03
add a comment |
I am making a NW.js app on Mac, and want to run the app in dev mode by double-clicking on an icon. First step, I'm trying to make my shell script work.
Using VSCode on Windows (I wanted to gain time), I have created a run-nw
file at the root of my project, containing this:
#!/bin/bash
cd "src"
npm install
cd ..
./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &
but I get this output:
$ sh ./run-nw
: command not found
: No such file or directory
: command not found
: No such file or directory
Usage: npm <command>
where <command> is one of: (snip commands list)
(snip npm help)
npm@3.10.3 /usr/local/lib/node_modules/npm
: command not found
: No such file or directory
: command not found
I really don't understand:
- it seems that it takes empty lines as commands. In my editor (VSCode) I have tried to replace
rn
withn
(in case ther
creates problems) but it changes nothing. - it seems that it doesn't find the folders (with or without the
dirname
instruction), or maybe it doesn't know about thecd
command ? - it seems that it doesn't understand the
install
argument tonpm
- the part that really weirds me out, is that it still runs the app (if I did a
npm install
manually)...
Not able to make it work properly, and suspecting something weird with the file itself, I created a new one directly on the Mac, using vim this time. I entered the exact same instructions, and... now it works without any issue.
A diff on the two files reveals exactly zero difference.
What can be the difference? What can make the first script not work? How can I find out?
Update
Following the accepted answer's recommandations, after the wrong line endings came back, I checked multiple things. It turns out that since I copied my ~/.gitconfig
from my Windows machine, I had autocrlf=true
, so every time I modified the bash file under Windows, it re-set the line endings to rn
.
So, in addition to running dos2unix (which you will have to install using Homebrew on mac), if you're using Git, check your config.
bash shell sh
I am making a NW.js app on Mac, and want to run the app in dev mode by double-clicking on an icon. First step, I'm trying to make my shell script work.
Using VSCode on Windows (I wanted to gain time), I have created a run-nw
file at the root of my project, containing this:
#!/bin/bash
cd "src"
npm install
cd ..
./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &
but I get this output:
$ sh ./run-nw
: command not found
: No such file or directory
: command not found
: No such file or directory
Usage: npm <command>
where <command> is one of: (snip commands list)
(snip npm help)
npm@3.10.3 /usr/local/lib/node_modules/npm
: command not found
: No such file or directory
: command not found
I really don't understand:
- it seems that it takes empty lines as commands. In my editor (VSCode) I have tried to replace
rn
withn
(in case ther
creates problems) but it changes nothing. - it seems that it doesn't find the folders (with or without the
dirname
instruction), or maybe it doesn't know about thecd
command ? - it seems that it doesn't understand the
install
argument tonpm
- the part that really weirds me out, is that it still runs the app (if I did a
npm install
manually)...
Not able to make it work properly, and suspecting something weird with the file itself, I created a new one directly on the Mac, using vim this time. I entered the exact same instructions, and... now it works without any issue.
A diff on the two files reveals exactly zero difference.
What can be the difference? What can make the first script not work? How can I find out?
Update
Following the accepted answer's recommandations, after the wrong line endings came back, I checked multiple things. It turns out that since I copied my ~/.gitconfig
from my Windows machine, I had autocrlf=true
, so every time I modified the bash file under Windows, it re-set the line endings to rn
.
So, in addition to running dos2unix (which you will have to install using Homebrew on mac), if you're using Git, check your config.
bash shell sh
bash shell sh
edited Mar 2 '17 at 8:20
tripleee
98.2k14139194
98.2k14139194
asked Sep 16 '16 at 9:05
thomasbthomasb
3,47934979
3,47934979
3
trydos2unix
?
– PS.
Sep 16 '16 at 9:10
2
If you run a shell script on Linux, at least all the shell implementations I have encountered so far, would get upset if they found a r somewhere. No you say that you have removed the r, and I hope you verified that they are really gone. For the safe side, you should look at your file at the hexadecimal level, to ensure that you don't have other weird characters in it. The next step would then be to execute the script withsh -x ./run-nw
, to get more information.
– user1934428
Sep 16 '16 at 9:34
Another good command to look for weird characters in a text file isLC_ALL=C cat -vet /path/to/file
. If the file is normal, it'll look normal (except for having a "$" at the end of each line). Anything abnormal should stand out fairly well. DOS/Windows files will have "^M$" at the end of lines.
– Gordon Davisson
Sep 16 '16 at 18:03
add a comment |
3
trydos2unix
?
– PS.
Sep 16 '16 at 9:10
2
If you run a shell script on Linux, at least all the shell implementations I have encountered so far, would get upset if they found a r somewhere. No you say that you have removed the r, and I hope you verified that they are really gone. For the safe side, you should look at your file at the hexadecimal level, to ensure that you don't have other weird characters in it. The next step would then be to execute the script withsh -x ./run-nw
, to get more information.
– user1934428
Sep 16 '16 at 9:34
Another good command to look for weird characters in a text file isLC_ALL=C cat -vet /path/to/file
. If the file is normal, it'll look normal (except for having a "$" at the end of each line). Anything abnormal should stand out fairly well. DOS/Windows files will have "^M$" at the end of lines.
– Gordon Davisson
Sep 16 '16 at 18:03
3
3
try
dos2unix
?– PS.
Sep 16 '16 at 9:10
try
dos2unix
?– PS.
Sep 16 '16 at 9:10
2
2
If you run a shell script on Linux, at least all the shell implementations I have encountered so far, would get upset if they found a r somewhere. No you say that you have removed the r, and I hope you verified that they are really gone. For the safe side, you should look at your file at the hexadecimal level, to ensure that you don't have other weird characters in it. The next step would then be to execute the script with
sh -x ./run-nw
, to get more information.– user1934428
Sep 16 '16 at 9:34
If you run a shell script on Linux, at least all the shell implementations I have encountered so far, would get upset if they found a r somewhere. No you say that you have removed the r, and I hope you verified that they are really gone. For the safe side, you should look at your file at the hexadecimal level, to ensure that you don't have other weird characters in it. The next step would then be to execute the script with
sh -x ./run-nw
, to get more information.– user1934428
Sep 16 '16 at 9:34
Another good command to look for weird characters in a text file is
LC_ALL=C cat -vet /path/to/file
. If the file is normal, it'll look normal (except for having a "$" at the end of each line). Anything abnormal should stand out fairly well. DOS/Windows files will have "^M$" at the end of lines.– Gordon Davisson
Sep 16 '16 at 18:03
Another good command to look for weird characters in a text file is
LC_ALL=C cat -vet /path/to/file
. If the file is normal, it'll look normal (except for having a "$" at the end of each line). Anything abnormal should stand out fairly well. DOS/Windows files will have "^M$" at the end of lines.– Gordon Davisson
Sep 16 '16 at 18:03
add a comment |
5 Answers
5
active
oldest
votes
Yes. Bash scripts are sensitive to line-endings, both in the script itself and in data it processes. They should have Unix-style line-endings, i.e., each line is terminated with a Line Feed character (decimal 10, hex 0A in ASCII).
DOS/Windows line endings in the script
With Windows or DOS-style line endings , each line is terminated with a Carriage Return followed by a Line Feed character. If a script file was saved with Windows line endings, Bash sees the file as
#!/bin/bash^M
^M
cd "src"^M
npm install^M
^M
cd ..^M
./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &^M
Note: I’ve used caret notation to represent non-printing characters, i.e., ^M
is used to to represent the Carriage Return characters (represented as r
in other contexts); this is the same technique used by cat -v
and Vim.
In this case, the carriage return (^M
or r
) is not treated as whitespace. Bash interprets the first line after the shebang (consisting of a single carriage return character) as the name of a command/program to run.
- Since there is no command named
^M
, it prints: command not found
- Since there is no directory named
"src"^M
(orsrc^M
), it prints: No such file or directory
- It passes
install^M
instead ofinstall
as an argument tonpm
which causesnpm
to complain.
DOS/Windows line endings in input data
Like above, if you have an input file with carriage returns:
hello^M
world^M
then it will look completely normal in editors and when writing it to screen, but tools may produce strange results. For example, grep
will fail to find lines that are obviously there:
$ grep 'hello$' file.txt || grep -x "hello" file.txt
(no match because the line actually ends in ^M)
Appended text will instead overwrite the line because the carriage returns moves the cursor to the start of the line:
$ sed -e 's/$/!/' file.txt
!ello
!orld
String comparison will seem to fail, even though strings appear to be the same when writing to screen:
$ a="hello"; read b < file.txt
$ if [[ "$a" = "$b" ]]
then echo "Variables are equal."
else echo "Sorry, $a is not equal to $b"
fi
Sorry, hello is not equal to hello
Solutions
The solution is to convert the file to use Unix-style line endings. There are a number of ways this can be accomplished:
This can be done using the
dos2unix
program:dos2unix filename
Open the file in a capable text editor (Sublime, Notepad++, not Notepad) and configure it to save files with Unix line endings, e.g., with Vim, run the following command before (re)saving:
:set fileformat=unix
If you have a version of the
sed
utility that supports the-i
or--in-place
option, e.g., GNUsed
, you could run the following command to strip trailing carriage returns:sed -i 's/r$//' filename
With other versions of
sed
, you could use output redirection to write to a new file. Be sure to use a different filename for the redirection target (it can be renamed later).sed 's/r$//' filename > filename.unix
Similarly, the
tr
translation filter can be used to delete unwanted characters from its input:tr -d 'r' <filename >filename.unix
Cygwin bash
With the Bash port for Cygwin, there’s a custom igncr
option that can be set to ignore the Carriage Return in line endings (presumably because many of its users use native Windows programs to edit their text files). Setting this option applies to the current shell process so it can be useful when sourcing files with extraneous carriage returns.
Useful utilities
The file
utility is useful for quickly seeing which line endings are used in a text file. Here’s what it prints for for each file type:
- Unix line endings:
Bourne-Again shell script, ASCII text executable
- Mac line endings:
Bourne-Again shell script, ASCII text executable, with CR line terminators
- DOS line endings:
Bourne-Again shell script, ASCII text executable, with CRLF line terminators
The GNU version of the cat
utility has a -v, --show-nonprinting
option that displays non-printing characters.
The dos2unix
utility is specifically written for converting text files between Unix, Mac and DOS line endings.
Useful links
Wikipedia has an excellent article covering the many different ways of marking the end of a line of text, the history of such encodings and how newlines are treated in different operating systems, programming languages and Internet protocols (e.g., FTP).
Files with classic Mac OS line endings
With Classic Mac OS (pre-OS X), each line was terminated with a Carriage Return (decimal 13, hex 0D in ASCII). If a script file was saved with such line endings, Bash would only see one long line like so:
#!/bin/bash^M^Mcd "src"^Mnpm install^M^Mcd ..^M./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &^M
Since this single long line begins with an octothorpe (#
), Bash treats the line (and the whole file) as a single comment.
Note: In 2001, Apple launched Mac OS X which was based on the BSD-derived NeXTSTEP operating system. As a result, OS X also uses Unix-style LF-only line endings and since then, text files terminated with a CR have become extremely rare. Nevertheless, I think it’s worthwhile to show how Bash would attempt to interpret such files.
dos2unix did the job for me. thanks for saving million hours.
– Prasad Lakmal
Feb 13 '18 at 10:11
Great explanation, only one little piece is missing here: is there any real reason these days for the genuine bash to continue to treatr
as a meaningful character at the end of the line?
– Alex Cohn
Feb 17 at 11:08
add a comment |
One more way to get rid of the unwanted CR ('r') character is to run the tr
command, for example:
$ tr -d 'r' < dosScript.py > nixScript.py
3
It should be noted that a new user might assume they can also dotr -d 'r' < myFile > myFile
which is NOT a good idea, as theirmyFile
will now be deleted or at least truncated. When using< infile > outFile
redirections, always use different filenames forinfile
andoutfile
. You can then rename as needed. Good luck to all.
– shellter
Apr 5 '18 at 13:53
Also,tr
is unusual in that it refuses to take a file name argument; you have to use redirection liketr x y <inputfile
(nottr x y inputfile
)
– tripleee
Nov 14 '18 at 8:49
add a comment |
On JetBrains products (PyCharm, PHPStorm, IDEA, etc.), you'll need to click on CRLF
/LF
to toggle between the two types of line separators (rn
and n
).
add a comment |
The simplest way on MAC / Linux - create a file using 'touch' command, open this file with VI or VIM editor, paste your code and save. This would automatically remove the windows characters.
1
This is very much not the simplest way, and would not necessarily remove the Windows characters, which are valid characters.
– thomasb
Mar 5 at 23:15
Easy to remember and works.
– danR
Mar 7 at 13:10
True, but copy/pasting in vi/vim is not what I'd call "easiest" :D I'll un-downvote, though.
– thomasb
Mar 8 at 14:08
Agree, kind of a life-hack for guys like me, who are not experts in shell scripting :)
– danR
Mar 11 at 8:32
touch
is a program
– souser12345
Apr 19 at 7:04
add a comment |
Coming from a duplicate, if the problem is that you have files whose names contain ^M
at the end, you can rename them with
for f in *$'r'; do
mv "$f" "$f%$'r'"
done
You properly want to fix whatever caused these files to have broken names in the first place (probably a script which created them should be dos2unix
ed and then rerun?) but sometimes this is not feasible.
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f39527571%2fare-shell-scripts-sensitive-to-encoding-and-line-endings%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
5 Answers
5
active
oldest
votes
5 Answers
5
active
oldest
votes
active
oldest
votes
active
oldest
votes
Yes. Bash scripts are sensitive to line-endings, both in the script itself and in data it processes. They should have Unix-style line-endings, i.e., each line is terminated with a Line Feed character (decimal 10, hex 0A in ASCII).
DOS/Windows line endings in the script
With Windows or DOS-style line endings , each line is terminated with a Carriage Return followed by a Line Feed character. If a script file was saved with Windows line endings, Bash sees the file as
#!/bin/bash^M
^M
cd "src"^M
npm install^M
^M
cd ..^M
./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &^M
Note: I’ve used caret notation to represent non-printing characters, i.e., ^M
is used to to represent the Carriage Return characters (represented as r
in other contexts); this is the same technique used by cat -v
and Vim.
In this case, the carriage return (^M
or r
) is not treated as whitespace. Bash interprets the first line after the shebang (consisting of a single carriage return character) as the name of a command/program to run.
- Since there is no command named
^M
, it prints: command not found
- Since there is no directory named
"src"^M
(orsrc^M
), it prints: No such file or directory
- It passes
install^M
instead ofinstall
as an argument tonpm
which causesnpm
to complain.
DOS/Windows line endings in input data
Like above, if you have an input file with carriage returns:
hello^M
world^M
then it will look completely normal in editors and when writing it to screen, but tools may produce strange results. For example, grep
will fail to find lines that are obviously there:
$ grep 'hello$' file.txt || grep -x "hello" file.txt
(no match because the line actually ends in ^M)
Appended text will instead overwrite the line because the carriage returns moves the cursor to the start of the line:
$ sed -e 's/$/!/' file.txt
!ello
!orld
String comparison will seem to fail, even though strings appear to be the same when writing to screen:
$ a="hello"; read b < file.txt
$ if [[ "$a" = "$b" ]]
then echo "Variables are equal."
else echo "Sorry, $a is not equal to $b"
fi
Sorry, hello is not equal to hello
Solutions
The solution is to convert the file to use Unix-style line endings. There are a number of ways this can be accomplished:
This can be done using the
dos2unix
program:dos2unix filename
Open the file in a capable text editor (Sublime, Notepad++, not Notepad) and configure it to save files with Unix line endings, e.g., with Vim, run the following command before (re)saving:
:set fileformat=unix
If you have a version of the
sed
utility that supports the-i
or--in-place
option, e.g., GNUsed
, you could run the following command to strip trailing carriage returns:sed -i 's/r$//' filename
With other versions of
sed
, you could use output redirection to write to a new file. Be sure to use a different filename for the redirection target (it can be renamed later).sed 's/r$//' filename > filename.unix
Similarly, the
tr
translation filter can be used to delete unwanted characters from its input:tr -d 'r' <filename >filename.unix
Cygwin bash
With the Bash port for Cygwin, there’s a custom igncr
option that can be set to ignore the Carriage Return in line endings (presumably because many of its users use native Windows programs to edit their text files). Setting this option applies to the current shell process so it can be useful when sourcing files with extraneous carriage returns.
Useful utilities
The file
utility is useful for quickly seeing which line endings are used in a text file. Here’s what it prints for for each file type:
- Unix line endings:
Bourne-Again shell script, ASCII text executable
- Mac line endings:
Bourne-Again shell script, ASCII text executable, with CR line terminators
- DOS line endings:
Bourne-Again shell script, ASCII text executable, with CRLF line terminators
The GNU version of the cat
utility has a -v, --show-nonprinting
option that displays non-printing characters.
The dos2unix
utility is specifically written for converting text files between Unix, Mac and DOS line endings.
Useful links
Wikipedia has an excellent article covering the many different ways of marking the end of a line of text, the history of such encodings and how newlines are treated in different operating systems, programming languages and Internet protocols (e.g., FTP).
Files with classic Mac OS line endings
With Classic Mac OS (pre-OS X), each line was terminated with a Carriage Return (decimal 13, hex 0D in ASCII). If a script file was saved with such line endings, Bash would only see one long line like so:
#!/bin/bash^M^Mcd "src"^Mnpm install^M^Mcd ..^M./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &^M
Since this single long line begins with an octothorpe (#
), Bash treats the line (and the whole file) as a single comment.
Note: In 2001, Apple launched Mac OS X which was based on the BSD-derived NeXTSTEP operating system. As a result, OS X also uses Unix-style LF-only line endings and since then, text files terminated with a CR have become extremely rare. Nevertheless, I think it’s worthwhile to show how Bash would attempt to interpret such files.
dos2unix did the job for me. thanks for saving million hours.
– Prasad Lakmal
Feb 13 '18 at 10:11
Great explanation, only one little piece is missing here: is there any real reason these days for the genuine bash to continue to treatr
as a meaningful character at the end of the line?
– Alex Cohn
Feb 17 at 11:08
add a comment |
Yes. Bash scripts are sensitive to line-endings, both in the script itself and in data it processes. They should have Unix-style line-endings, i.e., each line is terminated with a Line Feed character (decimal 10, hex 0A in ASCII).
DOS/Windows line endings in the script
With Windows or DOS-style line endings , each line is terminated with a Carriage Return followed by a Line Feed character. If a script file was saved with Windows line endings, Bash sees the file as
#!/bin/bash^M
^M
cd "src"^M
npm install^M
^M
cd ..^M
./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &^M
Note: I’ve used caret notation to represent non-printing characters, i.e., ^M
is used to to represent the Carriage Return characters (represented as r
in other contexts); this is the same technique used by cat -v
and Vim.
In this case, the carriage return (^M
or r
) is not treated as whitespace. Bash interprets the first line after the shebang (consisting of a single carriage return character) as the name of a command/program to run.
- Since there is no command named
^M
, it prints: command not found
- Since there is no directory named
"src"^M
(orsrc^M
), it prints: No such file or directory
- It passes
install^M
instead ofinstall
as an argument tonpm
which causesnpm
to complain.
DOS/Windows line endings in input data
Like above, if you have an input file with carriage returns:
hello^M
world^M
then it will look completely normal in editors and when writing it to screen, but tools may produce strange results. For example, grep
will fail to find lines that are obviously there:
$ grep 'hello$' file.txt || grep -x "hello" file.txt
(no match because the line actually ends in ^M)
Appended text will instead overwrite the line because the carriage returns moves the cursor to the start of the line:
$ sed -e 's/$/!/' file.txt
!ello
!orld
String comparison will seem to fail, even though strings appear to be the same when writing to screen:
$ a="hello"; read b < file.txt
$ if [[ "$a" = "$b" ]]
then echo "Variables are equal."
else echo "Sorry, $a is not equal to $b"
fi
Sorry, hello is not equal to hello
Solutions
The solution is to convert the file to use Unix-style line endings. There are a number of ways this can be accomplished:
This can be done using the
dos2unix
program:dos2unix filename
Open the file in a capable text editor (Sublime, Notepad++, not Notepad) and configure it to save files with Unix line endings, e.g., with Vim, run the following command before (re)saving:
:set fileformat=unix
If you have a version of the
sed
utility that supports the-i
or--in-place
option, e.g., GNUsed
, you could run the following command to strip trailing carriage returns:sed -i 's/r$//' filename
With other versions of
sed
, you could use output redirection to write to a new file. Be sure to use a different filename for the redirection target (it can be renamed later).sed 's/r$//' filename > filename.unix
Similarly, the
tr
translation filter can be used to delete unwanted characters from its input:tr -d 'r' <filename >filename.unix
Cygwin bash
With the Bash port for Cygwin, there’s a custom igncr
option that can be set to ignore the Carriage Return in line endings (presumably because many of its users use native Windows programs to edit their text files). Setting this option applies to the current shell process so it can be useful when sourcing files with extraneous carriage returns.
Useful utilities
The file
utility is useful for quickly seeing which line endings are used in a text file. Here’s what it prints for for each file type:
- Unix line endings:
Bourne-Again shell script, ASCII text executable
- Mac line endings:
Bourne-Again shell script, ASCII text executable, with CR line terminators
- DOS line endings:
Bourne-Again shell script, ASCII text executable, with CRLF line terminators
The GNU version of the cat
utility has a -v, --show-nonprinting
option that displays non-printing characters.
The dos2unix
utility is specifically written for converting text files between Unix, Mac and DOS line endings.
Useful links
Wikipedia has an excellent article covering the many different ways of marking the end of a line of text, the history of such encodings and how newlines are treated in different operating systems, programming languages and Internet protocols (e.g., FTP).
Files with classic Mac OS line endings
With Classic Mac OS (pre-OS X), each line was terminated with a Carriage Return (decimal 13, hex 0D in ASCII). If a script file was saved with such line endings, Bash would only see one long line like so:
#!/bin/bash^M^Mcd "src"^Mnpm install^M^Mcd ..^M./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &^M
Since this single long line begins with an octothorpe (#
), Bash treats the line (and the whole file) as a single comment.
Note: In 2001, Apple launched Mac OS X which was based on the BSD-derived NeXTSTEP operating system. As a result, OS X also uses Unix-style LF-only line endings and since then, text files terminated with a CR have become extremely rare. Nevertheless, I think it’s worthwhile to show how Bash would attempt to interpret such files.
dos2unix did the job for me. thanks for saving million hours.
– Prasad Lakmal
Feb 13 '18 at 10:11
Great explanation, only one little piece is missing here: is there any real reason these days for the genuine bash to continue to treatr
as a meaningful character at the end of the line?
– Alex Cohn
Feb 17 at 11:08
add a comment |
Yes. Bash scripts are sensitive to line-endings, both in the script itself and in data it processes. They should have Unix-style line-endings, i.e., each line is terminated with a Line Feed character (decimal 10, hex 0A in ASCII).
DOS/Windows line endings in the script
With Windows or DOS-style line endings , each line is terminated with a Carriage Return followed by a Line Feed character. If a script file was saved with Windows line endings, Bash sees the file as
#!/bin/bash^M
^M
cd "src"^M
npm install^M
^M
cd ..^M
./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &^M
Note: I’ve used caret notation to represent non-printing characters, i.e., ^M
is used to to represent the Carriage Return characters (represented as r
in other contexts); this is the same technique used by cat -v
and Vim.
In this case, the carriage return (^M
or r
) is not treated as whitespace. Bash interprets the first line after the shebang (consisting of a single carriage return character) as the name of a command/program to run.
- Since there is no command named
^M
, it prints: command not found
- Since there is no directory named
"src"^M
(orsrc^M
), it prints: No such file or directory
- It passes
install^M
instead ofinstall
as an argument tonpm
which causesnpm
to complain.
DOS/Windows line endings in input data
Like above, if you have an input file with carriage returns:
hello^M
world^M
then it will look completely normal in editors and when writing it to screen, but tools may produce strange results. For example, grep
will fail to find lines that are obviously there:
$ grep 'hello$' file.txt || grep -x "hello" file.txt
(no match because the line actually ends in ^M)
Appended text will instead overwrite the line because the carriage returns moves the cursor to the start of the line:
$ sed -e 's/$/!/' file.txt
!ello
!orld
String comparison will seem to fail, even though strings appear to be the same when writing to screen:
$ a="hello"; read b < file.txt
$ if [[ "$a" = "$b" ]]
then echo "Variables are equal."
else echo "Sorry, $a is not equal to $b"
fi
Sorry, hello is not equal to hello
Solutions
The solution is to convert the file to use Unix-style line endings. There are a number of ways this can be accomplished:
This can be done using the
dos2unix
program:dos2unix filename
Open the file in a capable text editor (Sublime, Notepad++, not Notepad) and configure it to save files with Unix line endings, e.g., with Vim, run the following command before (re)saving:
:set fileformat=unix
If you have a version of the
sed
utility that supports the-i
or--in-place
option, e.g., GNUsed
, you could run the following command to strip trailing carriage returns:sed -i 's/r$//' filename
With other versions of
sed
, you could use output redirection to write to a new file. Be sure to use a different filename for the redirection target (it can be renamed later).sed 's/r$//' filename > filename.unix
Similarly, the
tr
translation filter can be used to delete unwanted characters from its input:tr -d 'r' <filename >filename.unix
Cygwin bash
With the Bash port for Cygwin, there’s a custom igncr
option that can be set to ignore the Carriage Return in line endings (presumably because many of its users use native Windows programs to edit their text files). Setting this option applies to the current shell process so it can be useful when sourcing files with extraneous carriage returns.
Useful utilities
The file
utility is useful for quickly seeing which line endings are used in a text file. Here’s what it prints for for each file type:
- Unix line endings:
Bourne-Again shell script, ASCII text executable
- Mac line endings:
Bourne-Again shell script, ASCII text executable, with CR line terminators
- DOS line endings:
Bourne-Again shell script, ASCII text executable, with CRLF line terminators
The GNU version of the cat
utility has a -v, --show-nonprinting
option that displays non-printing characters.
The dos2unix
utility is specifically written for converting text files between Unix, Mac and DOS line endings.
Useful links
Wikipedia has an excellent article covering the many different ways of marking the end of a line of text, the history of such encodings and how newlines are treated in different operating systems, programming languages and Internet protocols (e.g., FTP).
Files with classic Mac OS line endings
With Classic Mac OS (pre-OS X), each line was terminated with a Carriage Return (decimal 13, hex 0D in ASCII). If a script file was saved with such line endings, Bash would only see one long line like so:
#!/bin/bash^M^Mcd "src"^Mnpm install^M^Mcd ..^M./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &^M
Since this single long line begins with an octothorpe (#
), Bash treats the line (and the whole file) as a single comment.
Note: In 2001, Apple launched Mac OS X which was based on the BSD-derived NeXTSTEP operating system. As a result, OS X also uses Unix-style LF-only line endings and since then, text files terminated with a CR have become extremely rare. Nevertheless, I think it’s worthwhile to show how Bash would attempt to interpret such files.
Yes. Bash scripts are sensitive to line-endings, both in the script itself and in data it processes. They should have Unix-style line-endings, i.e., each line is terminated with a Line Feed character (decimal 10, hex 0A in ASCII).
DOS/Windows line endings in the script
With Windows or DOS-style line endings , each line is terminated with a Carriage Return followed by a Line Feed character. If a script file was saved with Windows line endings, Bash sees the file as
#!/bin/bash^M
^M
cd "src"^M
npm install^M
^M
cd ..^M
./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &^M
Note: I’ve used caret notation to represent non-printing characters, i.e., ^M
is used to to represent the Carriage Return characters (represented as r
in other contexts); this is the same technique used by cat -v
and Vim.
In this case, the carriage return (^M
or r
) is not treated as whitespace. Bash interprets the first line after the shebang (consisting of a single carriage return character) as the name of a command/program to run.
- Since there is no command named
^M
, it prints: command not found
- Since there is no directory named
"src"^M
(orsrc^M
), it prints: No such file or directory
- It passes
install^M
instead ofinstall
as an argument tonpm
which causesnpm
to complain.
DOS/Windows line endings in input data
Like above, if you have an input file with carriage returns:
hello^M
world^M
then it will look completely normal in editors and when writing it to screen, but tools may produce strange results. For example, grep
will fail to find lines that are obviously there:
$ grep 'hello$' file.txt || grep -x "hello" file.txt
(no match because the line actually ends in ^M)
Appended text will instead overwrite the line because the carriage returns moves the cursor to the start of the line:
$ sed -e 's/$/!/' file.txt
!ello
!orld
String comparison will seem to fail, even though strings appear to be the same when writing to screen:
$ a="hello"; read b < file.txt
$ if [[ "$a" = "$b" ]]
then echo "Variables are equal."
else echo "Sorry, $a is not equal to $b"
fi
Sorry, hello is not equal to hello
Solutions
The solution is to convert the file to use Unix-style line endings. There are a number of ways this can be accomplished:
This can be done using the
dos2unix
program:dos2unix filename
Open the file in a capable text editor (Sublime, Notepad++, not Notepad) and configure it to save files with Unix line endings, e.g., with Vim, run the following command before (re)saving:
:set fileformat=unix
If you have a version of the
sed
utility that supports the-i
or--in-place
option, e.g., GNUsed
, you could run the following command to strip trailing carriage returns:sed -i 's/r$//' filename
With other versions of
sed
, you could use output redirection to write to a new file. Be sure to use a different filename for the redirection target (it can be renamed later).sed 's/r$//' filename > filename.unix
Similarly, the
tr
translation filter can be used to delete unwanted characters from its input:tr -d 'r' <filename >filename.unix
Cygwin bash
With the Bash port for Cygwin, there’s a custom igncr
option that can be set to ignore the Carriage Return in line endings (presumably because many of its users use native Windows programs to edit their text files). Setting this option applies to the current shell process so it can be useful when sourcing files with extraneous carriage returns.
Useful utilities
The file
utility is useful for quickly seeing which line endings are used in a text file. Here’s what it prints for for each file type:
- Unix line endings:
Bourne-Again shell script, ASCII text executable
- Mac line endings:
Bourne-Again shell script, ASCII text executable, with CR line terminators
- DOS line endings:
Bourne-Again shell script, ASCII text executable, with CRLF line terminators
The GNU version of the cat
utility has a -v, --show-nonprinting
option that displays non-printing characters.
The dos2unix
utility is specifically written for converting text files between Unix, Mac and DOS line endings.
Useful links
Wikipedia has an excellent article covering the many different ways of marking the end of a line of text, the history of such encodings and how newlines are treated in different operating systems, programming languages and Internet protocols (e.g., FTP).
Files with classic Mac OS line endings
With Classic Mac OS (pre-OS X), each line was terminated with a Carriage Return (decimal 13, hex 0D in ASCII). If a script file was saved with such line endings, Bash would only see one long line like so:
#!/bin/bash^M^Mcd "src"^Mnpm install^M^Mcd ..^M./tools/nwjs-sdk-v0.17.3-osx-x64/nwjs.app/Contents/MacOS/nwjs "src" &^M
Since this single long line begins with an octothorpe (#
), Bash treats the line (and the whole file) as a single comment.
Note: In 2001, Apple launched Mac OS X which was based on the BSD-derived NeXTSTEP operating system. As a result, OS X also uses Unix-style LF-only line endings and since then, text files terminated with a CR have become extremely rare. Nevertheless, I think it’s worthwhile to show how Bash would attempt to interpret such files.
edited Aug 2 '18 at 17:38
that other guy
77.1k890127
77.1k890127
answered Sep 16 '16 at 9:26
Anthony GeogheganAnthony Geoghegan
7,94543544
7,94543544
dos2unix did the job for me. thanks for saving million hours.
– Prasad Lakmal
Feb 13 '18 at 10:11
Great explanation, only one little piece is missing here: is there any real reason these days for the genuine bash to continue to treatr
as a meaningful character at the end of the line?
– Alex Cohn
Feb 17 at 11:08
add a comment |
dos2unix did the job for me. thanks for saving million hours.
– Prasad Lakmal
Feb 13 '18 at 10:11
Great explanation, only one little piece is missing here: is there any real reason these days for the genuine bash to continue to treatr
as a meaningful character at the end of the line?
– Alex Cohn
Feb 17 at 11:08
dos2unix did the job for me. thanks for saving million hours.
– Prasad Lakmal
Feb 13 '18 at 10:11
dos2unix did the job for me. thanks for saving million hours.
– Prasad Lakmal
Feb 13 '18 at 10:11
Great explanation, only one little piece is missing here: is there any real reason these days for the genuine bash to continue to treat
r
as a meaningful character at the end of the line?– Alex Cohn
Feb 17 at 11:08
Great explanation, only one little piece is missing here: is there any real reason these days for the genuine bash to continue to treat
r
as a meaningful character at the end of the line?– Alex Cohn
Feb 17 at 11:08
add a comment |
One more way to get rid of the unwanted CR ('r') character is to run the tr
command, for example:
$ tr -d 'r' < dosScript.py > nixScript.py
3
It should be noted that a new user might assume they can also dotr -d 'r' < myFile > myFile
which is NOT a good idea, as theirmyFile
will now be deleted or at least truncated. When using< infile > outFile
redirections, always use different filenames forinfile
andoutfile
. You can then rename as needed. Good luck to all.
– shellter
Apr 5 '18 at 13:53
Also,tr
is unusual in that it refuses to take a file name argument; you have to use redirection liketr x y <inputfile
(nottr x y inputfile
)
– tripleee
Nov 14 '18 at 8:49
add a comment |
One more way to get rid of the unwanted CR ('r') character is to run the tr
command, for example:
$ tr -d 'r' < dosScript.py > nixScript.py
3
It should be noted that a new user might assume they can also dotr -d 'r' < myFile > myFile
which is NOT a good idea, as theirmyFile
will now be deleted or at least truncated. When using< infile > outFile
redirections, always use different filenames forinfile
andoutfile
. You can then rename as needed. Good luck to all.
– shellter
Apr 5 '18 at 13:53
Also,tr
is unusual in that it refuses to take a file name argument; you have to use redirection liketr x y <inputfile
(nottr x y inputfile
)
– tripleee
Nov 14 '18 at 8:49
add a comment |
One more way to get rid of the unwanted CR ('r') character is to run the tr
command, for example:
$ tr -d 'r' < dosScript.py > nixScript.py
One more way to get rid of the unwanted CR ('r') character is to run the tr
command, for example:
$ tr -d 'r' < dosScript.py > nixScript.py
answered Mar 2 '18 at 19:25
Igor SoudakevitchIgor Soudakevitch
415715
415715
3
It should be noted that a new user might assume they can also dotr -d 'r' < myFile > myFile
which is NOT a good idea, as theirmyFile
will now be deleted or at least truncated. When using< infile > outFile
redirections, always use different filenames forinfile
andoutfile
. You can then rename as needed. Good luck to all.
– shellter
Apr 5 '18 at 13:53
Also,tr
is unusual in that it refuses to take a file name argument; you have to use redirection liketr x y <inputfile
(nottr x y inputfile
)
– tripleee
Nov 14 '18 at 8:49
add a comment |
3
It should be noted that a new user might assume they can also dotr -d 'r' < myFile > myFile
which is NOT a good idea, as theirmyFile
will now be deleted or at least truncated. When using< infile > outFile
redirections, always use different filenames forinfile
andoutfile
. You can then rename as needed. Good luck to all.
– shellter
Apr 5 '18 at 13:53
Also,tr
is unusual in that it refuses to take a file name argument; you have to use redirection liketr x y <inputfile
(nottr x y inputfile
)
– tripleee
Nov 14 '18 at 8:49
3
3
It should be noted that a new user might assume they can also do
tr -d 'r' < myFile > myFile
which is NOT a good idea, as their myFile
will now be deleted or at least truncated. When using < infile > outFile
redirections, always use different filenames for infile
and outfile
. You can then rename as needed. Good luck to all.– shellter
Apr 5 '18 at 13:53
It should be noted that a new user might assume they can also do
tr -d 'r' < myFile > myFile
which is NOT a good idea, as their myFile
will now be deleted or at least truncated. When using < infile > outFile
redirections, always use different filenames for infile
and outfile
. You can then rename as needed. Good luck to all.– shellter
Apr 5 '18 at 13:53
Also,
tr
is unusual in that it refuses to take a file name argument; you have to use redirection like tr x y <inputfile
(not tr x y inputfile
)– tripleee
Nov 14 '18 at 8:49
Also,
tr
is unusual in that it refuses to take a file name argument; you have to use redirection like tr x y <inputfile
(not tr x y inputfile
)– tripleee
Nov 14 '18 at 8:49
add a comment |
On JetBrains products (PyCharm, PHPStorm, IDEA, etc.), you'll need to click on CRLF
/LF
to toggle between the two types of line separators (rn
and n
).
add a comment |
On JetBrains products (PyCharm, PHPStorm, IDEA, etc.), you'll need to click on CRLF
/LF
to toggle between the two types of line separators (rn
and n
).
add a comment |
On JetBrains products (PyCharm, PHPStorm, IDEA, etc.), you'll need to click on CRLF
/LF
to toggle between the two types of line separators (rn
and n
).
On JetBrains products (PyCharm, PHPStorm, IDEA, etc.), you'll need to click on CRLF
/LF
to toggle between the two types of line separators (rn
and n
).
edited Mar 27 at 7:11
piet.t
10.2k73246
10.2k73246
answered Feb 26 at 3:39
Pedro LobitoPedro Lobito
51.9k16143173
51.9k16143173
add a comment |
add a comment |
The simplest way on MAC / Linux - create a file using 'touch' command, open this file with VI or VIM editor, paste your code and save. This would automatically remove the windows characters.
1
This is very much not the simplest way, and would not necessarily remove the Windows characters, which are valid characters.
– thomasb
Mar 5 at 23:15
Easy to remember and works.
– danR
Mar 7 at 13:10
True, but copy/pasting in vi/vim is not what I'd call "easiest" :D I'll un-downvote, though.
– thomasb
Mar 8 at 14:08
Agree, kind of a life-hack for guys like me, who are not experts in shell scripting :)
– danR
Mar 11 at 8:32
touch
is a program
– souser12345
Apr 19 at 7:04
add a comment |
The simplest way on MAC / Linux - create a file using 'touch' command, open this file with VI or VIM editor, paste your code and save. This would automatically remove the windows characters.
1
This is very much not the simplest way, and would not necessarily remove the Windows characters, which are valid characters.
– thomasb
Mar 5 at 23:15
Easy to remember and works.
– danR
Mar 7 at 13:10
True, but copy/pasting in vi/vim is not what I'd call "easiest" :D I'll un-downvote, though.
– thomasb
Mar 8 at 14:08
Agree, kind of a life-hack for guys like me, who are not experts in shell scripting :)
– danR
Mar 11 at 8:32
touch
is a program
– souser12345
Apr 19 at 7:04
add a comment |
The simplest way on MAC / Linux - create a file using 'touch' command, open this file with VI or VIM editor, paste your code and save. This would automatically remove the windows characters.
The simplest way on MAC / Linux - create a file using 'touch' command, open this file with VI or VIM editor, paste your code and save. This would automatically remove the windows characters.
edited Mar 7 at 13:07
answered Mar 5 at 18:01
danRdanR
4681615
4681615
1
This is very much not the simplest way, and would not necessarily remove the Windows characters, which are valid characters.
– thomasb
Mar 5 at 23:15
Easy to remember and works.
– danR
Mar 7 at 13:10
True, but copy/pasting in vi/vim is not what I'd call "easiest" :D I'll un-downvote, though.
– thomasb
Mar 8 at 14:08
Agree, kind of a life-hack for guys like me, who are not experts in shell scripting :)
– danR
Mar 11 at 8:32
touch
is a program
– souser12345
Apr 19 at 7:04
add a comment |
1
This is very much not the simplest way, and would not necessarily remove the Windows characters, which are valid characters.
– thomasb
Mar 5 at 23:15
Easy to remember and works.
– danR
Mar 7 at 13:10
True, but copy/pasting in vi/vim is not what I'd call "easiest" :D I'll un-downvote, though.
– thomasb
Mar 8 at 14:08
Agree, kind of a life-hack for guys like me, who are not experts in shell scripting :)
– danR
Mar 11 at 8:32
touch
is a program
– souser12345
Apr 19 at 7:04
1
1
This is very much not the simplest way, and would not necessarily remove the Windows characters, which are valid characters.
– thomasb
Mar 5 at 23:15
This is very much not the simplest way, and would not necessarily remove the Windows characters, which are valid characters.
– thomasb
Mar 5 at 23:15
Easy to remember and works.
– danR
Mar 7 at 13:10
Easy to remember and works.
– danR
Mar 7 at 13:10
True, but copy/pasting in vi/vim is not what I'd call "easiest" :D I'll un-downvote, though.
– thomasb
Mar 8 at 14:08
True, but copy/pasting in vi/vim is not what I'd call "easiest" :D I'll un-downvote, though.
– thomasb
Mar 8 at 14:08
Agree, kind of a life-hack for guys like me, who are not experts in shell scripting :)
– danR
Mar 11 at 8:32
Agree, kind of a life-hack for guys like me, who are not experts in shell scripting :)
– danR
Mar 11 at 8:32
touch
is a program– souser12345
Apr 19 at 7:04
touch
is a program– souser12345
Apr 19 at 7:04
add a comment |
Coming from a duplicate, if the problem is that you have files whose names contain ^M
at the end, you can rename them with
for f in *$'r'; do
mv "$f" "$f%$'r'"
done
You properly want to fix whatever caused these files to have broken names in the first place (probably a script which created them should be dos2unix
ed and then rerun?) but sometimes this is not feasible.
add a comment |
Coming from a duplicate, if the problem is that you have files whose names contain ^M
at the end, you can rename them with
for f in *$'r'; do
mv "$f" "$f%$'r'"
done
You properly want to fix whatever caused these files to have broken names in the first place (probably a script which created them should be dos2unix
ed and then rerun?) but sometimes this is not feasible.
add a comment |
Coming from a duplicate, if the problem is that you have files whose names contain ^M
at the end, you can rename them with
for f in *$'r'; do
mv "$f" "$f%$'r'"
done
You properly want to fix whatever caused these files to have broken names in the first place (probably a script which created them should be dos2unix
ed and then rerun?) but sometimes this is not feasible.
Coming from a duplicate, if the problem is that you have files whose names contain ^M
at the end, you can rename them with
for f in *$'r'; do
mv "$f" "$f%$'r'"
done
You properly want to fix whatever caused these files to have broken names in the first place (probably a script which created them should be dos2unix
ed and then rerun?) but sometimes this is not feasible.
answered Apr 19 at 6:12
tripleeetripleee
98.2k14139194
98.2k14139194
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f39527571%2fare-shell-scripts-sensitive-to-encoding-and-line-endings%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
3
try
dos2unix
?– PS.
Sep 16 '16 at 9:10
2
If you run a shell script on Linux, at least all the shell implementations I have encountered so far, would get upset if they found a r somewhere. No you say that you have removed the r, and I hope you verified that they are really gone. For the safe side, you should look at your file at the hexadecimal level, to ensure that you don't have other weird characters in it. The next step would then be to execute the script with
sh -x ./run-nw
, to get more information.– user1934428
Sep 16 '16 at 9:34
Another good command to look for weird characters in a text file is
LC_ALL=C cat -vet /path/to/file
. If the file is normal, it'll look normal (except for having a "$" at the end of each line). Anything abnormal should stand out fairly well. DOS/Windows files will have "^M$" at the end of lines.– Gordon Davisson
Sep 16 '16 at 18:03