[bash] How do I escape double and single quotes in sed?

From what I can find, when you use single quotes everything inside is considered literal. I want that for my substitution. But I also want to find a string that has single or double quotes.

For example,

sed -i 's/"http://www.fubar.com"/URL_FUBAR/g'

I want to replace "http://www.fubar.com" with URL_FUBAR. How is sed supposed to recognize my // or my double quotes?

Thanks for any help!

EDIT: Could I use s/\"http\:\/\/www\.fubar\.\com\"/URL_FUBAR/g ?

Does \ actually escape chars inside the single quotes?

This question is related to bash command-line sed

The answer is


May be the "\" char, try this one:

sed 's/\"http:\/\/www.fubar.com\"/URL_FUBAR/g'

You can use %

sed -i "s%http://www.fubar.com%URL_FUBAR%g"

Regarding the single quote, see the code below used to replace the string let's with let us:

command:

echo "hello, let's go"|sed 's/let'"'"'s/let us/g'

result:

hello, let us go


Aside: sed expressions containing BASH variables need to be double (")-quoted for the variable to be interpreted correctly.

If you also double-quote your $BASH variable (recommended practice)

... then you can escape the variable double quotes as shown:

sed -i "s/foo/bar ""$VARIABLE""/g" <file>

I.e., replace the $VARIABLE-associated " with "".

(Simply -escaping "$VAR" as \"$VAR\" results in a "-quoted output string.)


Examples

$ VAR='apples and bananas'
$ echo $VAR
apples and bananas

$ echo "$VAR"
apples and bananas

$ printf 'I like %s!\n' $VAR
I like apples!
I like and!
I like bananas!

$ printf 'I like %s!\n' "$VAR"
I like apples and bananas!

Here, $VAR is "-quoted before piping to sed (sed is either '- or "-quoted):

$ printf 'I like %s!\n' "$VAR" | sed 's/$VAR/cherries/g'
I like apples and bananas!

$ printf 'I like %s!\n' "$VAR" | sed 's/"$VAR"/cherries/g'
I like apples and bananas!

$ printf 'I like %s!\n' "$VAR" | sed 's/$VAR/cherries/g'
I like apples and bananas!

$ printf 'I like %s!\n' "$VAR" | sed 's/""$VAR""/cherries/g'
I like apples and bananas!

$ printf 'I like %s!\n' "$VAR" | sed "s/$VAR/cherries/g"
I like cherries!

$ printf 'I like %s!\n' "$VAR" | sed "s/""$VAR""/cherries/g"
I like cherries!

Compare that to:

$ printf 'I like %s!\n' $VAR | sed "s/$VAR/cherries/g"
I like apples!
I like and!
I like bananas!


$ printf 'I like %s!\n' $VAR | sed "s/""$VAR""/cherries/g"
I like apples!
I like and!
I like bananas!

... and so on ...

Conclusion

My recommendation, as standard practice, is to

  • "-quote BASH variables ("$VAR")
  • "-quote, again, those variables (""$VAR"") if they are used in a sed expression (which itself must be "-quoted, not '-quoted)
$ VAR='apples and bananas'

$ echo "$VAR"
apples and bananas

$ printf 'I like %s!\n' "$VAR" | sed "s/""$VAR""/cherries/g"
I like cherries!


You need to use \" for escaping " character (\ escape the following character

sed -i 's/\"http://www.fubar.com\"/URL_FUBAR/g'

Prompt% cat t1
This is "Unix"
This is "Unix sed"
Prompt% sed -i 's/\"Unix\"/\"Linux\"/g' t1
Prompt% sed -i 's/\"Unix sed\"/\"Linux SED\"/g' t1
Prompt% cat t1
This is "Linux"
This is "Linux SED"
Prompt%

My problem was that I needed to have the "" outside the expression since I have a dynamic variable inside the sed expression itself. So than the actual solution is that one from lenn jackman that you replace the " inside the sed regex with [\"].

So my complete bash is:

RELEASE_VERSION="0.6.6"

sed -i -e "s#value=[\"]trunk[\"]#value=\"tags/$RELEASE_VERSION\"#g" myfile.xml

Here is:

# is the sed separator

[\"] = " in regex

value = \"tags/$RELEASE_VERSION\" = my replacement string, important it has just the \" for the quotes


It's hard to escape a single quote within single quotes. Try this:

sed "s@['\"]http://www.\([^.]\+).com['\"]@URL_\U\1@g" 

Example:

$ sed "s@['\"]http://www.\([^.]\+\).com['\"]@URL_\U\1@g" <<END
this is "http://www.fubar.com" and 'http://www.example.com' here
END

produces

this is URL_FUBAR and URL_EXAMPLE here

Escaping a double quote can absolutely be necessary in sed: for instance, if you are using double quotes in the entire sed expression (as you need to do when you want to use a shell variable).

Here's an example that touches on escaping in sed but also captures some other quoting issues in bash:

# cat inventory
PURCHASED="2014-09-01"
SITE="Atlanta"
LOCATION="Room 154"

Let's say you wanted to change the room using a sed script that you can use over and over, so you variablize the input as follows:

# i="Room 101" (these quotes are there so the variable can contains spaces)

This script will add the whole line if it isn't there, or it will simply replace (using sed) the line that is there with the text plus the value of $i.

if grep -q LOCATION inventory; then 
## The sed expression is double quoted to allow for variable expansion; 
## the literal quotes are both escaped with \ 
    sed -i "/^LOCATION/c\LOCATION=\"$i\"" inventory
## Note the three layers of quotes to get echo to expand the variable
## AND insert the literal quotes
else 
    echo LOCATION='"'$i'"' >> inventory
fi

P.S. I wrote out the script above on multiple lines to make the comments parsable but I use it as a one-liner on the command line that looks like this:

i="your location"; if grep -q LOCATION inventory; then sed -i "/^LOCATION/c\LOCATION=\"$i\"" inventory; else echo LOCATION='"'$i'"' >> inventory; fi

Examples related to bash

Comparing a variable with a string python not working when redirecting from bash script Zipping a file in bash fails How do I prevent Conda from activating the base environment by default? Get first line of a shell command's output Fixing a systemd service 203/EXEC failure (no such file or directory) /bin/sh: apt-get: not found VSCode Change Default Terminal Run bash command on jenkins pipeline How to check if the docker engine and a docker container are running? How to switch Python versions in Terminal?

Examples related to command-line

Git is not working after macOS Update (xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools) Flutter command not found Angular - ng: command not found how to run python files in windows command prompt? How to run .NET Core console app from the command line Copy Paste in Bash on Ubuntu on Windows How to find which version of TensorFlow is installed in my system? How to install JQ on Mac by command-line? Python not working in the command line of git bash Run function in script from command line (Node JS)

Examples related to sed

Retrieve last 100 lines logs How to replace multiple patterns at once with sed? Insert multiple lines into a file after specified pattern using shell script Linux bash script to extract IP address Ansible playbook shell output remove white space from the end of line in linux bash, extract string before a colon invalid command code ., despite escaping periods, using sed RE error: illegal byte sequence on Mac OS X How to use variables in a command in sed?