Search and replace in bash using regular expressions

Question

I ve seen this example   hello ho02123ware38384you443d34o3434ingtod38384day echo   hello   0-9      Which follows this syntax    variable  pattern replacement   Unfortunately the pattern field doesn t seem to support full regex syntax  if I use   or  s  for example  it tries to match the literal characters    How can I search replace a string using full regex syntax

User · Answer

These examples also work in bash no need to use sed:

#!/bin/bash
MYVAR=ho02123ware38384you443d34o3434ingtod38384day
MYVAR=${MYVAR//[a-zA-Z]/X} 
echo ${MYVAR//[0-9]/N}

you can also use the character class bracket expressions

#!/bin/bash
MYVAR=ho02123ware38384you443d34o3434ingtod38384day
MYVAR=${MYVAR//[[:alpha:]]/X} 
echo ${MYVAR//[[:digit:]]/N}

output

XXNNNNNXXXXNNNNNXXXNNNXNNXNNNNXXXXXXNNNNNXXX

What @Lanaru wanted to know however, if I understand the question correctly, is why the "full" or PCRE extensions \s\S\w\W\d\D etc don't work as supported in php ruby python etc. These extensions are from Perl-compatible regular expressions (PCRE) and may not be compatible with other forms of shell based regular expressions.

These don't work:

#!/bin/bash
hello=ho02123ware38384you443d34o3434ingtod38384day
echo ${hello//\d/}


#!/bin/bash
hello=ho02123ware38384you443d34o3434ingtod38384day
echo $hello | sed 's/\d//g'

output with all literal "d" characters removed

ho02123ware38384you44334o3434ingto38384ay

but the following does work as expected

#!/bin/bash
hello=ho02123ware38384you443d34o3434ingtod38384day
echo $hello | perl -pe 's/\d//g'

output

howareyoudoingtodday

Hope that clarifies things a bit more but if you are not confused yet why don't you try this on Mac OS X which has the REG_ENHANCED flag enabled:

#!/bin/bash
MYVAR=ho02123ware38384you443d34o3434ingtod38384day;
echo $MYVAR | grep -o -E '\d'

On most flavours of *nix you will only see the following output:

d
d
d

nJoy!

User · Answer

Use    digit     note the double brackets  as the pattern     hello ho02123ware38384you443d34o3434ingtod38384day   echo   hello     digit      howareyoudoingtodday     Just wanted to summarize the answers  especially  nickl- s https   stackoverflow com a 22261334 2916086

User · Answer

I know this is an ancient thread  but it was my first hit on Google  and I wanted to share the following resub that I put together  which adds support for multiple  1   2  etc  backreferences       usr bin env bash                                                    resub - regex substitution in bash                                                    resub         local match  quot  1 quot  subst  quot  2 quot  tmp      if    -z  match     then         echo  quot Usage  echo   quot some text  quot    resub                2 me    1 time  quot   gt  amp 2         return 1     fi          First  convert  quot  1 quot  to  quot  BASH REMATCH 1  quot  and  single-quote  for later eval-ing             Utility function to  single-quote  a list of strings     squot     local a     for i in  quot    quot   do a      echo     i         quot     quot           done  echo  quot   a     quot          tmp  quot  quot      while     subst             0-9                   subst            0-9            do         tmp  quot    BASH REMATCH   BASH REMATCH 2      squot  quot   BASH REMATCH 3   quot    tmp  quot          subst  quot   BASH REMATCH 1   quot      done     subst  quot   squot  quot   subst  quot    tmp  quot           Now start  globally  substituting      tmp  quot  quot      while read line  do         counter 0         while     line     match         do             eval tmp   quot   tmp   line   BASH REMATCH 0    quot   quot   subst  quot              line  quot   BASH REMATCH        BASH REMATCH     - 1      quot          done         echo  quot   tmp   line  quot      done    resub  quot    quot                           EXAMPLES                                 echo  quot The quick brown fox jumps quickly over the lazy dog quot    resub quick slow        The slow brown fox jumps slowly over the lazy dog         echo  quot The quick brown fox jumps quickly over the lazy dog quot    resub  quick         fox   slow  1 sheep         The slow brown sheep jumps quickly over the lazy dog         animal  quot sheep quot         echo  quot The quick brown fox  jumps  quickly over the   quot lazy  quot    dog quot    resub  quick         fox   quot   quot   low  quot     1    animal  quot         The  quot  low quot  brown  sheep   jumps  quickly over the  quot lazy quot   dog         echo  quot one two three four five quot    resub  quot one         three         five quot   one  2 three  1 five         one four three two five         echo  quot one two one four five quot    resub  quot one          quot   XXX  1          XXX two XXX four five         echo  quot one two three four five one six three seven eight quot    resub  quot one         three          quot   XXX  1 YYY  2          XXX two YYY four five XXX six YYY seven eight  H T to  Charles Duffy re       match

User · Answer

This actually can be done in pure bash   hello ho02123ware38384you443d34o3434ingtod38384day re       0-9        while     hello     re     do   hello   BASH REMATCH 1    BASH REMATCH 2   done echo   hello       yields     howareyoudoingtodday

User · Answer

This example in the input hello ugly world it searches for the regex bad ugly and replaces it with nice    bin bash    THIS FUNCTION NEEDS THREE PARAMETERS   arg1   input              Example   hello ugly world   arg2   search regex       Example   bad ugly   arg3   replace            Example   nice function regex replace          1   hello ugly world      2   bad ugly      3   nice      REGEX   re  quot        2      quot     if     1     re     then       if there is a match              BASH REMATCH 0     hello ugly world         BASH REMATCH 1     hello          BASH REMATCH 2     ugly         BASH REMATCH 3     world            hello   nice   world     echo   BASH REMATCH 1   3  BASH REMATCH 3     else           if no match return original input  hello ugly world     echo  quot  1 quot    fi          prints  hello nice world  regex replace  hello ugly world   bad ugly   nice     to save output to a variable x   regex replace  hello ugly world   bad ugly   nice   echo  quot output of replacement is   x quot  exit

User · Answer

Use sed   MYVAR ho02123ware38384you443d34o3434ingtod38384day echo   MYVAR    sed -e  s  a-zA-Z  X g  -e  s  0-9  N g    prints XXNNNNNXXXXNNNNNXXXNNNXNNXNNNNXXXXXXNNNNNXXX   Note that the subsequent -e s are processed in order   Also  the g flag for the expression will match all occurrences in the input   You can also pick your favorite tool using this method  i e  perl  awk  e g    echo   MYVAR    perl -pe  s  a-zA-Z  X g and s  0-9  N g    This may allow you to do more creative matches     For example  in the snip above  the numeric replacement would not be used unless there was a match on the first expression  due to lazy and evaluation    And of course  you have the full language support of Perl to do your bidding

User · Answer

If you are making repeated calls and are concerned with performance  This test reveals the BASH method is  15x faster than forking to sed and likely any other external process   hello 123456789X123456789X123456789X123456789X123456789X123456789X123456789X123456789X123456789X123456789X123456789X  P1   date   s   for i in  1  10000  do    echo  hello   sed s X  g  gt   dev null done  P2   date   s  echo    P2- P1   for i in  1  10000  do    echo   hello  X    gt   dev null done  P3   date   s  echo    P3- P2

[regex] Search and replace in bash using regular expressions

Examples related to regex

Examples related to bash