How to do a recursive find replace of a string with awk or sed

Question

How do I find and replace every occurrence of   subdomainA example com   with   subdomainB example com   in every text file under the  home www  directory tree recursively

User · Answer

This one is compatible with git repositories, and a bit simpler:

Linux:

git grep -l 'original_text' | xargs sed -i 's/original_text/new_text/g'

Mac:

git grep -l 'original_text' | xargs sed -i '' -e 's/original_text/new_text/g'

(Thanks to http://blog.jasonmeridth.com/posts/use-git-grep-to-replace-strings-in-files-in-your-git-repository/)

User · Answer

A straight forward method if you need to exclude directories  --exclude-dir   folder  and also might have file names with spaces  solved by using 0Byte for both grep -Z and xargs -0  grep -rlZ oldtext   --exclude-dir  folder   xargs -0 sed -i  s oldtext newtext g

User · Answer

grep -lr  subdomainA example com    while read file  do sed -i  s subdomainA example com subdomainB example com g    file   done  I guess most people don t know that they can pipe something into a  while read file  and it avoids those nasty -print0 args  while presevering spaces in filenames   Further adding an echo before the sed allows you to see what files will change before actually doing it

User · Answer

For anyone using silver searcher  ag   ag SearchString -l0   xargs -0 sed -i  s SearchString Replacement g    Since ag ignores git hg svn file folders by default  this is safe to run inside a repository

User · Answer

Using combination of grep and sed  for pp in   grep -Rl looking for string  do     sed -i  s looking for string something other g     pp   done

User · Answer

You can use awk to solve this as below   for file in  find  home www -type f  do    awk   gsub  subdomainA example com   subdomainB example com    print  0     file  gt    tempFile  amp  amp  mv   tempFile  file  done   hope this will help you

User · Answer

or use the blazing fast GNU Parallel   grep -rl oldtext     parallel sed -i  s oldtext newtext g

User · Answer

For Qshell  qsh  on IBMi  not bash as tagged by OP   Limitations of qsh commands    find does not have the -print0 option xargs does not have -0 option sed does not have -i option   Thus the solution in qsh       PATH  your path here      SEARCH   subdomainA example com       REPLACE   subdomainB example com        for file in    find   PATH  -P -type f    do              TEMP FILE   file    RANDOM  temp file              if     -e   TEMP FILE     then                     touch -C 819   TEMP FILE                       sed -e  s   SEARCH    REPLACE  g                         lt    file   gt    TEMP FILE                       mv   TEMP FILE    file              fi     done   Caveats    Solution excludes error handling Not Bash as tagged by OP

User · Answer

I just needed this and was not happy with the speed of the available examples  So I came up with my own   cd  var www  amp  amp  ack-grep -l --print0 subdomainA example com   xargs -0 perl -i bak -pe  s subdomainA  example  com subdomainB example com g    Ack-grep is very efficient on finding relevant files  This command replaced  145 000 files with a breeze whereas others took so long I couldn t wait until they finish

User · Answer

For replace all occurrences in a git repository you can use   git ls-files -z   xargs -0 sed -i  s subdomainA  example  com subdomainB example com g    See List files in local git repo  for other options to list all files in a repository  The -z options tells git to separate the file names with a zero byte  which assures that xargs  with the option -0  can separate filenames  even if they contain spaces or whatnot

User · Answer

To cut down on files to recursively sed through  you could grep for your string instance   grep -rl  lt oldstring gt   path to folder   xargs sed -i s  lt oldstring gt   lt newstring gt  g   If you run man grep you ll notice you can also define an --exlude-dir    git  flag if you want to omit searching through  git directories  avoiding git index issues as others have politely pointed out   Leading you to   grep -rl --exclude-dir    git   lt oldstring gt   path to folder   xargs sed -i s  lt oldstring gt   lt newstring gt  g

User · Answer

If you wanted to use this without completely destroying your SVN repository  you can tell  find  to ignore all hidden files by doing   find        -regex              -type f -print0   xargs -0 sed -i  s subdomainA example com subdomainB example com g

User · Answer

For me the easiest solution to remember is https   stackoverflow com a 2113224 565525  i e    sed -i    -e  s subdomainA subdomainB g    find  home www  -type f    NOTE  -i    solves OSX problem sed  1         invalid command code    NOTE  If there are too many files to process you ll get Argument list too long  The workaround - use find -exec or xargs solution described above

User · Answer

Try this   sed -i  s subdomainA subdomainB g   grep -ril  subdomainA

User · Answer

If you do not mind using vim together with grep or find tools  you could follow up the answer given by user Gert in this link --  How to do a text replacement in a big folder hierarchy    Here s the deal    recursively grep for the string that you want to replace in a certain path  and take only the complete path of the matching file   that would be the   grep  string   pathname  -Rl    optional  if you want to make a pre-backup of those files on centralized directory maybe you can use this also  cp -iv   grep  string   pathname  -Rl   centralized-directory-pathname  after that you can edit replace at will in vim following a scheme similar to the one provided on the link given     bufdo  s string replacement gc   update

User · Answer

A bit old school but this worked on OS X   There are few trickeries       Will only edit files with extension  sls under the current directory        must be escaped to ensure sed does not evaluate them as  any character         is used as the sed delimiter instead of the usual    Also note this is to edit a Jinja template to pass a variable in the path of an import  but this is off topic    First  verify your sed command does what you want  this will only print the changes to stdout  it will not change the files    for file in   find   -name   sls -type f   do echo -e   n file     sed  s foo  bar foo bar    baz     g   file  done   Edit the sed command as needed  once you are ready to make changes   for file in   find   -name   sls -type f   do echo -e   n file     sed -i     s foo  bar foo bar    baz     g   file  done   Note the -i    in the sed command  I did not want to create a backup of the original files  as explained in In-place edits with sed on OS X or in Robert Lujo s comment in this page    Happy seding folks

User · Answer

If you have access to node you can do a npm install -g rexreplace and then  rexreplace  subdomainA example com   subdomainB example com   home www

User · Answer

All the tricks are almost the same  but I like this one   find  lt mydir gt  -type f -exec sed -i  s  lt string1 gt   lt string2 gt  g          find  lt mydir gt   look up in the  directory  -type f       File is of type  regular file  -exec command           This variant of the -exec action runs the specified command on the selected files  but the command line is built by appending             each  selected  file  name  at  the  end  the total number of invocations of the command will be much less than the number of             matched files   The command line is built in much the same way that xargs builds its command lines    Only  one  instance  of                  is allowed within the command   The command is executed in the starting directory

User · Answer

The simplest way for me is  grep -rl oldtext     xargs sed -i  s oldtext newtext g

User · Answer

find  home www  -type f -exec perl -i bak -pe  s subdomainA  example  com subdomainB example com g         find  home www  -type f  will list all files in  home www   and its subdirectories   The  -exec  flag tells find to run the following command on each file found   perl -i bak -pe  s subdomainA  example  com subdomainB example com g         is the command run on the files  many at a time   The    gets replaced by file names   The   at the end of the command tells find to build one command for many filenames   Per the find man page    The command line is built in much the same way that xargs builds its command lines    Thus it s possible to achieve your goal  and handle filenames containing spaces  without using xargs -0  or -print0

User · Answer

This is the best all around solution I ve found for OSX and Windows  msys2    Should work with anything that can get the gnu version of sed   Skips the  git directories so it won t corrupt your checksums   On mac  just install coreutils first and ensure gsed is in the path -  brew install coreutils   Then I stick this function in my zshrc bashrc -   replace-recursive         hash gsed 2 gt  dev null  amp  amp  local SED CMD  gsed     SED CMD  sed      find   -type f -name       -not -path     git    -print0   xargs -0  SED CMD -i  s  1  2 g     usage  replace-recursive  lt find gt   lt replace gt

User · Answer

Here s a version that should be more general than most  it doesn t require find  using du instead   for instance  It does require xargs  which are only found in some versions of Plan 9  like 9front     du -a   awk -F        print  2      xargs sed -i -e  s subdomainA  example  com subdomainB example com g    If you want to add filters like file extensions use grep    du -a   grep    scala     awk -F        print  2      xargs sed -i -e  s subdomainA  example  com subdomainB example com g

User · Answer

According to this blog post   find   -type f   xargs perl -pi -e  s oldtext newtext g

User · Answer

usr local bin bash -x  find    home www -type f   while read files do  sedtest   sed -n         p     files     sed -n   subdomainA p        if      sedtest         then     sed s  subdomainA subdomainB  g    files    gt     files   tmp     mv    files   tmp    files       fi  done

User · Answer

Simplest way to replace  all files  directory  recursive   find   -type f -not -path         -exec sed -i  s foo bar g         Note  Sometimes you might need to ignore some hidden files i e   git  you can use above command   If you want to include hidden files use   find   -type f  -exec sed -i  s foo bar g         In both case the string foo will be replaced with new string bar

User · Answer

find  home www    -type d -name  git -prune    -o -type f -print0   xargs -0 sed -i  s subdomainA  example  com subdomainB example com g    -print0 tells find to print each of the results separated by a null character  rather than a new line  In the unlikely event that your directory has files with newlines in the names  this still lets xargs work on the correct filenames      -type d -name  git -prune    is an expression which completely skips over all directories named  git  You could easily expand it  if you use SVN or have other folders you want to preserve -- just match against more names  It s roughly equivalent to -not -path  git  but more efficient  because rather than checking every file in the directory  it skips it entirely  The -o after it is required because of how -prune actually works   For more information  see man find

User · Answer

A simpler way is to use the below on the command line  find  home www  -type f xargs perl -pi -e  s subdomainA  example  com subdomainB example com g

User · Answer

Note  Do not run this command on a folder including a git repo - changes to  git could corrupt your git index   find  home www  -type f -exec       sed -i  s subdomainA  example  com subdomainB example com g         Compared to other answers here  this is simpler than most and uses sed instead of perl  which is what the original question asked for

User · Answer

cd  home www  amp  amp  find   -type f -print0     xargs -0 perl -i bak -pe  s subdomainA  example  com subdomainB example com g

User · Answer

I just use tops   find   -name     c cc cp cpp m mm h   -print0    xargs -0 tops -verbose  replace  verify noerr  lt b args gt    with    Verify noErr  lt args gt      replace  check  lt b args gt    with    Check  lt args gt

User · Answer

To replace all content matching string 1 with string 2 of all  c and  h files in the current directory and subdirectories  excluding  git     This works on Mac   find   -type f -path    git   -prune -o -name      ch   -exec   sed -i    -e  s   1    2  g         This should work on Linux  Have not tested yet    find   -type f -path    git   -prune -o -name      ch   -exec   sed -i  s string 1 string 2 g

User · Answer

just to avoid to change also   NearlysubdomainA example com subdomainA example comp other   but still    subdomainA example com IsIt good    maybe not good in the idea behind domain root   find  home www  -type f -exec sed -i  s  bsubdomainA  example  com b  1subdomainB example com 2 g

User · Answer

to change multiple files  and saving a backup as   bak    perl -p -i -e  s    x g      will take all files in directory and replace   with x called a    Perl pie     easy as a pie

User · Answer

perl -p -i -e  s oldthing new thingy g   grep -ril oldthing

User · Answer

An one nice oneliner as an extra  Using git grep    git grep -lz  subdomainA example com    xargs -0 perl -i   -pE  s subdomainA example com subdomainB example com g

[bash] How to do a recursive find/replace of a string with awk or sed?

Examples related to bash

Examples related to sed

Examples related to awk

Examples related to replace