Find and Replace Inside a Text File from a Bash Command

Question

What s the simplest way to do a find and replace for a given input string  say abc  and replace with another string  say XYZ in file  tmp file txt   I am writting an app and using IronPython to execute commands through SSH  mdash  but I don t know Unix that well and don t know what to look for   I have heard that Bash  apart from being a command line interface  can be a very powerful scripting language  So  if this is true  I assume you can perform actions like these   Can I do it with bash  and what s the simplest  one line  script to achieve my goal

User · Answer

You can use python within the bash script too  I didn t have much success with some of the top answers here  and found this to work without the need for loops      bin bash python filetosearch     home ubuntu ip table txt  texttoreplace    tcp443  texttoinsert    udp1194   s   open filetosearch  read   s   s replace texttoreplace  texttoinsert  f   open filetosearch   w   f write s  f close   quit

User · Answer

Try the following shell command   find     -type f -name  file  txt    xargs sed -i -e  s abc xyz g

User · Answer

I found this thread among others and I agree it contains the most complete answers so I m adding mine too    sed and ed are so useful   by hand  Look at this code from  Johnny   sed -i -e  s abc XYZ g   tmp file txt  When my restriction is to use it in a shell script  no variable can be used inside in place of  abc  or  XYZ   The BashFAQ seems to agree with what I understand at least  So  I can t use   x  abc  y  XYZ  sed -i -e  s  x  y g   tmp file txt  or  sed -i -e  s  x  y g   tmp file txt   but  what can we do  As   Johnny said use a while read    but  unfortunately that s not the end of the story  The following worked well with me    edit user s virtual domain result   if nullglob is set then  unset it temporarily is nullglob    shopt -s   egrep -i   nullglob    if    is nullglob     then    shopt -u nullglob fi while IFS  read -r line  do    line    line    lt servername gt    server      line    line    lt serveralias gt    alias      line    line    lt user gt    user      line    line    lt group gt    group      result   result   line   n  done  lt   tmp echo -e  result  gt   tmp  if nullglob was set then  re-enable it if    is nullglob     then    shopt -s nullglob fi  move user s virtual domain to Apache 2 domain directory         As one can see if nullglob is set then  it behaves strangely when there is a string containing a   as in    lt VirtualHost   80 gt   ServerName www example com   which becomes   lt VirtualHost ServerName www example com   there is no ending angle bracket and Apache2 can t even load  This kind of parsing should be slower than one-hit search and replace but  as you already saw  there are four variables for four different search patterns working out of one  parse cycle    The most suitable solution I can think of with the given assumptions of the problem

User · Answer

I was surprised when I stumbled over this     There is a replace command which ships with the  mysql-server  package  so if you have installed it try it out     replace string abc to XYZ in files replace  abc   XYZ  -- file txt file2 txt file3 txt    or pipe an echo to replace echo  abcdef   replace  abc   XYZ    See man replace for more on this

User · Answer

To edit text in the file non-interactively  you need in-place text editor such as vim   Here is simple example how to use it from the command line   vim -esnc   s foo bar g  wq  file txt   This is equivalent to  slim answer of ex editor which is basically the same thing   Here are few ex practical examples   Replacing text foo with bar in the file   ex -s   s foo bar ge -cwq file txt   Removing trailing whitespaces for multiple files   ex   bufdo  s  s     e  -cxa   txt   Troubleshooting  when terminal is stuck     Add -V1 param to show verbose messages  Force quit by  -cwq     See also    How to edit files non-interactively  e g  in pipeline   at Vi SE

User · Answer

Bash  like other shells  is just a tool for coordinating other commands  Typically you would try to use standard UNIX commands  but you can of course use Bash to invoke anything  including your own compiled programs  other shell scripts  Python and Perl scripts etc  In this case  there are a couple of ways to do it  If you want to read a file  and write it to another file  doing search replace as you go  use sed      sed  s abc XYZ g   lt infile  gt outfile  If you want to edit the file in place  as if opening the file in an editor  editing it  then saving it  supply instructions to the line editor  ex      echo  quot  s abc XYZ g     w     q      quot    ex file  Example is like vi without the fullscreen mode  You can give it the same commands you would at vi s   prompt

User · Answer

Be careful if you replace URLs with     character   An example of how to do it   sed -i  s http   domain com http   www domain com folder  g   test txt    Extracted from  http   www sysadmit com 2015 07 linux-reemplazar-texto-en-archivos-con-sed html

User · Answer

You may also use the ed command to do in-file search and replace     delete all lines matching foobar  ed -s test txt  lt  lt  lt    g foobar d nw     See more in  Editing files via scripts with ed

User · Answer

This is an old post but for anyone wanting to use variables as  centurian said the single quotes mean nothing will be expanded    A simple way to get variables in is to do string concatenation since this is done by juxtaposition in bash the following should work   sed -i -e  s  var1  var2 g   tmp file txt

User · Answer

The easiest way is to use sed  or perl   sed -i -e  s abc XYZ g   tmp file txt  Which will invoke sed to do an in-place edit due to the -i option   This can be called from bash  If you really really want to use just bash  then the following can work  while read a  do     echo   a  abc XYZ  done  lt   tmp file txt  gt   tmp file txt t mv  tmp file txt  t    This loops over each line  doing a substitution  and writing to a temporary file  don t want to clobber the input    The move at the end just moves temporary to the original name  For Mac users  sed -i     s abc XYZ g   tmp file txt  See the comment below why

User · Answer

If the file you are working on is not so big  and temporarily storing it in a variable is no problem  then you can use Bash string substitution on the whole file at once - there s no need to go over it line by line   file contents    lt  tmp file txt  echo    file contents  abc XYZ    gt   tmp file txt   The whole file contents will be treated as one long string  including linebreaks    XYZ can be a variable eg  replacement  and one advantage of not using sed here is that you need not be concerned that the search or replace string might contain the sed pattern delimiter character  usually  but not necessarily      A disadvantage is not being able to use regular expressions or any of sed s more sophisticated operations

User · Answer

You can use sed  sed -i  s abc XYZ gi   tmp file txt  You can use find and sed if you don t know your filename  find    -type f -exec sed -i  s abc XYZ gi         Find and replace in all Python files  find    -iname  quot   py quot  -type f -exec sed -i  s abc XYZ gi

User · Answer

You can use rpl command   For example you want to change domain name in whole php project   rpl -ivRpd -x  php   old domain name   new domain name    path to your project folder      This is  not clear bash of cause  but it s a very quick and usefull

User · Answer

File manipulation isn t normally done by Bash  but by programs invoked by Bash  e g    perl -pi -e  s abc XYZ g   tmp file txt   The -i flag tells it to do an in-place replacement   See man perlrun for more details  including how to take a backup of the original file

[bash] Find and Replace Inside a Text File from a Bash Command

Examples related to bash

Examples related to ssh

Examples related to replace

Examples related to scripting

Examples related to ironpython