Find and replace in file and overwrite file doesn t work it empties the file

Question

I would like to run a find and replace on an HTML file through the command line   My command looks something like this   sed -e s STRING TO REPLACE STRING TO REPLACE IT g index html  gt  index html   When I run this and look at the file afterward  it is empty  It deleted the contents of my file   When I run this after restoring the file again   sed -e s STRING TO REPLACE STRING TO REPLACE IT g index html   The stdout is the contents of the file  and the find and replace has been executed   Why is this happening

User · Answer

Warning  this is a dangerous method  It abuses the i o buffers in linux and with specific options of buffering it manages to work on small files  It is an interesting curiosity  But don t use it for a real situation   Besides the -i option of sed you can use the tee utility   From man      tee - read from standard input and write to standard output and files   So  the solution would be   sed s STRING TO REPLACE STRING TO REPLACE IT g index html   tee   tee index html   -- here the tee is repeated to make sure that the pipeline is buffered  Then all commands in the pipeline are blocked until they get some input to work on  Each command in the pipeline starts when the upstream commands have written 1 buffer of bytes  the size is defined somewhere  to the input of the command  So the last command tee index html  which opens the file for writing and therefore empties it  runs after the upstream pipeline has finished and the output is in the buffer within the pipeline   Most likely the following won t work   sed s STRING TO REPLACE STRING TO REPLACE IT g index html   tee index html   -- it will run both commands of the pipeline at the same time without any blocking   Without blocking the pipeline should pass the bytes line by line instead of buffer by buffer  Same as when you run cat   sed s bar GGG   Without blocking it s more interactive and usually pipelines of just 2 commands run without buffering and blocking  Longer pipelines are buffered   The tee index html will open the file for writing and it will be emptied  However  if you turn the buffering always on  the second version will work too

User · Answer

use sed s -i option  e g   sed -i bak -e s STRING TO REPLACE REPLACE WITH g index html

User · Answer

I was searching for the option where I can define the line range and found the answer  For example I want to change host1 to host2 from line 36-57    sed  36 57 s host1 host2 g  myfile txt  gt  myfile1 txt   You can use gi option as well to ignore the character case    sed  30 40 s version story gi  myfile txt  gt  myfile1 txt

User · Answer

When the shell sees   gt  index html in the command line it opens the file index html for writing  wiping off all its previous contents   To fix this you need to pass the -i option to sed to make the changes inline and create a backup of the original file before it does the changes in-place   sed -i bak s STRING TO REPLACE STRING TO REPLACE IT g index html   Without the  bak the command will fail on some platforms  such as Mac OSX

User · Answer

You can use Vim in Ex mode   ex -sc   s STRING TO REPLACE STRING TO REPLACE IT g x  index html      select all lines x save and close

User · Answer

An alternative  useful  pattern is   sed -e  script script  index html  gt  index html tmp  amp  amp  mv index html tmp index html   That has much the same effect  without using the -i option  and additionally means that  if the sed script fails for some reason  the input file isn t clobbered   Further  if the edit is successful  there s no backup file left lying around   This sort of idiom can be useful in Makefiles   Quite a lot of seds have the -i option  but not all of them  the posix sed is one which doesn t   If you re aiming for portability  therefore  it s best avoided

User · Answer

sed -i bak  s https    com  pub url g  MyHTMLFile html   If you have a link to be added  try this  Search for the URL as above  starting with https and ending with com here  and replace it with a URL string  I have used a variable  pub url here  s here means search and g means global replacement    It works

User · Answer

The problem with the command  sed  code  file  gt  file   is that file is truncated by the shell before sed actually gets to process it  As a result  you get an empty file    The sed way to do this is to use -i to edit in place  as other answers suggested  However  this is not always what you want  -i will create a temporary file that will then be used to replace the original file  This is problematic if your original file was a link  the link will be replaced by a regular file   If you need to preserve links  you can use a temporary variable to store the output of sed before writing it back to the file  like this   tmp   sed  code  file   echo -n   tmp   gt  file   Better yet  use printf instead of echo since echo is likely to process    as   in some shells  e g  dash    tmp   sed  code  file   printf   s    tmp   gt  file

User · Answer

To change multiple files  and saving a backup of each as   bak    perl -p -i -e  s    x g        will take all files in directory and replace   with x this is called a    Perl pie     easy as a pie

User · Answer

With all due respect to the above correct answers  it s always a good idea to   dry run  scripts like that  so that you don t corrupt your file and have to start again from scratch   Just get your script to spill the output to the command line instead of writing it to the file  for example  like that   sed -e s STRING TO REPLACE STRING TO REPLACE IT g index html   OR  less index html   sed -e s STRING TO REPLACE STRING TO REPLACE IT g    This way you can see and check the output of the command without getting your file truncated

User · Answer

You should try using the option -i for in-place editing

User · Answer

sed -i  s STRING TO REPLACE STRING TO REPLACE IT g  index html   This does a global in-place substitution on the file index html  Quoting the string prevents problems with whitespace in the query and replacement

User · Answer

And the ed answer    printf   s n   1  s STRING TO REPLACE STRING TO REPLACE IT g  w q   ed index html   To reiterate what codaddict answered  the shell handles the redirection first  wiping out the  input html  file  and then the shell invokes the  sed  command passing it a now empty file

[shell] Find and replace in file and overwrite file doesn't work, it empties the file

Examples related to shell

Examples related to unix

Examples related to sed

Examples related to io-redirection