How to grep a text file which contains some binary data

Question

grep returns   Binary file test log matches   For example  echo     line1 re  x00 r nline2 r nline3 re r n   gt  test log    in zsh echo -e  line1 re  x00 r nline2 r nline3 re r n   gt  test log    in bash grep re test log   I wish the result will show line1 and line3  total two lines        Is it possible to use tr convert the unprintable data into readable data  to let grep work again

User · Answer

You can use  strings  to extract strings from a binary file  for example  strings binary file   grep foo

User · Answer

grep -a   It can t get simpler than that

User · Answer

Here s what I used in a system that didn t have  strings  command installed  cat yourfilename   tr -cd    print      This prints the text and removes unprintable characters in one fell swoop  unlike  cat -v filename  which requires some postprocessing to remove unwanted stuff  Note that some of the binary data may be printable so you ll still get some gibberish between the good stuff  I think strings removes this gibberish too if you can use that

User · Answer

grep -a will force grep to search and output from a file that grep thinks is binary       grep -a re test log

User · Answer

You could run the data file through cat -v  e g   cat -v tmp test log   grep re line1 re    M line3 re M  which could be then further post-processed to remove the junk  this is most analogous to your query about using tr for the task  -v simply tells cat to display non-printing characters

User · Answer

You can force grep to look at binary files with   grep --binary-files text   You might also want to add -o  --only-matching  so you don t get tons of binary gibberish that will bork your terminal

User · Answer

One way is to simply treat binary files as text anyway  with grep --text but this may well result in binary information being sent to your terminal  That s not really a good idea if you re running a terminal that interprets the output stream  such as VT DEC or many others    Alternatively  you can send your file through tr with the following command   tr    000- 011 013- 037 177- 377        lt test log   grep whatever   This will change anything less than a space character  except newline  and anything greater than 126  into a   character  leaving only the printables     If you want every  illegal  character replaced by a different one  you can use something like the following C program  a classic standard input filter    include lt stdio h gt  int main  void        int ch      while   ch   getchar       EOF            if   ch      n        ch  gt         amp  amp   ch  lt                        putchar  ch             else               printf      02x     ch                       return 0      This will give you   NN    where NN is the hex code for the character  You can simply adjust the printf for whatever style of output you want   You can see that program in action here  where it   pax  printf  Hello  tBob nGoodbye  Bob n      filterProg Hello   09  Bob Goodbye  Bob

User · Answer

Starting with Grep 2 21  binary files are treated differently      When searching binary data  grep now may treat non-text bytes as line   terminators  This can boost performance significantly    So what happens now is that with binary data  all non-text bytes  including newlines  are treated as line terminators  If you want to change this behavior  you can    use --text  This will ensure that only newlines are line terminators use --null-data  This will ensure that only null bytes are line terminators

User · Answer

you can do   strings test log   grep -i   this will convert give output as a readable string to grep

User · Answer

You can also try Word Extractor tool  Word Extractor can be used with any file in your computer to separate the strings that contain human text   words from binary code  exe applications  DLLs

User · Answer

As James Selvakumar already said  grep -a does the trick  -a or --text forces Grep to handle the inputstream as text  See Manpage http   unixhelp ed ac uk CGI man-cgi grep  try  cat test log   grep -a somestring

[shell] How to grep a text file which contains some binary data?

Examples related to shell