Count all occurrences of a string in lots of files with grep

Question

I have a bunch of log files  I need to find out how many times a string occurs in all files   grep -c string     returns      file1 1 file2 0 file3 0       Using a pipe I was able to get only files that have one or more occurrences   grep -c string     grep -v  0      file4 5 file5 1 file6 2       How can I get only the combined count   If it returns file4 5  file5 1  file6 2  I want to get back 8

User · Answer

Here is a faster-than-grep AWK alternative way of doing this, which handles multiple matches of <url> per line, within a collection of XML files in a directory:

awk '/<url>/{m=gsub("<url>","");total+=m}END{print total}' some_directory/*.xml

This works well in cases where some XML files don't have line breaks.

User · Answer

short recursive variant   find   -type f -exec cat        grep -c  string

User · Answer

cat     grep -c string

User · Answer

Obligatory AWK solution   grep -c string     awk  BEGIN FS      x   2 END print x     Take care if your file names include     though

User · Answer

You can use a simple grep to capture the number of occurrences effectively  I will use the -i option to make sure STRING StrING string get captured properly   Command line that gives the files  name   grep -oci string     grep -v  0   Command line that removes the file names and prints 0 if there is a file without occurrences   grep -ochi string

User · Answer

cat     grep -c string   One of the rare useful applications of cat

User · Answer

Something different than all the previous answers   perl -lne   count   for m  lt pattern gt  g END print  count

User · Answer

cat     grep -c string

User · Answer

Another oneliner using basic command line functions handling multiple occurences per line     cat    sed s string    nstring   g  grep string  wc -l

User · Answer

Grep only solution which I tested with grep for windows   grep -ro  pattern to find in files   Directory to recursively search    grep -c  pattern to find in files    This solution will count all occurrences even if there are multiple on one line  -r recursively searches the directory  -o will  show only the part of a line matching PATTERN  -- this is what splits up multiple occurences on a single line and makes grep print each match on a new line  then pipe those newline-separated-results back into grep with -c to count the number of occurrences using the same pattern

User · Answer

short recursive variant   find   -type f -exec cat        grep -c  string

User · Answer

If you want number of occurrences per file  example for string  tcp     grep -RIci  tcp      awk -v FS     -v OFS   t    2 gt 0   print  2   1      sort -hr   Example output   53    HTTPClient src HTTPClient cpp 21    WiFi src WiFiSTA cpp 19    WiFi src ETH cpp 13    WiFi src WiFiAP cpp 4     WiFi src WiFiClient cpp 4     HTTPClient src HTTPClient h 3     WiFi src WiFiGeneric cpp 2     WiFi examples WiFiClientBasic WiFiClientBasic ino 2     WiFiClientSecure src ssl client cpp 1     WiFi src WiFiServer cpp   Explanation     grep -RIci NEEDLE   - looks for string NEEDLE recursively from current directory  following symlinks   ignoring binaries  counting number of occurrences  ignoring case awk     - this command ignores files with zero occurrences and formats lines sort -hr - sorts lines in reverse order by numbers in first column   Of course  it works with other grep commands with option -c  count  as well  For example   grep -c  tcp    txt   awk -v FS     -v OFS   t    2 gt 0   print  2   1      sort -hr

User · Answer

cat     grep -c string   One of the rare useful applications of cat

User · Answer

Instead of using -c  just pipe it to wc -l   grep string     wc -l   This will list each occurrence on a single line and then count the number of lines   This will miss instances where the string occurs 2  times on one line  though

User · Answer

Something different than all the previous answers   perl -lne   count   for m  lt pattern gt  g END print  count

User · Answer

grep -oh string     wc -w   will count multiple occurrences in a line

User · Answer

cat     grep -c string   One of the rare useful applications of cat

User · Answer

This works for multiple occurrences per line   grep -o string     wc -l

User · Answer

Another oneliner using basic command line functions handling multiple occurences per line     cat    sed s string    nstring   g  grep string  wc -l

User · Answer

You can add -R to search recursively  and avoid to use cat  and -I to ignore binary files   grep -RIc string

User · Answer

cat     grep -c string

User · Answer

Obligatory AWK solution   grep -c string     awk  BEGIN FS      x   2 END print x     Take care if your file names include     though

User · Answer

You can use a simple grep to capture the number of occurrences effectively  I will use the -i option to make sure STRING StrING string get captured properly   Command line that gives the files  name   grep -oci string     grep -v  0   Command line that removes the file names and prints 0 if there is a file without occurrences   grep -ochi string

User · Answer

cat     grep -c string   One of the rare useful applications of cat

User · Answer

Instead of using -c  just pipe it to wc -l   grep string     wc -l   This will list each occurrence on a single line and then count the number of lines   This will miss instances where the string occurs 2  times on one line  though

User · Answer

If you want number of occurrences per file  example for string  tcp     grep -RIci  tcp      awk -v FS     -v OFS   t    2 gt 0   print  2   1      sort -hr   Example output   53    HTTPClient src HTTPClient cpp 21    WiFi src WiFiSTA cpp 19    WiFi src ETH cpp 13    WiFi src WiFiAP cpp 4     WiFi src WiFiClient cpp 4     HTTPClient src HTTPClient h 3     WiFi src WiFiGeneric cpp 2     WiFi examples WiFiClientBasic WiFiClientBasic ino 2     WiFiClientSecure src ssl client cpp 1     WiFi src WiFiServer cpp   Explanation     grep -RIci NEEDLE   - looks for string NEEDLE recursively from current directory  following symlinks   ignoring binaries  counting number of occurrences  ignoring case awk     - this command ignores files with zero occurrences and formats lines sort -hr - sorts lines in reverse order by numbers in first column   Of course  it works with other grep commands with option -c  count  as well  For example   grep -c  tcp    txt   awk -v FS     -v OFS   t    2 gt 0   print  2   1      sort -hr

User · Answer

You can add -R to search recursively  and avoid to use cat  and -I to ignore binary files   grep -RIc string

User · Answer

Instead of using -c  just pipe it to wc -l   grep string     wc -l   This will list each occurrence on a single line and then count the number of lines   This will miss instances where the string occurs 2  times on one line  though

User · Answer

Here is a faster-than-grep AWK alternative way of doing this, which handles multiple matches of <url> per line, within a collection of XML files in a directory:

awk '/<url>/{m=gsub("<url>","");total+=m}END{print total}' some_directory/*.xml

This works well in cases where some XML files don't have line breaks.

User · Answer

The AWK solution which also handles file names including colons   grep -c string     sed -r  s           awk  BEGIN   x   1 END print x     Keep in mind that this method still does not find multiple occurrences of string on the same line

User · Answer

Grep only solution which I tested with grep for windows   grep -ro  pattern to find in files   Directory to recursively search    grep -c  pattern to find in files    This solution will count all occurrences even if there are multiple on one line  -r recursively searches the directory  -o will  show only the part of a line matching PATTERN  -- this is what splits up multiple occurences on a single line and makes grep print each match on a new line  then pipe those newline-separated-results back into grep with -c to count the number of occurrences using the same pattern

User · Answer

The AWK solution which also handles file names including colons   grep -c string     sed -r  s           awk  BEGIN   x   1 END print x     Keep in mind that this method still does not find multiple occurrences of string on the same line

User · Answer

Instead of using -c  just pipe it to wc -l   grep string     wc -l   This will list each occurrence on a single line and then count the number of lines   This will miss instances where the string occurs 2  times on one line  though

User · Answer

This works for multiple occurrences per line   grep -o string     wc -l

User · Answer

cat     grep -c string

User · Answer

grep -oh string     wc -w   will count multiple occurrences in a line

[grep] Count all occurrences of a string in lots of files with grep

Examples related to grep