How can I find all of the distinct file extensions in a folder hierarchy

Question

On a Linux machine I would like to traverse a folder hierarchy and get a list of all of the distinct file extensions within it   What would be the best way to achieve this from a shell

User · Answer

you could also do this  find   -type f -name    php  -exec PATHTOAPP

User · Answer

Since there s already another solution which uses Perl   If you have Python installed you could also do  from the shell    python -c  import os e set     e add os path splitext f  -1   for f in fn for     fn in os walk   home    print   n  join e

User · Answer

Find everythin with a dot and show only the suffix   find   -type f -name         awk -F    print  NF     sort -u   if you know all suffix have 3 characters then  find   -type f -name           awk -F    print  NF     sort -u   or with sed shows all suffixes with one to four characters  Change  1 4  to the range of characters you are expecting in the suffix   find   -type f   sed -n  s          1 4       1 p   sort -u

User · Answer

In Python using generators for very large directories  including blank extensions  and getting the number of times each extension shows up   import json import collections import itertools import os  root     home andres  files   itertools chain from iterable       files for     files in os walk root         counter   collections Counter       os path splitext file   1  for file  in files    print json dumps counter  indent 2

User · Answer

I think the most simple  amp  straightforward way is  for f in      do echo    f        done   sort -u   It s modified on ChristopheD s 3rd way

User · Answer

None of the replies so far deal with filenames with newlines properly  except for ChristopheD s  which just came in as I was typing this   The following is not a shell one-liner  but works  and is reasonably fast   import os  sys  def names roots       for root in roots          for a  b  basenames in os walk root               for basename in basenames                  yield basename  sufs   set os path splitext x  1  for x in names sys argv 1     for suf in sufs      if suf          print suf

User · Answer

I don t think this one was mentioned yet   find   -type f -exec sh -c  echo    0                sort   uniq -c

User · Answer

I tried a bunch of the answers here  even the  quot best quot  answer  They all came up short of what I specifically was after  So besides the past 12 hours of sitting in regex code for multiple programs and reading and testing these answers this is what I came up with which works EXACTLY like I want   find   -type f -name  quot     quot    grep -o -E  quot           quot    grep -o -E  quot    alpha    2 16  quot    awk   print tolower  0      sort -u   Finds all files which may have an extension  Greps only the extension Greps for file extensions between 2 and 16 characters  just adjust the numbers if they don t fit your need   This helps avoid cache files and system files  system file bit is to search jail   Awk to print the extensions in lower case  Sort and bring in only unique values  Originally I had attempted to try the awk answer but it would double print items that varied in case sensitivity   If you need a count of the file extensions then use the below code find   -type f -name  quot     quot    grep -o -E  quot           quot    grep -o -E  quot    alpha    2 16  quot    awk   print tolower  0      sort   uniq -c   sort -rn  While these methods will take some time to complete and probably aren t the best ways to go about the problem  they work  Update  Per  alpha 989 long file extensions will cause an issue  That s due to the original regex  quot    alpha    3 6  quot   I have updated the answer to include the regex  quot    alpha    2 16  quot   However anyone using this code should be aware that those numbers are the min and max of how long the extension is allowed for the final output  Anything outside that range will be split into multiple lines in the output  Note  Original post did read  quot - Greps for file extensions between 3 and 6 characters  just adjust the numbers if they don t fit your need   This helps avoid cache files and system files  system file bit is to search jail   quot  Idea  Could be used to find file extensions over a specific length via   find   -type f -name  quot     quot    grep -o -E  quot           quot    grep -o -E  quot    alpha    4   quot    awk   print tolower  0      sort -u  Where 4 is the file extensions length to include and then find also any extensions beyond that length

User · Answer

My awk-less  sed-less  Perl-less  Python-less POSIX-compliant alternative   find   -type f   rev   cut -d  -f1   rev    tr    upper       lower      sort   uniq --count   sort -rn   The trick is that it reverses the line and cuts the extension at the beginning  It also converts the extensions to lower case   Example output      3689 jpg    1036 png     610 mp4      90 webm      90 mkv      57 mov      12 avi      10 txt       3 zip       2 ogv       1 xcf       1 trashinfo       1 sh       1 m4v       1 jpeg       1 ini       1 gqv       1 gcs       1 dv

User · Answer

The accepted answer uses REGEX and you cannot create an alias command with REGEX  you have to put it into a shell script  I m using Amazon Linux 2 and did the following    I put the accepted answer code into a file using      sudo vim find sh   add this code   find    -type f   perl -ne  print  1 if m                  sort -u   save the file by typing    wq    sudo vim    bash profile alias getext     path to your find sh   wq       bash profile

User · Answer

No need for the pipe to sort  awk can do it all    find   -type f   awk -F    a  NF    print  NF

User · Answer

I ve found it simple and fast          find   -type f -exec basename         awk -F      print  NF    gt   tmp outfile txt      cat  tmp outfile txt   sort   uniq -c  sort -n  gt  tmp outfile sorted txt

User · Answer

Adding my own variation to the mix  I think it s the simplest of the lot and can be useful when efficiency is not a big concern   find   -type f   grep -o -E               sort -u

User · Answer

Try this  not sure if it s the best way  but it works    find   -type f   perl -ne  print  1 if m                  sort -u   It work as following    Find all files from current folder Prints extension of files if any Make a unique sorted list

User · Answer

Recursive version   find   -type f   sed -e  s           sed -e  s           sort -u   If you want totals  how may times the extension was seen    find   -type f   sed -e  s           sed -e  s           sort   uniq -c   sort -rn   Non-recursive  single folder    for f in      do printf   s n     f        done   sort -u   I ve based this upon this forum post  credit should go there

User · Answer

Powershell   dir -recurse   select-object extension -unique   Thanks to http   kevin-berridge blogspot com 2007 11 windows-powershell html

[linux] How can I find all of the distinct file extensions in a folder hierarchy?

Examples related to linux

Examples related to grep

Examples related to filesystems

Examples related to file-extension