Delete all but the most recent X files in bash

Question

Is there a simple way  in a pretty standard UNIX environment with bash  to run a command to delete all but the most recent X files from a directory   To give a bit more of a concrete example  imagine some cron job writing out a file  say  a log file or a tar-ed up backup  to a directory every hour  I d like a way to have another cron job running which would remove the oldest files in that directory until there are less than  say  5   And just to be clear  there s only one file present  it should never be deleted

User · Answer

If the filenames don t have spaces  this will work   ls -C1 -t  awk  NR gt 5  xargs rm   If the filenames do have spaces  something like  ls -C1 -t   awk  NR gt 5    sed -e  s   rm     -e  s         sh   Basic logic    get a listing of the files in time order  one column get all but the first 5  n 5 for this example  first version  send those to rm second version  gen a script that will remove them properly

User · Answer

The problems with the existing answers   inability to handle filenames with embedded spaces or newlines   in the case of solutions that invoke rm directly on an unquoted command substitution  rm         there s an added risk of unintended globbing    inability to distinguish between files and directories  i e   if directories happened to be among the 5 most recently modified filesystem items  you d effectively retain fewer than 5 files  and applying rm to directories will fail    wnoise s answer addresses these issues  but the solution is GNU-specific  and quite complex   Here s a pragmatic  POSIX-compliant solution that comes with only one caveat  it cannot handle filenames with embedded newlines - but I don t consider that a real-world concern for most people  For the record  here s the explanation for why it s generally not a good idea to parse ls output  http   mywiki wooledge org ParsingLs ls -tp   grep -v        tail -n  6   xargs -I    rm --     Note  This command operates in the current directory  to target a directory explicitly  use a subshell           cd  path to  amp  amp  ls -tp   grep -v        tail -n  6   xargs -I    rm --     The same applies analogously to the commands below  The above is inefficient  because xargs has to invoke rm once for each filename  Your platform s xargs may allow you to solve this problem  If you have GNU xargs  use -d   n   which makes xargs consider each input line a separate argument  yet passes as many arguments as will fit on a command line at once  ls -tp   grep -v        tail -n  6   xargs -d   n  -r rm --  -r  --no-run-if-empty  ensures that rm is not invoked if there s no input  If you have BSD xargs  including on macOS   you can use -0 to handle NUL-separated input  after first translating newlines to NUL  0x0  chars   which also passes  typically  all filenames at once  will also work with GNU xargs   ls -tp   grep -v        tail -n  6   tr   n    0    xargs -0 rm --  Explanation   ls -tp prints the names of filesystem items sorted by how recently they were modified   in descending order  most recently modified items first   -t   with directories printed with a trailing   to mark them as such  -p    Note  It is the fact that ls -tp always outputs file   directory names only  not full paths  that necessitates the subshell approach mentioned above for targeting a directory other than the current one   cd  path to  amp  amp  ls -tp          grep -v      then weeds out directories from the resulting listing  by omitting  -v  lines that have a trailing          Caveat  Since a symlink that points to a directory is technically not itself a directory  such symlinks will not be excluded    tail -n  6 skips the first 5 entries in the listing  in effect returning all but the 5 most recently modified files  if any  Note that in order to exclude N files  N 1 must be passed to tail -n     xargs -I    rm --     and its variations  then invokes on rm on all these files  if there are no matches at all  xargs won t do anything   xargs -I    rm --    defines placeholder    that represents each input line as a whole  so rm is then invoked once for each input line  but with filenames with embedded spaces handled correctly  -- in all cases ensures that any filenames that happen to start with - aren t mistaken for options by rm      A variation on the original problem  in case the matching files need to be processed individually or collected in a shell array    One by one  in a shell loop  POSIX-compliant   ls -tp   grep -v        tail -n  6   while IFS  read -r f  do echo  quot  f quot   done    One by one  but using a Bash process substitution   lt           so that the variables inside the  while  loop remain in scope  while IFS  read -r f  do echo  quot  f quot   done  lt   lt  ls -tp   grep -v        tail -n  6     Collecting the matches in a Bash  array   IFS    n  read -d    -ra files   lt   lt  ls -tp   grep -v        tail -n  6  printf   s n   quot   files     quot    print array elements

User · Answer

leaveCount 5 fileCount   ls -1   log   wc -l  tailCount    fileCount - leaveCount      avoid negative tail argument     tailCount  lt  0     amp  amp  tailCount 0  ls -t   log   tail - tailCount   xargs rm -f

User · Answer

All these answers fail when there are directories in the current directory  Here s something that works   find   -maxdepth 1 -type f   xargs -x ls -t   awk  NR gt 5    xargs -L1 rm   This    works when there are directories in the current directory tries to remove each file even if the previous one couldn t be removed  due to permissions  etc   fails safe when the number of files in the current directory is excessive and xargs would normally screw you over  the -x  doesn t cater for spaces in filenames  perhaps you re using the wrong OS

User · Answer

With zsh  Assuming you don t care about present directories and you will not have more than 999 files  choose a bigger number if you want  or create a while loop      6 -le  ls      wc -l     amp  amp  rm    om 6 999     In    om 6 999    the   means files  the o means sort order up  the m means by date of modification  put a for access time or c for inode change   the  6 999  chooses a range of file  so doesn t rm the 5 first

User · Answer

I realize this is an old thread  but maybe someone will benefit from this  This command will find files in the current directory    for F in   find   -maxdepth 1 -type f -name    srv logs   tar gz  -printf   T   p n    sort -r -z -n   tail -n 5   awk    print  2       do rm  F  done   This is a little more robust than some of the previous answers as it allows to limit your search domain to files matching expressions  First  find files matching whatever conditions you want  Print those files with the timestamps next to them   find   -maxdepth 1 -type f -name    srv logs   tar gz  -printf   T   p n    Next  sort them by the timestamps   sort -r -z -n   Then  knock off the 4 most recent files from the list   tail -n 5   Grab the 2nd column  the filename  not the timestamp    awk    print  2       And then wrap that whole thing up into a for statement   for F in      do rm  F  done   This may be a more verbose command  but I had much better luck being able to target conditional files and execute more complex commands against them

User · Answer

Ignoring newlines is ignoring security and good coding  wnoise had the only good answer  Here is a variation on his that puts the filenames in an array  x  while IFS  read -rd     do      x      REPLY         done  lt   lt  find   -maxdepth 1 -printf   T   p 0    sort -r -z -n

User · Answer

ls -tQ   tail -n 4   xargs rm   List filenames by modification time  quoting each filename  Exclude first 3  3 most recent   Remove remaining   EDIT after helpful comment from mklement0  thanks    corrected -n 3 argument  and note this will not work as expected if filenames contain newlines and or the directory contains subdirectories

User · Answer

I made this into a bash shell script  Usage  keep NUM DIR where NUM is the number of files to keep and DIR is the directory to scrub      bin bash   Keep last N files by date    Usage  keep NUMBER DIRECTORY echo    if      -lt 2    then     echo  Usage   0 NUMFILES DIR      echo  Keep last N newest files       exit 1 fi if     -e  2    then     echo  ERROR  directory   1  does not exist      exit 1 fi if     -d  2    then     echo  ERROR    1  is not a directory      exit 1 fi pushd  2  gt   dev null ls -tp   grep -v       tail -n    1    xargs -I    rm --    popd  gt   dev null echo  Done  Kept  1 most recent files in  2   ls  2 wc -l

User · Answer

I needed an elegant solution for the busybox  router   all xargs or array solutions were useless to me - no such command available there  find and mtime is not the proper answer as we are talking about 10 items and not necessarily 10 days  Espo s answer was the shortest and cleanest and likely the most unversal one   Error with spaces and when no files are to be deleted are both simply solved the standard way   rm    ls -td   tar   awk  NR gt 7    2 gt  amp -   Bit more educational version  We can do it all if we use awk differently  Normally  I use this method to pass  return  variables from the awk to the sh  As we read all the time that can not be done  I beg to differ  here is the method   Example for  tar files with no problem regarding the spaces in the filename  To test  replace  rm  with the  ls    eval   ls -td   tar   awk  NR gt 7   print  rm      0           Explanation   ls -td   tar lists all  tar files sorted by the time  To apply to all the files in the current folder  remove the  d   tar  part  awk  NR gt 7    skips the first 7 lines   print  rm      0      constructs a line  rm  file name   eval executes it  Since we are using rm  I would not use the above command in a script  Wiser usage is    cd  FolderToDeleteWithin  amp  amp  eval   ls -td   tar   awk  NR gt 7   print  rm      0            In the case of using ls -t command will not do any harm on such silly examples as  touch  foo   bar  and touch  hello   world   Not that we ever create files with such names in real life   Sidenote  If we wanted to pass a variable to the sh this way  we would simply modify the print  simple form  no spaces tolerated    print  VarName   1   to set the variable VarName to the value of  1  Multiple variables can be created in one go  This VarName becomes a normal sh variable and can be normally used in a script or shell afterwards  So  to create variables with awk and give them back to the shell   eval   ls -td   tar   awk  NR gt 7   print  VarName     1           echo   VarName

User · Answer

ls -t head -n 5 ls  sort uniq -u xargs rm   This version supports names with spaces    ls -t head -n 5 ls  sort uniq -u sed -e  s      amp   g  xargs rm

User · Answer

Removes all but the 10 latest  most recents  files  ls -t1   head -n   echo   ls -1   wc -l  - 10   bc    xargs rm   If less than 10 files no file is removed and you will have   error head  illegal line count -- 0  To count files with bash

User · Answer

find   -maxdepth 1 -type f -printf   T   p 0    sort -r -z -n   awk  BEGIN   RS   0   ORS   0   FS      NR  gt  5   sub    0-9     0-9             print      xargs -0 rm -f   Requires GNU find for -printf  and GNU sort for -z  and GNU awk for   0   and GNU xargs for -0  but handles files with embedded newlines or spaces

User · Answer

found interesting cmd in Sed-Onliners - Delete last 3 lines - fnd it perfect for another way to skin the cat  okay not  but idea       bin bash    sed cmd chng  2 to value file wish to retain   cd  opt depot    ls -1 MyMintFiles  zip  gt  BigList  sed -n -e  a -e  1 2  P N D   N ba  BigList  gt  DeList   for i in  cat DeList    do   echo  Deleted  i    rm -f  i     echo  File s  gonzo      read junk   done   exit 0

User · Answer

Remove all but 5  or whatever number  of the most recent files in a directory   rm  ls -t   awk  NR gt 5

User · Answer

Simpler variant of thelsdj s answer   ls -tr   head -n -5   xargs --no-run-if-empty rm    ls -tr displays all the files  oldest first  -t newest first  -r reverse    head -n -5 displays all but the 5 last lines  ie the 5 newest files    xargs rm calls rm for each selected file

[bash] Delete all but the most recent X files in bash

Examples related to bash

Examples related to unix

Examples related to scripting