[linux] Unix: How to delete files listed in a file

I have a long text file with list of file masks I want to delete

Example:

/tmp/aaa.jpg
/var/www1/*
/var/www/qwerty.php

I need delete them. Tried rm `cat 1.txt` and it says the list is too long.

Found this command, but when I check folders from the list, some of them still have files xargs rm <1.txt Manual rm call removes files from such folders, so no issue with permissions.

xargs -I{} sh -c 'rm {}' < 1.txt should do what you want. Be careful with this command as one incorrect entry in that file could cause a lot of trouble.

This answer was edited after @tdavies pointed out that the original did not do shell expansion.

Just to provide an another way, you can also simply use the following command

$ cat to_remove
/tmp/file1
/tmp/file2
/tmp/file3
$ rm $( cat to_remove )

Use this:

while IFS= read -r file ; do rm -- "$file" ; done < delete.list

If you need glob expansion you can omit quoting $file:

IFS=""
while read -r file ; do rm -- $file ; done < delete.list

But be warned that file names can contain "problematic" content and I would use the unquoted version. Imagine this pattern in the file

*
*/*
*/*/*

This would delete quite a lot from the current directory! I would encourage you to prepare the delete list in a way that glob patterns aren't required anymore, and then use quoting like in my first example.

This will allow file names to have spaces (reproducible example).

# Select files of interest, here, only text files for ex.
find -type f -exec file {} \; > findresult.txt
grep ": ASCII text$" findresult.txt > textfiles.txt
# leave only the path to the file removing suffix and prefix
sed -i -e 's/:.*$//' textfiles.txt
sed -i -e 's/\.\///' textfiles.txt

#write a script that deletes the files in textfiles.txt
IFS_backup=$IFS
IFS=$(echo "\n\b")
for f in $(cat textfiles.txt); 
do 
rm "$f"; 
done
IFS=$IFS_backup

# save script as "some.sh" and run: sh some.sh

You can use this one-liner:

cat 1.txt | xargs echo rm | sh

Which does shell expansion but executes rm the minimum number of times.

You could use '\n' for define the new line character as delimiter.

xargs -d '\n' rm < 1.txt

Be careful with the -rf because it can delete what you don't want to if the 1.txt contains paths with spaces. That's why the new line delimiter a bit safer.

On BSD systems, you could use -0 option to use new line characters as delimiter like this:

xargs -0 rm < 1.txt

Assuming that the list of files is in the file 1.txt, then do:

xargs rm -r <1.txt

The -r option causes recursion into any directories named in 1.txt.

If any files are read-only, use the -f option to force the deletion:

xargs rm -rf <1.txt

Be cautious with input to any tool that does programmatic deletions. Make certain that the files named in the input file are really to be deleted. Be especially careful about seemingly simple typos. For example, if you enter a space between a file and its suffix, it will appear to be two separate file names:

file .txt

is actually two separate files: file and .txt.

This may not seem so dangerous, but if the typo is something like this:

myoldfiles *

Then instead of deleting all files that begin with myoldfiles, you'll end up deleting myoldfiles and all non-dot-files and directories in the current directory. Probably not what you wanted.

Here you can use set of folders from deletelist.txt while avoiding some patterns as well

foreach f (cat deletelist.txt)
    rm -rf ls | egrep -v "needthisfile|*.cpp|*.h"
end

Here's another looping example. This one also contains an 'if-statement' as an example of checking to see if the entry is a 'file' (or a 'directory' for example):

for f in $(cat 1.txt); do if [ -f $f ]; then rm $f; fi; done

cat 1.txt | xargs rm -f | bash Run the command will do the following for files only.

cat 1.txt | xargs rm -rf | bash Run the command will do the following recursive behaviour.

In case somebody prefers sed and removing without wildcard expansion:

sed -e "s/^\(.*\)$/rm -f -- \'\1\'/" deletelist.txt | /bin/sh

Reminder: use absolute pathnames in the file or make sure you are in the right directory.

And for completeness the same with awk:

awk '{printf "rm -f -- '\''%s'\''\n",$1}' deletelist.txt | /bin/sh

Wildcard expansion will work if the single quotes are remove, but this is dangerous in case the filename contains spaces. This would need to add quotes around the wildcards.

In this particular case, due to the dangers cited in other answers, I would

1. Edit in e.g. Vim and :%s/\s/\\\0/g, escaping all space characters with a backslash.

2. Then :%s/^/rm -rf /, prepending the command. With -r you don't have to worry to have directories listed after the files contained therein, and with -f it won't complain due to missing files or duplicate entries.

3. Run all the commands: $ source 1.txt

This is not very efficient, but will work if you need glob patterns (as in /var/www/*)

for f in $(cat 1.txt) ; do 
  rm "$f"
done

If you don't have any patterns and are sure your paths in the file do not contain whitespaces or other weird things, you can use xargs like so:

xargs rm < 1.txt