How can I use inverse or negative wildcards when pattern matching in a unix linux shell

Question

Say I want to copy the contents of a directory excluding files and folders whose names contain the word  Music    cp  exclude-matches   Music   target directory   What should go in place of  exclude-matches  to accomplish this

User · Accepted Answer

In Bash you can do it by enabling the extglob option  like this  replace ls with cp and add the target directory  of course     foobar gt  shopt extglob extglob        off   foobar gt  ls abar  afoo  bbar  bfoo   foobar gt  ls   b   -bash     event not found   foobar gt  shopt -s extglob    Enables extglob   foobar gt  ls   b   abar  afoo   foobar gt  ls   a   bbar  bfoo   foobar gt  ls    foo  abar  bbar   You can later disable extglob with  shopt -u extglob

User · Answer

If you want to avoid the mem cost of using the exec command  I believe you can do better with xargs  I think the following is a more efficient alternative to  find foo -type f   -name   Music   -exec cp    bar      new proc for each exec    find   -maxdepth 1 -name   Music   -prune -o -print0   xargs -0 -i cp    dest

User · Answer

Not in bash  that I know of   but   cp  ls   grep -v Music   target directory   I know this is not exactly what you were looking for  but it will solve your example

User · Answer

In bash  an alternative to shopt -s extglob is the GLOBIGNORE variable  It s not really better  but I find it easier to remember   An example that may be what the original poster wanted   GLOBIGNORE   techno    cp  Music   only good music    When done  unset GLOBIGNORE to be able to rm  techno  in the source directory

User · Answer

You can also use a pretty simple for loop   for f in  find   -not -name   Music    do     cp  f  target dir done

User · Answer

My personal preference is to use grep and the while command  This allows one to write powerful yet readable scripts ensuring that you end up doing exactly what you want  Plus by using an echo command you can perform a dry run before carrying out the actual operation  For example   ls   grep -v  Music    while read filename do echo  filename done   will print out the files that you will end up copying  If the list is correct the next step is to simply replace the echo command with the copy command as follows   ls   grep -v  Music    while read filename do cp   filename   target directory done

User · Answer

The extglob shell option gives you more powerful pattern matching in the command line   You turn it on with shopt -s extglob  and turn it off with shopt -u extglob   In your example  you would initially do     shopt -s extglob   cp    Music    target directory   The full available extended globbing operators are  excerpt from man bash       If  the  extglob  shell option is enabled using the shopt builtin  several extended   pattern matching operators are recognized A pattern-list  is  a list of one or more patterns separated by a     Composite patterns may be formed using one or more of the following sub-patterns            pattern-list    Matches zero or one occurrence of the given patterns       pattern-list    Matches zero or more occurrences of the given patterns       pattern-list    Matches one or more occurrences of the given patterns       pattern-list    Matches one of the given patterns       pattern-list    Matches anything except one of the given patterns      So  for example  if you wanted to list all the files in the current directory that are not  c or  h files  you would do     ls -d       c  h     Of course  normal shell globing works  so the last example could also be written as     ls -d      ch

User · Answer

One solution for this can be found with find     mkdir foo bar   touch foo a txt foo Music txt   find foo -type f   -name   Music   -exec cp    bar      ls bar a txt   Find has quite a few options  you can get pretty specific on what you include and exclude   Edit  Adam in the comments noted that this is recursive   find options mindepth and maxdepth can be useful in controlling this

User · Answer

If you want to avoid the mem cost of using the exec command  I believe you can do better with xargs  I think the following is a more efficient alternative to  find foo -type f   -name   Music   -exec cp    bar      new proc for each exec    find   -maxdepth 1 -name   Music   -prune -o -print0   xargs -0 -i cp    dest

User · Answer

You can also use a pretty simple for loop   for f in  find   -not -name   Music    do     cp  f  target dir done

User · Answer

My personal preference is to use grep and the while command  This allows one to write powerful yet readable scripts ensuring that you end up doing exactly what you want  Plus by using an echo command you can perform a dry run before carrying out the actual operation  For example   ls   grep -v  Music    while read filename do echo  filename done   will print out the files that you will end up copying  If the list is correct the next step is to simply replace the echo command with the copy command as follows   ls   grep -v  Music    while read filename do cp   filename   target directory done

User · Answer

If you want to avoid the mem cost of using the exec command  I believe you can do better with xargs  I think the following is a more efficient alternative to  find foo -type f   -name   Music   -exec cp    bar      new proc for each exec    find   -maxdepth 1 -name   Music   -prune -o -print0   xargs -0 -i cp    dest

User · Answer

The extglob shell option gives you more powerful pattern matching in the command line   You turn it on with shopt -s extglob  and turn it off with shopt -u extglob   In your example  you would initially do     shopt -s extglob   cp    Music    target directory   The full available extended globbing operators are  excerpt from man bash       If  the  extglob  shell option is enabled using the shopt builtin  several extended   pattern matching operators are recognized A pattern-list  is  a list of one or more patterns separated by a     Composite patterns may be formed using one or more of the following sub-patterns            pattern-list    Matches zero or one occurrence of the given patterns       pattern-list    Matches zero or more occurrences of the given patterns       pattern-list    Matches one or more occurrences of the given patterns       pattern-list    Matches one of the given patterns       pattern-list    Matches anything except one of the given patterns      So  for example  if you wanted to list all the files in the current directory that are not  c or  h files  you would do     ls -d       c  h     Of course  normal shell globing works  so the last example could also be written as     ls -d      ch

User · Answer

The extglob shell option gives you more powerful pattern matching in the command line   You turn it on with shopt -s extglob  and turn it off with shopt -u extglob   In your example  you would initially do     shopt -s extglob   cp    Music    target directory   The full available extended globbing operators are  excerpt from man bash       If  the  extglob  shell option is enabled using the shopt builtin  several extended   pattern matching operators are recognized A pattern-list  is  a list of one or more patterns separated by a     Composite patterns may be formed using one or more of the following sub-patterns            pattern-list    Matches zero or one occurrence of the given patterns       pattern-list    Matches zero or more occurrences of the given patterns       pattern-list    Matches one or more occurrences of the given patterns       pattern-list    Matches one of the given patterns       pattern-list    Matches anything except one of the given patterns      So  for example  if you wanted to list all the files in the current directory that are not  c or  h files  you would do     ls -d       c  h     Of course  normal shell globing works  so the last example could also be written as     ls -d      ch

User · Answer

A trick I haven t seen on here yet that doesn t use extglob  find  or grep is to treat two file lists as sets and  diff  them using comm   comm -23  lt  ls   lt  ls  Music     comm is preferable over diff because it doesn t have extra cruft   This returns all elements of set 1  ls  that are not also in set 2  ls  Music   This requires both sets to be in sorted order to work properly  No problem for ls and glob expansion  but if you re using something like find  be sure to invoke sort   comm -23  lt  find     sort   lt  find     grep -i   jpg    sort    Potentially useful

User · Answer

The extglob shell option gives you more powerful pattern matching in the command line   You turn it on with shopt -s extglob  and turn it off with shopt -u extglob   In your example  you would initially do     shopt -s extglob   cp    Music    target directory   The full available extended globbing operators are  excerpt from man bash       If  the  extglob  shell option is enabled using the shopt builtin  several extended   pattern matching operators are recognized A pattern-list  is  a list of one or more patterns separated by a     Composite patterns may be formed using one or more of the following sub-patterns            pattern-list    Matches zero or one occurrence of the given patterns       pattern-list    Matches zero or more occurrences of the given patterns       pattern-list    Matches one or more occurrences of the given patterns       pattern-list    Matches one of the given patterns       pattern-list    Matches anything except one of the given patterns      So  for example  if you wanted to list all the files in the current directory that are not  c or  h files  you would do     ls -d       c  h     Of course  normal shell globing works  so the last example could also be written as     ls -d      ch

User · Answer

Not in bash  that I know of   but   cp  ls   grep -v Music   target directory   I know this is not exactly what you were looking for  but it will solve your example

User · Answer

this would do it excluding exactly  Music   cp -a   Music   target   this and that for excluding things like Music   or   Music  cp -a      complete   target cp -a   complete      target

User · Answer

You can also use a pretty simple for loop   for f in  find   -not -name   Music    do     cp  f  target dir done

User · Answer

Not in bash  that I know of   but   cp  ls   grep -v Music   target directory   I know this is not exactly what you were looking for  but it will solve your example

User · Answer

You can also use a pretty simple for loop   for f in  find   -not -name   Music    do     cp  f  target dir done

User · Answer

One solution for this can be found with find     mkdir foo bar   touch foo a txt foo Music txt   find foo -type f   -name   Music   -exec cp    bar      ls bar a txt   Find has quite a few options  you can get pretty specific on what you include and exclude   Edit  Adam in the comments noted that this is recursive   find options mindepth and maxdepth can be useful in controlling this

User · Answer

The following works lists all   txt files in the current dir  except those that begin with a number   This works in bash  dash  zsh and all other POSIX compatible shells   for FILE in  some dir   txt  do      for each   txt file     case    FILE       in              if file basename             0-9    continue               starts with digit  skip     esac        otherwise  do stuff with  FILE here done    In line one the pattern  some dir   txt will cause the for loop to iterate over all files in  some dir whose name end with  txt  In line two a case statement is used to weed out undesired files      The   FILE      expression strips off any leading dir name component from the filename  here  some dir   so that patters can match against only the basename of the file   If you re only weeding out filenames based on suffixes  you can shorten this to  FILE instead   In line three  all files matching the case pattern  0-9    line will be skipped  the continue statement jumps to the next iteration of the for loop       If you want to you can do something more interesting here  e g  like skipping all files which do not start with a letter  a   z  using   a-z    or you could use multiple patterns to skip several kinds of filenames e g   0-9     bak to skip files both  bak files  and files which does not start with a number

User · Answer

this would do it excluding exactly  Music   cp -a   Music   target   this and that for excluding things like Music   or   Music  cp -a      complete   target cp -a   complete      target

User · Answer

One solution for this can be found with find     mkdir foo bar   touch foo a txt foo Music txt   find foo -type f   -name   Music   -exec cp    bar      ls bar a txt   Find has quite a few options  you can get pretty specific on what you include and exclude   Edit  Adam in the comments noted that this is recursive   find options mindepth and maxdepth can be useful in controlling this

User · Answer

If you want to avoid the mem cost of using the exec command  I believe you can do better with xargs  I think the following is a more efficient alternative to  find foo -type f   -name   Music   -exec cp    bar      new proc for each exec    find   -maxdepth 1 -name   Music   -prune -o -print0   xargs -0 -i cp    dest

User · Answer

A trick I haven t seen on here yet that doesn t use extglob  find  or grep is to treat two file lists as sets and  diff  them using comm   comm -23  lt  ls   lt  ls  Music     comm is preferable over diff because it doesn t have extra cruft   This returns all elements of set 1  ls  that are not also in set 2  ls  Music   This requires both sets to be in sorted order to work properly  No problem for ls and glob expansion  but if you re using something like find  be sure to invoke sort   comm -23  lt  find     sort   lt  find     grep -i   jpg    sort    Potentially useful

User · Answer

The following works lists all   txt files in the current dir  except those that begin with a number   This works in bash  dash  zsh and all other POSIX compatible shells   for FILE in  some dir   txt  do      for each   txt file     case    FILE       in              if file basename             0-9    continue               starts with digit  skip     esac        otherwise  do stuff with  FILE here done    In line one the pattern  some dir   txt will cause the for loop to iterate over all files in  some dir whose name end with  txt  In line two a case statement is used to weed out undesired files      The   FILE      expression strips off any leading dir name component from the filename  here  some dir   so that patters can match against only the basename of the file   If you re only weeding out filenames based on suffixes  you can shorten this to  FILE instead   In line three  all files matching the case pattern  0-9    line will be skipped  the continue statement jumps to the next iteration of the for loop       If you want to you can do something more interesting here  e g  like skipping all files which do not start with a letter  a   z  using   a-z    or you could use multiple patterns to skip several kinds of filenames e g   0-9     bak to skip files both  bak files  and files which does not start with a number

User · Answer

Not in bash  that I know of   but   cp  ls   grep -v Music   target directory   I know this is not exactly what you were looking for  but it will solve your example

User · Answer

One solution for this can be found with find     mkdir foo bar   touch foo a txt foo Music txt   find foo -type f   -name   Music   -exec cp    bar      ls bar a txt   Find has quite a few options  you can get pretty specific on what you include and exclude   Edit  Adam in the comments noted that this is recursive   find options mindepth and maxdepth can be useful in controlling this

User · Answer

In bash  an alternative to shopt -s extglob is the GLOBIGNORE variable  It s not really better  but I find it easier to remember   An example that may be what the original poster wanted   GLOBIGNORE   techno    cp  Music   only good music    When done  unset GLOBIGNORE to be able to rm  techno  in the source directory

[bash] How can I use inverse or negative wildcards when pattern matching in a unix/linux shell?

Examples related to bash

Examples related to shell

Examples related to pattern-matching

Examples related to glob