You can exclude like this, the regex 'or' symbol, assuming a file you want doesn't have the same name as a folder you're excluding.
$exclude = 'dir1|dir2|dir3'
ls -r | where { $_.fullname -notmatch $exclude }
ls -r -dir | where fullname -notmatch 'dir1|dir2|dir3'