The issue is that, even though you removed the files, they are still present in previous revisions. That's the whole point of git, is that even if you delete something, you can still get it back by accessing the history.
What you are looking to do is called rewriting history, and it involved the git filter-branch
command.
GitHub has a good explanation of the issue on their site. https://help.github.com/articles/remove-sensitive-data
To answer your question more directly, what you basically need to run is this command with unwanted_filename_or_folder
replaced accordingly:
git filter-branch --index-filter 'git rm -r --cached --ignore-unmatch unwanted_filename_or_folder' --prune-empty
This will remove all references to the files from the active history of the repo.
Next step, to perform a GC cycle to force all references to the file to be expired and purged from the packfile. Nothing needs to be replaced in these commands.
git for-each-ref --format='delete %(refname)' refs/original | git update-ref --stdin
# or, for older git versions (e.g. 1.8.3.1) which don't support --stdin
# git update-ref $(git for-each-ref --format='delete %(refname)' refs/original)
git reflog expire --expire=now --all
git gc --aggressive --prune=now