A combination of the 2 answers worked for me
git rm -r one-of-the-directories
git commit . -m "Remove duplicated directory"
git push
if it still shows some warning, remove the files manually
git filter-branch --tree-filter 'rm -rf path/to/your/file' HEAD
git push