I tried experimenting with the answers given. My personal finding came out to be:
git rm -r --cached .
And then
git add .
This seemed to make my working directory nice and clean. You can put your fileName in place of the dot.
Source: Stackoverflow.com