The above solutions work fine for most cases. However, if you also need to remove all traces of that file (ie sensitive data such as passwords), you will also want to remove it from your entire commit history, as the file could still be retrieved from there.
Here is a solution that removes all traces of the file from your entire commit history, as though it never existed, yet keeps the file in place on your system.
https://help.github.com/articles/remove-sensitive-data/
You can actually skip to step 3 if you are in your local git repository, and don't need to perform a dry run. In my case, I only needed steps 3 and 6, as I had already created my .gitignore file, and was in the repository I wanted to work on.
To see your changes, you may need to go to the GitHub root of your repository and refresh the page. Then navigate through the links to get to an old commit that once had the file, to see that it has now been removed. For me, simply refreshing the old commit page did not show the change.
It looked intimidating at first, but really, was easy and worked like a charm ! :-)