I used to do the same until I started using libraries that are compiled differently depending on the environment such as PyCrypto. My PyCrypto mac wouldn't work on Cygwin wouldn't work on Ubuntu.
It becomes an utter nightmare to manage the repository.
Either way I found it easier to manage the pip freeze & a requirements file than having it all in git. It's cleaner too since you get to avoid the commit spam for thousands of files as those libraries get updated...