It varies based on the options that you pass to install
and the contents of the distutils configuration files on the system/in the package. I don't believe that any files are modified outside of directories specified in these ways.
Notably, distutils does not have an uninstall command at this time.
It's also noteworthy that deleting a package/egg can cause dependency issues – utilities like easy_install
attempt to alleviate such problems.