I just ran into the same issue for Ubuntu 13.04. These commands removed Postgres 9.1:
sudo apt-get purge postgresql
sudo apt-get autoremove postgresql
It occurs to me that perhaps only the second command is necessary, but from there I was able to install Postgres 9.2 (sudo apt-get install postgresql-9.2).