You must use 'gem uninstall gem_name' to uninstall a gem.
Note that if you installed the gem system-wide (ie. sudo bundle install) then you may need to specify the binary directory using the -n option, to ensure binaries belonging to the gem are removed. For example
sudo gem uninstall gem_name -n /usr/lib/ruby/gems/1.9.1/bin