If you are using RVM you can install your gems into gemsets. That way when you want to perform a full cleanup you can simply remove the gemset, which in turn removes all the gems installed in it. Your other option is to simply uninstall your unused gems and re-run your bundle install command.
Since bundler is meant to be a project-per-project gem versioning tool it does not provide a bundle clean command. Doing so would mean the possibility of removing gems associated with other projects as well, which would not be desirable. That means that bundler is probably the wrong tool to use to manage your gem directory. My personal recommendation would be to use RVM gemsets to sandbox your gems in certain projects or ruby versions.