Edit: As suggested in comments. This solution is for Linux OS. That too if you have installed ruby manually from package-manager.
If you want to have multiple ruby versions, better to have RVM. In that case you don't need to remove ruby older version.
Still if want to remove then follow the steps below:
First you should find where Ruby is:
whereis ruby
will list all the places where it exists on your system, then you can remove all them explicitly. Or you can use something like this:
rm -rf /usr/local/lib/ruby
rm -rf /usr/lib/ruby
rm -f /usr/local/bin/ruby
rm -f /usr/bin/ruby
rm -f /usr/local/bin/irb
rm -f /usr/bin/irb
rm -f /usr/local/bin/gem
rm -f /usr/bin/gem