I think you should have a look at the Pathogen plugin. After you have this installed, you can keep all of your plugins in separate folders in ~/.vim/bundle/, and Pathogen will take care of loading them.
Or, alternatively, perhaps you would prefer Vundle, which provides similar functionality (with the added bonus of automatic updates from plugins in github).