Quote from official documentation as of August 2013:
User-created libraries as of version 0017 go in a subdirectory of your default sketch directory. For example, on OSX, the new directory would be ~/Documents/Arduino/libraries/. On Windows, it would be My Documents\Arduino\libraries\. To add your own library, create a new directory in the libraries directory with the name of your library. The folder should contain a C or C++ file with your code and a header file with your function and variable declarations. It will then appear in the Sketch | Import Library menu in the Arduino IDE.
To remove a library, stop the Arduino IDE and remove the library directory from the aforementioned location.