The easiest way to do this is generate a KML file (see http://code.google.com/apis/kml/articles/csvtokml.html for a possible solution). You can then open that up in Google Maps by storing it online and linking to it from Google Maps as described at http://code.google.com/apis/kml/documentation/whatiskml.html
EDIT: http://www.gpsbabel.org/ may let you do it without coding.