I just ran across a free script file that will convert your CSV poi files to a KML file. If you have Google Earth installed, you can just double click on the KML file and see all of your poi entries in Google Earth. If you have a modest amount of entries, you can also import and create a map in Google Maps.
Many thanks to Luis Schwabe
link to the script file http://sites.google.com/site/lschwabe/