In my opinion, most answers proposed here as sub-optimal. The one coming closest is the original one proposed initially by Brann.
A Temp Filename must be
Because of these requirements, it is not a godd idea to program such a beast on your own. Smart People writing IO Libraries worry about things like locking (if needed) etc. Therefore, I see no need to rewrite System.IO.Path.GetTempFileName().
This, even if it looks clumsy, should do the job:
//Note that this already *creates* the file
string filename1 = System.IO.Path.GetTempFileName()
// Rename and move
filename = filename.Replace(".tmp", ".csv");
File.Move(filename1 , filename);