If you don't have the option to delete the already existing file in the new location, but still need to move and delete from the original location, this renaming trick might work:
string newFileLocation = @"c:\test\Test\SomeFile.txt";
while (File.Exists(newFileLocation)) {
newFileLocation = newFileLocation.Split('.')[0] + "_copy." + newFileLocation.Split('.')[1];
}
File.Move(@"c:\test\SomeFile.txt", newFileLocation);
This assumes the only '.' in the file name is before the extension. It splits the file in two before the extension, attaches "_copy." in between. This lets you move the file, but creates a copy if the file already exists or a copy of the copy already exists, or a copy of the copy of the copy exists... ;)