I've setup xampp for my localhost as well, I've not done anything with the files created by xampp during or after setup.
But in the '.htaccess' file, make sure you've set it to something like this. Works for me, and this should not make any difference for you.
RewriteEngine On
RewriteRule ^filename/?$ filename.html
Change .html to whatever format you're using.
Make sure your install is clean, and just make the .htaccess file. Also remember to put one .htaccess file for each directory (don't really know if you can use ONE file for all folders, but to be safe, just do this and it will always work.