I had the same problem in Safari and Chrome (the only ones I've tested) but I just did something that seems to work, at least I haven't been able to reproduce the problem since I added the solution. What I did was add a metatag to the header with a generated timstamp. Doesn't seem right but it's simple :)
<meta name="304workaround" content="2013-10-24 21:17:23">
Update P.S As far as I can tell, the problem disappears when I remove my node proxy (by proxy i mean both express.vhost and http-proxy module), which is weird...