I'm not about to repeat all the other answers, but there's one aspect that I haven't yet seen mentioned - it's the story of disappearing data. I don't know where to find it, but...
Basically it's about a web application that mysteriously every few night did loose all its data and nobody knew why. Inspecting the Logs later revealed that the site was found by google or another arbitrary spider, that happily GET (read: GOT) all the links it found on the site - including the "delete this entry" and "are you sure?" links.
Actually - part of this has been mentioned. This is the story behind "don't change data on GET but only on POST". Crawlers will happily follow GET, never POST. Even robots.txt doesn't help against misbehaving crawlers.