I've found that this might also happen if you rebuild a workspace with a project containing a lot of image data (such as a dedicated images project). Might be best to put something like that into its own workspace and handle it separately to the rest of the projects you deal with.
If you can't, then don't clean that project when you clean and rebuild. Only rebuild when necessary.