I have run to this same issue, and what I found is there are actually running mulitple Windows form application in the background. It happens when your application has two forms and you close the 2nd form which is not your main form so the application will not totally exited.
I usually run my application
Solution is close the other instance of Windows form application. This is one way to always close your application instance.