The biggest problem with a batch script is handling rollback when the user clicks cancel (or something goes wrong during your install). The correct way to handle this scenario is to create a CustomAction that adds temporary rows to the RemoveFiles table. That way the Windows Installer handles the rollback cases for you. It is insanely simpler when you see the solution.
Anyway, to have an action only execute during uninstall add a Condition element with:
REMOVE ~= "ALL"
the ~= says compare case insensitive (even though I think ALL is always uppercaesd). See the MSI SDK documentation about Conditions Syntax for more information.
PS: There has never been a case where I sat down and thought, "Oh, batch file would be a good solution in an installation package." Actually, finding an installation package that has a batch file in it would only encourage me to return the product for a refund.