Check Managing Exceptions with the Debugger page, it explains how to set this up.
Essentially, here are the steps (during debugging):
On the Debug menu, click Exceptions.
In the Exceptions dialog box, select Thrown for an entire category of exceptions, for example, Common Language Runtime Exceptions.
-or-
Expand the node for a category of exceptions, for example, Common Language Runtime Exceptions, and select Thrown for a specific exception within that category.