Using [Authorize]
attributes can help prevent security holes in your application. The way that MVC handles URL's (i.e. routing them to a controller rather than to an actual file) makes it difficult to actually secure everything via the web.config file.
Read more here: http://blogs.msdn.com/b/rickandy/archive/2012/03/23/securing-your-asp-net-mvc-4-app-and-the-new-allowanonymous-attribute.aspx (via archive.org)