You can also use the DeleteAllOnSubmit() method by passing it your results in a generic list rather than in var. This way your foreach reduces to one line of code:
List<Widgets> widgetList = context.Widgets
.Where(w => w.WidgetId == widgetId).ToList<Widgets>();
context.Widgets.DeleteAllOnSubmit(widgetList);
context.SubmitChanges();
It probably still uses a loop internally though.