I can see a number of reasons:
Readability
string s = string.Format("Hey, {0} it is the {1}st day of {2}. I feel {3}!", _name, _day, _month, _feeling);
vs:
string s = "Hey," + _name + " it is the " + _day + "st day of " + _month + ". I feel " + feeling + "!";
Format Specifiers (and this includes the fact you can write custom formatters)
string s = string.Format("Invoice number: {0:0000}", _invoiceNum);
vs:
string s = "Invoice Number = " + ("0000" + _invoiceNum).Substr(..... /*can't even be bothered to type it*/)
String Template Persistence
What if I want to store string templates in the database? With string formatting:
_id _translation
1 Welcome {0} to {1}. Today is {2}.
2 You have {0} products in your basket.
3 Thank-you for your order. Your {0} will arrive in {1} working days.
vs:
_id _translation
1 Welcome
2 to
3 . Today is
4 .
5 You have
6 products in your basket.
7 Someone
8 just shoot
9 the developer.