The answers provided (at the time of this post) are link only answers so I thought I would summarize the links into an answer and what I will be using.
When working to create Cross Browser Favicons (including touch icons) there are several things to consider.
The first (of course) is Internet Explorer. IE does not support PNG favicons until version 11. So our first line is a conditional comment for favicons in IE 9 and below:
<!--[if IE]><link rel="shortcut icon" href="path/to/favicon.ico"><![endif]-->
To cover the uses of the icon create it at 32x32 pixels. Notice the rel="shortcut icon" for IE to recognize the icon it needs the word shortcut which is not standard. Also we wrap the .ico favicon in a IE conditional comment because Chrome and Safari will use the .ico file if it is present, despite other options available, not what we would like.
The above covers IE up to IE 9. IE 11 accepts PNG favicons, however, IE 10 does not. Also IE 10 does not read conditional comments thus IE 10 won't show a favicon. With IE 11 and Edge available I don't see IE 10 in widespread use, so I ignore this browser.
For the rest of the browsers we are going to use the standard way to cite a favicon:
<link rel="icon" href="path/to/favicon.png">
This icon should be 196x196 pixels in size to cover all devices that may use this icon.
To cover touch icons on mobile devices we are going to use Apple's proprietary way to cite a touch icon:
<link rel="apple-touch-icon-precomposed" href="apple-touch-icon-precomposed.png">
Using rel="apple-touch-icon-precomposed" will not apply the reflective shine when bookmarked on iOS. To have iOS apply the shine use rel="apple-touch-icon". This icon should be sized to 180x180 pixels as that is the current size recommend by Apple for the latest iPhones and iPads. I have read Blackberry will also use rel="apple-touch-icon-precomposed".
As a note: Chrome for Android states:
The apple-touch-* are deprecated, and will be supported only for a short time. (Written as of beta for m31 of Chrome).
Custom Tiles for IE 11+ on Windows 8.1+
IE 11+ on Windows 8.1+ does offer a way to create pinned tiles for your site.
Microsoft recommends creating a few tiles at the following size:
Small: 128 x 128
Medium: 270 x 270
Wide: 558 x 270
Large: 558 x 558
These should be transparent images as we will define a color background next.
Once these images are created you should create an xml file called browserconfig.xml with the following code:
<?xml version="1.0" encoding="utf-8"?>
<browserconfig>
<msapplication>
<tile>
<square70x70logo src="images/smalltile.png"/>
<square150x150logo src="images/mediumtile.png"/>
<wide310x150logo src="images/widetile.png"/>
<square310x310logo src="images/largetile.png"/>
<TileColor>#009900</TileColor>
</tile>
</msapplication>
</browserconfig>
Save this xml file in the root of your site. When a site is pinned IE will look for this file. If you want to name the xml file something different or have it in a different location add this meta tag to the head:
<meta name="msapplication-config" content="path-to-browserconfig/custom-name.xml" />
For additional information on IE 11+ custom tiles and using the XML file visit Microsoft's website.
Putting it all together:
To put it all together the above code would look like this:
<!-- For IE 9 and below. ICO should be 32x32 pixels in size -->
<!--[if IE]><link rel="shortcut icon" href="path/to/favicon.ico"><![endif]-->
<!-- Touch Icons - iOS and Android 2.1+ 180x180 pixels in size. -->
<link rel="apple-touch-icon-precomposed" href="apple-touch-icon-precomposed.png">
<!-- Firefox, Chrome, Safari, IE 11+ and Opera. 196x196 pixels in size. -->
<link rel="icon" href="path/to/favicon.png">
Windows Phone Live Tiles
If a user is using a Windows Phone they can pin a website to the start screen of their phone. Unfortunately, when they do this it displays a screenshot of your phone, not a favicon (not even the MS specific code referenced above). To make a "Live Tile" for Windows Phone Users for your website one must use the following code:
Here are detailed instructions from Microsoft but here is a synopsis:
Step 1
Create a square image for your website, to support hi-res screens create it at 768x768 pixels in size.
Step 2
Add a hidden overlay of this image. Here is example code from Microsoft:
<div id="TileOverlay" onclick="ToggleTileOverlay()" style='background-color: Highlight; height: 100%; width: 100%; top: 0px; left: 0px; position: fixed; color: black; visibility: hidden'>
<img src="customtile.png" width="320" height="320" />
<div style='margin-top: 40px'>
Add text/graphic asking user to pin to start using the menu...
</div>
</div>
Step 3
You then can add thew following line to add a pin to start link:
<a href="javascript:ToggleTileOverlay()">Pin this site to your start screen</a>
Microsoft recommends that you detect windows phone and only show that link to those users since it won't work for other users.
Step 4
Next you add some JS to toggle the overlay visibility
<script>
function ToggleTileOverlay() {
var newVisibility = (document.getElementById('TileOverlay').style.visibility == 'visible') ? 'hidden' : 'visible';
document.getElementById('TileOverlay').style.visibility = newVisibility;
}
</script>
Note on Sizes
I am using one size as every browser will scale down the image as necessary. I could add more HTML to specify multiple sizes if desired for those with a lower bandwidth but I am already compressing the PNG files heavily using TinyPNG and I find this unnecessary for my purposes. Also, according to philippe_b's answer Chrome and Firefox have bugs that cause the browser to load all sizes of icons. Using one large icon may be better than multiple smaller ones because of this.
Further Reading
For those who would like more details see the links below: