Like ?egDwight said, you can use the HTML entity ∞ or ∞. A easy source of getting these codes, is to look at this entity list.
You can also use them in CSS, which was what I was looking for, just like font-awesome does. A simple CSS based solution would be to have something like:
.infinity-ico:before {
content: '\221E';
}