If you're having font size issues (your font isn't responding to size changes)...
@codester has the right code:
myButton.titleLabel!.font = UIFont(name: YourfontName, size: 20)
However, my font size wasn't changing. It turns out that I asking for a font that didn't exist ("HelveticaNeue-Regular"). It wasn't causing a crash, but seemed to be just ignoring that font statement because of it. Once I changed the font to something that does exist, changes to "size: x" did render.