I ended up writing my own program (really small) to display just the size. More information here: http://fwhacking.blogspot.com/2011/03/bfsize-print-file-size-in-bytes-and.html
The two most clean ways in my opinion with common Linux tools are:
$ stat -c %s /usr/bin/stat
50000
$ wc -c < /usr/bin/wc
36912
But I just don't want to be typing parameters or pipe the output just to get a file size, so I'm using my own bfsize.