Update (20160519): Firebase just released a new feature called Firebase Storage. This allows you to upload images and other non-JSON data to a dedicated storage service. We highly recommend that you use this for storing images, instead of storing them as base64 encoded data in the JSON database.
You certainly can! Depending on how big your images are, you have a couple options:
1. For smaller images (under 10mb)
We have an example project that does that here: https://github.com/firebase/firepano
The general approach is to load the file locally (using FileReader) so you can then store it in Firebase just as you would any other data. Since images are binary files, you'll want to get the base64-encoded contents so you can store it as a string. Or even more convenient, you can store it as a data: url which is then ready to plop in as the src of an img tag (this is what the example does)!
2. For larger images
Firebase does have a 10mb (of utf8-encoded string data) limit. If your image is bigger, you'll have to break it into 10mb chunks. You're right though that Firebase is more optimized for small strings that change frequently rather than multi-megabyte strings. If you have lots of large static data, I'd definitely recommend S3 or a CDN instead.