As I commented above, problem seems to be the conversion from PIL internal list format to a standard python list type. I've found that Image.tostring() is much faster, and depending on your needs it might be enough. In my case, I needed to calculate the CRC32 digest of image data, and it suited fine.
If you need to perform more complex calculations, tom10 response involving numpy might be what you need.