The following worked for me to get GPUImage compiling without errors on Xcode 5.1 for both the 64-bit simulator and retina iPad Mini, without needing to remove arm64 from the Valid Architectures list (which defeats the purpose of owning a 64-bit device for testing 64-bit performance).
Download the .zip folder from the GitHub page: https://github.com/BradLarson/GPUImage
Unzip, and navigate to the 'framework' folder. From here, add and copy the 'Source' folder into your Xcode project. Ensure 'Copy items into destination group's folder' is ticked, and that 'Create groups for any added folders' is also ticked. This will copy the generic, iOS and Mac header/implementation files into your project.
If you don't need the Mac files because you're compiling for iOS you can delete the Mac folder either before you copy the files into your project, or simply delete the group from within Xcode.
Once you've added the Source folder to your project just use the following to begin using GPUImage's classes/methods:
#import "Source/GPUImage.h"
A few things to point out:
Hope the above helps - it seems there were no clear instructions anywhere despite the question being asked multiple times, but fear not, GPUImage definitely works for arm64 architecture!