In case this happens to someone. I built my own library to use with a third party code. While I was building it to deliver, I accidentally left my iPhone 4S plugged in, and so Xcode built my library only for the plugged architecture instead of following the project settings. Remove any plugged in devices and rebuilt the library, link it, and you should be all right.
Hope it helps.