I had to be sure to change these settings in both the Target and Project settings on xCode 4.3.2 after doing that and setting it to build for both armv6 and armv7 everywhere I was able to submit my app.
For safe measure I also exited xCode between making the changes and doing a clean, build, archive cycle.