Update April 2013: Don't do this. It wasn't a good idea in 2009 when I first answered the question and it really isn't a good idea now. See this answer by hackbod for reasons:
Avoid reloading activity with asynctask on orientation change in android
Add android:configChanges="keyboardHidden|orientation" to your AndroidManifest.xml. This tells the system what configuration changes you are going to handle yourself - in this case by doing nothing.
<activity android:name="MainActivity"
android:screenOrientation="portrait"
android:configChanges="keyboardHidden|orientation">
See Developer reference configChanges for more details.
However, your application can be interrupted at any time, e.g. by a phone call, so you really should add code to save the state of your application when it is paused.
Update: As of Android 3.2, you also need to add "screenSize":
<activity
android:name="MainActivity"
android:screenOrientation="portrait"
android:configChanges="keyboardHidden|orientation|screenSize">
From Developer guide Handling the Configuration Change Yourself
Caution: Beginning with Android 3.2 (API level 13), the "screen size" also changes when the device switches between portrait and landscape orientation. Thus, if you want to prevent runtime restarts due to orientation change when developing for API level 13 or higher (as declared by the minSdkVersion and targetSdkVersion attributes), you must include the "screenSize" value in addition to the "orientation" value. That is, you must declare
android:configChanges="orientation|screenSize". However, if your application targets API level 12 or lower, then your activity always handles this configuration change itself (this configuration change does not restart your activity, even when running on an Android 3.2 or higher device).