Well I've tried everything I found on the internet and none of them worked. Calling System.gc() only drags down the speed of app. Recycling bitmaps in onDestroy didn't work for me too.
The only thing that works now is to have a static list of all the bitmap so that the bitmaps survive after a restart. And just use the saved bitmaps instead of creating new ones every time the activity if restarted.
In my case the code looks like this:
private static BitmapDrawable currentBGDrawable;
if (new File(uriString).exists()) {
if (!uriString.equals(currentBGUri)) {
freeBackground();
bg = BitmapFactory.decodeFile(uriString);
currentBGUri = uriString;
bgDrawable = new BitmapDrawable(bg);
currentBGDrawable = bgDrawable;
} else {
bgDrawable = currentBGDrawable;
}
}