I have tried below steps and it worked without stopping the instance. My requirement was - as I have changed my client machine, the old .pem file was not allowing me to log in to the ec2 instance.
You will see your old keys in that file.
ssh-keygen -f YOUR_PEM_FILE.pem -y It will generate a key. Append the key to ~/.ssh/authorized_keys opened in step#1. No need to delete the old key.
From AWS console, create a new key pair. Store it in your new machine. Rename it to the old pem file - reason is old pem file is still associated with the ec2 instance in AWS.
All done.
I am able to log in to the AWS ec2 from my new client machine.