Step 1:
Added a config file in ~/.ssh/config
file which looks like
User git
Hostname gitlab.com
IdentityFile ~/.ssh/id_rsa_gitlab
TCPKeepAlive yes
IdentitiesOnly yes
Step 2: Just clone the git repo WITHOUT sudo.
Documentation: https://gitlab.com/help/ssh/README#working-with-non-default-ssh-key-pair-paths