(Update: a few years later Google and Qwant "airlines" still send me here when searching for "git non-default ssh port") A probably better way in newer git versions is to use the GIT_SSH_COMMAND ENV.VAR like:
GIT_SSH_COMMAND="ssh -oPort=1234 -i ~/.ssh/myPrivate_rsa.key" \
git clone myuser@myGitRemoteServer:/my/remote/git_repo/path
This has the added advantage of allowing any other ssh suitable option (port, priv.key, IPv6, PKCS#11 device, ...).