Just to add to these answers, In my case I had no local mySQL server, it was running inside a docker container. So the socket file does not exist and will not be accessible for the "mysql" client.
The sock file gets created by mysqld and mysql uses this to communicate with it. However if your mySql server is not running local, it does not require the sock file.
By specifying a host name/ip the sock file is not required e.g.
mysql --host=127.0.0.1 --port=3306 --user=xyz --password=xyz