@John Woo's accepted answer has some caveats which you should be aware of:
So, there is an intermediate level between your SQL Server instance and the session which you can use to control the default language setting for session - login level.
SQL Server Instance level setting
-> User login level setting
-> Query Session level setting
This can help you in case you want to set default language of all new sessions belonging to some specific user only.
Simply change the default language setting of the target user login as per this link and you are all set. You can also do it from SQL Server Management Studio (SSMS) UI. Below you can see the default language setting in properties window of sa
user in SQL Server:
Note: Also, it is important to know that changing the setting doesn't affect the default language of already active sessions from that user login. It will affect only the new sessions created after changing the setting.