If you are running a linux server:
This system is quite secure/handy for homeservers -usually with all ports blocked by default-. You only have to forward the SSH port (don't use number 22).
If you like Microsoft Terminal Server you can even set a SSH Tunneling to your computer and connect securely to your web server throught it.
With ssh tunneling you even can forward the 3306 port of your remote server to a local port and connect using local phpMyAdmin or MySQL Workbench.
I understand that this option is an overkill, but is as secure as the access of your private key.