Paulo's help lead me to the solution. It was a combination of the following:
The bash shell treats the dollar sign as a special character for expansion to an environment variable, so we need to escape it with a backslash. Incidentally, we don't have to do this in the case where the dollar sign is the final character of the password.
As an example, if your password is "pas$word", from Linux bash we must connect as follows:
# mysql --host=192.168.233.142 --user=root --password=pas\$word