On new project, easy peasy:
rails new your_new_project_name -d mysql
On existing project, definitely trickier. This has given me a number of issues on existing rails projects. This kind of works with me:
# On Gemfile:
gem 'mysql2', '>= 0.3.18', '< 0.5' # copied from a new project for rails 5.1 :)
gem 'activerecord-mysql-adapter' # needed for mysql..
# On Dockerfile or on CLI:
sudo apt-get install -y mysql-client libmysqlclient-dev