As of Chrome58 you can now use controlsList to remove controls you don't want shown. This is available for both <audio>
and <video>
tags.
If you want to remove the download button in the controls do this:
<audio controls controlsList="nodownload">