Just access the element and append it to the value.
<input
type="button"
onclick="document.getElementById('myTextArea').value += '•'"
value="Add •">
See a live demo.
For the sake of keeping things simple, I haven't written unobtrusive JS. For a production system you should.
Also it needs to be a UTF8 character.
Browsers generally submit forms using the encoding they received the page in. Serve your page as UTF-8 if you want UTF-8 data submitted back.