The correct HTML to use for images with captions, is <figure>
with <figcaption>
.
There's no Markdown equivalent for this, so if you're only adding the occasional caption, I'd encourage you to just add that html into your Markdown document:
Lorem ipsum dolor sit amet, consectetur adipiscing elit...
<figure>
<img src="{{site.url}}/assets/image.jpg" alt="my alt text"/>
<figcaption>This is my caption text.</figcaption>
</figure>
Vestibulum eu vulputate magna...
The Markdown spec encourages you to embed HTML in cases like this, so it will display just fine. It's also a lot simpler than messing with plugins.
If you're trying to use other Markdown-y features (like tables, asterisks, etc) to produce captions, then you're just hacking around how Markdown was intended to be used.