Markdown does not officially support video embeddings but you can embed raw HTML in it. I tested out with GitHub Pages and it works flawlessly.
The snippet looks like:
<iframe width="560" height="315"
src="https://www.youtube.com/embed/MUQfKFzIOeU"
frameborder="0"
allow="accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture"
allowfullscreen></iframe>
PS: You can check out the live preview here