[html] Why shouldn't `'` be used to escape single quotes?

As stated in, When did single quotes in HTML become so popular? and Jquery embedded quote in attribute, the Wikipedia entry on HTML says the following:

The single-quote character ('), when used to quote an attribute value, must also be escaped as ' or ' (should NOT be escaped as ' except in XHTML documents) when it appears within the attribute value itself.

Why shouldn't ' be used? Also, is " safe to be used instead of "?

' is not part of the HTML 4 standard.

" is, though, so is fine to use.

If you really need single quotes, apostrophes, you can use

html    | numeric | hex
‘ | ‘  | ‘ // for the left/beginning single-quote and
’ | ’  | ’ // for the right/ending single-quote

" is valid in both HTML5 and HTML4.

' is valid in HTML5, but not HTML4. However, most browsers support ' for HTML4 anyway.

If you need to write semantically correct mark-up, even in HTML5, you must not use ' to escape single quotes. Although, I can imagine you actually meant apostrophe rather then single quote.

single quotes and apostrophes are not the same, semantically, although they might look the same.

Here's one apostrophe.

Use ' to insert it if you need HTML4 support. (edited)

In British English, single quotes are used like this:

"He told me to 'give it a try'", I said.

Quotes come in pairs. You can use:

<p><q>He told me to <q>give it a try</q></q>, I said.<p>

to have nested quotes in a semantically correct way, deferring the substitution of the actual characters to the rendering engine. This substitution can then be affected by CSS rules, like:

q {
  quotes: '"' '"' '<' '>';
} 

An old but seemingly still relevant article about semantically correct mark-up: The Trouble With EM ’n EN (and Other Shady Characters).

(edited) This used to be:

Use ’ to insert it if you need HTML4 support.

But, as @James_pic pointed out, that is not the straight single quote, but the "Single curved quote, right".