The shorter ones are vectorized, meaning they can return a vector, like this:
((-2:2) >= 0) & ((-2:2) <= 0)
# [1] FALSE FALSE TRUE FALSE FALSE
The longer form evaluates left to right examining only the first element of each vector, so the above gives
((-2:2) >= 0) && ((-2:2) <= 0)
# [1] FALSE
As the help page says, this makes the longer form "appropriate for programming control-flow and [is] typically preferred in if clauses."
So you want to use the long forms only when you are certain the vectors are length one.
You should be absolutely certain your vectors are only length 1, such as in cases where they are functions that return only length 1 booleans. You want to use the short forms if the vectors are length possibly >1. So if you're not absolutely sure, you should either check first, or use the short form and then use all and any to reduce it to length one for use in control flow statements, like if.
The functions all and any are often used on the result of a vectorized comparison to see if all or any of the comparisons are true, respectively. The results from these functions are sure to be length 1 so they are appropriate for use in if clauses, while the results from the vectorized comparison are not. (Though those results would be appropriate for use in ifelse.
One final difference: the && and || only evaluate as many terms as they need to (which seems to be what is meant by short-circuiting). For example, here's a comparison using an undefined value a; if it didn't short-circuit, as & and | don't, it would give an error.
a
# Error: object 'a' not found
TRUE || a
# [1] TRUE
FALSE && a
# [1] FALSE
TRUE | a
# Error: object 'a' not found
FALSE & a
# Error: object 'a' not found
Finally, see section 8.2.17 in The R Inferno, titled "and and andand".