1: No difference. It is kept around to allow old S-code to continue to function. This is documented a "Note" in ?Math
2: Yes: But you already know it:
`^`(x,y)
#[1] 1024
In R the mathematical operators are really functions that the parser takes care of rearranging arguments and function names for you to simulate ordinary mathematical infix notation. Also documented at ?Math.
Edit: Let me add that knowing how R handles infix operators (i.e. two argument functions) is very important in understanding the use of the foundational infix "[[" and "["-functions as (functional) second arguments to lapply and sapply:
> sapply( list( list(1,2,3), list(4,3,6) ), "[[", 1)
[1] 1 4
> firsts <- function(lis) sapply(lis, "[[", 1)
> firsts( list( list(1,2,3), list(4,3,6) ) )
[1] 1 4