I didn't find this solution here. It removes all of it using the cowplot package:
library(cowplot)
p + theme_nothing() +
theme(legend.position="none") +
scale_x_continuous(expand=c(0,0)) +
scale_y_continuous(expand=c(0,0)) +
labs(x = NULL, y = NULL)
Just noticed that the same thing can be accomplished using theme.void() like this:
p + theme_void() +
theme(legend.position="none") +
scale_x_continuous(expand=c(0,0)) +
scale_y_continuous(expand=c(0,0)) +
labs(x = NULL, y = NULL)