So I follow first answer, but my vnc viewer gives me grey screen when I connect to it. And I found this Ask Ubuntu link to solve that.
The only difference with previous answer is you need to install these extra packages:
apt-get install gnome-panel gnome-settings-daemon metacity nautilus gnome-terminal
And use this ~/.vnc/xstartup file:
#!/bin/sh
export XKL_XMODMAP_DISABLE=1
unset SESSION_MANAGER
unset DBUS_SESSION_BUS_ADDRESS
[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup
[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresources
xsetroot -solid grey
vncconfig -iconic &
gnome-panel &
gnome-settings-daemon &
metacity &
nautilus &
gnome-terminal &
Everything else is the same.
Tested on EC2 Ubuntu 14.04 LTS.