Answering my question based on the suggestions from Maudicus and Hit.
Check the WebView tutorial here. Just implement the web client and set it before loadUrl. The simplest way is:
myWebView.setWebViewClient(new WebViewClient());
For more advanced processing for the web content, consider the ChromeClient.