You should connect to the place that your actual application needs. Otherwise you're testing whether you have a connection to somewhere irrelevant (Google in this case).
In particular, if you're trying to talk to a web service, and if you're in control of the web service, it would be a good idea to have some sort of cheap "get the status" web method. That way you have a much better idea of whether your "real" call is likely to work.
In other cases, just opening a connection to a port that should be open may be enough - or sending a ping. InetAddress.isReachable
may well be an appropriate API for your needs here.