Read url to string in few lines of java code

Question

I m trying to find Java s equivalent to Groovy s   String content    http   www google com  toURL   getText      I want to read content from a URL into string   I don t want to pollute my code with buffered streams and loops for such a simple task   I looked into apache s HttpClient but I also don t see a one or two line implementation

User · Answer

Here s Jeanne s lovely answer  but wrapped in a tidy function for muppets like me   private static String getUrl String aUrl  throws MalformedURLException  IOException       String urlData           URL urlObj   new URL aUrl       URLConnection conn   urlObj openConnection        try  BufferedReader reader   new BufferedReader new InputStreamReader conn getInputStream    StandardCharsets UTF 8                   urlData   reader lines   collect Collectors joining   n               return urlData

User · Answer

If your url is of type java net URL  then you can just convert it to uri and then to string url toURI   toString

User · Answer

URL to String in pure Java  Example call   String str   getStringFromUrl  YourUrl      Implementation  You can use the method described in this answer  on How to read URL to an InputStream and combine it with this answer on How to read InputStream to String   The outcome will be something like  public String getStringFromUrl URL url  throws IOException           return inputStreamToString urlToInputStream url null       public String inputStreamToString InputStream inputStream  throws IOException       try ByteArrayOutputStream result   new ByteArrayOutputStream              byte   buffer   new byte 1024           int length          while   length   inputStream read buffer      -1                result write buffer  0  length                      return result toString UTF 8            private InputStream urlToInputStream URL url  Map lt String  String gt  args        HttpURLConnection con   null      InputStream inputStream   null      try           con    HttpURLConnection  url openConnection            con setConnectTimeout 15000           con setReadTimeout 15000           if  args    null                for  Entry lt String  String gt  e   args entrySet                      con setRequestProperty e getKey    e getValue                                     con connect            int responseCode   con getResponseCode               By default the connection will follow redirects  The following            block is only entered if the implementation of HttpURLConnection            does not perform the redirect  The exact behavior depends to             the actual implementation  e g  sun net                  Attention  This block allows the connection to             switch protocols  e g  HTTP to HTTPS   which is  lt b gt not lt  b gt              default behavior  See  https   stackoverflow com questions 1884230             for more info                        if  responseCode  lt  400  amp  amp  responseCode  gt  299                String redirectUrl   con getHeaderField  Location                try                   URL newUrl   new URL redirectUrl                   return urlToInputStream newUrl  args                 catch  MalformedURLException e                    URL newUrl   new URL url getProtocol             url getHost     redirectUrl                   return urlToInputStream newUrl  args                                                      inputStream   con getInputStream            return inputStream        catch  Exception e            throw new RuntimeException e             Pros   It is pure java It can be easily enhanced by adding different headers  instead of passing a null object  like the example above does   authentication  etc  Handling of protocol switches is supported

User · Answer

There s an even better way as of Java 9   URL u   new URL  http   www example com     try  InputStream in   u openStream          return new String in readAllBytes    StandardCharsets UTF 8       Like the original groovy example  this assumes that the content is UTF-8 encoded   If you need something more clever than that  you need to create a URLConnection and use it to figure out the encoding

User · Answer

Or just use Apache Commons IOUtils toString URL url   or the variant that also accepts an encoding parameter

User · Answer

Now that some time has passed since the original answer was accepted  there s a better approach    String out   new Scanner new URL  http   www google com   openStream     UTF-8   useDelimiter    A   next      If you want a slightly fuller implementation  which is not a single line  do this   public static String readStringFromURL String requestURL  throws IOException       try  Scanner scanner   new Scanner new URL requestURL  openStream                StandardCharsets UTF 8 toString                   scanner useDelimiter    A            return scanner hasNext     scanner next

User · Answer

Now that more time has passed  here s a way to do it in Java 8   URLConnection conn   url openConnection    try  BufferedReader reader   new BufferedReader new InputStreamReader conn getInputStream    StandardCharsets UTF 8          pageText   reader lines   collect Collectors joining   n

User · Answer

The following works with Java 7 8  secure urls  and shows how to add a cookie to your request as well  Note this is mostly a direct copy of this other great answer on this page  but added the cookie example  and clarification in that it works with secure urls as well  -    If you need to connect to a server with an invalid certificate or self signed certificate  this will throw security errors unless you import the certificate  If you need this functionality  you could consider the approach detailed in this answer to this related question on StackOverflow   Example  String result   getUrlAsString  https   www google com    System out println result     outputs   lt  doctype html gt  lt html itemscope         etc   Code  import java net URL  import java net URLConnection  import java io BufferedReader  import java io InputStreamReader   public static String getUrlAsString String url        try               URL urlObj   new URL url           URLConnection con   urlObj openConnection             con setDoOutput true      we want the response          con setRequestProperty  Cookie    myCookie test123            con connect             BufferedReader in   new BufferedReader new InputStreamReader con getInputStream               StringBuilder response   new StringBuilder            String inputLine           String newLine   System getProperty  line separator            while   inputLine   in readLine       null                        response append inputLine   newLine                      in close             return response toString              catch  Exception e                throw new RuntimeException e

User · Answer

Java 11   URI uri   URI create  quot http   www google com quot    HttpRequest request   HttpRequest newBuilder uri  build    String content   HttpClient newHttpClient   send request  BodyHandlers ofString    body

User · Answer

This answer refers to an older version of Java  You may want to look at ccleve s answer      Here is the traditional way to do this   import java net    import java io     public class URLConnectionReader       public static String getText String url  throws Exception           URL website   new URL url           URLConnection connection   website openConnection            BufferedReader in   new BufferedReader                                  new InputStreamReader                                      connection getInputStream               StringBuilder response   new StringBuilder            String inputLine           while   inputLine   in readLine       null               response append inputLine            in close             return response toString               public static void main String   args  throws Exception           String content   URLConnectionReader getText args 0            System out println content             As  extraneon has suggested  ioutils allows you to do this in a very eloquent way that s still in the Java spirit    InputStream in   new URL   http   jakarta apache org    openStream      try      System out println  IOUtils toString  in         finally      IOUtils closeQuietly in

User · Answer

If you have the input stream  see Joe s answer  also consider ioutils toString  inputstream     http   commons apache org io api-1 4 org apache commons io IOUtils html toString java io InputStream

User · Answer

Additional example using Guava   URL xmlData       String data   Resources toString xmlData  Charsets UTF 8

[java] Read url to string in few lines of java code

Examples related to java

Examples related to http

Examples related to url