[java] Android Reading from an Input stream efficiently

I am making an HTTP get request to a website for an android application I am making.

I am using a DefaultHttpClient and using HttpGet to issue the request. I get the entity response and from this obtain an InputStream object for getting the html of the page.

I then cycle through the reply doing as follows:

BufferedReader r = new BufferedReader(new InputStreamReader(inputStream));
String x = "";
x = r.readLine();
String total = "";

while(x!= null){
total += x;
x = r.readLine();
}

However this is horrendously slow.

Is this inefficient? I'm not loading a big web page - www.cokezone.co.uk so the file size is not big. Is there a better way to do this?

Thanks

Andy

To convert the InputStream to String we use the BufferedReader.readLine() method. We iterate until the BufferedReader return null which means there's no more data to read. Each line will appended to a StringBuilder and returned as String.

 public static String convertStreamToString(InputStream is) {

        BufferedReader reader = new BufferedReader(new InputStreamReader(is));
        StringBuilder sb = new StringBuilder();

        String line = null;
        try {
            while ((line = reader.readLine()) != null) {
                sb.append(line + "\n");
            }
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            try {
                is.close();
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
        return sb.toString();
    }
}`

And finally from any class where you want to convert call the function

String dataString = Utils.convertStreamToString(in);

complete

If the file is long, you can optimize your code by appending to a StringBuilder instead of using a String concatenation for each line.

What about this. Seems to give better performance.

byte[] bytes = new byte[1000];

StringBuilder x = new StringBuilder();

int numRead = 0;
while ((numRead = is.read(bytes)) >= 0) {
    x.append(new String(bytes, 0, numRead));
}

Edit: Actually this sort of encompasses both steelbytes and Maurice Perry's

Maybe rather then read 'one line at a time' and join the strings, try 'read all available' so as to avoid the scanning for end of line, and to also avoid string joins.

ie, InputStream.available() and InputStream.read(byte[] b), int offset, int length)

I am use to read full data:

// inputStream is one instance InputStream
byte[] data = new byte[inputStream.available()];
inputStream.read(data);
String dataString = new String(data);

Another possibility with Guava:

dependency: compile 'com.google.guava:guava:11.0.2'

import com.google.common.io.ByteStreams;
...

String total = new String(ByteStreams.toByteArray(inputStream ));

Have you tried the built in method to convert a stream to a string? It's part of the Apache Commons library (org.apache.commons.io.IOUtils).

Then your code would be this one line:

String total = IOUtils.toString(inputStream);

The documentation for it can be found here: http://commons.apache.org/io/api-1.4/org/apache/commons/io/IOUtils.html#toString%28java.io.InputStream%29

The Apache Commons IO library can be downloaded from here: http://commons.apache.org/io/download_io.cgi

Possibly somewhat faster than Jaime Soriano's answer, and without the multi-byte encoding problems of Adrian's answer, I suggest:

File file = new File("/tmp/myfile");
try {
    FileInputStream stream = new FileInputStream(file);

    int count;
    byte[] buffer = new byte[1024];
    ByteArrayOutputStream byteStream =
        new ByteArrayOutputStream(stream.available());

    while (true) {
        count = stream.read(buffer);
        if (count <= 0)
            break;
        byteStream.write(buffer, 0, count);
    }

    String string = byteStream.toString();
    System.out.format("%d bytes: \"%s\"%n", string.length(), string);
} catch (IOException e) {
    e.printStackTrace();
}

Reading one line of text at a time, and appending said line to a string individually is time-consuming both in extracting each line and the overhead of so many method invocations.

I was able to get better performance by allocating a decent-sized byte array to hold the stream data, and which is iteratively replaced with a larger array when needed, and trying to read as much as the array could hold.

For some reason, Android repeatedly failed to download the entire file when the code used the InputStream returned by HTTPUrlConnection, so I had to resort to using both a BufferedReader and a hand-rolled timeout mechanism to ensure I would either get the whole file or cancel the transfer.

private static  final   int         kBufferExpansionSize        = 32 * 1024;
private static  final   int         kBufferInitialSize          = kBufferExpansionSize;
private static  final   int         kMillisecondsFactor         = 1000;
private static  final   int         kNetworkActionPeriod        = 12 * kMillisecondsFactor;

private String loadContentsOfReader(Reader aReader)
{
    BufferedReader  br = null;
    char[]          array = new char[kBufferInitialSize];
    int             bytesRead;
    int             totalLength = 0;
    String          resourceContent = "";
    long            stopTime;
    long            nowTime;

    try
    {
        br = new BufferedReader(aReader);

        nowTime = System.nanoTime();
        stopTime = nowTime + ((long)kNetworkActionPeriod * kMillisecondsFactor * kMillisecondsFactor);
        while(((bytesRead = br.read(array, totalLength, array.length - totalLength)) != -1)
        && (nowTime < stopTime))
        {
            totalLength += bytesRead;
            if(totalLength == array.length)
                array = Arrays.copyOf(array, array.length + kBufferExpansionSize);
            nowTime = System.nanoTime();
        }

        if(bytesRead == -1)
            resourceContent = new String(array, 0, totalLength);
    }
    catch(Exception e)
    {
        e.printStackTrace();
    }

    try
    {
        if(br != null)
            br.close();
    }
    catch(IOException e)
    {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
}

EDIT: It turns out that if you don't need to have the content re-encoded (ie, you want the content AS IS) you shouldn't use any of the Reader subclasses. Just use the appropriate Stream subclass.

Replace the beginning of the preceding method with the corresponding lines of the following to speed it up an extra 2 to 3 times.

String  loadContentsFromStream(Stream aStream)
{
    BufferedInputStream br = null;
    byte[]              array;
    int                 bytesRead;
    int                 totalLength = 0;
    String              resourceContent;
    long                stopTime;
    long                nowTime;

    resourceContent = "";
    try
    {
        br = new BufferedInputStream(aStream);
        array = new byte[kBufferInitialSize];

I believe this is efficient enough... To get a String from an InputStream, I'd call the following method:

public static String getStringFromInputStream(InputStream stream) throws IOException
{
    int n = 0;
    char[] buffer = new char[1024 * 4];
    InputStreamReader reader = new InputStreamReader(stream, "UTF8");
    StringWriter writer = new StringWriter();
    while (-1 != (n = reader.read(buffer))) writer.write(buffer, 0, n);
    return writer.toString();
}

I always use UTF-8. You could, of course, set charset as an argument, besides InputStream.

    byte[] buffer = new byte[1024];  // buffer store for the stream
    int bytes; // bytes returned from read()

    // Keep listening to the InputStream until an exception occurs
    while (true) {
        try {
            // Read from the InputStream
            bytes = mmInStream.read(buffer);

            String TOKEN_ = new String(buffer, "UTF-8");

            String xx = TOKEN_.substring(0, bytes);