How to convert OutputStream to InputStream

Question

I am on the stage of development  where I have two modules and from one I got output as a OutputStream and second one  which accepts only InputStream  Do you know how to convert OutputStream to InputStream  not vice versa  I mean really this way  that I will be able to connect these two parts   Thanks

User · Answer

The easystream open source library has direct support to convert an OutputStream to an InputStream: http://io-tools.sourceforge.net/easystream/tutorial/tutorial.html

// create conversion
final OutputStreamToInputStream<Void> out = new OutputStreamToInputStream<Void>() {
    @Override
    protected Void doRead(final InputStream in) throws Exception {
           LibraryClass2.processDataFromInputStream(in);
           return null;
        }
    };
try {   
     LibraryClass1.writeDataToTheOutputStream(out);
} finally {
     // don't miss the close (or a thread would not terminate correctly).
     out.close();
}

They also list other options: http://io-tools.sourceforge.net/easystream/outputstream_to_inputstream/implementations.html

Write the data the data into a memory buffer (ByteArrayOutputStream) get the byteArray and read it again with a ByteArrayInputStream. This is the best approach if you're sure your data fits into memory.
Copy your data to a temporary file and read it back.
Use pipes: this is the best approach both for memory usage and speed (you can take full advantage of the multi-core processors) and also the standard solution offered by Sun.
Use InputStreamFromOutputStream and OutputStreamToInputStream from the easystream library.

User · Answer

As input and output streams are just start and end point  the solution is to temporary store data in byte array  So you must create intermediate ByteArrayOutputStream  from which you create byte   that is used as input for new ByteArrayInputStream    public void doTwoThingsWithStream InputStream inStream  OutputStream outStream        create temporary bayte array output stream   ByteArrayOutputStream baos   new ByteArrayOutputStream      doFirstThing inStream  baos       create input stream from baos   InputStream isFromFirstData   new ByteArrayInputStream baos toByteArray        doSecondThing isFromFirstData  outStream       Hope it helps

User · Answer

If you want to make an OutputStream from an InputStream there is one basic problem  A method writing to an OutputStream blocks until it is done  So the result is available when the writing method is finished  This has 2 consequences    If you use only one thread  you need to wait until everything is written  so you need to store the stream s data in memory or disk   If you want to access the data before it is finished  you need a second thread    Variant 1 can be implemented using byte arrays or filed  Variant 1 can be implemented using pipies  either directly or with extra abstraction - e g  RingBuffer or the google lib from the other comment    Indeed with standard java there is no other way to solve the problem  Each solution is an implementataion of one of these   There is one concept called  continuation   see wikipedia for details   In this case basically this means    there is a special output stream that expects a certain amount of data if the ammount is reached  the stream gives control to it s counterpart which is a special input stream the input stream makes the amount of data available until it is read  after that  it passes back the control to the output stream   While some languages have this concept built in  for java you need some  magic   For example  commons-javaflow  from apache implements such for java  The disadvantage is that this requires some special bytecode modifications at build time  So it would make sense to put all the stuff in an extra library whith custom build scripts

User · Answer

Old post but might help others  Use this way   OutputStream out   new ByteArrayOutputStream        out write        ObjectInputStream ois   new ObjectInputStream new ByteArrayInputStream out toString   getBytes

User · Answer

ByteArrayOutputStream buffer    ByteArrayOutputStream  aOutputStream  byte   bytes   buffer toByteArray    InputStream inputStream   new ByteArrayInputStream bytes

User · Answer

An OutputStream is one where you write data to  If some module exposes an OutputStream  the expectation is that there is something reading at the other end   Something that exposes an InputStream  on the other hand  is indicating that you will need to listen to this stream  and there will be data that you can read    So it is possible to connect an InputStream to an OutputStream  InputStream----read--- gt  intermediateBytes n  ----write---- gt  OutputStream  As someone metioned  this is what the copy   method from IOUtils lets you do  It does not make sense to go the other way    hopefully this makes some sense  UPDATE   Of course the more I think of this  the more I can see how this actually would be a requirement  I know some of the comments mentioned Piped input ouput streams  but there is another possibility   If the output stream that is exposed is a ByteArrayOutputStream  then you can always get the full contents by calling the toByteArray   method  Then you can create an input stream wrapper by using the ByteArrayInputStream sub-class  These two are pseudo-streams  they both basically just wrap an array of bytes  Using the streams this way  therefore  is technically possible  but to me it is still very strange

User · Answer

There seem to be many links and other such stuff  but no actual code using pipes  The advantage of using java io PipedInputStream and java io PipedOutputStream is that there is no additional consumption of memory  ByteArrayOutputStream toByteArray   returns a copy of the original buffer  so that means that whatever you have in memory  you now have two copies of it  Then writing to an InputStream means you now have three copies of the data   The code      take the copy of the stream and re-write it to an InputStream PipedInputStream in   new PipedInputStream    final PipedOutputStream out   new PipedOutputStream in   new Thread new Runnable         public void run              try                  write the original OutputStream to the PipedOutputStream                note that in order for the below method to work  you need                to ensure that the data has finished writing to the                ByteArrayOutputStream             originalByteArrayOutputStream writeTo out                     catch  IOException e                   logging and exception handling should go here                   finally                  close the PipedOutputStream here because we re done writing data                once this thread has completed its run             if  out    null                       close the PipedOutputStream cleanly                 out close                                        start      This code assumes that the originalByteArrayOutputStream is a ByteArrayOutputStream as it is usually the only usable output stream  unless you re writing to a file  I hope this helps  The great thing about this is that since it s in a separate thread  it also is working in parallel  so whatever is consuming your input stream will be streaming out of your old output stream too  That is beneficial because the buffer can remain smaller and you ll have less latency and less memory usage

User · Answer

The library io-extras may be useful  For example if you want to gzip an InputStream using GZIPOutputStream and you want it to happen synchronously  using the default buffer size of 8192    InputStream is       InputStream gz   IOUtil pipe is  o - gt  new GZIPOutputStream o      Note that the library has 100  unit test coverage  for what that s worth of course   and is on Maven Central  The Maven dependency is    lt dependency gt     lt groupId gt com github davidmoten lt  groupId gt     lt artifactId gt io-extras lt  artifactId gt     lt version gt 0 1 lt  version gt   lt  dependency gt    Be sure to check for a later version

User · Answer

I encountered the same problem with converting a ByteArrayOutputStream to a ByteArrayInputStream and solved it by using a derived class from ByteArrayOutputStream which is able to return a ByteArrayInputStream that is initialized with the internal buffer of the ByteArrayOutputStream  This way no additional memory is used and the  conversion  is very fast   package info whitebyte utils  import java io ByteArrayInputStream  import java io ByteArrayOutputStream          This class extends the ByteArrayOutputStream by     providing a method that returns a new ByteArrayInputStream    which uses the internal byte array buffer  This buffer    is not copied  so no additional memory is used  After    creating the ByteArrayInputStream the instance of the    ByteArrayInOutStream can not be used anymore      lt p gt     The ByteArrayInputStream can be retrieved using  lt code gt getInputStream   lt  code gt       author Nick Russler     public class ByteArrayInOutStream extends ByteArrayOutputStream                  Creates a new ByteArrayInOutStream  The buffer capacity is        initially 32 bytes  though its size increases if necessary              public ByteArrayInOutStream             super                          Creates a new ByteArrayInOutStream  with a buffer capacity of        the specified size  in bytes                 param   size   the initial size          exception  IllegalArgumentException if size is negative              public ByteArrayInOutStream int size            super size                         Creates a new ByteArrayInputStream that uses the internal byte array buffer         of this ByteArrayInOutStream instance as its buffer array  The initial value         of pos is set to zero and the initial value of count is the number of bytes         that can be read from the byte array  The buffer array is not copied  This         instance of ByteArrayInOutStream can not be used anymore after calling this        method          return the ByteArrayInputStream instance             public ByteArrayInputStream getInputStream                create new ByteArrayInputStream that respects the current count         ByteArrayInputStream in   new ByteArrayInputStream this buf  0  this count               set the buffer of the ByteArrayOutputStream             to null so it can t be altered anymore         this buf   null           return in            I put the stuff on github  https   github com nickrussler ByteArrayInOutStream

User · Answer

From my point of view  java io PipedInputStream java io PipedOutputStream is the best option to considere  In some situations you may want to use ByteArrayInputStream ByteArrayOutputStream  The problem is that you need to duplicate the buffer to convert a ByteArrayOutputStream to a ByteArrayInputStream  Also ByteArrayOutpuStream ByteArrayInputStream are limited to 2GB  Here is an OutpuStream InputStream implementation I wrote to bypass ByteArrayOutputStream ByteArrayInputStream limitations  Scala code  but easily understandable for java developpers    import java io  IOException  InputStream  OutputStream   import scala annotation tailrec      Acts as a replacement for ByteArrayOutputStream          class HugeMemoryOutputStream capacity  Long  extends OutputStream     private val PAGE SIZE  Int   1024000   private val ALLOC STEP  Int   1024        Pages array                private var streamBuffers  Array Array Byte     Array empty Array Byte          Allocated pages count                private var pageCount  Int   0        Allocated bytes count                private var allocatedBytes  Long   0        Current position in stream                private var position  Long   0        Stream length                private var length  Long   0    allocSpaceIfNeeded capacity         Gets page count based on given length              param length   Buffer length        return         Page count to hold the specified amount of data          private def getPageCount length  Long          var pageCount    length   PAGE SIZE  toInt   1      if   length   PAGE SIZE     0          pageCount -  1            pageCount            Extends pages array                private def extendPages    Unit         if  streamBuffers isEmpty          streamBuffers   new Array Array Byte   ALLOC STEP            else         val newStreamBuffers   new Array Array Byte   streamBuffers length   ALLOC STEP        Array copy streamBuffers  0  newStreamBuffers  0  streamBuffers length        streamBuffers   newStreamBuffers            pageCount   streamBuffers length            Ensures buffers are bug enough to hold specified amount of data              param value  Amount of data          private def allocSpaceIfNeeded value  Long   Unit          tailrec     def allocSpaceIfNeededIter value  Long   Unit           val currentPageCount   getPageCount allocatedBytes        val neededPageCount   getPageCount value         if  currentPageCount  lt  neededPageCount            if  currentPageCount    pageCount  extendPages            streamBuffers currentPageCount    new Array Byte  PAGE SIZE          allocatedBytes    currentPageCount   1  toLong   PAGE SIZE          allocSpaceIfNeededIter value                     if  value  lt  0  throw new Error  AllocSpaceIfNeeded  lt  0       if  value  gt  0          allocSpaceIfNeededIter value         length   Math max value  length        if  position  gt  length  position   length                        Writes the specified byte to this output stream  The general       contract for  lt code gt write lt  code gt  is that one byte is written       to the output stream  The byte to be written is the eight       low-order bits of the argument  lt code gt b lt  code gt   The 24       high-order bits of  lt code gt b lt  code gt  are ignored         lt p gt        Subclasses of  lt code gt OutputStream lt  code gt  must provide an       implementation for this method               param      b the  lt code gt byte lt  code gt             throws IOException    override def write b  Int   Unit         val buffer  Array Byte    new Array Byte  1       buffer 0    b toByte      write buffer                   Writes  lt code gt len lt  code gt  bytes from the specified byte array       starting at offset  lt code gt off lt  code gt  to this output stream        The general contract for  lt code gt write b  off  len  lt  code gt  is that       some of the bytes in the array  lt code gt b lt  code gt  are written to the       output stream in order  element  lt code gt b off  lt  code gt  is the first       byte written and  lt code gt b off len-1  lt  code gt  is the last byte written       by this operation         lt p gt        The  lt code gt write lt  code gt  method of  lt code gt OutputStream lt  code gt  calls       the write method of one argument on each of the bytes to be       written out  Subclasses are encouraged to override this method and       provide a more efficient implementation         lt p gt        If  lt code gt b lt  code gt  is  lt code gt null lt  code gt   a        lt code gt NullPointerException lt  code gt  is thrown         lt p gt        If  lt code gt off lt  code gt  is negative  or  lt code gt len lt  code gt  is negative  or        lt code gt off len lt  code gt  is greater than the length of the array        lt code gt b lt  code gt   then an  lt tt gt IndexOutOfBoundsException lt  tt gt  is thrown               param      b   the data         param      off the start offset in the data         param      len the number of bytes to write            throws IOException    override def write b  Array Byte   off  Int  len  Int   Unit          tailrec     def writeIter b  Array Byte   off  Int  len  Int   Unit           val currentPage  Int    position   PAGE SIZE  toInt       val currentOffset  Int    position   PAGE SIZE  toInt        if  len    0            val currentLength  Int   Math min PAGE SIZE - currentOffset  len          Array copy b  off  streamBuffers currentPage   currentOffset  currentLength           position    currentLength          writeIter b  off   currentLength  len - currentLength                     allocSpaceIfNeeded position   len      writeIter b  off  len             Gets an InputStream that points to HugeMemoryOutputStream buffer              return InputStream          def asInputStream    InputStream         new HugeMemoryInputStream streamBuffers  length         private class HugeMemoryInputStream streamBuffers  Array Array Byte    val length  Long  extends InputStream           Current position in stream                      private var position  Long   0                  Reads the next byte of data from the input stream  The value byte is         returned as an  lt code gt int lt  code gt  in the range  lt code gt 0 lt  code gt  to          lt code gt 255 lt  code gt   If no byte is available because the end of the stream         has been reached  the value  lt code gt -1 lt  code gt  is returned  This method         blocks until input data is available  the end of the stream is detected          or an exception is thrown                   lt p gt  A subclass must provide an implementation of this method                   return the next byte of data  or  lt code gt -1 lt  code gt  if the end of the                 stream is reached                throws IOException      def read  Int           val buffer  Array Byte    new Array Byte  1         if  read buffer     0  throw new Error  End of stream         else buffer 0                         Reads up to  lt code gt len lt  code gt  bytes of data from the input stream into         an array of bytes   An attempt is made to read as many as          lt code gt len lt  code gt  bytes  but a smaller number may be read          The number of bytes actually read is returned as an integer                   lt p gt  This method blocks until input data is available  end of file is         detected  or an exception is thrown                   lt p gt  If  lt code gt len lt  code gt  is zero  then no bytes are read and          lt code gt 0 lt  code gt  is returned  otherwise  there is an attempt to read at         least one byte  If no byte is available because the stream is at end of         file  the value  lt code gt -1 lt  code gt  is returned  otherwise  at least one         byte is read and stored into  lt code gt b lt  code gt                    lt p gt  The first byte read is stored into element  lt code gt b off  lt  code gt   the         next one into  lt code gt b off 1  lt  code gt   and so on  The number of bytes read         is  at most  equal to  lt code gt len lt  code gt   Let  lt i gt k lt  i gt  be the number of         bytes actually read  these bytes will be stored in elements          lt code gt b off  lt  code gt  through  lt code gt b off  lt  code gt  lt i gt k lt  i gt  lt code gt -1  lt  code gt           leaving elements  lt code gt b off  lt  code gt  lt i gt k lt  i gt  lt code gt   lt  code gt  through          lt code gt b off len-1  lt  code gt  unaffected                   lt p gt  In every case  elements  lt code gt b 0  lt  code gt  through          lt code gt b off  lt  code gt  and elements  lt code gt b off len  lt  code gt  through          lt code gt b b length-1  lt  code gt  are unaffected                   lt p gt  The  lt code gt read b  lt  code gt   lt code gt off  lt  code gt   lt code gt len  lt  code gt  method         for class  lt code gt InputStream lt  code gt  simply calls the method          lt code gt read   lt  code gt  repeatedly  If the first such call results in an          lt code gt IOException lt  code gt   that exception is returned from the call to         the  lt code gt read b  lt  code gt   lt code gt off  lt  code gt   lt code gt len  lt  code gt  method   If         any subsequent call to  lt code gt read   lt  code gt  results in a          lt code gt IOException lt  code gt   the exception is caught and treated as if it         were end of file  the bytes read up to that point are stored into          lt code gt b lt  code gt  and the number of bytes read before the exception         occurred is returned  The default implementation of this method blocks         until the requested amount of input data  lt code gt len lt  code gt  has been read          end of file is detected  or an exception is thrown  Subclasses are encouraged         to provide a more efficient implementation of this method                   param      b   the buffer into which the data is read           param      off the start offset in array  lt code gt b lt  code gt                          at which the data is written           param      len the maximum number of bytes to read           return the total number of bytes read into the buffer  or                  lt code gt -1 lt  code gt  if there is no more data because the end of                 the stream has been reached           see java io InputStream read                 throws IOException      override def read b  Array Byte   off  Int  len  Int   Int            tailrec       def readIter acc  Int  b  Array Byte   off  Int  len  Int   Int             val currentPage  Int    position   PAGE SIZE  toInt         val currentOffset  Int    position   PAGE SIZE  toInt          val count  Int   Math min len  length - position  toInt          if  count    0    position  gt   length  acc         else             val currentLength   Math min PAGE SIZE - currentOffset  count            Array copy streamBuffers currentPage   currentOffset  b  off  currentLength             position    currentLength            readIter acc   currentLength  b  off   currentLength  len - currentLength                           readIter 0  b  off  len                         Skips over and discards  lt code gt n lt  code gt  bytes of data from this input         stream  The  lt code gt skip lt  code gt  method may  for a variety of reasons  end         up skipping over some smaller number of bytes  possibly  lt code gt 0 lt  code gt           This may result from any of a number of conditions  reaching end of file         before  lt code gt n lt  code gt  bytes have been skipped is only one possibility          The actual number of bytes skipped is returned  If  lt code gt n lt  code gt  is         negative  the  lt code gt skip lt  code gt  method for class  lt code gt InputStream lt  code gt  always         returns 0  and no bytes are skipped  Subclasses may handle the negative         value differently                  The  lt code gt skip lt  code gt  method of this class creates a         byte array and then repeatedly reads into it until  lt code gt n lt  code gt  bytes         have been read or the end of the stream has been reached  Subclasses are         encouraged to provide a more efficient implementation of this method          For instance  the implementation may depend on the ability to seek                   param      n the number of bytes to be skipped           return the actual number of bytes skipped                throws IOException      override def skip n  Long   Long           if  n  lt  0  0       else           position   Math min position   n  length          length - position                       Easy to use  no buffer duplication  no 2GB memory limit  val out  HugeMemoryOutputStream   new HugeMemoryOutputStream initialCapacity   may be 0     out write           val in1  InputStream   out asInputStream    in1 read           val in2  InputStream   out asInputStream    in2 read

User · Answer

Though you cannot convert an OutputStream to an InputStream  java provides a way using PipedOutputStream and PipedInputStream that you can have data written to a  PipedOutputStream to become available through an associated PipedInputStream   Sometime back I faced a similar situation when dealing with third party libraries that required an InputStream instance to be passed to them instead of an OutputStream instance  The way I fixed this issue is to use the PipedInputStream and PipedOutputStream  By the way they are tricky to use and you must use multithreading to achieve what you want  I recently published an implementation on github which you can use  Here is the link   You can go through the wiki to understand how to use it

User · Answer

You will need an intermediate class which will buffer between  Each time InputStream read byte       is called  the buffering class will fill the passed in byte array with the next chunk passed in from OutputStream write byte        Since the sizes of the chunks may not be the same  the adapter class will need to store a certain amount until it has enough to fill the read buffer and or be able to store up any buffer overflow   This article has a nice breakdown of a few different approaches to this problem   http   blog ostermiller org convert-java-outputstream-inputstream

[java] How to convert OutputStream to InputStream?

Examples related to java

Examples related to inputstream

Examples related to outputstream