How to download a file using a Java REST service and a data stream

Question

I have 3 machines   server where the file is located server where REST service is running   Jersey   client browser  with access to 2nd server but no access to 1st server   How can I directly  without saving the file on 2nd server  download the file from 1st server to client s machine   From 2nd server I can get a ByteArrayOutputStream to get the file from 1st server  can I pass this stream further to the client using the REST service  It will work this way  So basically what I want to achieve is to allow the client to download a file from 1st server using the REST service on 2nd server  since there is no direct access from client to 1st server  using only data streams  so no data touching the file system of 2nd server   What I try now with EasyStream library  final FTDClient client   FTDClient getInstance     try       final InputStreamFromOutputStream  lt String gt  isOs   new InputStreamFromOutputStream  lt String gt               Override         public String produce final OutputStream dataSink  throws Exception               return client downloadFile2 location  Integer valueOf spaceId   URLDecoder decode filePath   quot UTF-8 quot    dataSink                        try           String fileName   filePath substring filePath lastIndexOf  quot   quot     1            StreamingOutput output   new StreamingOutput                  Override             public void write OutputStream outputStream  throws IOException  WebApplicationException                   int length                  byte   buffer   new byte 1024                   while   length   isOs read buffer      -1                        outputStream write buffer  0  length                                     outputStream flush                                     return Response ok output  MediaType APPLICATION OCTET STREAM               header  quot Content-Disposition quot    quot attachment  filename   quot  quot    fileName    quot   quot  quot                build             UPDATE2 So my code now with the custom MessageBodyWriter looks simple  ByteArrayOutputStream baos   new ByteArrayOutputStream 2048    client downloadFile location  spaceId  filePath  baos   return Response ok baos  build     But I get the same heap error when trying with large files  UPDATE3 Finally managed to get it working   StreamingOutput did the trick  Thank you  peeskillet   Many thanks

User · Answer

See example here  Input and Output binary streams using JERSEY   Pseudo code would be something like this  there are a few other similar options in above mentioned post     Path  file     GET  Produces   application pdf    public StreamingOutput getFileContent   throws Exception        public void write OutputStream output  throws IOException  WebApplicationException           try                             1  Get Stream to file from first server                        while  lt read stream from first server gt                   output write  lt bytes read from first server gt                         catch  Exception e                throw new WebApplicationException e             finally                    close input stream

User · Answer

Refer this    RequestMapping value  download   method RequestMethod GET  public void getDownload HttpServletResponse response        Get your file stream from wherever  InputStream myStream   someClass returnFile        Set the content type and attachment header  response addHeader  Content-disposition    attachment filename myfilename txt    response setContentType  txt plain        Copy the stream to the response s output stream  IOUtils copy myStream  response getOutputStream     response flushBuffer        Details at  https   twilblog github io java spring rest file stream 2015 08 14 return-a-file-stream-from-spring-rest html

User · Answer

How can I directly  without saving the file on 2nd server  download the file from 1st server to client s machine     Just use the Client API and get the InputStream from the response  Client client   ClientBuilder newClient    String url          final InputStream responseStream   client target url  request   get InputStream class     There are two flavors to get the InputStream  You can also use   Response response   client target url  request   get    InputStream is    InputStream response getEntity      Which one is the more efficient  I m not sure  but the returned InputStreams are different classes  so you may want to look into that if you care to      From 2nd server I can get a ByteArrayOutputStream to get the file from 1st server  can I pass this stream further to the client using the REST service    So most of the answers you ll see in the link provided by  GradyGCooper seem to favor the use of StreamingOutput  An example implementation might be something like  final InputStream responseStream   client target url  request   get InputStream class   System out println responseStream getClass     StreamingOutput output   new StreamingOutput          Override     public void write OutputStream out  throws IOException  WebApplicationException             int length          byte   buffer   new byte 1024           while  length   responseStream read buffer      -1                out write buffer  0  length                     out flush            responseStream close                return Response ok output  header           Content-Disposition    attachment  filename           build      But if we look at the source code for StreamingOutputProvider  you ll see in the writeTo  that it simply writes the data from one stream to another  So with our implementation above  we have to write twice    How can we get only one write  Simple return the InputStream as the Response  final InputStream responseStream   client target url  request   get InputStream class   return Response ok responseStream  header           Content-Disposition    attachment  filename           build      If we look at the source code for InputStreamProvider  it simply delegates to ReadWriter writeTo in  out   which simply does what we did above in the StreamingOutput implementation   public static void writeTo InputStream in  OutputStream out  throws IOException       int read      final byte   data   new byte BUFFER SIZE       while   read   in read data      -1            out write data  0  read             Asides    Client objects are expensive resources  You may want to reuse the same Client for request  You can extract a WebTarget from the client for each request   WebTarget target   client target url   InputStream is   target request   get InputStream class     I think the WebTarget can even be shared  I can t find anything in the Jersey 2 x documentation  only because it is a larger document  and I m too lazy to scan through it right now  -   but in the Jersey 1 x documentation  it says the Client and WebResource  which is equivalent to WebTarget in 2 x  can be shared between threads  So I m guessing Jersey 2 x would be the same  but you may want to confirm for yourself  You don t have to make use of the Client API  A download can be easily achieved with the java net package APIs  But since you re already using Jersey  it doesn t hurt to use its APIs The above is assuming Jersey 2 x  For Jersey 1 x  a simple Google search should get you a bunch of hits for working with the API  or the documentation I linked to above      UPDATE  I m such a dufus  While the OP and I are contemplating ways to turn a ByteArrayOutputStream to an InputStream  I missed the simplest solution  which is simply to write a MessageBodyWriter for the ByteArrayOutputStream  import java io ByteArrayOutputStream  import java io IOException  import java io OutputStream  import java lang annotation Annotation  import java lang reflect Type  import javax ws rs WebApplicationException  import javax ws rs core MediaType  import javax ws rs core MultivaluedMap  import javax ws rs ext MessageBodyWriter  import javax ws rs ext Provider    Provider public class OutputStreamWriter implements MessageBodyWriter lt ByteArrayOutputStream gt          Override     public boolean isWriteable Class lt   gt  type  Type genericType              Annotation   annotations  MediaType mediaType            return ByteArrayOutputStream class    type              Override     public long getSize ByteArrayOutputStream t  Class lt   gt  type  Type genericType              Annotation   annotations  MediaType mediaType            return -1              Override     public void writeTo ByteArrayOutputStream t  Class lt   gt  type  Type genericType              Annotation   annotations  MediaType mediaType              MultivaluedMap lt String  Object gt  httpHeaders  OutputStream entityStream              throws IOException  WebApplicationException           t writeTo entityStream             Then we can simply return the ByteArrayOutputStream in the response  return Response ok baos  build      D OH   UPDATE 2  Here are the tests I used    Resource class   Path  test   public class TestResource        final String path    some 150 mb file         GET      Produces MediaType APPLICATION OCTET STREAM      public Response doTest   throws Exception           InputStream is   new FileInputStream path           ByteArrayOutputStream baos   new ByteArrayOutputStream            int len          byte   buffer   new byte 4096           while   len   is read buffer  0  buffer length      -1                baos write buffer  0  len                     System out println  Server size      baos size             return Response ok baos  build              Client test  public class Main       public static void main String   args  throws Exception           Client client   ClientBuilder newClient            String url    http   localhost 8080 api test           Response response   client target url  request   get            String location    some location           FileOutputStream out   new FileOutputStream location           InputStream is    InputStream response getEntity            int len   0          byte   buffer   new byte 4096           while  len   is read buffer      -1                out write buffer  0  len                     out flush            out close            is close              UPDATE 3  So the final solution for this particular use case was for the OP to simply pass the OutputStream from the StreamingOutput s write method  Seems the third-party API  required a OutputStream as an argument   StreamingOutput output   new StreamingOutput          Override     public void write OutputStream out            thirdPartyApi downloadFile             out           return Response ok output  build      Not quite sure  but seems the reading writing within the resource method  using ByteArrayOutputStream   realized something into memory   The point of the downloadFile method accepting an OutputStream is so that it can write the result directly to the OutputStream provided  For instance a FileOutputStream  if you wrote it to file  while the download is coming in  it would get directly streamed to the file    It s not meant for us to keep a reference to the OutputStream  as you were trying to do with the baos  which is where the memory realization comes in   So with the way that works  we are writing directly to the response stream provided for us  The method write doesn t actually get called until the writeTo method  in the MessageBodyWriter   where the OutputStream is passed to it   You can get a better picture looking at the MessageBodyWriter I wrote  Basically in the writeTo method  replace the ByteArrayOutputStream with StreamingOutput  then inside the method  call streamingOutput write entityStream   You can see the link I provided in the earlier part of the answer  where I link to the StreamingOutputProvider  This is exactly what happens

[java] How to download a file using a Java REST service and a data stream

Examples related to java

Examples related to rest

Examples related to download

Examples related to jersey

Examples related to bytestream