Java NIO FileChannel versus FileOutputstream performance usefulness

Question

I am trying to figure out if there is any difference in performance  or advantages  when we use nio FileChannel versus normal FileInputStream FileOuputStream to read and write files to filesystem  I observed that on my machine both perform at the same level  also many times the FileChannel way is slower  Can I please know more details comparing these two methods  Here is the code I used  the file that I am testing with is around 350MB  Is it a good option to use NIO based classes for File I O  if I am not looking at random access or other such advanced features   package trialjavaprograms   import java io File  import java io FileInputStream  import java io FileOutputStream  import java io InputStream  import java nio ByteBuffer  import java nio channels FileChannel   public class JavaNIOTest       public static void main String   args  throws Exception           useNormalIO            useFileChannel               private static void useNormalIO   throws Exception           File file   new File   home developer test iso            File oFile   new File   home developer test2             long time1   System currentTimeMillis            InputStream is   new FileInputStream file           FileOutputStream fos   new FileOutputStream oFile           byte   buf   new byte 64   1024           int len   0          while  len   is read buf      -1                fos write buf  0  len                     fos flush            fos close            is close            long time2   System currentTimeMillis            System out println  Time taken     time2-time1    ms               private static void useFileChannel   throws Exception           File file   new File   home developer test iso            File oFile   new File   home developer test2             long time1   System currentTimeMillis            FileInputStream is   new FileInputStream file           FileOutputStream fos   new FileOutputStream oFile           FileChannel f   is getChannel            FileChannel f2   fos getChannel             ByteBuffer buf   ByteBuffer allocateDirect 64   1024           long len   0          while  len   f read buf      -1                buf flip                f2 write buf               buf clear                       f2 close            f close             long time2   System currentTimeMillis            System out println  Time taken     time2-time1    ms

User · Answer

My experience is  that NIO is much faster with small files  But when it comes to large files FileInputStream FileOutputStream is much faster

User · Answer

I tested the performance of FileInputStream vs  FileChannel for decoding base64 encoded files  In my experients I tested rather large file and traditional io was alway a bit faster than nio   FileChannel might have had an advantage in prior versions of the jvm because of synchonization overhead in several io related classes  but modern jvm are pretty good at removing unneeded locks

User · Answer

Based on my tests  Win7 64bit  6GB RAM  Java6   NIO transferFrom is fast only with small files and becomes very slow on larger files  NIO databuffer flip always outperforms standard IO     Copying 1000x2MB   NIO  transferFrom   2300ms NIO  direct datababuffer 5000b flip   3500ms  Standard IO  buffer 5000b   6000ms  Copying 100x20mb   NIO  direct datababuffer 5000b flip   4000ms  NIO  transferFrom   5000ms Standard IO  buffer 5000b   6500ms  Copying 1x1000mb   NIO  direct datababuffer 5000b flip   4500s  Standard IO  buffer 5000b   7000ms NIO  transferFrom   8000ms    The transferTo   method works on chunks of a file  wasn t intended as a high-level file copy method  How to copy a large file in Windows XP

User · Answer

If you are not using the transferTo feature or non-blocking features you will not notice a difference between traditional IO and NIO 2  because the traditional IO maps to NIO    But if you can use the NIO features like transferFrom To or want to use Buffers  then of course NIO is the way to go

User · Answer

My experience with larger files sizes has been that java nio is faster than java io   Solidly faster   Like in the  250  range   That said  I am eliminating obvious bottlenecks  which I suggest your micro-benchmark might suffer from   Potential areas for investigating   The buffer size   The algorithm you basically have is   copy from disk to buffer copy from buffer to disk   My own experience has been that this buffer size is ripe for tuning   I ve settled on 4KB for one part of my application  256KB for another   I suspect your code is suffering with such a large buffer   Run some benchmarks with buffers of 1KB  2KB  4KB  8KB  16KB  32KB and 64KB to prove it to yourself   Don t perform java benchmarks that read and write to the same disk   If you do  then you are really benchmarking the disk  and not Java   I would also suggest that if your CPU is not busy  then you are probably experiencing some other bottleneck   Don t use a buffer if you don t need to   Why copy to memory if your target is another disk or a NIC   With larger files  the latency incured is non-trivial     Like other have said  use FileChannel transferTo   or FileChannel transferFrom     The key advantage here is that the JVM uses the OS s access to DMA  Direct Memory Access   if present   This is implementation dependent  but modern Sun and IBM versions on general purpose CPUs are good to go    What happens is the data goes straight to from disc  to the bus  and then to the destination    bypassing any circuit through RAM or the CPU   The web app I spent my days and night working on is very IO heavy   I ve done micro benchmarks and real-world benchmarks too   And the results are up on my blog  have a look-see    Real world performance metrics  java io vs  java nio   Real world performance metrics  java io vs  java nio  The Sequel      Use production data and environments  Micro-benchmarks are prone to distortion   If you can  make the effort to gather data from exactly what you plan to do  with the load you expect  on the hardware you expect   My benchmarks are solid and reliable because they took place on a production system  a beefy system  a system under load  gathered in logs   Not my notebook s 7200 RPM 2 5  SATA drive while I watched intensely as the JVM work my hard disc     What are you running on   It matters

User · Answer

Answering the  usefulness  part of the question   One rather subtle gotcha of using FileChannel over FileOutputStream is that performing any of its blocking operations  e g  read   or write    from a thread that s in interrupted state will cause the channel to close abruptly with java nio channels ClosedByInterruptException   Now  this could be a good thing if whatever the FileChannel was used for is part of the thread s main function  and design took this into account   But it could also be pesky if used by some auxiliary feature such as a logging function   For example  you can find your logging output suddenly closed if the logging function happens to be called by a thread that s also interrupted   It s unfortunate this is so subtle because not accounting for this can lead to bugs that affect write integrity  1  2     1  https   bugs openjdk java net browse JDK-4469683  2  http   bugs java com bugdatabase view bug do bug id 6608965

User · Answer

If the thing you want to compare is performance of file copying  then for the channel test you should do this instead   final FileInputStream inputStream   new FileInputStream src   final FileOutputStream outputStream   new FileOutputStream dest   final FileChannel inChannel   inputStream getChannel    final FileChannel outChannel   outputStream getChannel    inChannel transferTo 0  inChannel size    outChannel   inChannel close    outChannel close    inputStream close    outputStream close      This won t be slower than buffering yourself from one channel to the other  and will potentially be massively faster  According to the Javadocs      Many operating systems can transfer bytes directly from the filesystem cache to the target channel without actually copying them

[java] Java NIO FileChannel versus FileOutputstream performance / usefulness

Examples related to java

Examples related to optimization

Examples related to file

Examples related to nio

Examples related to operations