Executors newCachedThreadPool versus Executors newFixedThreadPool

Question

newCachedThreadPool   versus newFixedThreadPool    When should I use one or the other  Which strategy is better in terms of resource utilization

User · Answer

Just to complete the other answers  I would like to quote Effective Java  2nd Edition  by Joshua Bloch  chapter 10  Item 68     quot Choosing the executor service for a particular application can be tricky  If you   re writing a small program  or a lightly loaded server  using Executors new- CachedThreadPool is generally a good choice  as it demands no configuration and generally    does the right thing     But a cached thread pool is not a good choice for a heavily loaded production server  In a cached thread pool  submitted tasks are not queued but immediately handed off to a thread for execution  If no threads are available  a new one is created  If a server is so heavily loaded that all of its CPUs are fully utilized  and more tasks arrive  more threads will be created  which will only make matters worse  Therefore  in a heavily loaded production server  you are much better off using Executors newFixedThreadPool  which gives you a pool with a fixed number of threads  or using the ThreadPoolExecutor class directly  for maximum control  quot

User · Answer

You must use newCachedThreadPool only when you have short-lived asynchronous tasks as stated in Javadoc  if you submit tasks which takes longer time to process  you will end up creating too many threads  You may hit 100  CPU if you submit long running tasks at faster rate to newCachedThreadPool  http   rashcoder com be-careful-while-using-executors-newcachedthreadpool

User · Answer

If you are not worried about an unbounded queue of Callable Runnable tasks  you can use one of them  As suggested by bruno  I too prefer newFixedThreadPool to newCachedThreadPool over these two   But ThreadPoolExecutor   provides more flexible features compared to either newFixedThreadPool or newCachedThreadPool  ThreadPoolExecutor int corePoolSize  int maximumPoolSize  long keepAliveTime   TimeUnit unit  BlockingQueue lt Runnable gt  workQueue  ThreadFactory threadFactory  RejectedExecutionHandler handler    Advantages    You have full control of BlockingQueue size  It s not un-bounded  unlike the earlier two options  I won t get an out of memory error due to a huge pile-up of pending Callable Runnable tasks when there is unexpected turbulence in the system  You can implement custom Rejection handling policy OR use one of the policies    In the default ThreadPoolExecutor AbortPolicy  the handler throws a runtime RejectedExecutionException upon rejection  In ThreadPoolExecutor CallerRunsPolicy  the thread that invokes execute itself runs the task  This provides a simple feedback control mechanism that will slow down the rate that new tasks are submitted  In ThreadPoolExecutor DiscardPolicy  a task that cannot be executed is simply dropped  In ThreadPoolExecutor DiscardOldestPolicy  if the executor is not shut down  the task at the head of the work queue is dropped  and then execution is retried  which can fail again  causing this to be repeated    You can implement a custom Thread factory for the below use cases    To set a more descriptive thread name To set thread daemon status To set thread priority

User · Answer

The ThreadPoolExecutor class is the base implementation for the executors that are returned from many of the Executors factory methods  So let s approach Fixed and Cached thread pools from ThreadPoolExecutor s perspective   ThreadPoolExecutor  The main constructor of this class looks like this   public ThreadPoolExecutor                    int corePoolSize                    int maximumPoolSize                    long keepAliveTime                    TimeUnit unit                    BlockingQueue lt Runnable gt  workQueue                    ThreadFactory threadFactory                    RejectedExecutionHandler handler     Core Pool Size  The corePoolSize determines the minimum size of the target thread pool  The implementation would maintain a pool of that size even if there are no tasks to execute   Maximum Pool Size  The maximumPoolSize is the maximum number of threads that can be active at once    After the thread pool grows and becomes bigger than the corePoolSize threshold  the executor can terminate idle threads and reach to the corePoolSize again  If allowCoreThreadTimeOut is true  then the executor can even terminate core pool threads if they were idle more than keepAliveTime threshold   So the bottom line is if threads remain idle more than keepAliveTime threshold  they may get terminated since there is no demand for them   Queuing  What happens when a new task comes in and all core threads are occupied  The new tasks will be queued inside that BlockingQueue lt Runnable gt  instance  When a thread becomes free  one of those queued tasks can be processed   There are different implementations of the BlockingQueue interface in Java  so we can implement different queuing approaches like    Bounded Queue  New tasks would be queued inside a bounded task queue  Unbounded Queue  New tasks would be queued inside an unbounded task queue  So this queue can grow as much as the heap size allows  Synchronous Handoff  We can also use the SynchronousQueue to queue the new tasks  In that case  when queuing a new task  another thread must already be waiting for that task    Work Submission  Here s how the ThreadPoolExecutor executes a new task    If fewer than corePoolSize threads are running  tries to start a new thread with the given task as its first job  Otherwise  it tries to enqueue the new task using the BlockingQueue offer method  The offer method won t block if the queue is full and immediately returns false  If it fails to queue the new task  i e  offer returns false   then it tries to add a new thread to the thread pool with this task as its first job  If it fails to add the new thread  then the executor is either shut down or saturated  Either way  the new task would be rejected using the provided RejectedExecutionHandler    The main difference between the fixed and cached thread pools boils down to these three factors    Core Pool Size Maximum Pool Size Queuing      ----------- ----------- ------------------- ---------------------------------    Pool Type   Core Size      Maximum Size             Queuing Strategy           ----------- ----------- ------------------- ---------------------------------      Fixed     n  fixed        n  fixed        Unbounded  LinkedBlockingQueue     ----------- ----------- ------------------- ---------------------------------      Cached        0       Integer MAX VALUE           SynchronousQueue           ----------- ----------- ------------------- ---------------------------------      Fixed Thread Pool   Here s how the Excutors newFixedThreadPool n  works   public static ExecutorService newFixedThreadPool int nThreads        return new ThreadPoolExecutor nThreads  nThreads                                    0L  TimeUnit MILLISECONDS                                    new LinkedBlockingQueue lt Runnable gt          As you can see    The thread pool size is fixed   If there is high demand  it won t grow  If threads are idle for quite some time  it won t shrink  Suppose all those threads are occupied with some long-running tasks and the arrival rate is still pretty high  Since the executor is using an unbounded queue  it may consume a huge part of the heap  Being unfortunate enough  we may experience an OutOfMemoryError       When should I use one or the other  Which strategy is better in terms of resource utilization    A fixed-size thread pool seems to be a good candidate when we re going to limit the number of concurrent tasks for resource management purposes    For example  if we re going to use an executor to handle web server requests  a fixed executor can handle the request bursts more reasonably    For even better resource management  it s highly recommended to create a custom ThreadPoolExecutor with a bounded BlockingQueue lt T gt  implementation coupled with reasonable RejectedExecutionHandler     Cached Thread Pool    Here s how the Executors newCachedThreadPool   works   public static ExecutorService newCachedThreadPool         return new ThreadPoolExecutor 0  Integer MAX VALUE                                    60L  TimeUnit SECONDS                                    new SynchronousQueue lt Runnable gt          As you can see    The thread pool can grow from zero threads to Integer MAX VALUE  Practically  the thread pool is unbounded  If any thread is idle for more than 1 minute  it may get terminated  So the pool can shrink if threads remain too much idle  If all allocated threads are occupied while a new task comes in  then it creates a new thread  as offering a new task to a SynchronousQueue always fails when there is no one on the other end to accept it        When should I use one or the other  Which strategy is better in terms of resource utilization    Use it when you have a lot of predictable short-running tasks

User · Answer

I think the docs explain the difference and usage of these two functions pretty well    newFixedThreadPool     Creates a thread pool that reuses a   fixed number of threads operating off   a shared unbounded queue  At any   point  at most nThreads threads will   be active processing tasks  If   additional tasks are submitted when   all threads are active  they will wait   in the queue until a thread is   available  If any thread terminates   due to a failure during execution   prior to shutdown  a new one will take   its place if needed to execute   subsequent tasks  The threads in the   pool will exist until it is explicitly   shutdown     newCachedThreadPool     Creates a thread pool that creates new   threads as needed  but will reuse   previously constructed threads when   they are available  These pools will   typically improve the performance of   programs that execute many short-lived   asynchronous tasks  Calls to execute   will reuse previously constructed   threads if available  If no existing   thread is available  a new thread will   be created and added to the pool    Threads that have not been used for   sixty seconds are terminated and   removed from the cache  Thus  a pool   that remains idle for long enough will   not consume any resources  Note that   pools with similar properties but   different details  for example    timeout parameters  may be created   using ThreadPoolExecutor constructors    In terms of resources  the newFixedThreadPool will keep all the threads running until they are explicitly terminated  In the newCachedThreadPool Threads that have not been used for sixty seconds are terminated and removed from the cache   Given this  the resource consumption will depend very much in the situation  For instance  If you have a huge number of long running tasks I would suggest the FixedThreadPool  As for the CachedThreadPool  the docs say that  These pools will typically improve the performance of programs that execute many short-lived asynchronous tasks

User · Answer

That   s right  Executors newCachedThreadPool   isn t a great choice for server code that s servicing multiple clients and concurrent requests   Why  There are basically two  related  problems with it    It s unbounded  which means that you re opening the door for anyone to cripple your JVM by simply injecting more work into the service  DoS attack   Threads consume a non-negligible amount of memory and also increase memory consumption based on their work-in-progress  so it s quite easy to topple a server this way  unless you have other circuit-breakers in place   The unbounded problem is exacerbated by the fact that the Executor is fronted by a SynchronousQueue which means there s a direct handoff between the task-giver and the thread pool  Each new task will create a new thread if all existing threads are busy  This is generally a bad strategy for server code  When the CPU gets saturated  existing tasks take longer to finish  Yet more tasks are being submitted and more threads created  so tasks take longer and longer to complete  When the CPU is saturated  more threads is definitely not what the server needs    Here are my recommendations   Use a fixed-size thread pool Executors newFixedThreadPool or a  ThreadPoolExecutor  with a set maximum number of threads

User · Answer

I do some quick tests and have the following findings   1  if using SynchronousQueue   After the threads reach the maximum size  any new work will be rejected with the exception like below      Exception in thread  main  java util concurrent RejectedExecutionException  Task java util concurrent FutureTask 3fee733d rejected from java util concurrent ThreadPoolExecutor 5acf9800 Running  pool size   3  active threads   3  queued tasks   0  completed tasks   0       at java util concurrent ThreadPoolExecutor AbortPolicy rejectedExecution ThreadPoolExecutor java 2047    2  if using LinkedBlockingQueue   The threads never increase from minimum size to maximum size  meaning the thread pool is fixed size as the minimum size

User · Answer

If you look at the source code  you will see  they are calling ThreadPoolExecutor  internally and setting their properties  You can create your one to have a better control of your requirement   public static ExecutorService newFixedThreadPool int nThreads       return new ThreadPoolExecutor nThreads  nThreads 0L  TimeUnit MILLISECONDS  new LinkedBlockingQueue lt Runnable gt         public static ExecutorService newCachedThreadPool             return new ThreadPoolExecutor 0  Integer MAX VALUE                                        60L  TimeUnit SECONDS                                        new SynchronousQueue lt Runnable gt

[java] Executors.newCachedThreadPool() versus Executors.newFixedThreadPool()

Examples related to java

Examples related to multithreading

Examples related to concurrency

Examples related to executorservice

Examples related to threadpoolexecutor