Is there a common Java utility to break a list into batches

Question

I wrote myself a utility to break a list into batches of given size  I just wanted to know if there is already any apache commons util for this   public static  lt T gt  List lt List lt T gt  gt  getBatches List lt T gt  collection int batchSize       int i   0      List lt List lt T gt  gt  batches   new ArrayList lt List lt T gt  gt         while i lt collection size             int nextInc   Math min collection size  -i batchSize           List lt T gt  batch   collection subList i i nextInc           batches add batch           i   i   nextInc             return batches      Please let me know if there any existing utility already for the same

User · Answer

There was another question that was closed as being a duplicate of this one, but if you read it closely, it's subtly different. So in case someone (like me) actually wants to split a list into a given number of almost equally sized sublists, then read on.

I simply ported the algorithm described here to Java.

@Test
public void shouldPartitionListIntoAlmostEquallySizedSublists() {

    List<String> list = Arrays.asList("a", "b", "c", "d", "e", "f", "g");
    int numberOfPartitions = 3;

    List<List<String>> split = IntStream.range(0, numberOfPartitions).boxed()
            .map(i -> list.subList(
                    partitionOffset(list.size(), numberOfPartitions, i),
                    partitionOffset(list.size(), numberOfPartitions, i + 1)))
            .collect(toList());

    assertThat(split, hasSize(numberOfPartitions));
    assertEquals(list.size(), split.stream().flatMap(Collection::stream).count());
    assertThat(split, hasItems(Arrays.asList("a", "b", "c"), Arrays.asList("d", "e"), Arrays.asList("f", "g")));
}

private static int partitionOffset(int length, int numberOfPartitions, int partitionIndex) {
    return partitionIndex * (length / numberOfPartitions) + Math.min(partitionIndex, length % numberOfPartitions);
}

User · Answer

Another approach is to use Collectors groupingBy of indices and then map the grouped indices to the actual elements      final List lt Integer gt  numbers   range 1  12               boxed                collect toList         System out println numbers        final List lt List lt Integer gt  gt  groups   range 0  numbers size                 boxed                collect groupingBy index - gt  index   4                values                stream                map indices - gt  indices                      stream                        map numbers  get                       collect toList                  collect toList         System out println groups    Output    1  2  3  4  5  6  7  8  9  10  11    1  2  3  4    5  6  7  8    9  10  11

User · Answer

List lt T gt  batch   collection subList i i nextInc   - gt  List lt T gt  batch   collection subList i  i   i   nextInc

User · Answer

The following example demonstrates chunking of a List   package de thomasdarimont labs   import java util ArrayList  import java util Arrays  import java util HashMap  import java util List  import java util Map   public class SplitIntoChunks        public static void main String   args             List lt Integer gt  ints   Arrays asList 1  2  3  4  5  6  7  8  9  10  11            List lt List lt Integer gt  gt  chunks   chunk ints  4            System out printf  Ints     s n   ints           System out printf  Chunks   s n   chunks              public static  lt T gt  List lt List lt T gt  gt  chunk List lt T gt  input  int chunkSize             int inputSize   input size            int chunkCount    int  Math ceil inputSize    double  chunkSize            Map lt Integer  List lt T gt  gt  map   new HashMap lt  gt  chunkCount           List lt List lt T gt  gt  chunks   new ArrayList lt  gt  chunkCount            for  int i   0  i  lt  inputSize  i                   map computeIfAbsent i   chunkSize   ignore  - gt                     List lt T gt  chunk   new ArrayList lt  gt                     chunks add chunk                   return chunk                  add input get i                       return chunks            Output   Ints     1  2  3  4  5  6  7  8  9  10  11  Chunks    1  2  3  4    5  6  7  8    9  10  11

User · Answer

I came up with this one   private static  lt T gt  List lt List lt T gt  gt  partition Collection lt T gt  members  int maxSize        List lt List lt T gt  gt  res   new ArrayList lt  gt          List lt T gt  internal   new ArrayList lt  gt          for  T member   members                internal add member            if  internal size      maxSize                        res add internal               internal   new ArrayList lt  gt                         if  internal isEmpty      false                res add internal             return res

User · Answer

Here an example  final AtomicInteger counter   new AtomicInteger    final int partitionSize 3  final List lt Object gt  list new ArrayList lt  gt                 list add  quot A quot                list add  quot B quot                list add  quot C quot                list add  quot D quot                list add  quot E quot                     final Collection lt List lt Object gt  gt  subLists list stream   collect Collectors groupingBy                  it- gt counter getAndIncrement     partitionSize                    values            System out println subLists    Input   A  B  C  D  E  Output    A  B  C    D  E   You can find examples here  https   e printstacktrace blog divide-a-list-to-lists-of-n-size-in-Java-8

User · Answer

import com google common collect Lists   List lt List lt T gt  gt  batches   Lists partition List lt T gt  batchSize   Use Lists partition List batchSize   You need to import Lists from google common package  com google common collect Lists   It will return List of List lt T gt  with and the size of every element equal to your batchSize

User · Answer

In case you want to produce a Java-8 stream of batches  you can try the following code  public static  lt T gt  Stream lt List lt T gt  gt  batches List lt T gt  source  int length        if  length  lt   0          throw new IllegalArgumentException  quot length    quot    length       int size   source size        if  size  lt   0          return Stream empty        int fullChunks    size - 1    length      return IntStream range 0  fullChunks   1  mapToObj          n - gt  source subList n   length  n    fullChunks   size    n   1    length       public static void main String   args        List lt Integer gt  list   Arrays asList 1  2  3  4  5  6  7  8  9  10  11  12  13  14        System out println  quot By 3  quot        batches list  3  forEach System out  println            System out println  quot By 4  quot        batches list  4  forEach System out  println      Output  By 3   1  2  3   4  5  6   7  8  9   10  11  12   13  14  By 4   1  2  3  4   5  6  7  8   9  10  11  12   13  14

User · Answer

Here is a simple solution for Java 8    public static  lt T gt  Collection lt List lt T gt  gt  prepareChunks List lt T gt  inputList  int chunkSize        AtomicInteger counter   new AtomicInteger        return inputList stream   collect Collectors groupingBy it - gt  counter getAndIncrement     chunkSize   values

User · Answer

You can use below code to get the batch of list   Iterable lt List lt T gt  gt  batchIds   Iterables partition list  batchSize     You need to import Google Guava library to use above code

User · Answer

With Java 9 you can use IntStream iterate   with hasNext condition  So you can simplify the code of your method to this   public static  lt T gt  List lt List lt T gt  gt  getBatches List lt T gt  collection  int batchSize        return IntStream iterate 0  i - gt  i  lt  collection size    i - gt  i   batchSize               mapToObj i - gt  collection subList i  Math min i   batchSize  collection size                   collect Collectors toList         Using  0  1  2  3  4  5  6  7  8  9   the result of getBatches numbers  4  will be     0  1  2  3    4  5  6  7    8  9

User · Answer

Check out   Lists partition java util List  int  from Google Guava      Returns consecutive sublists of a list  each of the same size  the final list may be smaller   For example  partitioning a list containing  a  b  c  d  e  with a partition size of 3 yields   a  b  c    d  e   -- an outer list containing two inner lists of three and two elements  all in the original order

User · Answer

A one-liner in Java 8 would be   import static java util function Function identity  import static java util stream Collectors     private static  lt T gt  Collection lt List lt T gt  gt  partition List lt T gt  xs  int size        return IntStream range 0  xs size                 boxed                collect collectingAndThen toMap identity    xs  get   Map  entrySet                stream                collect groupingBy x - gt  x getKey     size  mapping Map Entry  getValue  toList                   values

User · Answer

Use Apache Commons ListUtils partition   org apache commons collections4 ListUtils partition final List lt T gt  list  final int size

User · Answer

Another approach to solve this  question   public class CollectionUtils                  Splits the collection into lists with given batch size        param collection to split in to batches        param batchsize size of the batch        param  lt T gt  it maintains the input type to output type        return nested list            public static  lt T gt  List lt List lt T gt  gt  makeBatch Collection lt T gt  collection  int batchsize             List lt List lt T gt  gt  totalArrayList   new ArrayList lt  gt             List lt T gt  tempItems   new ArrayList lt  gt              Iterator lt T gt  iterator   collection iterator             for  int i   0  i  lt  collection size    i                  tempItems add iterator next                 if   i 1    batchsize    0                    totalArrayList add tempItems                   tempItems   new ArrayList lt  gt                                      if  tempItems size    gt  0                totalArrayList add tempItems                      return totalArrayList

User · Answer

Similar to OP without streams and libs  but conciser   public  lt T gt  List lt List lt T gt  gt  getBatches List lt T gt  collection  int batchSize        List lt List lt T gt  gt  batches   new ArrayList lt  gt         for  int i   0  i  lt  collection size    i    batchSize            batches add collection subList i  Math min i   batchSize  collection size                 return batches

User · Answer

Using various cheats from the web  I came to this solution   int   count   new int 1   final int CHUNK SIZE   500  Map lt Integer  List lt Long gt  gt  chunkedUsers   users stream   collect  Collectors groupingBy       user - gt            count 0             return Math floorDiv  count 0   CHUNK SIZE                 We use count to mimic a normal collection index  Then  we group the collection elements in buckets  using the algebraic quotient as bucket number  The final map contains as key the bucket number  as value the bucket itself   You can then easily do an operation on each of the buckets with   chunkedUsers values   forEach

[java] Is there a common Java utility to break a list into batches?

Examples related to java

Examples related to collections