Split large string in n-size chunks in JavaScript

Question

I would like to split a very large string (let's say, 10,000 characters) into N-size chunks.

What would be the best way in terms of performance to do this?

For instance: "1234567890" split by 2 would become ["12", "34", "56", "78", "90"].

Would something like this be possible using String.prototype.match and if so, would that be the best way to do it in terms of performance?

User · Answer

Here is the code that I am using  it uses String prototype slice   Yes it is quite long as an answer goes as it tries to follow current standards as close as possible and of course contains a reasonable amount of JSDOC comments  However  once minified  the code is only 828 bytes and once gzipped for transmission it is only 497 bytes   The 1 method that this adds to String prototype  using Object defineProperty where available  is    toChunks   A number of tests have been included to check the functionality   Worried that the length of code will affect the performance  No need to worry  http   jsperf com chunk-string 3  Much of the extra code is there to be sure that the code will respond the same across multiple javascript environments    x000D   x000D    jslint maxlen 80  browser true  devel true    x000D   x000D     x000D     Properties used by toChunks  x000D      x000D   x000D    property x000D      MAX SAFE INTEGER  abs  ceil  configurable  defineProperty  enumerable  x000D      floor  length  max  min  pow  prototype  slice  toChunks  value  x000D      writable x000D     x000D   x000D     x000D     Properties used in the testing of toChunks implimentation  x000D      x000D   x000D    property x000D      appendChild  createTextNode  floor  fromCharCode  getElementById  length  x000D      log  pow  push  random  toChunks x000D     x000D   x000D   function      x000D       use strict   x000D   x000D      var MAX SAFE INTEGER   Number MAX SAFE INTEGER    Math pow 2  53  - 1  x000D   x000D          x000D         Defines a new property directly on an object  or modifies an existing x000D         property on an object  and returns the object  x000D         x000D          private x000D          function x000D          param  Object  object x000D          param  string  property x000D          param  Object  descriptor x000D          return  Object  x000D          see https   goo gl CZnEqg x000D          x000D      function  defineProperty object  property  descriptor    x000D          if  Object defineProperty    x000D              Object defineProperty object  property  descriptor   x000D            else   x000D              object property    descriptor value  x000D            x000D   x000D          return object  x000D        x000D   x000D          x000D         Returns true if the operands are strictly equal with no type conversion  x000D         x000D          private x000D          function x000D          param     a x000D          param     b x000D          return  boolean  x000D          see http   www ecma-international org ecma-262 5 1  sec-11 9 4 x000D          x000D      function  strictEqual a  b    x000D          return a     b  x000D        x000D   x000D          x000D         Returns true if the operand inputArg is undefined  x000D         x000D          private x000D          function x000D          param     inputArg x000D          return  boolean  x000D          x000D      function  isUndefined inputArg    x000D          return  strictEqual typeof inputArg   undefined    x000D        x000D   x000D          x000D         The abstract operation throws an error if its argument is a value that x000D         cannot be converted to an Object  otherwise returns the argument  x000D         x000D          private x000D          function x000D          param     inputArg The object to be tested  x000D          throws  TypeError  If inputArg is null or undefined  x000D          return     The inputArg if coercible  x000D          see https   goo gl 5GcmVq x000D          x000D      function  requireObjectCoercible inputArg    x000D          var errStr  x000D   x000D          if  inputArg     null     isUndefined inputArg     x000D              errStr    Cannot convert argument to object      inputArg  x000D              throw new TypeError errStr   x000D            x000D   x000D          return inputArg  x000D        x000D   x000D          x000D         The abstract operation converts its argument to a value of type string x000D         x000D          private x000D          function x000D          param     inputArg x000D          return  string  x000D          see https   people mozilla org  jorendorff es6-draft html sec-tostring x000D          x000D      function  toString inputArg    x000D          var type  x000D              val  x000D   x000D          if  inputArg     null    x000D              val    null   x000D            else   x000D              type   typeof inputArg  x000D              if  type      string     x000D                  val   inputArg  x000D                else if  type      undefined     x000D                  val   type  x000D                else   x000D                  if  type      symbol     x000D                      throw new TypeError  Cannot convert symbol to string    x000D                    x000D   x000D                  val   String inputArg   x000D                x000D            x000D   x000D          return val  x000D        x000D   x000D          x000D         Returns a string only if the arguments is coercible otherwise throws an x000D         error  x000D         x000D          private x000D          function x000D          param     inputArg x000D          throws  TypeError  If inputArg is null or undefined  x000D          return  string  x000D          x000D      function  onlyCoercibleToString inputArg    x000D          return  toString  requireObjectCoercible inputArg    x000D        x000D   x000D          x000D         The function evaluates the passed value and converts it to an integer  x000D         x000D          private x000D          function x000D          param     inputArg The object to be converted to an integer  x000D          return  number  If the target value is NaN  null or undefined  0 is x000D                           returned  If the target value is false  0 is returned x000D                           and if true  1 is returned  x000D          see http   www ecma-international org ecma-262 5 1  sec-9 4 x000D          x000D      function  toInteger inputArg    x000D          var number    inputArg  x000D              val   0  x000D   x000D          if   strictEqual number  number     x000D              if   number    number     Infinity    number     -Infinity    x000D                  val   number  x000D                else   x000D                  val    number  gt  0    -1    Math floor Math abs number    x000D                x000D            x000D   x000D          return val  x000D        x000D   x000D          x000D         The abstract operation ToLength converts its argument to an integer x000D         suitable for use as the length of an array-like object  x000D         x000D          private x000D          function x000D          param     inputArg The object to be converted to a length  x000D          return  number  If len  lt    0 then  0 else if len is  INFINITY then x000D                           2 53-1 else min len  2 53-1   x000D          see https   people mozilla org  jorendorff es6-draft html sec-tolength x000D          x000D      function  toLength inputArg    x000D          return Math min Math max  toInteger inputArg   0   MAX SAFE INTEGER   x000D        x000D   x000D      if   String prototype toChunks    x000D              x000D             This method chunks a string into an array of strings of a specified x000D             chunk size  x000D             x000D              function x000D              this  string  The string to be chunked  x000D              param  Number  chunkSize The size of the chunks that the string will x000D                                       be chunked into  x000D              returns  Array  Returns an array of the chunked string  x000D              x000D           defineProperty String prototype   toChunks     x000D              enumerable  false  x000D              configurable  true  x000D              writable  true  x000D              value  function  chunkSize    x000D                  var str    onlyCoercibleToString this   x000D                      chunkLength    toInteger chunkSize   x000D                      chunked       x000D                      numChunks  x000D                      length  x000D                      index  x000D                      start  x000D                      end  x000D   x000D                  if  chunkLength  lt  1    x000D                      return chunked  x000D                    x000D   x000D                  length    toLength str length   x000D                  numChunks   Math ceil length   chunkLength   x000D                  index   0  x000D                  start   0  x000D                  end   chunkLength  x000D                  chunked length   numChunks  x000D                  while  index  lt  numChunks    x000D                      chunked index    str slice start  end   x000D                      start   end  x000D                      end    chunkLength  x000D                      index    1  x000D                    x000D   x000D                  return chunked  x000D                x000D              x000D        x000D        x000D   x000D     x000D     Some tests x000D      x000D   x000D   function      x000D       use strict   x000D   x000D      var pre   document getElementById  out    x000D          chunkSizes       x000D          maxChunkSize   512  x000D          testString       x000D          maxTestString   100000  x000D          chunkSize   0  x000D          index   1  x000D   x000D      while  chunkSize  lt  maxChunkSize    x000D          chunkSize   Math pow 2  index   x000D          chunkSizes push chunkSize   x000D          index    1  x000D        x000D   x000D      index   0  x000D      while  index  lt  maxTestString    x000D          testString    String fromCharCode Math floor Math random     95    32   x000D          index    1  x000D        x000D   x000D      function log result    x000D          pre appendChild document createTextNode result     n     x000D        x000D   x000D      function test     x000D          var strLength   testString length  x000D              czLength   chunkSizes length  x000D              czIndex   0  x000D              czValue  x000D              result  x000D              numChunks  x000D              pass  x000D   x000D          while  czIndex  lt  czLength    x000D              czValue   chunkSizes czIndex   x000D              numChunks   Math ceil strLength   czValue   x000D              result   testString toChunks czValue   x000D              czIndex    1  x000D              log  chunksize      czValue   x000D              log   Number of chunks     x000D              log    Calculated      numChunks   x000D              log    Actual     result length   x000D              pass   result length     numChunks  x000D              log   First chunk size      result 0  length   x000D              pass   pass  amp  amp  result 0  length     czValue  x000D              log   Passed      pass   x000D              log      x000D            x000D        x000D   x000D      test    x000D      log      x000D      log  Simple test result    x000D      log  abcdefghijklmnopqrstuvwxyz  toChunks 3    x000D        x000D   lt pre id  out  gt  lt  pre gt  x000D   x000D   x000D

User · Answer

I have written an extended function  so the chunk length can also be an array of numbers  like  1 3   String prototype chunkString   function len        var  ret      if  this length  lt  1            return               if  typeof len      number   amp  amp  len  gt  0            var  size   Math ceil this length   len    offset   0           ret   new Array  size           for  var  i   0   i  lt   size   i                   ret  i    this substring  offset   offset    offset   len                       else if  typeof len      object   amp  amp  len length            var n   0  l   this length  chunk  that   this           ret               do               len forEach function o                    chunk   that substring n  n   o                   if  chunk                                ret push chunk                       n    chunk length                                                if  n     0                    return undefined     prevent an endless loop when len    0                          while  n  lt  l             return  ret       The code   1234567890123  chunkString  1 3     will return      1    234    5    678    9    012    3

User · Answer

it Split s large string in to Small strings of given words    function chunkSubstr str  words      var parts   str split        values        i   0   tmpVar           each parts  function index  value          if tmpVar length  lt  words             tmpVar          value         else            values i    tmpVar replace   s  g                  i              tmpVar   value                  if values length  lt  1  amp  amp   parts length  gt  0         values 0    tmpVar        return values

User · Answer

window format   function b  a            if   b    isNaN  a   return a          var a   b charAt 0      -    -a    a              j   a  lt  0   a   -a   0              e   b match     d -     g               h   e  amp  amp  e e length - 1                      e   e  amp  amp  e 1   amp  amp  e 0                      b   b split h               a   a toFixed b 1   amp  amp  b 1  length               a    a                   d   b 1   amp  amp  b 1  lastIndexOf  0                c   a split               if   c 1     c 1   amp  amp  c 1  length  lt   d  a     a  toFixed d   1           d   b 0  split e           b 0    d join              var f   b 0   amp  amp  b 0  indexOf  0            if  f  gt  -1              for    c 0  length  lt  b 0  length - f   c 0     0    c 0           else  c 0     0  amp  amp   c 0                 a   a split               a 0    c 0           if  c   d 1   amp  amp  d d length -                 1  length                for  var d   a 0   f       k   d length   c  g   0  i   d length  g  lt  i  g    f    d charAt g      g - k   1    c   amp  amp  g  lt  i - c  amp  amp   f    e               a 0    f                   a 1    b 1   amp  amp  a 1    h   a 1                return  j    -          a 0    a 1          var str  1234567890   var formatstr format             str   alert formatstr     This will split the string in reverse order with comma separated after 3 char s  If you want you can change the position

User · Answer

const getChunksFromString    str  chunkSize    gt        var regexChunk   new RegExp    1   chunkSize      g            represents any character     return str match regexChunk      Call it as needed   console log getChunksFromString  Hello world   3          Hel    lo     wor    ld

User · Answer

My issue with the above solution is that it beark the string into formal size chunks regardless of the position in the sentences.

I think the following a better approach; although it needs some performance tweaking:

 static chunkString(str, length, size,delimiter='\n' ) {
        const result = [];
        for (let i = 0; i < str.length; i++) {
            const lastIndex = _.lastIndexOf(str, delimiter,size + i);
            result.push(str.substr(i, lastIndex - i));
            i = lastIndex;
        }
        return result;
    }

User · Answer

You can definitely do something like  let pieces    1234567890   split     2     filter x   gt  x length    2     to get this      12    34    56    78    90      If you want to dynamically input adjust the chunk size so that the chunks are of size n  you can do this   n   2  let pieces    1234567890   split new RegExp       n toString          filter x   gt  x length    n     To find all possible size n chunks in the original string  try this   let subs   new Set    let n   2  let str    1234567890    let regex   new RegExp       n toString                set up regex expression dynamically encoded with n  for  let i   0  i  lt  n  i                     starting from all possible offsets from position 0 in the string     let pieces   str split regex  filter x   gt  x length    n        divide the string into chunks of size n        for  let p of pieces                       and add the chunks to the set         subs add p       str   str substr 1        shift the string reading frame     You should end up with      12    23    34    45    56    67    78    89    90    0

User · Answer

Include both left and right version with pre-allocation  This is as fast as RegExp impl for small chunks but it goes faster as the chunk size grows  And it is memory efficent  function chunkLeft  str  size   3      if  typeof str      string         const length   str length     const chunks   Array Math ceil length   size       for  let i   0  index   0  index  lt  length  i            chunks i    str slice index  index    size            return chunks        function chunkRight  str  size   3      if  typeof str      string         const length   str length     const chunks   Array Math ceil length   size       if  length          chunks 0    str slice 0  length   size    size        for  let i   1  index   chunks 0  length  index  lt  length  i              chunks i    str slice index  index    size                    return chunks        console log chunkRight        undefined console log chunkRight             console log chunkRight  1          quot 1 quot   console log chunkRight  123          quot 123 quot   console log chunkRight  1234          quot 1 quot    quot 234 quot   console log chunkRight  12345          quot 12 quot    quot 345 quot   console log chunkRight  123456          quot 123 quot    quot 456 quot   console log chunkRight  1234567          quot 1 quot    quot 234 quot    quot 567 quot

User · Answer

var l   str length  lc   0  chunks       c   0  chunkSize   2  for    lc  lt  l  c        chunks c    str slice lc  lc    chunkSize

User · Answer

function chunkString str  length   10        let result               offset   0      if  str length  lt   length  return result push str   amp  amp  result      while  offset  lt  str length            result push str substr offset  length            offset    length            return result

User · Answer

I would use a regex     var chunkStr   function str  chunkLength        return str match new RegExp     s  S  1      chunkLength         g

User · Answer

You can do something like this    1234567890  match    1 2  g      Results in    12    34    56    78    90     The method will still work with strings whose size is not an exact multiple of the chunk-size    123456789  match    1 2  g      Results in    12    34    56    78    9     In general  for any string out of which you want to extract at-most n-sized substrings  you would do   str match    1 n  g      Replace n with the size of the substring   If your string can contain newlines or carriage returns  you would do   str match       r n   1 n  g      Replace n with the size of the substring   As far as performance  I tried this out with approximately 10k characters and it took a little over a second on Chrome  YMMV   This can also be used in a reusable function   function chunkString str  length      return str match new RegExp    1     length         g

User · Answer

You can use reduce   without any regex    str  n    gt      return str split     reduce       acc  rec  index    gt          return   index   n       index     acc concat rec    acc concat      rec                    split

User · Answer

Here s a solution I came up with for template strings after a little experimenting   Usage   chunkString 5  testing123     x000D   x000D  function chunkString nSize    x000D      return  strToChunk    gt    x000D          let result       x000D          let chars   String strToChunk  split      x000D   x000D          for let i   0  i  lt   String strToChunk  length   nSize   i      x000D              result   result concat chars slice i nSize  i 1  nSize  join       x000D            x000D          return result x000D        x000D    x000D   x000D  document write chunkString 5  testing123    x000D     returns  testi ng123 x000D   x000D  document write chunkString 3  testing123    x000D     returns  tes tin g12 3 x000D   x000D   x000D

User · Answer

comparison of match  slice  substr and substring comparison of match and slice for different chunk sizes comparison of match and slice with small chunk size   Bottom line    match is very inefficient  slice is better  on Firefox substr substring is better still match is even more inefficient for short strings  even with cached regex - probably due to regex parsing setup time  match is even more inefficient for large chunk size  probably due to inability to  jump   for longer strings with very small chunk size  match outperforms slice on older IE but still loses on all other systems jsperf rocks

User · Answer

I created several faster variants which you can see on jsPerf  My favorite one is this   function chunkSubstr str  size      const numChunks   Math ceil str length   size    const chunks   new Array numChunks     for  let i   0  o   0  i  lt  numChunks    i  o    size        chunks i    str substr o  size         return chunks

User · Answer

var str    123456789   var chunks       var chunkSize   2   while  str        if  str length  lt  chunkSize            chunks push str           break            else           chunks push str substr 0  chunkSize            str   str substr chunkSize            alert chunks      chunks    12 34 56 78 9

User · Answer

What about this small piece of code   function splitME str  size        let subStr   new RegExp    1     size         g        return str match subStr

User · Answer

In the form of a prototype function   String prototype lsplit   function        return this match new RegExp    1      arguments length  1   isFinite String arguments 0   trim    arguments 0  false  1         g

User · Answer

Using slice   method   function returnChunksArray str  chunkSize      var arr         while str               arr push str slice 0  chunkSize        str   str slice chunkSize         return arr      The same can be done using substring   method   function returnChunksArray str  chunkSize      var arr         while str               arr push str substring 0  chunkSize        str   str substring chunkSize         return arr

User · Answer

Surprise  You can use split to split   var parts    1234567890   split     2     filter O  gt O    Results in    12    34    56    78    90

User · Answer

This is a fast and straightforward solution -

_x000D_

function chunkString (str, len) {_x000D_
  const size = Math.ceil(str.length/len)_x000D_
  const r = Array(size)_x000D_
  let offset = 0_x000D_
  _x000D_
  for (let i = 0; i < size; i++) {_x000D_
    r[i] = str.substr(offset, len)_x000D_
    offset += len_x000D_
  }_x000D_
  _x000D_
  return r_x000D_
}_x000D_
_x000D_
console.log(chunkString("helloworld", 3))_x000D_
// => [ "hel", "low", "orl", "d" ]_x000D_
_x000D_
// 10,000 char string_x000D_
const bigString = "helloworld".repeat(1000)_x000D_
console.time("perf")_x000D_
const result = chunkString(bigString, 3)_x000D_
console.timeEnd("perf")_x000D_
console.log(result)_x000D_
// => perf: 0.385 ms_x000D_
// => [ "hel", "low", "orl", "dhe", "llo", "wor", ... ]

_x000D_

[javascript] Split large string in n-size chunks in JavaScript

The answer is

Examples related to javascript

Examples related to regex

Examples related to string

Examples related to split

Tags