[javascript] JavaScript, Node.js: is Array.forEach asynchronous?

I have a question regarding the native Array.forEach implementation of JavaScript: Does it behave asynchronously? For example, if I call:

[many many elements].forEach(function () {lots of work to do})

Will this be non-blocking?

This question is related to javascript arrays asynchronous foreach node.js

The answer is


If you need an asynchronous-friendly version of Array.forEach and similar, they're available in the Node.js 'async' module: http://github.com/caolan/async ...as a bonus this module also works in the browser.

async.each(openFiles, saveFile, function(err){
    // if any of the saves produced an error, err would equal that error
});

Array.forEach is meant for computing stuff not waiting, and there is nothing to be gained making computations asynchronous in an event loop (webworkers add multiprocessing, if you need multi-core computation). If you want to wait for multiple tasks to end, use a counter, which you can wrap in a semaphore class.


Use Promise.each of bluebird library.

Promise.each(
Iterable<any>|Promise<Iterable<any>> input,
function(any item, int index, int length) iterator
) -> Promise

This method iterates over an array, or a promise of an array, which contains promises (or a mix of promises and values) with the given iterator function with the signature (value, index, length) where the value is the resolved value of a respective promise in the input array. Iteration happens serially. If the iterator function returns a promise or a thenable, then the result of the promise is awaited before continuing with next iteration. If any promise in the input array is rejected, then the returned promise is rejected as well.

If all of the iterations resolve successfully, Promise.each resolves to the original array unmodified. However, if one iteration rejects or errors, Promise.each ceases execution immediately and does not process any further iterations. The error or rejected value is returned in this case instead of the original array.

This method is meant to be used for side effects.

var fileNames = ["1.txt", "2.txt", "3.txt"];

Promise.each(fileNames, function(fileName) {
    return fs.readFileAsync(fileName).then(function(val){
        // do stuff with 'val' here.  
    });
}).then(function() {
console.log("done");
});

There is a common pattern for doing a really heavy computation in Node that may be applicable to you...

Node is single-threaded (as a deliberate design choice, see What is Node.js?); this means that it can only utilize a single core. Modern boxes have 8, 16, or even more cores, so this could leave 90+% of the machine idle. The common pattern for a REST service is to fire up one node process per core, and put these behind a local load balancer like http://nginx.org/.

Forking a child - For what you are trying to do, there is another common pattern, forking off a child process to do the heavy lifting. The upside is that the child process can do heavy computation in the background while your parent process is responsive to other events. The catch is that you can't / shouldn't share memory with this child process (not without a LOT of contortions and some native code); you have to pass messages. This will work beautifully if the size of your input and output data is small compared to the computation that must be performed. You can even fire up a child node.js process and use the same code you were using previously.

For example:

var child_process = require('child_process');
function run_in_child(array, cb) {
    var process = child_process.exec('node libfn.js', function(err, stdout, stderr) {
        var output = JSON.parse(stdout);
        cb(err, output);
    });
    process.stdin.write(JSON.stringify(array), 'utf8');
    process.stdin.end();
}

Here is a small example you can run to test it:

[1,2,3,4,5,6,7,8,9].forEach(function(n){
    var sum = 0;
    console.log('Start for:' + n);
    for (var i = 0; i < ( 10 - n) * 100000000; i++)
        sum++;

    console.log('Ended for:' + n, sum);
});

It will produce something like this(if it takes too less/much time, increase/decrease the number of iterations):

(index):48 Start for:1
(index):52 Ended for:1 900000000
(index):48 Start for:2
(index):52 Ended for:2 800000000
(index):48 Start for:3
(index):52 Ended for:3 700000000
(index):48 Start for:4
(index):52 Ended for:4 600000000
(index):48 Start for:5
(index):52 Ended for:5 500000000
(index):48 Start for:6
(index):52 Ended for:6 400000000
(index):48 Start for:7
(index):52 Ended for:7 300000000
(index):48 Start for:8
(index):52 Ended for:8 200000000
(index):48 Start for:9
(index):52 Ended for:9 100000000
(index):45 [Violation] 'load' handler took 7285ms

Although Array.forEach is not asynchronous, you can get asynchronous "end result". Example below:

_x000D_
_x000D_
function delayFunction(x) {
    return new Promise(
        (resolve) => setTimeout(() => resolve(x), 1000)
    );
}

[1, 2, 3].forEach(async(x) => {
    console.log(x);
    console.log(await delayFunction(x));
});
_x000D_
_x000D_
_x000D_


There is a package on npm for easy asynchronous for each loops.

var forEachAsync = require('futures').forEachAsync;

// waits for one request to finish before beginning the next 
forEachAsync(['dogs', 'cats', 'octocats'], function (next, element, index, array) {
  getPics(element, next);
  // then after all of the elements have been handled 
  // the final callback fires to let you know it's all done 
  }).then(function () {
    console.log('All requests have finished');
});

Also another variation forAllAsync


This is a short asynchronous function to use without requiring third party libs

Array.prototype.each = function (iterator, callback) {
    var iterate = function () {
            pointer++;
            if (pointer >= this.length) {
                callback();
                return;
            }
            iterator.call(iterator, this[pointer], iterate, pointer);
    }.bind(this),
        pointer = -1;
    iterate(this);
};

Edit 2018-10-11: It looks like there is a good chance the standard described below may not go through, consider pipelineing as an alternative (does not behave exactly the same but methods could be implemented in a similar manor).

This is exactly why I am excited about es7, in future you will be able to do something like the code below (some of the specs are not complete so use with caution, I will try to keep this up to date). But basically using the new :: bind operator, you will be able to run a method on an object as if the object's prototype contains the method. eg [Object]::[Method] where normally you would call [Object].[ObjectsMethod]

Note to do this today (24-July-16) and have it work in all browsers you will need to transpile your code for the following functionality:Import / Export, Arrow functions, Promises, Async / Await and most importantly function bind. The code below could be modfied to use only function bind if nessesary, all this functionality is neatly available today by using babel.

YourCode.js (where 'lots of work to do' must simply return a promise, resolving it when the asynchronous work is done.)

import { asyncForEach } from './ArrayExtensions.js';

await [many many elements]::asyncForEach(() => lots of work to do);

ArrayExtensions.js

export function asyncForEach(callback)
{
    return Promise.resolve(this).then(async (ar) =>
    {
        for(let i=0;i<ar.length;i++)
        {
            await callback.call(ar, ar[i], i, ar);
        }
    });
};

export function asyncMap(callback)
{
    return Promise.resolve(this).then(async (ar) =>
    {
        const out = [];
        for(let i=0;i<ar.length;i++)
        {
            out[i] = await callback.call(ar, ar[i], i, ar);
        }
        return out;
    });
};

It is possible to code even the solution like this for example :

 var loop = function(i, data, callback) {
    if (i < data.length) {
        //TODO("SELECT * FROM stackoverflowUsers;", function(res) {
            //data[i].meta = res;
            console.log(i, data[i].title);
            return loop(i+1, data, errors, callback);
        //});
    } else {
       return callback(data);
    }
};

loop(0, [{"title": "hello"}, {"title": "world"}], function(data) {
    console.log("DONE\n"+data);
});

On the other hand, it is much slower than a "for".

Otherwise, the excellent Async library can do this: https://caolan.github.io/async/docs.html#each


Examples related to javascript

need to add a class to an element How to make a variable accessible outside a function? Hide Signs that Meteor.js was Used How to create a showdown.js markdown extension Please help me convert this script to a simple image slider Highlight Anchor Links when user manually scrolls? Summing radio input values How to execute an action before close metro app WinJS javascript, for loop defines a dynamic variable name Getting all files in directory with ajax

Examples related to arrays

PHP array value passes to next row Use NSInteger as array index How do I show a message in the foreach loop? Objects are not valid as a React child. If you meant to render a collection of children, use an array instead Iterating over arrays in Python 3 Best way to "push" into C# array Sort Array of object by object field in Angular 6 Checking for duplicate strings in JavaScript array what does numpy ndarray shape do? How to round a numpy array?

Examples related to asynchronous

How to read file with async/await properly? Use Async/Await with Axios in React.js Waiting until the task finishes How to reject in async/await syntax? React - Display loading screen while DOM is rendering? angular 2 how to return data from subscribe How do I access store state in React Redux? SyntaxError: Unexpected token function - Async Await Nodejs Why does .json() return a promise? Why is setState in reactjs Async instead of Sync?

Examples related to foreach

Angular ForEach in Angular4/Typescript? How to use forEach in vueJs? Python foreach equivalent Get current index from foreach loop TypeScript for ... of with index / key? JavaScript: Difference between .forEach() and .map() JSON forEach get Key and Value Laravel blade check empty foreach Go to "next" iteration in JavaScript forEach loop Why is "forEach not a function" for this object?

Examples related to node.js

Hide Signs that Meteor.js was Used Querying date field in MongoDB with Mongoose SyntaxError: Cannot use import statement outside a module Server Discovery And Monitoring engine is deprecated How to fix ReferenceError: primordials is not defined in node UnhandledPromiseRejectionWarning: This error originated either by throwing inside of an async function without a catch block dyld: Library not loaded: /usr/local/opt/icu4c/lib/libicui18n.62.dylib error running php after installing node with brew on Mac internal/modules/cjs/loader.js:582 throw err DeprecationWarning: Buffer() is deprecated due to security and usability issues when I move my script to another server Please run `npm cache clean`