Node js on multi-core machines

Question

Node js looks interesting  BUT I must miss something - isn t Node js tuned only to run on a single process and thread   Then how does it scale for multi-core CPUs and multi-CPU servers  After all  it is all great to make fast as possible single-thread server  but for high loads I would want to use several CPUs  And the same goes for making applications faster - seems today the way is use multiple CPUs and parallelize the tasks   How does Node js fit into this picture  Is its idea to somehow distribute multiple instances or what

User · Answer

The new kid on the block here is LearnBoost's "Up".

It provides "Zero-downtime reloads" and additionally creates multiple workers (by default the number of CPUs, but it is configurable) to provide the best of all Worlds.

It is new, but seems to be pretty stable, and I'm using it happily in one of my current projects.

User · Answer

I m using Node worker to run processes in a simple way from my main process  Seems to be working great while we wait for the official way to come around

User · Answer

I have to add an important difference between using node s build in cluster mode VS a process manager like PM2 s cluster mode  PM2 allows zero down time reloads when you are running  pm2 start app js -i 2 --wait-ready  In your codes add the following process send  ready      When you call pm2 reload app after code updates  PM2 will reload the first instance of the app  wait for the  ready  call  then it move on to reloads the next instance  ensuring you always have an app active to respond to requests   While if you use nodejs  cluster  there will be down time when you restart and waiting for server to be ready

User · Answer

Spark2 is based on Spark which is now no longer maintained  Cluster is its successor  and it has some cool features  like spawning one worker process per CPU core and respawning dead workers

User · Answer

Ryan Dahl answers this question in the tech talk he gave at Google last summer   To paraphrase   just run multiple node processes and use something sensible to allow them to communicate  e g  sendmsg  -style IPC or traditional RPC    If you want to get your hands dirty right away  check out the spark2 Forever module   It makes spawning multiple node processes trivially easy   It handles setting up port sharing  so they can each accept connections to the same port  and also auto-respawning if you want to make sure a process is restarted if when it dies   UPDATE - 10 11 11  Consensus in the node community seems to be that Cluster is now the preferred module for managing multiple node instances per machine   Forever is also worth a look

User · Answer

Future version of node will allow you to fork a process and pass messages to it and Ryan has stated he wants to find some way to also share file handlers  so it won t be a straight forward Web Worker implementation   At this time there is not an easy solution for this but it s still very early and node is one of the fastest moving open source projects I ve ever seen so expect something awesome in the near future

User · Answer

Node Js is supporting clustering to take full advantages of your cpu  If you are not not running it with cluster  then probably you are wasting your hardware capabilities   Clustering in Node js allows you to create separate processes which can share same server port  For example  if we run one HTTP server on Port 3000  it is one Server running on Single thread on single core of processor   Code shown below allow you to cluster your application  This code is official code represented by Node js   var cluster   require  cluster    var numCPUs   require  os   cpus   length   if  cluster isMaster           Fork workers      for  var i   0  i  lt  numCPUs  i              cluster fork               Object keys cluster workers  forEach function id            console log  I am running with ID       cluster workers id  process pid                cluster on  exit   function worker  code  signal            console log  worker     worker process pid     died              else          Do further processing      check this article for the full tutorial

User · Answer

You may run your node js application on multiple cores by using the cluster module on combination with os module which may be used to detect how many CPUs you have   For example let s imagine that you have a server module that runs simple http server on the backend and you want to run it for several CPUs    x000D   x000D     Dependencies  x000D  const server   require    lib server       This is our custom server module  x000D  const cluster   require  cluster    x000D  const os   require  os    x000D   x000D      If we re on the master thread start the forks  x000D  if  cluster isMaster    x000D       Fork the process  x000D    for  let i   0  i  lt  os cpus   length  i      x000D      cluster fork    x000D      x000D    else   x000D       If we re not on the master thread start the server  x000D    server init    x000D    x000D   x000D   x000D

User · Answer

You can use cluster module  Check this   var cluster   require  cluster    var http   require  http    var numCPUs   require  os   cpus   length   if  cluster isMaster           Fork workers      for  var i   0  i  lt  numCPUs  i              cluster fork               cluster on  exit   function worker  code  signal            console log  worker     worker process pid     died              else          Workers can share any TCP connection        In this case its a HTTP server     http createServer function req  res            res writeHead 200           res end  hello world n           listen 8000

User · Answer

As mentioned above  Cluster will scale and load-balance your app across all cores  adding something like  cluster on  exit   function        cluster fork          Will restart any failing workers   These days  a lot of people also prefer PM2  which handles the clustering for you and also provides some cool monitoring features   Then  add Nginx or HAProxy in front of several machines running with clustering and you have multiple levels of failover and a much higher load capacity

User · Answer

It s also possible to design the web-service as several stand alone servers that listen to unix sockets  so that you can push functions like data processing into seperate processes   This is similar to most scrpting database web server architectures where a cgi process handles business logic and then pushes and pulls the data via a unix socket to a database   the difference being that the data processing is written as  a node webserver listening on a port   it s more complex but ultimately its where multi-core development has to go  a multiprocess architecture using multiple components for each web request

User · Answer

This post is up-to-date as of 2012-09-02  newer than above     Node js absolutely does scale on multi-core machines   Yes  Node js is one-thread-per-process  This is a very deliberate design decision and eliminates the need to deal with locking semantics  If you don t agree with this  you probably don t yet realize just how insanely hard it is to debug multi-threaded code   For a deeper explanation of the Node js process model and why it works this way  and why it will NEVER support multiple threads   read my other post   So how do I take advantage of my 16 core box   Two ways    For big heavy compute tasks like image encoding  Node js can fire up child processes or send messages to additional worker processes  In this design  you d have one thread managing the flow of events and N processes doing heavy compute tasks and chewing up the other 15 CPUs  For scaling throughput on a webservice  you should run multiple Node js servers on one box  one per core and split request traffic between them  This provides excellent CPU-affinity and will scale throughput nearly linearly with core count      Scaling throughput on a webservice  Since v6 0 X Node js has included the cluster module straight out of the box  which makes it easy to set up multiple node workers that can listen on a single port  Note that this is NOT the same as the older learnboost  cluster  module available through npm   if  cluster isMaster         Fork workers    for  var i   0  i  lt  numCPUs  i          cluster fork          else     http Server function req  res           listen 8000       Workers will compete to accept new connections  and the least loaded process is most likely to win  It works pretty well and can scale up throughput quite well on a multi-core box   If you have enough load to care about multiple cores  then you are going to want to do a few more things too    Run your Node js service behind a web-proxy like Nginx or Apache - something that can do connection throttling  unless you want overload conditions to bring the box down completely   rewrite URLs  serve static content  and proxy other sub-services  Periodically recycle your worker processes  For a long-running process  even a small memory leak will eventually add up  Setup log collection   monitoring     PS  There s a discussion between Aaron and Christopher in the comments of another post  as of this writing  its the top post   A few comments on that    A shared socket model is very convenient for allowing multiple processes to listen on a single port and compete to accept new connections  Conceptually  you could think of preforked Apache doing this with the significant caveat that each process will only accept a single connection and then die  The efficiency loss for Apache is in the overhead of forking new processes and has nothing to do with the socket operations  For Node js  having N workers compete on a single socket is an extremely reasonable solution  The alternative is to set up an on-box front-end like Nginx and have that proxy traffic to the individual workers  alternating between workers for assigning new connections  The two solutions have very similar performance characteristics  And since  as I mentioned above  you will likely want to have Nginx  or an alternative  fronting your node service anyways  the choice here is really between    Shared Ports  nginx  port 80  --  Node workers x N  sharing port 3000 w  Cluster   vs  Individual Ports  nginx  port 80  --   Node worker  port 3000   Node worker  port 3001   Node worker  port 3002   Node worker  port 3003        There are arguably some benefits to the individual ports setup  potential to have less coupling between processes  have more sophisticated load-balancing decisions  etc    but it is definitely more work to set up and the built-in cluster module is a low-complexity alternative that works for most people

User · Answer

It s possible to scale NodeJS out to multiple boxes using a pure TCP load balancer  HAProxy  in front of multiple boxes running one NodeJS process each   If you then have some common knowledge to share between all instances you could use a central Redis store or similar which can then be accessed from all process instances  e g  from all boxes

User · Answer

Multi-node harnesses all the cores that you may have  Have a look at http   github com kriszyp multi-node   For simpler needs  you can start up multiple copies of node on different port numbers and put a load balancer in front of them

User · Answer

One method would be to run multiple instances of node js on the server and then put a load balancer  preferably a non-blocking one like nginx  in front of them

User · Answer

The cluster module allows you to utilise all cores of your machine  In fact you can take advantage of this in just 2 commands and without touching your code using a very popular process manager pm2   npm i -g pm2 pm2 start app js -i max

[javascript] Node.js on multi-core machines

Examples related to javascript

Examples related to node.js

Examples related to node-cluster