What is Node js

Question

I don t fully get what Node js is all about  Maybe it s because I am mainly a web based business application developer  What is it and what is the use of it   My understanding so far is that    The programming model is event driven  especially the way it handles I O  It uses JavaScript and the parser is V8  It can be easily used to create concurrent server applications    Are my understandings correct  If yes  then what are the benefits of evented I O  is it just more for the concurrency stuff  Also  is the direction of Node js to become a framework like  JavaScript based  V8 based  programming model

User · Answer

Node js is an open source command line tool built for the server side JavaScript code  You can download a tarball  compile and install the source  It lets you run JavaScript programs   The JavaScript is executed by the V8  a JavaScript engine developed by Google which is used in Chrome browser  It uses a JavaScript API to access the network and file system   It is popular for its performance and the ability to perform parallel operations      Understanding node js is the best explanation of node js I have found so far    Following are some good articles on the topic         Learning Server-Side JavaScript with Node js   This Time  You   ll Learn Node js

User · Answer

I use Node js at work  and find it to be very powerful  Forced to choose one word to describe Node js  I d say  interesting   which is not a purely positive adjective   The community is vibrant and growing  JavaScript  despite its oddities can be a great language to code in  And you will daily rethink your own understanding of  best practice  and the patterns of well-structured code  There s an enormous energy of ideas flowing into Node js right now  and working in it exposes you to all this thinking - great mental weightlifting   Node js in production is definitely possible  but far from the  turn-key  deployment seemingly promised by the documentation  With Node js v0 6 x   cluster  has been integrated into the platform  providing one of the essential building blocks  but my  production js  script is still  150 lines of logic to handle stuff like creating the log directory  recycling dead workers  etc  For a  serious  production service  you also need to be prepared to throttle incoming connections and do all the stuff that Apache does for PHP  To be fair  Ruby on Rails has this exact problem  It is solved via two complementary mechanisms  1  Putting Ruby on Rails Node js behind a dedicated webserver  written in C and tested to hell and back  like Nginx  or Apache   Lighttd   The webserver can efficiently serve static content  access logging  rewrite URLs  terminate SSL  enforce access rules  and manage multiple sub-services  For requests that hit the actual node service  the webserver proxies the request through  2  Using a framework like Unicorn that will manage the worker processes  recycle them periodically  etc  I ve yet to find a Node js serving framework that seems fully baked  it may exist  but I haven t found it yet and still use  150 lines in my hand-rolled  production js    Reading frameworks like Express makes it seem like the standard practice is to just serve everything through one jack-of-all-trades Node js service      app use express static   dirname     public      For lower-load services and development  that s probably fine  But as soon as you try to put big time load on your service and have it run 24 7  you ll quickly discover the motivations that push big sites to have well baked  hardened C-code like Nginx fronting their site and handling all of the static content requests     until you set up a CDN  like Amazon CloudFront    For a somewhat humorous and unabashedly negative take on this  see this guy   Node js is also finding more and more non-service uses  Even if you are using something else to serve web content  you might still use Node js as a build tool  using npm modules to organize your code  Browserify to stitch it into a single asset  and uglify-js to minify it for deployment  For dealing with the web  JavaScript is a perfect impedance match and frequently that makes it the easiest route of attack  For example  if you want to grovel through a bunch of JSON response payloads  you should use my underscore-CLI module  the utility-belt of structured data   Pros   Cons    Pro  For a server guy  writing JavaScript on the backend has been a  gateway drug  to learning modern UI patterns  I no longer dread writing client code  Pro  Tends to encourage proper error checking  err is returned by virtually all callbacks  nagging the programmer to handle it   also  async js and other libraries handle the  fail if any of these subtasks fails  paradigm much better than typical synchronous code  Pro  Some interesting and normally hard tasks become trivial - like getting status on tasks in flight  communicating between workers  or sharing cache state Pro  Huge community and tons of great libraries based on a solid package manager  npm  Con  JavaScript has no standard library  You get so used to importing functionality that it feels weird when you use JSON parse or some other build in method that doesn t require adding an npm module  This means that there are five versions of everything  Even the modules included in the Node js  core  have five more variants should you be unhappy with the default implementation   This leads to rapid evolution  but also some level of confusion    Versus a simple one-process-per-request model  LAMP     Pro  Scalable to thousands of active connections  Very fast and very efficient  For a web fleet  this could mean a 10X reduction in the number of boxes required versus PHP or Ruby Pro  Writing parallel patterns is easy  Imagine that you need to fetch three  or N  blobs from Memcached  Do this in PHP     did you just write code the fetches the first blob  then the second  then the third  Wow  that s slow  There s a special PECL module to fix that specific problem for Memcached  but what if you want to fetch some Memcached data in parallel with your database query  In Node js  because the paradigm is asynchronous  having a web request do multiple things in parallel is very natural  Con  Asynchronous code is fundamentally more complex than synchronous code  and the up-front learning curve can be hard for developers without a solid understanding of what concurrent execution actually means  Still  it s vastly less difficult than writing any kind of multithreaded code with locking  Con  If a compute-intensive request runs for  for example  100 nbsp ms  it will stall processing of other requests that are being handled in the same Node js process     AKA  cooperative-multitasking  This can be mitigated with the Web Workers pattern  spinning off a subprocess to deal with the expensive task   Alternatively  you could use a large number of Node js workers and only let each one handle a single request concurrently  still fairly efficient because there is no process recycle   Con  Running a production system is MUCH more complicated than a CGI model like Apache   PHP  Perl  Ruby  etc  Unhandled exceptions will bring down the entire process  necessitating logic to restart failed workers  see cluster   Modules with buggy native code can hard-crash the process  Whenever a worker dies  any requests it was handling are dropped  so one buggy API can easily degrade service for other cohosted APIs    Versus writing a  real  service in Java   C    C  C  really     Pro  Doing asynchronous in Node js is easier than doing thread-safety anywhere else and arguably provides greater benefit  Node js is by far the least painful asynchronous paradigm I ve ever worked in  With good libraries  it is only slightly harder than writing synchronous code  Pro  No multithreading   locking bugs   True  you invest up front in writing more verbose code that expresses a proper asynchronous workflow with no blocking operations   And you need to write some tests and get the thing to work  it is a scripting language and fat fingering variable names is only caught at unit-test time   BUT  once you get it to work  the surface area for heisenbugs -- strange problems that only manifest once in a million runs -- that surface area is just much much lower  The taxes writing Node js code are heavily front-loaded into the coding phase  Then you tend to end up with stable code  Pro  JavaScript is much more lightweight for expressing functionality  It s hard to prove this with words  but JSON  dynamic typing  lambda notation  prototypal inheritance  lightweight modules  whatever     it just tends to take less code to express the same ideas  Con  Maybe you really  really like coding services in Java    For another perspective on JavaScript and Node js  check out From Java to Node js  a blog post on a Java developer s impressions and experiences learning Node js     Modules When considering node  keep in mind that your choice of JavaScript libraries will DEFINE your experience  Most people use at least two  an asynchronous pattern helper  Step  Futures  Async   and a JavaScript sugar module  Underscore js    Helper   JavaScript Sugar    Underscore js - use this  Just do it  It makes your code nice and readable with stuff like   isString    and   isArray    I m not really sure how you could write safe code otherwise  Also  for enhanced command-line-fu  check out my own Underscore-CLI    Asynchronous Pattern Modules    Step - a very elegant way to express combinations of serial and parallel actions  My personal reccomendation  See my post on what Step code looks like  Futures - much more flexible  is that really a good thing   way to express ordering through requirements  Can express things like  start a  b  c in parallel  When A  and B finish  start AB  When A  and C finish  start AC   Such flexibility requires more care to avoid bugs in your workflow  like never calling the callback  or calling it multiple times   See Raynos s post on using futures  this is the post that made me  get  futures   Async - more traditional library with one method for each pattern  I started with this before my religious conversion to step and subsequent realization that all patterns in Async could be expressed in Step with a single more readable paradigm  TameJS - Written by OKCupid  it s a precompiler that adds a new language primative  await  for elegantly writing serial and parallel workflows  The pattern looks amazing  but it does require pre-compilation  I m still making up my mind on this one  StreamlineJS - competitor to TameJS  I m leaning toward Tame  but you can make up your own mind    Or to read all about the asynchronous libraries  see this panel-interview with the authors   Web Framework    Express Great Ruby on Rails-esk framework for organizing web sites  It uses JADE as a XML HTML templating engine  which makes building HTML far less painful  almost elegant even  jQuery While not technically a node module  jQuery is quickly becoming a de-facto standard for client-side user interface  jQuery provides CSS-like selectors to  query  for sets of DOM elements that can then be operated on  set handlers  properties  styles  etc   Along the same vein  Twitter s Bootstrap CSS framework  Backbone js for an MVC pattern  and Browserify js to stitch all your JavaScript files into a single file  These modules are all becoming de-facto standards so you should at least check them out if you haven t heard of them    Testing    JSHint - Must use  I didn t use this at first which now seems incomprehensible  JSLint adds back a bunch of the basic verifications you get with a compiled language like Java  Mismatched parenthesis  undeclared variables  typeos of many shapes and sizes  You can also turn on various forms of what I call  anal mode  where you verify style of whitespace and whatnot  which is OK if that s your cup of tea -- but the real value comes from getting instant feedback on the exact line number where you forgot a closing         without having to run your code and hit the offending line   JSHint  is a more-configurable variant of Douglas Crockford s JSLint  Mocha competitor to Vows which I m starting to prefer  Both frameworks handle the basics well enough  but complex patterns tend to be easier to express in Mocha  Vows Vows is really quite elegant  And it prints out a lovely report  --spec  showing you which test cases passed   failed  Spend 30 minutes learning it  and you can create basic tests for your modules with minimal effort  Zombie - Headless testing for HTML and JavaScript using JSDom as a virtual  browser   Very powerful stuff  Combine it with Replay to get lightning fast deterministic tests of in-browser code  A comment on how to  think about  testing    Testing is non-optional  With a dynamic language like JavaScript  there are very few static checks  For example  passing two parameters to a method that expects 4 won t break until the code is executed  Pretty low bar for creating bugs in JavaScript  Basic tests are essential to making up the verification gap with compiled languages  Forget validation  just make your code execute  For every method  my first validation case is  nothing breaks   and that s the case that fires most often  Proving that your code runs without throwing catches 80  of the bugs and will do so much to improve your code confidence that you ll find yourself going back and adding the nuanced validation cases you skipped  Start small and break the inertial barrier  We are all lazy  and pressed for time  and it s easy to see testing as  extra work   So start small  Write test case 0 - load your module and report success  If you force yourself to do just this much  then the inertial barrier to testing is broken  That s  lt 30 min to do it your first time  including reading the documentation  Now write test case 1 - call one of your methods and verify  nothing breaks   that is  that you don t get an error back  Test case 1 should take you less than one minute  With the inertia gone  it becomes easy to incrementally expand your test coverage  Now evolve your tests with your code  Don t get intimidated by what the  correct  end-to-end test would look like with mock servers and all that  Code starts simple and evolves to handle new cases  tests should too  As you add new cases and new complexity to your code  add test cases to exercise the new code  As you find bugs  add verifications and   or new cases to cover the flawed code  When you are debugging and lose confidence in a piece of code  go back and add tests to prove that it is doing what you think it is  Capture strings of example data  from other services you call  websites you scrape  whatever  and feed them to your parsing code  A few cases here  improved validation there  and you will end up with highly reliable code     Also  check out the official list of recommended Node js modules  However  GitHub s Node Modules Wiki is much more complete and a good resource     To understand Node  it s helpful to consider a few of the key design choices   Node js is EVENT BASED and ASYNCHRONOUS   NON-BLOCKING  Events  like an incoming HTTP connection will fire off a JavaScript function that does a little bit of work and kicks off other asynchronous tasks like connecting to a database or pulling content from another server  Once these tasks have been kicked off  the event function finishes and Node js goes back to sleep  As soon as something else happens  like the database connection being established or the external server responding with content  the callback functions fire  and more JavaScript code executes  potentially kicking off even more asynchronous tasks  like a database query   In this way  Node js will happily interleave activities for multiple parallel workflows  running whatever activities are unblocked at any point in time  This is why Node js does such a great job managing thousands of simultaneous connections   Why not just use one process thread per connection like everyone else  In Node js  a new connection is just a very small heap allocation  Spinning up a new process takes significantly more memory  a megabyte on some platforms  But the real cost is the overhead associated with context-switching  When you have 10 6 kernel threads  the kernel has to do a lot of work figuring out who should execute next  A bunch of work has gone into building an O 1  scheduler for Linux  but in the end  it s just way way more efficient to have a single event-driven process than 10 6 processes competing for CPU time  Also  under overload conditions  the multi-process model behaves very poorly  starving critical administration and management services  especially SSHD  meaning you can t even log into the box to figure out how screwed it really is    Node js is SINGLE THREADED and LOCK FREE  Node js  as a very deliberate design choice only has a single thread per process  Because of this  it s fundamentally impossible for multiple threads to access data simultaneously  Thus  no locks are needed  Threads are hard  Really really hard  If you don t believe that  you haven t done enough threaded programming  Getting locking right is hard and results in bugs that are really hard to track down  Eliminating locks and multi-threading makes one of the nastiest classes of bugs just go away  This might be the single biggest advantage of node   But how do I take advantage of my 16 core box   Two ways    For big heavy compute tasks like image encoding  Node js can fire up child processes or send messages to additional worker processes  In this design  you d have one thread managing the flow of events and N processes doing heavy compute tasks and chewing up the other 15 CPUs  For scaling throughput on a webservice  you should run multiple Node js servers on one box  one per core  using cluster  With Node js v0 6 x  the official  cluster  module linked here replaces the learnboost version which has a different API   These local Node js servers can then compete on a socket to accept new connections  balancing load across them  Once a connection is accepted  it becomes tightly bound to a single one of these shared processes  In theory  this sounds bad  but in practice it works quite well and allows you to avoid the headache of writing thread-safe code  Also  this means that Node js gets excellent CPU cache affinity  more effectively using memory bandwidth    Node js lets you do some really powerful things without breaking a sweat  Suppose you have a Node js program that does a variety of tasks  listens on a TCP port for commands  encodes some images  whatever  With five lines of code  you can add in an HTTP based web management portal that shows the current status of active tasks  This is EASY to do   var http   require  http    http createServer function  req  res        res writeHead 200    Content-Type    text plain         res end myJavascriptObject getSomeStatusInfo        listen 1337   127 0 0 1      Now you can hit a URL and check the status of your running process  Add a few buttons  and you have a  management portal   If you have a running Perl   Python   Ruby script  just  throwing in a management portal  isn t exactly simple   But isn t JavaScript slow   bad   evil   spawn-of-the-devil  JavaScript has some weird oddities  but with  the good parts  there s a very powerful language there  and in any case  JavaScript is THE language on the client  browser   JavaScript is here to stay  other languages are targeting it as an IL  and world class talent is competing to produce the most advanced JavaScript engines  Because of JavaScript s role in the browser  an enormous amount of engineering effort is being thrown at making JavaScript blazing fast   V8 is the latest and greatest javascript engine  at least for this month  It blows away the other scripting languages in both efficiency AND stability  looking at you  Ruby   And it s only going to get better with huge teams working on the problem at Microsoft  Google  and Mozilla  competing to build the best JavaScript engine  It s no longer a JavaScript  interpreter  as all the modern engines do tons of JIT compiling under the hood with interpretation only as a fallback for execute-once code    Yeah  we all wish we could fix a few of the odder JavaScript language choices  but it s really not that bad  And the language is so darn flexible that you really aren t coding JavaScript  you are coding Step or jQuery -- more than any other language  in JavaScript  the libraries define the experience  To build web applications  you pretty much have to know JavaScript anyway  so coding with it on the server has a sort of skill-set synergy   It has made me not dread writing client code   Besides  if you REALLY hate JavaScript  you can use syntactic sugar like CoffeeScript  Or anything else that creates JavaScript code  like Google nbsp Web nbsp Toolkit  GWT    Speaking of JavaScript  what s a  closure   - Pretty much a fancy way of saying that you retain lexically scoped variables across call chains     Like this   var myData    foo   database connect   user pass   function myCallback  result         database query  SELECT   from Foo where id       myData           Note that doSomethingElse   executes  BEFORE   database query  which is inside a callback doSomethingElse      See how you can just use  myData  without doing anything awkward like stashing it into an object  And unlike in Java  the  myData  variable doesn t have to be read-only  This powerful language feature makes asynchronous-programming much less verbose and less painful   Writing asynchronous code is always going to be more complex than writing a simple single-threaded script  but with Node js  it s not that much harder and you get a lot of benefits in addition to the efficiency and scalability to thousands of concurrent connections

User · Answer

The closures are a way to execute code in the context it was created in   What this means for concurency is that you can define variables  then initiate a nonblocking I O function  and send it an anonymous function for its callback   When the task is complete  the callback function will execute in the context with the variables  this is the closure   The reason closures are so good for writing applications with nonblocking I O is that it s very easy to manage the context of functions executing asynchronously

User · Answer

Well  I understand that         Node s goal is to provide an easy way   to build scalable network programs    Node is similar in design to and influenced by systems like Ruby s Event Machine or Python s Twisted     Evented I O for V8 javascript       For me that means that you were correct in all three assumptions  The library sure looks promising

User · Answer

Two good examples are regarding how you manage templates and use progressive enhancements with it  You just need a few lightweight pieces of JavaScript code to make it work perfectly   I strongly recommend that you watch and read these articles    Video glass node Node js YUI DOM manipulation   Pick up any language and try to remember how you would manage your HTML file templates and what you had to do to update a single CSS class name in your DOM structure  for instance  a user clicked on a menu item and you want that marked as  selected  and update the content of the page    With Node js it is as simple as doing it in client-side JavaScript code  Get your DOM node and apply your CSS class to that  Get your DOM node and innerHTML your content  you will need some additional JavaScript code to do this  Read the article to know more    Another good example  is that you can make your web page compatible both with JavaScript turned on or off with the same piece of code  Imagine you have a date selection made in JavaScript that would allow your users to pick up any date using a calendar  You can write  or use  the same piece of JavaScript code to make it work with your JavaScript  turned ON or OFF

User · Answer

Q  The programming model is event driven  especially the way it handles I O   Correct  It uses call-backs  so any request to access the file system would cause a request to be sent to the file system and then Node js would start processing its next request  It would only worry about the I O request once it gets a response back from the file system  at which time it will run the callback code  However  it is possible to make synchronous I O requests  that is  blocking requests   It is up to the developer to choose between asynchronous  callbacks  or synchronous  waiting    Q  It uses JavaScript and the parser is V8   Yes  Q  It can be easily used to create concurrent server applications   Yes  although you d need to hand-code quite a lot of JavaScript  It might be better to look at a framework  such as http   www easynodejs com  - which comes with full online documentation and a sample application

User · Answer

Also  do not forget to mention that Google s V8 is VERY fast  It actually converts the JavaScript code to machine code with the matched performance of compiled binary  So along with all the other great things  it s INSANELY fast

User · Answer

V8 is an implementation of JavaScript  It lets you run standalone JavaScript applications  among other things    Node js is simply a library written for V8 which does evented I O  This concept is a bit trickier to explain  and I m sure someone will answer with a better explanation than I    The gist is that rather than doing some input or output and waiting for it to happen  you just don t wait for it to finish  So for example  ask for the last edited time of a file      Pseudo code stat   somefile      That might take a couple of milliseconds  or it might take seconds  With evented I O you simply fire off the request and instead of waiting around you attach a callback that gets run when the request finishes      Pseudo code stat   somefile   function  result          Use the result here            more code here   This makes it a lot like JavaScript code in the browser  for example  with Ajax style functionality    For more information  you should check out the article Node js is genuinely exciting which was my introduction to the library platform    I found it quite good

User · Answer

I think the advantages are    Web development in a dynamic language  JavaScript  on a VM that is incredibly fast  V8   It is much faster than Ruby  Python  or Perl  Ability to handle thousands of concurrent connections with minimal overhead on a single process  JavaScript is perfect for event loops with first class function objects and closures  People already know how to use it this way having used it in the browser to respond to user initiated events  A lot of people already know JavaScript  even people who do not claim to be programmers  It is arguably the most popular programming language  Using JavaScript on a web server as well as the browser reduces the impedance mismatch between the two programming environments which can communicate data structures via JSON that work the same on both sides of the equation  Duplicate form validation code can be shared between server and client  etc

User · Answer

There is a very good fast food place analogy that best explains the event driven model of Node js  see the full article  Node js  Doctor   s Offices and Fast Food Restaurants     Understanding Event-driven Programming  Here is a summary      If the fast food joint followed a traditional thread-based model  you d order your food and wait in line until you received it  The person behind you wouldn t be able to order until your order was done  In an event-driven model  you order your food and then get out of line to wait  Everyone else is then free to order    Node js is event-driven  but most web servers are thread-based York explains how Node js works    You use your web browser to make a request for   about html  on a Node js web server  The Node js server accepts your request and calls a function to retrieve that file from disk  While the Node js server is waiting for the file to be retrieved  it services the next web request  When the file is retrieved  there is a callback function that is inserted in the Node js servers queue  The Node js server executes that function which in this case would render the   about html  page and send it back to your web browser

[javascript] What is Node.js?

Examples related to javascript

Examples related to node.js

Examples related to v8

Examples related to evented-io