Node js Best Practice Exception Handling

Question

I just started trying out node js a few days ago  I ve realized that the Node is terminated whenever I have an unhandled exception in my program  This is different than the normal server container that I have been exposed to where only the Worker Thread dies when unhandled exceptions occur and the container would still be able to receive the request  This raises a few questions    Is process on  uncaughtException   the only effective way to guard against it   Will process on  uncaughtException   catch the unhandled exception during execution of asynchronous processes as well  Is there a module that is already built  such as sending email or writing to a file  that I could leverage in the case of uncaught exceptions    I would appreciate any pointer article that would show me the common best practices for handling uncaught exceptions in node js

User · Answer

I would just like to add that Step js library helps you handle exceptions by always passing it to the next step function  Therefore you can have as a last step a function that check for any errors in any of the previous steps  This approach can greatly simplify your error handling   Below is a quote from the github page      any exceptions thrown are caught and passed as the first argument to   the next function  As long as you don t nest callback functions inline   your main functions this prevents there from ever being any uncaught   exceptions  This is very important for long running node JS servers   since a single uncaught exception can bring the whole server down    Furthermore  you can use Step to control execution of scripts to have a clean up section as the last step  For example if you want to write a build script in Node and report how long it took to write  the last step can do that  rather than trying to dig out the last callback

User · Answer

One instance where using a try-catch might be appropriate is when using a forEach loop  It is synchronous but at the same time you cannot just use a return statement in the inner scope  Instead a try and catch approach can be used to return an Error object in the appropriate scope  Consider    function processArray         try            1  2  3  forEach function     throw new Error  exception               catch  e            return e             It is a combination of the approaches described by  balupton above

User · Answer

I wrote about this recently at http   snmaynard com 2012 12 21 node-error-handling   A new feature of node in version 0 8 is domains and allow you to combine all the forms of error handling into one easier manage form  You can read about them in my post   You can also use something like Bugsnag to track your uncaught exceptions and be notified via email  chatroom or have a ticket created for an uncaught exception  I am the co-founder of Bugsnag

User · Answer

nodejs domains is the most up to date way of handling errors in nodejs  Domains can capture both error other events as well as traditionally thrown objects  Domains also provide functionality for handling callbacks with an error passed as the first argument via the intercept method   As with normal try catch-style error handling  is is usually best to throw errors when they occur  and block out areas where you want to isolate errors from affecting the rest of the code  The way to  block out  these areas are to call domain run with a function as a block of isolated code    In synchronous code  the above is enough - when an error happens you either let it be thrown through  or you catch it and handle there  reverting any data you need to revert   try         something   catch e         handle data reversion      probably log too     When the error happens in an asynchronous callback  you either need to be able to fully handle the rollback of data  shared state  external data like databases  etc   OR you have to set something to indicate that an exception has happened - where ever you care about that flag  you have to wait for the callback to complete   var err   null  var d   require  domain   create    d on  error   function e      err   e       any additional error handling   d run function     Fiber function          do stuff   var future   somethingAsynchronous         more stuff    future wait       here we care about the error   if err    null           handle data reversion        probably log too              Some of that above code is ugly  but you can create patterns for yourself to make it prettier  eg   var specialDomain   specialDomain function          do stuff   var future   somethingAsynchronous         more stuff    future wait       here we care about the error   if specialDomain error             handle data reversion        probably log too         function         catch       any additional error handling       UPDATE  2013-09    Above  I use a future that implies fibers semantics  which allow you to wait on futures in-line  This actually allows you to use traditional try-catch blocks for everything - which I find to be the best way to go  However  you can t always do this  ie in the browser      There are also futures that don t require fibers semantics  which then work with normal  browsery JavaScript   These can be called futures  promises  or deferreds  I ll just refer to futures from here on   Plain-old-JavaScript futures libraries allow errors to be propagated between futures  Only some of these libraries allow any thrown future to be correctly handled  so beware    An example   returnsAFuture   then function       console log  1     return doSomething      also returns a future     then function       console log  2     throw Error  oops an error was thrown       then function       console log  3       catch function exception      console log  handler        handle the exception    done     This mimics a normal try-catch  even though the pieces are asynchronous  It would print   1 2 handler   Note that it doesn t print  3  because an exception was thrown that interrupts that flow   Take a look at bluebird promises    https   github com petkaantonov bluebird   Note that I haven t found many other libraries other than these that properly handle thrown exceptions  jQuery s deferred  for example  don t - the  fail  handler would never get the exception thrown an a  then  handler  which in my opinion is a deal breaker

User · Answer

If you want use Services in Ubuntu Upstart   Node as a service in Ubuntu 11 04 with upstart  monit and forever js

User · Answer

You can catch uncaught exceptions  but it s of limited use  See http   debuggable com posts node-js-dealing-with-uncaught-exceptions 4c933d54-1428-443c-928d-4e1ecbdd56cb  monit  forever or upstart can be used to restart node process when it crashes  A graceful shutdown is best you can hope for  e g  save all in-memory data in uncaught exception handler

User · Answer

Catching errors has been very well discussed here  but it s worth remembering to log the errors out somewhere so you can view them and fix stuff up    Bunyan is a popular logging framework for NodeJS - it supporst writing out to a bunch of different output places which makes it useful for local debugging  as long as you avoid console log    In your domain s error handler you could spit the error out to a log file   var log   bunyan createLogger     name   myapp     streams                level   error         path    var tmp myapp-error log      log ERROR to this file                 This can get time consuming if you have lots of errors and or servers to check  so it could be worth looking into a tool like Raygun  disclaimer  I work at Raygun  to group errors together - or use them both together    If you decided to use Raygun as a tool  it s pretty easy to setup too  var raygunClient   new raygun Client   init   apiKey   your API key      raygunClient send theError       Crossed with using a tool like PM2 or forever  your app should be able to crash  log out what happened and reboot without any major issues

User · Answer

After reading this post some time ago I was wondering if it was safe to use domains for exception handling on an api   function level  I wanted to use them to simplify exception handling code in each async function I wrote  My concern was that using a new domain for each function would introduce significant overhead  My homework seems to indicate that there is minimal overhead and that performance is actually better with domains than with try catch in some situations   http   www lighthouselogic com   using-a-new-domain-for-each-async-function-in-node

User · Answer

Following is a summarization and curation from many different sources on this topic including code example and quotes from selected blog posts  The complete list of best practices can be found here    Best practices of Node JS error handling    Number1  Use promises for async error handling  TL DR  Handling async errors in callback style is probably the fastest way to hell  a k a the pyramid of doom   The best gift you can give to your code is using instead a reputable promise library which provides much compact and familiar code syntax like try-catch  Otherwise  Node JS callback style  function err  response   is a promising way to un-maintainable code due to the mix of error handling with casual code  excessive nesting and awkward coding patterns  Code example - good  doWork    then doWork   then doError   then doWork   catch errorHandler   then verify     code example anti pattern     callback style error handling  getData someParameter  function err  result       if err    null          do something like calling the given callback function and pass the error     getMoreData a  function err  result             if err    null                do something like calling the given callback function and pass the error         getMoreData b  function c                    getMoreData d  function e                                                                                          Blog quote   We have a problem with promises   From the blog pouchdb  ranked 11 for the keywords  Node Promises           And in fact  callbacks do something even more sinister  they deprive us of the stack  which is something we usually take for granted in programming languages  Writing code without a stack is a lot like driving a car without a brake pedal  you don   t realize how badly you need it  until you reach for it and it   s not there  The whole point of promises is to give us back the language fundamentals we lost when we went async  return  throw  and the stack  But you have to know how to use promises correctly in order to take advantage of them       Number2  Use only the built-in Error object  TL DR  It pretty common to see code that throws errors as string or as a custom type     this complicates the error handling logic and the interoperability between modules  Whether you reject a promise  throw exception or emit error     using Node JS built-in Error object increases uniformity and prevents loss of error information  Otherwise  When executing some module  being uncertain which type of errors come in return     makes it much harder to reason about the coming exception and handle it  Even worth  using custom types to describe errors might lead to loss of critical error information like the stack trace   Code example - doing it right        throwing an Error from typical function  whether sync or async  if  productToAdd   throw new Error  How can I add new product when no value provided         throwing  an Error from EventEmitter const myEmitter   new MyEmitter    myEmitter emit  error   new Error  whoops          throwing  an Error from a Promise  return new promise function  resolve  reject     DAL getProduct productToAdd id  then  existingProduct    gt    if existingProduct    null   return reject new Error  Why fooling us and trying to add an existing product        code example anti pattern    throwing a String lacks any stack trace information and other important properties if  productToAdd      throw   How can I add new product when no value provided       Blog quote   A string is not an error   From the blog devthought  ranked 6 for the keywords    Node JS error object             passing a string instead of an error results in reduced interoperability between modules  It breaks contracts with APIs that might be performing instanceof Error checks  or that want to know more about the error  Error objects  as we   ll see  have very interesting properties in modern JavaScript engines besides holding the message passed to the constructor        Number3  Distinguish operational vs programmer errors  TL DR  Operations errors  e g  API received an invalid input  refer to known cases where the error impact is fully understood and can be handled thoughtfully  On the other hand  programmer error  e g  trying to read undefined variable  refers to unknown code failures that dictate to gracefully restart the application  Otherwise  You may always restart the application when an error appear  but why letting  5000 online users down because of a minor and predicted error  operational error   the opposite is also not ideal     keeping the application up when unknown issue  programmer error  occurred might lead unpredicted behavior  Differentiating the two allows acting tactfully and applying a balanced approach based on the given context  Code example - doing it right        throwing an Error from typical function  whether sync or async  if  productToAdd   throw new Error  How can I add new product when no value provided         throwing  an Error from EventEmitter const myEmitter   new MyEmitter    myEmitter emit  error   new Error  whoops          throwing  an Error from a Promise  return new promise function  resolve  reject     DAL getProduct productToAdd id  then  existingProduct    gt    if existingProduct    null   return reject new Error  Why fooling us and trying to add an existing product        code example - marking an error as operational  trusted     marking an error object as operational  var myError   new Error  How can I add new product when no value provided     myError isOperational   true     or if you re using some centralized error factory  see other examples at the bullet  Use only the built-in Error object   function appError commonType  description  isOperational        Error call this       Error captureStackTrace this       this commonType   commonType      this description   description      this isOperational   isOperational      throw new appError errorManagement commonErrors InvalidInput   Describe here what happened   true      error handling code within middleware process on  uncaughtException   function error        if  error isOperational          process exit 1         Blog Quote   Otherwise you risk the state   From the blog debugable  ranked 3 for the keywords  Node JS uncaught exception           By the very nature of how throw works in JavaScript  there is almost never any way to safely    pick up where you left off     without leaking references  or creating some other sort of undefined brittle state  The safest way to respond to a thrown error is to shut down the process  Of course  in a normal web server  you might have many connections open  and it is not reasonable to abruptly shut those down because an error was triggered by someone else  The better approach is to send an error response to the request that triggered the error  while letting the others finish in their normal time  and stop listening for new requests in that worker      Number4  Handle errors centrally  through but not within middleware  TL DR  Error handling logic such as mail to admin and logging should be encapsulated in a dedicated and centralized object that all end-points  e g  Express middleware  cron jobs  unit-testing  call when an error comes in   Otherwise  Not handling errors within a single place will lead to code duplication and probably to errors that are handled improperly  Code example - a typical error flow    DAL layer  we don t handle errors here DB addDocument newCustomer   error  result    gt        if  error          throw new Error  Great error explanation comes here   other useful parameters         API route code  we catch both sync and async errors and forward to the middleware try       customerService addNew req body  then function  result            res status 200  json result          catch  error    gt            next error            catch  error        next error        Error handling middleware  we delegate the handling to the centrzlied error handler app use function  err  req  res  next        errorHandler handleError err  then  isOperationalError    gt            if   isOperationalError              next err                 Blog quote   Sometimes lower levels can   t do anything useful except propagate the error to their caller   From the blog Joyent  ranked 1 for the keywords    Node JS error handling             You may end up handling the same error at several levels of the stack  This happens when lower levels can   t do anything useful except propagate the error to their caller  which propagates the error to its caller  and so on  Often  only the top-level caller knows what the appropriate response is  whether that   s to retry the operation  report an error to the user  or something else  But that doesn   t mean you should try to report all errors to a single top-level callback  because that callback itself can   t know in what context the error occurred      Number5  Document API errors using Swagger  TL DR  Let your API callers know which errors might come in return so they can handle these thoughtfully without crashing  This is usually done with REST API documentation frameworks like Swagger  Otherwise  An API client might decide to crash and restart only because he received back an error he couldn   t understand  Note  the caller of your API might be you  very typical in a microservices environment   Blog quote   You have to tell your callers what errors can happen   From the blog Joyent  ranked 1 for the keywords    Node JS logging            We   ve talked about how to handle errors  but when you   re writing a new function  how do you deliver errors to the code that called your function     If you don   t know what errors can happen or don   t know what they mean  then your program cannot be correct except by accident  So if you   re writing a new function  you have to tell your callers what errors can happen and what they mea     Number6  Shut the process gracefully when a stranger comes to town  TL DR  When an unknown error occurs  a developer error  see best practice number  3 - there is uncertainty about the application healthiness  A common practice suggests restarting the process carefully using a    restarter    tool like Forever and PM2  Otherwise  When an unfamiliar exception is caught  some object might be in a faulty state  e g an event emitter which is used globally and not firing events anymore due to some internal failure  and all future requests might fail or behave crazily  Code example - deciding whether to crash    deciding whether to crash when an uncaught exception arrives   Assuming developers mark known operational errors with error isOperational true  read best practice  3 process on  uncaughtException   function error     errorManagement handler handleError error    if  errorManagement handler isTrustedError error    process exit 1          centralized error handler encapsulates error-handling related logic  function errorHandler     this handleError   function  error     return logger logError err  then sendMailToAdminIfCritical  then saveInOpsQueueIfCritical  then determineIfOperationalError        this isTrustedError   function error      return error isOperational       Blog quote   There are three schools of thoughts on error handling   From the blog jsrecipes         There are primarily three schools of thoughts on error handling  1  Let the application crash and restart it  2  Handle all possible errors and never crash  3  Balanced approach between the two     Number7  Use a mature logger to increase errors visibility  TL DR  A set of mature logging tools like Winston  Bunyan or Log4J  will speed-up error discovery and understanding  So forget about console log   Otherwise  Skimming through console logs or manually through messy text file without querying tools or a decent log viewer might keep you busy at work until late  Code example - Winston logger in action    your centralized logger object var logger   new winston Logger    level   info    transports     new  winston transports Console      new  winston transports File    filename   somefile log                custom code somewhere using the logger logger log  info    Test Log Message with some parameter  s    some parameter     anything   This is metadata        Blog quote   Lets identify a few requirements  for a logger     From the blog strongblog         Lets identify a few requirements  for a logger     1  Time stamp each log line  This one is pretty self explanatory     you should be able to tell when each log entry occured    2  Logging format should be easily digestible by humans as well as machines    3  Allows for multiple configurable destination streams  For example  you might be writing trace logs to one file but when an error is encountered  write to the same file  then into error file and send an email at the same time        Number8  Discover errors and downtime using APM products  TL DR  Monitoring and performance products  a k a APM  proactively gauge your codebase or API so they can auto-magically highlight errors  crashes and slow parts that you were missing  Otherwise  You might spend great effort on measuring API performance and downtimes  probably you   ll never be aware which are your slowest code parts under real world scenario and how these affects the UX  Blog quote   APM products segments   From the blog Yoni Goldberg          APM products constitutes 3 major segments 1  Website or API monitoring     external services that constantly monitor uptime and performance via HTTP requests  Can be setup in few minutes  Following are few selected contenders  Pingdom  Uptime Robot  and New Relic   2  Code instrumentation     products family which require to embed an agent within the application to benefit feature slow code detection  exceptions statistics  performance monitoring and many more  Following are few selected contenders  New Relic  App Dynamics   3  Operational intelligence dashboard     these line of products are focused on facilitating the ops team with metrics and curated content that helps to easily stay on top of application performance  This is usually involves aggregating multiple sources of information  application logs  DB logs  servers log  etc  and upfront dashboard design work  Following are few selected contenders  Datadog  Splunk      The above is a shortened version - see here more best practices and examples

User · Answer

Update  Joyent now has their own guide  The following information is more of a summary   Safely  throwing  errors  Ideally we d like to avoid uncaught errors as much as possible  as such  instead of literally throwing the error  we can instead safely  throw  the error using one of the following methods depending on our code architecture    For synchronous code  if an error happens  return the error      Define divider as a syncrhonous function var divideSync   function x y           if error condition      if   y     0                 throw  the error safely by returning it         return new Error  Can t divide by zero             else              no error occured  continue on         return x y             Divide 4 2 var result   divideSync 4 2     did an error occur  if   result instanceof Error            handle the error safely     console log  4 2 err   result    else          no error occured  continue on     console log  4 2   result        Divide 4 0 result   divideSync 4 0     did an error occur  if   result instanceof Error            handle the error safely     console log  4 0 err   result    else          no error occured  continue on     console log  4 0   result     For callback-based  ie  asynchronous  code  the first argument of the callback is err  if an error happens err is the error  if an error doesn t happen then err is null  Any other arguments follow the err argument   var divide   function x y next           if error condition      if   y     0                 throw  the error safely by calling the completion callback            with the first argument being the error         next new Error  Can t divide by zero              else              no error occured  continue on         next null  x y           divide 4 2 function err result          did an error occur      if   err                handle the error safely         console log  4 2 err   err            else              no error occured  continue on         console log  4 2   result            divide 4 0 function err result          did an error occur      if   err                handle the error safely         console log  4 0 err   err            else              no error occured  continue on         console log  4 0   result            For eventful code  where the error may happen anywhere  instead of throwing the error  fire the error event instead      Definite our Divider Event Emitter var events   require  events   var Divider   function        events EventEmitter call this    require  util   inherits Divider  events EventEmitter      Add the divide function Divider prototype divide   function x y          if error condition      if   y     0                 throw  the error safely by emitting it         var err   new Error  Can t divide by zero           this emit  error   err            else              no error occured  continue on         this emit  divided   x  y  x y                Chain     return this        Create our divider and listen for errors var divider   new Divider   divider on  error   function err          handle the error safely     console log err     divider on  divided   function x y result       console log x     y     result         Divide divider divide 4 2  divide 4 0     Safely  catching  errors  Sometimes though  there may still be code that throws an error somewhere which can lead to an uncaught exception and a potential crash of our application if we don t catch it safely  Depending on our code architecture we can use one of the following methods to catch it    When we know where the error is occurring  we can wrap that section in a node js domain  var d   require  domain   create   d on  error   function err          handle the error safely     console log err         catch the uncaught errors in this asynchronous or synchronous code block d run function           the asynchronous or synchronous code that we want to catch thrown errors on     var err   new Error  example       throw err     If we know where the error is occurring is synchronous code  and for whatever reason can t use domains  perhaps old version of node   we can use the try catch statement      catch the uncaught errors in this synchronous code block    try catch statements only work on synchronous code try          the synchronous code that we want to catch thrown errors on     var err   new Error  example       throw err   catch  err           handle the error safely     console log err      However  be careful not to use try   catch in asynchronous code  as an asynchronously thrown error will not be caught   try       setTimeout function            var err   new Error  example           throw err        1000    catch  err           Example error won t be caught here    crashing our app        hence the need for domains     If you do want to work with try  catch in conjunction with asynchronous code  when running Node 7 4 or higher you can use async await natively to write your asynchronous functions   Another thing to be careful about with try   catch is the risk of wrapping your completion callback inside the try statement like so   var divide   function x y next           if error condition      if   y     0                 throw  the error safely by calling the completion callback            with the first argument being the error         next new Error  Can t divide by zero              else              no error occured  continue on         next null  x y           var continueElsewhere   function err  result           throw new Error  elsewhere has failed      try           divide 4  2  continueElsewhere               the execution of divide  and the execution of               continueElsewhere will be inside the try statement   catch  err            console log err stack               will output the  unexpected  result of  elsewhere has failed     This gotcha is very easy to do as your code becomes more complex  As such  it is best to either use domains or to return errors to avoid  1  uncaught exceptions in asynchronous code  2  the try catch catching execution that you don t want it to  In languages that allow for proper threading instead of JavaScript s asynchronous event-machine style  this is less of an issue  Finally  in the case where an uncaught error happens in a place that wasn t wrapped in a domain or a try catch statement  we can make our application not crash by using the uncaughtException listener  however doing so can put the application in an unknown state       catch the uncaught errors that weren t wrapped in a domain or try catch statement    do not use this in modules  but only in applications  as otherwise we could have multiple of these bound process on  uncaughtException   function err           handle the error safely     console log err         the asynchronous or synchronous code that emits the otherwise uncaught error var err   new Error  example   throw err

[node.js] Node.js Best Practice Exception Handling

Examples related to node.js

Examples related to exception-handling

Examples related to serverside-javascript