Should I check in folder node modules to Git when creating a Node js app on Heroku

Question

I followed the basic getting started instructions for Node js on Heroku here  https   devcenter heroku com categories nodejs These instruction don t tell you to create a  gitignore node modules  and therefore imply that folder node modules should be checked in to Git   When I included node modules in Git repository  my getting started application ran correctly  When I followed the more advanced example at   Building a Real-time  Polyglot Application with Node js  Ruby  MongoDB and Socket IO https   github com mongolab tractorpush-server   source   It instructed me to add folder node modules to file  gitignore  So I removed folder node modules from Git  added it to file  gitignore  and then redeployed  This time the deployed failed like so  ----- gt  Heroku receiving push ----- gt  Node js app detected ----- gt  Resolving engine versions        Using Node js version  0 8 2        Using npm version  1 0 106 ----- gt  Fetching Node js binaries ----- gt  Vendoring node into slug ----- gt  Installing dependencies with npm        Error  npm doesn t work with node v0 8 2        Required  node 0 4    0 5    0 6            at  tmp node-npm-5iGk bin npm-cli js 57 23            at Object  lt anonymous gt    tmp node-npm-5iGk bin npm-cli js 77 3             at Module  compile  module js 449 26             at Object Module  extensions  js  module js 467 10             at Module load  module js 356 32             at Function Module  load  module js 312 12             at Module require  module js 362 17             at require  module js 378 17             at Object  lt anonymous gt    tmp node-npm-5iGk cli js 2 1             at Module  compile  module js 449 26         Error  npm doesn t work with node v0 8 2        Required  node 0 4    0 5    0 6            at  tmp node-npm-5iGk bin npm-cli js 57 23            at Object  lt anonymous gt    tmp node-npm-5iGk bin npm-cli js 77 3             at Module  compile  module js 449 26             at Object Module  extensions  js  module js 467 10             at Module load  module js 356 32             at Function Module  load  module js 312 12             at Module require  module js 362 17             at require  module js 378 17             at Object  lt anonymous gt    tmp node-npm-5iGk cli js 2 1             at Module  compile  module js 449 26         Dependencies installed ----- gt  Discovering process types        Procfile declares types - gt  mongod  redis  web ----- gt  Compiled slug size is 5 0MB ----- gt  Launching    done  v9  Running  quot heroku ps quot  confirms the crash  OK  no problem  so I rolled back the change  added folder node module back to the Git repository and removed it from file  gitignore  However  even after reverting  I still get the same error message on deploy  but now the application is running correctly again  Running  quot heroku ps quot  tells me the application is running  What s the right way to do this  Include folder node modules or not   And why would I still be getting the error message when I rollback  My guess is the Git repository is in a bad state on the Heroku side

User · Answer

Scenario 1  One scenario  You use a package that gets removed from npm  If you have all the modules in the folder node modules  then it won t be a problem for you  If you do only have the package name in the package json  you can t get it anymore  If a package is less than 24 hours old  you can easily remove it from npm  If it s older than 24 hours old  then you need to contact them  But   If you contact support  they will check to see if removing that version of your package would break any other installs  If so  we will not remove it   read more So the chances for this are low  but there is scenario 2     Scenario 2  An other scenario where this is the case  You develop an enterprise version of your software or a very important software and write in your package json   quot dependencies quot          quot studpid-package quot    quot  1 0 1 quot     You use the method function1 x of that package  Now the developers of studpid-package rename the method function1 x to function2 x  and they make a fault    They change the version of their package from 1 0 1 to 1 1 0  That s a problem because when you call npm install the next time  you will accept version 1 1 0 because you used the tilde   quot studpid-package quot    quot  1 0 1 quot    Calling function1 x  can cause errors and problems now   Pushing the whole node modules folder  often more than 100 MB  to your repository  will cost you memory space  A few kb  package json only  compared with hundreds of MB  package json  amp  node modules     Think about it  You could do it   should think about it if   the software is very important   it costs you money when something fails   you don t trust the npm registry  npm is centralized and could theoretically be shut down    You don t need to publish the node modules folder in 99 9  of the cases if   you develop a software just for yourself   you ve programmed something and just want to publish the result on GitHub because someone else could maybe be interested in it     If you don t want the node modules to be in your repository  just create a  gitignore file and add the line node modules

User · Answer

From  quot node modules quot  in Git   To recap   Only checkin node modules for applications you deploy  not reusable packages you maintain  Any compiled dependencies should have their source checked in  not the compile targets  and should   npm rebuild on deploy    My favorite part   All you people who added node modules to your gitignore  remove that shit  today  it   s an artifact of an era we   re all too happy to leave behind  The era of global modules is dead    The original link was this one  but it is now dead  Thanks  Flavio for pointing it out

User · Answer

I believe that npm install should not run in a production environment  There are several things that can go wrong - npm outage  download of newer dependencies  shrinkwrap seems to have solved this  are two of them  On the other hand  folder node modules should not be committed to Git  Apart from their big size  commits including them can become distracting  The best solutions would be this  npm install should run in a CI environment that is similar to the production environment  All tests will run and a zipped release file will be created that will include all dependencies

User · Answer

I have been using both committing the node modules folder and shrink-wrapping  Both solutions did not make me happy  In short  a committed node modules folder adds too much noise to the repository And shrinkwrap json is not easy to manage and there isn t any guarantee that some shrink-wrapped project will build in a few years  I found that Mozilla was using a separate repository for one of their projects  https   github com mozilla-b2g gaia-node-modules So it did not take me long to implement this idea in a Node js CLI tool  https   github com bestander npm-git-lock Just before every build  add  npm-git-lock --repo  git bitbucket org your dedicated node modules git repository git   It will calculate the hash of your package json file and will either check out folder node modules content from a remote repository  or  if it is a first build for this package json file  will do a clean npm install and push the results to the remote repository

User · Answer

Instead of checking in folder node modules  make a package json file for your application  The package json file specifies the dependencies of your application  Heroku can then tell npm to install all of those dependencies  The tutorial you linked to contains a section on package json files

User · Answer

My biggest concern with not checking folder node modules into Git is that 10 years down the road  when your production application is still in use  npm may not be around  Or npm might become corrupted  or the maintainers might decide to remove the library that you rely on from their repository  or the version you use might be trimmed out  This can be mitigated with repository managers like Maven  because you can always use your own local Nexus  Sonatype  or Artifactory to maintain a mirror with the packages that you use  As far as I understand  such a system doesn t exist for npm  The same goes for client-side library managers like Bower and Jam js  If you ve committed the files to your own Git repository  then you can update them when you like  and you have the comfort of repeatable builds and the knowledge that your application won t break because of some third-party action

User · Answer

Second Update The FAQ is not available anymore  From the documentation of shrinkwrap   If you wish to lock down the specific bytes included in a package  for example to have 100  confidence in being able to reproduce a deployment or build  then you ought to check your dependencies into source control  or pursue some other mechanism that can verify contents rather than versions   Shannon and Steven mentioned this before but I think  it should be part of the accepted answer   Update The source listed for the below recommendation has been updated   They are no longer recommending the node modules folder be committed   Usually  no  Allow npm to resolve dependencies for your packages  For packages you deploy  such as websites and apps  you should use npm shrinkwrap to lock down your full dependency tree  https   docs npmjs com cli shrinkwrap   Original Post For reference  npm FAQ answers your question clearly   Check node modules into git for things you deploy  such as websites and apps  Do not check node modules into git for libraries and modules intended to be reused  Use npm to manage dependencies in your dev environment  but not in your deployment scripts   and for some good rationale for this  read Mikeal Rogers  post on this   Source  https   docs npmjs com misc faq should-i-check-my-node-modules-folder-into-git

User · Answer

I am using this solution   Create a separate repository that holds folder node modules  If you have native modules that should be build for specific platform then create a separate repository for each platform   Attach these repositories to your project repository with git submodule  git submodule add     your project node modules windows git node modules windows git submodule add     your project node modules linux x86 64 node modules linux x86 64  Create a link from platform-specific node modules to node modules directory and add node modules to  gitignore   Run npm install   Commit submodule repository changes   Commit your project repository changes    So you can easily switch between node modules on different platforms  for example  if you are developing on OS X and deploying to Linux

User · Answer

I was going to leave this after this comment  Should I check in folder  quot node modules quot  to Git when creating a Node js app on Heroku  But Stack  Overflow was formatting it weirdly  If you don t have identical machines and are checking in node modules  do a  gitignore on the native extensions  Our  gitignore looks like    Ignore native extensions in the node modules folder  things changed by npm rebuild  node modules      node node modules      o node modules      a node modules      mk node modules      gypi node modules      target node modules     deps  node modules    build Makefile node modules       build Makefile  Test this by first checking everything in  and then have another developer do the following  rm -rf node modules git checkout -- node modules npm rebuild git status  Ensure that no files changed

User · Answer

You should not include folder node modules in your  gitignore file  or rather you should include folder node modules in your source deployed to Heroku   If folder node modules   exists then npm install will use those vendored libraries and will rebuild any binary dependencies with npm rebuild  doesn t exist then npm install will have to fetch all dependencies itself which adds time to the slug compile step   See the Node js buildpack source for these exact steps  However  the original error looks to be an incompatibility between the versions of npm and Node js  It is a good idea to always explicitly set the engines section of your packages json file according to this guide to avoid these types of situations       quot name quot    quot myapp quot      quot version quot    quot 0 0 1 quot      quot engines quot          quot node quot    quot 0 8 x quot        quot npm quot     quot 1 1 x quot         This will ensure development production parity and reduce the likelihood of such situations in the future

User · Answer

If you re rolling your own modules specific to your application  you can either   Keep those  and only those  in your application s  node modules folder and move out all the other dependencies to parent    node modules folder  This will work because of how NodeJS CommonJS modules system works by moving up to the parent directory  and so on  until the root of the tree is reached  See  https   nodejs org api modules html  Or gitignore all  node modules   except your  node modules your-modules  See  Make  gitignore ignore everything except a few files   This use case is pretty awesome  It lets you keep modules you created specifically for your application nicely with it and doesn t clutter with dependencies which can be installed later

User · Answer

Explicitly adding a npm version to file package json   quot npm quot     quot 1 1 x quot   and not checking in folder node modules to Git worked for me  It may be slower to deploy  since it downloads the packages each time   but I couldn t get the packages to compile when they were checked in  Heroku was looking for files that only existed on my local box

[git] Should I check in folder "node_modules" to Git when creating a Node.js app on Heroku?

Examples related to git

Examples related to node.js

Examples related to heroku

Examples related to npm

Examples related to gitignore