How to store Node js deployment settings configuration files

Question

I have been working on a few Node apps  and I ve been looking for a good pattern of storing deployment-related settings  In the Django world  where I come from   the common practise would be to have a settings py file containing the standard settings  timezone  etc   and then a local settings py for deployment specific settings  ie  what database to talk to  what memcache socket  e-mail address for the admins and so on   I have been looking for similar patterns for Node  Just a config file would be nice  so it does not have to be jammed in with everything else in app js  but I find it important to have a way to have server-specific configuration in a file that is not in source control  The same app could well be deployed across different servers with wildly different settings  and having to deal with merge conflicts and all that is not my idea of fun   So is there some kind of framework tool for this  or does everyone just hack something together themselves

User · Answer

Here is a neat approach inspired by this article. It does not require any additional packages except the ubiquitous lodash package. Moreover, it lets you manage nested defaults with environment-specific overwrites.

First, create a config folder in the package root path that looks like this

package
  |_config
      |_ index.js
      |_ defaults.json
      |_ development.json
      |_ test.json
      |_ production.json

here is the index.js file

const _ = require("lodash");
const defaults = require("./defaults.json");
const envConf = require("./" + (process.env.NODE_ENV || "development") + ".json" );
module.exports = _.defaultsDeep(envConf, defaults);

Now let's assume we have a defaults.json like so

{
  "confKey1": "value1",
  "confKey2": {
    "confKey3": "value3",
    "confKey4": "value4"
  }
}

and development.json like so

{
  "confKey2": {
    "confKey3": "value10",
  }
}

if you do config = require('./config') here is what you will get

{
  "confKey1": "value1",
  "confKey2": {
    "confKey3": "value10",
    "confKey4": "value4"
  }
}

Notice that you get all the default values except for those defined in environment-specific files. So you can manage a config hierarchy. Using defaultsDeep makes sure that you can even have nested defaults.

User · Answer

I know this is a really old post  But I want to share my module for configuring environment variables  I think it is very flexible solution  Here is the module json-configurator  var configJson        baseUrl    http   test com       prod baseUrl    https   prod com      endpoints          users     lt    baseUrl   gt  users        accounts     lt    baseUrl   gt  accounts           foo   bar     foobar   foobar      prod foo   foo in prod      test foo   foo in test     deep       veryDeep          publicKey   abc         secret   secret          prod secret   super secret                var config   require  json-configurator   configJson   prod     console log config deep veryDeep secret      super secret   console log config endpoints users     https   prod com users    Then you can use process env NODE ENV to get all the variables for your environment

User · Answer

Much later  I found a pretty good Node js module for managing configuration  nconf   A simple example   var nconf   require  nconf        First consider commandline arguments and environment variables  respectively  nconf argv   env        Then load configuration from a designated file  nconf file   file   config json          Provide default values for settings not provided above  nconf defaults        http              port   1337               Once this is in place  you can just use nconf get to get your settings     So this would configure  myApp  to listen on port 1337 if the port    has not been overridden by any of the three configuration inputs    mentioned above  myApp listen nconf get  http port       It also supports storing settings in Redis  writing configuration files  and has a fairly solid API  and is also backed by one of the more well-respected Node js shops  Nodejitsu  as part of the Flatiron framework initiative  so it should be fairly future-proof    Check out nconf at Github

User · Answer

npm i config  In config default json        quot app quot              quot port quot   3000             quot db quot              quot port quot   27017           quot name quot    quot dev db name quot           In config production json        quot app quot              quot port quot   4000             quot db quot              quot port quot   27000           quot name quot    quot prod db name quot           In index js  const config   require  config     let appPort   config get  app port    console log  Application port    appPort      let dbPort   config get  db port    console log  Database port    dbPort      let dbName   config get  db name    console log  Database name    dbName      console log  NODE ENV      config util getEnv  NODE ENV        node index js Application port  3000 Database port  27017 Database name  dev db name NODE ENV  development  For production   set NODE ENV production   node index js Application port  4000 Database port  27000 Database name  prod db name NODE ENV  production

User · Answer

I m going to throw my hat into the ring here because none of these answers address all the critical components that pretty much any system needs  Considerations    Public configuration  that can be seen by the frontend  vs private configuration  guy mograbi got this one right   And ensuring these are kept separate  Secrets like keys Defaults vs environment-specific overrides Frontend bundles   Here s how I do my configuration    config default private js - In version control  these are default configuration options that can only be seen by your backend  config default public js - In version control  these are default configuration options that can be seen by backend and frontend config dev private js - If you need different private defaults for dev  config dev public js - If you need different public defaults for dev  config private js - Not in version control  these are environment specific options that override config default private js config public js - Not in version control  these are environment specific options that override config default public js  keys  - A folder where each file stores a different secret of some kind  This is also not under version control  keys should never be under version control      I use plain-old javascript files for configuration so I have the full power of the javascript langauge  including comments and the ability to do things like load the default config file in the environment-specific file so they can then be overridden   If you want to use environment variables  you can load them inside those config files  tho I recommend against using env vars for the same reason I don t recommend using json files - you don t have the power of a programming language to construct your config    The reason each key is in a separate file is for installer use  This allows you to have an installer that creates keys on-machine and stores them in the keys folder  Without this  your installer might fail when you load your configuration file that can t access your keys  This way you can traverse the directory and load any key files that are in that folder without having to worry about what exists and what doesn t in any given version of your code    Since you probably have keys loaded in your private configuration  you definitely don t want to load your private config in any frontend code  While its probably strictly more ideal to completely separate your frontend codebase from your backend  a lot of times that PITA is a big enough barrier to prevent people from doing it  thus private vs public config  But there s two things I do to prevent private config being loaded in the frontend    I have a unit test that ensures my frontend bundles don t contain one of the secret keys I have in the private config   I have my frontend code in a different folder than my backend code  and I have two different files named  config js  - one for each end  For backend  config js loads the private config  for frontend  it loads the public config  Then you always just require  config   and don t worry about where it comes from     One last thing  your configuration should be loaded into the browser via a completely separate file than any of your other frontend code  If you bundle your frontend code  the public configuration should be built as a completely separate bundle  Otherwise  your config isn t really config anymore - its just part of your code  Config needs to be able to be different on different machines

User · Answer

You might also look to node-config which loads configuration file depending on  HOST and  NODE ENV variable  a little bit like RoR    documentation   This can be quite useful for different deployment settings  development  test or production

User · Answer

an alt example I just used because I wanted more flexibility than a typical  json file but didn t want it abstracted away into a library which would require a dependency is something like this  Basically  exporting a function invoked immediately which returned an object with values I wanted set  Gives a lot of flexibility         module exports   function           switch node env            case  dev              return              var1    development                          There is a much better explanation with full example here  Using Config Files in Node js

User · Answer

It s better to separate  development  and  production  configs   I use following way  Here is my config index js file   const config         dev             ip address    0 0 0 0           port   8080          mongo                url    mongodb   localhost 27017 story box dev               options                           prod             ip address    0 0 0 0           port   3000          mongo                url    mongodb   localhost 27017 story box prod               options                           For require the config use following   const config   require     config   process env NODE ENV     Than you can use your config object   const ip address   config ip address  const port   config port

User · Answer

You can require JSON files as of Node v0 5 x  referencing this answer   config json          username     root        password     foot      app js   var config   require    config json    log in config username  config password

User · Answer

Just do a simple settings js with exports   exports my password    value    Then  in your script  do a require   var settings   require    settings js      All your settings now will be availabe via settings variable   settings my password     value

User · Answer

I use a package json for my packages and a config js for my configuration  which looks like   var config        config twitter       config redis       config web        config default stuff      red   green   blue   apple   yellow   orange   politics    config twitter user name   process env TWITTER USER     username   config twitter password   process env TWITTER PASSWORD     password   config redis uri   process env DUOSTACK DB REDIS  config redis host    hostname   config redis port   6379  config web port   process env WEB PORT    9980   module exports   config    I load the config from my project   var config   require    config      and then I can access my things from config db host  config db port  etc        This lets me either use hardcoded parameters  or parameters stored in environmental variables if I don t want to store passwords in source control   I also generate a package json and insert a dependencies section    dependencies        cradle    0 5 5      jade    0 10 4      redis    0 5 11      socket io    0 6 16      twitter-node    0 0 2      express    2 2 0      When I clone the project to my local machine  I run npm install to install the packages  More info on that here   The project is stored in GitHub  with remotes added for my production server

User · Answer

You can use pconf  https   www npmjs com package pconf  Example   var Config   require  pconf    var testConfig   new Config  testConfig    testConfig onload   function       testConfig setValue  test   1     testConfig getValue  test        testConfig saveConfig    Not needed

User · Answer

I tryied some of suggested solutions here  but was not sattisfied with them  so I created my own module  It is called mikro-config and the main difference is that it honors convention over configuration  so you can just require the module and start using it    You store your configuration in either plain js  or json files from  config folder  First it loads default js file  then all other files from  config directory  then it loads environment specific configuration based on  NODE ENV variable    It also allows to override this configuration for local development with local js or environment specific  config env  NODE ENV local js    You can take at look at it here   https   www npmjs com package mikro-config  https   github com B4nan mikro-config

User · Answer

I will create a folder as config a file naming as config js and later I will use this file wherever required as below  Example of config js  module exports         proxyURL   http   url port       TWITTER            consumerkey   yourconsumerkey           consumerSecrete   yourconsumersecrete             GOOGLE            consumerkey   yourconsumerkey           consumerSecrete   yourconsumersecrete             FACEBOOK            consumerkey   yourconsumerkey           consumerSecrete   yourconsumersecrete            Then if i want to use this config file somewhere  I will first import as below  var config   require    config     and I can access the values as below  const oauth   OAuth       consumer            key  config TWITTER consumerkey          secret  config TWITTER consumerSecrete            signature method   HMAC-SHA1       hash function base string  key            return crypto createHmac  sha1   key  update base string  digest  base64

User · Answer

Are you guys using npm to start your scripts  env etc     If you use  env files you can include them in your package json and use npm to source start them   Example        name    server      version    0 0 1      private   true     scripts          start    node test js        start-dev    source dev env  node test js        start-prod    source prod env  node test js          dependencies          mysql               then run the npm scripts     npm start-dev   Its described here https   gist github com ericelliott 4152984 All credit to Eric Elliot

User · Answer

You can use Konfig for environment specific config files  It loads json or yaml config files automatically  it has default value and dynamic configuration features   An example from Konfig repo   File  config app json ----------------------------        default              port   3000           cache assets   true           secret key    7EHDWHD9W9UW9FBFB949394BWYFG8WE78F               development              cache assets   false              test              port   3001              staging              port     process env PORT            secret key    3F8RRJR30UHERGUH8UERHGIUERHG3987GH8               production              port     process env PORT            secret key    3F8RRJR30UHERGUH8UERHGIUERHG3987GH8            In development    gt  config app port 3000   In production  assume we start application with   NODE ENV production PORT 4567 node app js   gt  config app port 4567   More details   https   github com vngrs konfig

User · Answer

In addition to the nconf module mentioned in this answer  and node-config mentioned in this answer  there are also node-iniparser and IniReader  which appear to be simpler  ini configuration file parsers

User · Answer

For those who are visiting this old thread here is a package I find to be good   https   www npmjs org package config

User · Answer

My solution is fairly simple   Load the environment config in   config index js  var env   process env NODE ENV     development      cfg   require    config   env    module exports   cfg    Define some defaults in   config config global js  var config   module exports        config env    development   config hostname    dev example com      mongo database config mongo       config mongo uri   process env MONGO URI     localhost   config mongo db    example dev     Override the defaults in   config config test js  var config   require    config global     config env    test   config hostname    test example   config mongo db    example test    module exports   config    Using it in   models user js   var mongoose   require  mongoose     cfg   require     config     db   mongoose createConnection cfg mongo uri  cfg mongo db     Running your app in test environment   NODE ENV test node   app js

User · Answer

I just recently released a small module to load any type of configuration files   It s pretty straight-forward  you can check it at https   github com flesler config-node

User · Answer

I am a bit late in the game  but I couldn t find what I needed here- or anywhere else - so I wrote something myself   My requirements for a configuration mechanism are the following     Support front-end  What is the point if the front-end cannot use the configuration  Support settings-overrides js - which looks the same but allows overrides of configuration at settings js  The idea here is to modify configuration easily without changing the code  I find it useful for saas     Even though I care less about supporting environments - the will explain how to add it easily to my solution  var publicConfiguration          title     Hello World       demoAuthToken    undefined        demoUserId    undefined        errorEmail    null    if null we will not send emails on errors        var privateConfiguration          port  9040       adminAuthToken  undefined       adminUserId  undefined    var meConf   null  try      meConf   require     conf dev meConf     catch  e     console log  meConf does not exist  ignoring           var publicConfigurationInitialized   false  var privateConfigurationInitialized   false   function getPublicConfiguration        if   publicConfigurationInitialized            publicConfigurationInitialized   true          if  meConf    null                for  var i in publicConfiguration                    if  meConf hasOwnProperty i                         publicConfiguration i    meConf i                                                       return publicConfiguration      function getPrivateConfiguration        if    privateConfigurationInitialized             privateConfigurationInitialized   true           var pubConf   getPublicConfiguration             if   pubConf    null                for   var j in pubConf                    privateConfiguration j    pubConf j                                   if   meConf    null                  for   var i in meConf                      privateConfiguration i    meConf i                                       return privateConfiguration       exports sendPublicConfiguration   function  req  res        var name   req param  name       conf        res send   window     name           JSON stringify getPublicConfiguration                 var prConf   getPrivateConfiguration    if   prConf    null        for   var i in prConf            if   prConf i      undefined                 throw new Error  undefined configuration      i                           exports i    prConf i             return exports    Explanation   undefined means this property is required null means it is optional  meConf - currently the code is target to a file under app  meConf is the overrides files which is targeted to conf dev - which is ignored by my vcs   publicConfiguration - will be visible from front-end and back-end   privateConfiguration - will be visible from back-end only   sendPublicConfiguration - a route that will expose the public configuration and assign it to a global variable  For example the code below will expose the public configuration as global variable myConf in the front-end  By default it will use the global variable name conf    app get   backend conf   require  conf   sendPublicConfiguration       Logic of overrides   privateConfiguration is merged with publicConfiguration and then meConf  publicConfiguration checks each key if it has an override  and uses that override  This way we are not exposing anything private     Adding environment support  Even though I don t find an  environment support  useful  maybe someone will    To add environment support you need to change the meConf require statement to something like this   pseudocode    if   environment     production              meConf   require     conf dev meConf   production        if   environment     development              meConf   require     conf dev meConf   development        Similarly you can have a file per environment   meConf development js  meConf production js   and import the right one   The rest of the logic stays the same

User · Answer

Convict is another option that adds a schema for validation   Like nconf  it supports loading settings from any combination of environment variables  arguments  files  and json objects   Example from the README   var convict   require  convict    var conf   convict     env        doc   The applicaton environment        format    production    development    test        default   development       env   NODE ENV         ip        doc   The IP address to bind        format   ipaddress       default   127 0 0 1       env   IP ADDRESS          port        doc   The port to bind        format   port       default  0      env   PORT            Getting started article  Taming Configurations with node-convict

User · Answer

You might also look to dotenv which follows the tenets of a twelve-factor app   I used to use node-config  but created dotenv for that reason  It was completely inspired by ruby s dotenv library   Usage is quite easy   var dotenv   require  dotenv    dotenv load      Then you just create a  env file and put your settings in there like so   S3 BUCKET YOURS3BUCKET SECRET KEY YOURSECRETKEYGOESHERE OTHER SECRET STUFF my cats middle name   That s dotenv for nodejs

User · Answer

Just use npm module config  more than 300000 downloads   https   www npmjs com package config   Node-config organizes hierarchical configurations for your app deployments   It lets you define a set of default parameters  and extend them for different deployment environments  development  qa  staging  production  etc       npm install config   mkdir config   vi config default json              Customer module configs        Customer              dbConfig                host    localhost              port   5984             dbName    customers                      credit                initialLimit   100               Set low for development            initialDays   1                          vi config production json       Customer          dbConfig            host    prod-db-server              credit            initialDays   30                  vi index js  var config   require  config          var dbConfig   config get  Customer dbConfig    db connect dbConfig         if  config has  optionalFeature detail        var detail   config get  optionalFeature detail                  export NODE ENV production   node index js

User · Answer

For long  I used to use the approach mentioned in the solution here  There is a concern however  about security of the secrets in clear text  You can use another package on top of config so that the security bits are taken care of    Check this out  https   www attosol com secure-application-secrets-using-masterkey-in-azure-key-vault

[node.js] How to store Node.js deployment settings/configuration files?

Examples related to node.js

Examples related to configuration-files