[node.js] How to store Node.js deployment settings/configuration files?

I have been working on a few Node apps, and I've been looking for a good pattern of storing deployment-related settings. In the Django world (where I come from), the common practise would be to have a settings.py file containing the standard settings (timezone, etc), and then a local_settings.py for deployment specific settings, ie. what database to talk to, what memcache socket, e-mail address for the admins and so on.

I have been looking for similar patterns for Node. Just a config file would be nice, so it does not have to be jammed in with everything else in app.js, but I find it important to have a way to have server-specific configuration in a file that is not in source control. The same app could well be deployed across different servers with wildly different settings, and having to deal with merge conflicts and all that is not my idea of fun.

So is there some kind of framework/tool for this, or does everyone just hack something together themselves?

npm i config

In config/default.json
{
    "app": {
        "port": 3000
    },
    "db": {
        "port": 27017,
        "name": "dev_db_name"
    }
}

In config/production.json
{
    "app": {
        "port": 4000
    },
    "db": {
        "port": 27000,
        "name": "prod_db_name"
    }
}

In index.js

const config = require('config');

let appPort = config.get('app.port');
console.log(`Application port: ${appPort}`);

let dbPort = config.get('db.port');
console.log(`Database port: ${dbPort}`);

let dbName = config.get('db.name');
console.log(`Database name: ${dbName}`);

console.log('NODE_ENV: ' + config.util.getEnv('NODE_ENV'));

$ node index.js
Application port: 3000
Database port: 27017
Database name: dev_db_name
NODE_ENV: development

For production
$ set NODE_ENV=production
$ node index.js
Application port: 4000
Database port: 27000
Database name: prod_db_name
NODE_ENV: production

I know this is a really old post. But I want to share my module for configuring environment variables, I think it is very flexible solution. Here is the module json-configurator

var configJson = {
  'baseUrl': 'http://test.com',
  '$prod_baseUrl': 'https://prod.com',
  'endpoints': {
    'users': '<%= baseUrl %>/users',
    'accounts': '<%= baseUrl %>/accounts'
    },
  foo: 'bar',
  foobar: 'foobar',
  $prod_foo: 'foo in prod',
  $test_foo: 'foo in test',
  deep:{
    veryDeep: {
      publicKey: 'abc',
      secret: 'secret',
      $prod_secret: 'super secret'
    }
  }
};

var config = require('json-configurator')(configJson, 'prod');

console.log(config.deep.veryDeep.secret) 
// super secret 

console.log(config.endpoints.users)
// https://prod.com/users 

Then you can use process.env.NODE_ENV to get all the variables for your environment.

Here is a neat approach inspired by this article. It does not require any additional packages except the ubiquitous lodash package. Moreover, it lets you manage nested defaults with environment-specific overwrites.

First, create a config folder in the package root path that looks like this

package
  |_config
      |_ index.js
      |_ defaults.json
      |_ development.json
      |_ test.json
      |_ production.json

here is the index.js file

const _ = require("lodash");
const defaults = require("./defaults.json");
const envConf = require("./" + (process.env.NODE_ENV || "development") + ".json" );
module.exports = _.defaultsDeep(envConf, defaults);

Now let's assume we have a defaults.json like so

{
  "confKey1": "value1",
  "confKey2": {
    "confKey3": "value3",
    "confKey4": "value4"
  }
}

and development.json like so

{
  "confKey2": {
    "confKey3": "value10",
  }
}

if you do config = require('./config') here is what you will get

{
  "confKey1": "value1",
  "confKey2": {
    "confKey3": "value10",
    "confKey4": "value4"
  }
}

Notice that you get all the default values except for those defined in environment-specific files. So you can manage a config hierarchy. Using defaultsDeep makes sure that you can even have nested defaults.

For those who are visiting this old thread here is a package I find to be good.

https://www.npmjs.org/package/config

You can require JSON files as of Node v0.5.x (referencing this answer)

config.json:

{
    "username" : "root",
    "password" : "foot"
}

app.js:

var config = require('./config.json');
log_in(config.username, config.password);

I just recently released a small module to load any type of configuration files. It's pretty straight-forward, you can check it at https://github.com/flesler/config-node

I will create a folder as config a file naming as config.js and later I will use this file wherever required as below

Example of config.js

module.exports = {
    proxyURL: 'http://url:port',
    TWITTER: {
        consumerkey: 'yourconsumerkey',
        consumerSecrete: 'yourconsumersecrete'
    },
    GOOGLE: {
        consumerkey: 'yourconsumerkey',
        consumerSecrete: 'yourconsumersecrete'
    },
    FACEBOOK: {
        consumerkey: 'yourconsumerkey',
        consumerSecrete: 'yourconsumersecrete'
    }
}

Then if i want to use this config file somewhere

I will first import as below

var config = require('./config');

and I can access the values as below

const oauth = OAuth({
    consumer: {
        key: config.TWITTER.consumerkey,
        secret: config.TWITTER.consumerSecrete
    },
    signature_method: 'HMAC-SHA1',
    hash_function(base_string, key) {
        return crypto.createHmac('sha1', key).update(base_string).digest('base64');
    }
});

You might also look to node-config which loads configuration file depending on $HOST and $NODE_ENV variable (a little bit like RoR) : documentation.

This can be quite useful for different deployment settings (development, test or production).

It's better to separate 'development' and 'production' configs.

I use following way: Here is my config/index.js file:

const config = {
    dev : {
        ip_address : '0.0.0.0',
        port : 8080,
        mongo :{
            url : "mongodb://localhost:27017/story_box_dev",
            options : ""
        }
    },
    prod : {
        ip_address : '0.0.0.0',
        port : 3000,
        mongo :{
            url : "mongodb://localhost:27017/story_box_prod",
            options : ""
        }
    }
} 

For require the config use following:

const config = require('../config')[process.env.NODE_ENV];

Than you can use your config object:

const ip_address = config.ip_address;
const port = config.port;

an alt example I just used because I wanted more flexibility than a typical .json file but didn't want it abstracted away into a library which would require a dependency is something like this. Basically, exporting a function invoked immediately which returned an object with values I wanted set. Gives a lot of flexibility.

     module.exports = function(){
       switch(node_env){
         case 'dev':
           return
           { var1 = 'development'};
         }
    }();

There is a much better explanation with full example here. Using Config Files in Node.js

You can use pconf: https://www.npmjs.com/package/pconf

Example:

var Config = require("pconf");
var testConfig = new Config("testConfig");
testConfig.onload = function(){

  testConfig.setValue("test", 1);
  testConfig.getValue("test");
  //testConfig.saveConfig(); Not needed

}

I tryied some of suggested solutions here, but was not sattisfied with them, so I created my own module. It is called mikro-config and the main difference is that it honors convention over configuration, so you can just require the module and start using it.

You store your configuration in either plain js, or json files from /config folder. First it loads default.js file, then all other files from /config directory, then it loads environment specific configuration based on $NODE_ENV variable.

It also allows to override this configuration for local development with local.js or environment specific /config/env/$NODE_ENV.local.js.

You can take at look at it here:

https://www.npmjs.com/package/mikro-config

https://github.com/B4nan/mikro-config

Convict is another option that adds a schema for validation. Like nconf, it supports loading settings from any combination of environment variables, arguments, files, and json objects.

Example from the README:

var convict = require('convict');
var conf = convict({
  env: {
    doc: "The applicaton environment.",
    format: ["production", "development", "test"],
    default: "development",
    env: "NODE_ENV"
  },
  ip: {
    doc: "The IP address to bind.",
    format: "ipaddress",
    default: "127.0.0.1",
    env: "IP_ADDRESS",
  },
  port: {
    doc: "The port to bind.",
    format: "port",
    default: 0,
    env: "PORT"
  }
});

Getting started article: Taming Configurations with node-convict

I use a package.json for my packages and a config.js for my configuration, which looks like:

var config = {};

config.twitter = {};
config.redis = {};
config.web = {};

config.default_stuff =  ['red','green','blue','apple','yellow','orange','politics'];
config.twitter.user_name = process.env.TWITTER_USER || 'username';
config.twitter.password=  process.env.TWITTER_PASSWORD || 'password';
config.redis.uri = process.env.DUOSTACK_DB_REDIS;
config.redis.host = 'hostname';
config.redis.port = 6379;
config.web.port = process.env.WEB_PORT || 9980;

module.exports = config;

I load the config from my project:

var config = require('./config');

and then I can access my things from config.db_host, config.db_port, etc... This lets me either use hardcoded parameters, or parameters stored in environmental variables if I don't want to store passwords in source control.

I also generate a package.json and insert a dependencies section:

"dependencies": {
  "cradle": "0.5.5",
  "jade": "0.10.4",
  "redis": "0.5.11",
  "socket.io": "0.6.16",
  "twitter-node": "0.0.2",
  "express": "2.2.0"
}

When I clone the project to my local machine, I run npm install to install the packages. More info on that here.

The project is stored in GitHub, with remotes added for my production server.

I'm going to throw my hat into the ring here because none of these answers address all the critical components that pretty much any system needs. Considerations:

• Public configuration (that can be seen by the frontend) vs private configuration (guy mograbi got this one right). And ensuring these are kept separate.
• Secrets like keys
• Defaults vs environment-specific overrides
• Frontend bundles

Here's how I do my configuration:

• config.default.private.js - In version control, these are default configuration options that can only be seen by your backend.
• config.default.public.js - In version control, these are default configuration options that can be seen by backend and frontend
• config.dev.private.js - If you need different private defaults for dev.
• config.dev.public.js - If you need different public defaults for dev.
• config.private.js - Not in version control, these are environment specific options that override config.default.private.js
• config.public.js - Not in version control, these are environment specific options that override config.default.public.js
• keys/ - A folder where each file stores a different secret of some kind. This is also not under version control (keys should never be under version control).

I use plain-old javascript files for configuration so I have the full power of the javascript langauge (including comments and the ability to do things like load the default config file in the environment-specific file so they can then be overridden). If you want to use environment variables, you can load them inside those config files (tho I recommend against using env vars for the same reason I don't recommend using json files - you don't have the power of a programming language to construct your config).

The reason each key is in a separate file is for installer use. This allows you to have an installer that creates keys on-machine and stores them in the keys folder. Without this, your installer might fail when you load your configuration file that can't access your keys. This way you can traverse the directory and load any key files that are in that folder without having to worry about what exists and what doesn't in any given version of your code.

Since you probably have keys loaded in your private configuration, you definitely don't want to load your private config in any frontend code. While its probably strictly more ideal to completely separate your frontend codebase from your backend, a lot of times that PITA is a big enough barrier to prevent people from doing it, thus private vs public config. But there's two things I do to prevent private config being loaded in the frontend:

1. I have a unit test that ensures my frontend bundles don't contain one of the secret keys I have in the private config.
2. I have my frontend code in a different folder than my backend code, and I have two different files named "config.js" - one for each end. For backend, config.js loads the private config, for frontend, it loads the public config. Then you always just require('config') and don't worry about where it comes from.

One last thing: your configuration should be loaded into the browser via a completely separate file than any of your other frontend code. If you bundle your frontend code, the public configuration should be built as a completely separate bundle. Otherwise, your config isn't really config anymore - its just part of your code. Config needs to be able to be different on different machines.

In addition to the nconf module mentioned in this answer, and node-config mentioned in this answer, there are also node-iniparser and IniReader, which appear to be simpler .ini configuration file parsers.

Just use npm module config (more than 300000 downloads)

https://www.npmjs.com/package/config

Node-config organizes hierarchical configurations for your app deployments.

It lets you define a set of default parameters, and extend them for different deployment environments (development, qa, staging, production, etc.).

$ npm install config
$ mkdir config
$ vi config/default.json


{
      // Customer module configs
      "Customer": {
        "dbConfig": {
          "host": "localhost",
          "port": 5984,
          "dbName": "customers"
        },
        "credit": {
          "initialLimit": 100,
          // Set low for development
          "initialDays": 1
        }
      }
}



$ vi config/production.json

{
  "Customer": {
    "dbConfig": {
      "host": "prod-db-server"
    },
    "credit": {
      "initialDays": 30
    }
  }
}



$ vi index.js

var config = require('config');
//...
var dbConfig = config.get('Customer.dbConfig');
db.connect(dbConfig, ...);

if (config.has('optionalFeature.detail')) {
  var detail = config.get('optionalFeature.detail');
  //...
}


$ export NODE_ENV=production
$ node index.js

I am a bit late in the game, but I couldn't find what I needed here- or anywhere else - so I wrote something myself.

My requirements for a configuration mechanism are the following:

1. Support front-end. What is the point if the front-end cannot use the configuration?
2. Support settings-overrides.js - which looks the same but allows overrides of configuration at settings.js. The idea here is to modify configuration easily without changing the code. I find it useful for saas.

Even though I care less about supporting environments - the will explain how to add it easily to my solution

var publicConfiguration = {
    "title" : "Hello World"
    "demoAuthToken" : undefined, 
    "demoUserId" : undefined, 
    "errorEmail" : null // if null we will not send emails on errors. 

};

var privateConfiguration = {
    "port":9040,
    "adminAuthToken":undefined,
    "adminUserId":undefined
}

var meConf = null;
try{
    meConf = require("../conf/dev/meConf");
}catch( e ) { console.log("meConf does not exist. ignoring.. ")}




var publicConfigurationInitialized = false;
var privateConfigurationInitialized = false;

function getPublicConfiguration(){
    if (!publicConfigurationInitialized) {
        publicConfigurationInitialized = true;
        if (meConf != null) {
            for (var i in publicConfiguration) {
                if (meConf.hasOwnProperty(i)) {
                    publicConfiguration[i] = meConf[i];
                }
            }
        }
    }
    return publicConfiguration;
}


function getPrivateConfiguration(){
    if ( !privateConfigurationInitialized ) {
        privateConfigurationInitialized = true;

        var pubConf = getPublicConfiguration();

        if ( pubConf != null ){
            for ( var j in pubConf ){
                privateConfiguration[j] = pubConf[j];
            }
        }
        if ( meConf != null ){
              for ( var i in meConf ){
                  privateConfiguration[i] = meConf[i];
              }
        }
    }
    return privateConfiguration;

}


exports.sendPublicConfiguration = function( req, res ){
    var name = req.param("name") || "conf";

    res.send( "window." + name + " = " + JSON.stringify(getPublicConfiguration()) + ";");
};


var prConf = getPrivateConfiguration();
if ( prConf != null ){
    for ( var i in prConf ){
        if ( prConf[i] === undefined ){

            throw new Error("undefined configuration [" + i + "]");
        }
        exports[i] = prConf[i];
    }
}


return exports;

Explanation

• undefined means this property is required
• null means it is optional
• meConf - currently the code is target to a file under app. meConf is the overrides files which is targeted to conf/dev - which is ignored by my vcs.
• publicConfiguration - will be visible from front-end and back-end.
• privateConfiguration - will be visible from back-end only.

• sendPublicConfiguration - a route that will expose the public configuration and assign it to a global variable. For example the code below will expose the public configuration as global variable myConf in the front-end. By default it will use the global variable name conf.

  app.get("/backend/conf", require("conf").sendPublicConfiguration);

Logic of overrides

• privateConfiguration is merged with publicConfiguration and then meConf.
• publicConfiguration checks each key if it has an override, and uses that override. This way we are not exposing anything private.

Adding environment support

Even though I don't find an "environment support" useful, maybe someone will.

To add environment support you need to change the meConf require statement to something like this (pseudocode)

if ( environment == "production" ) { meConf = require("../conf/dev/meConf").production; }

if ( environment == "development" ) { meConf = require("../conf/dev/meConf").development; }

Similarly you can have a file per environment

 meConf.development.js
 meConf.production.js

and import the right one. The rest of the logic stays the same.

For long, I used to use the approach mentioned in the solution here. There is a concern however, about security of the secrets in clear text. You can use another package on top of config so that the security bits are taken care of.

Check this out: https://www.attosol.com/secure-application-secrets-using-masterkey-in-azure-key-vault/

Are you guys using npm to start your scripts (env etc) ?

If you use .env files you can include them in your package.json and use npm to source/start them.

Example:

{
  "name": "server",
  "version": "0.0.1",
  "private": true,
  "scripts": {
    "start": "node test.js",
    "start-dev": "source dev.env; node test.js",
    "start-prod": "source prod.env; node test.js"
  },
  "dependencies": {
    "mysql": "*"
  }
}

then run the npm scripts:

$ npm start-dev

Its described here https://gist.github.com/ericelliott/4152984 All credit to Eric Elliot

You might also look to dotenv which follows the tenets of a twelve-factor app.

I used to use node-config, but created dotenv for that reason. It was completely inspired by ruby's dotenv library.

Usage is quite easy:

var dotenv = require('dotenv');
dotenv.load();

Then you just create a .env file and put your settings in there like so:

S3_BUCKET=YOURS3BUCKET
SECRET_KEY=YOURSECRETKEYGOESHERE
OTHER_SECRET_STUFF=my_cats_middle_name

That's dotenv for nodejs.

Just do a simple settings.js with exports:

exports.my_password = 'value'

Then, in your script, do a require:

var settings = require('./settings.js');

All your settings now will be availabe via settings variable:

settings.my_password // 'value'

You can use Konfig for environment specific config files. It loads json or yaml config files automatically, it has default value and dynamic configuration features.

An example from Konfig repo:

File: config/app.json
----------------------------
{
    "default": {
        "port": 3000,
        "cache_assets": true,
        "secret_key": "7EHDWHD9W9UW9FBFB949394BWYFG8WE78F"
    },

    "development": {
        "cache_assets": false
    },

    "test": {
        "port": 3001
    },

    "staging": {
        "port": #{process.env.PORT},
        "secret_key": "3F8RRJR30UHERGUH8UERHGIUERHG3987GH8"
    },

    "production": {
        "port": #{process.env.PORT},
        "secret_key": "3F8RRJR30UHERGUH8UERHGIUERHG3987GH8"
    }
}

In development:

> config.app.port
3000

In production, assume we start application with $ NODE_ENV=production PORT=4567 node app.js

> config.app.port
4567

More details : https://github.com/vngrs/konfig

My solution is fairly simple:

Load the environment config in ./config/index.js

var env = process.env.NODE_ENV || 'development'
  , cfg = require('./config.'+env);

module.exports = cfg;

Define some defaults in ./config/config.global.js

var config = module.exports = {};

config.env = 'development';
config.hostname = 'dev.example.com';

//mongo database
config.mongo = {};
config.mongo.uri = process.env.MONGO_URI || 'localhost';
config.mongo.db = 'example_dev';

Override the defaults in ./config/config.test.js

var config = require('./config.global');

config.env = 'test';
config.hostname = 'test.example';
config.mongo.db = 'example_test';

module.exports = config;

Using it in ./models/user.js:

var mongoose = require('mongoose')
, cfg = require('../config')
, db = mongoose.createConnection(cfg.mongo.uri, cfg.mongo.db);

Running your app in test environment:

NODE_ENV=test node ./app.js