[javascript] HTTP GET Request in Node.js Express

How can I make an HTTP request from within Node.js or Express.js? I need to connect to another service. I am hoping the call is asynchronous and that the callback contains the remote server's response.

This question is related to javascript node.js express httprequest

The answer is


Here is a snippet of some code from a sample of mine. It's asynchronous and returns a JSON object. It can do any form of GET request.

Note that there are more optimal ways (just a sample) - for example, instead of concatenating the chunks you put into an array and join it etc... Hopefully, it gets you started in the right direction:

const http = require('http');
const https = require('https');

/**
 * getJSON:  RESTful GET request returning JSON object(s)
 * @param options: http options object
 * @param callback: callback to pass the results JSON object(s) back
 */

module.exports.getJSON = (options, onResult) => {
  console.log('rest::getJSON');
  const port = options.port == 443 ? https : http;

  let output = '';

  const req = port.request(options, (res) => {
    console.log(`${options.host} : ${res.statusCode}`);
    res.setEncoding('utf8');

    res.on('data', (chunk) => {
      output += chunk;
    });

    res.on('end', () => {
      let obj = JSON.parse(output);

      onResult(res.statusCode, obj);
    });
  });

  req.on('error', (err) => {
    // res.send('error: ' + err.message);
  });

  req.end();
};

It's called by creating an options object like:

const options = {
  host: 'somesite.com',
  port: 443,
  path: '/some/path',
  method: 'GET',
  headers: {
    'Content-Type': 'application/json'
  }
};

And providing a callback function.

For example, in a service, I require the REST module above and then do this:

rest.getJSON(options, (statusCode, result) => {
  // I could work with the resulting HTML/JSON here. I could also just return it
  console.log(`onResult: (${statusCode})\n\n${JSON.stringify(result)}`);

  res.statusCode = statusCode;

  res.send(result);
});

UPDATE

If you're looking for async/await (linear, no callback), promises, compile time support and intellisense, we created a lightweight HTTP and REST client that fits that bill:

Microsoft typed-rest-client


Try using the simple http.get(options, callback) function in node.js:

var http = require('http');
var options = {
  host: 'www.google.com',
  path: '/index.html'
};

var req = http.get(options, function(res) {
  console.log('STATUS: ' + res.statusCode);
  console.log('HEADERS: ' + JSON.stringify(res.headers));

  // Buffer the body entirely for processing as a whole.
  var bodyChunks = [];
  res.on('data', function(chunk) {
    // You can process streamed parts here...
    bodyChunks.push(chunk);
  }).on('end', function() {
    var body = Buffer.concat(bodyChunks);
    console.log('BODY: ' + body);
    // ...and/or process the entire body here.
  })
});

req.on('error', function(e) {
  console.log('ERROR: ' + e.message);
});

There is also a general http.request(options, callback) function which allows you to specify the request method and other request details.


Request and Superagent are pretty good libraries to use.

note: request is deprecated, use at your risk!

Using request:

var request=require('request');

request.get('https://someplace',options,function(err,res,body){
  if(err) //TODO: handle err
  if(res.statusCode === 200 ) //etc
  //TODO Do something with response
});

This version is based on the initially proposed by bryanmac function which uses promises, better error handling, and is rewritten in ES6.

let http = require("http"),
    https = require("https");

/**
* getJSON:  REST get request returning JSON object(s)
* @param options: http options object
*/
exports.getJSON = function (options) {
    console.log('rest::getJSON');
    let reqHandler = +options.port === 443 ? https : http;

    return new Promise((resolve, reject) => {
        let req = reqHandler.request(options, (res) => {
            let output = '';
            console.log('rest::', options.host + ':' + res.statusCode);
            res.setEncoding('utf8');

            res.on('data', function (chunk) {
                output += chunk;
            });

            res.on('end', () => {
                try {
                    let obj = JSON.parse(output);
                    // console.log('rest::', obj);
                    resolve({
                        statusCode: res.statusCode,
                        data: obj
                    });
                }
                catch (err) {
                    console.error('rest::end', err);
                    reject(err);
                }
            });
        });

        req.on('error', (err) => {
            console.error('rest::request', err);
            reject(err);
        });

        req.end();
    });
};

As a result you don't have to pass in a callback function, instead getJSON() returns a promise. In the following example the function is used inside of an ExpressJS route handler

router.get('/:id', (req, res, next) => {
    rest.getJSON({
        host: host,
        path: `/posts/${req.params.id}`,
        method: 'GET'
    }).then(({ statusCode, data }) => {
        res.json(data);
    }, (error) => {
        next(error);
    });
});

On error it delegates the error to the server error handling middleware.


Check out shred. It's a node HTTP client created and maintained by spire.io that handles redirects, sessions, and JSON responses. It's great for interacting with rest APIs. See this blog post for more details.


Check out httpreq: it's a node library I created because I was frustrated there was no simple http GET or POST module out there ;-)


Use reqclient: not designed for scripting purpose like request or many other libraries. Reqclient allows in the constructor specify many configurations useful when you need to reuse the same configuration again and again: base URL, headers, auth options, logging options, caching, etc. Also has useful features like query and URL parsing, automatic query encoding and JSON parsing, etc.

The best way to use the library is create a module to export the object pointing to the API and the necessary configurations to connect with:

Module client.js:

let RequestClient = require("reqclient").RequestClient

let client = new RequestClient({
  baseUrl: "https://myapp.com/api/v1",
  cache: true,
  auth: {user: "admin", pass: "secret"}
})

module.exports = client

And in the controllers where you need to consume the API use like this:

let client = require('client')
//let router = ...

router.get('/dashboard', (req, res) => {
  // Simple GET with Promise handling to https://myapp.com/api/v1/reports/clients
  client.get("reports/clients")
    .then(response => {
       console.log("Report for client", response.userId)  // REST responses are parsed as JSON objects
       res.render('clients/dashboard', {title: 'Customer Report', report: response})
    })
    .catch(err => {
      console.error("Ups!", err)
      res.status(400).render('error', {error: err})
    })
})

router.get('/orders', (req, res, next) => {
  // GET with query (https://myapp.com/api/v1/orders?state=open&limit=10)
  client.get({"uri": "orders", "query": {"state": "open", "limit": 10}})
    .then(orders => {
      res.render('clients/orders', {title: 'Customer Orders', orders: orders})
    })
    .catch(err => someErrorHandler(req, res, next))
})

router.delete('/orders', (req, res, next) => {
  // DELETE with params (https://myapp.com/api/v1/orders/1234/A987)
  client.delete({
    "uri": "orders/{client}/{id}",
    "params": {"client": "A987", "id": 1234}
  })
  .then(resp => res.status(204))
  .catch(err => someErrorHandler(req, res, next))
})

reqclient supports many features, but it has some that are not supported by other libraries: OAuth2 integration and logger integration with cURL syntax, and always returns native Promise objects.


## you can use request module and promise in express to make any request ##
const promise                       = require('promise');
const requestModule                 = require('request');

const curlRequest =(requestOption) =>{
    return new Promise((resolve, reject)=> {
        requestModule(requestOption, (error, response, body) => {
            try {
                if (error) {
                    throw error;
                }
                if (body) {

                    try {
                        body = (body) ? JSON.parse(body) : body;
                        resolve(body);
                    }catch(error){
                        resolve(body);
                    }

                } else {

                    throw new Error('something wrong');
                }
            } catch (error) {

                reject(error);
            }
        })
    })
};

const option = {
    url : uri,
    method : "GET",
    headers : {

    }
};


curlRequest(option).then((data)=>{
}).catch((err)=>{
})

Unirest is the best library I've come across for making HTTP requests from Node. It's aiming at being a multiplatform framework, so learning how it works on Node will serve you well if you need to use an HTTP client on Ruby, PHP, Java, Python, Objective C, .Net or Windows 8 as well. As far as I can tell the unirest libraries are mostly backed by existing HTTP clients (e.g. on Java, the Apache HTTP client, on Node, Mikeal's Request libary) - Unirest just puts a nicer API on top.

Here are a couple of code examples for Node.js:

var unirest = require('unirest')

// GET a resource
unirest.get('http://httpbin.org/get')
  .query({'foo': 'bar'})
  .query({'stack': 'overflow'})
  .end(function(res) {
    if (res.error) {
      console.log('GET error', res.error)
    } else {
      console.log('GET response', res.body)
    }
  })

// POST a form with an attached file
unirest.post('http://httpbin.org/post')
  .field('foo', 'bar')
  .field('stack', 'overflow')
  .attach('myfile', 'examples.js')
  .end(function(res) {
    if (res.error) {
      console.log('POST error', res.error)
    } else {
      console.log('POST response', res.body)
    }
  })

You can jump straight to the Node docs here


If you just need to make simple get requests and don't need support for any other HTTP methods take a look at: simple-get:

var get = require('simple-get');

get('http://example.com', function (err, res) {
  if (err) throw err;
  console.log(res.statusCode); // 200
  res.pipe(process.stdout); // `res` is a stream
});

If you ever need to send GET request to an IP as well as a Domain (Other answers did not mention you can specify a port variable), you can make use of this function:

function getCode(host, port, path, queryString) {
    console.log("(" + host + ":" + port + path + ")" + "Running httpHelper.getCode()")

    // Construct url and query string
    const requestUrl = url.parse(url.format({
        protocol: 'http',
        hostname: host,
        pathname: path,
        port: port,
        query: queryString
    }));

    console.log("(" + host + path + ")" + "Sending GET request")
    // Send request
    console.log(url.format(requestUrl))
    http.get(url.format(requestUrl), (resp) => {
        let data = '';

        // A chunk of data has been received.
        resp.on('data', (chunk) => {
            console.log("GET chunk: " + chunk);
            data += chunk;
        });

        // The whole response has been received. Print out the result.
        resp.on('end', () => {
            console.log("GET end of response: " + data);
        });

    }).on("error", (err) => {
        console.log("GET Error: " + err);
    });
}

Don't miss requiring modules at the top of your file:

http = require("http");
url = require('url')

Also bare in mind that you may use https module for communicating over secured network. so these two lines would change:

https = require("https");
...
https.get(url.format(requestUrl), (resp) => { ......

You can also use Requestify, a really cool and very simple HTTP client I wrote for nodeJS + it supports caching.

Just do the following for GET method request:

var requestify = require('requestify');

requestify.get('http://example.com/api/resource')
  .then(function(response) {
      // Get the response body (JSON parsed or jQuery object for XMLs)
      response.getBody();
  }
);

Examples related to javascript

need to add a class to an element How to make a variable accessible outside a function? Hide Signs that Meteor.js was Used How to create a showdown.js markdown extension Please help me convert this script to a simple image slider Highlight Anchor Links when user manually scrolls? Summing radio input values How to execute an action before close metro app WinJS javascript, for loop defines a dynamic variable name Getting all files in directory with ajax

Examples related to node.js

Hide Signs that Meteor.js was Used Querying date field in MongoDB with Mongoose SyntaxError: Cannot use import statement outside a module Server Discovery And Monitoring engine is deprecated How to fix ReferenceError: primordials is not defined in node UnhandledPromiseRejectionWarning: This error originated either by throwing inside of an async function without a catch block dyld: Library not loaded: /usr/local/opt/icu4c/lib/libicui18n.62.dylib error running php after installing node with brew on Mac internal/modules/cjs/loader.js:582 throw err DeprecationWarning: Buffer() is deprecated due to security and usability issues when I move my script to another server Please run `npm cache clean`

Examples related to express

UnhandledPromiseRejectionWarning: This error originated either by throwing inside of an async function without a catch block jwt check if token expired Avoid "current URL string parser is deprecated" warning by setting useNewUrlParser to true MongoNetworkError: failed to connect to server [localhost:27017] on first connect [MongoNetworkError: connect ECONNREFUSED 127.0.0.1:27017] npm notice created a lockfile as package-lock.json. You should commit this file Make Axios send cookies in its requests automatically What does body-parser do with express? SyntaxError: Unexpected token function - Async Await Nodejs Route.get() requires callback functions but got a "object Undefined" How to redirect to another page in node.js

Examples related to httprequest

Http post and get request in angular 6 Postman: How to make multiple requests at the same time Adding header to all request with Retrofit 2 Understanding Chrome network log "Stalled" state Why is this HTTP request not working on AWS Lambda? Simulate a specific CURL in PostMan HTTP Request in Swift with POST method What are all the possible values for HTTP "Content-Type" header? How to get host name with port from a http or https request Why I get 411 Length required error?