[rest] How to make remote REST call inside Node.js? any CURL?

In Node.js, other than using child process to make CURL call, is there a way to make CURL call to remote server REST API and get the return data?

I also need to set up the request header to the remote REST call, and also query string as well in GET (or POST).

I find this one: http://blog.nodejitsu.com/jsdom-jquery-in-5-lines-on-nodejs

but it doesn't show any way to POST query string.

This question is related to rest curl node.js

The answer is


var http = require('http');
var url = process.argv[2];

http.get(url, function(response) {
  var finalData = "";

  response.on("data", function (data) {
    finalData += data.toString();
  });

  response.on("end", function() {
    console.log(finalData.length);
    console.log(finalData.toString());
  });

});

If you have Node.js 4.4+, take a look at reqclient, it allows you to make calls and log the requests in cURL style, so you can easily check and reproduce the calls outside the application.

Returns Promise objects instead of pass simple callbacks, so you can handle the result in a more "fashion" way, chain the result easily, and handle errors in a standard way. Also removes a lot of boilerplate configurations on each request: base URL, time out, content type format, default headers, parameters and query binding in the URL, and basic cache features.

This is an example of how to initialize it, make a call and log the operation with curl style:

var RequestClient = require("reqclient").RequestClient;
var client = new RequestClient({
    baseUrl:"http://baseurl.com/api/", debugRequest:true, debugResponse:true});
client.post("client/orders", {"client": 1234, "ref_id": "A987"},{"x-token": "AFF01XX"});

This will log in the console...

[Requesting client/orders]-> -X POST http://baseurl.com/api/client/orders -d '{"client": 1234, "ref_id": "A987"}' -H '{"x-token": "AFF01XX"}' -H Content-Type:application/json

And when the response is returned ...

[Response   client/orders]<- Status 200 - {"orderId": 1320934}

This is an example of how to handle the response with the promise object:

client.get("reports/clients")
  .then(function(response) {
    // Do something with the result
  }).catch(console.error);  // In case of error ...

Of course, it can be installed with: npm install reqclient.


I found superagent to be really useful, it is very simple for example

const superagent=require('superagent')
superagent
.get('google.com')
.set('Authorization','Authorization object')
.set('Accept','application/json')

Axios

An example (axios_example.js) using Axios in Node.js:

const axios = require('axios');
const express = require('express');
const app = express();
const port = process.env.PORT || 5000;

app.get('/search', function(req, res) {
    let query = req.query.queryStr;
    let url = `https://your.service.org?query=${query}`;

    axios({
        method:'get',
        url,
        auth: {
            username: 'the_username',
            password: 'the_password'
        }
    })
    .then(function (response) {
        res.send(JSON.stringify(response.data));
    })
    .catch(function (error) {
        console.log(error);
    });
});

var server = app.listen(port);

Be sure in your project directory you do:

npm init
npm install express
npm install axios
node axios_example.js

You can then test the Node.js REST API using your browser at: http://localhost:5000/search?queryStr=xxxxxxxxx

Similarly you can do post, such as:

axios({
  method: 'post',
  url: 'https://your.service.org/user/12345',
  data: {
    firstName: 'Fred',
    lastName: 'Flintstone'
  }
});

SuperAgent

Similarly you can use SuperAgent.

superagent.get('https://your.service.org?query=xxxx')
.end((err, response) => {
    if (err) { return console.log(err); }
    res.send(JSON.stringify(response.body));
});

And if you want to do basic authentication:

superagent.get('https://your.service.org?query=xxxx')
.auth('the_username', 'the_password')
.end((err, response) => {
    if (err) { return console.log(err); }
    res.send(JSON.stringify(response.body));
});

Ref:


How about using Request — Simplified HTTP client.

Edit February 2020: Request has been deprecated so you probably shouldn't use it any more.

Here's a GET:

var request = require('request');
request('http://www.google.com', function (error, response, body) {
    if (!error && response.statusCode == 200) {
        console.log(body) // Print the google web page.
     }
})

OP also wanted a POST:

request.post('http://service.com/upload', {form:{key:'value'}})

I didn't find any with cURL so I wrote a wrapper around node-libcurl and can be found at https://www.npmjs.com/package/vps-rest-client.

To make a POST is like so:

var host = 'https://api.budgetvm.com/v2/dns/record';
var key = 'some___key';
var domain_id = 'some___id';

var rest = require('vps-rest-client');
var client = rest.createClient(key, {
  verbose: false
});

var post = {
  domain: domain_id,
  record: 'test.example.net',
  type: 'A',
  content: '111.111.111.111'
};

client.post(host, post).then(function(resp) {
  console.info(resp);

  if (resp.success === true) {
    // some action
  }
  client.close();
}).catch((err) => console.info(err));

Look at http://isolasoftware.it/2012/05/28/call-rest-api-with-node-js/

var https = require('https');

/**
 * HOW TO Make an HTTP Call - GET
 */
// options for GET
var optionsget = {
    host : 'graph.facebook.com', // here only the domain name
    // (no http/https !)
    port : 443,
    path : '/youscada', // the rest of the url with parameters if needed
    method : 'GET' // do GET
};

console.info('Options prepared:');
console.info(optionsget);
console.info('Do the GET call');

// do the GET request
var reqGet = https.request(optionsget, function(res) {
    console.log("statusCode: ", res.statusCode);
    // uncomment it for header details
//  console.log("headers: ", res.headers);


    res.on('data', function(d) {
        console.info('GET result:\n');
        process.stdout.write(d);
        console.info('\n\nCall completed');
    });

});

reqGet.end();
reqGet.on('error', function(e) {
    console.error(e);
});

/**
 * HOW TO Make an HTTP Call - POST
 */
// do a POST request
// create the JSON object
jsonObject = JSON.stringify({
    "message" : "The web of things is approaching, let do some tests to be ready!",
    "name" : "Test message posted with node.js",
    "caption" : "Some tests with node.js",
    "link" : "http://www.youscada.com",
    "description" : "this is a description",
    "picture" : "http://youscada.com/wp-content/uploads/2012/05/logo2.png",
    "actions" : [ {
        "name" : "youSCADA",
        "link" : "http://www.youscada.com"
    } ]
});

// prepare the header
var postheaders = {
    'Content-Type' : 'application/json',
    'Content-Length' : Buffer.byteLength(jsonObject, 'utf8')
};

// the post options
var optionspost = {
    host : 'graph.facebook.com',
    port : 443,
    path : '/youscada/feed?access_token=your_api_key',
    method : 'POST',
    headers : postheaders
};

console.info('Options prepared:');
console.info(optionspost);
console.info('Do the POST call');

// do the POST call
var reqPost = https.request(optionspost, function(res) {
    console.log("statusCode: ", res.statusCode);
    // uncomment it for header details
//  console.log("headers: ", res.headers);

    res.on('data', function(d) {
        console.info('POST result:\n');
        process.stdout.write(d);
        console.info('\n\nPOST completed');
    });
});

// write the json data
reqPost.write(jsonObject);
reqPost.end();
reqPost.on('error', function(e) {
    console.error(e);
});

/**
 * Get Message - GET
 */
// options for GET
var optionsgetmsg = {
    host : 'graph.facebook.com', // here only the domain name
    // (no http/https !)
    port : 443,
    path : '/youscada/feed?access_token=you_api_key', // the rest of the url with parameters if needed
    method : 'GET' // do GET
};

console.info('Options prepared:');
console.info(optionsgetmsg);
console.info('Do the GET call');

// do the GET request
var reqGet = https.request(optionsgetmsg, function(res) {
    console.log("statusCode: ", res.statusCode);
    // uncomment it for header details
//  console.log("headers: ", res.headers);


    res.on('data', function(d) {
        console.info('GET result after POST:\n');
        process.stdout.write(d);
        console.info('\n\nCall completed');
    });

});

reqGet.end();
reqGet.on('error', function(e) {
    console.error(e);
});

To use latest Async/Await features

https://www.npmjs.com/package/request-promise-native

npm install --save request
npm install --save request-promise-native

//code

async function getData (){
    try{
          var rp = require ('request-promise-native');
          var options = {
          uri:'https://reqres.in/api/users/2',
          json:true
        };

        var response = await rp(options);
        return response;
    }catch(error){
        throw error;
    }        
}

try{
    console.log(getData());
}catch(error){
    console.log(error);
}

I use node-fetch because it uses the familiar (if you are a web developer) fetch() API. fetch() is the new way to make arbitrary HTTP requests from the browser.

Yes I know this is a node js question, but don't we want to reduce the number of API's developers have to memorize and understand, and improve re-useability of our javascript code? Fetch is a standard so how about we converge on that?

The other nice thing about fetch() is that it returns a javascript Promise, so you can write async code like this:

let fetch = require('node-fetch');

fetch('http://localhost', {
  method: 'POST',
  headers: {'Content-Type': 'application/json'},
  body: '{}'
}).then(response => {
  return response.json();
}).catch(err => {console.log(err);});

Fetch supersedes XMLHTTPRequest. Here's some more info.


Warning: As of Feb 11th 2020, request is fully deprecated.

One another example - you need to install request module for that

var request = require('request');
function get_trustyou(trust_you_id, callback) {
    var options = {
        uri : 'https://api.trustyou.com/hotels/'+trust_you_id+'/seal.json',
        method : 'GET'
    }; 
    var res = '';
    request(options, function (error, response, body) {
        if (!error && response.statusCode == 200) {
            res = body;
        }
        else {
            res = 'Not Found';
        }
        callback(res);
    });
}

get_trustyou("674fa44c-1fbd-4275-aa72-a20f262372cd", function(resp){
    console.log(resp);
});

You can use curlrequest to easily set what time of request you want to do... you can even set headers in the options to "fake" a browser call.


I have been using restler for making webservices call, works like charm and is pretty neat.


Warning: As of Feb 11th 2020, request is fully deprecated.

If you implement with form-data, for more info (https://tanaikech.github.io/2017/07/27/multipart-post-request-using-node.js):

var fs = require('fs');
var request = require('request');
request.post({
  url: 'https://slack.com/api/files.upload',
  formData: {
    file: fs.createReadStream('sample.zip'),
    token: '### access token ###',
    filetype: 'zip',
    filename: 'samplefilename',
    channels: 'sample',
    title: 'sampletitle',
  },
}, function (error, response, body) {
  console.log(body);
});

Examples related to rest

Access blocked by CORS policy: Response to preflight request doesn't pass access control check Returning data from Axios API Access Control Origin Header error using Axios in React Web throwing error in Chrome JSON parse error: Can not construct instance of java.time.LocalDate: no String-argument constructor/factory method to deserialize from String value How to send json data in POST request using C# How to enable CORS in ASP.net Core WebAPI RestClientException: Could not extract response. no suitable HttpMessageConverter found REST API - Use the "Accept: application/json" HTTP Header 'Field required a bean of type that could not be found.' error spring restful API using mongodb MultipartException: Current request is not a multipart request

Examples related to curl

What is the incentive for curl to release the library for free? curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number Converting a POSTMAN request to Curl git clone error: RPC failed; curl 56 OpenSSL SSL_read: SSL_ERROR_SYSCALL, errno 10054 How to post raw body data with curl? Curl : connection refused How to use the curl command in PowerShell? Curl to return http status code along with the response How to install php-curl in Ubuntu 16.04 curl: (35) SSL connect error

Examples related to node.js

Hide Signs that Meteor.js was Used Querying date field in MongoDB with Mongoose SyntaxError: Cannot use import statement outside a module Server Discovery And Monitoring engine is deprecated How to fix ReferenceError: primordials is not defined in node UnhandledPromiseRejectionWarning: This error originated either by throwing inside of an async function without a catch block dyld: Library not loaded: /usr/local/opt/icu4c/lib/libicui18n.62.dylib error running php after installing node with brew on Mac internal/modules/cjs/loader.js:582 throw err DeprecationWarning: Buffer() is deprecated due to security and usability issues when I move my script to another server Please run `npm cache clean`