[api] Automated testing for REST Api

I would like to write an automated testing suite for a REST API. As we complete new services, we'd like to check to make sure all the previously created services are working as expected. Any suggestions on the best tools to use to accomplish this? I know tools like Apigee exist that allow you to test 1 service at a time, but we'd like for a way to test all services with the click of a button.

One of the problems of doing automated testing for APIs is that many of the tools require you to have the API server up and running before you run your test suite. It can be a real advantage to have a unit testing framework that is capable of running and querying the APIs in a fully automated test environment.

An option that's good for APIs implemented with Node.JS / Express is to use mocha for automated testing. In addition to unit tests, its easy to write functional tests against the APIs, separated into different test suites. You can start up the API server automatically in the local test environment and set up a local test database. Using make, npm, and a build server, you can create a "make test" target and an incremental build that will run the entire test suite every time a piece of code is submitted to your repository. For the truly fastidious developer, it will even generate a nice HTML code-coverage report showing you which parts of your code base are covered by tests or not. If this sounds interesting, here's a blog post that provides all the technical details.

If you're not using node, then whatever the defacto unit testing framework for the language is (jUnit, cucumber/capybara, etc) - look at its support for spinning up servers in the local test environment and running the HTTP queries. If it's a large project, the effort to get automated API testing and continual integration working will pay off pretty quickly.

Hope that helps.

API test automation, up to once per minute, is a service available through theRightAPI. You create your test scenarios, and execute them. Once those tests do what you expect them to, you can then schedule them. Tests can be 'chained' together for scenarios that require authentication. For example, you can have a test that make an OAuth request to Twitter, and creates a shared token that can then be used by any other test. Tests can also have validation criteria attached to ensure http status codes, or even detailed inspection of the responses using javascript or schema validation. Once tests are scheduled, you can then have alerts notify you as soon as a particular test fails validation, or is behaving out of established ranges for response time or response size.

At my work we have recently put together a couple of test suites written in Java to test some RESTful APIs we built. Our Services could invoke other RESTful APIs they depend on. We split it into two suites.

• Suite 1 - Testing each service in isolation
  • Mock any peer services the API depends on using restito. Other alternatives include rest-driver, wiremock and betamax.
  • Tests the service we are testing and the mocks all run in a single JVM
  • Launches the service in Jetty

I would definitely recommend doing this. It has worked really well for us. The main advantages are:

• Peer services are mocked, so you needn't perform any complicated data setup. Before each test you simply use restito to define how you want peer services to behave, just like you would with classes in unit tests with Mockito.
• You can ask the mocked peer services if they were called. You can't do these asserts as easily with real peer services.
• The suite is super fast as mocked services serve pre-canned in-memory responses. So we can get good coverage without the suite taking an age to run.
• The suite is reliable and repeatable as its isolated in it's own JVM, so no need to worry about other suites/people mucking about with an shared environment at the same time the suite is running and causing tests to fail.

• Suite 2 - Full End to End
  • Suite runs against a full environment deployed across several machines
  • API deployed on Tomcat in environment
  • Peer services are real 'as live' full deployments

This suite requires us to do data set up in peer services which means tests generally take more time to write. As much as possible we use REST clients to do data set up in peer services.

Tests in this suite usually take longer to write, so we put most of our coverage in Suite 1. That being said there is still clear value in this suite as our mocks in Suite 1 may not be behaving quite like the real services.

I have used TestNG and Apache HTTP classes to build my own REST API test framework, I developed this concept after working in Selenium for two years.

Everything is same, except you should use Apache HTTP classes instead of Selenium classes.

give a try, its really cute and good, you've all the power to customize your test framework to your fullest possibilities.

Frisby is a REST API testing framework built on node.js and Jasmine that makes testing API endpoints easy, fast, and fun. http://frisbyjs.com

Example:

var frisby = require('../lib/frisby');

var URL = 'http://localhost:3000/';
var URL_AUTH = 'http://username:password@localhost:3000/';

frisby.globalSetup({ // globalSetup is for ALL requests
  request: {
    headers: { 'X-Auth-Token': 'fa8426a0-8eaf-4d22-8e13-7c1b16a9370c' }
  }
});

frisby.create('GET user johndoe')
  .get(URL + '/users/3.json')
  .expectStatus(200)
  .expectJSONTypes({
    id: Number,
    username: String,
    is_admin: Boolean
  })
  .expectJSON({
    id: 3,
    username: 'johndoe',
    is_admin: false
  })
  // 'afterJSON' automatically parses response body as JSON and passes it as an argument
  .afterJSON(function(user) {
    // You can use any normal jasmine-style assertions here
    expect(1+1).toEqual(2);

    // Use data from previous result in next test
    frisby.create('Update user')
      .put(URL_AUTH + '/users/' + user.id + '.json', {tags: ['jasmine', 'bdd']})
      .expectStatus(200)
    .toss();
  })
.toss();

I collaborated with one of my coworkers to start the PyRestTest framework for this reason: https://github.com/svanoort/pyresttest

Although you can work with the tests in Python, the normal test format is in YAML.

Sample test suite for a basic REST app -- verifies that APIs respond correctly, checking HTTP status codes, though you can make it examine response bodies as well:

---
- config:
    - testset: "Tests using test app"

- test: # create entity
    - name: "Basic get"
    - url: "/api/person/"
- test: # create entity
    - name: "Get single person"
    - url: "/api/person/1/"
- test: # create entity
    - name: "Get single person"
    - url: "/api/person/1/"
    - method: 'DELETE'
- test: # create entity by PUT
    - name: "Create/update person"
    - url: "/api/person/1/"
    - method: "PUT"
    - body: '{"first_name": "Gaius","id": 1,"last_name": "Baltar","login": "gbaltar"}'
    - headers: {'Content-Type': 'application/json'}
- test: # create entity by POST
    - name: "Create person"
    - url: "/api/person/"
    - method: "POST"
    - body: '{"first_name": "Willim","last_name": "Adama","login": "theadmiral"}'
    - headers: {Content-Type: application/json}

Runscope is a cloud based service that can monitor Web APIs using a set of tests. Tests can be , scheduled and/or run via parameterized web hooks. Tests can also be executed from data centers around the world to ensure response times are acceptable to global client base.

The free tier of Runscope supports up to 10K requests per month.

Disclaimer: I am a developer advocate for Runscope.

You can also use Rest Assured library. For a demo with sample script, refer to http://artoftesting.com/automationTesting/restAPIAutomationGetRequest.html

I implemented many automation cases based on REST Assured , a jave DSL for testing restful service. https://code.google.com/p/rest-assured/

The syntax is easy, it supports json and xml. https://code.google.com/p/rest-assured/wiki/Usage

Before that, I tried SOAPUI and had some issues with the free version. Plus the cases are in xml files which hard to extend and reuse, simply I don't like

I used SOAP UI for functional and automated testing. SOAP UI allows you to run the tests on the click of a button. There is also a spring controllers testing page created by Ted Young. I used this article to create Rest unit tests in our application.