Asynchronous Node.js and testing

Let’s face it, most of our application code we write is going to be doing something asynchronously. Whether it’s talking to a database of some form, or making API calls, there is going to be quite a bit of asynchronous code in any system.

In the Node.js world, or JavaScript world in general, many people don’t know whether they should just stick with callbacks, go with the promises route, or even use generators for async flow control. There are pros and cons to each approach.

As well as the application code, many devs also wonder how they should even be testing this code. On many occassions, I’ve seen very capable developers not follow a TDD approach, or fail to write tests that they know they should, because getting your head around how best to test asynchronous code is not necessarily easy if you aren’t used to it.


So, in this post, I want to tackle this and demonstrate what a simple asynchronous function might look like if using callbacks, promises, or generators. I want to then show how to write tests for each implementation. Hopefully this demonstration should give you an understanding of the differences in code structure and test structure for each approach.
For our example, each function will perform the same action of fetching a list of ‘posts’ from an API, and then for the first post, will fetch details of the user that created that post in a seperate API call. If you want to see the complete application and test code for this example, then please see the github repo.

The function will then return the name of the user and title of the post

So, first up, the typical callback approach. Here’s the application code:

function fetchWithCallback(cb) {
  var request = require('request')

  request('http://jsonplaceholder.typicode.com/posts', function(err, res, body1) {
    if (err) return cb(err)
    var posts = JSON.parse(body1),
        userId = posts[0].userId
    request('http://jsonplaceholder.typicode.com/users/'+userId, function(err, res, body2) {
      if (err) return cb(err)
      var user = JSON.parse(body2)
      return cb(null, {
        name: user.name,
        post: posts[0].title
      })
    })
  })
}

 

And now the code that tests this. Note, we are using mocha as the test runner, as well as chai, sinon and a couple of other modules. You can see the full source code and run the tests for all examples at https://github.com/fluentsoftware/async-code-test

it('should make an API request using callbacks', function(done) {
    var requestStub = sinon.stub()

    requestStub.onCall(0).callsArgWith(1, null, null, '[{"title": "first post", "userId": 1}]')
      .onCall(1).callsArgWith(1, null, null, '{"name": "alice"}')

    var client = proxyquire('../src/index', {
      request: requestStub
    })

    client.fetchWithCallback(function(err, data) {
      try {
        requestStub.should.have.been.calledWith('http://jsonplaceholder.typicode.com/posts')
        requestStub.should.have.been.calledWith('http://jsonplaceholder.typicode.com/users/1')

        data.should.eql({
          name: 'alice',
          post: 'first post'
        })
        done()
      } catch (e) {
        done(e)
      }
    })
  })

 

Hopefully there’s nothing too revolutionary here. We are using the request library to make the http requests, using data from the first request to make an additional request, and then invoking the given callback with the transformed data.

In our next example, we will see what this looks like using Promises rather than callbacks. We are using the request-promise library here, which provides a ‘promisified’ version of request.

function fetchWithPromise() {
  var request = require('request-promise')
  return request('http://jsonplaceholder.typicode.com/posts')
    .then(function(body1) {
      var posts = JSON.parse(body1),
          userId = posts[0].userId
      return request('http://jsonplaceholder.typicode.com/users/' + userId)
        .then(function(body2) {
          var user = JSON.parse(body2)
          return {
            name: user.name,
            post: posts[0].title
          }
        })
    })
}

 

And again, our code to test this

it('should make an API request using promises', function() {
    var requestPromiseStub = sinon.stub()

    requestPromiseStub.onCall(0).returns(Promise.resolve('[{"title": "first post", "userId": 1}]'))
      .onCall(1).returns(Promise.resolve('{"name": "alice"}'))

    var client = proxyquire('../src/index', {
      'request-promise': requestPromiseStub
    })

    return client.fetchWithPromise().then(function(res) {
      requestPromiseStub.should.have.been.calledWith('http://jsonplaceholder.typicode.com/posts')
      requestPromiseStub.should.have.been.calledWith('http://jsonplaceholder.typicode.com/users/1')

      res.should.eql({
        name: 'alice',
        post: 'first post'
      })
    })
  })

 

Some points to note comparing the promise based version to callbacks are

  1. Error handling has changed. The chaining of promises means that we don’t need to handle the error cases in each callback function. If any error happens in either API call, or the functions passed into ‘then’, they bubble up to the caller of our API function.
  2. The levels of nesting are still present as we are making an http request based on data returned from the first. We could remove this nesting by having the first callback return intermediate data containing the post title and user id, but it would complicate the code

So, the promise based approach looks pretty similar to the callback version, but has improved error handling behaviour, and in some cases could have better levels of nesting than the callback approach.

From a test code perspective, we no longer need the ‘done’ parameter in our test function. Instead, we can just return a promise from our test function. Mocha will mark the test as failed if the promise results in an error being thrown.

So, now onto the generator approach. In this example, we are still using the promisified version of request, but are also using the ‘co’ library, which will allow us to use generator functions and the ‘yield’ keyword to have asynchronous flow control. Promises can be yielded, with execution effectively resumable when the promise is resolved or rejected.

function* fetchWithGenerator() {
  var request = require('request-promise'),
      body1 = yield request('http://jsonplaceholder.typicode.com/posts'),
      posts = JSON.parse(body1),
      userId = posts[0].userId,
      body2 = yield request('http://jsonplaceholder.typicode.com/users/' + userId),
      user = JSON.parse(body2)
  return {
    name: user.name,
    post: posts[0].title
  }
}

 

And the test code again:

it('should make an API request using generators and co', function() {
    var requestPromiseStub = sinon.stub()

    requestPromiseStub.onCall(0).returns(Promise.resolve('[{"title": "first post", "userId": 1}]'))
      .onCall(1).returns(Promise.resolve('{"name": "alice"}'))

    var client = proxyquire('../src/index', {
      'request-promise': requestPromiseStub
    })

    return co(function*() {
      var res = yield client.fetchWithGenerator()
      requestPromiseStub.should.have.been.calledWith('http://jsonplaceholder.typicode.com/posts')
      requestPromiseStub.should.have.been.calledWith('http://jsonplaceholder.typicode.com/users/1')

      res.should.eql({
        name: 'alice',
        post: 'first post'
      })
    })
  })

 

In this case, you can see the application code is greatly simplified, assuming you are happy with developers understanding the yield behaviour, and how the co module works.

The code reads much more like a synchronous version except it executes the same way the promise based version would. Errors and exceptions are handled in the same way they would be in synchronous code, so if the http requests fail, or there is another exception in the code, the caller can just surround the code with a try catch block.

This approach just requires that at the top level, generator functions are wrapped in a ‘co’ call which will return a promise that is resolved/rejected based on the generator function call. The test for this, then remains very similar to the promise version except we having this wrapping ‘co’ function call.
I hope these examples have given an illustration as to how the same set of asynchronous code can look implemented in 3 different ways. I hope that this post also illustrates how to go about testing asynchronous code so that whether you use TDD or write unit tests after the fact, you can develop quality tested code quickly.

For our handy cheatsheet on Node.js testing, including how to test asynchronous code, download the PDF here.

Leave a Reply