Software quality and agile

I want to share some thoughts based on recent discussions I've been having with developers around software quality and agile. I have seen many cases where development teams dogmatically following their view of an agile (mainly scrum) process has a negative effect on software quality. What do you think?

 

 

Please don’t rewrite your system from scratch

It seems there have been a lot of discussions online recently around rewriting software.

I think it’s a great discussion to have, as on many many different projects, I have seen developers clamouring to rewrite the software, to update to modern tooling, to fix large sets of issues as they see them in the codebase.

For the most part, prototypes of a newer, modern codebase can be created in a matter of hours. It can be very easy to assume that with modern tooling, and a prototype, rewriting a system is an easy job.

For the most part it is not.

I have personally seen a number of rewrite projects, and for the most part these rewrites are far from easy. With a rewrite, all of the necessary complexity of the system has to be addressed all at once.

Most of these recent discussions would also advocate incremental improvement over a rewrite, refactoring towards a target architecture. By all means, create a prototype of where you think the system should be. Then, armed with that information, work out how you can get there gradually.

The first part of this has to be understanding your existing technical debt.

Work out what is costing the business the most right now, and look at refactoring that towards your vision of where you should be. It might take a while to get there, and you will have to take many steps along that path, but you will be doing so in a much more controlled less risky way, providing more benefit up front.

In my new course, ‘Dealing With Technical Debt’, I go into this in more detail, and look at ways we can compare instances of technical debt in order to prioritise them. Once prioritised, we can look at improving the system in a deliberate way, tackling items that will bring the most benefit first.

Sometimes you do indeed NEED a rewrite. And sometimes you can even pull it off, but often it’s better for the business and all developers involved to put in place a strategy for dealing with your technical debt.

When done right, this can actually allow developers to use the languages, techniques and tools needed to develop as they should, but do so in a more gradual way, allowing results to be seen early.

What do you think about rewrite vs incremental improvement? Have you seen any cases where a rewrite was successful? How do you manage your technical debt?

On the value of not being busy

As part of teaching courses for developers online, in person, and through actually working as part of development teams, I receive many questions and am involved in many conversations around software development.

I want to start sharing some of the topics discussed and thoughts here in the hope that others will find this useful too.

I've just started using anchor, and think it's a very interesting platform for this kind of micro podcasting and discussion. The anchor is embedded below so you can listen here, but if you haven't heard of anchor, I would highly encourage you to check it out, and reply to my wave with your own audio!

 

 

New course on technical debt!

I’m pleased to announce a new mini course, and a different format for this one.

This new course, “Dealing With Technical Debt”, as the title suggests is all about technical debt. This course covers:

  • What we mean by technical debt and why it doesn’t matter if some people disagree with that definition
  • The different causes and types of technical debt
  • A model for comparing technical debt and why not all tech debt is equal
  • A way to break down technical debt and find the most important issues to start fixing

This is a 4 part email based course, and if you are interested in signing up for FREE click here and start fixing your technical debt.

Asynchronous Node.js and testing

Let’s face it, most of our application code we write is going to be doing something asynchronously. Whether it’s talking to a database of some form, or making API calls, there is going to be quite a bit of asynchronous code in any system.

In the Node.js world, or JavaScript world in general, many people don’t know whether they should just stick with callbacks, go with the promises route, or even use generators for async flow control. There are pros and cons to each approach.

As well as the application code, many devs also wonder how they should even be testing this code. On many occassions, I’ve seen very capable developers not follow a TDD approach, or fail to write tests that they know they should, because getting your head around how best to test asynchronous code is not necessarily easy if you aren’t used to it.


So, in this post, I want to tackle this and demonstrate what a simple asynchronous function might look like if using callbacks, promises, or generators. I want to then show how to write tests for each implementation. Hopefully this demonstration should give you an understanding of the differences in code structure and test structure for each approach.
For our example, each function will perform the same action of fetching a list of ‘posts’ from an API, and then for the first post, will fetch details of the user that created that post in a seperate API call. If you want to see the complete application and test code for this example, then please see the github repo.

The function will then return the name of the user and title of the post

So, first up, the typical callback approach. Here’s the application code:

function fetchWithCallback(cb) {
  var request = require('request')

  request('http://jsonplaceholder.typicode.com/posts', function(err, res, body1) {
    if (err) return cb(err)
    var posts = JSON.parse(body1),
        userId = posts[0].userId
    request('http://jsonplaceholder.typicode.com/users/'+userId, function(err, res, body2) {
      if (err) return cb(err)
      var user = JSON.parse(body2)
      return cb(null, {
        name: user.name,
        post: posts[0].title
      })
    })
  })
}

 

And now the code that tests this. Note, we are using mocha as the test runner, as well as chai, sinon and a couple of other modules. You can see the full source code and run the tests for all examples at https://github.com/fluentsoftware/async-code-test

it('should make an API request using callbacks', function(done) {
    var requestStub = sinon.stub()

    requestStub.onCall(0).callsArgWith(1, null, null, '[{"title": "first post", "userId": 1}]')
      .onCall(1).callsArgWith(1, null, null, '{"name": "alice"}')

    var client = proxyquire('../src/index', {
      request: requestStub
    })

    client.fetchWithCallback(function(err, data) {
      try {
        requestStub.should.have.been.calledWith('http://jsonplaceholder.typicode.com/posts')
        requestStub.should.have.been.calledWith('http://jsonplaceholder.typicode.com/users/1')

        data.should.eql({
          name: 'alice',
          post: 'first post'
        })
        done()
      } catch (e) {
        done(e)
      }
    })
  })

 

Hopefully there’s nothing too revolutionary here. We are using the request library to make the http requests, using data from the first request to make an additional request, and then invoking the given callback with the transformed data.

In our next example, we will see what this looks like using Promises rather than callbacks. We are using the request-promise library here, which provides a ‘promisified’ version of request.

function fetchWithPromise() {
  var request = require('request-promise')
  return request('http://jsonplaceholder.typicode.com/posts')
    .then(function(body1) {
      var posts = JSON.parse(body1),
          userId = posts[0].userId
      return request('http://jsonplaceholder.typicode.com/users/' + userId)
        .then(function(body2) {
          var user = JSON.parse(body2)
          return {
            name: user.name,
            post: posts[0].title
          }
        })
    })
}

 

And again, our code to test this

it('should make an API request using promises', function() {
    var requestPromiseStub = sinon.stub()

    requestPromiseStub.onCall(0).returns(Promise.resolve('[{"title": "first post", "userId": 1}]'))
      .onCall(1).returns(Promise.resolve('{"name": "alice"}'))

    var client = proxyquire('../src/index', {
      'request-promise': requestPromiseStub
    })

    return client.fetchWithPromise().then(function(res) {
      requestPromiseStub.should.have.been.calledWith('http://jsonplaceholder.typicode.com/posts')
      requestPromiseStub.should.have.been.calledWith('http://jsonplaceholder.typicode.com/users/1')

      res.should.eql({
        name: 'alice',
        post: 'first post'
      })
    })
  })

 

Some points to note comparing the promise based version to callbacks are

  1. Error handling has changed. The chaining of promises means that we don’t need to handle the error cases in each callback function. If any error happens in either API call, or the functions passed into ‘then’, they bubble up to the caller of our API function.
  2. The levels of nesting are still present as we are making an http request based on data returned from the first. We could remove this nesting by having the first callback return intermediate data containing the post title and user id, but it would complicate the code

So, the promise based approach looks pretty similar to the callback version, but has improved error handling behaviour, and in some cases could have better levels of nesting than the callback approach.

From a test code perspective, we no longer need the ‘done’ parameter in our test function. Instead, we can just return a promise from our test function. Mocha will mark the test as failed if the promise results in an error being thrown.

So, now onto the generator approach. In this example, we are still using the promisified version of request, but are also using the ‘co’ library, which will allow us to use generator functions and the ‘yield’ keyword to have asynchronous flow control. Promises can be yielded, with execution effectively resumable when the promise is resolved or rejected.

function* fetchWithGenerator() {
  var request = require('request-promise'),
      body1 = yield request('http://jsonplaceholder.typicode.com/posts'),
      posts = JSON.parse(body1),
      userId = posts[0].userId,
      body2 = yield request('http://jsonplaceholder.typicode.com/users/' + userId),
      user = JSON.parse(body2)
  return {
    name: user.name,
    post: posts[0].title
  }
}

 

And the test code again:

it('should make an API request using generators and co', function() {
    var requestPromiseStub = sinon.stub()

    requestPromiseStub.onCall(0).returns(Promise.resolve('[{"title": "first post", "userId": 1}]'))
      .onCall(1).returns(Promise.resolve('{"name": "alice"}'))

    var client = proxyquire('../src/index', {
      'request-promise': requestPromiseStub
    })

    return co(function*() {
      var res = yield client.fetchWithGenerator()
      requestPromiseStub.should.have.been.calledWith('http://jsonplaceholder.typicode.com/posts')
      requestPromiseStub.should.have.been.calledWith('http://jsonplaceholder.typicode.com/users/1')

      res.should.eql({
        name: 'alice',
        post: 'first post'
      })
    })
  })

 

In this case, you can see the application code is greatly simplified, assuming you are happy with developers understanding the yield behaviour, and how the co module works.

The code reads much more like a synchronous version except it executes the same way the promise based version would. Errors and exceptions are handled in the same way they would be in synchronous code, so if the http requests fail, or there is another exception in the code, the caller can just surround the code with a try catch block.

This approach just requires that at the top level, generator functions are wrapped in a ‘co’ call which will return a promise that is resolved/rejected based on the generator function call. The test for this, then remains very similar to the promise version except we having this wrapping ‘co’ function call.
I hope these examples have given an illustration as to how the same set of asynchronous code can look implemented in 3 different ways. I hope that this post also illustrates how to go about testing asynchronous code so that whether you use TDD or write unit tests after the fact, you can develop quality tested code quickly.

For our handy cheatsheet on Node.js testing, including how to test asynchronous code, download the PDF here.

Testing made easy

We know that writing tests as you code can be annoying, but it’s something that most of us agree we really should be doing.

To make this easy, and avoid having to remember which libraries to use, how to use them, and have a browser full of tabs just to remember how best to write the test, we’ve created a handy Node.js testing cheatsheet.

We hope you find this useful, and if you have any thoughts or comments, feel free to let us know below.

Download the cheatsheet

Developers in the real world

As part of teaching courses online, and also mentoring devs that I work with, I often get asked about the systems I’ve worked on, designed and developed. I think there is a massive amount of value to be had in going beyond the basics and understanding how other systems are put together.

After all, you wouldn’t expect an author to write a book who hadn’t read quite a few too.

To that end, I’m pleased to announce I’m going to be starting a podcast soon. It will start off as an experiment and if people find it useful I will continue recording further episodes. In this podcast I want to interview other software developers, dev leads and architects to understand how their systems are designed, and help share the lessons they have learned. I want to expose how teams are building software now, and help improve the way we work.

If you are interested in sharing what you are working on and lessons you feel could benefit others, or even just have an idea as to the kind of things you would like to hear about,just fill out the form below and let us know what you think!

Your Name (required)

Your Email (required)

Your Message

Test driven development in Java

Firstly, welcome to the new site. We’ve been making a few changes to how we run our courses, and the new site is going to give you access to better content more relevant to support your online learning with us.

If you’re one of the many students taking our Java TDD course, then we want to hear from you and understand more about what you are interested in learning. Do you feel there are gaps in your knowledge relating to other aspects of testing, software design or development?

So, please click here to take a super quick survey (less than 2 minutes) and help us choose what course content we will create next.