The importance of testing – unit tests, acceptance tests and so on – is difficult to overstate.

Tests are essential for developers, testers and the business, and getting your testing right is especially important since it will remain a critical element even when the software is in production.

With tests being so crucial, then, every team should have a strategy around testing. It should cover:

• Choosing the right tools
• Deciding when the tests will be written
• How those tests should be written, to provide early feedback on what is broken

Getting early, detailed feedback from any failed test is critical. For example, is it the business logic, an integration with a dependent service, or the GUI layer that is broken? Only with the right tests can your team get the necessary confidence to fix things promptly.

In this article, I’ll touch upon a few points I believe can help development teams to create a faster, effective, more reliable test structure.

Choosing the right tools

Tool selection will depend upon what you want to test, as well as the team’s familiarity with the options available. You can then segregate tests accordingly, using different types to test the right aspect of your software effectively, and in less time.

The testing pyramid is a well-known and effective concept that we use to describe our test segregation. It’s essential that the entire team decides and knows its testing strategy (documented within a pyramid if you so wish), so that everyone follows the decided route. In the spirit of continual improvement, the approach should also be reviewed on a regular basis, with changes made when required to improve confidence in tests and build times.

Why use contract tests?

When we write our acceptance tests, we want to exclude any external services (e.g. SOAP/REST services); after all, we’re only looking to test the behaviour of our application. So we write our acceptance tests using a mock or a stub (fakes) as the foundation, mimicking the dependent service in question.

But how do you know that these fakes are correctly mimicking the real external services (the ‘contract’ in contract tests)? And how do you get feedback if and when one of the contracts breaks (ie. a dependent service introduces a breaking change)?

Without the right contract tests, we wouldn’t get to know of a breaking change until our application is deployed in an environment against real dependent services. To get around this we introduce contract tests to test our fakes against real dependent services.

The important aspect here is to create an abstraction layer around your fakes. Our acceptance tests and contract tests use the same abstraction, to ensure that the acceptance test uses the same fakes that have been tested against the real service.

As we add or modify contracts on which our acceptance tests depends, we add a corresponding contract test.

There is an argument that this layer is redundant, because integration points or the contracts will automatically be tested in a smaller suite of tests, run against a real environment (sometimes called end to end tests). But here are three reasons why end to end tests shouldn’t take the place of your contract tests:

• End to end tests are expensive, since they take longer to run;
• They provide late feedback, because they run very late in the build pipeline;
• An end to end test can break for multiple reasons (a change in contract, a bug in your application, or an environment issue).

Contract tests address these problems. They offer a faster, dedicated suite to test the correctness of fakes and catch any changes in the contract, and also isolate where any problem lies. With a solid foundation of contract tests, your acceptance test will provide your team with good confidence on how the software will behave against real services.

Why proper acceptance testing doesn’t mean long build times

I’ve occasionally heard developers advocating faster unit tests to cover most acceptance criteria – and only use slower acceptance tests to cover the basic acceptance criteria. The idea is that this will provide faster build times. But for several reasons, I’d argue that to release software with confidence, you must have acceptance tests (‘black box tests’) around all acceptance criteria.

Firstly, a unit test cannot provide any guarantee on how your software is going to behave. It can only guarantee how a specific unit is going to behave, and usually the boundary of these units is very small – only up to a method or class level. Unit tests are for helping you to evolve better design incrementally, and break bigger problems into chunks that can be tested individually.

The second issue with using unit tests for acceptance criteria, in my opinion, is that it papers over the real issue of longer build times, without addressing the real problem of what’s causing the long build times.

Certainly, if an application is a web-based application with a GUI, a very big chunk of build time will go into running browser-based acceptance tests. But it’s a myth that acceptance tests can only be written using a browser-based testing style.

To overcome the problem, we adopt a new style of testing – one which does not use a browser for a large number of acceptance criterias.

A web application essentially exposes a bunch of endpoints (for example, GET, POST and so on). By setting up your application data correctly for those endpoints correctly, you can then write tests that check them using any HTTP client. You can then test the behaviour of each endpoint in isolation.

This is very similar to testing a RESTful service – the only difference is the response for a GUI-based application will be HTML. We can use HTML parsing libraries (e.g. jsoup) to parse the response and write appropriate assertions. Note that this will work effectively only for the cases where there is no client side rendering. If required, client side rendering can be tested well in faster javascript tests, or browser-based tests.

Browser-based testing is still useful

Having said that, I’m not suggesting we completely throw away browser tests, or don’t write them. Browser-based testing tools are ideal to test the stitching of pages and core acceptance criteria – call them GUI acceptance tests. Ultimately, it boils down to correctly identifying which aspects can be tested without an actual browser – and which can help you get shorter build times – without ‘papering over the cracks’ by only using unit tests.

Of course, the methods and views I’ve expressed here are not the only way to build a robust, reliable and quick test suite – but I do think teams will reap good benefits from introducing them if they are not doing so already.

It’s also worth remembering that automation shouldn’t completely replace manual processes – exploratory testing remains a crucial part of an overall test strategy. However you approach it, evolving the right testing strategy is essential for any development team, but be sure to review and adjust it continuously as the work develops.