Based on our scientific structure, the experiment was summarised by looking at why, what and how. The current landscape is predominantly centred around end to end testing which simulates user behaviour.
We believe that writing unit tests before implementing a change will result in faster implementation and less risk of problems with the change.
We will be experimenting by writing unit tests for a change involving complex logic before implementing the change itself.
We will know this experiment is a success when the unit tests are passing, the change is implemented and passes UAT.
Firstly, lets re-visit the differences between unit testing and end to end testing. The idea of a unit, is to break down the code into small, easily testable parts
. It could be a single function, class or in our case, complex algorithm. This differs from an end to end test where you're testing the entire application. The point of an end to end test is to simulate how a user would use the application. Scratch
creates a good template around this. If I do X then I expect to see XYZ.
To evaluate the success, test driven development was adopted as part of four pieces of functionality on a complex public sector project. The team implemented a number of unit tests for the four pieces of functionality while still creating a significant number of end-to-end tests.
Unit tests work well as another tool in our toolkit to compliment our end-to-end tests.
They are not intended as a replacement for end-to-end tests, but given they're more reliable, faster and lower maintenance, they are a good option in some situations.
The biggest benefit is for long-running products with frequent maintenance (eg. large public sector projects), or for areas of code where the logic is complex and difficult to decipher when results go wrong.
Everyone should keep an eye out for appropriate opportunities where unit tests would provide value.
Before the experiment, we gauged how the industry perceived unit testing. Google's Mike Wacker summed up the value of unit testing in this blog
. Unit tests are fast, reliable and help isolate features.
The other key benefit that we saw reflected in our results was the speed to finding and fixing bugs. With end-to-end tests, you have to wait for the software to be built, then for it to be deployed, and finally for all end-to-end tests to run.
While this is a great tool to have in our toolkit for complex logic, there is still significant value in running end to end tests.