What should we test?

This is always a good question. But to properly answer it, we need to make some assumptions:

  • Tests don’t give us a guarantee that our systems will work. It helps, but it is not a guarantee.
  • If the tests take too long to run, no one will run the tests
  • Good coverage is not synonymous for good tests.
  • Tests are good when they are meaningful, when they are test something important. Otherwise, they are useless.

Having said that, what we should test? If we write thousands of integration and/or functional tests, we would test a lot of features, but our build would take too much time to run. It would be easy to developers forget about those tests. So it is not a good approach. We could create thousands of unit tests, the build would be quick and we would have a very fast feedback. It is a good approach, however, it wouldn’t test the classes working together, so  it is still not the best approach.

In my opinion, the best approach is the test pyramid.

We still would have lots of unit tests. They are good for testing classes and allow you to use TDD as a tool for Emergent Design. They are not suffiicent though.

We could also create some integration tests to test our classes together (e.g. add tests in the Service Layer). However, we should only add tests that are really necessary since they would take too long to run. The goal would be to test critical features or only happy-path scenarios.

After that, we should have the end-to-end tests (in the picture above called UI tests). In these tests, we want to test a feature like the user would. This DOESN’T mean testing javascript. For that, we have frameworks like Jasmine.  We could add only few of those end-to-end tests. Only to test a whole feature that we cannot tests on our integration tests.

In summary, there are some approaches to test our software. But the most important thing is that our tests should be meaningful and should have value. It should test what is important. The test pyramid approach is a good guide to make us test the right thing at the right level.

Advertisements

2 thoughts on “What should we test?

  1. Hi Mario,

    I believe that building “thousands” of test units will be counter-productive. Testing should be done thoroughly – but automating testing for every single feature can be a complete mess – especially if the requirements are fluid.

    I’m a firm believer that automating tests is not for all projects and should not be used for every feature of the project…

    • Hi (I wasn’t able to figure out your name, sorry)!

      Thanks for your comment. Unit tests should test only public methods (mocking the dependencies), not an entire feature. Create tests for every single feature is not good, I agree, but it is not an unit test. Unit tests, in my opinion, is also a tool for evolutionary design. It is more about help you to keep your methods with single responsibility and simple. Like that concepts (KISS and DRY). But I still think that automating tests is for all projects, you need to have some kind of feedback often. There are some ways to have it, like show for the client, do manual testing, but automated tests is a good way too.

      I’m curious to know in which situation you wouldn’t create automated tests!

      Thanks again!

      Mário

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s