20 January 2016

What's the point of unit testing?

I'll start this post by saying that I have never been a huge fan of unit testing (this is in PHP), and though I have read a lot about it and also written unit tests myself, I have never read about or experienced anything with unit tests that really shows me it's worth in my development cycle.

Let me explain why.  Firstly I'll make this clear - I can absolutely see the importance of testing code and I always thoroughly test every execution path of every method I write.  Releasing code that you just write and don't test is clearly not a good idea.  So it's not that I am against testing the code I write at all.

The reason I don't see the point in unit testing is that I develop, and always have developed using an IDE and xdebug (which is the same thing I would hope any serious developer would be doing).  This of course allows me to fully test any code I want by setting breakpoints and stepping through code execution to inspect what is happening on a line by line basis.  To me stepping through code execution like this line by line, being able to see exactly what variables are being set to what, inspect objects and their contents, step into/out/over code offers so much more than unit testing when it comes to testing your code and verifying that it is working correctly. Both at a single method/class level as with unit testing, and at a wider level relying on external objects and services as with integration testing. While a unit test allows you to verify the end result of a particular code execution path, there are serious limitations on what can be realistically tested, for instance methods which don't return anything and just contain essentially business logic, protected/private methods etc. You can't directly test this with unit testing (or at least not without some kind of hacky approach using reflection and so on) but you can step through all of this with xdebug and breakpoints as you are testing the actual code execution flow on the actual class and method you want to test.

You will probably not have missed the fairly recent general release of Magento 2, and may well have noted that this ships with it's own testing framework for unit and integration testing.  Finding unit testing to now be such an integral part of the Magento codebase prompted me to revisit it in a bid to find what I may have missed before.  Surely there is just some important point to unit testing I just haven't managed to grasp yet?  Well...not that I have managed to find at the time of writing.

One of the things I have been doing recently is deciding whether or not to port our current Magento 1 extensions to work under Magento 2.  I say port, but in reality it's basically a total rewrite from scratch for every extension (which isn't a thrilling proposition, but that's another story).  I have been really enjoying working with the Magento 2 codebase and I think it genuinely is a long way ahead of Magento 1.  So rewriting extensions under Magento 2 is largely an enjoyable, if not substantial task.

So why not write unit tests at the same time for these extensions?  And this is how I was prompted to revisit unit testing.  So after a quick refresher on assertions, mock objects, mock and stub methods etc I came to writing some tests for one of our modules, which quickly moved to looking at existing unit tests in core modules.  I came across the following test for the catalog module in test class Magento\Catalog\Test\Unit\Block\NavigationTest:
public function testGetIdentities()
{
    $this->assertEquals(
        [\Magento\Catalog\Model\Category::CACHE_TAG, \Magento\Store\Model\Group::CACHE_TAG],
        $this->block->getIdentities()
    );
}
To me this is a good test to demonstrate the title of the post, what's the point of unit testing?

So as you can probably see, this is a simple test that checks that an array containing 2 class constants is the same as the return value of the getIdentities() method called on an object. Looking at $this->block we see that it is an instance of class Magento\Catalog\Block\Navigation instantiated by the object manager in the setUp() method (so not a mock). Looking at the class we can see that the getIdentities() method does the following:
public function getIdentities()
{
    return [\Magento\Catalog\Model\Category::CACHE_TAG, \Magento\Store\Model\Group::CACHE_TAG];
}
Yes, the method returns exactly the class constants tested for in the assert. Now unless I am missing something pretty fundamental, what is the point in this test exactly? It is nothing more than testing if a value equals itself, which of course it does. This is a test which can never fail as whatever either class constant may be changed to in the future it will still equal itself. To my mind this test just has no value at all, and due to the limited scope unit tests are designed for, summarises the overall usefulness of any unit test. I could understand if the class constant needed to be a certain value in order to be used to say pull some data from a database (which would cause the test to fail in the case of a change), but allowing reliance on external dependencies like this is not what unit tests are for. Any external dependency like this would be mocked with any methods which return data collected from some external source like this stubbed with the return value hard coded. Again it wouldn't matter what the class constants were changed to, the test would still pass because of the hard coded return value in the stub method. To me it seems that with the removal of dependencies all unit tests boil down to essentially the above and actually achieve nothing of worth.

So if somebody can tell me that the above test is just a poor example and enlighten me on the real point to unit testing I have been missing all this time, that would be great. At the moment the only point I can see to unit testing is to catch badly written code where the developer doesn't really understand the classes they are working/integrating with - and no I don't think that's a good enough reason for unit testing being as prevalent as it is.

I do understand that unit testing does have side benefits such as essentially acting as a type of documentation to the methods being tested, and giving developers who might be looking at code for the first time confidence in making changes by allowing them to run tests after making those changes. But that still doesn't get away from the fact that the developer still has to come to the point where they do understand the code they are working with, and in my company at least if you don't understand the code you are working with completely yet, the code you write doesn't get released until you do. Otherwise you are just intruding on cut and paste territory which is somewhere no worthwhile developer should ever be found.

Another side benefit, and the one that is perhaps the greatest is the way unit testing forces you to write your code. If you want to unit test you need to write code such that you can unit test, and this generally results in each method being as short and concise as possible, achieving only one goal. Dependencies should be passed to the methods that need them or be stored to the object rather than instantiated in the method itself, but Magento 2 pretty much has that covered with the object managers automatic dependency injection functionality for class constructors. Also logic which queries or uses any external dependency or service should happen in it's own method so when the object is mocked, that method can be stubbed to remove that dependency.

These side benefits are all great, and if wanting to write unit tests in a round about way forces you to follow these kind of coding conventions and principles then that can only be a good thing. But again I would hope that any developer with reasonable knowledge and experience would automatically be writing short and concise methods which are named sensibly, and perform only one task anyway. Regardless of coding this way being advantageous when it comes to writing unit tests, common sense should tell you this is a sensible way of doing things anyway as it makes your code so much more maintainable with it being very much quicker and easier to understand.

So please, use the comments below to give me your opinion and convince me I should be writing unit tests along side testing my code with xdebug!

No comments:

Post a Comment