18 March 2016

Rising to the top

I've been involved in web development for over 15 years now, and have been doing it professionally for the last six years, but how did I get to this point?  This is the tale of how I made setting out in business on my own work (and still am)!

Six years ago I quit my successful central London job where I was an imaging engineer in one of the top three post production facilities in the UK, working on film and and TV projects from the biggest names in the business.  The company employed hundreds of both technical and creative staff, and it was a place where I was gradually rising through the engineering ranks and approaching senior level.

I really enjoyed the work I did there, but occasionally times just change.  I was to be getting married and London just didn't work for me or my wife to be any more, so a massive decision - and consequently change in circumstance was needed.  We moved from the big city to a small village in South Wales where I discovered there wasn't really much of a post production industry (which is mainly because it's really all in central London!).  After moving I looked extensively for work in the local area and found very little, but....

Freelance to the rescue!!!

Easy right?  I'll just become a freelancer, draw on and develop my web skills and make a living that way.  Unfortunately it seems at least one other person had the same idea.  Actually make that a LOT of people.  Freelance web development was a whole new ball game, no longer restricted by physical location, I was now competing with anyone else in any part of the world who had the same idea.  In order to survive in this world, you either need to just be ridiculously lucky (which will never realistically last), or pursue a better option - 'rise to the top' in your chosen area of work.

You may not be aware, but the online world gives freelancers an amazingly huge market place you can quickly leverage to look for work - there are countless freelance foccussed sites such as upwork, freelancer and peopleperhour where buyers can very easily post jobs, and sellers like me can very easily submit proposals for those jobs.  Simple, easy work, but almost seems to good to be true.

Well you know how that saying goes...
While this might not always be true, in this case it really, really is.  When you open that lovely looking jar of sweets that is the freelance jobs sites, you find out the jar is actually filled with those Jelly Belly beans that are made to look like the tasty ones, but bite into them and you find the complete opposite.

These freelance sites are in a way a victim of their own success - everything is just too easy, and too accessible, for anyone and everyone.  The result?  The market gets completely flooded and overwhelmed by the sheer number of freelancers offering their services.  There are people with all levels of skill and professionalism, and while there are both buyers with money to spend and good reputable sellers who charge a decent hourly rate, there are so many on there that fall well below this that they drown out the voices of good buyers and good sellers so that it comes down to pure luck when a genuinely good buyer who is willing to pay well for their work finds a genuinely good seller with skills and experience to justify the hourly rate they demand.

The end result is good buyers with great projects don't even use the sites because they know they will get flooded with at the very least 50+ very average to poor proposals and not end up finding anyone who can do the job to the level of professionalism they need.  Equally good sellers who rightly ask a decent hourly rate give up on the sites because the good buyers are no longer there.  So in the end all you get are what I call 'moon on a stick' jobs where the buyer wants everything for virtually nothing, and the sellers that are left take part in a ridiculous low bidding war for the work where no-one really wins.

Believe me, I can see the attraction from the buyers point of view, why have the work done for amount x when you can get the same work done for a quarter of the price on a freelance site.  The end result of these kinds of jobs are the sort I pick up later down the line when the buyer realises they actually do need to spend money after all to get their site built to a good standard.

My conclusion - these kind of freelance sites have their place, but not if you are looking for a great project which pays honestly and which needs to be completed by a competent, experienced professional who demands an hourly rate their skill and knowledge justifies.

Surely there must be a better way? (I knew you were beginning to wonder when the point was coming!).  Well while attempting to find out if there is indeed a better way, I came across this article on the top 15 freelance sites to find work.  While I don't agree with the number one slot being upwork, the number two slot definitely caught my attention.  Toptal sounded interesting as soon as I read the description:
"Toptal is for seasoned, talented freelancers. Passing Toptal's screening process gives you unparalleled access to meaningful projects with great clients (JPMorgan, Zendesk, Airbnb, etc.) and fair compensation (no low-bid contests)."
Going to the Toptal site it claims to allow buyers to:
"Hire the top 3% of freelance talent."
That's where the good buyers must have gone then?  And while hoping not to sound too self important, I do consider myself an expert in my field and believe I could certainly be in that top 3%.

While I am not part of Toptal just yet, I am genuinely looking forward to joining their Web Engineering Community and certainly have high hopes for the opportunities that might bring with it.  The great part about the joining process is that this one isn't 'too easy' or 'too accessible' - which is such a good thing because it addresses everything which doesn't work about other freelance sites.  A definite thumbs up from me.

So if you are an experienced freelancer who has the skills to complete work at the level expected by the best companies and businesses out there, I suggest you head over to Toptal like I did and start the joining process yourself.  Good luck and see you in the community!

27 January 2016

Magento database clone script

We recently added a new gist to our github account - a Magento 1 command line database syncing tool.  Most store owners will find the need to have at least a staging site for their store, and probably a development site as well.  While it's easy enough to get the staging environment mirroring production in terms of configuration, and the fileset handled by version control, syncing the database contents is less straightforward - but none the less desirable.

In an environment where you want to mirror production as closely as possible both in terms of configuration and data, having database contents lagging far behind production isn't ideal.  Take for instance a situation where you find a bug that is apparent on production but not in any other environment.  In my experience 99% of the time you eventually find issues like this to be down to differences in admin configuration, and therefore the database.  So it's important to have your stores staging environment properly synced to production to give yourself the best chance of finding any bugs before code is deployed to your production environment.

That's exactly what our syncing tool does - it requires n98-magerun and takes an already exported dump of your production database, imports it to your staging site and updates the install specific contents such as base url, cookie domain and secure front/backend settings.

You can find the script here with full usage instructions underneath.  I hope you find it useful!

20 January 2016

What's the point of unit testing?

I'll start this post by saying that I have never been a huge fan of unit testing (this is in PHP), and though I have read a lot about it and also written unit tests myself, I have never read about or experienced anything with unit tests that really shows me it's worth in my development cycle.

Let me explain why.  Firstly I'll make this clear - I can absolutely see the importance of testing code and I always thoroughly test every execution path of every method I write.  Releasing code that you just write and don't test is clearly not a good idea.  So it's not that I am against testing the code I write at all.

The reason I don't see the point in unit testing is that I develop, and always have developed using an IDE and xdebug (which is the same thing I would hope any serious developer would be doing).  This of course allows me to fully test any code I want by setting breakpoints and stepping through code execution to inspect what is happening on a line by line basis.  To me stepping through code execution like this line by line, being able to see exactly what variables are being set to what, inspect objects and their contents, step into/out/over code offers so much more than unit testing when it comes to testing your code and verifying that it is working correctly. Both at a single method/class level as with unit testing, and at a wider level relying on external objects and services as with integration testing. While a unit test allows you to verify the end result of a particular code execution path, there are serious limitations on what can be realistically tested, for instance methods which don't return anything and just contain essentially business logic, protected/private methods etc. You can't directly test this with unit testing (or at least not without some kind of hacky approach using reflection and so on) but you can step through all of this with xdebug and breakpoints as you are testing the actual code execution flow on the actual class and method you want to test.

You will probably not have missed the fairly recent general release of Magento 2, and may well have noted that this ships with it's own testing framework for unit and integration testing.  Finding unit testing to now be such an integral part of the Magento codebase prompted me to revisit it in a bid to find what I may have missed before.  Surely there is just some important point to unit testing I just haven't managed to grasp yet?  Well...not that I have managed to find at the time of writing.

One of the things I have been doing recently is deciding whether or not to port our current Magento 1 extensions to work under Magento 2.  I say port, but in reality it's basically a total rewrite from scratch for every extension (which isn't a thrilling proposition, but that's another story).  I have been really enjoying working with the Magento 2 codebase and I think it genuinely is a long way ahead of Magento 1.  So rewriting extensions under Magento 2 is largely an enjoyable, if not substantial task.

So why not write unit tests at the same time for these extensions?  And this is how I was prompted to revisit unit testing.  So after a quick refresher on assertions, mock objects, mock and stub methods etc I came to writing some tests for one of our modules, which quickly moved to looking at existing unit tests in core modules.  I came across the following test for the catalog module in test class Magento\Catalog\Test\Unit\Block\NavigationTest:
public function testGetIdentities()
        [\Magento\Catalog\Model\Category::CACHE_TAG, \Magento\Store\Model\Group::CACHE_TAG],
To me this is a good test to demonstrate the title of the post, what's the point of unit testing?

So as you can probably see, this is a simple test that checks that an array containing 2 class constants is the same as the return value of the getIdentities() method called on an object. Looking at $this->block we see that it is an instance of class Magento\Catalog\Block\Navigation instantiated by the object manager in the setUp() method (so not a mock). Looking at the class we can see that the getIdentities() method does the following:
public function getIdentities()
    return [\Magento\Catalog\Model\Category::CACHE_TAG, \Magento\Store\Model\Group::CACHE_TAG];
Yes, the method returns exactly the class constants tested for in the assert. Now unless I am missing something pretty fundamental, what is the point in this test exactly? It is nothing more than testing if a value equals itself, which of course it does. This is a test which can never fail as whatever either class constant may be changed to in the future it will still equal itself. To my mind this test just has no value at all, and due to the limited scope unit tests are designed for, summarises the overall usefulness of any unit test. I could understand if the class constant needed to be a certain value in order to be used to say pull some data from a database (which would cause the test to fail in the case of a change), but allowing reliance on external dependencies like this is not what unit tests are for. Any external dependency like this would be mocked with any methods which return data collected from some external source like this stubbed with the return value hard coded. Again it wouldn't matter what the class constants were changed to, the test would still pass because of the hard coded return value in the stub method. To me it seems that with the removal of dependencies all unit tests boil down to essentially the above and actually achieve nothing of worth.

So if somebody can tell me that the above test is just a poor example and enlighten me on the real point to unit testing I have been missing all this time, that would be great. At the moment the only point I can see to unit testing is to catch badly written code where the developer doesn't really understand the classes they are working/integrating with - and no I don't think that's a good enough reason for unit testing being as prevalent as it is.

I do understand that unit testing does have side benefits such as essentially acting as a type of documentation to the methods being tested, and giving developers who might be looking at code for the first time confidence in making changes by allowing them to run tests after making those changes. But that still doesn't get away from the fact that the developer still has to come to the point where they do understand the code they are working with, and in my company at least if you don't understand the code you are working with completely yet, the code you write doesn't get released until you do. Otherwise you are just intruding on cut and paste territory which is somewhere no worthwhile developer should ever be found.

Another side benefit, and the one that is perhaps the greatest is the way unit testing forces you to write your code. If you want to unit test you need to write code such that you can unit test, and this generally results in each method being as short and concise as possible, achieving only one goal. Dependencies should be passed to the methods that need them or be stored to the object rather than instantiated in the method itself, but Magento 2 pretty much has that covered with the object managers automatic dependency injection functionality for class constructors. Also logic which queries or uses any external dependency or service should happen in it's own method so when the object is mocked, that method can be stubbed to remove that dependency.

These side benefits are all great, and if wanting to write unit tests in a round about way forces you to follow these kind of coding conventions and principles then that can only be a good thing. But again I would hope that any developer with reasonable knowledge and experience would automatically be writing short and concise methods which are named sensibly, and perform only one task anyway. Regardless of coding this way being advantageous when it comes to writing unit tests, common sense should tell you this is a sensible way of doing things anyway as it makes your code so much more maintainable with it being very much quicker and easier to understand.

So please, use the comments below to give me your opinion and convince me I should be writing unit tests along side testing my code with xdebug!