Speeding up Magento

If you are a Magento store owner there's a good chance that one of the things that is pretty high on your todo list is reducing the time it takes your stores pages to load.  If you are a developer you have no doubt been asked to impart your knowledge on how this is best achieved on more than one occasion.

A quick search will soon reveal a multitude of pages and discussions talking about how to improve the speed of your store, with loading times often featuring heavily in any arguments against using Magento.  It's true that Magento is complex software, and is also a heavy application for a web server to run.  Until you are experienced it can also seem mind bogglingly complex from a development point of view.

However over the years of developing Magento I have come to find the framework to be the best developed, most capable ecommerce platform in the market place hands down.  Once you have mastered it it's also extremely intuitive and amazingly versatile to develop - it quite comfortably stands up in it's own right as a fully fledged MVC framework.

So even though the versatility you will find in Magento brings a down side in the form of potentially slow page loading times, rest assured that with so much scope for development it's certainly not a problem which can't be addressed with very effective results.

You have probably come across this post after doing some research into exactly how you might achieve faster page loading times, and that probably also means you have come across the near endless multitude of guides and suggestions on the topic.  You will find some good pointers, and tips like minifying JS/CSS, enabling Gzip compression and installing an opcode cache such as APC or XCache are all worthwhile, but in my experience pretty much all of the solutions in these guides don't actually make a great deal of noticeable difference to page loading times.

The reality is that for every request Magento processes, thousands of lines of code need to be executed.  This takes both time and processing power and until each request completes and the resulting HTML response has been sent, the user browsing the site won't see any page content.  While tweaking the server setup and Magento configuration might improve overall performance by probably no more than 5%, the process of Magento building each page accounts for at least 95% of the total work the server has to do to deliver the entire page and all assets (images, JS, CSS) to the browser.

Many of the guides and suggestions out there don't even make, let alone address this fundamental point, and rather cloud the issue with endless server and Magento tweaks which would take a huge number of hours to complete if you were to follow them all, but result in just a very modest improvement in performance.

If however you want to see a real improvement in page loading times, then there are really only three areas you need to concentrate your efforts.  Be warned though that all three of these areas are likely to require investment, so if you are only interested in free solutions, this probably isn't the post for you.  If however you have some money to put into it then read on...

Server Hardware

The first area you need to look at it is the hardware your store is running on, so your web hosting package.  This is of course the foundation of your entire store setup and anything that you build on this foundation can only ever be as good as the hosting will allow.  For instance if you are trying to run a Magento store on a cheap shared hosting package, then you can tweak the store, and spend as much time and money on optimising it as you like, but you will never achieve good performance because the raw processing power is just not there in the first place.

So review your hosting and make sure it is up to standard.  Which hosting package you choose directly relates to how much traffic you expect to see on the store as the higher the traffic, the more capable the server will have to be as it will be fulfilling more simultaneous requests.  Always steer clear of shared hosting when it comes to Magento, and instead go for a suitable VPS or dedicated hosting package.  As for which specific package to go for, well that entirely depends on your store setup, and your sites traffic.  If you are running a smaller, low traffic store, then you can probably get away with a mid to low range VPS package, if however you are running a high traffic store with lots of products, you will almost definitely want a dedicated server package.  As your site grows you will discover that the possibilities for Magento hosting continue to scale up to complex, multi server, load balanced setups - but if your store is at that level, you probably don't need to read this post.

To determine the best hosting package for your store I would recommend contacting someone with good knowledge on the subject.  There are many companies who offer Magento specific hosting and should be happy to advise, or a good Magento developer will also be able to give you some pointers on a suitable package to go for.

If you want to get in touch with us for some advice on which hosting package is best suited for your store, then just drop us an email to support@husseycoding.co.uk with the following information:

- Store address
- Summary of typical monthly traffic
- Geographical location of main target audience

We will be happy to have a look at your site and give you some free advice on hosting packages which suit your setup.

Time for first request to complete

When viewing a web page the first request that is sent to the web server is a request for the complete page HTML, and once this has been completed everything else relating to the page can be requested, so the images, CSS, JS and so on.

In the case of Magento, fulfilling this first request means initialising the Magento application, routing the request to the relevant controller in the framework, generating the layout for the page, and then finally rendering the page out to HTML from the layout.  This resulting HTML is then returned in the server response to the browser, at which point the browser can display the page HTML and begin sending requests for images, JS and CSS files.

Each step in generating the pages HTML takes time, and as a whole quite a lot of time.  When I talk about rendering out a page accounting for at least 95% of the work the server has to do for a page and all assets to be delivered to the browser, it's this complete process I'm talking about.  The rest of the requests which come after this are just for files so all the server has to do is deliver each file - no processing of the request is required.

So it makes sense that this first request for the page HTML is the one that could do with improving, and that's exactly what all of the Magento full page cache extensions, and also Varnish aim to do.  I'm sure you are aware there are quite a few Magento full page cache solutions out there, with varying levels of performance.

As already mentioned, the process of rendering out a Magento page is time consuming, and as soon you need to enter the Magento framework to fulfil a request you immediately significantly increase the time it takes for that request to complete.  This is the main difference between many of the full page cache solutions available, and our own solution, Evolved Caching.  Although all of the full page caching solutions available do actually cache the full page HTML, the vast majority of the solutions also need to enter the Magento framework in order to fulfil the cached page request, and this in turn means that the speed at which this request can be fulfilled becomes entirely dependant on the speed of your Magento install (which I'm sure you will agree to a large extent defeats the object of the exercise).  Evolved Caching operates much the same as Varnish and fulfils this initial request for the page HTML entirely outside of the Magento framework giving vastly improved performance when compared to other solutions.

In order to add any dynamic user specific content to the site, Varnish has ESI, which requires tags to be put in relevant places in the site code.  The major down side of using ESI with Magento is that there will often be multiple parts of the page which need to have dynamic content added, and for each of these, a separate request must be sent in to Magento.  This of course kicks off the slow process of initialising the framework and rendering out an HTML response.  In addition to this, Varnish will not deliver the page to the browser until all the ESI content has been collected, and although ESI responses can also be cached, the content Magento produces is just too dynamic to make this a realistic possibility.  So by using Varnish and ESI you might actually end up sending 2 or 3, or more requests into Magento before the page is delivered to the browser, thus resulting in page load times which are actually slower than without Varnish!

Evolved Caching on the other hand delivers the page HTML immediately after it is pulled from cache, and then fulfils the requirement for dynamic content on the page via a single AJAX request while other requests for things like images, JS and CSS are still running.  Although this request does have to enter the Magento framework, the request is quite a bit 'lighter' than a standard request, and is also only made after content has already been delivered to the browser thus eliminating the waiting time of other full page cache solutions, and also Varnish + ESI.

Evolved Caching was the first Magento full page cache extension to introduce this innovation and has a large amount of equally valuable functionality on top of this.  It also integrates directly with Varnish by way of it's caching key cookie and this greatly increases the ability of Varnish to cache and fulfil requests for your store.

So in summary reducing the time it takes for the server to fulfil this first request for the full page HTML is exactly what full page cache extensions, and also Varnish do - but the key to this being really successful is to not enter the Magento framework before content is delivered as this will immediately have a major impact on the performance of the cache.

If you are interested in Evolved Caching, then you can find a summary of functionality on Magento Connect, read the full guide here, and purchase the extension from our store here.

Requests per page

The final area which you need to look at is the least easily addressed, particularly if your store is already developed and themed.  That's certainly not to say that it can't still be looked at, it's just more easily done when the site is initially being built.

When viewing a page, multiple requests are sent to the server before that page fully loads, in fact every separate item on the page is requested individually from the server, so every image, JS file, CSS file and of course the full page HTML as we have learned above, are all received as a response to an individual request for that content.

If you look at a typical web page using the net panel of Firebug, or if you don't have that, a site such as GTmetrix, then you are able to see just how many requests have to be fulfilled in order for that page to fully load (if you are looking at GTmetrix, look at the waterfall diagram).  Each item in the waterfall which describes the loading of the page represents an individual request which has been made to the server for content.

This is significant because a browser can only fulfil a certain number of requests in parallel.  At the time of writing, in modern browsers this is along the lines of 6-8 - so 6-8 of the requests shown in the waterfall diagram can be completed simultaneously, but no more.  When the maximum possible number of requests for your browser are all in progress, further requests must be queued and become idle until an existing request completes, therefore making room for a queued request to start.  When looking at the waterfall diagram, the left hand most block of colour for each request represents the time that request is queueing and idle, and you will probably see that all requests after a certain point are queued.

This queueing is time that is entirely wasted, and no queueing would result in a significant reduction in the time taken for pages to fully load.  So one way to alleviate this is to reduce the number of requests per page.  Merging of CSS and JS is a good, easy way of going some way towards doing this, and for a Magento store will often mean reducing perhaps 25+ inidividual requests down to just 2, one for merged JS, and the other for merged CSS.

There is some discussion surrounding whether or not to use JS/CSS merging within Magento as the JS/CSS files which are required on each page tends to differ a little between different areas of the site, and this results in different merged JS/CSS being created wherever this is the case.  The end result is that your browser ends up needing to download a new JS/CSS merged file for each of these different areas but the actual file content contains very little that differs from what was perhaps already downloaded for another area of the site.  Without merging only the new JS/CSS files that differs for each area would need to be downloaded rather than the whole lot again (in merged form).  The argument against merging however ignores the fact that without merging the number of requests for a page will likely increase by at least 20, and as long as you have sensible file sizes for your images, request queueing has a much greater impact on page loading times than the amount of data which must be downloaded for each page (merged CSS/JS combined should only account for perhaps 30k of data, around the same as 1 small image).  In case you have come across any discussions like this, my recommendation is to definitely enable JS and CSS merging.

So that does a pretty good job of reducing the number of requests for JS and CSS content, however it's likely that there are still quite a few more requests that can potentially be reduced down, particularly for images.  Typically Magento themes use quite a high number of images and while things like product and category images can't realistically be worked on to this end, it's well worth reviewing what images are to be found in common areas of your store such as the header, footer and sidebars.  There is very likely scope here for reducing the number of images required by creating suitable sprites.  A sprite is simply a single image which contains several images within it.  This means you can effectively download a number of images in a single request, and then use CSS to position the sprite within an element to show just the image you require.  That way you can have a number of different images in different places on the site, but only require one server request to obtain them all.  A bit of common sense is of course required in balancing file size vs number of requests when using sprites.

As well as thinking about sprites also consider using more CCS3 styles.  All modern browsers now support CSS3, and support in older browser is getting better all the time as new browsers continue to be released and older ones fall away.  As time goes on of course, older browsers that don't support CSS3 will disappear entirely.  Using a few simple CSS3 styles can really help you cut down the number of images you need to use, and styles like border-radius address design problems which historically have created the need for quite a large number of individual images (a box with rounded corners for instance).  Using CSS3 these kinds of things can be achieved using no images at all, meaning the only request sent to the server is for the CSS file containing the styles.

Working on the images and considering the CSS3 styles to use is the reason why I say this is easier done at the initial design and build phase where the possibility of using sprites and styles can be taken into account - this is of course significantly easier than reworking an existing theme.

Going back to talking about the browser limit of 6-8 simultaneous requests at a time, it's worth noting that this limit is applicable on a domain basis, so if you had a page from domain1.com and domain2.com both loading at the same time in your browser, both pages could send 6-8 simultaneous requests, making a total of 12-16 simultaneous requests sent by the browser.  This is exactly the functionality which a CDN leverages.  As CDN's reside under a different domain to the main domain you see in the address bar, it allows a browser to send an additional 6-8 simultaneous requests for every CDN domain used.  The result of course is a reduction in requests queueing, and therefore a decrease in the time it takes for a pages content to load entirely.

Getting a CDN setup for your store isn't an entirely straight forward process, and something you should probably approach your hosting provider about if you don't have the skills in house, however if you have done all you can to reduce the number of images served by creating sprites and using CSS3 styles, then it's a good option to look at.



I hope that this 3 point guide has proved informative, and if you want to get in touch with us regarding the performance of your store, just drop us an email to support@husseycoding.co.uk with your site address and we will be happy to evaluate your stores performance for free and come back to you with some pointers on further steps you could take.  You can also assess your store using our profiler for some straight forward, clear, and specific advice on improving the performance of your store.