Varnish and Joomla - how does it work?

Our most popular product, Varnish-Joomla integration, makes your website fly fast. But how does it achieve its goal?

Most visitors to virtually any website are anonymous visitors. These people don't log in, don't interact, they just view your content. In an e-commerce store they may be visitors who didn't buy anything. On a forum website they may be people who have found some thread on Google, read it and forget about it. On a newspaper site, readers who don't leave comments and don't have an account are considered anonymous visitors.

These visitors generate the most traffic. But they are also the easiest to handle, because all they want to do is read your content - so why serve their pages using full Joomla - PHP - MySQL - Apache stack, if they don't really utilize any of these in full?

We took advantage of that fact and we serve content to such users before they reach Joomla. We put a piece of software in front of Joomla which serves content to anonymous visitors from a very fast cache. If it encounters a visitor who's logged in, it forwards his request to Joomla so he gets a full experience.

If you don't believe - check the numbers. Varnish makes Joomla fast.

HTTP 417 errors in lighttpd

One of the most annoying parts of Lighttpd is that it sometimes fails with error 417 when you upload a form or use an API with libcurl. The proper, HTTP/1.1 compiliant solution will be available in version 1.5, which is in development for years now, but since 1.4.21 they have implemented a solution to behave like a HTTP/1.0 server, which is just what 99.9% of users need.

To fix 417 errors in lighttpd, simply add this to your lighttpd.conf:
server.reject-expect-100-with-417 = "disable"
and restart your lighttpd server. Viola!

Christmas in e-commerce

Christmas can be hard on servers

If you're running an e-commerce website, you already know the pattern: during Christmas and Valentine's Day your revenue increases ten-fold, but so does the server load. Pages load slowly, customers are annoyed and so is your staff. Can this be avoided?

Handling more users doesn't necessarily mean getting more expensive hardware and moving from one server to another. More often than not, you can increase the efficiency of your existing infrastructure. Sometimes you could even lower the total cost of operation just by optimizing resource usage. This is equally true for servers and for other business assets.

We work with websites handling lots of traffic all the time. We know how to optimize them for speed, performance and scalability. We know how to test how much they can handle. If that's needed, we also know how to design a larger infrastructure tailored to your needs.

Google research shows that page load time increased by half a second causes a 20% drop in traffic. For e-commerce, more traffic equals more sales. Do we really need to elaborate?

Read more to learn what we can do to help you.

Joomla website made fast - case study

I've just finished optimizing the infrastructure behind WomensForum.com. They have used our Joomla-Varnish package and a few hours of my website optimization consulting time. The amount of money they spent on optimization is comparable to monthly hosting fee for a big website.

While the website was usually fast before optimization, the server couldn't handle traffic spikes. At one point, there were more than 1500 requests per second - that's a lot for Joomla, but not for Varnish. Page load time was consistently low even during traffic spikes. Additionally, we have used the failover capability of Varnish to be less reliable on backend server failures. Speed up your website too.

Virtual desktop

What would you do with a virtual Ubuntu desktop? We have a few ideas. You could view facebook while you're at work, no one will know. You could work on a document with other people. You could download torrents or rapidshare/megaupload files while your computer is off. Your imagination is the limit, but check out our remote VNC desktop page to see a few ideas.

Gitlab, the private Github clone

We programmers know and love Github, the source code hosting website that's free for open source projects. But we often like to keep our projects private before releasing them publicly, and that's one thing Github is not good at. Sure, you can get the personal plan to have 5 private repositories, but is that really enough for all your personal mini-projects?

Meet Gitlab, the open source Github clone that you can host on your server. The number of repositories and collaborators is limited just by your server's capabilities. But you need to set up the server, and it takes a few hours unless you're proficient with Ruby.

We're proud to introduce the Git Server Appliance, featuring gitlab as web GUI and gitosis as accounts manager. All this starting at the price of Github personal plan!

There's no set up, you're provided with administrator login and password to your newly provisioned git server appliance. All you need to do is log in and start working.

Lamson, the Python SMTP server

Do the words "sendmail", "postfix" and "qmail" sound scary to you? You're not alone, and some developers were so fed up with 1970-s style software traditionally used to handle email routing, that they have written an email server in Python.

The front page of Lamson features some simple examples of how routing rules may be created - it's so logical, it's hard to believe it's handling email.

Don't know where to start? We're here to support developers.

Spanish website

We are proud to announce MassiveScale website in Spanish. We're also introducing live chat with sales in both English and Spanish.

Secure Joomla 1.0 hosting

At MassiveScale, we are starting to offer VPS hosting. While we don't have any generally available offer yet, we have started marketing a niche service. Joomla 1.0 websites hosting.

"It's as if you offered Windows 3.11 support", a friend of mine said. Well, yes and no. The newest version of Joomla CMS is 1.7, and 1.0 has been unsupported since 2008. However, there are many websites made on 1.0 that will not upgrade to 1.7 simply because it's too expensive. If someone ordered a page made with Joomla and a slightly modified off-the-shelf template, it's not because he has a large budget. Now, try explaining that person that he needs to spend the money twice or there will be countless break-ins into his page.

That's why we decided to offer a service to help these people out. Most of these websites are now in maintenance mode, only content is being added/edited and no major redesigns are taking place. We just lock all the files and, optionally, parts of the database, and that's the secret sauce. It makes Joomla 1.0 hosting pretty secure.

Varnish and FLV streaming

When doing usual website speed optimization to a website that has lots of videos, I have encountered a problem. Videos started breaking for no apparent reason, they loaded very slowly and in general were unusable.

It turned out it was because of two things. One was that Apache has been set up to compress everything except images with mod_deflate - this does not play well with streaming. Another problem was that Varnish running in front of the web server wasn't set up to do streaming, so it first fetched whole video to its cache and then sent it to the browser. It was slow.

The solution was to upgrade Varnish to 3.0.1 and add the following code to vcl_fetch:

if(beresp.http.Content-Type && beresp.http.Content-Type ~ "video") {
        set beresp.do_stream = true;
}

Optimizing website performance is not an easy job and has tons of catches like this.Order optimization now!

Celery task queue with PHP

We have released a Celery client for PHP some time ago. What does it do and why is it useful?

Celery is a piece of software that helps you easily run time-consuming tasks. If you have ever used system() or exec() to create a big ZIP file or encode a video, you probably already know what the problem is. If you didn't - well, time-consuming tasks executed from within a PHP script (or any other web application) usually generate many problems, unresponsive user interface and timeouts being two major ones. There are many ways to overcome these issues, but face it - spending many hours and hundreds of lines of code just to create a ZIP file isn't especially effective.

So how does a basic Celery application to create a ZIP file look like? Let's see:

#!/usr/bin/python
from celery.task import task
from subprocess import Popen, PIPE, STDOUT

@task
def create_zip(zip_path, files_path):
    command = ("zip", zip_path, files_path)
    return Popen(command, stdout=PIPE, stderr=STDOUT).communicate()[0]


The above code calls the command-line zip command with two arguments: path to a zip file and path of the files to be archived. "def" is Python for "function". (this is just an example, in real world you will probably want to validate paths and replace zip command with Python's zipfile module)

Now you will probably want to call that code from your PHP application. First schedule the task for execution:

$c = new Celery('localhost', 'myuser', 'mypass', 'myvhost');
$_SESSION['zip_result'] = $c->PostTask('tasks.create_zip', array('/home/user/file.zip', '/home/user/images/'));

Then you will want to somehow asynchronously display the result to user. You can use AJAX or some other technique to display the result of this script every second:

$result = $_SESSION['zip_result'];
if(!$result->isReady())
{
    echo 'Please wait...';
}
else
{
    echo '<a href="/file.zip">Ready!</a>';
}


That's it, as easy as this. Check out the Celery-PHP documentation and try it out.

Also, the examples above are oversimplified - make sure you validate your input data, handle errors and don't unnecessarily poll the server.

RabbitMQ 2.6.0

It looks like new RabbitMQ is just behind the corner.