Tuesday, December 11, 2012

Making Yootheme Widgetkit load faster

While optimizing a customer's web site today I noticed that resources related to the (otherwise excellent) Yootheme Widgetkit were not cached by Varnish or by clients' browsers. This had a measurable impact on page loading time - it was about 5 scripts fetched on every page load, over and over again.

This happened because on every page load, a different request has been made. For example the media player URL was /media/widgetkit/widgets/mediaplayer/mediaelement/mediaelement-and-player.js?_=1355230620557, then /media/widgetkit/widgets/mediaplayer/mediaelement/mediaelement-and-player.js?_=1355230620586 and so on. Clearly something has been appending a time stamp as a query parameter.

After some investigation, I've found that jQuery was guilty. Its default behavior is to append the time stamp to prevent caching. Fortunately, this behavior is configurable. After adding the following line of code to the beginning of media/widgetkit/js/jquery.plugins.js, the query parameter was gone and the 5 scripts were cached once and for all:

jQuery.ajaxSetup({cache: true});

Yootheme support has been notified, hopefully they will include it or make it configurable in the next release.

Tuesday, October 23, 2012

Varnish integration for Joomla! 3.0

A version of our Varnish-Joomla integration for Joomla! 3.0 is ready. It doesn't bring any significant updates to older Joomla! versions. The upgrade is free for customers who ordered an older version of this patch.

Varnish is a website accelerator which can help you handle hundreds of requests per second on your Joomla! site.

Saturday, October 13, 2012

Introducing TurnKey templates

Turn Key Linux is an open source project providing tens of virtual server templates. We're proud to introduce the ability to order VPS servers with these templates pre-installed directly from our web site: TurnKey based VPS servers.

Tuesday, August 28, 2012

Cron log in Debian 6

By default, Debian only sends results of cron jobs by email. In some cases, however (no direct internet connection), you want them logged in a regular log file in /var/log. In Debian Squeeze it's not the default but it's very easy to change it. Just edit /etc/rsyslog.conf and uncomment the following line:

cron.*                          /var/log/cron.log

Then, restart rsyslogd by executing service rsyslog restart. Viola!

Tuesday, August 21, 2012

Amazon Glacier

Amazon has released a new tool: Glacier. Similarly to S3, it is storage, but Glacier is a different beast. It's for keeping long term backups which you will hardly ever need, but which you need to store for one reason or another. The storage fee is just 1 US cent per gigabyte per month. The catch is, however, getting back your data. Retrieval of storage is free if you're only grabbing 5 percent of your data per month.

Thursday, August 16, 2012

vztop in vzprocps: fixed

There has been a frequently encountered problem with vztop on OpenVZ making the tool totally useless - it worked perfectly except for it didn't show virtual machine IDs for processes. Instead of actual numerical ID, it showed "N/A" in the VEID column.

So I spent an hour or so debugging, and it turned out that vztop, when requested to show the VEID column, didn't set a flag which enabled collection of VEIDs and other data. While diving into vzprocps source, I also made vztop show VEID by default. The result is this simple patch against vzprocps 2.0.11-2.

Apply with patch -p1 <vztop_veid.txt

If you need to debug a problem with some piece of software, I'm for hire at Massive Scale. I have never seen vzprocps source code before and still the solution was available in just 1 hour of work. That's $70. Would your $15/h programmer solve the problem as efficiently?

Thursday, May 10, 2012

Varnish-Joomla patch upgraded to 1.2.1

Varnish integration for Joomla! CMS is our solution for websites based on Joomla which have many page visits by anonymous users. It allows owners to serve even hundreds times more traffic without investing in hardware. Should I mention that it makes Joomla! fast, too?

One of our customers, Free Speaker Plans, was experiencing some issues with the patch he purchased from us. After a quick debugging session, it looked like forum software was not compatible with the patch. So I've modified the patch and now the forum component may run unmodified. If you've purchased an older version and have trouble running Kunena Forum, upgrade now!

At this point, I'd like to thank the webmaster of Free Speaker Plans for patiently testing my subsequent attempts at fixing the problem and providing rich feedback. Thank you!

Technical list of changes:

  • Support for Joomla < 2.5.4 has been abandoned

  • Fixed a bug introduced in 1.1, causing that pages calling Jerror::raise() raised a Fatal Error

  • JFactory is now creating a guest JUser object if anonymous session detected (fixes Kunena and possibly other social software)

  • There is no upgrade patch from 1.2 because of numerous Joomla core changes

Monday, May 7, 2012

New server

There was a disk failure on a server serving our customers' virtual private servers. Its disks were not hot swappable, so in an effort to prevent downtime, we're moving to a new server instead of repairing the current one. This was a good opportunity to upgrade the hardware, too. The gigabit network connection is pretty impressive, it downloaded a Linode test file in 1.5 seconds.

Thursday, February 2, 2012

Personal server - how to save on memory?

This blog is usually about scaling systems to hundreds of thousands of users, but let's make an exception this time. I've been setting up a virtual server for my private use last few days, and I'd like to share my experience on saving the most precious resource when it comes to VPSes - RAM. There are many different virtual server providers with many different pricing options, but one thing they have in common is that the pricing is mainly dependent on RAM.

Let's make a quick list of demands for a private server. We'd like it to be as cheap as possible. Home budget is important, after all. We won't serve much traffic or do heavy computation, we also don't need to care about speed, with one user at a time the system will be fast enough regardless of the setup. In my case, I wanted to run OwnCloud 3 for syncing files, calendars and contacts with my phone. I'll also need an FTP server for easy sharing photos from my smartphone (using FTP is a bad idea in general, but I can't find anything like URLy with SFTP support).

A traditional approach used in large systems would be to install a full-featured web server with PHP running via FPM, regular FastCGI or mod_php, and an FTP daemon like proftpd or vsftpd. The processes would run in the background and wait for connections. This is very good for performance, but takes a lot of RAM.

So I took the old-school path. I'm running PHP code via CGI, and FTP daemon is running behind inetd. Let's expand this topic.

inetd is called an internet super-server. It's a tiny daemon which listens for connections on different ports and, when it accepts a connection, starts a program to handle this connection. So, in our case, it listens on port 21 (FTP), starts a FTP daemon when a connection comes in, and kills it immediately afterwards. inetd itself uses less than 2MB of RAM, less than any FTP program I'm fond of, so the extra memory is used only when my phone is actually uploading something.

CGI is a similar idea for processing scripts in a web server. When lighttpd needs to process some PHP, it sets some environment variables reflecting script name, HTTP headers and so on, and starts a php-cgi process. The interpreter processes script and exits. The RAM is free again. Of course, starting a PHP interpreter every time is slow and takes resources, but on an almost idle system it takes fraction of a second. It may not be acceptable on a massive scale, but is good enough for personal use.

What do we gain by using inetd and cgi? It's highly unlikely that I upload a file and use ownCloud from my computer at the same time, so I can estimate that the maximum memory usage is max(sizeof(php), sizeof(ftpd)) as opposed to sizeof(php) + sizeof(ftpd) in the case when they're running all the time. Even if this assumption isn't true and it happens that I'm using FTP while browsing ownCloud, nothing bad will happen - I will see an error 500 page, hit Refresh and live on.

ownCloud also needs a database. A standard database for PHP applications is MySQL. It runs as a regular server, so it takes memory all the time too. Fortunately, ownCloud also supports SQLite databases. SQLite is nowhere as robust or scalable as MySQL, in fact it doesn't even allow concurrent access to database. But it's enough for a single user, and even good enough for several users. SQLite needs no daemon running in the background enforcing ACID and taking up RAM - it just works as a PHP extension, a library which is loaded into PHP interpreter when it starts up.

Let's talk about implementation. Running PHP in CGI mode is easy with lighttpd, you just need to execute lighty-enable-mod cgi, edit /etc/lighttpd/conf-enabled/10-cgi.conf and uncomment the cgi.assign section in the bottom. You may also need to apt-get install php5-cgi.

Setting up inetd and ftpd is even easier. apt-get install openbsd-inetd ftpd leaves you with a running FTP server for all your users defined in /etc/passwd. You will also want to edit /etc/ftpchroot and /etc/ftpusers files. man ftpchroot and man ftpusers should explain everything.

As you see, using the right tool for the right job is beneficial - in enterprise I wouldn't dare using CGI or inetd as they would quickly make the server unresponsive, but for personal use it's good to step back and use an old school solution.

If you need a server set up for your personal needs, order one or talk to us about your needs. We're passionate about computing in general, not just at a massive scale.

Tuesday, January 31, 2012

Varnish/Joomla patch 1.2 in testing

We're testing the new version of Joomla support for Varnish HTTP accelerator. Version 1.2 is available for Joomla! 1.5 and 2.5. Take a look at the changelog.


Don't cache a page with error box - when JError::raise() is called or JDocumentRendererMessage is used, a header which forbids caching is being sent


Prevent displaying cached pages directly after logging in.

After successful login the user is usually redirected to referring page. In many cases the page is cached by browser, and contains a login form, thus causing confusion to the user. The problem is solved by adding a random GET variable called _rnd.

Saturday, January 14, 2012

How to disable annoying ipython exit prompt

ipython is an interactive Python shell which is generally great with its history and autocompletion, but has one annoying problem. Whenever you hit Ctrl-D to exit from ipython to bash, it asks you this question:

Do you really want to exit ([y]/n)?

Perhaps this prompt would be useful if I were using it to control an atomic power plant in emergency mode, but for my humble needs it's just annoying. It's easy to disable it - create a file called ~/.ipython/profile_default/ipython_config.py with the following content.

c = get_config()
c.InteractiveShell.confirm_exit = False

If you need a hand with Python, see what we can do for you.

Thursday, January 12, 2012

Varnish/Joomla patch update

Our patch for making Joomla work with Varnish Cache has been upgraded today. It contained a rarely occuring bug causing pages with error boxes to be cached under certain conditions.

Because of this, regular visitors may see errors like the on below, even though they didn't try to log in:

The upgrade is free to all our customers who have purchased the patch.