Thursday, December 12, 2013

Joomla 1.0 spam

This news comes from our secure Joomla 1.0 hosting department. Recently some of our Joomla 1.0 sites became sources of spam. Apparently somebody (with a narrow range of IP addresses geolocating to Malaysia) has found a security problem in VirtueMart recommendation module which allows them to send emails with POST requests.
The solution was to paste the following code into Apache configuration (or .htaccess):
# vm spam patch
RewriteEngine On
RewriteCond %{THE_REQUEST} ^.*(page=shop\.recommend).* [NC]
RewriteRule ^.*$ - [F,L]


If you're fed up dealing with lack of security in old Joomla versions, give us a shot. We run Joomla on latest Apache, PHP and MySQL versions, keep a month worth of backups and have several security measures.

Tuesday, November 19, 2013

Varnish-Joomla Pack 1.3.1 with Joomla 3.2 support is out

New features:

  • Support for Joomla! 3.2
  • Support for HikaShop
Joomla! 3.2 support has been tested with core components and the following extensions:
  • JomSocial
  • Akeeba Backup
  • Admin Tools
  • Kunena Forum
  • Community Builder
  • HikaShop
Virtuemart does not support 3.2 (and never will), and that's the only reason why it's not included in the test. We continue to support it. As explained here, the patch for Joomla! 3.2 does not prevent creating new sessions for anonymous users. If you have a lot of cache misses caused by anonymous users, there will be an extra disk write operation per cache miss - if this concerns you, move your sessions to memcached or XCache.

Upgrade from 1.3.0 and 1.2.2: Just replace the default.vcl file. As always, the upgrade is free for existing users.

New users - order to speed up your Joomla! and its extensions.

Friday, November 15, 2013

Varnish for Joomla! 3.2 on its way

Varnish for Joomla 3.2 will be released next week. The plan was to release it this week, but Joomla 3.2 increased its dependence on sessions and the patch can not operate the same way it did before.
In previous versions, we did what we could to not even create sessions in the database, thus lowering the number of disk writes required to serve users' requests. Unfortunately it doesn't look like it's possible any more and the best we can do is not sending session cookies to the user.
This is a new approach and while it passes all internal tests against a vanilla Joomla install, it needs more testing with supported 3rd party components.
I'm sorry for the delay, we got many requests about 3.2 availability in the past two weeks.

Tuesday, November 5, 2013

Varnish-Joomla package 1.3.0 released, with cache purging

The new Varnish-Joomla package 1.3.0 is now available. New features include:

  • Purging cache when saving articles
  • Support for more Joomla! extensions and static file types
  • Improved cache hit rate because of Accept-Encoding normalization
  • Support for Piwik and CloudFlare
  • More debugging headers
The biggest news is cache purging - this means that the moment you save your article, it is visible to your visitors. Behind the curtains, our Joomla plugin connects to your Varnish and tells it to remove cached copies of the saved page, pages of categories it belongs to, and the front page. This feature is optional, you don't need to install the plugin if this is not the desired behavior.
The package has support for Joomla 2.5.4+, 3.0.x and 3.1.x, just like the previous version.
There is good news for our existing customers: the patch to Joomla core didn't change, all features are implemented with the Varnish VCL file and an easily installable Joomla plugin.
As usually, the upgrade is free, contact us to receive the latest package if you've already purchased it.

Monday, October 28, 2013

Make Joomla's Nextend Menu faster

If you're using Nextend's Accordion Menu for Joomla, there's a good chance it's slowing you down. Adding this simple patch to make it use cache cut down average loading time by 900 ms on a Joomla 2.5 site.


  • Open libraries/nextend/accordionmenu/treebase.php
  • Find the following part:

        $this->renderItem();
    }
    
    function renderItem() {

  • Replace it with:
        $cache = & JFactory::getCache();
        $cache->call(array($this, 'renderItem'), JURI::getInstance()->toString());
    }
    
    function renderItem($dummy=NULL) {

Thursday, October 24, 2013

Alternatives to mod_rpaf

For years, mod_rpaf has been the standard module for running Apache behind a reverse proxy. Fortunately, it's not the only one and there are alternatives.

  • mod_extract_forwarded which is available in EPEL, so it's available for CentOS systems without compling
  • mod_remoteip which is built into Apache 2.4 (but good luck finding a production server running 2.4)
  • mod_rpaf modified to support X-Forwarded-Proto/X-Forwarded-HTTPS/X-HTTPS

Tuesday, October 22, 2013

New software available: HTML5 prerender support for Joomla! CMS

Massive Scale is proud to announce its new software: HTML5 Prefetch/Prerender for Joomla!. It makes use of prerendering capabilities of web browsers like Chrome, Firefox or Internet Explorer 11 using machine learning techniques. You will learn more about prefetching, prerendering and Joomla! on the product page.

Monday, October 7, 2013

Thread creation error in Varnish

Today I installed Varnish on a pretty standard system (CentOS + WHM) and the service wouldn't start. The system log (/var/log/messages) contained the following:

Oct  7 04:15:46 server varnishd[20836]: Child (20846) Panic message: Assert error in WRK_BgThread(), cache_pool.c line 582:#012  Condition((pthread_create(thr, ((void *)0), wrk_bgthread, bt)) == 0) not true.#012errno = 11 (Resource temporarily unavailable)#012thread = (cache-main)#012ident = Linux,2.6.32-358.2.1.el6.x86_64,x86_64,-sfile,-smalloc,-hcritbit,no_waiter#012

The settings for number of processes and file descriptors in /etc/security/limits.{conf,d} were correct and it still didn't start. The problem turned out to be with stack size - I have no idea why this is happening in just this one system and works with every other centos, but the solution was as following:

1. Patch /etc/init.d/varnish adding a new ulimit call in start()

        # Stack size
        ulimit -s unlimited

2. Increase system limits in /etc/security/limits.conf accordingly

varnish         stack   soft            unlimited
varnish         stack   hard            unlimited

3. Restart Varnish

After successfully running Varnish this way, be sure to change the stack limit back to a reasonable size! Unlimited is not a very wise setting, but helps debugging the problem. Now Varnish and Joomla could happily work together.

Wednesday, September 4, 2013

Nextend modules for Joomla behind Varnish Cache

Nextend makes pretty and well-written modules for Joomla and Wordpress. I've optimized a website which uses one of them and it had a weird issue: sometimes the module would appear unstyled and CSS/JS files in the "media/nextend/cache/" directory returned 404. After reading some code the solution was obvious: Varnish was set to cache pages for at least 2 hours, while the Nextend module only cached it for 15 minutes, and deleted old files. Because every time it creates a new directory for updated cache content, it deletes old ones - cached HTML pages were referencing URLs which did not exist any more.

The solution was very simple: in Joomla admin go to Extensions - Plugins - Nextend Library, then click Configure in the Basic Options and set the cache time to the same value your Varnish installation uses. For the best Varnish + Joomla experience, see our offer.

Sunday, June 23, 2013

Zend opcode cacher in PHP 5.5: a security perspective

The new zend opcache extension built into the latest version of PHP is, as we've checked, a bit faster than the current best - XCache. There's, however, a security concern.

Both extensions make it possible to clear the whole cache programatically (from within a script). In Zend it's the opcache_reset() function and in XCache it's xcache_clear_cache(XC_TYPE_PHP).

If you run shared hosting, you allow your users to execute arbitrary code in your PHP interpreter. This, naturally, includes the two functions mentioned above. If you rely on the opcode cache to provide desired level of service for your customers, you don't want them to clear everybody's caches. If a malicious or badly written script clears your cache, say, every minute - there is no benefit from caching. The whole idea is that you cache the most used files once, and use the cached version from RAM. What this means for a shared host is increased CPU and possibly disk IO usage. In some cases, this can turn into a Denial of Service (DoS) attack.

So let's take this simple script and see what Zend Opcache will do when you execute it:

<?php

error_reporting(E_ALL);
ini_set('display_errors', 1);

opcache_reset();

echo 'OK';

Well, Zend happily cleared the cache. Now let's see what XCache does:

<?php

error_reporting(E_ALL);
ini_set('display_errors', 1);

xcache_clear_cache(XC_TYPE_PHP);

echo 'OK';

The XCache developer envisioned this problem. Certain functions, which affect other users' experience, trigger a login page. The username and password is set by an administrator system-wide in php.ini and is not viewable by regular users.

Conclusion

If you run a shared host and want to speed it up using opcode caching, you should use XCache rather than the new Zend Opcache. Zend is only better by 10%, it's not a number worth risking a DoS attack. On the other hand, if you and only you control the code that's run on your servers, this security issue won't affect you.

Possible fix

You could (and should) use the disable_functions directive in php.ini

(Need help installing XCache or securing Zend?)

PHP 5.5: Zend Optimiser+ OPcache vs XCache

PHP 5.5 has a new feature: built-in opcode cache! It's an alternative for extensions like XCache, APC or eAccelerator. Let's see how it performs compared to established solutions.

Test environment

Server: OpenVZ container running on 1 Xeon W3520 CPU core, high I/O priority (to ensure CPU-bound execution), 1 GB RAM. Requests originating from another VPS on that system.

Software: Debian 7.0 i386, nginx 1.2.1, PHP 5.5.0 from dotdeb, XCache SVN trunk from 2013-06-23 (r1269).

(This test may or may not reflect what is happening on dedicated servers. Unfortunately, we don't have a spare server-grade physical machine at the moment. That being said, the machine which held the VPS is under a small network and IO load.

PHP-FPM configuration

pm = dynamic
pm.max_children = 25
pm.start_servers = 4
pm.min_spare_servers = 2
pm.max_spare_servers = 10
pm.max_requests = 5000

Zend Opcache has been configured as following:

root@opcode:/var/www# grep ^opcache /etc/php5/fpm/php.ini
opcache.enable=1
opcache.enable_cli=1
opcache.memory_consumption=128
opcache.interned_strings_buffer=4
opcache.max_accelerated_files=3907
opcache.fast_shutdown=1

XCache has been configured as following:

xcache.size = 128M
xcache.count = 3
xcache.cacher = On

Joomla

Joomla! 3.1.1. Installation was tricky, installation/application/framework.php had to be edited, this line in particular: ini_set('display_errors', true); - otherwise AJAX calls didn't succeed because JSON has been polluted with error messages.

The "Default English" sample data set has been installed. Joomla cache has been disabled because it's not what we want to test.

Test

The Apache Benchmark has been used to measure average response time and number of requests served per seconds. The command line was:

ab -n 10000 -c 20 

Ten thousands requests have been performed by 20 concurrent threads. This corresponds with the pm.max_children=25 setting in php.ini.

No cache

Performance is clearly CPU-bound. CPU usage is 100%, RAM utilization is about 880 MB out of 1GB. Disk I/O is negligible measured from both within the container (%wa 0) and the hypervisor (%wa 7).

Apache Benchmark output (Unfold)

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 176.31.28.48 (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests


Server Software:        nginx/1.2.1
Server Hostname:        176.31.28.48
Server Port:            80

Document Path:          /
Document Length:        8626 bytes

Concurrency Level:      20
Time taken for tests:   1743.994 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      89890000 bytes
HTML transferred:       86260000 bytes
Requests per second:    5.73 [#/sec] (mean)
Time per request:       3487.989 [ms] (mean)
Time per request:       174.399 [ms] (mean, across all concurrent requests)
Transfer rate:          50.33 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:   656 3483 1020.6   3166    9587
Waiting:      403 2999 778.5   2812    8757
Total:        656 3483 1020.6   3166    9587

Percentage of the requests served within a certain time (ms)
  50%   3166
  66%   3528
  75%   3849
  80%   4091
  90%   4797
  95%   5540
  98%   6695
  99%   7420
 100%   9587 (longest request)

Zend OPcache enabled

It looks like the CPU hasn't been fully utilized with 20 concurrent requests. The usage fluctuated between 70% and 90%. I/O and memory usage were, as previously, within reasonable limits.

Apache Benchmark output (Unfold)

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 176.31.28.48 (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests


Server Software:        nginx/1.2.1
Server Hostname:        176.31.28.48
Server Port:            80

Document Path:          /
Document Length:        8626 bytes

Concurrency Level:      20
Time taken for tests:   658.408 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      89890000 bytes
HTML transferred:       86260000 bytes
Requests per second:    15.19 [#/sec] (mean)
Time per request:       1316.817 [ms] (mean)
Time per request:       65.841 [ms] (mean, across all concurrent requests)
Transfer rate:          133.33 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:   421 1316 426.2   1214    5664
Waiting:      275 1053 355.8    986    4329
Total:        421 1316 426.2   1214    5664

Percentage of the requests served within a certain time (ms)
  50%   1214
  66%   1333
  75%   1448
  80%   1537
  90%   1826
  95%   2065
  98%   2484
  99%   2780
 100%   5664 (longest request)

XCache

Just like previously, the load has been CPU-bound even after enabling XCache.

Apache Benchmark output (Unfold)

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 176.31.28.48 (be patient)
Completed 1000 requests
Completed 2000 requests
Completed 3000 requests
Completed 4000 requests
Completed 5000 requests
Completed 6000 requests
Completed 7000 requests
Completed 8000 requests
Completed 9000 requests
Completed 10000 requests
Finished 10000 requests


Server Software:        nginx/1.2.1
Server Hostname:        176.31.28.48
Server Port:            80

Document Path:          /
Document Length:        8626 bytes

Concurrency Level:      20
Time taken for tests:   769.788 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      89890000 bytes
HTML transferred:       86260000 bytes
Requests per second:    12.99 [#/sec] (mean)
Time per request:       1539.576 [ms] (mean)
Time per request:       76.979 [ms] (mean, across all concurrent requests)
Transfer rate:          114.04 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    0   0.1      0       1
Processing:   601 1538 431.1   1439    4401
Waiting:      393 1232 361.6   1178    4205
Total:        601 1539 431.1   1440    4401

Percentage of the requests served within a certain time (ms)
  50%   1440
  66%   1582
  75%   1696
  80%   1784
  90%   2079
  95%   2333
  98%   2847
  99%   3179
 100%   4401 (longest request)

Graphs

Time graph: less is better

Requests per second graph: more is better

Comment

Well, XCache is no more the best opcode cache around. We're looking forward to widespread adoption of PHP 5.5+. Unfortunately, there is currently no GUI for nicely analyzing cache usage, like XCache has. However, Zend exposes a new function, accelerator_get_status(), which could be used to create such a GUI - so certainly somebody will release relevant GUI in the next weeks.

EDIT: We didn't have to wait long - as a reader pointed out, there is a GUI for Zend Optimizer.

As usual, if you need help, we will make your website fast for you.

Wednesday, June 12, 2013

Pure-PHP Celery connector

Our contribution to the open source community, the Celery-PHP library, had one drawback: it needed a PECL extension that was pretty hard to compile, and not shipped with the most popular distributions. I can only imagine how hard it must have been for some people to set up a working environment.

So during the last month the library has undergone massive refactoring to abstract away the parts responsible for connecting to a queue. This allowed to create pluggable queue backend objects. Right now two are implemented: "pecl", the old code utilizing a PECL extension, and "php-amqplib", using a pure PHP AMQP library.

If you've used Celery-PHP before, you won't notice any difference. The library will just default to using PECL.

Here's how it works: I've added an extra parameter to the constructor of Celery class. If not supplied, it triggers an auto-detection system which first looks for the PECL extension, then checks if PHP-amqplib has been installed by composer, and gives up if neither of these is present. You can force the queue type by passing a string - currently 'pecl' and 'php-amqplib' are accepted.

The new PHP-AMQPLib backend passed most unit tests, but the ones checking how it behaves on long running tasks. The problem is that AMQPLib doesn't allow specifying an arbitrary interval when waiting for a new message, it's always an integer number of seconds. For this reason some tests don't behave as expected, but it shouldn't affect real-life code other than that "asynchronous" checks for result take a whole second. I've prepared a pull request to address this problem, but the developer didn't decide to pull it in yet.

If you want to try it out, pull the code from its branch on github. It will be available in the main branch once it's well tested and the php-amqplib pull request goes through (so vote for it if you want to use PECL-less Celery!).

Thursday, May 9, 2013

Celery-PHP updated, works with Celery 3.0

Updated Celery binding for PHP is available from Github.

I've ran unit tests with latest Celery and RabbitMQ from Debian Squeeze. They went well without any change in code logic.

I've came across some problems during setup of testing environment, mainly during installation of php-pecl-amqp. The install_amqp.sh script is now updated with current librabbitmq-c URL and other minor changes to build properly on Squeeze. I've also updated documentation about testing and updated celeryconfig.py in test scenario to use Celery 3.0-style BROKER_URL variable instead of BROKER_HOST etc. used in Celery 2.x.

Software versions used:

  • Celery 3.0.19
  • RabbitMQ 2.8.4
  • PHP 5.4.4
  • PHP-AMQP 1.0.10
  • librabbitmq 0.3.0

Monday, April 29, 2013

Varnish-Joomla patch version 1.2.2 released

A new version of Varnish-Joomla patch is available. The most interesting news is Joomla! 3.1 support.

Other than that, patches for Joomla! 2.5 and 3.0 have been updated to the latest version of their respective families (2.5.11 and 3.0.4).

A bug pointed by Yves Lavoie has been fixed too. It could make the patched system error out in some rare circumstances.

If you're an existing customer, contact us to get the latest package.

If you're interested in purchasing the patch, please head to Order section of the product page: make your Joomla! faster

Friday, March 8, 2013

MySQL administration tool

Administrating a server set up by someone else can be a real pain. One of the problems I have regularly encountered on clients' systems has been database administration. While you can perform some tasks from command line, I really can't imagine working without a tool like PHPMyAdmin.

PHPMyAdmin, however, has a number of problems. The main one for me is its installation process which is much harder than necessary and involves editing configuration files. Consulting time spent installing PMA vs doing actual work is rather hard to explain to customers. So I started looking for alternatives.

The tool I've found performed beyond my wildest expectations. It consists of a single file (which is, by the way, interesting - they "compile" a nice, readable PHP source tree into a single minified PHP file). Not only it supports MySQL, but works with Postgres, SQLite, MS SQL and Oracle. It has a pretty, minimalistic interface. It's fast - not only it executes fast on the server, but it generates small pages so working on a slow network isn't a problem. And - best of all - the one-step installation process consists of uploading a single PHP file to the server.

Adminer, the tool I'm writing about, supports everything I care for. Some things could be done better or easier, but it's a really viable PHPMyAdmin alternative - I never found myself in a position where I couldn't do something and had to install PMA.

As usual, if you need help making your website faster, be sure to contact us.

Friday, January 11, 2013

Rails + supervisord (named sockets)

A recent upgrade of our project management software, Chili Project, was a good opportunity to make it run like other services - ran from Supervisord, connected via FastCGI. Unfortunately, Rails doesn't make it easy to listen for FCGI connections on a named socket.

After some research, I've found a way to run Chilli Project (and, in fact, any rails app) this way. I've put this little script into scripts/ subdirectory of the rails app:

Also, for completeness: contents of /etc/supervisor/conf.d/redmine.conf

Need help hosting Rails?