Wednesday, June 12, 2013

Pure-PHP Celery connector

Our contribution to the open source community, the Celery-PHP library, had one drawback: it needed a PECL extension that was pretty hard to compile, and not shipped with the most popular distributions. I can only imagine how hard it must have been for some people to set up a working environment.

So during the last month the library has undergone massive refactoring to abstract away the parts responsible for connecting to a queue. This allowed to create pluggable queue backend objects. Right now two are implemented: "pecl", the old code utilizing a PECL extension, and "php-amqplib", using a pure PHP AMQP library.

If you've used Celery-PHP before, you won't notice any difference. The library will just default to using PECL.

Here's how it works: I've added an extra parameter to the constructor of Celery class. If not supplied, it triggers an auto-detection system which first looks for the PECL extension, then checks if PHP-amqplib has been installed by composer, and gives up if neither of these is present. You can force the queue type by passing a string - currently 'pecl' and 'php-amqplib' are accepted.

The new PHP-AMQPLib backend passed most unit tests, but the ones checking how it behaves on long running tasks. The problem is that AMQPLib doesn't allow specifying an arbitrary interval when waiting for a new message, it's always an integer number of seconds. For this reason some tests don't behave as expected, but it shouldn't affect real-life code other than that "asynchronous" checks for result take a whole second. I've prepared a pull request to address this problem, but the developer didn't decide to pull it in yet.

If you want to try it out, pull the code from its branch on github. It will be available in the main branch once it's well tested and the php-amqplib pull request goes through (so vote for it if you want to use PECL-less Celery!).

No comments:

Post a Comment