Tarantool Queue

Tarantool is a NoSQL database running in a Lua application server. It integrates Lua modules, called LuaRocks. This package provides PHP bindings for Tarantool Queue LuaRock.

Installation

The recommended way to install the library is through Composer:

$ composer require tarantool/queue

Before start

In order to use queue, you first need to make sure that your Tarantool instance is configured, up and running. The minimal required configuration might look like this:

-- queues.lua

box.cfg {}

queue = require('queue')
queue.start()

queue.create_tube('foobar', 'fifottl', { if_not_exists=true })

You can read more about the box configuration in the official Tarantool documentation. For more information about the queue configuration check out queue's README.

To start the instance you need to copy (or symlink) queues.lua file into the /etc/tarantool/instances.enabled directory and run the following command:

$ sudo tarantoolctl start queues

Working with queue

Once you have your instance running, you can start by creating a queue object with the queue (tube) name you defined in the Lua script:

use Tarantool\Queue\Queue;

...

$queue = new Queue($client, 'foobar');

where $client is either an instance of the Tarantool class from the pecl extension or an instance of Tarantool\Client\Client from the pure PHP package.

Data types

Under the hood Tarantool uses MessagePack binary format to serialize/deserialize data being stored in a queue. It can handle most of the PHP data types (except resources and closures) without any manual pre- or post-processing:

$queue->put('foo');
$queue->put(true);
$queue->put(42);
$queue->put(4.2);
$queue->put(['foo' => ['bar' => ['baz' => null]]]);
$queue->put(new MyObject());

Note

Object serialization is only supported when tarantool/client is used.

Tasks

Most of the Queue API methods return back a Task object containing the following getters:

Task::getId()
Task::getState() // States::READY, States::TAKEN, States::DONE, States::BURY or States::DELAYED
Task::getData()

And some sugar methods:

Task::isReady()
Task::isTaken()
Task::isDone()
Task::isBuried()
Task::isDelayed()

Producer API

As you've already seen, to insert a task into a queue you need to call put() method, which accepts two arguments: the data you want to process and optional array of task options, which this particular queue supports. For example, fifottl queue (which we defined earlier in our Lua config file), supports delay, ttl, ttr, pri and temporary options:

$queue->put('foo', ['delay' => 30]);
$queue->put('bar', ['ttl' => 5]);
$queue->put('baz', ['ttr' => 10, 'pri' => 42]);

See the full list of available options here.

Consumer API

To reserve a task for execution, call take() method. It accepts an optional timeout parameter. If a timeout value is supplied the call will wait timeout seconds until a READY task appears in the queue. The method returns either a Task object or null:

$task = $queue->take();

// wait 2 seconds
$task = $queue->take(2);

// wait 100 milliseconds
$task = $queue->take(.1);

After successful execution, a task can be marked as acknowledged (that will also delete the task from a queue):

$data = $task->getData();

// process $data

$task = $queue->ack($task->getId());

Or put back into the queue in case it cannot be executed:

$task = $queue->release($task->getId());

// for ttl-like queues you can specify a delay
$task = $queue->release($task->getId(), ['delay' => 30]);

To look at a task without changing its state, use:

$task = $queue->peek($task->getId());

To bury (disable) a task:

$task = $queue->bury($task->getId());

To reset buried task(s) back to READY state:

$count = $queue->kick(3); // kick 3 buried tasks

A task (in any state) can be deleted permanently with delete():

$task = $queue->delete($task->getId());

To delete all tasks in a queue:

$queue->truncate();

For a detailed API documentation, please read the section "Using the queue module" of the queue's README.

Statistics

The stats() method provides access to the statistical information accumulated since a queue was created:

$stats = $queue->stats();

The result of this call might look like this:

[
    'tasks' => [
        'taken'   => 1,
        'buried'  => 1,
        'ready'   => 1,
        'done'    => 0,
        'delayed' => 0,
        'total'   => 3,
    ],
    'calls' => [
        'bury' => 1,
        'put'  => 3,
        'take' => 1,
        ...
    ],
]

In addition, you can specify a key to return only a subset of the array:

$calls = $queue->stats('calls');
$total = $queue->stats('tasks.total');

Tests

The easiest way to run tests is with Docker. First, build an image using the dockerfile.sh generator:

$ ./dockerfile.sh | docker build -t queue -

Then run Tarantool instance (needed for integration tests):

$ docker network create tarantool-php
$ docker run -d --net=tarantool-php --name=tarantool -v `pwd`:/queue \
    tarantool/tarantool:1.7 tarantool /queue/tests/Integration/queues.lua

And then run both unit and integration tests:

$ docker run --rm --net=tarantool-php --name=queue -v `pwd`:/queue -w /queue queue

To run only integration or unit tests, set the PHPUNIT_OPTS environment variable to either --testsuite Integration or --testsuite Unit respectively, e.g.:

$ docker run --rm --net=tarantool-php --name=queue -v `pwd`:/queue -w /queue \
    -e PHPUNIT_OPTS='--testsuite Unit' queue

License

The library is released under the MIT License. See the bundled LICENSE file for details.