├── LICENSE ├── README.md ├── composer.json └── src ├── Options.php ├── Queue.php ├── States.php └── Task.php /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015-2023 Eugene Leonovich 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Tarantool Queue 2 | 3 | [![Quality Assurance](https://github.com/tarantool-php/queue/workflows/QA/badge.svg)](https://github.com/tarantool-php/queue/actions?query=workflow%3AQA) 4 | [![Scrutinizer Code Quality](https://scrutinizer-ci.com/g/tarantool-php/queue/badges/quality-score.png?b=master)](https://scrutinizer-ci.com/g/tarantool-php/queue/?branch=master) 5 | [![Code Coverage](https://scrutinizer-ci.com/g/tarantool-php/queue/badges/coverage.png?b=master)](https://scrutinizer-ci.com/g/tarantool-php/queue/?branch=master) 6 | [![Mentioned in Awesome PHP](https://awesome.re/mentioned-badge.svg)](https://github.com/ziadoz/awesome-php) 7 | [![Telegram](https://img.shields.io/badge/Telegram-join%20chat-blue.svg)](https://t.me/tarantool_php) 8 | 9 | [Tarantool](https://www.tarantool.io/en/developers/) is a NoSQL database running in a Lua application server. It integrates 10 | Lua modules, called [LuaRocks](https://luarocks.org/). This package provides PHP bindings for 11 | [Tarantool Queue LuaRock](https://github.com/tarantool/queue/). 12 | 13 | 14 | ## Table of contents 15 | 16 | * [Installation](#installation) 17 | * [Before start](#before-start) 18 | * [Working with queue](#working-with-queue) 19 | * [Data types](#data-types) 20 | * [Tasks](#tasks) 21 | * [Producer API](#producer-api) 22 | * [Consumer API](#consumer-api) 23 | * [Statistics](#statistics) 24 | * [Custom methods](#custom-methods) 25 | * [Testing](#testing) 26 | * [License](#license) 27 | 28 | 29 | ## Installation 30 | 31 | The recommended way to install the library is through [Composer](http://getcomposer.org): 32 | 33 | ```bash 34 | composer require tarantool/queue 35 | ``` 36 | 37 | 38 | ## Before start 39 | 40 | In order to use queue, you first need to make sure that your Tarantool instance 41 | is configured, up and running. The minimal required configuration might look like this: 42 | 43 | ```lua 44 | -- queues.lua 45 | 46 | box.cfg {listen = 3301} 47 | 48 | queue = require('queue') 49 | queue.create_tube('foobar', 'fifottl', {if_not_exists = true}) 50 | ``` 51 | 52 | > *You can read more about the box configuration in the official Tarantool [documentation](http://tarantool.org/doc/book/configuration/index.html#initialization-file). 53 | > More information on queue configuration can be found [here](https://github.com/tarantool/queue/blob/master/README.md).* 54 | 55 | To start the instance you need to copy (or symlink) `queues.lua` file into the `/etc/tarantool/instances.enabled` 56 | directory and run the following command: 57 | 58 | ```bash 59 | sudo tarantoolctl start queues 60 | ``` 61 | 62 | 63 | ## Working with queue 64 | 65 | Once you have your instance running, you can start by creating a queue object with the queue (tube) name you defined 66 | in the Lua script: 67 | 68 | ```php 69 | use Tarantool\Queue\Queue; 70 | 71 | ... 72 | 73 | $queue = new Queue($client, 'foobar'); 74 | ``` 75 | 76 | where `$client` is an instance of `Tarantool\Client\Client` from the [tarantool/client](https://github.com/tarantool-php/client) package. 77 | 78 | 79 | ### Data types 80 | 81 | Under the hood Tarantool uses [MessagePack](http://msgpack.org/) binary format to serialize/deserialize 82 | data being stored in a queue. It can handle most of the PHP data types (except resources and closures) without 83 | any manual pre- or post-processing: 84 | 85 | ```php 86 | $queue->put('foo'); 87 | $queue->put(true); 88 | $queue->put(42); 89 | $queue->put(4.2); 90 | $queue->put(['foo' => ['bar' => ['baz' => null]]]); 91 | $queue->put(new MyObject()); 92 | ``` 93 | 94 | > *To learn more about object serialization, please follow this [link](https://github.com/tarantool-php/client#user-defined-types).* 95 | 96 | 97 | ### Tasks 98 | 99 | Most of the [Queue API](src/Queue.php) methods return a [Task](src/Task.php) object 100 | containing the following getters: 101 | 102 | ```php 103 | Task::getId() 104 | Task::getState() // States::READY, States::TAKEN, States::DONE, States::BURY or States::DELAYED 105 | Task::getData() 106 | ``` 107 | 108 | And some sugar methods: 109 | 110 | ```php 111 | Task::isReady() 112 | Task::isTaken() 113 | Task::isDone() 114 | Task::isBuried() 115 | Task::isDelayed() 116 | ``` 117 | 118 | 119 | ### Producer API 120 | 121 | As you've already seen, to insert a task into a queue you need to call `put()` method, which accepts 122 | two arguments: the data you want to process and optional array of task options, which this particular 123 | queue supports. For example, `fifottl` queue (which we defined [earlier](#before-start) in our Lua config 124 | file), supports `delay`, `ttl`, `ttr` and `pri` options: 125 | 126 | ```php 127 | use Tarantool\Queue\Options; 128 | 129 | $queue->put('foo', [Options::DELAY => 30.0]); 130 | $queue->put('bar', [Options::TTL => 5.0]); 131 | $queue->put('baz', [Options::TTR => 10.0, Options::PRI => 42]); 132 | ``` 133 | 134 | > *See the full list of available options [here](https://github.com/tarantool/queue#queue-types).* 135 | 136 | 137 | ### Consumer API 138 | 139 | To reserve a task for execution, call `take()` method. It accepts an optional `timeout` parameter. 140 | If a timeout value is supplied the call will wait `timeout` seconds until a `READY` task appears in the queue. 141 | The method returns either a [Task](#tasks) object or `null`: 142 | 143 | ```php 144 | $taskOrNull = $queue->take(); 145 | 146 | // wait 2 seconds 147 | $taskOrNull = $queue->take(2.0); 148 | 149 | // wait 100 milliseconds 150 | $taskOrNull = $queue->take(.1); 151 | ``` 152 | 153 | After successful execution, a task can be marked as acknowledged (that will also delete the task from a queue): 154 | 155 | ```php 156 | $data = $task->getData(); 157 | 158 | // process $data 159 | 160 | $task = $queue->ack($task->getId()); 161 | ``` 162 | 163 | Or put back into the queue in case it cannot be executed: 164 | 165 | ```php 166 | $task = $queue->release($task->getId()); 167 | 168 | // for *ttl queues you can specify a delay 169 | $task = $queue->release($task->getId(), [Options::DELAY => 30.0]); 170 | ``` 171 | 172 | To look at a task without changing its state, use: 173 | 174 | ```php 175 | $task = $queue->peek($task->getId()); 176 | ``` 177 | 178 | To bury (disable) a task: 179 | 180 | ```php 181 | $task = $queue->bury($task->getId()); 182 | ``` 183 | 184 | To reset buried task(s) back to `READY` state: 185 | 186 | ```php 187 | $count = $queue->kick(3); // kick 3 buried tasks 188 | ``` 189 | 190 | To increase TTR and/or TTL of a running task (only for *ttl queues): 191 | 192 | ```php 193 | $taskOrNull = $queue->touch($takenTask->getId(), 5.0); // increase ttr/ttl to 5 seconds 194 | ``` 195 | 196 | A task (in any state) can be deleted permanently with `delete()`: 197 | 198 | ```php 199 | $task = $queue->delete($task->getId()); 200 | ``` 201 | 202 | To delete all tasks in a queue: 203 | 204 | ```php 205 | $queue->truncate(); 206 | ``` 207 | 208 | > *For a detailed API documentation, please read the section 209 | > "[Using the queue module](https://github.com/tarantool/queue#using-the-queue-module)" 210 | > of the queue README.* 211 | 212 | 213 | ### Statistics 214 | 215 | The `stats()` method provides access to the statistical information accumulated 216 | since a queue was created: 217 | 218 | ```php 219 | $stats = $queue->stats(); 220 | ``` 221 | 222 | The result of this call might look like this: 223 | 224 | ```php 225 | [ 226 | 'tasks' => [ 227 | 'taken' => 1, 228 | 'buried' => 1, 229 | 'ready' => 1, 230 | 'done' => 0, 231 | 'delayed' => 0, 232 | 'total' => 3, 233 | ], 234 | 'calls' => [ 235 | 'bury' => 1, 236 | 'put' => 3, 237 | 'take' => 1, 238 | ... 239 | ], 240 | ] 241 | ``` 242 | 243 | In addition, you can specify a key to return only a subset of the array: 244 | 245 | ```php 246 | $calls = $queue->stats('calls'); 247 | $total = $queue->stats('tasks.total'); 248 | ``` 249 | 250 | 251 | ### Custom methods 252 | 253 | Thanks to flexible nature of the [queue](https://github.com/tarantool/queue/) Lua module, you can easily create 254 | your own queue drivers or extend existing ones with an additional functionality. For example, suppose you added 255 | the `put_many` method to your `foobar` queue, which inserts multiple tasks atomically: 256 | 257 | ```lua 258 | -- queues.lua 259 | 260 | ... 261 | 262 | queue.tube.foobar.put_many = function(self, items) 263 | local put = {} 264 | 265 | box.begin() 266 | for k, item in pairs(items) do 267 | put[k] = tube:put(unpack(item)) 268 | end 269 | box.commit() 270 | 271 | return put 272 | end 273 | ``` 274 | 275 | To invoke this method from php, use `Queue::call()`: 276 | 277 | ```php 278 | $result = $queue->call('put_many', [ 279 | 'foo' => ['foo', [Options::DELAY => 30.0]], 280 | 'bar' => ['bar'], 281 | ]); 282 | ``` 283 | 284 | 285 | ## Testing 286 | 287 | The easiest way to run tests is with Docker. First, build an image using the [dockerfile.sh](dockerfile.sh) generator: 288 | 289 | ```bash 290 | ./dockerfile.sh | docker build -t queue - 291 | ``` 292 | 293 | Then run a Tarantool instance (needed for integration tests): 294 | 295 | ```bash 296 | docker network create tarantool-php 297 | docker run -d --net=tarantool-php -p 3301:3301 --name=tarantool \ 298 | -v $(pwd)/tests/Integration/queues.lua:/queues.lua \ 299 | tarantool/tarantool:2 tarantool /queues.lua 300 | ``` 301 | 302 | And then run both unit and integration tests: 303 | 304 | ```bash 305 | docker run --rm --net=tarantool-php -v $(pwd):/queue -w /queue queue 306 | ``` 307 | 308 | The library uses [PHPUnit](https://phpunit.de/) under the hood, and if needed, 309 | you can pass additional arguments and options to the `phpunit` command. 310 | For example, to run only unit tests, execute: 311 | 312 | ```bash 313 | docker run --rm --net=tarantool-php -v $(pwd):/queue -w /queue queue \ 314 | vendor/bin/phpunit --testsuite=unit 315 | ``` 316 | 317 | 318 | ## License 319 | 320 | The library is released under the MIT License. See the bundled [LICENSE](LICENSE) file for details. 321 | -------------------------------------------------------------------------------- /composer.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "tarantool/queue", 3 | "description": "PHP bindings for Tarantool Queue.", 4 | "keywords": ["queue", "schedule", "delayed", "priority", "ttl", "ttr", "task", "job", "worker", "tarantool", "nosql"], 5 | "type": "library", 6 | "license": "MIT", 7 | "authors": [ 8 | { 9 | "name": "Eugene Leonovich", 10 | "email": "gen.work@gmail.com" 11 | } 12 | ], 13 | "require": { 14 | "php": "^7.2.5|^8", 15 | "tarantool/client": "^0.10" 16 | }, 17 | "require-dev": { 18 | "friendsofphp/php-cs-fixer": "^2.19", 19 | "tarantool/phpunit-extras": "^0.2", 20 | "vimeo/psalm": "^3.9|^4" 21 | }, 22 | "autoload": { 23 | "psr-4": { 24 | "Tarantool\\Queue\\": "src/" 25 | } 26 | }, 27 | "autoload-dev" : { 28 | "psr-4": { 29 | "Tarantool\\Queue\\Tests\\": "tests/" 30 | } 31 | }, 32 | "config": { 33 | "preferred-install": { 34 | "*": "dist" 35 | }, 36 | "sort-packages": true 37 | } 38 | } 39 | -------------------------------------------------------------------------------- /src/Options.php: -------------------------------------------------------------------------------- 1 | 7 | * 8 | * For the full copyright and license information, please view the LICENSE 9 | * file that was distributed with this source code. 10 | */ 11 | 12 | declare(strict_types=1); 13 | 14 | namespace Tarantool\Queue; 15 | 16 | final class Options 17 | { 18 | public const DELAY = 'delay'; 19 | public const PRI = 'pri'; 20 | public const TTL = 'ttl'; 21 | public const TTR = 'ttr'; 22 | public const UTUBE = 'utube'; 23 | 24 | private function __construct() 25 | { 26 | } 27 | } 28 | -------------------------------------------------------------------------------- /src/Queue.php: -------------------------------------------------------------------------------- 1 | 7 | * 8 | * For the full copyright and license information, please view the LICENSE 9 | * file that was distributed with this source code. 10 | */ 11 | 12 | declare(strict_types=1); 13 | 14 | namespace Tarantool\Queue; 15 | 16 | use Tarantool\Client\Client; 17 | 18 | class Queue 19 | { 20 | private $client; 21 | private $name; 22 | 23 | public function __construct(Client $client, string $name) 24 | { 25 | $this->client = $client; 26 | $this->name = $name; 27 | } 28 | 29 | public function getClient() : Client 30 | { 31 | return $this->client; 32 | } 33 | 34 | public function getName() : string 35 | { 36 | return $this->name; 37 | } 38 | 39 | /** 40 | * @param mixed $data 41 | */ 42 | public function put($data, array $options = []) : Task 43 | { 44 | return Task::fromTuple( 45 | $this->client->call("queue.tube.$this->name:put", $data, $options)[0] 46 | ); 47 | } 48 | 49 | public function take(float $timeout = null) : ?Task 50 | { 51 | $result = $this->client->call("queue.tube.$this->name:take", $timeout); 52 | 53 | return empty($result[0]) ? null : Task::fromTuple($result[0]); 54 | } 55 | 56 | public function touch(int $taskId, float $increment) : ?Task 57 | { 58 | $result = $this->client->call("queue.tube.$this->name:touch", $taskId, $increment); 59 | 60 | return empty($result[0]) ? null : Task::fromTuple($result[0]); 61 | } 62 | 63 | public function ack(int $taskId) : Task 64 | { 65 | return Task::fromTuple( 66 | $this->client->call("queue.tube.$this->name:ack", $taskId)[0] 67 | ); 68 | } 69 | 70 | public function release(int $taskId, array $options = []) : Task 71 | { 72 | return Task::fromTuple( 73 | $this->client->call("queue.tube.$this->name:release", $taskId, $options)[0] 74 | ); 75 | } 76 | 77 | public function peek(int $taskId) : Task 78 | { 79 | return Task::fromTuple( 80 | $this->client->call("queue.tube.$this->name:peek", $taskId)[0] 81 | ); 82 | } 83 | 84 | public function bury(int $taskId) : Task 85 | { 86 | return Task::fromTuple( 87 | $this->client->call("queue.tube.$this->name:bury", $taskId)[0] 88 | ); 89 | } 90 | 91 | public function kick(int $count) : int 92 | { 93 | return $this->client->call("queue.tube.$this->name:kick", $count)[0]; 94 | } 95 | 96 | public function delete(int $taskId) : Task 97 | { 98 | return Task::fromTuple( 99 | $this->client->call("queue.tube.$this->name:delete", $taskId)[0] 100 | ); 101 | } 102 | 103 | public function truncate() : void 104 | { 105 | $this->client->call("queue.tube.$this->name:truncate"); 106 | } 107 | 108 | /** 109 | * @throws \InvalidArgumentException 110 | * 111 | * @return array|int 112 | */ 113 | public function stats(?string $path = null) 114 | { 115 | [$stats] = $this->client->call('queue.stats', $this->name); 116 | 117 | if (null === $path) { 118 | return $stats; 119 | } 120 | 121 | foreach (\explode('.', $path) as $key) { 122 | if (!isset($stats[$key])) { 123 | throw new \InvalidArgumentException(\sprintf('Invalid path "%s"', $path)); 124 | } 125 | $stats = $stats[$key]; 126 | } 127 | 128 | return $stats; 129 | } 130 | 131 | /** 132 | * @param mixed ...$args 133 | */ 134 | public function call(string $methodName, ...$args) : array 135 | { 136 | return $this->client->call("queue.tube.$this->name:$methodName", ...$args); 137 | } 138 | } 139 | -------------------------------------------------------------------------------- /src/States.php: -------------------------------------------------------------------------------- 1 | 7 | * 8 | * For the full copyright and license information, please view the LICENSE 9 | * file that was distributed with this source code. 10 | */ 11 | 12 | declare(strict_types=1); 13 | 14 | namespace Tarantool\Queue; 15 | 16 | final class States 17 | { 18 | public const READY = 'r'; 19 | public const TAKEN = 't'; 20 | public const DONE = '-'; 21 | public const BURIED = '!'; 22 | public const DELAYED = '~'; 23 | 24 | private function __construct() 25 | { 26 | } 27 | } 28 | -------------------------------------------------------------------------------- /src/Task.php: -------------------------------------------------------------------------------- 1 | 7 | * 8 | * For the full copyright and license information, please view the LICENSE 9 | * file that was distributed with this source code. 10 | */ 11 | 12 | declare(strict_types=1); 13 | 14 | namespace Tarantool\Queue; 15 | 16 | final class Task 17 | { 18 | private $id; 19 | private $state; 20 | 21 | /** @var mixed */ 22 | private $data; 23 | 24 | /** 25 | * @param mixed $data 26 | */ 27 | private function __construct(int $id, string $state, $data) 28 | { 29 | $this->id = $id; 30 | $this->state = $state; 31 | $this->data = $data; 32 | } 33 | 34 | public static function fromTuple(array $tuple) : self 35 | { 36 | [$id, $state, $data] = $tuple + [2 => null]; 37 | 38 | return new self($id, $state, $data); 39 | } 40 | 41 | public function getId() : int 42 | { 43 | return $this->id; 44 | } 45 | 46 | public function getState() : string 47 | { 48 | return $this->state; 49 | } 50 | 51 | /** 52 | * @return mixed 53 | */ 54 | public function getData() 55 | { 56 | return $this->data; 57 | } 58 | 59 | public function isReady() : bool 60 | { 61 | return States::READY === $this->state; 62 | } 63 | 64 | public function isTaken() : bool 65 | { 66 | return States::TAKEN === $this->state; 67 | } 68 | 69 | public function isDone() : bool 70 | { 71 | return States::DONE === $this->state; 72 | } 73 | 74 | public function isBuried() : bool 75 | { 76 | return States::BURIED === $this->state; 77 | } 78 | 79 | public function isDelayed() : bool 80 | { 81 | return States::DELAYED === $this->state; 82 | } 83 | } 84 | --------------------------------------------------------------------------------