├── .formatter.exs ├── .gitignore ├── LICENSE ├── README.md ├── lib ├── cubq.ex └── cubq │ └── queue.ex ├── mix.exs ├── mix.lock └── test ├── cubq_test.exs └── test_helper.exs /.formatter.exs: -------------------------------------------------------------------------------- 1 | # Used by "mix format" 2 | [ 3 | inputs: ["{mix,.formatter}.exs", "{config,lib,test}/**/*.{ex,exs}"] 4 | ] 5 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # The directory Mix will write compiled artifacts to. 2 | /_build/ 3 | 4 | # If you run "mix test --cover", coverage assets end up here. 5 | /cover/ 6 | 7 | # The directory Mix downloads your dependencies sources to. 8 | /deps/ 9 | 10 | # Where third-party dependencies like ExDoc output generated docs. 11 | /doc/ 12 | 13 | # Ignore .fetch files in case you like to edit your project deps locally. 14 | /.fetch 15 | 16 | # If the VM crashes, it generates a dump, let's ignore it too. 17 | erl_crash.dump 18 | 19 | # Also ignore archive artifacts (built via "mix archive.build"). 20 | *.ez 21 | 22 | # Ignore package tarball (built via "mix hex.build"). 23 | cubq-*.tar 24 | 25 | .elixir_ls/ 26 | tmp/ 27 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2020 Luca Ongaro 2 | 3 | Licensed under the Apache License, Version 2.0 (the "License"); 4 | you may not use this file except in compliance with the License. 5 | You may obtain a copy of the License at 6 | 7 | http://www.apache.org/licenses/LICENSE-2.0 8 | 9 | Unless required by applicable law or agreed to in writing, software 10 | distributed under the License is distributed on an "AS IS" BASIS, 11 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | See the License for the specific language governing permissions and 13 | limitations under the License. 14 | 15 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![Hex Version](https://img.shields.io/hexpm/v/cubq.svg)](https://hex.pm/packages/cubq) [![docs](https://img.shields.io/badge/docs-hexpm-blue.svg)](https://hexdocs.pm/cubq/) 2 | 3 | # CubQ 4 | 5 | An embedded queue and stack abstraction in Elixir on top of 6 | [CubDB](https://github.com/lucaong/cubdb). 7 | 8 | It implements persistent local (double-ended) queue and stack semantics. 9 | 10 | ## Usage 11 | 12 | `CubQ` is given a `CubDB` process and a queue identifier upon start: 13 | 14 | ```elixir 15 | {:ok, db} = CubDB.start_link(data_dir: "my/data/directory") 16 | {:ok, pid} = CubQ.start_link(db: db, queue: :my_queue_id) 17 | ``` 18 | 19 | ### Queues 20 | 21 | Queue semantics are implemented by the `enqueue` and `dequeue` functions: 22 | 23 | ```elixir 24 | CubQ.enqueue(pid, :one) 25 | #=> :ok 26 | 27 | CubQ.enqueue(pid, :two) 28 | #=> :ok 29 | 30 | CubQ.dequeue(pid) 31 | #=> {:ok, :one} 32 | 33 | CubQ.dequeue(pid) 34 | #=> {:ok, :two} 35 | 36 | # When there are no more items in the queue, `dequeue` returns `nil`: 37 | 38 | CubQ.dequeue(pid) 39 | #=> nil 40 | ``` 41 | 42 | Note that items can be any Elixir (or Erlang) term: 43 | 44 | ```elixir 45 | CubQ.enqueue(pid, %SomeStruct{foo: "bar"}) 46 | #=> :ok 47 | 48 | CubQ.dequeue(pid) 49 | #=> {:ok, %SomeStruct{foo: "bar"}} 50 | ``` 51 | 52 | The queue is actually double-ended, so items can be prepended too: 53 | 54 | ```elixir 55 | CubQ.enqueue(pid, :one) 56 | #=> :ok 57 | 58 | CubQ.prepend(pid, :zero) 59 | #=> :ok 60 | 61 | CubQ.dequeue(pid) 62 | #=> {:ok, :zero} 63 | ``` 64 | 65 | ### Stacks 66 | 67 | Stack semantics are implemented by the `push` and `pop` functions: 68 | 69 | ```elixir 70 | CubQ.push(pid, :one) 71 | #=> :ok 72 | 73 | CubQ.push(pid, :two) 74 | #=> :ok 75 | 76 | CubQ.pop(pid) 77 | #=> {:ok, :two} 78 | 79 | CubQ.pop(pid) 80 | #=> {:ok, :one} 81 | 82 | # When there are no more items in the stack, `pop` returns `nil`: 83 | 84 | CubQ.pop(pid) 85 | #=> nil 86 | ``` 87 | 88 | ### Mixing things up 89 | 90 | As the underlying data structure used for stacks and queues is the same, queue 91 | and stack semantics can be mixed on the same queue. 92 | 93 | ### At-least-once semantics 94 | 95 | When multiple consumers are taking items from a queue, and "at least once" 96 | semantics are required, the `dequeue_ack` and `pop_ack` functions allow to 97 | explicitly acknowledge the successful consumption of an item, or else put it 98 | back in the queue after a given timeout elapses: 99 | 100 | ```elixir 101 | CubQ.enqueue(pid, :one) 102 | #=> :ok 103 | 104 | CubQ.enqueue(pid, :two) 105 | #=> :ok 106 | 107 | {:ok, item, ack_id} = CubQ.dequeue_ack(pid, 3000) 108 | #=> {:ok, :one, ack_id} 109 | 110 | # If 3 seconds elapse without `ack` being called, or if `nack` is called, the 111 | # item `:one` is put back to the queue, so it can be consumed again. 112 | 113 | # When successful consumption is confirmed by calling `ack`, the item 114 | # is finally discarded and won't be put back in the queue anymore: 115 | CubQ.ack(pid, ack_id) 116 | #=> :ok 117 | ``` 118 | 119 | ## Installation 120 | 121 | The package can be installed by adding `cubq` to your list of dependencies in 122 | `mix.exs`: 123 | 124 | ```elixir 125 | def deps do 126 | [ 127 | {:cubq, "~> 0.3.0"} 128 | ] 129 | end 130 | ``` 131 | 132 | Documentation can be generated with 133 | [ExDoc](https://github.com/elixir-lang/ex_doc) and published on 134 | [HexDocs](https://hexdocs.pm). The docs can be found at 135 | [https://hexdocs.pm/cubq](https://hexdocs.pm/cubq). 136 | -------------------------------------------------------------------------------- /lib/cubq.ex: -------------------------------------------------------------------------------- 1 | defmodule CubQ do 2 | use GenServer 3 | alias CubQ.Queue 4 | 5 | @moduledoc """ 6 | `CubQ` is a queue abstraction on top of `CubDB`. It implements persistent 7 | local (double-ended) queue and stack semantics. 8 | 9 | ## Usage 10 | 11 | `CubQ` is given a `CubDB` process and a queue identifier upon start: 12 | 13 | ``` 14 | {:ok, db} = CubDB.start_link(data_dir: "my/data/directory") 15 | {:ok, pid} = CubQ.start_link(db: db, queue: :my_queue_id) 16 | ``` 17 | 18 | ### Queues 19 | 20 | Queue semantics are implemented by the `enqueue/2` and `dequeue/1` functions: 21 | 22 | ``` 23 | CubQ.enqueue(pid, :one) 24 | #=> :ok 25 | 26 | CubQ.enqueue(pid, :two) 27 | #=> :ok 28 | 29 | CubQ.dequeue(pid) 30 | #=> {:ok, :one} 31 | 32 | CubQ.dequeue(pid) 33 | #=> {:ok, :two} 34 | 35 | # When there are no more items in the queue, `dequeue/1` returns `nil`: 36 | 37 | CubQ.dequeue(pid) 38 | #=> nil 39 | ``` 40 | 41 | Note that items can be any Elixir (or Erlang) term: 42 | 43 | ``` 44 | CubQ.enqueue(pid, %SomeStruct{foo: "bar"}) 45 | #=> :ok 46 | 47 | CubQ.dequeue(pid) 48 | #=> {:ok, %SomeStruct{foo: "bar"}} 49 | ``` 50 | 51 | The queue is actually double-ended, so items can be prepended too: 52 | 53 | ``` 54 | CubQ.enqueue(pid, :one) 55 | #=> :ok 56 | 57 | CubQ.prepend(pid, :zero) 58 | #=> :ok 59 | 60 | CubQ.dequeue(pid) 61 | #=> {:ok, :zero} 62 | ``` 63 | 64 | ### Stacks 65 | 66 | Stack semantics are implemented by the `push/2` and `pop/1` functions: 67 | 68 | ``` 69 | CubQ.push(pid, :one) 70 | #=> :ok 71 | 72 | CubQ.push(pid, :two) 73 | #=> :ok 74 | 75 | CubQ.pop(pid) 76 | #=> {:ok, :two} 77 | 78 | CubQ.pop(pid) 79 | #=> {:ok, :one} 80 | 81 | # When there are no more items in the stack, `pop/1` returns `nil`: 82 | 83 | CubQ.pop(pid) 84 | #=> nil 85 | ``` 86 | 87 | ### Mixing things up 88 | 89 | As the underlying data structure used for stacks and queues is the same, queue 90 | and stack semantics can be mixed on the same queue. 91 | 92 | ### At-least-once semantics 93 | 94 | When multiple consumers are taking items from a queue, and "at least once" 95 | semantics are required, the `dequeue_ack/2` and `pop_ack/2` functions allow to 96 | explicitly acknowledge the successful consumption of an item, or else put it 97 | back in the queue after a given timeout elapses: 98 | 99 | ``` 100 | CubQ.enqueue(pid, :one) 101 | #=> :ok 102 | 103 | CubQ.enqueue(pid, :two) 104 | #=> :ok 105 | 106 | {:ok, item, ack_id} = CubQ.dequeue_ack(pid, 3000) 107 | #=> {:ok, :one, ack_id} 108 | 109 | # If 3 seconds elapse without `ack` being called, or if `nack` is called, the 110 | # item `:one` is put back to the queue, so it can be consumed again. 111 | 112 | # When successful consumption is confirmed by calling `ack`, the item 113 | # is finally discarded and won't be put back in the queue anymore: 114 | CubQ.ack(pid, ack_id) 115 | #=> :ok 116 | ``` 117 | """ 118 | 119 | defmodule State do 120 | @type t :: %CubQ.State{ 121 | db: GenServer.server(), 122 | queue: term, 123 | pending_acks: %{} 124 | } 125 | 126 | @enforce_keys [:db, :queue] 127 | defstruct [:db, :queue, pending_acks: %{}] 128 | end 129 | 130 | @gen_server_options [:name, :timeout, :debug, :spawn_opt, :hibernate_after] 131 | 132 | @type item :: term 133 | @opaque ack_id :: {term, reference, number, :start | :end} 134 | @type option :: {:db, GenServer.server()} | {:queue, term} 135 | 136 | @spec start_link([option | GenServer.option()]) :: GenServer.on_start() 137 | 138 | @doc """ 139 | Starts a `CubQ` process linked to the calling process. 140 | 141 | The argument is a keyword list of options: 142 | 143 | - `db` (required): the `pid` (or name) of the `CubDB` process for storing 144 | the queue 145 | 146 | - `queue` (required): the identifier of the queue. It can be any Elixir 147 | term, but typically one would use an atom, like `:my_queue` 148 | 149 | `GenServer` options like `name` and `timeout` can also be given, and are 150 | forwarded to `GenServer.start_link/3` as the third argument. 151 | """ 152 | def start_link(options) do 153 | {gen_server_options, options} = Keyword.split(options, @gen_server_options) 154 | 155 | GenServer.start_link(__MODULE__, options, gen_server_options) 156 | end 157 | 158 | @spec start([option | GenServer.option()]) :: GenServer.on_start() 159 | 160 | @doc """ 161 | Starts a `CubQ` process without link. 162 | 163 | The argument is a keyword list of options, see `start_link/1` for details. 164 | """ 165 | def start(options) do 166 | {gen_server_options, options} = Keyword.split(options, @gen_server_options) 167 | 168 | GenServer.start(__MODULE__, options, gen_server_options) 169 | end 170 | 171 | @spec enqueue(GenServer.server(), item) :: :ok | {:error, term} 172 | 173 | @doc """ 174 | Inserts an item at the end of the queue. 175 | 176 | The item can be any Elixir (or Erlang) term. 177 | 178 | ## Example: 179 | 180 | ``` 181 | CubQ.enqueue(pid, :one) 182 | #=> :ok 183 | 184 | CubQ.enqueue(pid, :two) 185 | #=> :ok 186 | 187 | CubQ.dequeue(pid) 188 | #=> {:ok, :one} 189 | ``` 190 | """ 191 | def enqueue(pid, item) do 192 | GenServer.call(pid, {:enqueue, item}) 193 | end 194 | 195 | @spec dequeue(GenServer.server()) :: {:ok, item} | nil | {:error, term} 196 | 197 | @doc """ 198 | Removes an item from the beginning of the queue and returns it. 199 | 200 | It returns `{:ok, item}` if there are items in the queue, `nil` if the 201 | queue is empty, and `{:error, cause}` on error. 202 | 203 | ## Example: 204 | 205 | ``` 206 | CubQ.enqueue(pid, :one) 207 | #=> :ok 208 | 209 | CubQ.enqueue(pid, :two) 210 | #=> :ok 211 | 212 | CubQ.dequeue(pid) 213 | #=> {:ok, :one} 214 | 215 | CubQ.dequeue(pid) 216 | #=> {:ok, :two} 217 | ``` 218 | """ 219 | def dequeue(pid) do 220 | GenServer.call(pid, :dequeue) 221 | end 222 | 223 | @spec dequeue_ack(GenServer.server(), timeout) :: {:ok, item, ack_id} | nil | {:error, term} 224 | 225 | @doc """ 226 | Removes an item from the beginning of the queue and returns it, expecting a 227 | manual confirmation of its successful processing with `ack/2`. If `ack/2` is 228 | not called before `timeout`, the item is put back at the beginning of the 229 | queue. 230 | 231 | The `dequeue_ack/2` function is useful when implementing "at least once" 232 | semantics, especially when more than one consumer takes items from the same 233 | queue, to ensure that each item is successfully consumed before being 234 | discarded. 235 | 236 | The problem is the following: if a consumer took an item with `dequeue/1` and 237 | then crashed before processing it, the item would be lost. With 238 | `dequeue_ack/2` instead, the item is not immediately removed, but instead 239 | atomically transfered to a staging storage. If the consumer successfully 240 | processes the item, it can call `ack/2` (acknowledgement) to confirm the 241 | success, after which the item is discarded. If `ack/2` is not called within 242 | the `timeout` (expressed in milliseconds, 5000 by default), the item is 243 | automatically put back to the queue, so it can be dequeued again by a 244 | consumer. If the consumer wants to put back the item to the queue immediately, 245 | it can also call `nack/2` (negative acknowledgement) explicitly. 246 | 247 | The return value of `dequeue_ack/2` is a 3-tuple: `{:ok, item, ack_id}`. 248 | The `ack_id` is the argument that must be passed to `ack/2` or `nack/2` to 249 | confirm the successful (or insuccessful) consumption of the item. 250 | 251 | Note that `dequeue_ack/2` performs its operation in an atomic and durable way, 252 | so even if the `CubQ` process crashes, after restarting it will still 253 | re-insert the items pending acknowledgement in the queue after the timeout 254 | elapses. After restarting though, the timeouts are also restarted, so the 255 | effective time before the item goes back to the queue can be larger than the 256 | original timeout. 257 | 258 | In case of timeout or negative acknowledgement, the item is put back in the 259 | queue from the start, so while global ordering cannot be enforced in case of 260 | items being put back to the queue, the items will be ready to be dequeued 261 | again immediately after being back to the queue. 262 | 263 | ## Example 264 | 265 | ``` 266 | CubQ.enqueue(pid, :one) 267 | #=> :ok 268 | 269 | CubQ.enqueue(pid, :two) 270 | #=> :ok 271 | 272 | {:ok, item, ack_id} = CubQ.dequeue_ack(pid, 3000) 273 | #=> {:ok, :one, ack_id} 274 | 275 | # If 3 seconds elapse without `ack` being called, or `nack` is called, the 276 | # item `:one` is put back to the queue, so it can be dequeued again: 277 | CubQ.nack(pid, ack_id) 278 | #=> :ok 279 | 280 | # The item `:one` is back in the queue, as `nack/2` was called: 281 | {:ok, item, ack_id} = CubQ.dequeue_ack(pid, 3000) 282 | #=> {:ok, :one, ack_id} 283 | 284 | # When successful consumption is confirmed by calling `ack`, the item 285 | # is finally discarded and won't be put back in the queue anymore: 286 | CubQ.ack(pid, ack_id) 287 | #=> :ok 288 | ``` 289 | """ 290 | def dequeue_ack(pid, timeout \\ 5000) do 291 | GenServer.call(pid, {:dequeue_ack, timeout}) 292 | end 293 | 294 | @spec pop_ack(GenServer.server(), timeout) :: {:ok, item, ack_id} | nil | {:error, term} 295 | 296 | @doc """ 297 | Removes an item from the end of the queue and returns it, expecting a manual 298 | confirmation of its successful processing with `ack/2`. If `ack/2` is not 299 | called before `timeout`, the item is put back at the end of the queue. 300 | 301 | See the documentation for `dequeue_ack/2` for more details: the `pop_ack/2` 302 | function works in the same way as `dequeue_ack/2`, but with stack semantics 303 | instead of queue semantics. 304 | """ 305 | def pop_ack(pid, timeout \\ 5000) do 306 | GenServer.call(pid, {:pop_ack, timeout}) 307 | end 308 | 309 | @spec ack(GenServer.server(), ack_id) :: :ok | {:error, term} 310 | 311 | @doc """ 312 | Positively acknowledges the successful consumption of an item taken with 313 | `dequeue_ack/2` or `pop_ack/2`. 314 | 315 | See the documentation for `dequeue_ack/2` for more details. 316 | """ 317 | def ack(pid, ack_id) do 318 | GenServer.call(pid, {:ack, ack_id}) 319 | end 320 | 321 | @spec nack(GenServer.server(), ack_id) :: :ok | {:error, term} 322 | 323 | @doc """ 324 | Negatively acknowledges an item taken with `dequeue_ack/2` or `pop_ack/2`, 325 | causing it to be put back in the queue. 326 | 327 | See the documentation for `dequeue_ack/2` for more details. 328 | """ 329 | def nack(pid, ack_id) do 330 | GenServer.call(pid, {:nack, ack_id}) 331 | end 332 | 333 | @spec peek_first(GenServer.server()) :: {:ok, item} | nil | {:error, term} 334 | 335 | @doc """ 336 | Returns the item at the beginning of the queue without removing it. 337 | 338 | It returns `{:ok, item}` if there are items in the queue, `nil` if the 339 | queue is empty, and `{:error, cause}` on error. 340 | 341 | ## Example: 342 | 343 | ``` 344 | CubQ.enqueue(pid, :one) 345 | #=> :ok 346 | 347 | CubQ.enqueue(pid, :two) 348 | #=> :ok 349 | 350 | CubQ.peek_first(pid) 351 | #=> {:ok, :one} 352 | 353 | CubQ.dequeue(pid) 354 | #=> {:ok, :one} 355 | ``` 356 | """ 357 | def peek_first(pid) do 358 | GenServer.call(pid, :peek_first) 359 | end 360 | 361 | @spec append(GenServer.server(), item) :: :ok | {:error, term} 362 | 363 | @doc """ 364 | Same as `enqueue/2` 365 | """ 366 | def append(pid, item) do 367 | enqueue(pid, item) 368 | end 369 | 370 | @spec prepend(GenServer.server(), item) :: :ok | {:error, term} 371 | 372 | @doc """ 373 | Inserts an item at the beginning of the queue. 374 | 375 | The item can be any Elixir (or Erlang) term. 376 | 377 | ## Example: 378 | 379 | ``` 380 | CubQ.enqueue(pid, :one) 381 | #=> :ok 382 | 383 | CubQ.prepend(pid, :zero) 384 | #=> :ok 385 | 386 | CubQ.dequeue(pid) 387 | #=> {:ok, :zero} 388 | 389 | CubQ.dequeue(pid) 390 | #=> {:ok, :one} 391 | ``` 392 | """ 393 | def prepend(pid, item) do 394 | GenServer.call(pid, {:prepend, item}) 395 | end 396 | 397 | @spec push(GenServer.server(), item) :: :ok | {:error, term} 398 | 399 | @doc """ 400 | Same as `enqueue/2`. 401 | 402 | Normally used together with `pop/1` for stack semantics. 403 | """ 404 | def push(pid, item) do 405 | enqueue(pid, item) 406 | end 407 | 408 | @spec pop(GenServer.server()) :: {:ok, item} | nil | {:error, term} 409 | 410 | @doc """ 411 | Removes an item from the end of the queue and returns it. 412 | 413 | It returns `{:ok, item}` if there are items in the queue, `nil` if the 414 | queue is empty, and `{:error, cause}` on error. 415 | 416 | ## Example: 417 | 418 | ``` 419 | CubQ.push(pid, :one) 420 | #=> :ok 421 | 422 | CubQ.push(pid, :two) 423 | #=> :ok 424 | 425 | CubQ.pop(pid) 426 | #=> {:ok, :two} 427 | 428 | CubQ.pop(pid) 429 | #=> {:ok, :one} 430 | ``` 431 | """ 432 | def pop(pid) do 433 | GenServer.call(pid, :pop) 434 | end 435 | 436 | @spec peek_last(GenServer.server()) :: {:ok, item} | nil | {:error, term} 437 | 438 | @doc """ 439 | Returns the item at the end of the queue without removing it. 440 | 441 | It returns `{:ok, item}` if there are items in the queue, `nil` if the 442 | queue is empty, and `{:error, cause}` on error. 443 | 444 | ## Example: 445 | 446 | ``` 447 | CubQ.enqueue(pid, :one) 448 | #=> :ok 449 | 450 | CubQ.enqueue(pid, :two) 451 | #=> :ok 452 | 453 | CubQ.peek_last(pid) 454 | #=> {:ok, :two} 455 | 456 | CubQ.pop(pid) 457 | #=> {:ok, :two} 458 | ``` 459 | """ 460 | def peek_last(pid) do 461 | GenServer.call(pid, :peek_last) 462 | end 463 | 464 | @spec delete_all(GenServer.server(), pos_integer) :: :ok | {:error, term} 465 | 466 | @doc """ 467 | Deletes all items from the queue. 468 | 469 | The items are deleted in batches, and the size of the batch can be 470 | specified as the optional second argument. 471 | """ 472 | def delete_all(pid, batch_size \\ 100) do 473 | GenServer.call(pid, {:delete_all, batch_size}) 474 | end 475 | 476 | # GenServer callbacks 477 | 478 | @impl true 479 | 480 | def init(options) do 481 | db = Keyword.fetch!(options, :db) 482 | queue = Keyword.fetch!(options, :queue) 483 | {:ok, %State{db: db, queue: queue}, {:continue, nil}} 484 | end 485 | 486 | @impl true 487 | 488 | def handle_continue(_continue, state = %State{db: db, queue: queue}) do 489 | pending_acks = 490 | Enum.reduce(Queue.get_pending_acks!(db, queue), %{}, fn {key, {_, timeout}}, map -> 491 | Map.put(map, key, schedule_ack_timeout(key, timeout)) 492 | end) 493 | 494 | {:noreply, %State{state | pending_acks: pending_acks}} 495 | end 496 | 497 | @impl true 498 | 499 | def handle_call({:enqueue, item}, _from, state = %State{db: db, queue: queue}) do 500 | reply = Queue.enqueue(db, queue, item) 501 | {:reply, reply, state} 502 | end 503 | 504 | def handle_call(:dequeue, _from, state = %State{db: db, queue: queue}) do 505 | reply = Queue.dequeue(db, queue) 506 | {:reply, reply, state} 507 | end 508 | 509 | def handle_call(:peek_first, _from, state = %State{db: db, queue: queue}) do 510 | reply = Queue.peek_first(db, queue) 511 | {:reply, reply, state} 512 | end 513 | 514 | def handle_call(:pop, _from, state = %State{db: db, queue: queue}) do 515 | reply = Queue.pop(db, queue) 516 | {:reply, reply, state} 517 | end 518 | 519 | def handle_call(:peek_last, _from, state = %State{db: db, queue: queue}) do 520 | reply = Queue.peek_last(db, queue) 521 | {:reply, reply, state} 522 | end 523 | 524 | def handle_call({:prepend, item}, _from, state = %State{db: db, queue: queue}) do 525 | reply = Queue.prepend(db, queue, item) 526 | {:reply, reply, state} 527 | end 528 | 529 | def handle_call({:delete_all, batch_size}, _from, state = %State{db: db, queue: queue}) do 530 | reply = Queue.delete_all(db, queue, batch_size) 531 | {:reply, reply, state} 532 | end 533 | 534 | def handle_call({:dequeue_ack, timeout}, _from, state) do 535 | %State{db: db, queue: queue, pending_acks: pending_acks} = state 536 | reply = Queue.dequeue_ack(db, queue, timeout) 537 | 538 | state = 539 | case reply do 540 | {:ok, _, ack_id} -> 541 | ref = schedule_ack_timeout(ack_id, timeout) 542 | %State{state | pending_acks: Map.put(pending_acks, ack_id, ref)} 543 | 544 | _ -> 545 | state 546 | end 547 | 548 | {:reply, reply, state} 549 | end 550 | 551 | def handle_call({:pop_ack, timeout}, _from, state) do 552 | %State{db: db, queue: queue, pending_acks: pending_acks} = state 553 | reply = Queue.pop_ack(db, queue, timeout) 554 | 555 | state = 556 | case reply do 557 | {:ok, _, ack_id} -> 558 | ref = schedule_ack_timeout(ack_id, timeout) 559 | %State{state | pending_acks: Map.put(pending_acks, ack_id, ref)} 560 | 561 | _ -> 562 | state 563 | end 564 | 565 | {:reply, reply, state} 566 | end 567 | 568 | def handle_call({:ack, ack_id}, _from, state) do 569 | %State{db: db, queue: queue, pending_acks: pending_acks} = state 570 | 571 | with :ok <- Queue.ack(db, queue, ack_id) do 572 | case Map.pop(pending_acks, ack_id) do 573 | {nil, _} -> 574 | {:reply, {:error, :not_found}, state} 575 | 576 | {timer, pending_acks} -> 577 | Process.cancel_timer(timer) 578 | {:reply, :ok, %State{state | pending_acks: pending_acks}} 579 | end 580 | else 581 | reply -> {:reply, reply, state} 582 | end 583 | end 584 | 585 | def handle_call({:nack, ack_id}, _from, state) do 586 | %State{db: db, queue: queue, pending_acks: pending_acks} = state 587 | 588 | with :ok <- Queue.nack(db, queue, ack_id) do 589 | case Map.pop(pending_acks, ack_id) do 590 | {nil, _} -> 591 | {:reply, {:error, :not_found}, state} 592 | 593 | {timer, pending_acks} -> 594 | Process.cancel_timer(timer) 595 | {:reply, :ok, %State{state | pending_acks: pending_acks}} 596 | end 597 | else 598 | reply -> {:reply, reply, state} 599 | end 600 | end 601 | 602 | @impl true 603 | 604 | def handle_info({:ack_timeout, ack_id}, state) do 605 | %State{db: db, queue: queue, pending_acks: pending_acks} = state 606 | :ok = Queue.nack(db, queue, ack_id) 607 | {:noreply, %State{state | pending_acks: Map.delete(pending_acks, ack_id)}} 608 | end 609 | 610 | @spec schedule_ack_timeout(ack_id, timeout) :: reference 611 | 612 | defp schedule_ack_timeout(ack_id, timeout) do 613 | Process.send_after(self(), {:ack_timeout, ack_id}, timeout) 614 | end 615 | end 616 | -------------------------------------------------------------------------------- /lib/cubq/queue.ex: -------------------------------------------------------------------------------- 1 | defmodule CubQ.Queue do 2 | @moduledoc false 3 | 4 | @typep key :: {term, number} 5 | @typep entry :: {key, CubQ.item()} 6 | @typep select_option :: 7 | {:min_key, key} 8 | | {:max_key, key} 9 | | {:reverse, boolean} 10 | | {:pipe, [{:take, pos_integer}]} 11 | 12 | @spec enqueue(GenServer.server(), term, CubQ.item()) :: :ok | {:error, term} 13 | 14 | def enqueue(db, queue, item) do 15 | conditions = select_conditions(queue) 16 | 17 | case get_entry(db, queue, [{:reverse, true} | conditions]) do 18 | {:ok, {{_queue, n}, _value}} -> 19 | CubDB.put(db, {queue, n + 1}, item) 20 | 21 | nil -> 22 | CubDB.put(db, {queue, 0}, item) 23 | 24 | other -> 25 | other 26 | end 27 | end 28 | 29 | @spec prepend(GenServer.server(), term, CubQ.item()) :: :ok | {:error, term} 30 | 31 | def prepend(db, queue, item) do 32 | case get_entry(db, queue, select_conditions(queue)) do 33 | {:ok, {{_queue, n}, _value}} -> 34 | CubDB.put(db, {queue, n - 1}, item) 35 | 36 | nil -> 37 | CubDB.put(db, {queue, 0}, item) 38 | 39 | other -> 40 | other 41 | end 42 | end 43 | 44 | @spec dequeue(GenServer.server(), term) :: {:ok, CubQ.item()} | nil | {:error, term} 45 | 46 | def dequeue(db, queue) do 47 | case get_entry(db, queue, select_conditions(queue)) do 48 | {:ok, {key, value}} -> 49 | with :ok <- CubDB.delete(db, key), do: {:ok, value} 50 | 51 | other -> 52 | other 53 | end 54 | end 55 | 56 | @spec pop(GenServer.server(), term) :: {:ok, CubQ.item()} | nil | {:error, term} 57 | 58 | def pop(db, queue) do 59 | conditions = select_conditions(queue) 60 | 61 | case get_entry(db, queue, [{:reverse, true} | conditions]) do 62 | {:ok, {key, value}} -> 63 | with :ok <- CubDB.delete(db, key), do: {:ok, value} 64 | 65 | other -> 66 | other 67 | end 68 | end 69 | 70 | @spec peek_first(GenServer.server(), term) :: {:ok, CubQ.item()} | nil | {:error, term} 71 | 72 | def peek_first(db, queue) do 73 | case get_entry(db, queue, select_conditions(queue)) do 74 | {:ok, {_key, value}} -> 75 | {:ok, value} 76 | 77 | other -> 78 | other 79 | end 80 | end 81 | 82 | @spec peek_last(GenServer.server(), term) :: {:ok, CubQ.item()} | nil | {:error, term} 83 | 84 | def peek_last(db, queue) do 85 | conditions = select_conditions(queue) 86 | 87 | case get_entry(db, queue, [{:reverse, true} | conditions]) do 88 | {:ok, {_key, value}} -> 89 | {:ok, value} 90 | 91 | other -> 92 | other 93 | end 94 | end 95 | 96 | @spec delete_all(GenServer.server(), term, pos_integer) :: :ok | {:error, term} 97 | 98 | def delete_all(db, queue, batch_size \\ 100) do 99 | conditions = select_conditions(queue) 100 | pipe = Keyword.get(conditions, :pipe, []) |> Keyword.put(:take, batch_size) 101 | batch_conditions = Keyword.put(conditions, :pipe, pipe) 102 | 103 | case CubDB.select(db, batch_conditions) do 104 | {:ok, []} -> 105 | :ok 106 | 107 | {:ok, items} when is_list(items) -> 108 | keys = Enum.map(items, fn {key, _value} -> key end) 109 | 110 | with :ok <- CubDB.delete_multi(db, keys), 111 | do: delete_all(db, queue, batch_size) 112 | 113 | {:error, error} -> 114 | {:error, error} 115 | end 116 | end 117 | 118 | @spec dequeue_ack(GenServer.server(), term, timeout) :: 119 | {:ok, CubQ.item(), CubQ.ack_id()} | nil | {:error, term} 120 | 121 | def dequeue_ack(db, queue, timeout) do 122 | case get_entry(db, queue, select_conditions(queue)) do 123 | {:ok, entry} -> 124 | stage_item(db, queue, entry, timeout, :start) 125 | 126 | other -> 127 | other 128 | end 129 | end 130 | 131 | @spec pop_ack(GenServer.server(), term, timeout) :: 132 | {:ok, CubQ.item(), CubQ.ack_id()} | nil | {:error, term} 133 | 134 | def pop_ack(db, queue, timeout) do 135 | conditions = select_conditions(queue) 136 | 137 | case get_entry(db, queue, [{:reverse, true} | conditions]) do 138 | {:ok, entry} -> 139 | stage_item(db, queue, entry, timeout, :end) 140 | 141 | other -> 142 | other 143 | end 144 | end 145 | 146 | @spec ack(GenServer.server(), term, CubQ.ack_id()) :: :ok | {:error, term} 147 | 148 | def ack(db, _queue, ack_id) do 149 | CubDB.delete(db, ack_id) 150 | end 151 | 152 | @spec nack(GenServer.server(), term, CubQ.ack_id()) :: :ok | {:error, term} 153 | 154 | def nack(db, queue, ack_id) do 155 | # item can be requeued at the start or at the end of the queue 156 | {conditions, increment} = case ack_id do 157 | {_, _, _, :end} -> 158 | {[{:reverse, true} | select_conditions(queue)], 1} 159 | 160 | _ -> 161 | {select_conditions(queue), -1} 162 | end 163 | 164 | case get_entry(db, queue, conditions) do 165 | {:ok, {{_queue, n}, _value}} -> 166 | with {:ok, nil} <- commit_nack(db, queue, ack_id, n + increment), do: :ok 167 | 168 | nil -> 169 | with {:ok, nil} <- commit_nack(db, queue, ack_id, 0), do: :ok 170 | 171 | other -> 172 | other 173 | end 174 | end 175 | 176 | @spec get_pending_acks!(GenServer.server(), term) :: [ 177 | {CubQ.ack_id(), {CubQ.item(), timeout}} 178 | ] 179 | 180 | def get_pending_acks!(db, queue) do 181 | case CubDB.select(db, 182 | min_key: {queue, nil, 0, 0}, 183 | max_key: {queue, [], nil, []} 184 | ) do 185 | {:ok, entries} -> entries 186 | {:error, error} -> raise(error) 187 | end 188 | end 189 | 190 | @spec select_conditions(term) :: [select_option] 191 | 192 | defp select_conditions(queue) do 193 | [min_key: {queue, -1.0e32}, max_key: {queue, nil}, pipe: [take: 1]] 194 | end 195 | 196 | @spec get_entry(GenServer.server(), term, [select_option]) :: 197 | {:ok, entry} | nil | {:error, term} 198 | 199 | defp get_entry(db, queue, conditions) do 200 | case CubDB.select(db, conditions) do 201 | {:ok, [entry = {{^queue, n}, _value}]} when is_number(n) -> 202 | {:ok, entry} 203 | 204 | {:ok, []} -> 205 | nil 206 | 207 | {:error, error} -> 208 | {:error, error} 209 | end 210 | end 211 | 212 | @spec stage_item(GenServer.server(), term, entry, timeout, :start | :end) :: 213 | {:ok, CubQ.item(), term} | nil | {:error, term} 214 | 215 | defp stage_item(db, queue, entry, timeout, requeue_pos) do 216 | {key, item} = entry 217 | ack_id = {queue, make_ref(), :rand.uniform_real(), requeue_pos} 218 | 219 | case CubDB.get_and_update_multi(db, [], fn _ -> 220 | {nil, %{ack_id => {item, timeout}}, [key]} 221 | end) do 222 | {:ok, nil} -> 223 | {:ok, item, ack_id} 224 | 225 | other -> 226 | other 227 | end 228 | end 229 | 230 | @spec commit_nack(GenServer.server(), term, CubQ.ack_id(), number) :: 231 | {:ok, term} | {:error, term} 232 | 233 | defp commit_nack(db, queue, ack_id, n) do 234 | CubDB.get_and_update_multi(db, [ack_id], fn 235 | %{^ack_id => {item, _timeout}} -> 236 | {nil, %{{queue, n} => item}, [ack_id]} 237 | 238 | _ -> 239 | {nil, %{}, []} 240 | end) 241 | end 242 | end 243 | -------------------------------------------------------------------------------- /mix.exs: -------------------------------------------------------------------------------- 1 | defmodule CubQ.MixProject do 2 | use Mix.Project 3 | 4 | def project do 5 | [ 6 | app: :cubq, 7 | version: "0.3.0", 8 | elixir: "~> 1.10", 9 | start_permanent: Mix.env() == :prod, 10 | deps: deps(), 11 | package: package(), 12 | source_url: "https://github.com/lucaong/cubq", 13 | docs: [ 14 | main: "CubQ" 15 | ], 16 | test_coverage: [tool: ExCoveralls], 17 | preferred_cli_env: [coveralls: :test, "coveralls.detail": :test, "coveralls.post": :test, "coveralls.html": :test, "coveralls.travis": :test] 18 | ] 19 | end 20 | 21 | def application do 22 | [ 23 | extra_applications: [:logger] 24 | ] 25 | end 26 | 27 | defp deps do 28 | [ 29 | {:cubdb, "~> 0.17 or ~> 1.0"}, 30 | {:dialyxir, "~> 1.0.0", only: [:dev], runtime: false}, 31 | {:ex_doc, "~> 0.21", only: :dev, runtime: false}, 32 | {:excoveralls, "~> 0.12", only: :test} 33 | ] 34 | end 35 | 36 | defp package() do 37 | [ 38 | description: "An embedded queue and stack abstraction for Elixir on top of CubDB", 39 | files: ["lib", "LICENSE", "mix.exs"], 40 | maintainers: ["Luca Ongaro"], 41 | licenses: ["Apache-2.0"], 42 | links: %{} 43 | ] 44 | end 45 | end 46 | -------------------------------------------------------------------------------- /mix.lock: -------------------------------------------------------------------------------- 1 | %{ 2 | "certifi": {:hex, :certifi, "2.5.1", "867ce347f7c7d78563450a18a6a28a8090331e77fa02380b4a21962a65d36ee5", [:rebar3], [{:parse_trans, "~>3.3", [hex: :parse_trans, repo: "hexpm", optional: false]}], "hexpm"}, 3 | "cubdb": {:hex, :cubdb, "0.17.0", "a0a1ce830bc74b94a2e78aea1c529750028e822ae741752245d7f9bdd10c116f", [:mix], [], "hexpm"}, 4 | "dialyxir": {:hex, :dialyxir, "1.0.0", "6a1fa629f7881a9f5aaf3a78f094b2a51a0357c843871b8bc98824e7342d00a5", [:mix], [{:erlex, ">= 0.2.6", [hex: :erlex, repo: "hexpm", optional: false]}], "hexpm"}, 5 | "earmark": {:hex, :earmark, "1.4.3", "364ca2e9710f6bff494117dbbd53880d84bebb692dafc3a78eb50aa3183f2bfd", [:mix], [], "hexpm"}, 6 | "erlex": {:hex, :erlex, "0.2.6", "c7987d15e899c7a2f34f5420d2a2ea0d659682c06ac607572df55a43753aa12e", [:mix], [], "hexpm"}, 7 | "ex_doc": {:hex, :ex_doc, "0.21.3", "857ec876b35a587c5d9148a2512e952e24c24345552259464b98bfbb883c7b42", [:mix], [{:earmark, "~> 1.4", [hex: :earmark, repo: "hexpm", optional: false]}, {:makeup_elixir, "~> 0.14", [hex: :makeup_elixir, repo: "hexpm", optional: false]}], "hexpm"}, 8 | "excoveralls": {:hex, :excoveralls, "0.12.3", "2142be7cb978a3ae78385487edda6d1aff0e482ffc6123877bb7270a8ffbcfe0", [:mix], [{:hackney, "~> 1.0", [hex: :hackney, repo: "hexpm", optional: false]}, {:jason, "~> 1.0", [hex: :jason, repo: "hexpm", optional: false]}], "hexpm"}, 9 | "hackney": {:hex, :hackney, "1.15.2", "07e33c794f8f8964ee86cebec1a8ed88db5070e52e904b8f12209773c1036085", [:rebar3], [{:certifi, "2.5.1", [hex: :certifi, repo: "hexpm", optional: false]}, {:idna, "6.0.0", [hex: :idna, repo: "hexpm", optional: false]}, {:metrics, "1.0.1", [hex: :metrics, repo: "hexpm", optional: false]}, {:mimerl, "~>1.1", [hex: :mimerl, repo: "hexpm", optional: false]}, {:ssl_verify_fun, "1.1.5", [hex: :ssl_verify_fun, repo: "hexpm", optional: false]}], "hexpm"}, 10 | "idna": {:hex, :idna, "6.0.0", "689c46cbcdf3524c44d5f3dde8001f364cd7608a99556d8fbd8239a5798d4c10", [:rebar3], [{:unicode_util_compat, "0.4.1", [hex: :unicode_util_compat, repo: "hexpm", optional: false]}], "hexpm"}, 11 | "jason": {:hex, :jason, "1.2.0", "10043418c42d2493d0ee212d3fddd25d7ffe484380afad769a0a38795938e448", [:mix], [{:decimal, "~> 1.0", [hex: :decimal, repo: "hexpm", optional: true]}], "hexpm"}, 12 | "makeup": {:hex, :makeup, "1.0.1", "82f332e461dc6c79dbd82fbe2a9c10d48ed07146f0a478286e590c83c52010b5", [:mix], [{:nimble_parsec, "~> 0.5.0", [hex: :nimble_parsec, repo: "hexpm", optional: false]}], "hexpm"}, 13 | "makeup_elixir": {:hex, :makeup_elixir, "0.14.0", "cf8b7c66ad1cff4c14679698d532f0b5d45a3968ffbcbfd590339cb57742f1ae", [:mix], [{:makeup, "~> 1.0", [hex: :makeup, repo: "hexpm", optional: false]}], "hexpm"}, 14 | "metrics": {:hex, :metrics, "1.0.1", "25f094dea2cda98213cecc3aeff09e940299d950904393b2a29d191c346a8486", [:rebar3], [], "hexpm"}, 15 | "mimerl": {:hex, :mimerl, "1.2.0", "67e2d3f571088d5cfd3e550c383094b47159f3eee8ffa08e64106cdf5e981be3", [:rebar3], [], "hexpm"}, 16 | "nimble_parsec": {:hex, :nimble_parsec, "0.5.3", "def21c10a9ed70ce22754fdeea0810dafd53c2db3219a0cd54cf5526377af1c6", [:mix], [], "hexpm"}, 17 | "parse_trans": {:hex, :parse_trans, "3.3.0", "09765507a3c7590a784615cfd421d101aec25098d50b89d7aa1d66646bc571c1", [:rebar3], [], "hexpm"}, 18 | "ssl_verify_fun": {:hex, :ssl_verify_fun, "1.1.5", "6eaf7ad16cb568bb01753dbbd7a95ff8b91c7979482b95f38443fe2c8852a79b", [:make, :mix, :rebar3], [], "hexpm"}, 19 | "unicode_util_compat": {:hex, :unicode_util_compat, "0.4.1", "d869e4c68901dd9531385bb0c8c40444ebf624e60b6962d95952775cac5e90cd", [:rebar3], [], "hexpm"}, 20 | } 21 | -------------------------------------------------------------------------------- /test/cubq_test.exs: -------------------------------------------------------------------------------- 1 | defmodule CubQTest do 2 | use ExUnit.Case, async: true 3 | doctest CubQ 4 | 5 | setup do 6 | tmp_dir = :os.cmd('mktemp -d') |> List.to_string() |> String.trim() |> String.to_charlist() 7 | cubdb_options = [data_dir: tmp_dir, auto_file_sync: false, auto_compact: false] 8 | 9 | {:ok, db} = CubDB.start_link(cubdb_options) 10 | 11 | on_exit(fn -> 12 | with {:ok, files} <- File.ls(tmp_dir) do 13 | for file <- files, do: File.rm(Path.join(tmp_dir, file)) 14 | end 15 | 16 | :ok = File.rmdir(tmp_dir) 17 | end) 18 | 19 | {:ok, db: db} 20 | end 21 | 22 | test "queue operations", %{db: db} do 23 | {:ok, q} = CubQ.start_link(db: db, queue: :my_queue) 24 | {:ok, q2} = CubQ.start_link(db: db, queue: :my_other_queue) 25 | 26 | assert :ok = CubQ.enqueue(q2, "one") 27 | 28 | assert CubQ.dequeue(q) == nil 29 | assert CubQ.peek_first(q) == nil 30 | 31 | assert :ok = CubQ.enqueue(q, :one) 32 | assert :ok = CubQ.enqueue(q, :two) 33 | assert :ok = CubQ.enqueue(q, :three) 34 | 35 | assert {:ok, :one} = CubQ.peek_first(q) 36 | 37 | assert {:ok, :one} = CubQ.dequeue(q) 38 | assert {:ok, :two} = CubQ.dequeue(q) 39 | assert {:ok, :three} = CubQ.peek_first(q) 40 | assert {:ok, :three} = CubQ.dequeue(q) 41 | 42 | assert CubQ.dequeue(q) == nil 43 | assert CubQ.peek_first(q) == nil 44 | 45 | assert {:ok, "one"} = CubQ.peek_first(q2) 46 | end 47 | 48 | test "stack operations", %{db: db} do 49 | {:ok, q} = CubQ.start_link(db: db, queue: :my_queue) 50 | {:ok, q2} = CubQ.start_link(db: db, queue: :my_other_queue) 51 | 52 | assert :ok = CubQ.push(q2, "one") 53 | 54 | assert CubQ.pop(q) == nil 55 | assert CubQ.peek_last(q) == nil 56 | 57 | assert :ok = CubQ.push(q, :one) 58 | assert :ok = CubQ.push(q, :two) 59 | assert :ok = CubQ.push(q, :three) 60 | 61 | assert {:ok, :three} = CubQ.peek_last(q) 62 | 63 | assert {:ok, :three} = CubQ.pop(q) 64 | assert {:ok, :two} = CubQ.pop(q) 65 | assert {:ok, :one} = CubQ.peek_last(q) 66 | assert {:ok, :one} = CubQ.pop(q) 67 | 68 | assert CubQ.pop(q) == nil 69 | assert CubQ.peek_last(q) == nil 70 | 71 | assert {:ok, "one"} = CubQ.peek_last(q2) 72 | end 73 | 74 | test "append and prepend", %{db: db} do 75 | {:ok, q} = CubQ.start_link(db: db, queue: :my_queue) 76 | {:ok, q2} = CubQ.start_link(db: db, queue: :my_other_queue) 77 | 78 | assert :ok = CubQ.enqueue(q2, "one") 79 | 80 | assert :ok = CubQ.append(q, :one) 81 | assert :ok = CubQ.prepend(q, :zero) 82 | assert :ok = CubQ.append(q, :two) 83 | 84 | assert {:ok, :zero} = CubQ.peek_first(q) 85 | assert {:ok, :two} = CubQ.peek_last(q) 86 | 87 | assert {:ok, :zero} = CubQ.dequeue(q) 88 | assert {:ok, :one} = CubQ.dequeue(q) 89 | assert {:ok, :two} = CubQ.dequeue(q) 90 | 91 | assert CubQ.dequeue(q) == nil 92 | assert CubQ.peek_first(q) == nil 93 | 94 | assert {:ok, "one"} = CubQ.peek_first(q2) 95 | end 96 | 97 | test "delete_all/1 deletes all items in the queue leaving other entries unchanged", %{db: db} do 98 | {:ok, q} = CubQ.start_link(db: db, queue: :my_queue) 99 | {:ok, q2} = CubQ.start_link(db: db, queue: :my_other_queue) 100 | 101 | assert :ok = CubQ.enqueue(q2, "one") 102 | 103 | assert :ok = CubQ.enqueue(q, :one) 104 | assert :ok = CubQ.enqueue(q, :two) 105 | assert :ok = CubQ.enqueue(q, :three) 106 | 107 | assert :ok = CubQ.delete_all(q, 2) 108 | assert CubQ.dequeue(q) == nil 109 | 110 | assert :ok = CubQ.delete_all(q) 111 | 112 | assert {:ok, "one"} = CubQ.peek_first(q2) 113 | end 114 | 115 | test "queue operations with explicit ack/nack", %{db: db} do 116 | {:ok, q} = CubQ.start_link(db: db, queue: :my_queue) 117 | {:ok, q2} = CubQ.start_link(db: db, queue: :my_other_queue) 118 | 119 | assert :ok = CubQ.enqueue(q2, "one") 120 | assert :ok = CubQ.enqueue(q2, "two") 121 | assert _ = CubQ.dequeue_ack(q2, 10_000) 122 | 123 | assert :ok = CubQ.enqueue(q, :one) 124 | assert :ok = CubQ.enqueue(q, :two) 125 | assert :ok = CubQ.enqueue(q, :three) 126 | 127 | assert {:ok, :one} = CubQ.peek_first(q) 128 | 129 | assert {:ok, :one, ack_id} = CubQ.dequeue_ack(q) 130 | assert {:ok, :two} = CubQ.peek_first(q) 131 | 132 | assert :ok = CubQ.nack(q, ack_id) 133 | assert {:ok, :one} = CubQ.peek_first(q) 134 | 135 | assert {:ok, :one, ack_id} = CubQ.dequeue_ack(q) 136 | assert {:ok, :two} = CubQ.peek_first(q) 137 | 138 | assert :ok = CubQ.ack(q, ack_id) 139 | assert {:ok, :two} = CubQ.peek_first(q) 140 | 141 | assert {:ok, :two, _ack_id} = CubQ.dequeue_ack(q, 150) 142 | assert {:ok, :three, _ack_id} = CubQ.dequeue_ack(q, 100) 143 | 144 | Process.sleep(1000) 145 | assert {:ok, :two} = CubQ.peek_first(q) 146 | 147 | assert {:ok, "two"} = CubQ.peek_first(q2) 148 | end 149 | 150 | test "stack operations with explicit ack/nack", %{db: db} do 151 | {:ok, q} = CubQ.start_link(db: db, queue: :my_queue) 152 | {:ok, q2} = CubQ.start_link(db: db, queue: :my_other_queue) 153 | 154 | assert :ok = CubQ.push(q2, "one") 155 | assert :ok = CubQ.push(q2, "two") 156 | assert _ = CubQ.pop_ack(q2, 10_000) 157 | 158 | assert :ok = CubQ.push(q, :zero) 159 | assert :ok = CubQ.push(q, :one) 160 | assert :ok = CubQ.push(q, :two) 161 | 162 | assert {:ok, :two} = CubQ.peek_last(q) 163 | 164 | assert {:ok, :two, ack_id} = CubQ.pop_ack(q) 165 | assert {:ok, :one} = CubQ.peek_last(q) 166 | 167 | assert :ok = CubQ.nack(q, ack_id) 168 | assert {:ok, :two} = CubQ.peek_last(q) 169 | 170 | assert {:ok, :two, ack_id} = CubQ.pop_ack(q) 171 | assert {:ok, :one} = CubQ.peek_last(q) 172 | 173 | assert :ok = CubQ.ack(q, ack_id) 174 | assert {:ok, :one} = CubQ.peek_last(q) 175 | 176 | assert {:ok, :one, _ack_id} = CubQ.pop_ack(q, 150) 177 | assert {:ok, :zero, _ack_id} = CubQ.pop_ack(q, 100) 178 | 179 | Process.sleep(1000) 180 | assert {:ok, :one} = CubQ.peek_last(q) 181 | 182 | assert {:ok, "one"} = CubQ.peek_last(q2) 183 | end 184 | end 185 | -------------------------------------------------------------------------------- /test/test_helper.exs: -------------------------------------------------------------------------------- 1 | ExUnit.start() 2 | --------------------------------------------------------------------------------