├── .gems ├── test ├── helper.rb └── all.rb ├── makefile ├── stal.gemspec ├── lib └── stal.rb ├── CONTRIBUTING ├── LICENSE ├── data └── stal.lua └── README.md /.gems: -------------------------------------------------------------------------------- 1 | cutest -v 1.2.2 2 | redic -v 1.5.0 3 | -------------------------------------------------------------------------------- /test/helper.rb: -------------------------------------------------------------------------------- 1 | require_relative "../lib/stal" 2 | -------------------------------------------------------------------------------- /makefile: -------------------------------------------------------------------------------- 1 | .PHONY: test 2 | 3 | test: 4 | cutest -r ./test/helper.rb ./test/*.rb 5 | -------------------------------------------------------------------------------- /stal.gemspec: -------------------------------------------------------------------------------- 1 | Gem::Specification.new do |s| 2 | s.name = "stal" 3 | s.version = "0.3.0" 4 | s.summary = %{Set algebra solver for Redis} 5 | s.description = %Q{Stal receives s-expressions and resolves the set operations in Redis} 6 | s.authors = ["Michel Martens"] 7 | s.email = ["michel@soveran.com"] 8 | s.homepage = "https://github.com/soveran/stal" 9 | s.license = "MIT" 10 | 11 | s.files = `git ls-files`.split("\n") 12 | 13 | s.add_dependency "redic", "~> 1.5" 14 | s.add_development_dependency "cutest", "~> 0" 15 | end 16 | -------------------------------------------------------------------------------- /lib/stal.rb: -------------------------------------------------------------------------------- 1 | # encoding: UTF-8 2 | 3 | require "json" 4 | require "redic" 5 | 6 | module Stal 7 | LUA = File.expand_path("../../data/stal.lua", __FILE__) 8 | SHA = "4bd605bfee5f1e809089c5f98d10fab8aec38bd3" 9 | 10 | # Evaluate expression `expr` in the Redis client `c`. 11 | def self.solve(c, expr) 12 | begin 13 | c.call!("EVALSHA", SHA, 0, JSON.dump(expr)) 14 | rescue RuntimeError 15 | if $!.message["NOSCRIPT"] 16 | c.call!("SCRIPT", "LOAD", File.read(LUA)) 17 | c.call!("EVALSHA", SHA, 0, JSON.dump(expr)) 18 | else 19 | raise $! 20 | end 21 | end 22 | end 23 | end 24 | -------------------------------------------------------------------------------- /test/all.rb: -------------------------------------------------------------------------------- 1 | require_relative "helper" 2 | 3 | setup do 4 | Redic.new.tap do |c| 5 | c.call("FLUSHDB") 6 | c.call("SCRIPT", "FLUSH") 7 | c.call("SADD", "foo", "a", "b", "c") 8 | c.call("SADD", "bar", "b", "c", "d") 9 | c.call("SADD", "baz", "c", "d", "e") 10 | c.call("SADD", "qux", "x", "y", "z") 11 | end 12 | end 13 | 14 | test do |c| 15 | 16 | # Example expression 17 | expr = ["SUNION", "qux", ["SDIFF", ["SINTER", "foo", "bar"], "baz"]] 18 | 19 | assert_equal ["b", "x", "y", "z"], Stal.solve(c, expr).sort 20 | 21 | # Commands without sub expressions also work 22 | expr = ["SINTER", "foo", "bar"] 23 | 24 | assert_equal ["b", "c"], Stal.solve(c, expr).sort 25 | 26 | # Verify there's no keyspace pollution 27 | assert_equal ["bar", "baz", "foo", "qux"], c.call("KEYS", "*").sort 28 | 29 | expr = ["SCARD", ["SINTER", "foo", "bar"]] 30 | end 31 | -------------------------------------------------------------------------------- /CONTRIBUTING: -------------------------------------------------------------------------------- 1 | This code tries to solve a particular problem with a very simple 2 | implementation. We try to keep the code to a minimum while making 3 | it as clear as possible. The design is very likely finished, and 4 | if some feature is missing it is possible that it was left out on 5 | purpose. That said, new usage patterns may arise, and when that 6 | happens we are ready to adapt if necessary. 7 | 8 | A good first step for contributing is to meet us on IRC and discuss 9 | ideas. We spend a lot of time on #lesscode at freenode, always ready 10 | to talk about code and simplicity. If connecting to IRC is not an 11 | option, you can create an issue explaining the proposed change and 12 | a use case. We pay a lot of attention to use cases, because our 13 | goal is to keep the code base simple. Usually the result of a 14 | conversation is the creation of a different tool. 15 | 16 | Please don't start the conversation with a pull request. The code 17 | should come at last, and even though it may help to convey an idea, 18 | more often than not it draws the attention to a particular 19 | implementation. 20 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2015 Michel Martens 2 | 3 | Permission is hereby granted, free of charge, to any person obtaining a copy 4 | of this software and associated documentation files (the "Software"), to deal 5 | in the Software without restriction, including without limitation the rights 6 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 7 | copies of the Software, and to permit persons to whom the Software is 8 | furnished to do so, subject to the following conditions: 9 | 10 | The above copyright notice and this permission notice shall be included in 11 | all copies or substantial portions of the Software. 12 | 13 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 14 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 15 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 16 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 17 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 18 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 19 | THE SOFTWARE. 20 | -------------------------------------------------------------------------------- /data/stal.lua: -------------------------------------------------------------------------------- 1 | -- Copyright (c) 2016 Michel Martens 2 | 3 | local expr = cjson.decode(ARGV[1]) 4 | 5 | local tr = { 6 | SDIFF = "SDIFFSTORE", 7 | SINTER = "SINTERSTORE", 8 | SUNION = "SUNIONSTORE", 9 | ZINTER = "ZINTERSTORE", 10 | ZUNION = "ZUNIONSTORE", 11 | } 12 | 13 | local function append(t1, t2) 14 | for _, item in ipairs(t2) do 15 | table.insert(t1, item) 16 | end 17 | end 18 | 19 | local function map(t, f) 20 | local nt = {} 21 | 22 | for k, v in pairs(t) do 23 | nt[k] = f(v) 24 | end 25 | 26 | return nt 27 | end 28 | 29 | local compile, convert 30 | 31 | function compile(expr, ids, ops) 32 | return map(expr, function(v) 33 | if (type(v) == "table") then 34 | return convert(v, ids, ops) 35 | else 36 | return v 37 | end 38 | end) 39 | end 40 | 41 | function convert(expr, ids, ops) 42 | local tail = {unpack(expr)} 43 | local head = table.remove(tail, 1) 44 | 45 | -- Key where partial results will be stored 46 | local id = "stal:" .. #ids 47 | 48 | -- Keep a reference to clean it up later 49 | table.insert(ids, id) 50 | 51 | -- Translate into command and destination key 52 | local op = {tr[head] or head, id} 53 | 54 | -- Compile the rest recursively 55 | append(op, compile(tail, ids, ops)) 56 | 57 | -- Append the outermost operation 58 | table.insert(ops, op) 59 | 60 | return id 61 | end 62 | 63 | local function solve(expr) 64 | local ids = {} 65 | local ops = {} 66 | local res = nil 67 | 68 | table.insert(ops, compile(expr, ids, ops)) 69 | 70 | if (#ops == 1) then 71 | return redis.call(unpack(ops[1])) 72 | else 73 | for _, op in ipairs(ops) do 74 | if (#op > 1) then 75 | res = redis.call(unpack(op)) 76 | end 77 | end 78 | 79 | redis.call("DEL", unpack(ids)) 80 | 81 | return res 82 | end 83 | end 84 | 85 | if redis.replicate_commands then 86 | redis.replicate_commands() 87 | redis.set_repl(redis.REPL_NONE) 88 | end 89 | 90 | return solve(expr) 91 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Stal 2 | ==== 3 | 4 | Set algebra solver for Redis. 5 | 6 | Description 7 | ----------- 8 | 9 | `Stal` receives an array with an s-expression composed of commands 10 | and key names and resolves the set operations in [Redis][redis]. 11 | 12 | Community 13 | --------- 14 | 15 | Meet us on IRC: [#lesscode](irc://chat.freenode.net/#lesscode) on 16 | [freenode.net](http://freenode.net/). 17 | 18 | Getting started 19 | --------------- 20 | 21 | Install [Redis][redis]. On most platforms it's as easy as grabbing 22 | the sources, running make and then putting the `redis-server` binary 23 | in the PATH. 24 | 25 | Once you have it installed, you can execute `redis-server` and it 26 | will run on `localhost:6379` by default. Check the `redis.conf` 27 | file that comes with the sources if you want to change some settings. 28 | 29 | Usage 30 | ----- 31 | 32 | `Stal` requires a [Redic][redic] compatible client. To make things 33 | easier, `redic` is listed as a runtime dependency so the examples 34 | in this document will work. 35 | 36 | ```ruby 37 | require "stal" 38 | 39 | # Connect the client to the default host 40 | redis = Redic.new 41 | 42 | # Use the Redis client to populate some sets 43 | redis.call("SADD", "foo", "a", "b", "c") 44 | redis.call("SADD", "bar", "b", "c", "d") 45 | redis.call("SADD", "baz", "c", "d", "e") 46 | redis.call("SADD", "qux", "x", "y", "z") 47 | ``` 48 | 49 | Now we can perform some set operations with `Stal`: 50 | 51 | ```ruby 52 | expr = ["SUNION", "qux", ["SDIFF", ["SINTER", "foo", "bar"], "baz"]] 53 | 54 | Stal.solve(redis, expr) 55 | #=> ["b", "x", "y", "z"] 56 | ``` 57 | 58 | `Stal` translates the internal calls to `SUNION`, `SDIFF` and 59 | `SINTER` into `SDIFFSTORE`, `SINTERSTORE` and `SUNIONSTORE` to 60 | perform the underlying operations, and it takes care of generating 61 | and deleting any temporary keys. 62 | 63 | For more information, refer to the repository of the [Stal][stal] 64 | script. 65 | 66 | Installation 67 | ------------ 68 | 69 | ``` 70 | $ gem install stal 71 | ``` 72 | 73 | [redis]: http://redis.io 74 | [redic]: https://github.com/amakawa/redic 75 | [stal]: https://github.com/soveran/stal --------------------------------------------------------------------------------