└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # Introduction to using Redis as a JSON store 2 | 3 | The following is a document that provides information on storing JSON in Redis, and the different strategies that are available with advantages and disadvantages. The examples with utilize node and the Redis client to store JSON. The majority of this information comes from the amazing presentation by [@itamarhaber](https://github.com/itamarhaber) 4 | 5 | ## Why did I create this README? 6 | 7 | To give people an understanding as to the different possibilities in using Redis as a JSON store and showing a working code that allows anyone to jump start their project. I will be querying a json endpoint and saving the json in three different ways: 8 | 9 | **As a key- value, where the value will be the JSON:** 10 | 11 | ``` 12 | SET key value 13 | ``` 14 | 15 | > Advantages 16 | 17 | * Data is store serialized. "BLOB" read/write 18 | 19 | > Disadvantages 20 | 21 | * Element access is impossible - entire bulk must be read 22 | 23 | **As a hash, where the value will be a Javascript Object:** 24 | 25 | ```sh 26 | HMSET key obj 27 | ``` 28 | 29 | > Advantages 30 | 31 | - Elements are accessible in O(1) 32 | 33 | > Disadvantages: 34 | 35 | * No native way to decode/encode to/from JSON, means a client side implementation 36 | 37 | * Only String/Number types 38 | 39 | ​ 40 | 41 | **Using ReJSON, a module created within Redis, that fully supports the JSON data type.** 42 | 43 | ``` 44 | JSON.SET key accessor value 45 | ``` 46 | 47 | > Advantages: 48 | 49 | * Full JSON support 50 | 51 | * Works with any Redis client 52 | 53 | > Disadvantages 54 | 55 | * Serializing the JSON is "expensive"; transform the input JSON to a tree structure internally 56 | 57 | * Higher memory overhead 58 | 59 | ## Requirements 60 | 61 | *Download the ReJSON docker image that provides Redis with the JSON module* 62 | 63 | ``` 64 | docker run --rm -d -p 6379:6379 --name redis-rejson redislabs/rejson:latest 65 | ``` 66 | 67 | * Dependencies 68 | 69 | ```sh 70 | npm install axios redis bluebird 71 | ``` 72 | 73 | * index.js 74 | 75 | ```javascript 76 | const redis = require('redis'); 77 | const bluebird = require("bluebird"); 78 | const axios = require('axios'); 79 | const client = redis.createClient(); 80 | 81 | bluebird.promisifyAll(redis.RedisClient.prototype); 82 | bluebird.promisifyAll(redis.Multi.prototype); 83 | ``` 84 | 85 | Below I will be showing the three different strategies described above: 86 | 87 | * Setting the JSON as a string. I would use this strategy is the data will not change and there is no necessity in changing anything in the JSON 88 | 89 | ```javascript 90 | // Calling a JSON endpoint that contains a list of emoji with descriptions. Saving the return as a string in Redis 91 | axios.get("https://gist.githubusercontent.com/oliveratgithub/0bf11a9aff0d6da7b46f1490f86a71eb/raw/ac8dde8a374066bcbcf44a8296fc0522c7392244/emojis.json").then((response) => { 92 | client.setAsync("emojis", JSON.stringify(response.data)); 93 | }); 94 | ``` 95 | 96 | * Setting the JSON as a hash. The problem with using this strategy is that only String/Number types are supported. If you have a null, or array with null values, you will be responsible with processing the data before saving in Redis. With simple json, this may not be an issue, but if your JSON has many nested levels, the cleaning process can become complex; where you will end up having to use [libraries](https://github.com/hughsk/flat) to flatten the JSON, and then unflatten when retrieving from Redis. 97 | * Below we can see an example of a more complex JSON with a nested structure and null values. 98 | 99 | ```json 100 | exampleJSON = { "address": 101 | {"location": null, "person": [{"name": "Joe Smith", "age": null}, {"name": "John Doe", "age": null}]} 102 | } 103 | 104 | client.hmsetAsync("emojis-hash", exampleJSON); 105 | ``` 106 | 107 | > Node Redis will provide the following message. So all validation and formatting code need to be added client-side 108 | 109 | !["warning provided by node redis"](https://api.monosnap.com/rpc/file/download?id=H6Ww0xPuTfdRSbKetaqHBu0brVxRpF) 110 | 111 | 112 | 113 | 114 | 115 | * Using **ReJSON** to set the JSON object 116 | 117 | ```javascript 118 | axios.get("https://gist.githubusercontent.com/oliveratgithub/0bf11a9aff0d6da7b46f1490f86a71eb/raw/ac8dde8a374066bcbcf44a8296fc0522c7392244/emojis.json").then((response) => { 119 | //Set the JSON 120 | client.sendCommandAsync("JSON.SET", ["emoji-json", "." , JSON.stringify(response.data)]).then(reply=> console.log(reply)) 121 | }); 122 | ``` 123 | 124 | > Using ReJSON, you can now query by keys, and you can also modify the object without having to read it back into Node 125 | 126 | ```javascript 127 | //Query of the JSON 128 | client.sendCommandAsync("JSON.GET", ["emoji-json", "."]).then(reply => console.log(JSON.parse(reply))) 129 | 130 | //You can also query specific keys 131 | client.sendCommandAsync("JSON.GET", ["emoji-json", ".emojis[0]"]).then(reply => console.log(JSON.parse(reply))) 132 | 133 | 134 | //With ReJSON you can also modify the JSON structure 135 | client.sendCommandAsync("JSON.SET", ["emoji-json", ".emojis[0]", '{"name":"Example"}']).then(reply => console.log(reply)) 136 | ``` 137 | 138 | * Using npm libraries - rejson 139 | There are also npm libraries that allow for easier integration with rejson by providing semantically easier API than utilizing `sendCommandAsync`. One such library is [rejson](https://github.com/stockholmux/node_redis-rejson). 140 | Below is an example of how to use the library to set json and get specific json keys from redis, allowing you to pull specific keys without having to read the entire json in memory. 141 | 142 | ```sh 143 | ╭─josecolella at MacBookAir in /redis-with-json-introduction 144 | ╰─λ node 0 < 21:22:42 145 | > const redis = require("redis"); 146 | undefined 147 | > const rejson = require("redis-rejson") 148 | undefined 149 | > rejson(redis) 150 | undefined 151 | > const client = redis.createClient() 152 | undefined 153 | > client.json_set("message", ".", JSON.stringify({key: "Hello Redis!!"})) 154 | true 155 | > client.json_get("message", ".key", (err, payload) => { 156 | ... console.log(payload); 157 | ... }) 158 | true 159 | > "Hello Redis!!" 160 | 161 | ``` 162 | 163 | ## Conclusions 164 | 165 | If your application json does not change over time, and you require a simple caching of JSON, the string key value store strategy. 166 | 167 | If you application json is not overly complex with nested arrays, and potential null values, the hash strategy is probably the best way to go. With access at constant time, this strategy wins out. 168 | 169 | Finally, if your application json changes and you require support of complex JSON objects, where you can modify and access specific keys, ReJSON is the way to go. 170 | 171 | 172 | 173 | ### Contributing 174 | 175 | If there is anything that can be improved in this document, or if you feel that something does not make sense, make sure to open an issue. This document was created to help people get started with JSON in redis, with code examples. 176 | 177 | :octocat: 178 | 179 | ## Additional Resources 180 | 181 | 182 | 183 | - RedisLab Video: https://youtu.be/NLRbq2FtcIk 184 | 185 | - RedisLab blog post on Redis as a JSON store: https://redislabs.com/blog/redis-as-a-json-store/ 186 | 187 | ​ 188 | --------------------------------------------------------------------------------