Rockstar Games has another hard-boiled hit on its hands.
"},{"type":"text","value":"Old West adventure game "Red Dead Redemption 2," which landed Friday, lassoed $725 million in sales worldwide in its first three days.
"},{"type":"text","value":"That places the massive explorable open-world game as the No. 2 game out of the gate, just behind Rockstar's "Grand Theft Auto V," the biggest seller of all time, which took in $1 billion in its first three days when it launched on Sept. 17, 2013.
"},{"type":"text","value":""GTA V" has gone on to make more money than any other single game title, selling nearly 100 million copies and reaping $6 billion in revenue, according to MarketWatch.
"},{"type":"text","value":"The three-day start makes "Red Dead Redemption 2" the single-biggest opening weekend in "the history of entertainment," Rockstar said in a press release detailing the game's achievements. That means the three-day sales for the game, prices of which start at $59.99 (rated Mature for those 17-up), surpasses opening weekends for blockbuster movies such as "Avengers: Infinity War" and "Star Wars: The Force Awakens."
"},{"type":"text","value":"More: 'Red Dead Redemption 2': First impressions from once upon a time in the West
\n
\nMore: Sony lists the 20 games coming to PlayStation Classic retro video game console
"Red Dead Redemption 2" also tallied the biggest full game sales marks for one and for three days on Sony's PlayStation Network, Rockstar said. It was also the most preordered game on Sony's online network.
"},{"type":"text","value":"Reviews for the game have rated it among the best ever. It earned a 97 on Metacritic, earning it a tie for No. 6 all-time, along with games such as "Super Mario Galaxy" and "GTA V."
"},{"type":"text","value":"Piper Jaffray & Co. senior research analyst Michael Olson estimated Rockstar sold about 11 million copies in its first three days. That means Olson's initial estimate of Rockstar selling 15.5 million copies of "Red Dead 2" in its fiscal year, which ends in March 2019, "appears conservative," he said in a note to investors Tuesday.
"},{"type":"text","value":""Clearly, with RDR2 first weekend sell-through exceeding CoD: Black Ops 4, it now appears RDR2 estimates may have been overly conservative," Olson wrote.
"},{"type":"text","value":"Shares of Rockstar’s parent company Take-Two Interactive (TTWO) rose about 8 percent in early trading Tuesday to $120.55.
"},{"type":"asset","value":"1765924002"},{"type":"text","value":"Follow USA TODAY reporter Mike Snider on Twitter: @MikeSnider.
"}],"mobile":[{"type":"asset","value":"1763879002"},{"type":"text","value":"Rockstar Games has another hard-boiled hit on its hands.
"},{"type":"text","value":"Old West adventure game "Red Dead Redemption 2," which landed Friday, lassoed $725 million in sales worldwide in its first three days.
"},{"type":"text","value":"That places the massive explorable open-world game as the No. 2 game out of the gate, just behind Rockstar's "Grand Theft Auto V," the biggest seller of all time, which took in $1 billion in its first three days when it launched on Sept. 17, 2013.
"},{"type":"text","value":""GTA V" has gone on to make more money than any other single game title, selling nearly 100 million copies and reaping $6 billion in revenue, according to MarketWatch.
"},{"type":"text","value":"The three-day start makes "Red Dead Redemption 2" the single-biggest opening weekend in "the history of entertainment," Rockstar said in a press release detailing the game's achievements. That means the three-day sales for the game, prices of which start at $59.99 (rated Mature for those 17-up), surpasses opening weekends for blockbuster movies such as "Avengers: Infinity War" and "Star Wars: The Force Awakens."
"},{"type":"text","value":"More: 'Red Dead Redemption 2': First impressions from once upon a time in the West
\n
\nMore: Sony lists the 20 games coming to PlayStation Classic retro video game console
"Red Dead Redemption 2" also tallied the biggest full game sales marks for one and for three days on Sony's PlayStation Network, Rockstar said. It was also the most preordered game on Sony's online network.
"},{"type":"text","value":"Reviews for the game have rated it among the best ever. It earned a 97 on Metacritic, earning it a tie for No. 6 all-time, along with games such as "Super Mario Galaxy" and "GTA V."
"},{"type":"text","value":"Piper Jaffray & Co. senior research analyst Michael Olson estimated Rockstar sold about 11 million copies in its first three days. That means Olson's initial estimate of Rockstar selling 15.5 million copies of "Red Dead 2" in its fiscal year, which ends in March 2019, "appears conservative," he said in a note to investors Tuesday.
"},{"type":"text","value":""Clearly, with RDR2 first weekend sell-through exceeding CoD: Black Ops 4, it now appears RDR2 estimates may have been overly conservative," Olson wrote.
"},{"type":"text","value":"Shares of Rockstar’s parent company Take-Two Interactive (TTWO) rose about 8 percent in early trading Tuesday to $120.55.
"},{"type":"asset","value":"1765924002"},{"type":"text","value":"Follow USA TODAY reporter Mike Snider on Twitter: @MikeSnider.
"}]},"fullText":"Rockstar Games has another hard-boiled hit on its hands.
\n\nOld West adventure game "Red Dead Redemption 2," which landed Friday, lassoed $725 million in sales worldwide in its first three days.
\n\nThat places the massive explorable open-world game as the No. 2 game out of the gate, just behind Rockstar's "Grand Theft Auto V," the biggest seller of all time, which took in $1 billion in its first three days when it launched on Sept. 17, 2013.
\n\n"GTA V" has gone on to make more money than any other single game title, selling nearly 100 million copies and reaping $6 billion in revenue, according to MarketWatch.
\n\nThe three-day start makes "Red Dead Redemption 2" the single-biggest opening weekend in "the history of entertainment," Rockstar said in a press release detailing the game's achievements. That means the three-day sales for the game, prices of which start at $59.99 (rated Mature for those 17-up), surpasses opening weekends for blockbuster movies such as "Avengers: Infinity War" and "Star Wars: The Force Awakens."
\n\nMore: 'Red Dead Redemption 2': First impressions from once upon a time in the West
\n
\nMore: Sony lists the 20 games coming to PlayStation Classic retro video game console
"Red Dead Redemption 2" also tallied the biggest full game sales marks for one and for three days on Sony's PlayStation Network, Rockstar said. It was also the most preordered game on Sony's online network.
\n\nReviews for the game have rated it among the best ever. It earned a 97 on Metacritic, earning it a tie for No. 6 all-time, along with games such as "Super Mario Galaxy" and "GTA V."
\n\nPiper Jaffray & Co. senior research analyst Michael Olson estimated Rockstar sold about 11 million copies in its first three days. That means Olson's initial estimate of Rockstar selling 15.5 million copies of "Red Dead 2" in its fiscal year, which ends in March 2019, "appears conservative," he said in a note to investors Tuesday.
\n\n"Clearly, with RDR2 first weekend sell-through exceeding CoD: Black Ops 4, it now appears RDR2 estimates may have been overly conservative," Olson wrote.
\n\nShares of Rockstar’s parent company Take-Two Interactive (TTWO) rose about 8 percent in early trading Tuesday to $120.55.
\n\nFollow USA TODAY reporter Mike Snider on Twitter: @MikeSnider.
\n","layoutPriorityAssetId":"1763879002","seoTitle":"Red Dead Redemption 2 makes $725 million in debut for Rockstar Games","type":"text","versionHash":"fe60306b2e7574a8d65e690753deb666"}`) 32 | _ = jsonparser.Delete(data1, "body") 33 | } 34 | } 35 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [](https://goreportcard.com/report/github.com/buger/jsonparser)  2 | # Alternative JSON parser for Go (10x times faster standard library) 3 | 4 | It does not require you to know the structure of the payload (eg. create structs), and allows accessing fields by providing the path to them. It is up to **10 times faster** than standard `encoding/json` package (depending on payload size and usage), **allocates no memory**. See benchmarks below. 5 | 6 | ## Rationale 7 | Originally I made this for a project that relies on a lot of 3rd party APIs that can be unpredictable and complex. 8 | I love simplicity and prefer to avoid external dependecies. `encoding/json` requires you to know exactly your data structures, or if you prefer to use `map[string]interface{}` instead, it will be very slow and hard to manage. 9 | I investigated what's on the market and found that most libraries are just wrappers around `encoding/json`, there is few options with own parsers (`ffjson`, `easyjson`), but they still requires you to create data structures. 10 | 11 | 12 | Goal of this project is to push JSON parser to the performance limits and not sacrifice with compliance and developer user experience. 13 | 14 | ## Example 15 | For the given JSON our goal is to extract the user's full name, number of github followers and avatar. 16 | 17 | ```go 18 | import "github.com/buger/jsonparser" 19 | 20 | ... 21 | 22 | data := []byte(`{ 23 | "person": { 24 | "name": { 25 | "first": "Leonid", 26 | "last": "Bugaev", 27 | "fullName": "Leonid Bugaev" 28 | }, 29 | "github": { 30 | "handle": "buger", 31 | "followers": 109 32 | }, 33 | "avatars": [ 34 | { "url": "https://avatars1.githubusercontent.com/u/14009?v=3&s=460", "type": "thumbnail" } 35 | ] 36 | }, 37 | "company": { 38 | "name": "Acme" 39 | } 40 | }`) 41 | 42 | // You can specify key path by providing arguments to Get function 43 | jsonparser.Get(data, "person", "name", "fullName") 44 | 45 | // There is `GetInt` and `GetBoolean` helpers if you exactly know key data type 46 | jsonparser.GetInt(data, "person", "github", "followers") 47 | 48 | // When you try to get object, it will return you []byte slice pointer to data containing it 49 | // In `company` it will be `{"name": "Acme"}` 50 | jsonparser.Get(data, "company") 51 | 52 | // If the key doesn't exist it will throw an error 53 | var size int64 54 | if value, err := jsonparser.GetInt(data, "company", "size"); err == nil { 55 | size = value 56 | } 57 | 58 | // You can use `ArrayEach` helper to iterate items [item1, item2 .... itemN] 59 | jsonparser.ArrayEach(data, func(value []byte, dataType jsonparser.ValueType, offset int, err error) { 60 | fmt.Println(jsonparser.Get(value, "url")) 61 | }, "person", "avatars") 62 | 63 | // Or use can access fields by index! 64 | jsonparser.GetString(data, "person", "avatars", "[0]", "url") 65 | 66 | // You can use `ObjectEach` helper to iterate objects { "key1":object1, "key2":object2, .... "keyN":objectN } 67 | jsonparser.ObjectEach(data, func(key []byte, value []byte, dataType jsonparser.ValueType, offset int) error { 68 | fmt.Printf("Key: '%s'\n Value: '%s'\n Type: %s\n", string(key), string(value), dataType) 69 | return nil 70 | }, "person", "name") 71 | 72 | // The most efficient way to extract multiple keys is `EachKey` 73 | 74 | paths := [][]string{ 75 | []string{"person", "name", "fullName"}, 76 | []string{"person", "avatars", "[0]", "url"}, 77 | []string{"company", "url"}, 78 | } 79 | jsonparser.EachKey(data, func(idx int, value []byte, vt jsonparser.ValueType, err error){ 80 | switch idx { 81 | case 0: // []string{"person", "name", "fullName"} 82 | ... 83 | case 1: // []string{"person", "avatars", "[0]", "url"} 84 | ... 85 | case 2: // []string{"company", "url"}, 86 | ... 87 | } 88 | }, paths...) 89 | 90 | // For more information see docs below 91 | ``` 92 | 93 | ## Reference 94 | 95 | Library API is really simple. You just need the `Get` method to perform any operation. The rest is just helpers around it. 96 | 97 | You also can view API at [godoc.org](https://godoc.org/github.com/buger/jsonparser) 98 | 99 | 100 | ### **`Get`** 101 | ```go 102 | func Get(data []byte, keys ...string) (value []byte, dataType jsonparser.ValueType, offset int, err error) 103 | ``` 104 | Receives data structure, and key path to extract value from. 105 | 106 | Returns: 107 | * `value` - Pointer to original data structure containing key value, or just empty slice if nothing found or error 108 | * `dataType` - Can be: `NotExist`, `String`, `Number`, `Object`, `Array`, `Boolean` or `Null` 109 | * `offset` - Offset from provided data structure where key value ends. Used mostly internally, for example for `ArrayEach` helper. 110 | * `err` - If the key is not found or any other parsing issue, it should return error. If key not found it also sets `dataType` to `NotExist` 111 | 112 | Accepts multiple keys to specify path to JSON value (in case of quering nested structures). 113 | If no keys are provided it will try to extract the closest JSON value (simple ones or object/array), useful for reading streams or arrays, see `ArrayEach` implementation. 114 | 115 | Note that keys can be an array indexes: `jsonparser.GetInt("person", "avatars", "[0]", "url")`, pretty cool, yeah? 116 | 117 | ### **`GetString`** 118 | ```go 119 | func GetString(data []byte, keys ...string) (val string, err error) 120 | ``` 121 | Returns strings properly handing escaped and unicode characters. Note that this will cause additional memory allocations. 122 | 123 | ### **`GetUnsafeString`** 124 | If you need string in your app, and ready to sacrifice with support of escaped symbols in favor of speed. It returns string mapped to existing byte slice memory, without any allocations: 125 | ```go 126 | s, _, := jsonparser.GetUnsafeString(data, "person", "name", "title") 127 | switch s { 128 | case 'CEO': 129 | ... 130 | case 'Engineer' 131 | ... 132 | ... 133 | } 134 | ``` 135 | Note that `unsafe` here means that your string will exist until GC will free underlying byte slice, for most of cases it means that you can use this string only in current context, and should not pass it anywhere externally: through channels or any other way. 136 | 137 | 138 | ### **`GetBoolean`**, **`GetInt`** and **`GetFloat`** 139 | ```go 140 | func GetBoolean(data []byte, keys ...string) (val bool, err error) 141 | 142 | func GetFloat(data []byte, keys ...string) (val float64, err error) 143 | 144 | func GetInt(data []byte, keys ...string) (val int64, err error) 145 | ``` 146 | If you know the key type, you can use the helpers above. 147 | If key data type do not match, it will return error. 148 | 149 | ### **`ArrayEach`** 150 | ```go 151 | func ArrayEach(data []byte, cb func(value []byte, dataType jsonparser.ValueType, offset int, err error), keys ...string) 152 | ``` 153 | Needed for iterating arrays, accepts a callback function with the same return arguments as `Get`. 154 | 155 | ### **`ObjectEach`** 156 | ```go 157 | func ObjectEach(data []byte, callback func(key []byte, value []byte, dataType ValueType, offset int) error, keys ...string) (err error) 158 | ``` 159 | Needed for iterating object, accepts a callback function. Example: 160 | ```go 161 | var handler func([]byte, []byte, jsonparser.ValueType, int) error 162 | handler = func(key []byte, value []byte, dataType jsonparser.ValueType, offset int) error { 163 | //do stuff here 164 | } 165 | jsonparser.ObjectEach(myJson, handler) 166 | ``` 167 | 168 | 169 | ### **`EachKey`** 170 | ```go 171 | func EachKey(data []byte, cb func(idx int, value []byte, dataType jsonparser.ValueType, err error), paths ...[]string) 172 | ``` 173 | When you need to read multiple keys, and you do not afraid of low-level API `EachKey` is your friend. It read payload only single time, and calls callback function once path is found. For example when you call multiple times `Get`, it has to process payload multiple times, each time you call it. Depending on payload `EachKey` can be multiple times faster than `Get`. Path can use nested keys as well! 174 | 175 | ```go 176 | paths := [][]string{ 177 | []string{"uuid"}, 178 | []string{"tz"}, 179 | []string{"ua"}, 180 | []string{"st"}, 181 | } 182 | var data SmallPayload 183 | 184 | jsonparser.EachKey(smallFixture, func(idx int, value []byte, vt jsonparser.ValueType, err error){ 185 | switch idx { 186 | case 0: 187 | data.Uuid, _ = value 188 | case 1: 189 | v, _ := jsonparser.ParseInt(value) 190 | data.Tz = int(v) 191 | case 2: 192 | data.Ua, _ = value 193 | case 3: 194 | v, _ := jsonparser.ParseInt(value) 195 | data.St = int(v) 196 | } 197 | }, paths...) 198 | ``` 199 | 200 | ### **`Set`** 201 | ```go 202 | func Set(data []byte, setValue []byte, keys ...string) (value []byte, err error) 203 | ``` 204 | Receives existing data structure, key path to set, and value to set at that key. *This functionality is experimental.* 205 | 206 | Returns: 207 | * `value` - Pointer to original data structure with updated or added key value. 208 | * `err` - If any parsing issue, it should return error. 209 | 210 | Accepts multiple keys to specify path to JSON value (in case of updating or creating nested structures). 211 | 212 | Note that keys can be an array indexes: `jsonparser.Set(data, []byte("http://github.com"), "person", "avatars", "[0]", "url")` 213 | 214 | ### **`Delete`** 215 | ```go 216 | func Delete(data []byte, keys ...string) value []byte 217 | ``` 218 | Receives existing data structure, and key path to delete. *This functionality is experimental.* 219 | 220 | Returns: 221 | * `value` - Pointer to original data structure with key path deleted if it can be found. If there is no key path, then the whole data structure is deleted. 222 | 223 | Accepts multiple keys to specify path to JSON value (in case of updating or creating nested structures). 224 | 225 | Note that keys can be an array indexes: `jsonparser.Delete(data, "person", "avatars", "[0]", "url")` 226 | 227 | 228 | ## What makes it so fast? 229 | * It does not rely on `encoding/json`, `reflection` or `interface{}`, the only real package dependency is `bytes`. 230 | * Operates with JSON payload on byte level, providing you pointers to the original data structure: no memory allocation. 231 | * No automatic type conversions, by default everything is a []byte, but it provides you value type, so you can convert by yourself (there is few helpers included). 232 | * Does not parse full record, only keys you specified 233 | 234 | 235 | ## Benchmarks 236 | 237 | There are 3 benchmark types, trying to simulate real-life usage for small, medium and large JSON payloads. 238 | For each metric, the lower value is better. Time/op is in nanoseconds. Values better than standard encoding/json marked as bold text. 239 | Benchmarks run on standard Linode 1024 box. 240 | 241 | Compared libraries: 242 | * https://golang.org/pkg/encoding/json 243 | * https://github.com/Jeffail/gabs 244 | * https://github.com/a8m/djson 245 | * https://github.com/bitly/go-simplejson 246 | * https://github.com/antonholmquist/jason 247 | * https://github.com/mreiferson/go-ujson 248 | * https://github.com/ugorji/go/codec 249 | * https://github.com/pquerna/ffjson 250 | * https://github.com/mailru/easyjson 251 | * https://github.com/buger/jsonparser 252 | 253 | #### TLDR 254 | If you want to skip next sections we have 2 winner: `jsonparser` and `easyjson`. 255 | `jsonparser` is up to 10 times faster than standard `encoding/json` package (depending on payload size and usage), and almost infinitely (literally) better in memory consumption because it operates with data on byte level, and provide direct slice pointers. 256 | `easyjson` wins in CPU in medium tests and frankly i'm impressed with this package: it is remarkable results considering that it is almost drop-in replacement for `encoding/json` (require some code generation). 257 | 258 | It's hard to fully compare `jsonparser` and `easyjson` (or `ffson`), they a true parsers and fully process record, unlike `jsonparser` which parse only keys you specified. 259 | 260 | If you searching for replacement of `encoding/json` while keeping structs, `easyjson` is an amazing choice. If you want to process dynamic JSON, have memory constrains, or more control over your data you should try `jsonparser`. 261 | 262 | `jsonparser` performance heavily depends on usage, and it works best when you do not need to process full record, only some keys. The more calls you need to make, the slower it will be, in contrast `easyjson` (or `ffjson`, `encoding/json`) parser record only 1 time, and then you can make as many calls as you want. 263 | 264 | With great power comes great responsibility! :) 265 | 266 | 267 | #### Small payload 268 | 269 | Each test processes 190 bytes of http log as a JSON record. 270 | It should read multiple fields. 271 | https://github.com/buger/jsonparser/blob/master/benchmark/benchmark_small_payload_test.go 272 | 273 | Library | time/op | bytes/op | allocs/op 274 | ------ | ------- | -------- | ------- 275 | encoding/json struct | 7879 | 880 | 18 276 | encoding/json interface{} | 8946 | 1521 | 38 277 | Jeffail/gabs | 10053 | 1649 | 46 278 | bitly/go-simplejson | 10128 | 2241 | 36 279 | antonholmquist/jason | 27152 | 7237 | 101 280 | github.com/ugorji/go/codec | 8806 | 2176 | 31 281 | mreiferson/go-ujson | **7008** | **1409** | 37 282 | a8m/djson | 3862 | 1249 | 30 283 | pquerna/ffjson | **3769** | **624** | **15** 284 | mailru/easyjson | **2002** | **192** | **9** 285 | buger/jsonparser | **1367** | **0** | **0** 286 | buger/jsonparser (EachKey API) | **809** | **0** | **0** 287 | 288 | Winners are ffjson, easyjson and jsonparser, where jsonparser is up to 9.8x faster than encoding/json and 4.6x faster than ffjson, and slightly faster than easyjson. 289 | If you look at memory allocation, jsonparser has no rivals, as it makes no data copy and operates with raw []byte structures and pointers to it. 290 | 291 | #### Medium payload 292 | 293 | Each test processes a 2.4kb JSON record (based on Clearbit API). 294 | It should read multiple nested fields and 1 array. 295 | 296 | https://github.com/buger/jsonparser/blob/master/benchmark/benchmark_medium_payload_test.go 297 | 298 | | Library | time/op | bytes/op | allocs/op | 299 | | ------- | ------- | -------- | --------- | 300 | | encoding/json struct | 57749 | 1336 | 29 | 301 | | encoding/json interface{} | 79297 | 10627 | 215 | 302 | | Jeffail/gabs | 83807 | 11202 | 235 | 303 | | bitly/go-simplejson | 88187 | 17187 | 220 | 304 | | antonholmquist/jason | 94099 | 19013 | 247 | 305 | | github.com/ugorji/go/codec | 114719 | 6712 | 152 | 306 | | mreiferson/go-ujson | **56972** | 11547 | 270 | 307 | | a8m/djson | 28525 | 10196 | 198 | 308 | | pquerna/ffjson | **20298** | **856** | **20** | 309 | | mailru/easyjson | **10512** | **336** | **12** | 310 | | buger/jsonparser | **15955** | **0** | **0** | 311 | | buger/jsonparser (EachKey API) | **8916** | **0** | **0** | 312 | 313 | The difference between ffjson and jsonparser in CPU usage is smaller, while the memory consumption difference is growing. On the other hand `easyjson` shows remarkable performance for medium payload. 314 | 315 | `gabs`, `go-simplejson` and `jason` are based on encoding/json and map[string]interface{} and actually only helpers for unstructured JSON, their performance correlate with `encoding/json interface{}`, and they will skip next round. 316 | `go-ujson` while have its own parser, shows same performance as `encoding/json`, also skips next round. Same situation with `ugorji/go/codec`, but it showed unexpectedly bad performance for complex payloads. 317 | 318 | 319 | #### Large payload 320 | 321 | Each test processes a 24kb JSON record (based on Discourse API) 322 | It should read 2 arrays, and for each item in array get a few fields. 323 | Basically it means processing a full JSON file. 324 | 325 | https://github.com/buger/jsonparser/blob/master/benchmark/benchmark_large_payload_test.go 326 | 327 | | Library | time/op | bytes/op | allocs/op | 328 | | --- | --- | --- | --- | 329 | | encoding/json struct | 748336 | 8272 | 307 | 330 | | encoding/json interface{} | 1224271 | 215425 | 3395 | 331 | | a8m/djson | 510082 | 213682 | 2845 | 332 | | pquerna/ffjson | **312271** | **7792** | **298** | 333 | | mailru/easyjson | **154186** | **6992** | **288** | 334 | | buger/jsonparser | **85308** | **0** | **0** | 335 | 336 | `jsonparser` now is a winner, but do not forget that it is way more lightweight parser than `ffson` or `easyjson`, and they have to parser all the data, while `jsonparser` parse only what you need. All `ffjson`, `easysjon` and `jsonparser` have their own parsing code, and does not depend on `encoding/json` or `interface{}`, thats one of the reasons why they are so fast. `easyjson` also use a bit of `unsafe` package to reduce memory consuption (in theory it can lead to some unexpected GC issue, but i did not tested enough) 337 | 338 | Also last benchmark did not included `EachKey` test, because in this particular case we need to read lot of Array values, and using `ArrayEach` is more efficient. 339 | 340 | ## Questions and support 341 | 342 | All bug-reports and suggestions should go though Github Issues. 343 | 344 | ## Contributing 345 | 346 | 1. Fork it 347 | 2. Create your feature branch (git checkout -b my-new-feature) 348 | 3. Commit your changes (git commit -am 'Added some feature') 349 | 4. Push to the branch (git push origin my-new-feature) 350 | 5. Create new Pull Request 351 | 352 | ## Development 353 | 354 | All my development happens using Docker, and repo include some Make tasks to simplify development. 355 | 356 | * `make build` - builds docker image, usually can be called only once 357 | * `make test` - run tests 358 | * `make fmt` - run go fmt 359 | * `make bench` - run benchmarks (if you need to run only single benchmark modify `BENCHMARK` variable in make file) 360 | * `make profile` - runs benchmark and generate 3 files- `cpu.out`, `mem.mprof` and `benchmark.test` binary, which can be used for `go tool pprof` 361 | * `make bash` - enter container (i use it for running `go tool pprof` above) 362 | -------------------------------------------------------------------------------- /benchmark/benchmark_easyjson.go: -------------------------------------------------------------------------------- 1 | package benchmark 2 | 3 | import ( 4 | json "encoding/json" 5 | jlexer "github.com/mailru/easyjson/jlexer" 6 | jwriter "github.com/mailru/easyjson/jwriter" 7 | ) 8 | 9 | var _ = json.RawMessage{} // suppress unused package warning 10 | 11 | func easyjson_decode_github_com_buger_jsonparser_benchmark_LargePayload(in *jlexer.Lexer, out *LargePayload) { 12 | in.Delim('{') 13 | for !in.IsDelim('}') { 14 | key := in.UnsafeString() 15 | in.WantColon() 16 | if in.IsNull() { 17 | in.Skip() 18 | in.WantComma() 19 | continue 20 | } 21 | switch key { 22 | case "users": 23 | in.Delim('[') 24 | if !in.IsDelim(']') { 25 | out.Users = make([]*DSUser, 0, 8) 26 | } else { 27 | out.Users = nil 28 | } 29 | for !in.IsDelim(']') { 30 | var v1 *DSUser 31 | if in.IsNull() { 32 | in.Skip() 33 | v1 = nil 34 | } else { 35 | v1 = new(DSUser) 36 | (*v1).UnmarshalEasyJSON(in) 37 | } 38 | out.Users = append(out.Users, v1) 39 | in.WantComma() 40 | } 41 | in.Delim(']') 42 | case "topics": 43 | if in.IsNull() { 44 | in.Skip() 45 | out.Topics = nil 46 | } else { 47 | out.Topics = new(DSTopicsList) 48 | (*out.Topics).UnmarshalEasyJSON(in) 49 | } 50 | default: 51 | in.SkipRecursive() 52 | } 53 | in.WantComma() 54 | } 55 | in.Delim('}') 56 | } 57 | func easyjson_encode_github_com_buger_jsonparser_benchmark_LargePayload(out *jwriter.Writer, in *LargePayload) { 58 | out.RawByte('{') 59 | first := true 60 | _ = first 61 | if !first { 62 | out.RawByte(',') 63 | } 64 | first = false 65 | out.RawString("\"users\":") 66 | out.RawByte('[') 67 | for v2, v3 := range in.Users { 68 | if v2 > 0 { 69 | out.RawByte(',') 70 | } 71 | if v3 == nil { 72 | out.RawString("null") 73 | } else { 74 | (*v3).MarshalEasyJSON(out) 75 | } 76 | } 77 | out.RawByte(']') 78 | if !first { 79 | out.RawByte(',') 80 | } 81 | first = false 82 | out.RawString("\"topics\":") 83 | if in.Topics == nil { 84 | out.RawString("null") 85 | } else { 86 | (*in.Topics).MarshalEasyJSON(out) 87 | } 88 | out.RawByte('}') 89 | } 90 | func (v *LargePayload) MarshalEasyJSON(w *jwriter.Writer) { 91 | easyjson_encode_github_com_buger_jsonparser_benchmark_LargePayload(w, v) 92 | } 93 | func (v *LargePayload) UnmarshalEasyJSON(l *jlexer.Lexer) { 94 | easyjson_decode_github_com_buger_jsonparser_benchmark_LargePayload(l, v) 95 | } 96 | func easyjson_decode_github_com_buger_jsonparser_benchmark_DSTopicsList(in *jlexer.Lexer, out *DSTopicsList) { 97 | in.Delim('{') 98 | for !in.IsDelim('}') { 99 | key := in.UnsafeString() 100 | in.WantColon() 101 | if in.IsNull() { 102 | in.Skip() 103 | in.WantComma() 104 | continue 105 | } 106 | switch key { 107 | case "topics": 108 | in.Delim('[') 109 | if !in.IsDelim(']') { 110 | out.Topics = make([]*DSTopic, 0, 8) 111 | } else { 112 | out.Topics = nil 113 | } 114 | for !in.IsDelim(']') { 115 | var v4 *DSTopic 116 | if in.IsNull() { 117 | in.Skip() 118 | v4 = nil 119 | } else { 120 | v4 = new(DSTopic) 121 | (*v4).UnmarshalEasyJSON(in) 122 | } 123 | out.Topics = append(out.Topics, v4) 124 | in.WantComma() 125 | } 126 | in.Delim(']') 127 | case "more_topics_url": 128 | out.MoreTopicsUrl = in.String() 129 | default: 130 | in.SkipRecursive() 131 | } 132 | in.WantComma() 133 | } 134 | in.Delim('}') 135 | } 136 | func easyjson_encode_github_com_buger_jsonparser_benchmark_DSTopicsList(out *jwriter.Writer, in *DSTopicsList) { 137 | out.RawByte('{') 138 | first := true 139 | _ = first 140 | if !first { 141 | out.RawByte(',') 142 | } 143 | first = false 144 | out.RawString("\"topics\":") 145 | out.RawByte('[') 146 | for v5, v6 := range in.Topics { 147 | if v5 > 0 { 148 | out.RawByte(',') 149 | } 150 | if v6 == nil { 151 | out.RawString("null") 152 | } else { 153 | (*v6).MarshalEasyJSON(out) 154 | } 155 | } 156 | out.RawByte(']') 157 | if !first { 158 | out.RawByte(',') 159 | } 160 | first = false 161 | out.RawString("\"more_topics_url\":") 162 | out.String(in.MoreTopicsUrl) 163 | out.RawByte('}') 164 | } 165 | func (v *DSTopicsList) MarshalEasyJSON(w *jwriter.Writer) { 166 | easyjson_encode_github_com_buger_jsonparser_benchmark_DSTopicsList(w, v) 167 | } 168 | func (v *DSTopicsList) UnmarshalEasyJSON(l *jlexer.Lexer) { 169 | easyjson_decode_github_com_buger_jsonparser_benchmark_DSTopicsList(l, v) 170 | } 171 | func easyjson_decode_github_com_buger_jsonparser_benchmark_DSTopic(in *jlexer.Lexer, out *DSTopic) { 172 | in.Delim('{') 173 | for !in.IsDelim('}') { 174 | key := in.UnsafeString() 175 | in.WantColon() 176 | if in.IsNull() { 177 | in.Skip() 178 | in.WantComma() 179 | continue 180 | } 181 | switch key { 182 | case "id": 183 | out.Id = in.Int() 184 | case "slug": 185 | out.Slug = in.String() 186 | default: 187 | in.SkipRecursive() 188 | } 189 | in.WantComma() 190 | } 191 | in.Delim('}') 192 | } 193 | func easyjson_encode_github_com_buger_jsonparser_benchmark_DSTopic(out *jwriter.Writer, in *DSTopic) { 194 | out.RawByte('{') 195 | first := true 196 | _ = first 197 | if !first { 198 | out.RawByte(',') 199 | } 200 | first = false 201 | out.RawString("\"id\":") 202 | out.Int(in.Id) 203 | if !first { 204 | out.RawByte(',') 205 | } 206 | first = false 207 | out.RawString("\"slug\":") 208 | out.String(in.Slug) 209 | out.RawByte('}') 210 | } 211 | func (v *DSTopic) MarshalEasyJSON(w *jwriter.Writer) { 212 | easyjson_encode_github_com_buger_jsonparser_benchmark_DSTopic(w, v) 213 | } 214 | func (v *DSTopic) UnmarshalEasyJSON(l *jlexer.Lexer) { 215 | easyjson_decode_github_com_buger_jsonparser_benchmark_DSTopic(l, v) 216 | } 217 | func easyjson_decode_github_com_buger_jsonparser_benchmark_DSUser(in *jlexer.Lexer, out *DSUser) { 218 | in.Delim('{') 219 | for !in.IsDelim('}') { 220 | key := in.UnsafeString() 221 | in.WantColon() 222 | if in.IsNull() { 223 | in.Skip() 224 | in.WantComma() 225 | continue 226 | } 227 | switch key { 228 | case "username": 229 | out.Username = in.String() 230 | default: 231 | in.SkipRecursive() 232 | } 233 | in.WantComma() 234 | } 235 | in.Delim('}') 236 | } 237 | func easyjson_encode_github_com_buger_jsonparser_benchmark_DSUser(out *jwriter.Writer, in *DSUser) { 238 | out.RawByte('{') 239 | first := true 240 | _ = first 241 | if !first { 242 | out.RawByte(',') 243 | } 244 | first = false 245 | out.RawString("\"username\":") 246 | out.String(in.Username) 247 | out.RawByte('}') 248 | } 249 | func (v *DSUser) MarshalEasyJSON(w *jwriter.Writer) { 250 | easyjson_encode_github_com_buger_jsonparser_benchmark_DSUser(w, v) 251 | } 252 | func (v *DSUser) UnmarshalEasyJSON(l *jlexer.Lexer) { 253 | easyjson_decode_github_com_buger_jsonparser_benchmark_DSUser(l, v) 254 | } 255 | func easyjson_decode_github_com_buger_jsonparser_benchmark_MediumPayload(in *jlexer.Lexer, out *MediumPayload) { 256 | in.Delim('{') 257 | for !in.IsDelim('}') { 258 | key := in.UnsafeString() 259 | in.WantColon() 260 | if in.IsNull() { 261 | in.Skip() 262 | in.WantComma() 263 | continue 264 | } 265 | switch key { 266 | case "person": 267 | if in.IsNull() { 268 | in.Skip() 269 | out.Person = nil 270 | } else { 271 | out.Person = new(CBPerson) 272 | (*out.Person).UnmarshalEasyJSON(in) 273 | } 274 | case "company": 275 | in.Delim('{') 276 | if !in.IsDelim('}') { 277 | out.Company = make(map[string]interface{}) 278 | } else { 279 | out.Company = nil 280 | } 281 | for !in.IsDelim('}') { 282 | key := in.String() 283 | in.WantColon() 284 | var v7 interface{} 285 | v7 = in.Interface() 286 | (out.Company)[key] = v7 287 | in.WantComma() 288 | } 289 | in.Delim('}') 290 | default: 291 | in.SkipRecursive() 292 | } 293 | in.WantComma() 294 | } 295 | in.Delim('}') 296 | } 297 | func easyjson_encode_github_com_buger_jsonparser_benchmark_MediumPayload(out *jwriter.Writer, in *MediumPayload) { 298 | out.RawByte('{') 299 | first := true 300 | _ = first 301 | if !first { 302 | out.RawByte(',') 303 | } 304 | first = false 305 | out.RawString("\"person\":") 306 | if in.Person == nil { 307 | out.RawString("null") 308 | } else { 309 | (*in.Person).MarshalEasyJSON(out) 310 | } 311 | if !first { 312 | out.RawByte(',') 313 | } 314 | first = false 315 | out.RawString("\"company\":") 316 | out.RawByte('{') 317 | v8_first := true 318 | for v8_name, v8_value := range in.Company { 319 | if !v8_first { 320 | out.RawByte(',') 321 | } 322 | v8_first = false 323 | out.String(v8_name) 324 | out.Raw(json.Marshal(v8_value)) 325 | } 326 | out.RawByte('}') 327 | out.RawByte('}') 328 | } 329 | func (v *MediumPayload) MarshalEasyJSON(w *jwriter.Writer) { 330 | easyjson_encode_github_com_buger_jsonparser_benchmark_MediumPayload(w, v) 331 | } 332 | func (v *MediumPayload) UnmarshalEasyJSON(l *jlexer.Lexer) { 333 | easyjson_decode_github_com_buger_jsonparser_benchmark_MediumPayload(l, v) 334 | } 335 | func easyjson_decode_github_com_buger_jsonparser_benchmark_CBPerson(in *jlexer.Lexer, out *CBPerson) { 336 | in.Delim('{') 337 | for !in.IsDelim('}') { 338 | key := in.UnsafeString() 339 | in.WantColon() 340 | if in.IsNull() { 341 | in.Skip() 342 | in.WantComma() 343 | continue 344 | } 345 | switch key { 346 | case "name": 347 | if in.IsNull() { 348 | in.Skip() 349 | out.Name = nil 350 | } else { 351 | out.Name = new(CBName) 352 | (*out.Name).UnmarshalEasyJSON(in) 353 | } 354 | case "github": 355 | if in.IsNull() { 356 | in.Skip() 357 | out.Github = nil 358 | } else { 359 | out.Github = new(CBGithub) 360 | (*out.Github).UnmarshalEasyJSON(in) 361 | } 362 | case "gravatar": 363 | if in.IsNull() { 364 | in.Skip() 365 | out.Gravatar = nil 366 | } else { 367 | out.Gravatar = new(CBGravatar) 368 | (*out.Gravatar).UnmarshalEasyJSON(in) 369 | } 370 | default: 371 | in.SkipRecursive() 372 | } 373 | in.WantComma() 374 | } 375 | in.Delim('}') 376 | } 377 | func easyjson_encode_github_com_buger_jsonparser_benchmark_CBPerson(out *jwriter.Writer, in *CBPerson) { 378 | out.RawByte('{') 379 | first := true 380 | _ = first 381 | if !first { 382 | out.RawByte(',') 383 | } 384 | first = false 385 | out.RawString("\"name\":") 386 | if in.Name == nil { 387 | out.RawString("null") 388 | } else { 389 | (*in.Name).MarshalEasyJSON(out) 390 | } 391 | if !first { 392 | out.RawByte(',') 393 | } 394 | first = false 395 | out.RawString("\"github\":") 396 | if in.Github == nil { 397 | out.RawString("null") 398 | } else { 399 | (*in.Github).MarshalEasyJSON(out) 400 | } 401 | if !first { 402 | out.RawByte(',') 403 | } 404 | first = false 405 | out.RawString("\"gravatar\":") 406 | if in.Gravatar == nil { 407 | out.RawString("null") 408 | } else { 409 | (*in.Gravatar).MarshalEasyJSON(out) 410 | } 411 | out.RawByte('}') 412 | } 413 | func (v *CBPerson) MarshalEasyJSON(w *jwriter.Writer) { 414 | easyjson_encode_github_com_buger_jsonparser_benchmark_CBPerson(w, v) 415 | } 416 | func (v *CBPerson) UnmarshalEasyJSON(l *jlexer.Lexer) { 417 | easyjson_decode_github_com_buger_jsonparser_benchmark_CBPerson(l, v) 418 | } 419 | func easyjson_decode_github_com_buger_jsonparser_benchmark_CBName(in *jlexer.Lexer, out *CBName) { 420 | in.Delim('{') 421 | for !in.IsDelim('}') { 422 | key := in.UnsafeString() 423 | in.WantColon() 424 | if in.IsNull() { 425 | in.Skip() 426 | in.WantComma() 427 | continue 428 | } 429 | switch key { 430 | case "full_name": 431 | out.FullName = in.String() 432 | default: 433 | in.SkipRecursive() 434 | } 435 | in.WantComma() 436 | } 437 | in.Delim('}') 438 | } 439 | func easyjson_encode_github_com_buger_jsonparser_benchmark_CBName(out *jwriter.Writer, in *CBName) { 440 | out.RawByte('{') 441 | first := true 442 | _ = first 443 | if !first { 444 | out.RawByte(',') 445 | } 446 | first = false 447 | out.RawString("\"full_name\":") 448 | out.String(in.FullName) 449 | out.RawByte('}') 450 | } 451 | func (v *CBName) MarshalEasyJSON(w *jwriter.Writer) { 452 | easyjson_encode_github_com_buger_jsonparser_benchmark_CBName(w, v) 453 | } 454 | func (v *CBName) UnmarshalEasyJSON(l *jlexer.Lexer) { 455 | easyjson_decode_github_com_buger_jsonparser_benchmark_CBName(l, v) 456 | } 457 | func easyjson_decode_github_com_buger_jsonparser_benchmark_CBGithub(in *jlexer.Lexer, out *CBGithub) { 458 | in.Delim('{') 459 | for !in.IsDelim('}') { 460 | key := in.UnsafeString() 461 | in.WantColon() 462 | if in.IsNull() { 463 | in.Skip() 464 | in.WantComma() 465 | continue 466 | } 467 | switch key { 468 | case "followers": 469 | out.Followers = in.Int() 470 | default: 471 | in.SkipRecursive() 472 | } 473 | in.WantComma() 474 | } 475 | in.Delim('}') 476 | } 477 | func easyjson_encode_github_com_buger_jsonparser_benchmark_CBGithub(out *jwriter.Writer, in *CBGithub) { 478 | out.RawByte('{') 479 | first := true 480 | _ = first 481 | if !first { 482 | out.RawByte(',') 483 | } 484 | first = false 485 | out.RawString("\"followers\":") 486 | out.Int(in.Followers) 487 | out.RawByte('}') 488 | } 489 | func (v *CBGithub) MarshalEasyJSON(w *jwriter.Writer) { 490 | easyjson_encode_github_com_buger_jsonparser_benchmark_CBGithub(w, v) 491 | } 492 | func (v *CBGithub) UnmarshalEasyJSON(l *jlexer.Lexer) { 493 | easyjson_decode_github_com_buger_jsonparser_benchmark_CBGithub(l, v) 494 | } 495 | func easyjson_decode_github_com_buger_jsonparser_benchmark_CBGravatar(in *jlexer.Lexer, out *CBGravatar) { 496 | in.Delim('{') 497 | for !in.IsDelim('}') { 498 | key := in.UnsafeString() 499 | in.WantColon() 500 | if in.IsNull() { 501 | in.Skip() 502 | in.WantComma() 503 | continue 504 | } 505 | switch key { 506 | case "avatars": 507 | in.Delim('[') 508 | if !in.IsDelim(']') { 509 | out.Avatars = make([]*CBAvatar, 0, 8) 510 | } else { 511 | out.Avatars = nil 512 | } 513 | for !in.IsDelim(']') { 514 | var v9 *CBAvatar 515 | if in.IsNull() { 516 | in.Skip() 517 | v9 = nil 518 | } else { 519 | v9 = new(CBAvatar) 520 | (*v9).UnmarshalEasyJSON(in) 521 | } 522 | out.Avatars = append(out.Avatars, v9) 523 | in.WantComma() 524 | } 525 | in.Delim(']') 526 | default: 527 | in.SkipRecursive() 528 | } 529 | in.WantComma() 530 | } 531 | in.Delim('}') 532 | } 533 | func easyjson_encode_github_com_buger_jsonparser_benchmark_CBGravatar(out *jwriter.Writer, in *CBGravatar) { 534 | out.RawByte('{') 535 | first := true 536 | _ = first 537 | if !first { 538 | out.RawByte(',') 539 | } 540 | first = false 541 | out.RawString("\"avatars\":") 542 | out.RawByte('[') 543 | for v10, v11 := range in.Avatars { 544 | if v10 > 0 { 545 | out.RawByte(',') 546 | } 547 | if v11 == nil { 548 | out.RawString("null") 549 | } else { 550 | (*v11).MarshalEasyJSON(out) 551 | } 552 | } 553 | out.RawByte(']') 554 | out.RawByte('}') 555 | } 556 | func (v *CBGravatar) MarshalEasyJSON(w *jwriter.Writer) { 557 | easyjson_encode_github_com_buger_jsonparser_benchmark_CBGravatar(w, v) 558 | } 559 | func (v *CBGravatar) UnmarshalEasyJSON(l *jlexer.Lexer) { 560 | easyjson_decode_github_com_buger_jsonparser_benchmark_CBGravatar(l, v) 561 | } 562 | func easyjson_decode_github_com_buger_jsonparser_benchmark_CBAvatar(in *jlexer.Lexer, out *CBAvatar) { 563 | in.Delim('{') 564 | for !in.IsDelim('}') { 565 | key := in.UnsafeString() 566 | in.WantColon() 567 | if in.IsNull() { 568 | in.Skip() 569 | in.WantComma() 570 | continue 571 | } 572 | switch key { 573 | case "url": 574 | out.Url = in.String() 575 | default: 576 | in.SkipRecursive() 577 | } 578 | in.WantComma() 579 | } 580 | in.Delim('}') 581 | } 582 | func easyjson_encode_github_com_buger_jsonparser_benchmark_CBAvatar(out *jwriter.Writer, in *CBAvatar) { 583 | out.RawByte('{') 584 | first := true 585 | _ = first 586 | if !first { 587 | out.RawByte(',') 588 | } 589 | first = false 590 | out.RawString("\"url\":") 591 | out.String(in.Url) 592 | out.RawByte('}') 593 | } 594 | func (v *CBAvatar) MarshalEasyJSON(w *jwriter.Writer) { 595 | easyjson_encode_github_com_buger_jsonparser_benchmark_CBAvatar(w, v) 596 | } 597 | func (v *CBAvatar) UnmarshalEasyJSON(l *jlexer.Lexer) { 598 | easyjson_decode_github_com_buger_jsonparser_benchmark_CBAvatar(l, v) 599 | } 600 | func easyjson_decode_github_com_buger_jsonparser_benchmark_SmallPayload(in *jlexer.Lexer, out *SmallPayload) { 601 | in.Delim('{') 602 | for !in.IsDelim('}') { 603 | key := in.UnsafeString() 604 | in.WantColon() 605 | if in.IsNull() { 606 | in.Skip() 607 | in.WantComma() 608 | continue 609 | } 610 | switch key { 611 | case "st": 612 | out.St = in.Int() 613 | case "sid": 614 | out.Sid = in.Int() 615 | case "tt": 616 | out.Tt = in.String() 617 | case "gr": 618 | out.Gr = in.Int() 619 | case "uuid": 620 | out.Uuid = in.String() 621 | case "ip": 622 | out.Ip = in.String() 623 | case "ua": 624 | out.Ua = in.String() 625 | case "tz": 626 | out.Tz = in.Int() 627 | case "v": 628 | out.V = in.Int() 629 | default: 630 | in.SkipRecursive() 631 | } 632 | in.WantComma() 633 | } 634 | in.Delim('}') 635 | } 636 | func easyjson_encode_github_com_buger_jsonparser_benchmark_SmallPayload(out *jwriter.Writer, in *SmallPayload) { 637 | out.RawByte('{') 638 | first := true 639 | _ = first 640 | if !first { 641 | out.RawByte(',') 642 | } 643 | first = false 644 | out.RawString("\"st\":") 645 | out.Int(in.St) 646 | if !first { 647 | out.RawByte(',') 648 | } 649 | first = false 650 | out.RawString("\"sid\":") 651 | out.Int(in.Sid) 652 | if !first { 653 | out.RawByte(',') 654 | } 655 | first = false 656 | out.RawString("\"tt\":") 657 | out.String(in.Tt) 658 | if !first { 659 | out.RawByte(',') 660 | } 661 | first = false 662 | out.RawString("\"gr\":") 663 | out.Int(in.Gr) 664 | if !first { 665 | out.RawByte(',') 666 | } 667 | first = false 668 | out.RawString("\"uuid\":") 669 | out.String(in.Uuid) 670 | if !first { 671 | out.RawByte(',') 672 | } 673 | first = false 674 | out.RawString("\"ip\":") 675 | out.String(in.Ip) 676 | if !first { 677 | out.RawByte(',') 678 | } 679 | first = false 680 | out.RawString("\"ua\":") 681 | out.String(in.Ua) 682 | if !first { 683 | out.RawByte(',') 684 | } 685 | first = false 686 | out.RawString("\"tz\":") 687 | out.Int(in.Tz) 688 | if !first { 689 | out.RawByte(',') 690 | } 691 | first = false 692 | out.RawString("\"v\":") 693 | out.Int(in.V) 694 | out.RawByte('}') 695 | } 696 | func (v *SmallPayload) MarshalEasyJSON(w *jwriter.Writer) { 697 | easyjson_encode_github_com_buger_jsonparser_benchmark_SmallPayload(w, v) 698 | } 699 | func (v *SmallPayload) UnmarshalEasyJSON(l *jlexer.Lexer) { 700 | easyjson_decode_github_com_buger_jsonparser_benchmark_SmallPayload(l, v) 701 | } 702 | -------------------------------------------------------------------------------- /benchmark/benchmark.go: -------------------------------------------------------------------------------- 1 | package benchmark 2 | 3 | /* 4 | Small paylod, http log like structure. Size: 190 bytes 5 | */ 6 | var smallFixture []byte = []byte(`{ 7 | "st": 1, 8 | "sid": 486, 9 | "tt": "active", 10 | "gr": 0, 11 | "uuid": "de305d54-75b4-431b-adb2-eb6b9e546014", 12 | "ip": "127.0.0.1", 13 | "ua": "user_agent", 14 | "tz": -6, 15 | "v": 1 16 | }`) 17 | 18 | type SmallPayload struct { 19 | St int 20 | Sid int 21 | Tt string 22 | Gr int 23 | Uuid string 24 | Ip string 25 | Ua string 26 | Tz int 27 | V int 28 | } 29 | 30 | /* 31 | Medium payload (based on Clearbit API response) 32 | */ 33 | type CBAvatar struct { 34 | Url string 35 | } 36 | 37 | type CBGravatar struct { 38 | Avatars []*CBAvatar 39 | } 40 | 41 | type CBGithub struct { 42 | Followers int 43 | } 44 | 45 | type CBName struct { 46 | FullName string 47 | } 48 | 49 | type CBPerson struct { 50 | Name *CBName 51 | Github *CBGithub 52 | Gravatar *CBGravatar 53 | } 54 | 55 | type MediumPayload struct { 56 | Person *CBPerson 57 | Company map[string]interface{} 58 | } 59 | 60 | // Reponse from Clearbit API. Size: 2.4kb 61 | var mediumFixture []byte = []byte(`{ 62 | "person": { 63 | "id": "d50887ca-a6ce-4e59-b89f-14f0b5d03b03", 64 | "name": { 65 | "fullName": "Leonid Bugaev", 66 | "givenName": "Leonid", 67 | "familyName": "Bugaev" 68 | }, 69 | "email": "leonsbox@gmail.com", 70 | "gender": "male", 71 | "location": "Saint Petersburg, Saint Petersburg, RU", 72 | "geo": { 73 | "city": "Saint Petersburg", 74 | "state": "Saint Petersburg", 75 | "country": "Russia", 76 | "lat": 59.9342802, 77 | "lng": 30.3350986 78 | }, 79 | "bio": "Senior engineer at Granify.com", 80 | "site": "http://flickfaver.com", 81 | "avatar": "https://d1ts43dypk8bqh.cloudfront.net/v1/avatars/d50887ca-a6ce-4e59-b89f-14f0b5d03b03", 82 | "employment": { 83 | "name": "www.latera.ru", 84 | "title": "Software Engineer", 85 | "domain": "gmail.com" 86 | }, 87 | "facebook": { 88 | "handle": "leonid.bugaev" 89 | }, 90 | "github": { 91 | "handle": "buger", 92 | "id": 14009, 93 | "avatar": "https://avatars.githubusercontent.com/u/14009?v=3", 94 | "company": "Granify", 95 | "blog": "http://leonsbox.com", 96 | "followers": 95, 97 | "following": 10 98 | }, 99 | "twitter": { 100 | "handle": "flickfaver", 101 | "id": 77004410, 102 | "bio": null, 103 | "followers": 2, 104 | "following": 1, 105 | "statuses": 5, 106 | "favorites": 0, 107 | "location": "", 108 | "site": "http://flickfaver.com", 109 | "avatar": null 110 | }, 111 | "linkedin": { 112 | "handle": "in/leonidbugaev" 113 | }, 114 | "googleplus": { 115 | "handle": null 116 | }, 117 | "angellist": { 118 | "handle": "leonid-bugaev", 119 | "id": 61541, 120 | "bio": "Senior engineer at Granify.com", 121 | "blog": "http://buger.github.com", 122 | "site": "http://buger.github.com", 123 | "followers": 41, 124 | "avatar": "https://d1qb2nb5cznatu.cloudfront.net/users/61541-medium_jpg?1405474390" 125 | }, 126 | "klout": { 127 | "handle": null, 128 | "score": null 129 | }, 130 | "foursquare": { 131 | "handle": null 132 | }, 133 | "aboutme": { 134 | "handle": "leonid.bugaev", 135 | "bio": null, 136 | "avatar": null 137 | }, 138 | "gravatar": { 139 | "handle": "buger", 140 | "urls": [ 141 | 142 | ], 143 | "avatar": "http://1.gravatar.com/avatar/f7c8edd577d13b8930d5522f28123510", 144 | "avatars": [ 145 | { 146 | "url": "http://1.gravatar.com/avatar/f7c8edd577d13b8930d5522f28123510", 147 | "type": "thumbnail" 148 | } 149 | ] 150 | }, 151 | "fuzzy": false 152 | }, 153 | "company": null 154 | }`) 155 | 156 | /* 157 | Large payload, based on Discourse API. Size: 28kb 158 | */ 159 | 160 | type DSUser struct { 161 | Username string 162 | } 163 | 164 | type DSTopic struct { 165 | Id int 166 | Slug string 167 | } 168 | 169 | type DSTopicsList struct { 170 | Topics []*DSTopic 171 | MoreTopicsUrl string 172 | } 173 | 174 | type LargePayload struct { 175 | Users []*DSUser 176 | Topics *DSTopicsList 177 | } 178 | 179 | var largeFixture []byte = []byte(` 180 | {"users":[{"id":-1,"username":"system","avatar_template":"/user_avatar/discourse.metabase.com/system/{size}/6_1.png"},{"id":89,"username":"zergot","avatar_template":"https://avatars.discourse.org/v2/letter/z/0ea827/{size}.png"},{"id":1,"username":"sameer","avatar_template":"https://avatars.discourse.org/v2/letter/s/bbce88/{size}.png"},{"id":84,"username":"HenryMirror","avatar_template":"https://avatars.discourse.org/v2/letter/h/ecd19e/{size}.png"},{"id":73,"username":"fimp","avatar_template":"https://avatars.discourse.org/v2/letter/f/ee59a6/{size}.png"},{"id":14,"username":"agilliland","avatar_template":"/user_avatar/discourse.metabase.com/agilliland/{size}/26_1.png"},{"id":87,"username":"amir","avatar_template":"https://avatars.discourse.org/v2/letter/a/c37758/{size}.png"},{"id":82,"username":"waseem","avatar_template":"https://avatars.discourse.org/v2/letter/w/9dc877/{size}.png"},{"id":78,"username":"tovenaar","avatar_template":"https://avatars.discourse.org/v2/letter/t/9de0a6/{size}.png"},{"id":74,"username":"Ben","avatar_template":"https://avatars.discourse.org/v2/letter/b/df788c/{size}.png"},{"id":71,"username":"MarkLaFay","avatar_template":"https://avatars.discourse.org/v2/letter/m/3bc359/{size}.png"},{"id":72,"username":"camsaul","avatar_template":"/user_avatar/discourse.metabase.com/camsaul/{size}/70_1.png"},{"id":53,"username":"mhjb","avatar_template":"/user_avatar/discourse.metabase.com/mhjb/{size}/54_1.png"},{"id":58,"username":"jbwiv","avatar_template":"https://avatars.discourse.org/v2/letter/j/6bbea6/{size}.png"},{"id":70,"username":"Maggs","avatar_template":"https://avatars.discourse.org/v2/letter/m/bbce88/{size}.png"},{"id":69,"username":"andrefaria","avatar_template":"/user_avatar/discourse.metabase.com/andrefaria/{size}/65_1.png"},{"id":60,"username":"bencarter78","avatar_template":"/user_avatar/discourse.metabase.com/bencarter78/{size}/59_1.png"},{"id":55,"username":"vikram","avatar_template":"https://avatars.discourse.org/v2/letter/v/e47774/{size}.png"},{"id":68,"username":"edchan77","avatar_template":"/user_avatar/discourse.metabase.com/edchan77/{size}/66_1.png"},{"id":9,"username":"karthikd","avatar_template":"https://avatars.discourse.org/v2/letter/k/cab0a1/{size}.png"},{"id":23,"username":"arthurz","avatar_template":"/user_avatar/discourse.metabase.com/arthurz/{size}/32_1.png"},{"id":3,"username":"tom","avatar_template":"/user_avatar/discourse.metabase.com/tom/{size}/21_1.png"},{"id":50,"username":"LeoNogueira","avatar_template":"/user_avatar/discourse.metabase.com/leonogueira/{size}/52_1.png"},{"id":66,"username":"ss06vi","avatar_template":"https://avatars.discourse.org/v2/letter/s/3ab097/{size}.png"},{"id":34,"username":"mattcollins","avatar_template":"/user_avatar/discourse.metabase.com/mattcollins/{size}/41_1.png"},{"id":51,"username":"krmmalik","avatar_template":"/user_avatar/discourse.metabase.com/krmmalik/{size}/53_1.png"},{"id":46,"username":"odysseas","avatar_template":"https://avatars.discourse.org/v2/letter/o/5f8ce5/{size}.png"},{"id":5,"username":"jonthewayne","avatar_template":"/user_avatar/discourse.metabase.com/jonthewayne/{size}/18_1.png"},{"id":11,"username":"anandiyer","avatar_template":"/user_avatar/discourse.metabase.com/anandiyer/{size}/23_1.png"},{"id":25,"username":"alnorth","avatar_template":"/user_avatar/discourse.metabase.com/alnorth/{size}/34_1.png"},{"id":52,"username":"j_at_svg","avatar_template":"https://avatars.discourse.org/v2/letter/j/96bed5/{size}.png"},{"id":42,"username":"styts","avatar_template":"/user_avatar/discourse.metabase.com/styts/{size}/47_1.png"}],"topics":{"can_create_topic":false,"more_topics_url":"/c/uncategorized/l/latest?page=1","draft":null,"draft_key":"new_topic","draft_sequence":null,"per_page":30,"topics":[{"id":8,"title":"Welcome to Metabase's Discussion Forum","fancy_title":"Welcome to Metabase’s Discussion Forum","slug":"welcome-to-metabases-discussion-forum","posts_count":1,"reply_count":0,"highest_post_number":1,"image_url":"/images/welcome/discourse-edit-post-animated.gif","created_at":"2015-10-17T00:14:49.526Z","last_posted_at":"2015-10-17T00:14:49.557Z","bumped":true,"bumped_at":"2015-10-21T02:32:22.486Z","unseen":false,"pinned":true,"unpinned":null,"excerpt":"Welcome to Metabase's discussion forum. This is a place to get help on installation, setting up as well as sharing tips and tricks.","visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":197,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"system","category_id":1,"pinned_globally":true,"posters":[{"extras":"latest single","description":"Original Poster, Most Recent Poster","user_id":-1}]},{"id":169,"title":"Formatting Dates","fancy_title":"Formatting Dates","slug":"formatting-dates","posts_count":1,"reply_count":0,"highest_post_number":1,"image_url":null,"created_at":"2016-01-14T06:30:45.311Z","last_posted_at":"2016-01-14T06:30:45.397Z","bumped":true,"bumped_at":"2016-01-14T06:30:45.397Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":11,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"zergot","category_id":1,"pinned_globally":false,"posters":[{"extras":"latest single","description":"Original Poster, Most Recent Poster","user_id":89}]},{"id":168,"title":"Setting for google api key","fancy_title":"Setting for google api key","slug":"setting-for-google-api-key","posts_count":2,"reply_count":0,"highest_post_number":2,"image_url":null,"created_at":"2016-01-13T17:14:31.799Z","last_posted_at":"2016-01-14T06:24:03.421Z","bumped":true,"bumped_at":"2016-01-14T06:24:03.421Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":16,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"zergot","category_id":1,"pinned_globally":false,"posters":[{"extras":"latest single","description":"Original Poster, Most Recent Poster","user_id":89}]},{"id":167,"title":"Cannot see non-US timezones on the admin","fancy_title":"Cannot see non-US timezones on the admin","slug":"cannot-see-non-us-timezones-on-the-admin","posts_count":1,"reply_count":0,"highest_post_number":1,"image_url":null,"created_at":"2016-01-13T17:07:36.764Z","last_posted_at":"2016-01-13T17:07:36.831Z","bumped":true,"bumped_at":"2016-01-13T17:07:36.831Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":11,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"zergot","category_id":1,"pinned_globally":false,"posters":[{"extras":"latest single","description":"Original Poster, Most Recent Poster","user_id":89}]},{"id":164,"title":"External (Metabase level) linkages in data schema","fancy_title":"External (Metabase level) linkages in data schema","slug":"external-metabase-level-linkages-in-data-schema","posts_count":4,"reply_count":1,"highest_post_number":4,"image_url":null,"created_at":"2016-01-11T13:51:02.286Z","last_posted_at":"2016-01-12T11:06:37.259Z","bumped":true,"bumped_at":"2016-01-12T11:06:37.259Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":32,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"zergot","category_id":1,"pinned_globally":false,"posters":[{"extras":"latest","description":"Original Poster, Most Recent Poster","user_id":89},{"extras":null,"description":"Frequent Poster","user_id":1}]},{"id":155,"title":"Query working on \"Questions\" but not in \"Pulses\"","fancy_title":"Query working on “Questions” but not in “Pulses”","slug":"query-working-on-questions-but-not-in-pulses","posts_count":3,"reply_count":0,"highest_post_number":3,"image_url":null,"created_at":"2016-01-01T14:06:10.083Z","last_posted_at":"2016-01-08T22:37:51.772Z","bumped":true,"bumped_at":"2016-01-08T22:37:51.772Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":72,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"agilliland","category_id":1,"pinned_globally":false,"posters":[{"extras":null,"description":"Original Poster","user_id":84},{"extras":null,"description":"Frequent Poster","user_id":73},{"extras":"latest","description":"Most Recent Poster","user_id":14}]},{"id":161,"title":"Pulses posted to Slack don't show question output","fancy_title":"Pulses posted to Slack don’t show question output","slug":"pulses-posted-to-slack-dont-show-question-output","posts_count":2,"reply_count":0,"highest_post_number":2,"image_url":"/uploads/default/original/1X/9d2806517bf3598b10be135b2c58923b47ba23e7.png","created_at":"2016-01-08T22:09:58.205Z","last_posted_at":"2016-01-08T22:28:44.685Z","bumped":true,"bumped_at":"2016-01-08T22:28:44.685Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":34,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"sameer","category_id":1,"pinned_globally":false,"posters":[{"extras":null,"description":"Original Poster","user_id":87},{"extras":"latest","description":"Most Recent Poster","user_id":1}]},{"id":152,"title":"Should we build Kafka connecter or Kafka plugin","fancy_title":"Should we build Kafka connecter or Kafka plugin","slug":"should-we-build-kafka-connecter-or-kafka-plugin","posts_count":4,"reply_count":1,"highest_post_number":4,"image_url":null,"created_at":"2015-12-28T20:37:23.501Z","last_posted_at":"2015-12-31T18:16:45.477Z","bumped":true,"bumped_at":"2015-12-31T18:16:45.477Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":84,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"sameer","category_id":1,"pinned_globally":false,"posters":[{"extras":null,"description":"Original Poster","user_id":82},{"extras":"latest","description":"Most Recent Poster, Frequent Poster","user_id":1}]},{"id":147,"title":"Change X and Y on graph","fancy_title":"Change X and Y on graph","slug":"change-x-and-y-on-graph","posts_count":1,"reply_count":0,"highest_post_number":1,"image_url":null,"created_at":"2015-12-21T17:52:46.581Z","last_posted_at":"2015-12-21T17:52:46.684Z","bumped":true,"bumped_at":"2015-12-21T18:19:13.003Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":68,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"tovenaar","category_id":1,"pinned_globally":false,"posters":[{"extras":"latest single","description":"Original Poster, Most Recent Poster","user_id":78}]},{"id":142,"title":"Issues sending mail via office365 relay","fancy_title":"Issues sending mail via office365 relay","slug":"issues-sending-mail-via-office365-relay","posts_count":5,"reply_count":2,"highest_post_number":5,"image_url":null,"created_at":"2015-12-16T10:38:47.315Z","last_posted_at":"2015-12-21T09:26:27.167Z","bumped":true,"bumped_at":"2015-12-21T09:26:27.167Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":122,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"Ben","category_id":1,"pinned_globally":false,"posters":[{"extras":"latest","description":"Original Poster, Most Recent Poster","user_id":74},{"extras":null,"description":"Frequent Poster","user_id":1}]},{"id":137,"title":"I see triplicates of my mongoDB collections","fancy_title":"I see triplicates of my mongoDB collections","slug":"i-see-triplicates-of-my-mongodb-collections","posts_count":3,"reply_count":0,"highest_post_number":3,"image_url":null,"created_at":"2015-12-14T13:33:03.426Z","last_posted_at":"2015-12-17T18:40:05.487Z","bumped":true,"bumped_at":"2015-12-17T18:40:05.487Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":97,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"MarkLaFay","category_id":1,"pinned_globally":false,"posters":[{"extras":"latest","description":"Original Poster, Most Recent Poster","user_id":71},{"extras":null,"description":"Frequent Poster","user_id":14}]},{"id":140,"title":"Google Analytics plugin","fancy_title":"Google Analytics plugin","slug":"google-analytics-plugin","posts_count":1,"reply_count":0,"highest_post_number":1,"image_url":null,"created_at":"2015-12-15T13:00:55.644Z","last_posted_at":"2015-12-15T13:00:55.705Z","bumped":true,"bumped_at":"2015-12-15T13:00:55.705Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":105,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"fimp","category_id":1,"pinned_globally":false,"posters":[{"extras":"latest single","description":"Original Poster, Most Recent Poster","user_id":73}]},{"id":138,"title":"With-mongo-connection failed: bad connection details:","fancy_title":"With-mongo-connection failed: bad connection details:","slug":"with-mongo-connection-failed-bad-connection-details","posts_count":1,"reply_count":0,"highest_post_number":1,"image_url":null,"created_at":"2015-12-14T17:28:11.041Z","last_posted_at":"2015-12-14T17:28:11.111Z","bumped":true,"bumped_at":"2015-12-14T17:28:11.111Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":56,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"MarkLaFay","category_id":1,"pinned_globally":false,"posters":[{"extras":"latest single","description":"Original Poster, Most Recent Poster","user_id":71}]},{"id":133,"title":"\"We couldn't understand your question.\" when I query mongoDB","fancy_title":"“We couldn’t understand your question.” when I query mongoDB","slug":"we-couldnt-understand-your-question-when-i-query-mongodb","posts_count":3,"reply_count":0,"highest_post_number":3,"image_url":null,"created_at":"2015-12-11T17:38:30.576Z","last_posted_at":"2015-12-14T13:31:26.395Z","bumped":true,"bumped_at":"2015-12-14T13:31:26.395Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":107,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"MarkLaFay","category_id":1,"pinned_globally":false,"posters":[{"extras":"latest","description":"Original Poster, Most Recent Poster","user_id":71},{"extras":null,"description":"Frequent Poster","user_id":72}]},{"id":129,"title":"My bar charts are all thin","fancy_title":"My bar charts are all thin","slug":"my-bar-charts-are-all-thin","posts_count":4,"reply_count":1,"highest_post_number":4,"image_url":"/uploads/default/original/1X/41bcf3b2a00dc7cfaff01cb3165d35d32a85bf1d.png","created_at":"2015-12-09T22:09:56.394Z","last_posted_at":"2015-12-11T19:00:45.289Z","bumped":true,"bumped_at":"2015-12-11T19:00:45.289Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":116,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"mhjb","category_id":1,"pinned_globally":false,"posters":[{"extras":"latest","description":"Original Poster, Most Recent Poster","user_id":53},{"extras":null,"description":"Frequent Poster","user_id":1}]},{"id":106,"title":"What is the expected return order of columns for graphing results when using raw SQL?","fancy_title":"What is the expected return order of columns for graphing results when using raw SQL?","slug":"what-is-the-expected-return-order-of-columns-for-graphing-results-when-using-raw-sql","posts_count":3,"reply_count":0,"highest_post_number":3,"image_url":null,"created_at":"2015-11-24T19:07:14.561Z","last_posted_at":"2015-12-11T17:04:14.149Z","bumped":true,"bumped_at":"2015-12-11T17:04:14.149Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":153,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"jbwiv","category_id":1,"pinned_globally":false,"posters":[{"extras":"latest","description":"Original Poster, Most Recent Poster","user_id":58},{"extras":null,"description":"Frequent Poster","user_id":14}]},{"id":131,"title":"Set site url from admin panel","fancy_title":"Set site url from admin panel","slug":"set-site-url-from-admin-panel","posts_count":2,"reply_count":0,"highest_post_number":2,"image_url":null,"created_at":"2015-12-10T06:22:46.042Z","last_posted_at":"2015-12-10T19:12:57.449Z","bumped":true,"bumped_at":"2015-12-10T19:12:57.449Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":77,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"sameer","category_id":1,"pinned_globally":false,"posters":[{"extras":null,"description":"Original Poster","user_id":70},{"extras":"latest","description":"Most Recent Poster","user_id":1}]},{"id":127,"title":"Internationalization (i18n)","fancy_title":"Internationalization (i18n)","slug":"internationalization-i18n","posts_count":2,"reply_count":0,"highest_post_number":2,"image_url":null,"created_at":"2015-12-08T16:55:37.397Z","last_posted_at":"2015-12-09T16:49:55.816Z","bumped":true,"bumped_at":"2015-12-09T16:49:55.816Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":85,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"agilliland","category_id":1,"pinned_globally":false,"posters":[{"extras":null,"description":"Original Poster","user_id":69},{"extras":"latest","description":"Most Recent Poster","user_id":14}]},{"id":109,"title":"Returning raw data with no filters always returns We couldn't understand your question","fancy_title":"Returning raw data with no filters always returns We couldn’t understand your question","slug":"returning-raw-data-with-no-filters-always-returns-we-couldnt-understand-your-question","posts_count":3,"reply_count":1,"highest_post_number":3,"image_url":null,"created_at":"2015-11-25T21:35:01.315Z","last_posted_at":"2015-12-09T10:26:12.255Z","bumped":true,"bumped_at":"2015-12-09T10:26:12.255Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":133,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"bencarter78","category_id":1,"pinned_globally":false,"posters":[{"extras":"latest","description":"Original Poster, Most Recent Poster","user_id":60},{"extras":null,"description":"Frequent Poster","user_id":14}]},{"id":103,"title":"Support for Cassandra?","fancy_title":"Support for Cassandra?","slug":"support-for-cassandra","posts_count":5,"reply_count":1,"highest_post_number":5,"image_url":null,"created_at":"2015-11-20T06:45:31.741Z","last_posted_at":"2015-12-09T03:18:51.274Z","bumped":true,"bumped_at":"2015-12-09T03:18:51.274Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":169,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"vikram","category_id":1,"pinned_globally":false,"posters":[{"extras":"latest","description":"Original Poster, Most Recent Poster","user_id":55},{"extras":null,"description":"Frequent Poster","user_id":1}]},{"id":128,"title":"Mongo query with Date breaks [solved: Mongo 3.0 required]","fancy_title":"Mongo query with Date breaks [solved: Mongo 3.0 required]","slug":"mongo-query-with-date-breaks-solved-mongo-3-0-required","posts_count":5,"reply_count":0,"highest_post_number":5,"image_url":null,"created_at":"2015-12-08T18:30:56.562Z","last_posted_at":"2015-12-08T21:03:02.421Z","bumped":true,"bumped_at":"2015-12-08T21:03:02.421Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":102,"like_count":1,"has_summary":false,"archetype":"regular","last_poster_username":"edchan77","category_id":1,"pinned_globally":false,"posters":[{"extras":"latest","description":"Original Poster, Most Recent Poster","user_id":68},{"extras":null,"description":"Frequent Poster","user_id":1}]},{"id":23,"title":"Can this connect to MS SQL Server?","fancy_title":"Can this connect to MS SQL Server?","slug":"can-this-connect-to-ms-sql-server","posts_count":7,"reply_count":1,"highest_post_number":7,"image_url":null,"created_at":"2015-10-21T18:52:37.987Z","last_posted_at":"2015-12-07T17:41:51.609Z","bumped":true,"bumped_at":"2015-12-07T17:41:51.609Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":367,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"sameer","category_id":1,"pinned_globally":false,"posters":[{"extras":null,"description":"Original Poster","user_id":9},{"extras":null,"description":"Frequent Poster","user_id":23},{"extras":null,"description":"Frequent Poster","user_id":3},{"extras":null,"description":"Frequent Poster","user_id":50},{"extras":"latest","description":"Most Recent Poster","user_id":1}]},{"id":121,"title":"Cannot restart metabase in docker","fancy_title":"Cannot restart metabase in docker","slug":"cannot-restart-metabase-in-docker","posts_count":5,"reply_count":1,"highest_post_number":5,"image_url":null,"created_at":"2015-12-04T21:28:58.137Z","last_posted_at":"2015-12-04T23:02:00.488Z","bumped":true,"bumped_at":"2015-12-04T23:02:00.488Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":96,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"sameer","category_id":1,"pinned_globally":false,"posters":[{"extras":null,"description":"Original Poster","user_id":66},{"extras":"latest","description":"Most Recent Poster, Frequent Poster","user_id":1}]},{"id":85,"title":"Edit Max Rows Count","fancy_title":"Edit Max Rows Count","slug":"edit-max-rows-count","posts_count":4,"reply_count":2,"highest_post_number":4,"image_url":null,"created_at":"2015-11-11T23:46:52.917Z","last_posted_at":"2015-11-24T01:01:14.569Z","bumped":true,"bumped_at":"2015-11-24T01:01:14.569Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":169,"like_count":1,"has_summary":false,"archetype":"regular","last_poster_username":"sameer","category_id":1,"pinned_globally":false,"posters":[{"extras":null,"description":"Original Poster","user_id":34},{"extras":"latest","description":"Most Recent Poster, Frequent Poster","user_id":1}]},{"id":96,"title":"Creating charts by querying more than one table at a time","fancy_title":"Creating charts by querying more than one table at a time","slug":"creating-charts-by-querying-more-than-one-table-at-a-time","posts_count":6,"reply_count":4,"highest_post_number":6,"image_url":null,"created_at":"2015-11-17T11:20:18.442Z","last_posted_at":"2015-11-21T02:12:25.995Z","bumped":true,"bumped_at":"2015-11-21T02:12:25.995Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":217,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"sameer","category_id":1,"pinned_globally":false,"posters":[{"extras":null,"description":"Original Poster","user_id":51},{"extras":"latest","description":"Most Recent Poster, Frequent Poster","user_id":1}]},{"id":90,"title":"Trying to add RDS postgresql as the database fails silently","fancy_title":"Trying to add RDS postgresql as the database fails silently","slug":"trying-to-add-rds-postgresql-as-the-database-fails-silently","posts_count":4,"reply_count":2,"highest_post_number":4,"image_url":null,"created_at":"2015-11-14T23:45:02.967Z","last_posted_at":"2015-11-21T01:08:45.915Z","bumped":true,"bumped_at":"2015-11-21T01:08:45.915Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":162,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"sameer","category_id":1,"pinned_globally":false,"posters":[{"extras":null,"description":"Original Poster","user_id":46},{"extras":"latest","description":"Most Recent Poster, Frequent Poster","user_id":1}]},{"id":17,"title":"Deploy to Heroku isn't working","fancy_title":"Deploy to Heroku isn’t working","slug":"deploy-to-heroku-isnt-working","posts_count":9,"reply_count":3,"highest_post_number":9,"image_url":null,"created_at":"2015-10-21T16:42:03.096Z","last_posted_at":"2015-11-20T18:34:14.044Z","bumped":true,"bumped_at":"2015-11-20T18:34:14.044Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":332,"like_count":2,"has_summary":false,"archetype":"regular","last_poster_username":"agilliland","category_id":1,"pinned_globally":false,"posters":[{"extras":null,"description":"Original Poster","user_id":5},{"extras":null,"description":"Frequent Poster","user_id":3},{"extras":null,"description":"Frequent Poster","user_id":11},{"extras":null,"description":"Frequent Poster","user_id":25},{"extras":"latest","description":"Most Recent Poster","user_id":14}]},{"id":100,"title":"Can I use DATEPART() in SQL queries?","fancy_title":"Can I use DATEPART() in SQL queries?","slug":"can-i-use-datepart-in-sql-queries","posts_count":2,"reply_count":0,"highest_post_number":2,"image_url":null,"created_at":"2015-11-17T23:15:58.033Z","last_posted_at":"2015-11-18T00:19:48.763Z","bumped":true,"bumped_at":"2015-11-18T00:19:48.763Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":112,"like_count":1,"has_summary":false,"archetype":"regular","last_poster_username":"sameer","category_id":1,"pinned_globally":false,"posters":[{"extras":null,"description":"Original Poster","user_id":53},{"extras":"latest","description":"Most Recent Poster","user_id":1}]},{"id":98,"title":"Feature Request: LDAP Authentication","fancy_title":"Feature Request: LDAP Authentication","slug":"feature-request-ldap-authentication","posts_count":1,"reply_count":0,"highest_post_number":1,"image_url":null,"created_at":"2015-11-17T17:22:44.484Z","last_posted_at":"2015-11-17T17:22:44.577Z","bumped":true,"bumped_at":"2015-11-17T17:22:44.577Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":97,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"j_at_svg","category_id":1,"pinned_globally":false,"posters":[{"extras":"latest single","description":"Original Poster, Most Recent Poster","user_id":52}]},{"id":87,"title":"Migrating from internal H2 to Postgres","fancy_title":"Migrating from internal H2 to Postgres","slug":"migrating-from-internal-h2-to-postgres","posts_count":2,"reply_count":0,"highest_post_number":2,"image_url":null,"created_at":"2015-11-12T14:36:06.745Z","last_posted_at":"2015-11-12T18:05:10.796Z","bumped":true,"bumped_at":"2015-11-12T18:05:10.796Z","unseen":false,"pinned":false,"unpinned":null,"visible":true,"closed":false,"archived":false,"bookmarked":null,"liked":null,"views":111,"like_count":0,"has_summary":false,"archetype":"regular","last_poster_username":"sameer","category_id":1,"pinned_globally":false,"posters":[{"extras":null,"description":"Original Poster","user_id":42},{"extras":"latest","description":"Most Recent Poster","user_id":1}]}]}} 181 | `) 182 | -------------------------------------------------------------------------------- /parser.go: -------------------------------------------------------------------------------- 1 | package jsonparser 2 | 3 | import ( 4 | "bytes" 5 | "errors" 6 | "fmt" 7 | "strconv" 8 | ) 9 | 10 | // Errors 11 | var ( 12 | KeyPathNotFoundError = errors.New("Key path not found") 13 | UnknownValueTypeError = errors.New("Unknown value type") 14 | MalformedJsonError = errors.New("Malformed JSON error") 15 | MalformedStringError = errors.New("Value is string, but can't find closing '\"' symbol") 16 | MalformedArrayError = errors.New("Value is array, but can't find closing ']' symbol") 17 | MalformedObjectError = errors.New("Value looks like object, but can't find closing '}' symbol") 18 | MalformedValueError = errors.New("Value looks like Number/Boolean/None, but can't find its end: ',' or '}' symbol") 19 | OverflowIntegerError = errors.New("Value is number, but overflowed while parsing") 20 | MalformedStringEscapeError = errors.New("Encountered an invalid escape sequence in a string") 21 | NullValueError = errors.New("Value is null") 22 | ) 23 | 24 | // How much stack space to allocate for unescaping JSON strings; if a string longer 25 | // than this needs to be escaped, it will result in a heap allocation 26 | const unescapeStackBufSize = 64 27 | 28 | func tokenEnd(data []byte) int { 29 | for i, c := range data { 30 | switch c { 31 | case ' ', '\n', '\r', '\t', ',', '}', ']': 32 | return i 33 | } 34 | } 35 | 36 | return len(data) 37 | } 38 | 39 | func findTokenStart(data []byte, token byte) int { 40 | for i := len(data) - 1; i >= 0; i-- { 41 | switch data[i] { 42 | case token: 43 | return i 44 | case '[', '{': 45 | return 0 46 | } 47 | } 48 | 49 | return 0 50 | } 51 | 52 | func findKeyStart(data []byte, key string) (int, error) { 53 | i := nextToken(data) 54 | if i == -1 { 55 | return i, KeyPathNotFoundError 56 | } 57 | ln := len(data) 58 | if ln > 0 && (data[i] == '{' || data[i] == '[') { 59 | i += 1 60 | } 61 | var stackbuf [unescapeStackBufSize]byte // stack-allocated array for allocation-free unescaping of small strings 62 | 63 | if ku, err := Unescape(StringToBytes(key), stackbuf[:]); err == nil { 64 | key = bytesToString(&ku) 65 | } 66 | 67 | for i < ln { 68 | switch data[i] { 69 | case '"': 70 | i++ 71 | keyBegin := i 72 | 73 | strEnd, keyEscaped := stringEnd(data[i:]) 74 | if strEnd == -1 { 75 | break 76 | } 77 | i += strEnd 78 | keyEnd := i - 1 79 | 80 | valueOffset := nextToken(data[i:]) 81 | if valueOffset == -1 { 82 | break 83 | } 84 | 85 | i += valueOffset 86 | 87 | // if string is a key, and key level match 88 | k := data[keyBegin:keyEnd] 89 | // for unescape: if there are no escape sequences, this is cheap; if there are, it is a 90 | // bit more expensive, but causes no allocations unless len(key) > unescapeStackBufSize 91 | if keyEscaped { 92 | if ku, err := Unescape(k, stackbuf[:]); err != nil { 93 | break 94 | } else { 95 | k = ku 96 | } 97 | } 98 | 99 | if data[i] == ':' && len(key) == len(k) && bytesToString(&k) == key { 100 | return keyBegin - 1, nil 101 | } 102 | 103 | case '[': 104 | end := blockEnd(data[i:], data[i], ']') 105 | if end != -1 { 106 | i = i + end 107 | } 108 | case '{': 109 | end := blockEnd(data[i:], data[i], '}') 110 | if end != -1 { 111 | i = i + end 112 | } 113 | } 114 | i++ 115 | } 116 | 117 | return -1, KeyPathNotFoundError 118 | } 119 | 120 | func tokenStart(data []byte) int { 121 | for i := len(data) - 1; i >= 0; i-- { 122 | switch data[i] { 123 | case '\n', '\r', '\t', ',', '{', '[': 124 | return i 125 | } 126 | } 127 | 128 | return 0 129 | } 130 | 131 | // Find position of next character which is not whitespace 132 | func nextToken(data []byte) int { 133 | for i, c := range data { 134 | switch c { 135 | case ' ', '\n', '\r', '\t': 136 | continue 137 | default: 138 | return i 139 | } 140 | } 141 | 142 | return -1 143 | } 144 | 145 | // Find position of last character which is not whitespace 146 | func lastToken(data []byte) int { 147 | for i := len(data) - 1; i >= 0; i-- { 148 | switch data[i] { 149 | case ' ', '\n', '\r', '\t': 150 | continue 151 | default: 152 | return i 153 | } 154 | } 155 | 156 | return -1 157 | } 158 | 159 | // Tries to find the end of string 160 | // Support if string contains escaped quote symbols. 161 | func stringEnd(data []byte) (int, bool) { 162 | escaped := false 163 | for i, c := range data { 164 | if c == '"' { 165 | if !escaped { 166 | return i + 1, false 167 | } else { 168 | j := i - 1 169 | for { 170 | if j < 0 || data[j] != '\\' { 171 | return i + 1, true // even number of backslashes 172 | } 173 | j-- 174 | if j < 0 || data[j] != '\\' { 175 | break // odd number of backslashes 176 | } 177 | j-- 178 | 179 | } 180 | } 181 | } else if c == '\\' { 182 | escaped = true 183 | } 184 | } 185 | 186 | return -1, escaped 187 | } 188 | 189 | // Find end of the data structure, array or object. 190 | // For array openSym and closeSym will be '[' and ']', for object '{' and '}' 191 | func blockEnd(data []byte, openSym byte, closeSym byte) int { 192 | level := 0 193 | i := 0 194 | ln := len(data) 195 | 196 | for i < ln { 197 | switch data[i] { 198 | case '"': // If inside string, skip it 199 | se, _ := stringEnd(data[i+1:]) 200 | if se == -1 { 201 | return -1 202 | } 203 | i += se 204 | case openSym: // If open symbol, increase level 205 | level++ 206 | case closeSym: // If close symbol, increase level 207 | level-- 208 | 209 | // If we have returned to the original level, we're done 210 | if level == 0 { 211 | return i + 1 212 | } 213 | } 214 | i++ 215 | } 216 | 217 | return -1 218 | } 219 | 220 | func searchKeys(data []byte, keys ...string) int { 221 | keyLevel := 0 222 | level := 0 223 | i := 0 224 | ln := len(data) 225 | lk := len(keys) 226 | lastMatched := true 227 | 228 | if lk == 0 { 229 | return 0 230 | } 231 | 232 | var stackbuf [unescapeStackBufSize]byte // stack-allocated array for allocation-free unescaping of small strings 233 | 234 | for i < ln { 235 | switch data[i] { 236 | case '"': 237 | i++ 238 | keyBegin := i 239 | 240 | strEnd, keyEscaped := stringEnd(data[i:]) 241 | if strEnd == -1 { 242 | return -1 243 | } 244 | i += strEnd 245 | keyEnd := i - 1 246 | 247 | valueOffset := nextToken(data[i:]) 248 | if valueOffset == -1 { 249 | return -1 250 | } 251 | 252 | i += valueOffset 253 | 254 | // if string is a key 255 | if data[i] == ':' { 256 | if level < 1 { 257 | return -1 258 | } 259 | 260 | key := data[keyBegin:keyEnd] 261 | 262 | // for unescape: if there are no escape sequences, this is cheap; if there are, it is a 263 | // bit more expensive, but causes no allocations unless len(key) > unescapeStackBufSize 264 | var keyUnesc []byte 265 | if !keyEscaped { 266 | keyUnesc = key 267 | } else if ku, err := Unescape(key, stackbuf[:]); err != nil { 268 | return -1 269 | } else { 270 | keyUnesc = ku 271 | } 272 | 273 | if level <= len(keys) { 274 | if equalStr(&keyUnesc, keys[level-1]) { 275 | lastMatched = true 276 | 277 | // if key level match 278 | if keyLevel == level-1 { 279 | keyLevel++ 280 | // If we found all keys in path 281 | if keyLevel == lk { 282 | return i + 1 283 | } 284 | } 285 | } else { 286 | lastMatched = false 287 | } 288 | } else { 289 | return -1 290 | } 291 | } else { 292 | i-- 293 | } 294 | case '{': 295 | 296 | // in case parent key is matched then only we will increase the level otherwise can directly 297 | // can move to the end of this block 298 | if !lastMatched { 299 | end := blockEnd(data[i:], '{', '}') 300 | if end == -1 { 301 | return -1 302 | } 303 | i += end - 1 304 | } else { 305 | level++ 306 | } 307 | case '}': 308 | level-- 309 | if level == keyLevel { 310 | keyLevel-- 311 | } 312 | case '[': 313 | // If we want to get array element by index 314 | if keyLevel == level && keys[level][0] == '[' { 315 | keyLen := len(keys[level]) 316 | if keyLen < 3 || keys[level][0] != '[' || keys[level][keyLen-1] != ']' { 317 | return -1 318 | } 319 | aIdx, err := strconv.Atoi(keys[level][1 : keyLen-1]) 320 | if err != nil { 321 | return -1 322 | } 323 | var curIdx int 324 | var valueFound []byte 325 | var valueOffset int 326 | curI := i 327 | ArrayEach(data[i:], func(value []byte, dataType ValueType, offset int, err error) { 328 | if curIdx == aIdx { 329 | valueFound = value 330 | valueOffset = offset 331 | if dataType == String { 332 | valueOffset = valueOffset - 2 333 | valueFound = data[curI+valueOffset : curI+valueOffset+len(value)+2] 334 | } 335 | } 336 | curIdx += 1 337 | }) 338 | 339 | if valueFound == nil { 340 | return -1 341 | } else { 342 | subIndex := searchKeys(valueFound, keys[level+1:]...) 343 | if subIndex < 0 { 344 | return -1 345 | } 346 | return i + valueOffset + subIndex 347 | } 348 | } else { 349 | // Do not search for keys inside arrays 350 | if arraySkip := blockEnd(data[i:], '[', ']'); arraySkip == -1 { 351 | return -1 352 | } else { 353 | i += arraySkip - 1 354 | } 355 | } 356 | case ':': // If encountered, JSON data is malformed 357 | return -1 358 | } 359 | 360 | i++ 361 | } 362 | 363 | return -1 364 | } 365 | 366 | func sameTree(p1, p2 []string) bool { 367 | minLen := len(p1) 368 | if len(p2) < minLen { 369 | minLen = len(p2) 370 | } 371 | 372 | for pi_1, p_1 := range p1[:minLen] { 373 | if p2[pi_1] != p_1 { 374 | return false 375 | } 376 | } 377 | 378 | return true 379 | } 380 | 381 | const stackArraySize = 128 382 | 383 | func EachKey(data []byte, cb func(int, []byte, ValueType, error), paths ...[]string) int { 384 | var x struct{} 385 | var level, pathsMatched, i int 386 | ln := len(data) 387 | 388 | pathFlags := make([]bool, stackArraySize)[:] 389 | if len(paths) > cap(pathFlags) { 390 | pathFlags = make([]bool, len(paths))[:] 391 | } 392 | pathFlags = pathFlags[0:len(paths)] 393 | 394 | var maxPath int 395 | for _, p := range paths { 396 | if len(p) > maxPath { 397 | maxPath = len(p) 398 | } 399 | } 400 | 401 | pathsBuf := make([]string, stackArraySize)[:] 402 | if maxPath > cap(pathsBuf) { 403 | pathsBuf = make([]string, maxPath)[:] 404 | } 405 | pathsBuf = pathsBuf[0:maxPath] 406 | 407 | for i < ln { 408 | switch data[i] { 409 | case '"': 410 | i++ 411 | keyBegin := i 412 | 413 | strEnd, keyEscaped := stringEnd(data[i:]) 414 | if strEnd == -1 { 415 | return -1 416 | } 417 | i += strEnd 418 | 419 | keyEnd := i - 1 420 | 421 | valueOffset := nextToken(data[i:]) 422 | if valueOffset == -1 { 423 | return -1 424 | } 425 | 426 | i += valueOffset 427 | 428 | // if string is a key, and key level match 429 | if data[i] == ':' { 430 | match := -1 431 | key := data[keyBegin:keyEnd] 432 | 433 | // for unescape: if there are no escape sequences, this is cheap; if there are, it is a 434 | // bit more expensive, but causes no allocations unless len(key) > unescapeStackBufSize 435 | var keyUnesc []byte 436 | if !keyEscaped { 437 | keyUnesc = key 438 | } else { 439 | var stackbuf [unescapeStackBufSize]byte 440 | if ku, err := Unescape(key, stackbuf[:]); err != nil { 441 | return -1 442 | } else { 443 | keyUnesc = ku 444 | } 445 | } 446 | 447 | if maxPath >= level { 448 | if level < 1 { 449 | cb(-1, nil, Unknown, MalformedJsonError) 450 | return -1 451 | } 452 | 453 | pathsBuf[level-1] = bytesToString(&keyUnesc) 454 | for pi, p := range paths { 455 | if len(p) != level || pathFlags[pi] || !equalStr(&keyUnesc, p[level-1]) || !sameTree(p, pathsBuf[:level]) { 456 | continue 457 | } 458 | 459 | match = pi 460 | 461 | pathsMatched++ 462 | pathFlags[pi] = true 463 | 464 | v, dt, _, e := Get(data[i+1:]) 465 | cb(pi, v, dt, e) 466 | 467 | if pathsMatched == len(paths) { 468 | break 469 | } 470 | } 471 | if pathsMatched == len(paths) { 472 | return i 473 | } 474 | } 475 | 476 | if match == -1 { 477 | tokenOffset := nextToken(data[i+1:]) 478 | i += tokenOffset 479 | 480 | if data[i] == '{' { 481 | blockSkip := blockEnd(data[i:], '{', '}') 482 | i += blockSkip + 1 483 | } 484 | } 485 | 486 | if i < ln { 487 | switch data[i] { 488 | case '{', '}', '[', '"': 489 | i-- 490 | } 491 | } 492 | } else { 493 | i-- 494 | } 495 | case '{': 496 | level++ 497 | case '}': 498 | level-- 499 | case '[': 500 | var ok bool 501 | arrIdxFlags := make(map[int]struct{}) 502 | 503 | pIdxFlags := make([]bool, stackArraySize)[:] 504 | if len(paths) > cap(pIdxFlags) { 505 | pIdxFlags = make([]bool, len(paths))[:] 506 | } 507 | pIdxFlags = pIdxFlags[0:len(paths)] 508 | 509 | if level < 0 { 510 | cb(-1, nil, Unknown, MalformedJsonError) 511 | return -1 512 | } 513 | 514 | for pi, p := range paths { 515 | if len(p) < level+1 || pathFlags[pi] || p[level][0] != '[' || !sameTree(p, pathsBuf[:level]) { 516 | continue 517 | } 518 | if len(p[level]) >= 2 { 519 | aIdx, _ := strconv.Atoi(p[level][1 : len(p[level])-1]) 520 | arrIdxFlags[aIdx] = x 521 | pIdxFlags[pi] = true 522 | } 523 | } 524 | 525 | if len(arrIdxFlags) > 0 { 526 | level++ 527 | 528 | var curIdx int 529 | arrOff, _ := ArrayEach(data[i:], func(value []byte, dataType ValueType, offset int, err error) { 530 | if _, ok = arrIdxFlags[curIdx]; ok { 531 | for pi, p := range paths { 532 | if pIdxFlags[pi] { 533 | aIdx, _ := strconv.Atoi(p[level-1][1 : len(p[level-1])-1]) 534 | 535 | if curIdx == aIdx { 536 | of := searchKeys(value, p[level:]...) 537 | 538 | pathsMatched++ 539 | pathFlags[pi] = true 540 | 541 | if of != -1 { 542 | v, dt, _, e := Get(value[of:]) 543 | cb(pi, v, dt, e) 544 | } 545 | } 546 | } 547 | } 548 | } 549 | 550 | curIdx += 1 551 | }) 552 | 553 | if pathsMatched == len(paths) { 554 | return i 555 | } 556 | 557 | i += arrOff - 1 558 | } else { 559 | // Do not search for keys inside arrays 560 | if arraySkip := blockEnd(data[i:], '[', ']'); arraySkip == -1 { 561 | return -1 562 | } else { 563 | i += arraySkip - 1 564 | } 565 | } 566 | case ']': 567 | level-- 568 | } 569 | 570 | i++ 571 | } 572 | 573 | return -1 574 | } 575 | 576 | // Data types available in valid JSON data. 577 | type ValueType int 578 | 579 | const ( 580 | NotExist = ValueType(iota) 581 | String 582 | Number 583 | Object 584 | Array 585 | Boolean 586 | Null 587 | Unknown 588 | ) 589 | 590 | func (vt ValueType) String() string { 591 | switch vt { 592 | case NotExist: 593 | return "non-existent" 594 | case String: 595 | return "string" 596 | case Number: 597 | return "number" 598 | case Object: 599 | return "object" 600 | case Array: 601 | return "array" 602 | case Boolean: 603 | return "boolean" 604 | case Null: 605 | return "null" 606 | default: 607 | return "unknown" 608 | } 609 | } 610 | 611 | var ( 612 | trueLiteral = []byte("true") 613 | falseLiteral = []byte("false") 614 | nullLiteral = []byte("null") 615 | ) 616 | 617 | func createInsertComponent(keys []string, setValue []byte, comma, object bool) []byte { 618 | isIndex := string(keys[0][0]) == "[" 619 | offset := 0 620 | lk := calcAllocateSpace(keys, setValue, comma, object) 621 | buffer := make([]byte, lk, lk) 622 | if comma { 623 | offset += WriteToBuffer(buffer[offset:], ",") 624 | } 625 | if isIndex && !comma { 626 | offset += WriteToBuffer(buffer[offset:], "[") 627 | } else { 628 | if object { 629 | offset += WriteToBuffer(buffer[offset:], "{") 630 | } 631 | if !isIndex { 632 | offset += WriteToBuffer(buffer[offset:], "\"") 633 | offset += WriteToBuffer(buffer[offset:], keys[0]) 634 | offset += WriteToBuffer(buffer[offset:], "\":") 635 | } 636 | } 637 | 638 | for i := 1; i < len(keys); i++ { 639 | if string(keys[i][0]) == "[" { 640 | offset += WriteToBuffer(buffer[offset:], "[") 641 | } else { 642 | offset += WriteToBuffer(buffer[offset:], "{\"") 643 | offset += WriteToBuffer(buffer[offset:], keys[i]) 644 | offset += WriteToBuffer(buffer[offset:], "\":") 645 | } 646 | } 647 | offset += WriteToBuffer(buffer[offset:], string(setValue)) 648 | for i := len(keys) - 1; i > 0; i-- { 649 | if string(keys[i][0]) == "[" { 650 | offset += WriteToBuffer(buffer[offset:], "]") 651 | } else { 652 | offset += WriteToBuffer(buffer[offset:], "}") 653 | } 654 | } 655 | if isIndex && !comma { 656 | offset += WriteToBuffer(buffer[offset:], "]") 657 | } 658 | if object && !isIndex { 659 | offset += WriteToBuffer(buffer[offset:], "}") 660 | } 661 | return buffer 662 | } 663 | 664 | func calcAllocateSpace(keys []string, setValue []byte, comma, object bool) int { 665 | isIndex := string(keys[0][0]) == "[" 666 | lk := 0 667 | if comma { 668 | // , 669 | lk += 1 670 | } 671 | if isIndex && !comma { 672 | // [] 673 | lk += 2 674 | } else { 675 | if object { 676 | // { 677 | lk += 1 678 | } 679 | if !isIndex { 680 | // "keys[0]" 681 | lk += len(keys[0]) + 3 682 | } 683 | } 684 | 685 | lk += len(setValue) 686 | for i := 1; i < len(keys); i++ { 687 | if string(keys[i][0]) == "[" { 688 | // [] 689 | lk += 2 690 | } else { 691 | // {"keys[i]":setValue} 692 | lk += len(keys[i]) + 5 693 | } 694 | } 695 | 696 | if object && !isIndex { 697 | // } 698 | lk += 1 699 | } 700 | 701 | return lk 702 | } 703 | 704 | func WriteToBuffer(buffer []byte, str string) int { 705 | copy(buffer, str) 706 | return len(str) 707 | } 708 | 709 | /* 710 | 711 | Del - Receives existing data structure, path to delete. 712 | 713 | Returns: 714 | `data` - return modified data 715 | 716 | */ 717 | func Delete(data []byte, keys ...string) []byte { 718 | lk := len(keys) 719 | if lk == 0 { 720 | return data[:0] 721 | } 722 | 723 | array := false 724 | if len(keys[lk-1]) > 0 && string(keys[lk-1][0]) == "[" { 725 | array = true 726 | } 727 | 728 | var startOffset, keyOffset int 729 | endOffset := len(data) 730 | var err error 731 | if !array { 732 | if len(keys) > 1 { 733 | _, _, startOffset, endOffset, err = internalGet(data, keys[:lk-1]...) 734 | if err == KeyPathNotFoundError { 735 | // problem parsing the data 736 | return data 737 | } 738 | } 739 | 740 | keyOffset, err = findKeyStart(data[startOffset:endOffset], keys[lk-1]) 741 | if err == KeyPathNotFoundError { 742 | // problem parsing the data 743 | return data 744 | } 745 | keyOffset += startOffset 746 | _, _, _, subEndOffset, _ := internalGet(data[startOffset:endOffset], keys[lk-1]) 747 | endOffset = startOffset + subEndOffset 748 | tokEnd := tokenEnd(data[endOffset:]) 749 | tokStart := findTokenStart(data[:keyOffset], ","[0]) 750 | 751 | if data[endOffset+tokEnd] == ","[0] { 752 | endOffset += tokEnd + 1 753 | } else if data[endOffset+tokEnd] == " "[0] && len(data) > endOffset+tokEnd+1 && data[endOffset+tokEnd+1] == ","[0] { 754 | endOffset += tokEnd + 2 755 | } else if data[endOffset+tokEnd] == "}"[0] && data[tokStart] == ","[0] { 756 | keyOffset = tokStart 757 | } 758 | } else { 759 | _, _, keyOffset, endOffset, err = internalGet(data, keys...) 760 | if err == KeyPathNotFoundError { 761 | // problem parsing the data 762 | return data 763 | } 764 | 765 | tokEnd := tokenEnd(data[endOffset:]) 766 | tokStart := findTokenStart(data[:keyOffset], ","[0]) 767 | 768 | if data[endOffset+tokEnd] == ","[0] { 769 | endOffset += tokEnd + 1 770 | } else if data[endOffset+tokEnd] == "]"[0] && data[tokStart] == ","[0] { 771 | keyOffset = tokStart 772 | } 773 | } 774 | 775 | // We need to remove remaining trailing comma if we delete las element in the object 776 | prevTok := lastToken(data[:keyOffset]) 777 | remainedValue := data[endOffset:] 778 | 779 | var newOffset int 780 | if nextToken(remainedValue) > -1 && remainedValue[nextToken(remainedValue)] == '}' && data[prevTok] == ',' { 781 | newOffset = prevTok 782 | } else { 783 | newOffset = prevTok + 1 784 | } 785 | 786 | // We have to make a copy here if we don't want to mangle the original data, because byte slices are 787 | // accessed by reference and not by value 788 | dataCopy := make([]byte, len(data)) 789 | copy(dataCopy, data) 790 | data = append(dataCopy[:newOffset], dataCopy[endOffset:]...) 791 | 792 | return data 793 | } 794 | 795 | /* 796 | 797 | Set - Receives existing data structure, path to set, and data to set at that key. 798 | 799 | Returns: 800 | `value` - modified byte array 801 | `err` - On any parsing error 802 | 803 | */ 804 | func Set(data []byte, setValue []byte, keys ...string) (value []byte, err error) { 805 | // ensure keys are set 806 | if len(keys) == 0 { 807 | return nil, KeyPathNotFoundError 808 | } 809 | 810 | _, _, startOffset, endOffset, err := internalGet(data, keys...) 811 | if err != nil { 812 | if err != KeyPathNotFoundError { 813 | // problem parsing the data 814 | return nil, err 815 | } 816 | // full path doesnt exist 817 | // does any subpath exist? 818 | var depth int 819 | for i := range keys { 820 | _, _, start, end, sErr := internalGet(data, keys[:i+1]...) 821 | if sErr != nil { 822 | break 823 | } else { 824 | endOffset = end 825 | startOffset = start 826 | depth++ 827 | } 828 | } 829 | comma := true 830 | object := false 831 | if endOffset == -1 { 832 | firstToken := nextToken(data) 833 | // We can't set a top-level key if data isn't an object 834 | if firstToken < 0 || data[firstToken] != '{' { 835 | return nil, KeyPathNotFoundError 836 | } 837 | // Don't need a comma if the input is an empty object 838 | secondToken := firstToken + 1 + nextToken(data[firstToken+1:]) 839 | if data[secondToken] == '}' { 840 | comma = false 841 | } 842 | // Set the top level key at the end (accounting for any trailing whitespace) 843 | // This assumes last token is valid like '}', could check and return error 844 | endOffset = lastToken(data) 845 | } 846 | depthOffset := endOffset 847 | if depth != 0 { 848 | // if subpath is a non-empty object, add to it 849 | // or if subpath is a non-empty array, add to it 850 | if (data[startOffset] == '{' && data[startOffset+1+nextToken(data[startOffset+1:])] != '}') || 851 | (data[startOffset] == '[' && data[startOffset+1+nextToken(data[startOffset+1:])] == '{') && keys[depth:][0][0] == 91 { 852 | depthOffset-- 853 | startOffset = depthOffset 854 | // otherwise, over-write it with a new object 855 | } else { 856 | comma = false 857 | object = true 858 | } 859 | } else { 860 | startOffset = depthOffset 861 | } 862 | value = append(data[:startOffset], append(createInsertComponent(keys[depth:], setValue, comma, object), data[depthOffset:]...)...) 863 | } else { 864 | // path currently exists 865 | startComponent := data[:startOffset] 866 | endComponent := data[endOffset:] 867 | 868 | value = make([]byte, len(startComponent)+len(endComponent)+len(setValue)) 869 | newEndOffset := startOffset + len(setValue) 870 | copy(value[0:startOffset], startComponent) 871 | copy(value[startOffset:newEndOffset], setValue) 872 | copy(value[newEndOffset:], endComponent) 873 | } 874 | return value, nil 875 | } 876 | 877 | func getType(data []byte, offset int) ([]byte, ValueType, int, error) { 878 | var dataType ValueType 879 | endOffset := offset 880 | 881 | // if string value 882 | if data[offset] == '"' { 883 | dataType = String 884 | if idx, _ := stringEnd(data[offset+1:]); idx != -1 { 885 | endOffset += idx + 1 886 | } else { 887 | return nil, dataType, offset, MalformedStringError 888 | } 889 | } else if data[offset] == '[' { // if array value 890 | dataType = Array 891 | // break label, for stopping nested loops 892 | endOffset = blockEnd(data[offset:], '[', ']') 893 | 894 | if endOffset == -1 { 895 | return nil, dataType, offset, MalformedArrayError 896 | } 897 | 898 | endOffset += offset 899 | } else if data[offset] == '{' { // if object value 900 | dataType = Object 901 | // break label, for stopping nested loops 902 | endOffset = blockEnd(data[offset:], '{', '}') 903 | 904 | if endOffset == -1 { 905 | return nil, dataType, offset, MalformedObjectError 906 | } 907 | 908 | endOffset += offset 909 | } else { 910 | // Number, Boolean or None 911 | end := tokenEnd(data[endOffset:]) 912 | 913 | if end == -1 { 914 | return nil, dataType, offset, MalformedValueError 915 | } 916 | 917 | value := data[offset : endOffset+end] 918 | 919 | switch data[offset] { 920 | case 't', 'f': // true or false 921 | if bytes.Equal(value, trueLiteral) || bytes.Equal(value, falseLiteral) { 922 | dataType = Boolean 923 | } else { 924 | return nil, Unknown, offset, UnknownValueTypeError 925 | } 926 | case 'u', 'n': // undefined or null 927 | if bytes.Equal(value, nullLiteral) { 928 | dataType = Null 929 | } else { 930 | return nil, Unknown, offset, UnknownValueTypeError 931 | } 932 | case '0', '1', '2', '3', '4', '5', '6', '7', '8', '9', '-': 933 | dataType = Number 934 | default: 935 | return nil, Unknown, offset, UnknownValueTypeError 936 | } 937 | 938 | endOffset += end 939 | } 940 | return data[offset:endOffset], dataType, endOffset, nil 941 | } 942 | 943 | /* 944 | Get - Receives data structure, and key path to extract value from. 945 | 946 | Returns: 947 | `value` - Pointer to original data structure containing key value, or just empty slice if nothing found or error 948 | `dataType` - Can be: `NotExist`, `String`, `Number`, `Object`, `Array`, `Boolean` or `Null` 949 | `offset` - Offset from provided data structure where key value ends. Used mostly internally, for example for `ArrayEach` helper. 950 | `err` - If key not found or any other parsing issue it should return error. If key not found it also sets `dataType` to `NotExist` 951 | 952 | Accept multiple keys to specify path to JSON value (in case of quering nested structures). 953 | If no keys provided it will try to extract closest JSON value (simple ones or object/array), useful for reading streams or arrays, see `ArrayEach` implementation. 954 | */ 955 | func Get(data []byte, keys ...string) (value []byte, dataType ValueType, offset int, err error) { 956 | a, b, _, d, e := internalGet(data, keys...) 957 | return a, b, d, e 958 | } 959 | 960 | func internalGet(data []byte, keys ...string) (value []byte, dataType ValueType, offset, endOffset int, err error) { 961 | if len(keys) > 0 { 962 | if offset = searchKeys(data, keys...); offset == -1 { 963 | return nil, NotExist, -1, -1, KeyPathNotFoundError 964 | } 965 | } 966 | 967 | // Go to closest value 968 | nO := nextToken(data[offset:]) 969 | if nO == -1 { 970 | return nil, NotExist, offset, -1, MalformedJsonError 971 | } 972 | 973 | offset += nO 974 | value, dataType, endOffset, err = getType(data, offset) 975 | if err != nil { 976 | return value, dataType, offset, endOffset, err 977 | } 978 | 979 | // Strip quotes from string values 980 | if dataType == String { 981 | value = value[1 : len(value)-1] 982 | } 983 | 984 | return value[:len(value):len(value)], dataType, offset, endOffset, nil 985 | } 986 | 987 | // ArrayEach is used when iterating arrays, accepts a callback function with the same return arguments as `Get`. 988 | func ArrayEach(data []byte, cb func(value []byte, dataType ValueType, offset int, err error), keys ...string) (offset int, err error) { 989 | if len(data) == 0 { 990 | return -1, MalformedObjectError 991 | } 992 | 993 | nT := nextToken(data) 994 | if nT == -1 { 995 | return -1, MalformedJsonError 996 | } 997 | 998 | offset = nT + 1 999 | 1000 | if len(keys) > 0 { 1001 | if offset = searchKeys(data, keys...); offset == -1 { 1002 | return offset, KeyPathNotFoundError 1003 | } 1004 | 1005 | // Go to closest value 1006 | nO := nextToken(data[offset:]) 1007 | if nO == -1 { 1008 | return offset, MalformedJsonError 1009 | } 1010 | 1011 | offset += nO 1012 | 1013 | if data[offset] != '[' { 1014 | return offset, MalformedArrayError 1015 | } 1016 | 1017 | offset++ 1018 | } 1019 | 1020 | nO := nextToken(data[offset:]) 1021 | if nO == -1 { 1022 | return offset, MalformedJsonError 1023 | } 1024 | 1025 | offset += nO 1026 | 1027 | if data[offset] == ']' { 1028 | return offset, nil 1029 | } 1030 | 1031 | for true { 1032 | v, t, o, e := Get(data[offset:]) 1033 | 1034 | if e != nil { 1035 | return offset, e 1036 | } 1037 | 1038 | if o == 0 { 1039 | break 1040 | } 1041 | 1042 | if t != NotExist { 1043 | cb(v, t, offset+o-len(v), e) 1044 | } 1045 | 1046 | if e != nil { 1047 | break 1048 | } 1049 | 1050 | offset += o 1051 | 1052 | skipToToken := nextToken(data[offset:]) 1053 | if skipToToken == -1 { 1054 | return offset, MalformedArrayError 1055 | } 1056 | offset += skipToToken 1057 | 1058 | if data[offset] == ']' { 1059 | break 1060 | } 1061 | 1062 | if data[offset] != ',' { 1063 | return offset, MalformedArrayError 1064 | } 1065 | 1066 | offset++ 1067 | } 1068 | 1069 | return offset, nil 1070 | } 1071 | 1072 | // ObjectEach iterates over the key-value pairs of a JSON object, invoking a given callback for each such entry 1073 | func ObjectEach(data []byte, callback func(key []byte, value []byte, dataType ValueType, offset int) error, keys ...string) (err error) { 1074 | offset := 0 1075 | 1076 | // Descend to the desired key, if requested 1077 | if len(keys) > 0 { 1078 | if off := searchKeys(data, keys...); off == -1 { 1079 | return KeyPathNotFoundError 1080 | } else { 1081 | offset = off 1082 | } 1083 | } 1084 | 1085 | // Validate and skip past opening brace 1086 | if off := nextToken(data[offset:]); off == -1 { 1087 | return MalformedObjectError 1088 | } else if offset += off; data[offset] != '{' { 1089 | return MalformedObjectError 1090 | } else { 1091 | offset++ 1092 | } 1093 | 1094 | // Skip to the first token inside the object, or stop if we find the ending brace 1095 | if off := nextToken(data[offset:]); off == -1 { 1096 | return MalformedJsonError 1097 | } else if offset += off; data[offset] == '}' { 1098 | return nil 1099 | } 1100 | 1101 | // Loop pre-condition: data[offset] points to what should be either the next entry's key, or the closing brace (if it's anything else, the JSON is malformed) 1102 | for offset < len(data) { 1103 | // Step 1: find the next key 1104 | var key []byte 1105 | 1106 | // Check what the the next token is: start of string, end of object, or something else (error) 1107 | switch data[offset] { 1108 | case '"': 1109 | offset++ // accept as string and skip opening quote 1110 | case '}': 1111 | return nil // we found the end of the object; stop and return success 1112 | default: 1113 | return MalformedObjectError 1114 | } 1115 | 1116 | // Find the end of the key string 1117 | var keyEscaped bool 1118 | if off, esc := stringEnd(data[offset:]); off == -1 { 1119 | return MalformedJsonError 1120 | } else { 1121 | key, keyEscaped = data[offset:offset+off-1], esc 1122 | offset += off 1123 | } 1124 | 1125 | // Unescape the string if needed 1126 | if keyEscaped { 1127 | var stackbuf [unescapeStackBufSize]byte // stack-allocated array for allocation-free unescaping of small strings 1128 | if keyUnescaped, err := Unescape(key, stackbuf[:]); err != nil { 1129 | return MalformedStringEscapeError 1130 | } else { 1131 | key = keyUnescaped 1132 | } 1133 | } 1134 | 1135 | // Step 2: skip the colon 1136 | if off := nextToken(data[offset:]); off == -1 { 1137 | return MalformedJsonError 1138 | } else if offset += off; data[offset] != ':' { 1139 | return MalformedJsonError 1140 | } else { 1141 | offset++ 1142 | } 1143 | 1144 | // Step 3: find the associated value, then invoke the callback 1145 | if value, valueType, off, err := Get(data[offset:]); err != nil { 1146 | return err 1147 | } else if err := callback(key, value, valueType, offset+off); err != nil { // Invoke the callback here! 1148 | return err 1149 | } else { 1150 | offset += off 1151 | } 1152 | 1153 | // Step 4: skip over the next comma to the following token, or stop if we hit the ending brace 1154 | if off := nextToken(data[offset:]); off == -1 { 1155 | return MalformedArrayError 1156 | } else { 1157 | offset += off 1158 | switch data[offset] { 1159 | case '}': 1160 | return nil // Stop if we hit the close brace 1161 | case ',': 1162 | offset++ // Ignore the comma 1163 | default: 1164 | return MalformedObjectError 1165 | } 1166 | } 1167 | 1168 | // Skip to the next token after the comma 1169 | if off := nextToken(data[offset:]); off == -1 { 1170 | return MalformedArrayError 1171 | } else { 1172 | offset += off 1173 | } 1174 | } 1175 | 1176 | return MalformedObjectError // we shouldn't get here; it's expected that we will return via finding the ending brace 1177 | } 1178 | 1179 | // GetUnsafeString returns the value retrieved by `Get`, use creates string without memory allocation by mapping string to slice memory. It does not handle escape symbols. 1180 | func GetUnsafeString(data []byte, keys ...string) (val string, err error) { 1181 | v, _, _, e := Get(data, keys...) 1182 | 1183 | if e != nil { 1184 | return "", e 1185 | } 1186 | 1187 | return bytesToString(&v), nil 1188 | } 1189 | 1190 | // GetString returns the value retrieved by `Get`, cast to a string if possible, trying to properly handle escape and utf8 symbols 1191 | // If key data type do not match, it will return an error. 1192 | func GetString(data []byte, keys ...string) (val string, err error) { 1193 | v, t, _, e := Get(data, keys...) 1194 | 1195 | if e != nil { 1196 | return "", e 1197 | } 1198 | 1199 | if t != String { 1200 | if t == Null { 1201 | return "", NullValueError 1202 | } 1203 | return "", fmt.Errorf("Value is not a string: %s", string(v)) 1204 | } 1205 | 1206 | // If no escapes return raw content 1207 | if bytes.IndexByte(v, '\\') == -1 { 1208 | return string(v), nil 1209 | } 1210 | 1211 | return ParseString(v) 1212 | } 1213 | 1214 | // GetFloat returns the value retrieved by `Get`, cast to a float64 if possible. 1215 | // The offset is the same as in `Get`. 1216 | // If key data type do not match, it will return an error. 1217 | func GetFloat(data []byte, keys ...string) (val float64, err error) { 1218 | v, t, _, e := Get(data, keys...) 1219 | 1220 | if e != nil { 1221 | return 0, e 1222 | } 1223 | 1224 | if t != Number { 1225 | if t == Null { 1226 | return 0, NullValueError 1227 | } 1228 | return 0, fmt.Errorf("Value is not a number: %s", string(v)) 1229 | } 1230 | 1231 | return ParseFloat(v) 1232 | } 1233 | 1234 | // GetInt returns the value retrieved by `Get`, cast to a int64 if possible. 1235 | // If key data type do not match, it will return an error. 1236 | func GetInt(data []byte, keys ...string) (val int64, err error) { 1237 | v, t, _, e := Get(data, keys...) 1238 | 1239 | if e != nil { 1240 | return 0, e 1241 | } 1242 | 1243 | if t != Number { 1244 | if t == Null { 1245 | return 0, NullValueError 1246 | } 1247 | return 0, fmt.Errorf("Value is not a number: %s", string(v)) 1248 | } 1249 | 1250 | return ParseInt(v) 1251 | } 1252 | 1253 | // GetBoolean returns the value retrieved by `Get`, cast to a bool if possible. 1254 | // The offset is the same as in `Get`. 1255 | // If key data type do not match, it will return error. 1256 | func GetBoolean(data []byte, keys ...string) (val bool, err error) { 1257 | v, t, _, e := Get(data, keys...) 1258 | 1259 | if e != nil { 1260 | return false, e 1261 | } 1262 | 1263 | if t != Boolean { 1264 | if t == Null { 1265 | return false, NullValueError 1266 | } 1267 | return false, fmt.Errorf("Value is not a boolean: %s", string(v)) 1268 | } 1269 | 1270 | return ParseBoolean(v) 1271 | } 1272 | 1273 | // ParseBoolean parses a Boolean ValueType into a Go bool (not particularly useful, but here for completeness) 1274 | func ParseBoolean(b []byte) (bool, error) { 1275 | switch { 1276 | case bytes.Equal(b, trueLiteral): 1277 | return true, nil 1278 | case bytes.Equal(b, falseLiteral): 1279 | return false, nil 1280 | default: 1281 | return false, MalformedValueError 1282 | } 1283 | } 1284 | 1285 | // ParseString parses a String ValueType into a Go string (the main parsing work is unescaping the JSON string) 1286 | func ParseString(b []byte) (string, error) { 1287 | var stackbuf [unescapeStackBufSize]byte // stack-allocated array for allocation-free unescaping of small strings 1288 | if bU, err := Unescape(b, stackbuf[:]); err != nil { 1289 | return "", MalformedValueError 1290 | } else { 1291 | return string(bU), nil 1292 | } 1293 | } 1294 | 1295 | // ParseNumber parses a Number ValueType into a Go float64 1296 | func ParseFloat(b []byte) (float64, error) { 1297 | if v, err := parseFloat(&b); err != nil { 1298 | return 0, MalformedValueError 1299 | } else { 1300 | return v, nil 1301 | } 1302 | } 1303 | 1304 | // ParseInt parses a Number ValueType into a Go int64 1305 | func ParseInt(b []byte) (int64, error) { 1306 | if v, ok, overflow := parseInt(b); !ok { 1307 | if overflow { 1308 | return 0, OverflowIntegerError 1309 | } 1310 | return 0, MalformedValueError 1311 | } else { 1312 | return v, nil 1313 | } 1314 | } 1315 | --------------------------------------------------------------------------------