├── .gitmodules ├── Makefile ├── README.md ├── c++ ├── .gitignore ├── Makefile ├── bench.h └── log.cc ├── go ├── .gitignore └── Makefile └── rust ├── .gitignore ├── Cargo.lock ├── Cargo.toml ├── Makefile ├── benches └── bench.rs ├── build.rs └── src ├── lib.rs ├── log.capnp ├── log.fbs └── log_proto.proto /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "c++/rapidjson"] 2 | path = c++/rapidjson 3 | url = https://github.com/erickt/rapidjson.git 4 | branch = master 5 | [submodule "go/goser"] 6 | path = go/goser 7 | url = https://github.com/erickt/goser.git 8 | branch = master 9 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | all: c++ go rust 2 | 3 | .PHONY: c++ go rust 4 | 5 | c++: 6 | cd c++ && $(MAKE) bench 7 | 8 | go: 9 | cd go && $(MAKE) bench 10 | 11 | rust: 12 | cd rust && cargo bench 13 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | This is a suite of serialization benchmarks. It comprises of the following projects: 2 | 3 | * C++ 4 | * [rapidjson](https://github.com/erickt/rapidjson) 5 | * Go 6 | * [encoding/json](http://golang.org/pkg/encoding/json) 7 | * [ffjson](https://github.com/pquerna/ffjson) 8 | * [go-capnproto](https://github.com/glycerine/go-capnproto) 9 | * [gogoprotobuf](http://code.google.com/p/gogoprotobuf/) 10 | * [goprotobuf](http://code.google.com/p/goprotobuf/) 11 | * Rust 12 | * [bincode](https://github.com/TyOverby/bincode) 13 | * [capnproto-rust](https://github.com/dwrensha/capnproto-rust) 14 | * [rust-msgpack](https://github.com/mneumann/rust-msgpack) 15 | * [rust-protobuf](https://github.com/stepancheg/rust-protobuf) 16 | * [rust-serde](https://github.com/erickt/rust-serde) 17 | * [serialize](http://doc.rust-lang.org/serialize/) 18 | * [flatbuffers](https://github.com/google/flatbuffers) 19 | 20 | The initial benchmark suite is derived from 21 | [Goser](https://github.com/cloudflare/goser) benchmark, which is all about 22 | serializing and serializing from this JSON record: 23 | 24 | ```json 25 | { 26 | "timestamp": 25469139677502, 27 | "zone_id": 123456, 28 | "zone_plan": 1, 29 | "http": { 30 | "protocol": 2, 31 | "status": 200, 32 | "host_status": 503, 33 | "up_status": 520, 34 | "method": 1, 35 | "content_type": "text/html", 36 | "user_agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML,like Gecko) Chrome/33.0.1750.146 Safari/537.36", 37 | "referer": "https: //www.cloudflare.com/", 38 | "request_uri": "/cdn-cgi/trace" 39 | }, 40 | "origin": { 41 | "ip": "1.2.3.4", 42 | "port": 8000, 43 | "hostname": "www.example.com", 44 | "protocol": 2 45 | }, 46 | "country": 238, 47 | "cache_status": 3, 48 | "server_ip": "192.168.1.1", 49 | "server_name": "metal.cloudflare.com", 50 | "remote_ip": "10.1.2.3", 51 | "bytes_dlv": 123456, 52 | "ray_id": "10c73629cce30078-LAX" 53 | } 54 | ``` 55 | 56 | To run the tests, you'll need to install: 57 | 58 | * [Cap'n Proto](https://capnproto.org/) 59 | * [Protocol Buffers](https://developers.google.com/protocol-buffers/?hl=en) 60 | * [Go](https://golang.org/) 61 | * [Rust](https://rust-lang.org/) 62 | 63 | On OS X and [Homebrew](http://brew.sh/), this can all be done with: 64 | 65 | ``` 66 | brew install capnp protobuf rust go 67 | ``` 68 | 69 | I'm not yet sure what's needed for other operating systems. Once that is 70 | installed, run: 71 | 72 | ``` 73 | % git submodule init 74 | % git submodule update 75 | % make 76 | ``` 77 | 78 | --- 79 | 80 | Current results: 81 | 82 | RapidJSON: 83 | 84 | ``` 85 | log populate : 549 ns/iter 86 | log serialize : 1930 ns/iter = 310 MB/s 87 | log dom deserialize: 3972 ns/iter = 151 MB/s 88 | log sax deserialize: 3821 ns/iter = 157 MB/s 89 | str serialize : 2642 ns/iter = 388 MB/s 90 | str dom deserialize: 3006 ns/iter = 341 MB/s 91 | str sax deserialize: 3536 ns/iter = 290 MB/s 92 | ``` 93 | 94 | Go: 95 | 96 | ``` 97 | BenchmarkPopulatePb 10000000 1235 ns/op 400 B/op 25 allocs/op 98 | BenchmarkPopulateGogopb 50000000 331 ns/op 48 B/op 3 allocs/op 99 | BenchmarkPopulateCapnp 10000000 2225 ns/op 112 B/op 2 allocs/op 100 | BenchmarkMarshalJSON 2000000 6707 ns/op 89.16 MB/s 568 B/op 29 allocs/op 101 | BenchmarkMarshalPb 10000000 1237 ns/op 233.52 MB/s 0 B/op 0 allocs/op 102 | BenchmarkMarshalGogopb 30000000 448 ns/op 644.71 MB/s 320 B/op 1 allocs/op 103 | BenchmarkMarshalCapnp 100000000 114 ns/op 4039.83 MB/s 8 B/op 1 allocs/op 104 | BenchmarkUnmarshalJSON 500000 24924 ns/op 23.99 MB/s 2144 B/op 41 allocs/op 105 | BenchmarkUnmarshalPb 5000000 3486 ns/op 82.89 MB/s 858 B/op 23 allocs/op 106 | BenchmarkUnmarshalGogopb 20000000 1040 ns/op 277.68 MB/s 266 B/op 10 allocs/op 107 | BenchmarkUnmarshalCapnp 20000000 613 ns/op 756.17 MB/s 256 B/op 5 allocs/op 108 | BenchmarkUnmarshalCapnpZeroCopy 50000000 321 ns/op 1445.07 MB/s 88 B/op 3 allocs/op 109 | ``` 110 | 111 | Rust: 112 | 113 | ``` 114 | test goser::bench_clone ... bench: 333 ns/iter (+/- 129) = 1645 MB/s 115 | test goser::bincode::bench_decoder ... bench: 1,399 ns/iter (+/- 571) = 285 MB/s 116 | test goser::bincode::bench_encoder ... bench: 135 ns/iter (+/- 43) = 2962 MB/s 117 | test goser::bincode::bench_populate ... bench: 878 ns/iter (+/- 116) 118 | test goser::bincode_serde::bench_deserialize ... bench: 1,188 ns/iter (+/- 460) = 301 MB/s 119 | test goser::bincode_serde::bench_populate ... bench: 900 ns/iter (+/- 312) 120 | test goser::bincode_serde::bench_serialize ... bench: 170 ns/iter (+/- 46) = 2105 MB/s 121 | test goser::capnp::bench_deserialize ... bench: 344 ns/iter (+/- 56) = 1302 MB/s 122 | test goser::capnp::bench_deserialize_packed ... bench: 812 ns/iter (+/- 360) = 415 MB/s 123 | test goser::capnp::bench_populate ... bench: 644 ns/iter (+/- 344) 124 | test goser::capnp::bench_serialize ... bench: 32 ns/iter (+/- 19) = 14000 MB/s 125 | test goser::capnp::bench_serialize_packed ... bench: 564 ns/iter (+/- 307) = 597 MB/s 126 | test goser::msgpack::bench_decoder ... bench: 2,234 ns/iter (+/- 831) = 128 MB/s 127 | test goser::msgpack::bench_deserializer ... bench: 2,686 ns/iter (+/- 1,117) = 106 MB/s 128 | test goser::msgpack::bench_encoder ... bench: 784 ns/iter (+/- 471) = 366 MB/s 129 | test goser::msgpack::bench_populate ... bench: 1,063 ns/iter (+/- 471) 130 | test goser::msgpack::bench_serializer ... bench: 922 ns/iter (+/- 183) = 311 MB/s 131 | test goser::protobuf::bench_decoder ... bench: 2,016 ns/iter (+/- 554) = 141 MB/s 132 | test goser::protobuf::bench_encoder ... bench: 779 ns/iter (+/- 444) = 367 MB/s 133 | test goser::protobuf::bench_populate ... bench: 908 ns/iter (+/- 264) 134 | test goser::rustc_serialize_json::bench_decoder ... bench: 30,541 ns/iter (+/- 7,753) = 19 MB/s 135 | test goser::rustc_serialize_json::bench_encoder ... bench: 3,469 ns/iter (+/- 1,583) = 174 MB/s 136 | test goser::rustc_serialize_json::bench_populate ... bench: 1,010 ns/iter (+/- 400) 137 | test goser::serde_json::bench_deserializer ... bench: 4,726 ns/iter (+/- 2,393) = 128 MB/s 138 | test goser::serde_json::bench_populate ... bench: 949 ns/iter (+/- 216) 139 | test goser::serde_json::bench_serializer ... bench: 1,966 ns/iter (+/- 692) = 307 MB/s 140 | ``` 141 | -------------------------------------------------------------------------------- /c++/.gitignore: -------------------------------------------------------------------------------- 1 | /log 2 | -------------------------------------------------------------------------------- /c++/Makefile: -------------------------------------------------------------------------------- 1 | all: log 2 | 3 | .PHONY: bench 4 | 5 | bench: log 6 | ./log 7 | 8 | log: log.cc bench.h 9 | c++ -O3 -o log -I rapidjson/include log.cc 10 | 11 | clean: 12 | rm -f log 13 | -------------------------------------------------------------------------------- /c++/bench.h: -------------------------------------------------------------------------------- 1 | #ifndef __BENCH_H__ 2 | #define __BENCH_H__ 3 | 4 | #include 5 | #include 6 | #include 7 | 8 | #ifdef __MACH__ 9 | #include 10 | #include 11 | #endif 12 | 13 | void current_utc_time(struct timespec *ts) { 14 | #ifdef __MACH__ // OS X does not have clock_gettime, use clock_get_time 15 | clock_serv_t cclock; 16 | mach_timespec_t mts; 17 | host_get_clock_service(mach_host_self(), CALENDAR_CLOCK, &cclock); 18 | clock_get_time(cclock, &mts); 19 | mach_port_deallocate(mach_task_self(), cclock); 20 | ts->tv_sec = mts.tv_sec; 21 | ts->tv_nsec = mts.tv_nsec; 22 | #else 23 | clock_gettime(CLOCK_REALTIME, ts); 24 | #endif 25 | } 26 | 27 | template 28 | void bench(const char* name, T& state, void (*f)(T&)) { 29 | int iters = 100000; 30 | 31 | struct timespec start_ts, end_ts; 32 | current_utc_time(&start_ts); 33 | 34 | for (int i = 0; i < iters; ++i) { 35 | (f)(state); 36 | } 37 | 38 | current_utc_time(&end_ts); 39 | 40 | unsigned long sec_delta = end_ts.tv_sec - start_ts.tv_sec; 41 | unsigned long nsec_delta = end_ts.tv_nsec - start_ts.tv_nsec; 42 | unsigned long delta = sec_delta * 1000000000 + nsec_delta; 43 | double delta_per_iter = (double)delta / (double)iters; 44 | 45 | printf("%s:\t%.0f ns/iter\n", name, delta_per_iter); 46 | } 47 | 48 | template 49 | void bench(const char* name, T& state, size_t size, void (*f)(T&)) { 50 | int iters = 100000; 51 | 52 | struct timespec start_ts, end_ts; 53 | current_utc_time(&start_ts); 54 | 55 | for (int i = 0; i < iters; ++i) { 56 | (f)(state); 57 | } 58 | 59 | current_utc_time(&end_ts); 60 | 61 | unsigned long sec_delta = end_ts.tv_sec - start_ts.tv_sec; 62 | unsigned long nsec_delta = end_ts.tv_nsec - start_ts.tv_nsec; 63 | unsigned long delta = sec_delta * 1000000000 + nsec_delta; 64 | double delta_per_iter = (double)delta / (double)iters; 65 | double bandwidth = ((double)size / (double)delta_per_iter) * 1000.0; 66 | 67 | printf("%s:\t%.0f ns/iter = %.0f MB/s\n", name, delta_per_iter, bandwidth); 68 | } 69 | 70 | #endif 71 | -------------------------------------------------------------------------------- /c++/log.cc: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | 6 | #include "rapidjson/rapidjson.h" 7 | #include "rapidjson/error/en.h" 8 | #include "rapidjson/document.h" 9 | #include "rapidjson/reader.h" 10 | #include "rapidjson/writer.h" 11 | #include "rapidjson/stringbuffer.h" 12 | 13 | #include "bench.h" 14 | 15 | using namespace std; 16 | using namespace rapidjson; 17 | 18 | enum HttpProtocol { 19 | HTTP_PROTOCOL_UNKNOWN, 20 | HTTP10, 21 | HTTP11, 22 | }; 23 | 24 | enum HttpMethod { 25 | METHOD_UNKNOWN, 26 | GET, 27 | POST, 28 | DELETE, 29 | PUT, 30 | HEAD, 31 | PURGE, 32 | OPTIONS, 33 | PROPFIND, 34 | MKCOL, 35 | PATCH, 36 | }; 37 | 38 | struct Http { 39 | HttpProtocol protocol; 40 | unsigned status; 41 | unsigned host_status; 42 | unsigned up_status; 43 | HttpMethod method; 44 | std::string content_type; 45 | std::string user_agent; 46 | std::string referer; 47 | std::string request_uri; 48 | 49 | template 50 | void Serialize(Writer& writer) const { 51 | writer.StartObject(); 52 | 53 | writer.String("protocol"); 54 | writer.Uint(protocol); 55 | 56 | writer.String("status"); 57 | writer.Uint(status); 58 | 59 | writer.String("host_status"); 60 | writer.Uint(host_status); 61 | 62 | writer.String("up_status"); 63 | writer.Uint(up_status); 64 | 65 | writer.String("method"); 66 | writer.Uint(method); 67 | 68 | writer.String("content_type"); 69 | writer.String(content_type.c_str()); 70 | 71 | writer.String("user_agent"); 72 | writer.String(user_agent.c_str()); 73 | 74 | writer.String("referer"); 75 | writer.String(referer.c_str()); 76 | 77 | writer.String("request_uri"); 78 | writer.String(request_uri.c_str()); 79 | 80 | writer.EndObject(); 81 | } 82 | 83 | template 84 | void Deserialize(Doc& document) { 85 | protocol = static_cast(document["protocol"].GetUint()); 86 | status = document["status"].GetUint(); 87 | host_status = document["host_status"].GetUint(); 88 | up_status = document["up_status"].GetUint(); 89 | method = static_cast(document["method"].GetUint()); 90 | content_type = document["content_type"].GetString(); 91 | user_agent = document["user_agent"].GetString(); 92 | referer = document["referer"].GetString(); 93 | request_uri = document["request_uri"].GetString(); 94 | } 95 | 96 | enum State { 97 | ExpectObjectStart, 98 | ExpectNameOrObjectEnd, 99 | ExpectProtocol, 100 | ExpectStatus, 101 | ExpectHostStatus, 102 | ExpectUpStatus, 103 | ExpectMethod, 104 | ExpectContentType, 105 | ExpectUserAgent, 106 | ExpectReferer, 107 | ExpectRequestUri, 108 | }; 109 | 110 | struct Handler: BaseReaderHandler<> { 111 | Http& http; 112 | State state; 113 | 114 | Handler(Http& http): http(http), state(ExpectObjectStart) {} 115 | 116 | bool StartObject() { 117 | switch (state) { 118 | case ExpectObjectStart: 119 | state = ExpectNameOrObjectEnd; 120 | return true; 121 | default: 122 | return false; 123 | } 124 | } 125 | 126 | bool Int(int value) { 127 | return Uint64(value); 128 | } 129 | 130 | bool Int64(int64_t value) { 131 | return Uint64(value); 132 | } 133 | 134 | bool Uint(int value) { 135 | return Uint64(value); 136 | } 137 | 138 | bool Uint64(uint64_t value) { 139 | switch (state) { 140 | case ExpectProtocol: 141 | http.protocol = static_cast(value); 142 | state = ExpectNameOrObjectEnd; 143 | return true; 144 | 145 | case ExpectStatus: 146 | http.status = value; 147 | state = ExpectNameOrObjectEnd; 148 | return true; 149 | 150 | case ExpectUpStatus: 151 | http.up_status = value; 152 | state = ExpectNameOrObjectEnd; 153 | return true; 154 | 155 | case ExpectHostStatus: 156 | http.host_status = value; 157 | state = ExpectNameOrObjectEnd; 158 | return true; 159 | 160 | case ExpectMethod: 161 | http.method = static_cast(value); 162 | state = ExpectNameOrObjectEnd; 163 | return true; 164 | 165 | default: 166 | return false; 167 | } 168 | } 169 | 170 | bool String(const char* str, SizeType length, bool) { 171 | switch (state) { 172 | case ExpectContentType: 173 | http.content_type = std::string(str, length); 174 | state = ExpectNameOrObjectEnd; 175 | return true; 176 | 177 | case ExpectUserAgent: 178 | http.user_agent = std::string(str, length); 179 | state = ExpectNameOrObjectEnd; 180 | return true; 181 | 182 | case ExpectReferer: 183 | http.referer = std::string(str, length); 184 | state = ExpectNameOrObjectEnd; 185 | return true; 186 | 187 | case ExpectRequestUri: 188 | http.request_uri = std::string(str, length); 189 | state = ExpectNameOrObjectEnd; 190 | return true; 191 | 192 | default: 193 | return false; 194 | } 195 | } 196 | 197 | bool Key(const char* str, SizeType length, bool) { 198 | switch (state) { 199 | case ExpectNameOrObjectEnd: 200 | if (memcmp(str, "protocol", sizeof("protocol")) == 0) { 201 | state = ExpectProtocol; 202 | } else if (memcmp(str, "status", sizeof("status")) == 0) { 203 | state = ExpectStatus; 204 | } else if (memcmp(str, "host_status", sizeof("host_status")) == 0) { 205 | state = ExpectHostStatus; 206 | } else if (memcmp(str, "up_status", sizeof("up_status")) == 0) { 207 | state = ExpectUpStatus; 208 | } else if (memcmp(str, "method", sizeof("method")) == 0) { 209 | state = ExpectMethod; 210 | } else if (memcmp(str, "content_type", sizeof("content_type")) == 0) { 211 | state = ExpectContentType; 212 | } else if (memcmp(str, "user_agent", sizeof("user_agent")) == 0) { 213 | state = ExpectUserAgent; 214 | } else if (memcmp(str, "referer", sizeof("referer")) == 0) { 215 | state = ExpectReferer; 216 | } else if (memcmp(str, "request_uri", sizeof("request_uri")) == 0) { 217 | state = ExpectRequestUri; 218 | } else { 219 | return false; 220 | } 221 | 222 | return true; 223 | default: 224 | return false; 225 | } 226 | } 227 | 228 | bool EndObject(SizeType) { 229 | return state == ExpectNameOrObjectEnd; 230 | } 231 | 232 | bool Default() { 233 | return false; 234 | } 235 | }; 236 | }; 237 | 238 | enum CacheStatus { 239 | CACHESTATUS_UNKNOWN, 240 | Miss, 241 | Expired, 242 | Hit, 243 | }; 244 | 245 | enum OriginProtocol { 246 | ORIGIN_PROTOCOL_UNKNOWN, 247 | HTTP, 248 | HTTPS, 249 | }; 250 | 251 | struct Origin { 252 | std::string ip; 253 | unsigned port; 254 | std::string hostname; 255 | OriginProtocol protocol; 256 | 257 | template 258 | void Serialize(Writer& writer) const { 259 | writer.StartObject(); 260 | 261 | writer.String("ip"); 262 | writer.String(ip.c_str()); 263 | 264 | writer.String("port"); 265 | writer.Uint(port); 266 | 267 | writer.String("hostname"); 268 | writer.String(hostname.c_str()); 269 | 270 | writer.String("protocol"); 271 | writer.Uint(protocol); 272 | 273 | writer.EndObject(); 274 | } 275 | 276 | template 277 | void Deserialize(Doc& document) { 278 | ip = document["ip"].GetString(); 279 | port = document["port"].GetUint(); 280 | hostname = document["hostname"].GetString(); 281 | protocol = static_cast(document["protocol"].GetUint()); 282 | } 283 | 284 | enum State { 285 | ExpectObjectStart, 286 | ExpectNameOrObjectEnd, 287 | ExpectIp, 288 | ExpectPort, 289 | ExpectHostname, 290 | ExpectProtocol, 291 | }; 292 | 293 | struct Handler: BaseReaderHandler<> { 294 | Origin& origin; 295 | State state; 296 | 297 | Handler(Origin& origin): origin(origin), state(ExpectObjectStart) {} 298 | 299 | bool StartObject() { 300 | switch (state) { 301 | case ExpectObjectStart: 302 | state = ExpectNameOrObjectEnd; 303 | return true; 304 | default: 305 | return false; 306 | } 307 | } 308 | 309 | bool Int(int value) { 310 | return Uint64(value); 311 | } 312 | 313 | bool Int64(int64_t value) { 314 | return Uint64(value); 315 | } 316 | 317 | bool Uint(int value) { 318 | return Uint64(value); 319 | } 320 | 321 | bool Uint64(uint64_t value) { 322 | switch (state) { 323 | case ExpectPort: 324 | origin.port = value; 325 | state = ExpectNameOrObjectEnd; 326 | return true; 327 | 328 | case ExpectProtocol: 329 | origin.protocol = static_cast(value); 330 | state = ExpectNameOrObjectEnd; 331 | return true; 332 | 333 | default: 334 | return false; 335 | } 336 | } 337 | 338 | bool String(const char* str, SizeType length, bool) { 339 | switch (state) { 340 | case ExpectIp: 341 | origin.ip = std::string(str, length); 342 | state = ExpectNameOrObjectEnd; 343 | return true; 344 | 345 | case ExpectHostname: 346 | origin.hostname = std::string(str, length); 347 | state = ExpectNameOrObjectEnd; 348 | return true; 349 | 350 | default: 351 | return false; 352 | } 353 | } 354 | 355 | bool Key(const char* str, SizeType length, bool) { 356 | switch (state) { 357 | case ExpectNameOrObjectEnd: 358 | if (memcmp(str, "ip", sizeof("ip")) == 0) { 359 | state = ExpectIp; 360 | } else if (memcmp(str, "port", sizeof("port")) == 0) { 361 | state = ExpectPort; 362 | } else if (memcmp(str, "hostname", sizeof("hostname")) == 0) { 363 | state = ExpectHostname; 364 | } else if (memcmp(str, "protocol", sizeof("protocol")) == 0) { 365 | state = ExpectProtocol; 366 | } else { 367 | return false; 368 | } 369 | 370 | return true; 371 | default: 372 | return false; 373 | } 374 | } 375 | 376 | bool EndObject(SizeType) { 377 | return state == ExpectNameOrObjectEnd; 378 | } 379 | 380 | bool Default() { 381 | return false; 382 | } 383 | }; 384 | }; 385 | 386 | enum ZonePlan { 387 | ZONEPLAN_UNKNOWN, 388 | FREE, 389 | PRO, 390 | BIZ, 391 | ENT, 392 | }; 393 | 394 | enum Country { 395 | UNKNOWN, 396 | A1, 397 | A2, 398 | O1, 399 | AD, 400 | AE, 401 | AF, 402 | AG, 403 | AI, 404 | AL, 405 | AM, 406 | AO, 407 | AP, 408 | AQ, 409 | AR, 410 | AS, 411 | AT, 412 | AU, 413 | AW, 414 | AX, 415 | AZ, 416 | BA, 417 | BB, 418 | BD, 419 | BE, 420 | BF, 421 | BG, 422 | BH, 423 | BI, 424 | BJ, 425 | BL, 426 | BM, 427 | BN, 428 | BO, 429 | BQ, 430 | BR, 431 | BS, 432 | BT, 433 | BV, 434 | BW, 435 | BY, 436 | BZ, 437 | CA, 438 | CC, 439 | CD, 440 | CF, 441 | CG, 442 | CH, 443 | CI, 444 | CK, 445 | CL, 446 | CM, 447 | CN, 448 | CO, 449 | CR, 450 | CU, 451 | CV, 452 | CW, 453 | CX, 454 | CY, 455 | CZ, 456 | DE, 457 | DJ, 458 | DK, 459 | DM, 460 | DO, 461 | DZ, 462 | EC, 463 | EE, 464 | EG, 465 | EH, 466 | ER, 467 | ES, 468 | ET, 469 | EU, 470 | FI, 471 | FJ, 472 | FK, 473 | FM, 474 | FO, 475 | FR, 476 | GA, 477 | GB, 478 | GD, 479 | GE, 480 | GF, 481 | GG, 482 | GH, 483 | GI, 484 | GL, 485 | GM, 486 | GN, 487 | GP, 488 | GQ, 489 | GR, 490 | GS, 491 | GT, 492 | GU, 493 | GW, 494 | GY, 495 | HK, 496 | HM, 497 | HN, 498 | HR, 499 | HT, 500 | HU, 501 | ID, 502 | IE, 503 | IL, 504 | IM, 505 | IN, 506 | IO, 507 | IQ, 508 | IR, 509 | IS, 510 | IT, 511 | JE, 512 | JM, 513 | JO, 514 | JP, 515 | KE, 516 | KG, 517 | KH, 518 | KI, 519 | KM, 520 | KN, 521 | KP, 522 | KR, 523 | KW, 524 | KY, 525 | KZ, 526 | LA, 527 | LB, 528 | LC, 529 | LI, 530 | LK, 531 | LR, 532 | LS, 533 | LT, 534 | LU, 535 | LV, 536 | LY, 537 | MA, 538 | MC, 539 | MD, 540 | ME, 541 | MF, 542 | MG, 543 | MH, 544 | MK, 545 | ML, 546 | MM, 547 | MN, 548 | MO, 549 | MP, 550 | MQ, 551 | MR, 552 | MS, 553 | MT, 554 | MU, 555 | MV, 556 | MW, 557 | MX, 558 | MY, 559 | MZ, 560 | NA, 561 | NC, 562 | NE, 563 | NF, 564 | NG, 565 | NI, 566 | NL, 567 | NO, 568 | NP, 569 | NR, 570 | NU, 571 | NZ, 572 | OM, 573 | PA, 574 | PE, 575 | PF, 576 | PG, 577 | PH, 578 | PK, 579 | PL, 580 | PM, 581 | PN, 582 | PR, 583 | PS, 584 | PT, 585 | PW, 586 | PY, 587 | QA, 588 | RE, 589 | RO, 590 | RS, 591 | RU, 592 | RW, 593 | SA, 594 | SB, 595 | SC, 596 | SD, 597 | SE, 598 | SG, 599 | SH, 600 | SI, 601 | SJ, 602 | SK, 603 | SL, 604 | SM, 605 | SN, 606 | SO, 607 | SR, 608 | SS, 609 | ST, 610 | SV, 611 | SX, 612 | SY, 613 | SZ, 614 | TC, 615 | TD, 616 | TF, 617 | TG, 618 | TH, 619 | TJ, 620 | TK, 621 | TL, 622 | TM, 623 | TN, 624 | TO, 625 | TR, 626 | TT, 627 | TV, 628 | TW, 629 | TZ, 630 | UA, 631 | UG, 632 | UM, 633 | US, 634 | UY, 635 | UZ, 636 | VA, 637 | VC, 638 | VE, 639 | VG, 640 | VI, 641 | VN, 642 | VU, 643 | WF, 644 | WS, 645 | XX, 646 | YE, 647 | YT, 648 | ZA, 649 | ZM, 650 | ZW, 651 | }; 652 | 653 | struct Log { 654 | long timestamp; 655 | unsigned zone_id; 656 | ZonePlan zone_plan; 657 | Http http; 658 | Origin origin; 659 | Country country; 660 | CacheStatus cache_status; 661 | std::string server_ip; 662 | std::string server_name; 663 | std::string remote_ip; 664 | unsigned long bytes_dlv; 665 | std::string ray_id; 666 | 667 | template 668 | void Serialize(Writer& writer) const { 669 | writer.StartObject(); 670 | 671 | writer.String("timestamp"); 672 | writer.Int64(timestamp); 673 | 674 | writer.String("zone_id"); 675 | writer.Uint(zone_id); 676 | 677 | writer.String("zone_plan"); 678 | writer.Uint(zone_plan); 679 | 680 | writer.String("http"); 681 | http.Serialize(writer); 682 | 683 | writer.String("origin"); 684 | origin.Serialize(writer); 685 | 686 | writer.String("country"); 687 | writer.Uint(country); 688 | 689 | writer.String("cache_status"); 690 | writer.Uint(cache_status); 691 | 692 | writer.String("server_ip"); 693 | writer.String(server_ip.c_str()); 694 | 695 | writer.String("server_name"); 696 | writer.String(server_name.c_str()); 697 | 698 | writer.String("remote_ip"); 699 | writer.String(remote_ip.c_str()); 700 | 701 | writer.String("bytes_dlv"); 702 | writer.Uint(bytes_dlv); 703 | 704 | writer.String("ray_id"); 705 | writer.String(ray_id.c_str()); 706 | 707 | writer.EndObject(); 708 | } 709 | 710 | void Deserialize(Document& document) { 711 | timestamp = document["timestamp"].GetInt64(); 712 | zone_id = document["zone_id"].GetUint(); 713 | zone_plan = static_cast(document["zone_plan"].GetUint()); 714 | http.Deserialize(document["http"]); 715 | origin.Deserialize(document["origin"]); 716 | country = static_cast(document["country"].GetUint()); 717 | cache_status = static_cast(document["cache_status"].GetUint()); 718 | server_ip = document["server_ip"].GetString(); 719 | server_name = document["server_name"].GetString(); 720 | remote_ip = document["remote_ip"].GetString(); 721 | bytes_dlv = document["bytes_dlv"].GetUint(); 722 | ray_id = document["ray_id"].GetString(); 723 | } 724 | 725 | enum State { 726 | ExpectObjectStart, 727 | ExpectNameOrObjectEnd, 728 | ExpectTimestamp, 729 | ExpectZoneId, 730 | ExpectZonePlan, 731 | ExpectHttp, 732 | HandleHttp, 733 | ExpectOrigin, 734 | HandleOrigin, 735 | ExpectCountry, 736 | ExpectCacheStatus, 737 | ExpectServerIp, 738 | ExpectServerName, 739 | ExpectRemoteIp, 740 | ExpectBytesDlv, 741 | ExpectRayId, 742 | }; 743 | 744 | struct Handler: BaseReaderHandler<> { 745 | Log& log; 746 | Http::Handler http_handler; 747 | Origin::Handler origin_handler; 748 | State state; 749 | 750 | Handler(Log& log): log(log), http_handler(log.http), origin_handler(log.origin), state(ExpectObjectStart) {} 751 | 752 | bool StartObject() { 753 | switch (state) { 754 | case ExpectObjectStart: 755 | state = ExpectNameOrObjectEnd; 756 | return true; 757 | 758 | case ExpectHttp: 759 | http_handler.StartObject(); 760 | state = HandleHttp; 761 | return true; 762 | 763 | case ExpectOrigin: 764 | origin_handler.StartObject(); 765 | state = HandleOrigin; 766 | return true; 767 | 768 | default: 769 | return false; 770 | } 771 | } 772 | 773 | bool Int(int value) { 774 | return Uint64(value); 775 | } 776 | 777 | bool Int64(int64_t value) { 778 | return Uint64(value); 779 | } 780 | 781 | bool Uint(uint value) { 782 | return Uint64(value); 783 | } 784 | 785 | 786 | bool Uint64(uint64_t value) { 787 | switch (state) { 788 | case HandleHttp: 789 | return http_handler.Uint64(value); 790 | 791 | case HandleOrigin: 792 | return origin_handler.Uint64(value); 793 | 794 | case ExpectTimestamp: 795 | log.timestamp = value; 796 | state = ExpectNameOrObjectEnd; 797 | return true; 798 | 799 | case ExpectZoneId: 800 | log.zone_id = value; 801 | state = ExpectNameOrObjectEnd; 802 | return true; 803 | 804 | case ExpectZonePlan: 805 | log.zone_plan = static_cast(value); 806 | state = ExpectNameOrObjectEnd; 807 | return true; 808 | 809 | case ExpectCountry: 810 | log.country = static_cast(value); 811 | state = ExpectNameOrObjectEnd; 812 | return true; 813 | 814 | case ExpectCacheStatus: 815 | log.cache_status = static_cast(value); 816 | state = ExpectNameOrObjectEnd; 817 | return true; 818 | 819 | case ExpectBytesDlv: 820 | log.bytes_dlv = value; 821 | state = ExpectNameOrObjectEnd; 822 | return true; 823 | 824 | default: 825 | abort(); 826 | return false; 827 | } 828 | } 829 | 830 | bool Double(double value) { 831 | return Uint(value); 832 | } 833 | 834 | bool String(const char* str, SizeType length, bool copy) { 835 | switch (state) { 836 | case HandleHttp: 837 | return http_handler.String(str, length, copy); 838 | 839 | case HandleOrigin: 840 | return origin_handler.String(str, length, copy); 841 | 842 | case ExpectServerIp: 843 | log.server_ip = std::string(str, length); 844 | state = ExpectNameOrObjectEnd; 845 | return true; 846 | 847 | case ExpectServerName: 848 | log.server_name = std::string(str, length); 849 | state = ExpectNameOrObjectEnd; 850 | return true; 851 | 852 | case ExpectRemoteIp: 853 | log.remote_ip = std::string(str, length); 854 | state = ExpectNameOrObjectEnd; 855 | return true; 856 | 857 | case ExpectRayId: 858 | log.ray_id = std::string(str, length); 859 | state = ExpectNameOrObjectEnd; 860 | return true; 861 | 862 | default: 863 | return false; 864 | } 865 | } 866 | 867 | bool Key(const char* str, SizeType length, bool copy) { 868 | switch (state) { 869 | case ExpectNameOrObjectEnd: 870 | if (memcmp(str, "timestamp", sizeof("timestamp")) == 0) { 871 | state = ExpectTimestamp; 872 | } else if (memcmp(str, "zone_id", sizeof("zone_id")) == 0) { 873 | state = ExpectZoneId; 874 | } else if (memcmp(str, "zone_plan", sizeof("zone_plan")) == 0) { 875 | state = ExpectZonePlan; 876 | } else if (memcmp(str, "http", sizeof("http")) == 0) { 877 | state = ExpectHttp; 878 | } else if (memcmp(str, "origin", sizeof("origin")) == 0) { 879 | state = ExpectOrigin; 880 | } else if (memcmp(str, "country", sizeof("country")) == 0) { 881 | state = ExpectCountry; 882 | } else if (memcmp(str, "cache_status", sizeof("cache_status")) == 0) { 883 | state = ExpectCacheStatus; 884 | } else if (memcmp(str, "server_ip", sizeof("server_ip")) == 0) { 885 | state = ExpectServerIp; 886 | } else if (memcmp(str, "server_name", sizeof("server_name")) == 0) { 887 | state = ExpectServerName; 888 | } else if (memcmp(str, "remote_ip", sizeof("remote_ip")) == 0) { 889 | state = ExpectRemoteIp; 890 | } else if (memcmp(str, "bytes_dlv", sizeof("bytes_dlv")) == 0) { 891 | state = ExpectBytesDlv; 892 | } else if (memcmp(str, "ray_id", sizeof("ray_id")) == 0) { 893 | state = ExpectRayId; 894 | } else { 895 | return false; 896 | } 897 | 898 | return true; 899 | 900 | case HandleHttp: 901 | return http_handler.Key(str, length, copy); 902 | 903 | case HandleOrigin: 904 | return origin_handler.Key(str, length, copy); 905 | 906 | default: 907 | return false; 908 | } 909 | } 910 | 911 | bool EndObject(SizeType length) { 912 | switch (state) { 913 | case ExpectNameOrObjectEnd: 914 | return true; 915 | 916 | case HandleHttp: 917 | state = ExpectNameOrObjectEnd; 918 | return http_handler.EndObject(length); 919 | 920 | case HandleOrigin: 921 | state = ExpectNameOrObjectEnd; 922 | return origin_handler.EndObject(length); 923 | 924 | default: 925 | return false; 926 | } 927 | } 928 | 929 | bool Default() { 930 | return false; 931 | } 932 | }; 933 | }; 934 | 935 | std::string log_serialize(Log& log, StringBuffer& buffer) { 936 | Writer writer(buffer); 937 | log.Serialize(writer); 938 | 939 | return buffer.GetString(); 940 | } 941 | 942 | std::string log_serialize(Log& log) { 943 | StringBuffer buffer(0, 1024); 944 | return log_serialize(log, buffer); 945 | } 946 | 947 | struct LogSerializeBencher { 948 | Log& log; 949 | StringBuffer buffer; 950 | 951 | LogSerializeBencher(Log& log): log(log), buffer(0, 1024) {} 952 | }; 953 | 954 | void bench_log_serialize(LogSerializeBencher& bencher) { 955 | bencher.buffer.Clear(); 956 | log_serialize(bencher.log, bencher.buffer); 957 | } 958 | 959 | void bench_log_deserialize_dom(string& json) { 960 | Document document; 961 | document.Parse(json.c_str()); 962 | Log log; 963 | log.Deserialize(document); 964 | } 965 | 966 | void bench_log_deserialize_sax(string& json) { 967 | Reader reader; 968 | Log log; 969 | Log::Handler handler(log); 970 | StringStream ss(json.c_str()); 971 | if (!reader.Parse(ss, handler)) { 972 | ParseErrorCode e = reader.GetParseErrorCode(); 973 | size_t o = reader.GetErrorOffset(); 974 | cout << "Error: " << GetParseError_En(e) << endl; 975 | exit(1); 976 | } 977 | } 978 | 979 | Log make_log() { 980 | Http http = { 981 | HTTP11, 982 | 200, 983 | 503, 984 | 520, 985 | GET, 986 | "text/html", 987 | "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.146 Safari/537.36", 988 | "https://www.cloudflare.com/", 989 | "/cdn-cgi/trace" 990 | }; 991 | 992 | Origin origin = { 993 | "1.2.3.4", 994 | 8000, 995 | "www.example.com", 996 | HTTPS 997 | }; 998 | 999 | Log log = { 1000 | 2837513946597, 1001 | 123, 1002 | FREE, 1003 | http, 1004 | origin, 1005 | US, 1006 | Hit, 1007 | "192.168.1.1", 1008 | "metal.cloudflare.com", 1009 | "10.1.2.3", 1010 | 456, 1011 | "10c73629cce30078-LAX", 1012 | }; 1013 | 1014 | return log; 1015 | } 1016 | 1017 | void bench_log_populate(int& unused) { 1018 | make_log(); 1019 | } 1020 | 1021 | string make_str() { 1022 | string s; 1023 | for (int i = 0; i < 1024; i++) { 1024 | s.push_back('a'); 1025 | } 1026 | return s; 1027 | } 1028 | 1029 | string make_json_str() { 1030 | string s; 1031 | s.push_back('"'); 1032 | for (int i = 0; i < 1024; i++) { 1033 | s.push_back('a'); 1034 | } 1035 | s.push_back('"'); 1036 | return s; 1037 | } 1038 | 1039 | string str_serialize(string& str) { 1040 | StringBuffer s(0, 1024); 1041 | Writer writer(s); 1042 | writer.String(str.c_str(), str.size()); 1043 | 1044 | return s.GetString(); 1045 | } 1046 | 1047 | void bench_str_serialize(string& str) { 1048 | str_serialize(str); 1049 | } 1050 | 1051 | void bench_str_deserialize_dom(string& json) { 1052 | Document document; 1053 | document.Parse(json.c_str()); 1054 | document.GetString(); 1055 | } 1056 | 1057 | struct StringHandler: BaseReaderHandler<> { 1058 | string str; 1059 | 1060 | StringHandler(): str("") {} 1061 | 1062 | bool String(const char* str, SizeType length, bool copy) { 1063 | this->str = string(str, length); 1064 | 1065 | return true; 1066 | } 1067 | }; 1068 | 1069 | void bench_str_deserialize_sax(string& json) { 1070 | Reader reader; 1071 | StringHandler handler; 1072 | StringStream ss(json.c_str()); 1073 | if (!reader.Parse(ss, handler)) { 1074 | ParseErrorCode e = reader.GetParseErrorCode(); 1075 | size_t o = reader.GetErrorOffset(); 1076 | cout << "Error: " << GetParseError_En(e) << endl; 1077 | exit(1); 1078 | } 1079 | } 1080 | 1081 | int main(int, char*[]) { 1082 | Log log = make_log(); 1083 | int unused; 1084 | bench("log populate ", unused, bench_log_populate); 1085 | string log_json = log_serialize(log); 1086 | LogSerializeBencher bencher(log); 1087 | bench("log serialize ", bencher, log_json.size(), bench_log_serialize); 1088 | bench("log dom deserialize", log_json, log_json.size(), bench_log_deserialize_dom); 1089 | bench("log sax deserialize", log_json, log_json.size(), bench_log_deserialize_sax); 1090 | 1091 | string str = make_str(); 1092 | string str_json = str_serialize(str); 1093 | bench("str serialize ", str, str_json.size(), bench_str_serialize); 1094 | bench("str dom deserialize", str_json, str_json.size(), bench_str_deserialize_dom); 1095 | bench("str sax deserialize", str_json, str_json.size(), bench_str_deserialize_sax); 1096 | 1097 | return 0; 1098 | } 1099 | -------------------------------------------------------------------------------- /go/.gitignore: -------------------------------------------------------------------------------- 1 | /gopath/ 2 | -------------------------------------------------------------------------------- /go/Makefile: -------------------------------------------------------------------------------- 1 | all: bench 2 | 3 | bench: goser 4 | 5 | .PHONY: goser 6 | goser: 7 | cd goser && $(MAKE) 8 | -------------------------------------------------------------------------------- /rust/.gitignore: -------------------------------------------------------------------------------- 1 | target/ 2 | src/log_proto.rs 3 | -------------------------------------------------------------------------------- /rust/Cargo.lock: -------------------------------------------------------------------------------- 1 | [[package]] 2 | name = "autocfg" 3 | version = "0.1.1" 4 | source = "registry+https://github.com/rust-lang/crates.io-index" 5 | 6 | [[package]] 7 | name = "bincode" 8 | version = "1.0.1" 9 | source = "registry+https://github.com/rust-lang/crates.io-index" 10 | dependencies = [ 11 | "byteorder 1.2.7 (registry+https://github.com/rust-lang/crates.io-index)", 12 | "serde 1.0.84 (registry+https://github.com/rust-lang/crates.io-index)", 13 | ] 14 | 15 | [[package]] 16 | name = "bitflags" 17 | version = "1.0.4" 18 | source = "registry+https://github.com/rust-lang/crates.io-index" 19 | 20 | [[package]] 21 | name = "byteorder" 22 | version = "1.2.7" 23 | source = "registry+https://github.com/rust-lang/crates.io-index" 24 | 25 | [[package]] 26 | name = "capnp" 27 | version = "0.9.4" 28 | source = "registry+https://github.com/rust-lang/crates.io-index" 29 | dependencies = [ 30 | "byteorder 1.2.7 (registry+https://github.com/rust-lang/crates.io-index)", 31 | ] 32 | 33 | [[package]] 34 | name = "capnpc" 35 | version = "0.9.3" 36 | source = "registry+https://github.com/rust-lang/crates.io-index" 37 | dependencies = [ 38 | "capnp 0.9.4 (registry+https://github.com/rust-lang/crates.io-index)", 39 | ] 40 | 41 | [[package]] 42 | name = "cfg-if" 43 | version = "0.1.6" 44 | source = "registry+https://github.com/rust-lang/crates.io-index" 45 | 46 | [[package]] 47 | name = "cloudabi" 48 | version = "0.0.3" 49 | source = "registry+https://github.com/rust-lang/crates.io-index" 50 | dependencies = [ 51 | "bitflags 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)", 52 | ] 53 | 54 | [[package]] 55 | name = "flatbuffers" 56 | version = "0.5.0" 57 | source = "registry+https://github.com/rust-lang/crates.io-index" 58 | dependencies = [ 59 | "smallvec 0.6.7 (registry+https://github.com/rust-lang/crates.io-index)", 60 | ] 61 | 62 | [[package]] 63 | name = "flatc-rust" 64 | version = "0.1.0" 65 | source = "registry+https://github.com/rust-lang/crates.io-index" 66 | dependencies = [ 67 | "log 0.4.6 (registry+https://github.com/rust-lang/crates.io-index)", 68 | ] 69 | 70 | [[package]] 71 | name = "fuchsia-zircon" 72 | version = "0.3.3" 73 | source = "registry+https://github.com/rust-lang/crates.io-index" 74 | dependencies = [ 75 | "bitflags 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)", 76 | "fuchsia-zircon-sys 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)", 77 | ] 78 | 79 | [[package]] 80 | name = "fuchsia-zircon-sys" 81 | version = "0.3.3" 82 | source = "registry+https://github.com/rust-lang/crates.io-index" 83 | 84 | [[package]] 85 | name = "itoa" 86 | version = "0.4.3" 87 | source = "registry+https://github.com/rust-lang/crates.io-index" 88 | 89 | [[package]] 90 | name = "libc" 91 | version = "0.2.47" 92 | source = "registry+https://github.com/rust-lang/crates.io-index" 93 | 94 | [[package]] 95 | name = "log" 96 | version = "0.4.6" 97 | source = "registry+https://github.com/rust-lang/crates.io-index" 98 | dependencies = [ 99 | "cfg-if 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", 100 | ] 101 | 102 | [[package]] 103 | name = "num-derive" 104 | version = "0.2.3" 105 | source = "registry+https://github.com/rust-lang/crates.io-index" 106 | dependencies = [ 107 | "num-traits 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)", 108 | "proc-macro2 0.4.24 (registry+https://github.com/rust-lang/crates.io-index)", 109 | "quote 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)", 110 | "syn 0.15.26 (registry+https://github.com/rust-lang/crates.io-index)", 111 | ] 112 | 113 | [[package]] 114 | name = "num-traits" 115 | version = "0.1.43" 116 | source = "registry+https://github.com/rust-lang/crates.io-index" 117 | dependencies = [ 118 | "num-traits 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)", 119 | ] 120 | 121 | [[package]] 122 | name = "num-traits" 123 | version = "0.2.6" 124 | source = "registry+https://github.com/rust-lang/crates.io-index" 125 | 126 | [[package]] 127 | name = "proc-macro2" 128 | version = "0.4.24" 129 | source = "registry+https://github.com/rust-lang/crates.io-index" 130 | dependencies = [ 131 | "unicode-xid 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", 132 | ] 133 | 134 | [[package]] 135 | name = "protobuf" 136 | version = "2.2.4" 137 | source = "registry+https://github.com/rust-lang/crates.io-index" 138 | 139 | [[package]] 140 | name = "protobuf-codegen" 141 | version = "2.2.4" 142 | source = "registry+https://github.com/rust-lang/crates.io-index" 143 | dependencies = [ 144 | "protobuf 2.2.4 (registry+https://github.com/rust-lang/crates.io-index)", 145 | ] 146 | 147 | [[package]] 148 | name = "protoc" 149 | version = "2.2.4" 150 | source = "registry+https://github.com/rust-lang/crates.io-index" 151 | dependencies = [ 152 | "log 0.4.6 (registry+https://github.com/rust-lang/crates.io-index)", 153 | ] 154 | 155 | [[package]] 156 | name = "protoc-rust" 157 | version = "2.2.4" 158 | source = "registry+https://github.com/rust-lang/crates.io-index" 159 | dependencies = [ 160 | "protobuf 2.2.4 (registry+https://github.com/rust-lang/crates.io-index)", 161 | "protobuf-codegen 2.2.4 (registry+https://github.com/rust-lang/crates.io-index)", 162 | "protoc 2.2.4 (registry+https://github.com/rust-lang/crates.io-index)", 163 | "tempfile 3.0.5 (registry+https://github.com/rust-lang/crates.io-index)", 164 | ] 165 | 166 | [[package]] 167 | name = "quote" 168 | version = "0.6.10" 169 | source = "registry+https://github.com/rust-lang/crates.io-index" 170 | dependencies = [ 171 | "proc-macro2 0.4.24 (registry+https://github.com/rust-lang/crates.io-index)", 172 | ] 173 | 174 | [[package]] 175 | name = "rand" 176 | version = "0.6.4" 177 | source = "registry+https://github.com/rust-lang/crates.io-index" 178 | dependencies = [ 179 | "autocfg 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", 180 | "libc 0.2.47 (registry+https://github.com/rust-lang/crates.io-index)", 181 | "rand_chacha 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", 182 | "rand_core 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", 183 | "rand_hc 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", 184 | "rand_isaac 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", 185 | "rand_os 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", 186 | "rand_pcg 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", 187 | "rand_xorshift 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", 188 | "winapi 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)", 189 | ] 190 | 191 | [[package]] 192 | name = "rand_chacha" 193 | version = "0.1.1" 194 | source = "registry+https://github.com/rust-lang/crates.io-index" 195 | dependencies = [ 196 | "autocfg 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)", 197 | "rand_core 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", 198 | ] 199 | 200 | [[package]] 201 | name = "rand_core" 202 | version = "0.3.0" 203 | source = "registry+https://github.com/rust-lang/crates.io-index" 204 | 205 | [[package]] 206 | name = "rand_hc" 207 | version = "0.1.0" 208 | source = "registry+https://github.com/rust-lang/crates.io-index" 209 | dependencies = [ 210 | "rand_core 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", 211 | ] 212 | 213 | [[package]] 214 | name = "rand_isaac" 215 | version = "0.1.1" 216 | source = "registry+https://github.com/rust-lang/crates.io-index" 217 | dependencies = [ 218 | "rand_core 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", 219 | ] 220 | 221 | [[package]] 222 | name = "rand_os" 223 | version = "0.1.1" 224 | source = "registry+https://github.com/rust-lang/crates.io-index" 225 | dependencies = [ 226 | "cloudabi 0.0.3 (registry+https://github.com/rust-lang/crates.io-index)", 227 | "fuchsia-zircon 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)", 228 | "libc 0.2.47 (registry+https://github.com/rust-lang/crates.io-index)", 229 | "rand_core 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", 230 | "rdrand 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", 231 | "winapi 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)", 232 | ] 233 | 234 | [[package]] 235 | name = "rand_pcg" 236 | version = "0.1.1" 237 | source = "registry+https://github.com/rust-lang/crates.io-index" 238 | dependencies = [ 239 | "rand_core 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", 240 | "rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", 241 | ] 242 | 243 | [[package]] 244 | name = "rand_xorshift" 245 | version = "0.1.1" 246 | source = "registry+https://github.com/rust-lang/crates.io-index" 247 | dependencies = [ 248 | "rand_core 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", 249 | ] 250 | 251 | [[package]] 252 | name = "rdrand" 253 | version = "0.4.0" 254 | source = "registry+https://github.com/rust-lang/crates.io-index" 255 | dependencies = [ 256 | "rand_core 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)", 257 | ] 258 | 259 | [[package]] 260 | name = "redox_syscall" 261 | version = "0.1.50" 262 | source = "registry+https://github.com/rust-lang/crates.io-index" 263 | 264 | [[package]] 265 | name = "remove_dir_all" 266 | version = "0.5.1" 267 | source = "registry+https://github.com/rust-lang/crates.io-index" 268 | dependencies = [ 269 | "winapi 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)", 270 | ] 271 | 272 | [[package]] 273 | name = "rmp" 274 | version = "0.8.7" 275 | source = "registry+https://github.com/rust-lang/crates.io-index" 276 | dependencies = [ 277 | "byteorder 1.2.7 (registry+https://github.com/rust-lang/crates.io-index)", 278 | "num-traits 0.1.43 (registry+https://github.com/rust-lang/crates.io-index)", 279 | ] 280 | 281 | [[package]] 282 | name = "rmp-serde" 283 | version = "0.13.7" 284 | source = "registry+https://github.com/rust-lang/crates.io-index" 285 | dependencies = [ 286 | "byteorder 1.2.7 (registry+https://github.com/rust-lang/crates.io-index)", 287 | "rmp 0.8.7 (registry+https://github.com/rust-lang/crates.io-index)", 288 | "serde 1.0.84 (registry+https://github.com/rust-lang/crates.io-index)", 289 | ] 290 | 291 | [[package]] 292 | name = "rust-serde-benchmarks" 293 | version = "0.0.1" 294 | dependencies = [ 295 | "bincode 1.0.1 (registry+https://github.com/rust-lang/crates.io-index)", 296 | "capnp 0.9.4 (registry+https://github.com/rust-lang/crates.io-index)", 297 | "capnpc 0.9.3 (registry+https://github.com/rust-lang/crates.io-index)", 298 | "flatbuffers 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)", 299 | "flatc-rust 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", 300 | "num-derive 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)", 301 | "num-traits 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)", 302 | "protobuf 2.2.4 (registry+https://github.com/rust-lang/crates.io-index)", 303 | "protoc-rust 2.2.4 (registry+https://github.com/rust-lang/crates.io-index)", 304 | "rmp-serde 0.13.7 (registry+https://github.com/rust-lang/crates.io-index)", 305 | "serde 1.0.84 (registry+https://github.com/rust-lang/crates.io-index)", 306 | "serde_derive 1.0.84 (registry+https://github.com/rust-lang/crates.io-index)", 307 | "serde_json 1.0.35 (registry+https://github.com/rust-lang/crates.io-index)", 308 | ] 309 | 310 | [[package]] 311 | name = "rustc_version" 312 | version = "0.2.3" 313 | source = "registry+https://github.com/rust-lang/crates.io-index" 314 | dependencies = [ 315 | "semver 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)", 316 | ] 317 | 318 | [[package]] 319 | name = "ryu" 320 | version = "0.2.7" 321 | source = "registry+https://github.com/rust-lang/crates.io-index" 322 | 323 | [[package]] 324 | name = "semver" 325 | version = "0.9.0" 326 | source = "registry+https://github.com/rust-lang/crates.io-index" 327 | dependencies = [ 328 | "semver-parser 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)", 329 | ] 330 | 331 | [[package]] 332 | name = "semver-parser" 333 | version = "0.7.0" 334 | source = "registry+https://github.com/rust-lang/crates.io-index" 335 | 336 | [[package]] 337 | name = "serde" 338 | version = "1.0.84" 339 | source = "registry+https://github.com/rust-lang/crates.io-index" 340 | 341 | [[package]] 342 | name = "serde_derive" 343 | version = "1.0.84" 344 | source = "registry+https://github.com/rust-lang/crates.io-index" 345 | dependencies = [ 346 | "proc-macro2 0.4.24 (registry+https://github.com/rust-lang/crates.io-index)", 347 | "quote 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)", 348 | "syn 0.15.26 (registry+https://github.com/rust-lang/crates.io-index)", 349 | ] 350 | 351 | [[package]] 352 | name = "serde_json" 353 | version = "1.0.35" 354 | source = "registry+https://github.com/rust-lang/crates.io-index" 355 | dependencies = [ 356 | "itoa 0.4.3 (registry+https://github.com/rust-lang/crates.io-index)", 357 | "ryu 0.2.7 (registry+https://github.com/rust-lang/crates.io-index)", 358 | "serde 1.0.84 (registry+https://github.com/rust-lang/crates.io-index)", 359 | ] 360 | 361 | [[package]] 362 | name = "smallvec" 363 | version = "0.6.7" 364 | source = "registry+https://github.com/rust-lang/crates.io-index" 365 | dependencies = [ 366 | "unreachable 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)", 367 | ] 368 | 369 | [[package]] 370 | name = "syn" 371 | version = "0.15.26" 372 | source = "registry+https://github.com/rust-lang/crates.io-index" 373 | dependencies = [ 374 | "proc-macro2 0.4.24 (registry+https://github.com/rust-lang/crates.io-index)", 375 | "quote 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)", 376 | "unicode-xid 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)", 377 | ] 378 | 379 | [[package]] 380 | name = "tempfile" 381 | version = "3.0.5" 382 | source = "registry+https://github.com/rust-lang/crates.io-index" 383 | dependencies = [ 384 | "cfg-if 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)", 385 | "libc 0.2.47 (registry+https://github.com/rust-lang/crates.io-index)", 386 | "rand 0.6.4 (registry+https://github.com/rust-lang/crates.io-index)", 387 | "redox_syscall 0.1.50 (registry+https://github.com/rust-lang/crates.io-index)", 388 | "remove_dir_all 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)", 389 | "winapi 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)", 390 | ] 391 | 392 | [[package]] 393 | name = "unicode-xid" 394 | version = "0.1.0" 395 | source = "registry+https://github.com/rust-lang/crates.io-index" 396 | 397 | [[package]] 398 | name = "unreachable" 399 | version = "1.0.0" 400 | source = "registry+https://github.com/rust-lang/crates.io-index" 401 | dependencies = [ 402 | "void 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)", 403 | ] 404 | 405 | [[package]] 406 | name = "void" 407 | version = "1.0.2" 408 | source = "registry+https://github.com/rust-lang/crates.io-index" 409 | 410 | [[package]] 411 | name = "winapi" 412 | version = "0.3.6" 413 | source = "registry+https://github.com/rust-lang/crates.io-index" 414 | dependencies = [ 415 | "winapi-i686-pc-windows-gnu 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", 416 | "winapi-x86_64-pc-windows-gnu 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)", 417 | ] 418 | 419 | [[package]] 420 | name = "winapi-i686-pc-windows-gnu" 421 | version = "0.4.0" 422 | source = "registry+https://github.com/rust-lang/crates.io-index" 423 | 424 | [[package]] 425 | name = "winapi-x86_64-pc-windows-gnu" 426 | version = "0.4.0" 427 | source = "registry+https://github.com/rust-lang/crates.io-index" 428 | 429 | [metadata] 430 | "checksum autocfg 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "4e5f34df7a019573fb8bdc7e24a2bfebe51a2a1d6bfdbaeccedb3c41fc574727" 431 | "checksum bincode 1.0.1 (registry+https://github.com/rust-lang/crates.io-index)" = "9f2fb9e29e72fd6bc12071533d5dc7664cb01480c59406f656d7ac25c7bd8ff7" 432 | "checksum bitflags 1.0.4 (registry+https://github.com/rust-lang/crates.io-index)" = "228047a76f468627ca71776ecdebd732a3423081fcf5125585bcd7c49886ce12" 433 | "checksum byteorder 1.2.7 (registry+https://github.com/rust-lang/crates.io-index)" = "94f88df23a25417badc922ab0f5716cc1330e87f71ddd9203b3a3ccd9cedf75d" 434 | "checksum capnp 0.9.4 (registry+https://github.com/rust-lang/crates.io-index)" = "5f3809f2a9dae96253a32f079db1d9887a495fa0e3181bc2a64476d7ff58eac8" 435 | "checksum capnpc 0.9.3 (registry+https://github.com/rust-lang/crates.io-index)" = "def7dcaba60ae98fb6fd1614e141f7c9b98ac7dd5daddb9f79771c4296b1ed21" 436 | "checksum cfg-if 0.1.6 (registry+https://github.com/rust-lang/crates.io-index)" = "082bb9b28e00d3c9d39cc03e64ce4cea0f1bb9b3fde493f0cbc008472d22bdf4" 437 | "checksum cloudabi 0.0.3 (registry+https://github.com/rust-lang/crates.io-index)" = "ddfc5b9aa5d4507acaf872de71051dfd0e309860e88966e1051e462a077aac4f" 438 | "checksum flatbuffers 0.5.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ea0c34f669be9911826facafe996adfda978aeee67285a13556869e2d8b8331f" 439 | "checksum flatc-rust 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "5e69e3f68d91fa29633d1a463b10a65a8ae49d9bc6cccc00526ed8be9fa30c0f" 440 | "checksum fuchsia-zircon 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "2e9763c69ebaae630ba35f74888db465e49e259ba1bc0eda7d06f4a067615d82" 441 | "checksum fuchsia-zircon-sys 0.3.3 (registry+https://github.com/rust-lang/crates.io-index)" = "3dcaa9ae7725d12cdb85b3ad99a434db70b468c09ded17e012d86b5c1010f7a7" 442 | "checksum itoa 0.4.3 (registry+https://github.com/rust-lang/crates.io-index)" = "1306f3464951f30e30d12373d31c79fbd52d236e5e896fd92f96ec7babbbe60b" 443 | "checksum libc 0.2.47 (registry+https://github.com/rust-lang/crates.io-index)" = "48450664a984b25d5b479554c29cc04e3150c97aa4c01da5604a2d4ed9151476" 444 | "checksum log 0.4.6 (registry+https://github.com/rust-lang/crates.io-index)" = "c84ec4b527950aa83a329754b01dbe3f58361d1c5efacd1f6d68c494d08a17c6" 445 | "checksum num-derive 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "8af1847c907c2f04d7bfd572fb25bbb4385c637fe5be163cf2f8c5d778fe1e7d" 446 | "checksum num-traits 0.1.43 (registry+https://github.com/rust-lang/crates.io-index)" = "92e5113e9fd4cc14ded8e499429f396a20f98c772a47cc8622a736e1ec843c31" 447 | "checksum num-traits 0.2.6 (registry+https://github.com/rust-lang/crates.io-index)" = "0b3a5d7cc97d6d30d8b9bc8fa19bf45349ffe46241e8816f50f62f6d6aaabee1" 448 | "checksum proc-macro2 0.4.24 (registry+https://github.com/rust-lang/crates.io-index)" = "77619697826f31a02ae974457af0b29b723e5619e113e9397b8b82c6bd253f09" 449 | "checksum protobuf 2.2.4 (registry+https://github.com/rust-lang/crates.io-index)" = "9e18cbbffd1481c115cfb809816dbe6a8f53cf5b4ac6df03d1589d9ccb6c609a" 450 | "checksum protobuf-codegen 2.2.4 (registry+https://github.com/rust-lang/crates.io-index)" = "eb6cb1a50e66ca517bd25cb871211be326fc5f9736b4a6eae7d277842d904e95" 451 | "checksum protoc 2.2.4 (registry+https://github.com/rust-lang/crates.io-index)" = "183abe5a3ca0c2e7b4568514d70b2b8f0cf234c159925b2a376347da0e70dc6d" 452 | "checksum protoc-rust 2.2.4 (registry+https://github.com/rust-lang/crates.io-index)" = "81f3950d2968a6d7f860fb8171383385b444902ed0323b6cdaa9556d8de474b1" 453 | "checksum quote 0.6.10 (registry+https://github.com/rust-lang/crates.io-index)" = "53fa22a1994bd0f9372d7a816207d8a2677ad0325b073f5c5332760f0fb62b5c" 454 | "checksum rand 0.6.4 (registry+https://github.com/rust-lang/crates.io-index)" = "3906503e80ac6cbcacb2c2973fa8e473f24d7e2747c8c92bb230c2441cad96b5" 455 | "checksum rand_chacha 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "556d3a1ca6600bfcbab7c7c91ccb085ac7fbbcd70e008a98742e7847f4f7bcef" 456 | "checksum rand_core 0.3.0 (registry+https://github.com/rust-lang/crates.io-index)" = "0905b6b7079ec73b314d4c748701f6931eb79fd97c668caa3f1899b22b32c6db" 457 | "checksum rand_hc 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "7b40677c7be09ae76218dc623efbf7b18e34bced3f38883af07bb75630a21bc4" 458 | "checksum rand_isaac 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "ded997c9d5f13925be2a6fd7e66bf1872597f759fd9dd93513dd7e92e5a5ee08" 459 | "checksum rand_os 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "f46fbd5550acf75b0c2730f5dd1873751daf9beb8f11b44027778fae50d7feca" 460 | "checksum rand_pcg 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "086bd09a33c7044e56bb44d5bdde5a60e7f119a9e95b0775f545de759a32fe05" 461 | "checksum rand_xorshift 0.1.1 (registry+https://github.com/rust-lang/crates.io-index)" = "cbf7e9e623549b0e21f6e97cf8ecf247c1a8fd2e8a992ae265314300b2455d5c" 462 | "checksum rdrand 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "678054eb77286b51581ba43620cc911abf02758c91f93f479767aed0f90458b2" 463 | "checksum redox_syscall 0.1.50 (registry+https://github.com/rust-lang/crates.io-index)" = "52ee9a534dc1301776eff45b4fa92d2c39b1d8c3d3357e6eb593e0d795506fc2" 464 | "checksum remove_dir_all 0.5.1 (registry+https://github.com/rust-lang/crates.io-index)" = "3488ba1b9a2084d38645c4c08276a1752dcbf2c7130d74f1569681ad5d2799c5" 465 | "checksum rmp 0.8.7 (registry+https://github.com/rust-lang/crates.io-index)" = "a3d45d7afc9b132b34a2479648863aa95c5c88e98b32285326a6ebadc80ec5c9" 466 | "checksum rmp-serde 0.13.7 (registry+https://github.com/rust-lang/crates.io-index)" = "011e1d58446e9fa3af7cdc1fb91295b10621d3ac4cb3a85cc86385ee9ca50cd3" 467 | "checksum rustc_version 0.2.3 (registry+https://github.com/rust-lang/crates.io-index)" = "138e3e0acb6c9fb258b19b67cb8abd63c00679d2851805ea151465464fe9030a" 468 | "checksum ryu 0.2.7 (registry+https://github.com/rust-lang/crates.io-index)" = "eb9e9b8cde282a9fe6a42dd4681319bfb63f121b8a8ee9439c6f4107e58a46f7" 469 | "checksum semver 0.9.0 (registry+https://github.com/rust-lang/crates.io-index)" = "1d7eb9ef2c18661902cc47e535f9bc51b78acd254da71d375c2f6720d9a40403" 470 | "checksum semver-parser 0.7.0 (registry+https://github.com/rust-lang/crates.io-index)" = "388a1df253eca08550bef6c72392cfe7c30914bf41df5269b68cbd6ff8f570a3" 471 | "checksum serde 1.0.84 (registry+https://github.com/rust-lang/crates.io-index)" = "0e732ed5a5592c17d961555e3b552985baf98d50ce418b7b655f31f6ba7eb1b7" 472 | "checksum serde_derive 1.0.84 (registry+https://github.com/rust-lang/crates.io-index)" = "b4d6115a3ca25c224e409185325afc16a0d5aaaabc15c42b09587d6f1ba39a5b" 473 | "checksum serde_json 1.0.35 (registry+https://github.com/rust-lang/crates.io-index)" = "dfb1277d4d0563e4593e0b8b5d23d744d277b55d2bc0bf1c38d0d8a6589d38aa" 474 | "checksum smallvec 0.6.7 (registry+https://github.com/rust-lang/crates.io-index)" = "b73ea3738b47563803ef814925e69be00799a8c07420be8b996f8e98fb2336db" 475 | "checksum syn 0.15.26 (registry+https://github.com/rust-lang/crates.io-index)" = "f92e629aa1d9c827b2bb8297046c1ccffc57c99b947a680d3ccff1f136a3bee9" 476 | "checksum tempfile 3.0.5 (registry+https://github.com/rust-lang/crates.io-index)" = "7e91405c14320e5c79b3d148e1c86f40749a36e490642202a31689cb1a3452b2" 477 | "checksum unicode-xid 0.1.0 (registry+https://github.com/rust-lang/crates.io-index)" = "fc72304796d0818e357ead4e000d19c9c174ab23dc11093ac919054d20a6a7fc" 478 | "checksum unreachable 1.0.0 (registry+https://github.com/rust-lang/crates.io-index)" = "382810877fe448991dfc7f0dd6e3ae5d58088fd0ea5e35189655f84e6814fa56" 479 | "checksum void 1.0.2 (registry+https://github.com/rust-lang/crates.io-index)" = "6a02e4885ed3bc0f2de90ea6dd45ebcbb66dacffe03547fadbb0eeae2770887d" 480 | "checksum winapi 0.3.6 (registry+https://github.com/rust-lang/crates.io-index)" = "92c1eb33641e276cfa214a0522acad57be5c56b10cb348b3c5117db75f3ac4b0" 481 | "checksum winapi-i686-pc-windows-gnu 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6" 482 | "checksum winapi-x86_64-pc-windows-gnu 0.4.0 (registry+https://github.com/rust-lang/crates.io-index)" = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f" 483 | -------------------------------------------------------------------------------- /rust/Cargo.toml: -------------------------------------------------------------------------------- 1 | [package] 2 | name = "rust-serde-benchmarks" 3 | version = "0.0.1" 4 | authors = ["erick.tryzelaar@gmail.com"] 5 | build = "build.rs" 6 | 7 | [build-dependencies] 8 | capnpc = "0.9" 9 | flatc-rust = "0.1" 10 | protoc-rust = "2.0" 11 | 12 | [dependencies] 13 | bincode = "1.0" 14 | capnp = "0.9" 15 | flatbuffers = "0.5" 16 | num-derive = "0.2" 17 | num-traits = "0.2" 18 | protobuf = "2.0" 19 | rmp-serde = "0.13" 20 | serde = "1.0" 21 | serde_derive = "1.0" 22 | serde_json = "1.0" 23 | -------------------------------------------------------------------------------- /rust/Makefile: -------------------------------------------------------------------------------- 1 | all: 2 | cargo +nightly build 3 | 4 | test: 5 | cargo +nightly test 6 | 7 | bench: # only bench requires nightly 8 | cargo +nightly bench 9 | 10 | .PHONY: test bench 11 | -------------------------------------------------------------------------------- /rust/benches/bench.rs: -------------------------------------------------------------------------------- 1 | #![feature(test)] 2 | extern crate bincode; 3 | extern crate capnp; 4 | extern crate flatbuffers; 5 | extern crate protobuf; 6 | extern crate rmp_serde; 7 | extern crate rust_serde_benchmarks; 8 | extern crate serde; 9 | extern crate serde_json; 10 | extern crate test; 11 | 12 | use capnp::message::{Builder, ReaderOptions}; 13 | use protobuf::{Clear, Message}; 14 | use rust_serde_benchmarks::{log_capnp, log_proto, log_flatbuffers, protocol_capnp, protocol_protobuf, protocol_flatbuffers, Log}; 15 | use serde::{Deserialize, Serialize}; 16 | use test::Bencher; 17 | 18 | #[bench] 19 | fn clone(b: &mut Bencher) { 20 | let log = Log::new(); 21 | 22 | let mut size = ::std::mem::size_of_val(&log); 23 | 24 | size += log.http.content_type.len(); 25 | size += log.http.user_agent.len(); 26 | size += log.http.referer.len(); 27 | size += log.http.request_uri.len(); 28 | 29 | size += log.origin.ip.len(); 30 | size += log.origin.hostname.len(); 31 | 32 | size += log.server_ip.len(); 33 | size += log.server_name.len(); 34 | size += log.remote_ip.len(); 35 | size += log.ray_id.len(); 36 | 37 | b.bytes = size as u64; 38 | 39 | b.iter(|| log.clone()); 40 | } 41 | 42 | #[bench] 43 | fn serde_json_serialize(b: &mut Bencher) { 44 | let log = Log::new(); 45 | let mut buf = Vec::new(); 46 | 47 | serde_json::to_writer(&mut buf, &log).unwrap(); 48 | b.bytes = buf.len() as u64; 49 | 50 | b.iter(|| { 51 | buf.clear(); 52 | serde_json::to_writer(&mut buf, &log).unwrap(); 53 | }); 54 | } 55 | 56 | #[bench] 57 | fn serde_json_deserialize(b: &mut Bencher) { 58 | let log = Log::new(); 59 | let json = serde_json::to_string(&log).unwrap(); 60 | 61 | b.bytes = json.len() as u64; 62 | 63 | b.iter(|| serde_json::from_str::(&json).unwrap()); 64 | } 65 | 66 | #[bench] 67 | fn capnp_populate(b: &mut Bencher) { 68 | let mut msg = Builder::new_default(); 69 | 70 | b.iter(|| { 71 | protocol_capnp::populate_log(&mut msg); 72 | }); 73 | } 74 | 75 | #[bench] 76 | fn capnp_serialize(b: &mut Bencher) { 77 | let mut msg = Builder::new_default(); 78 | protocol_capnp::populate_log(&mut msg); 79 | 80 | let mut buf = Vec::new(); 81 | ::capnp::serialize::write_message(&mut buf, &msg).unwrap(); 82 | b.bytes = buf.len() as u64; 83 | 84 | b.iter(|| { 85 | buf.clear(); 86 | ::capnp::serialize::write_message(&mut buf, &msg).unwrap(); 87 | }); 88 | } 89 | 90 | #[bench] 91 | fn capnp_serialize_packed(b: &mut Bencher) { 92 | let mut msg = Builder::new_default(); 93 | protocol_capnp::populate_log(&mut msg); 94 | 95 | let mut buf = Vec::new(); 96 | ::capnp::serialize_packed::write_message(&mut buf, &msg).unwrap(); 97 | b.bytes = buf.len() as u64; 98 | 99 | b.iter(|| { 100 | buf.clear(); 101 | ::capnp::serialize_packed::write_message(&mut buf, &msg).unwrap(); 102 | }); 103 | } 104 | 105 | #[bench] 106 | fn capnp_deserialize(b: &mut Bencher) { 107 | let mut msg = Builder::new_default(); 108 | protocol_capnp::populate_log(&mut msg); 109 | 110 | let mut buf = Vec::new(); 111 | ::capnp::serialize::write_message(&mut buf, &msg).unwrap(); 112 | b.bytes = buf.len() as u64; 113 | 114 | b.iter(|| { 115 | let mut rdr = &*buf; 116 | let message_reader = 117 | ::capnp::serialize::read_message(&mut rdr, ReaderOptions::new()).unwrap(); 118 | message_reader.get_root::().unwrap(); 119 | }); 120 | } 121 | 122 | #[bench] 123 | fn capnp_deserialize_packed(b: &mut Bencher) { 124 | let mut msg = Builder::new_default(); 125 | protocol_capnp::populate_log(&mut msg); 126 | 127 | let mut buf = Vec::new(); 128 | ::capnp::serialize_packed::write_message(&mut buf, &msg).unwrap(); 129 | b.bytes = buf.len() as u64; 130 | 131 | b.iter(|| { 132 | let mut rdr = &*buf; 133 | let message_reader = 134 | ::capnp::serialize_packed::read_message(&mut rdr, ReaderOptions::new()).unwrap(); 135 | message_reader.get_root::().unwrap(); 136 | }); 137 | } 138 | 139 | #[bench] 140 | fn flatbuffers_populate_with_args(b: &mut Bencher) { 141 | let mut msg = flatbuffers::FlatBufferBuilder::new(); 142 | 143 | b.iter(|| { 144 | msg.reset(); 145 | let log = protocol_flatbuffers::populate_log_with_args(&mut msg); 146 | msg.finish(log, None); 147 | }); 148 | } 149 | 150 | #[bench] 151 | fn flatbuffers_populate_with_builder(b: &mut Bencher) { 152 | let mut msg = flatbuffers::FlatBufferBuilder::new(); 153 | 154 | b.iter(|| { 155 | msg.reset(); 156 | let log = protocol_flatbuffers::populate_log_with_builder(&mut msg); 157 | msg.finish(log, None); 158 | }); 159 | } 160 | 161 | #[bench] 162 | fn flatbuffers_serialize(b: &mut Bencher) { 163 | let mut msg = flatbuffers::FlatBufferBuilder::new(); 164 | let log = protocol_flatbuffers::populate_log_with_builder(&mut msg); 165 | msg.finish(log, None); 166 | 167 | let buf = msg.finished_data(); 168 | b.bytes = buf.len() as u64; 169 | 170 | b.iter(|| { 171 | msg.finished_data() 172 | }); 173 | } 174 | 175 | #[bench] 176 | fn flatbuffers_deserialize(b: &mut Bencher) { 177 | let mut msg = flatbuffers::FlatBufferBuilder::new(); 178 | let log = protocol_flatbuffers::populate_log_with_builder(&mut msg); 179 | msg.finish(log, None); 180 | 181 | let buf = msg.finished_data(); 182 | b.bytes = buf.len() as u64; 183 | 184 | b.iter(|| { 185 | log_flatbuffers::get_root_as_log(&buf) 186 | }); 187 | } 188 | 189 | #[bench] 190 | fn rmp_serde_serialize(b: &mut Bencher) { 191 | let mut buf = Vec::new(); 192 | let log = Log::new(); 193 | 194 | log.serialize(&mut ::rmp_serde::Serializer::new(&mut buf)) 195 | .unwrap(); 196 | b.bytes = buf.len() as u64; 197 | 198 | b.iter(|| { 199 | buf.clear(); 200 | log.serialize(&mut ::rmp_serde::Serializer::new(&mut buf)) 201 | .unwrap(); 202 | }); 203 | } 204 | 205 | #[bench] 206 | fn rmp_serde_deserialize(b: &mut Bencher) { 207 | let mut buf = Vec::new(); 208 | let log = Log::new(); 209 | log.serialize(&mut ::rmp_serde::Serializer::new(&mut buf)) 210 | .unwrap(); 211 | b.bytes = buf.len() as u64; 212 | 213 | b.iter(|| { 214 | let mut decoder = ::rmp_serde::Deserializer::new(&*buf); 215 | let _log: Log = Deserialize::deserialize(&mut decoder).unwrap(); 216 | }); 217 | } 218 | 219 | #[bench] 220 | fn rust_protobuf_populate(b: &mut Bencher) { 221 | b.iter(|| protocol_protobuf::new_log()); 222 | } 223 | 224 | #[bench] 225 | fn rust_protobuf_serialize(b: &mut Bencher) { 226 | let mut buf = Vec::new(); 227 | let log = protocol_protobuf::new_log(); 228 | 229 | log.write_to_writer(&mut buf).unwrap(); 230 | b.bytes = buf.len() as u64; 231 | 232 | b.iter(|| { 233 | buf.clear(); 234 | log.write_to_writer(&mut buf).unwrap(); 235 | }); 236 | } 237 | 238 | #[bench] 239 | fn rust_protobuf_deserialize(b: &mut Bencher) { 240 | let log = protocol_protobuf::new_log(); 241 | let buf = log.write_to_bytes().unwrap(); 242 | b.bytes = buf.len() as u64; 243 | 244 | let mut log = log_proto::Log::new(); 245 | b.iter(|| { 246 | log.clear(); 247 | log.merge_from_bytes(&buf).unwrap(); 248 | }); 249 | } 250 | 251 | #[bench] 252 | fn rust_bincode_serialize(b: &mut Bencher) { 253 | let mut buf = Vec::new(); 254 | let log = Log::new(); 255 | 256 | ::bincode::serialize_into(&mut buf, &log).unwrap(); 257 | b.bytes = buf.len() as u64; 258 | 259 | b.iter(|| { 260 | buf.clear(); 261 | ::bincode::serialize_into(&mut buf, &log).unwrap(); 262 | }); 263 | } 264 | 265 | #[bench] 266 | fn rust_bincode_deserialize(b: &mut Bencher) { 267 | let buf = bincode::serialize(&Log::new()).unwrap(); 268 | b.bytes = buf.len() as u64; 269 | 270 | b.iter(|| { 271 | let _log: Log = ::bincode::deserialize_from(&*buf).unwrap(); 272 | }); 273 | } 274 | -------------------------------------------------------------------------------- /rust/build.rs: -------------------------------------------------------------------------------- 1 | extern crate capnpc; 2 | extern crate flatc_rust; 3 | extern crate protoc_rust; 4 | 5 | use std::path::Path; 6 | 7 | fn main() { 8 | println!("cargo:rerun-if-changed=src/log.capnp"); 9 | capnpc::CompilerCommand::new() 10 | .src_prefix("src") 11 | .file("src/log.capnp") 12 | .run() 13 | .expect("schema compiler command"); 14 | 15 | println!("cargo:rerun-if-changed=src/log_proto.proto"); 16 | let out_dir = "target/protobufs/"; 17 | std::fs::create_dir_all(out_dir).expect("could not create protobufs output folder"); 18 | protoc_rust::run(protoc_rust::Args { 19 | input: &["src/log_proto.proto"], 20 | out_dir, 21 | ..Default::default() 22 | }).expect("protoc"); 23 | 24 | println!("cargo:rerun-if-changed=src/log.fbs"); 25 | flatc_rust::run(flatc_rust::Args { 26 | inputs: &[Path::new("src/log.fbs")], 27 | out_dir: Path::new("target/flatbuffers/"), 28 | ..Default::default() 29 | }).expect("flatc"); 30 | } 31 | -------------------------------------------------------------------------------- /rust/src/lib.rs: -------------------------------------------------------------------------------- 1 | extern crate bincode; 2 | extern crate capnp; 3 | extern crate flatbuffers; 4 | extern crate num_traits; 5 | #[macro_use] 6 | extern crate num_derive; 7 | extern crate protobuf; 8 | extern crate serde; 9 | #[macro_use] 10 | extern crate serde_derive; 11 | extern crate serde_json; 12 | 13 | pub mod log_capnp { 14 | include!(concat!(env!("OUT_DIR"), "/log_capnp.rs")); 15 | } 16 | 17 | #[path = "../target/protobufs/log_proto.rs"] 18 | pub mod log_proto; 19 | 20 | #[allow(non_snake_case)] 21 | #[path = "../target/flatbuffers/log_generated.rs"] 22 | pub mod log_flatbuffers; 23 | 24 | use std::fmt; 25 | 26 | macro_rules! enum_number { 27 | ($name:ident { $($variant:ident = $value:expr, )* }) => { 28 | #[derive(Clone, Copy, Debug, FromPrimitive)] 29 | pub enum $name { 30 | $($variant = $value,)* 31 | } 32 | 33 | impl ::serde::Serialize for $name { 34 | fn serialize(&self, serializer: S) -> Result 35 | where 36 | S: ::serde::Serializer, 37 | { 38 | serializer.serialize_u64(*self as u64) 39 | } 40 | } 41 | 42 | impl<'de> ::serde::Deserialize<'de> for $name { 43 | fn deserialize(deserializer: D) -> Result 44 | where 45 | D: ::serde::Deserializer<'de>, 46 | { 47 | struct Visitor; 48 | 49 | impl<'de> ::serde::de::Visitor<'de> for Visitor { 50 | type Value = $name; 51 | 52 | fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result { 53 | formatter.write_str("positive integer") 54 | } 55 | 56 | fn visit_u64(self, value: u64) -> Result<$name, E> 57 | where 58 | E: ::serde::de::Error, 59 | { 60 | match value { 61 | $( $value => Ok($name::$variant), )* 62 | _ => Err(E::custom( 63 | format!("unknown {} value: {}", 64 | stringify!($name), value))), 65 | } 66 | } 67 | } 68 | 69 | deserializer.deserialize_u64(Visitor) 70 | } 71 | } 72 | } 73 | } 74 | 75 | #[derive(Clone, Serialize, Deserialize, Debug)] 76 | pub struct Http { 77 | pub protocol: HttpProtocol, 78 | pub status: u32, 79 | pub host_status: u32, 80 | pub up_status: u32, 81 | pub method: HttpMethod, 82 | pub content_type: String, 83 | pub user_agent: String, 84 | pub referer: String, 85 | pub request_uri: String, 86 | } 87 | 88 | enum_number!(HttpProtocol { 89 | HttpProtocolUnknown = 0, 90 | HTTP10 = 1, 91 | HTTP11 = 2, 92 | }); 93 | 94 | enum_number!(HttpMethod { 95 | MethodUnknown = 0, 96 | GET = 1, 97 | POST = 2, 98 | DELETE = 3, 99 | PUT = 4, 100 | HEAD = 5, 101 | PURGE = 6, 102 | OPTIONS = 7, 103 | PROPFIND = 8, 104 | MKCOL = 9, 105 | PATCH = 10, 106 | }); 107 | 108 | enum_number!(CacheStatus { 109 | CacheStatusUnknown = 0, 110 | Miss = 1, 111 | Expired = 2, 112 | Hit = 3, 113 | }); 114 | 115 | #[derive(Clone, Serialize, Deserialize, Debug)] 116 | pub struct Origin { 117 | pub ip: String, 118 | pub port: u32, 119 | pub hostname: String, 120 | pub protocol: OriginProtocol, 121 | } 122 | 123 | enum_number!(OriginProtocol { 124 | OriginProtocolUnknown = 0, 125 | HTTP = 1, 126 | HTTPS = 2, 127 | }); 128 | 129 | enum_number!(ZonePlan { 130 | ZonePlanUnknown = 0, 131 | FREE = 1, 132 | PRO = 2, 133 | BIZ = 3, 134 | ENT = 4, 135 | }); 136 | 137 | enum_number!(Country { 138 | UNKNOWN = 0, 139 | A1 = 1, 140 | A2 = 2, 141 | O1 = 3, 142 | AD = 4, 143 | AE = 5, 144 | AF = 6, 145 | AG = 7, 146 | AI = 8, 147 | AL = 9, 148 | AM = 10, 149 | AO = 11, 150 | AP = 12, 151 | AQ = 13, 152 | AR = 14, 153 | AS = 15, 154 | AT = 16, 155 | AU = 17, 156 | AW = 18, 157 | AX = 19, 158 | AZ = 20, 159 | BA = 21, 160 | BB = 22, 161 | BD = 23, 162 | BE = 24, 163 | BF = 25, 164 | BG = 26, 165 | BH = 27, 166 | BI = 28, 167 | BJ = 29, 168 | BL = 30, 169 | BM = 31, 170 | BN = 32, 171 | BO = 33, 172 | BQ = 34, 173 | BR = 35, 174 | BS = 36, 175 | BT = 37, 176 | BV = 38, 177 | BW = 39, 178 | BY = 40, 179 | BZ = 41, 180 | CA = 42, 181 | CC = 43, 182 | CD = 44, 183 | CF = 45, 184 | CG = 46, 185 | CH = 47, 186 | CI = 48, 187 | CK = 49, 188 | CL = 50, 189 | CM = 51, 190 | CN = 52, 191 | CO = 53, 192 | CR = 54, 193 | CU = 55, 194 | CV = 56, 195 | CW = 57, 196 | CX = 58, 197 | CY = 59, 198 | CZ = 60, 199 | DE = 61, 200 | DJ = 62, 201 | DK = 63, 202 | DM = 64, 203 | DO = 65, 204 | DZ = 66, 205 | EC = 67, 206 | EE = 68, 207 | EG = 69, 208 | EH = 70, 209 | ER = 71, 210 | ES = 72, 211 | ET = 73, 212 | EU = 74, 213 | FI = 75, 214 | FJ = 76, 215 | FK = 77, 216 | FM = 78, 217 | FO = 79, 218 | FR = 80, 219 | GA = 81, 220 | GB = 82, 221 | GD = 83, 222 | GE = 84, 223 | GF = 85, 224 | GG = 86, 225 | GH = 87, 226 | GI = 88, 227 | GL = 89, 228 | GM = 90, 229 | GN = 91, 230 | GP = 92, 231 | GQ = 93, 232 | GR = 94, 233 | GS = 95, 234 | GT = 96, 235 | GU = 97, 236 | GW = 98, 237 | GY = 99, 238 | HK = 100, 239 | HM = 101, 240 | HN = 102, 241 | HR = 103, 242 | HT = 104, 243 | HU = 105, 244 | ID = 106, 245 | IE = 107, 246 | IL = 108, 247 | IM = 109, 248 | IN = 110, 249 | IO = 111, 250 | IQ = 112, 251 | IR = 113, 252 | IS = 114, 253 | IT = 115, 254 | JE = 116, 255 | JM = 117, 256 | JO = 118, 257 | JP = 119, 258 | KE = 120, 259 | KG = 121, 260 | KH = 122, 261 | KI = 123, 262 | KM = 124, 263 | KN = 125, 264 | KP = 126, 265 | KR = 127, 266 | KW = 128, 267 | KY = 129, 268 | KZ = 130, 269 | LA = 131, 270 | LB = 132, 271 | LC = 133, 272 | LI = 134, 273 | LK = 135, 274 | LR = 136, 275 | LS = 137, 276 | LT = 138, 277 | LU = 139, 278 | LV = 140, 279 | LY = 141, 280 | MA = 142, 281 | MC = 143, 282 | MD = 144, 283 | ME = 145, 284 | MF = 146, 285 | MG = 147, 286 | MH = 148, 287 | MK = 149, 288 | ML = 150, 289 | MM = 151, 290 | MN = 152, 291 | MO = 153, 292 | MP = 154, 293 | MQ = 155, 294 | MR = 156, 295 | MS = 157, 296 | MT = 158, 297 | MU = 159, 298 | MV = 160, 299 | MW = 161, 300 | MX = 162, 301 | MY = 163, 302 | MZ = 164, 303 | NA = 165, 304 | NC = 166, 305 | NE = 167, 306 | NF = 168, 307 | NG = 169, 308 | NI = 170, 309 | NL = 171, 310 | NO = 172, 311 | NP = 173, 312 | NR = 174, 313 | NU = 175, 314 | NZ = 176, 315 | OM = 177, 316 | PA = 178, 317 | PE = 179, 318 | PF = 180, 319 | PG = 181, 320 | PH = 182, 321 | PK = 183, 322 | PL = 184, 323 | PM = 185, 324 | PN = 186, 325 | PR = 187, 326 | PS = 188, 327 | PT = 189, 328 | PW = 190, 329 | PY = 191, 330 | QA = 192, 331 | RE = 193, 332 | RO = 194, 333 | RS = 195, 334 | RU = 196, 335 | RW = 197, 336 | SA = 198, 337 | SB = 199, 338 | SC = 200, 339 | SD = 201, 340 | SE = 202, 341 | SG = 203, 342 | SH = 204, 343 | SI = 205, 344 | SJ = 206, 345 | SK = 207, 346 | SL = 208, 347 | SM = 209, 348 | SN = 210, 349 | SO = 211, 350 | SR = 212, 351 | SS = 213, 352 | ST = 214, 353 | SV = 215, 354 | SX = 216, 355 | SY = 217, 356 | SZ = 218, 357 | TC = 219, 358 | TD = 220, 359 | TF = 221, 360 | TG = 222, 361 | TH = 223, 362 | TJ = 224, 363 | TK = 225, 364 | TL = 226, 365 | TM = 227, 366 | TN = 228, 367 | TO = 229, 368 | TR = 230, 369 | TT = 231, 370 | TV = 232, 371 | TW = 233, 372 | TZ = 234, 373 | UA = 235, 374 | UG = 236, 375 | UM = 237, 376 | US = 238, 377 | UY = 239, 378 | UZ = 240, 379 | VA = 241, 380 | VC = 242, 381 | VE = 243, 382 | VG = 244, 383 | VI = 245, 384 | VN = 246, 385 | VU = 247, 386 | WF = 248, 387 | WS = 249, 388 | XX = 250, 389 | YE = 251, 390 | YT = 252, 391 | ZA = 253, 392 | ZM = 254, 393 | ZW = 255, 394 | }); 395 | 396 | #[derive(Clone, Serialize, Deserialize, Debug)] 397 | pub struct Log { 398 | pub timestamp: i64, 399 | pub zone_id: u32, 400 | pub zone_plan: ZonePlan, 401 | pub http: Http, 402 | pub origin: Origin, 403 | pub country: Country, 404 | pub cache_status: CacheStatus, 405 | pub server_ip: String, 406 | pub server_name: String, 407 | pub remote_ip: String, 408 | pub bytes_dlv: u64, 409 | pub ray_id: String, 410 | } 411 | 412 | impl Log { 413 | pub fn new() -> Log { 414 | Log { 415 | timestamp: 2837513946597, 416 | zone_id: 123456, 417 | zone_plan: ZonePlan::FREE, 418 | http: Http { 419 | protocol: HttpProtocol::HTTP11, 420 | status: 200, 421 | host_status: 503, 422 | up_status: 520, 423 | method: HttpMethod::GET, 424 | content_type: "text/html".to_string(), 425 | user_agent: "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.146 Safari/537.36".to_string(), 426 | referer: "https://www.cloudflare.com/".to_string(), 427 | request_uri: "/cdn-cgi/trace".to_string(), 428 | }, 429 | origin: Origin { 430 | ip: "1.2.3.4".to_string(), 431 | port: 8000, 432 | hostname: "www.example.com".to_string(), 433 | protocol: OriginProtocol::HTTPS, 434 | }, 435 | country: Country::US, 436 | cache_status: CacheStatus::Hit, 437 | server_ip: "192.168.1.1".to_string(), 438 | server_name: "metal.cloudflare.com".to_string(), 439 | remote_ip: "10.1.2.3".to_string(), 440 | bytes_dlv: 123456, 441 | ray_id: "10c73629cce30078-LAX".to_string(), 442 | } 443 | } 444 | } 445 | 446 | pub mod protocol_capnp { 447 | use capnp::message::{Allocator, Builder}; 448 | use crate::log_capnp::{h_t_t_p, log, origin, CacheStatus, Country, ZonePlan}; 449 | 450 | pub fn populate_log<'a, A: Allocator>(msg: &'a mut Builder) -> log::Builder<'a> { 451 | let mut log = msg.init_root::(); 452 | log.set_timestamp(2837513946597); 453 | log.set_zone_id(123456); 454 | log.set_zone_plan(ZonePlan::Free); 455 | 456 | { 457 | let mut http = log.reborrow().init_http(); 458 | http.set_protocol(h_t_t_p::Protocol::Http11); 459 | http.set_host_status(200); 460 | http.set_up_status(520); 461 | http.set_method(h_t_t_p::Method::Get); 462 | http.set_content_type("text/html"); 463 | http.set_user_agent("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.146 Safari/537.36"); 464 | http.set_referer("https://www.cloudflare.com/"); 465 | http.set_request_u_r_i("/cdn-cgi/trace"); 466 | } 467 | 468 | { 469 | let mut origin = log.reborrow().init_origin(); 470 | origin.set_ip("1.2.3.4"); 471 | origin.set_port(8000); 472 | origin.set_hostname("www.example.com"); 473 | origin.set_protocol(origin::Protocol::Https); 474 | } 475 | 476 | log.set_country(Country::Us); 477 | log.set_cache_status(CacheStatus::Hit); 478 | log.set_server_ip("192.168.1.1"); 479 | log.set_server_name("metal.cloudflare.com"); 480 | log.set_remote_ip("10.1.2.3"); 481 | log.set_bytes_dlv(123456); 482 | log.set_ray_id("10c73629cce30078-LAX"); 483 | 484 | log 485 | } 486 | } 487 | 488 | pub mod protocol_protobuf { 489 | use crate::log_proto; 490 | 491 | pub fn new_log() -> log_proto::Log { 492 | let mut log = log_proto::Log::new(); 493 | log.set_timestamp(2837513946597); 494 | log.set_zone_id(123456); 495 | log.set_zone_plan(log_proto::ZonePlan::FREE); 496 | 497 | let mut http = log_proto::HTTP::new(); 498 | http.set_protocol(log_proto::HTTP_Protocol::HTTP11); 499 | http.set_host_status(200); 500 | http.set_up_status(520); 501 | http.set_method(log_proto::HTTP_Method::GET); 502 | http.set_content_type("text/html".to_string()); 503 | http.set_user_agent("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.146 Safari/537.36".to_string()); 504 | http.set_referer("https://www.cloudflare.com/".to_string()); 505 | http.set_request_uri("/cdn-cgi/trace".to_string()); 506 | log.set_http(http); 507 | 508 | let mut origin = log_proto::Origin::new(); 509 | origin.set_ip([1, 2, 3, 4].to_vec()); 510 | origin.set_port(8000); 511 | origin.set_hostname("www.example.com".to_string()); 512 | origin.set_protocol(log_proto::Origin_Protocol::HTTPS); 513 | log.set_origin(origin); 514 | 515 | log.set_country(log_proto::Country::US); 516 | log.set_cache_status(log_proto::CacheStatus::HIT); 517 | log.set_server_ip([192, 168, 1, 1].to_vec()); 518 | log.set_server_name("metal.cloudflare.com".to_string()); 519 | log.set_remote_ip([10, 1, 2, 3].to_vec()); 520 | log.set_bytes_dlv(123456); 521 | log.set_ray_id("10c73629cce30078-LAX".to_string()); 522 | 523 | log 524 | } 525 | } 526 | 527 | pub mod protocol_flatbuffers { 528 | use crate::log_flatbuffers; 529 | 530 | pub fn populate_log_with_args<'a, 'b>(msg: &'b mut flatbuffers::FlatBufferBuilder<'a>) -> flatbuffers::WIPOffset> { 531 | let content_type = msg.create_string("text/html"); 532 | let user_agent = msg.create_string("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.146 Safari/537.36"); 533 | let referer = msg.create_string("https://www.cloudflare.com/"); 534 | let request_uri = msg.create_string("/cdn-cgi/trace"); 535 | 536 | let http = log_flatbuffers::HTTP::create( 537 | msg, 538 | &log_flatbuffers::HTTPArgs { 539 | protocol: log_flatbuffers::HTTP_Protocol::http11, 540 | hostStatus: 200, 541 | upStatus: 520, 542 | method: log_flatbuffers::HTTP_Method::get, 543 | contentType: Some(content_type), 544 | userAgent: Some(user_agent), 545 | referer: Some(referer), 546 | requestURI: Some(request_uri), 547 | ..Default::default() 548 | } 549 | ); 550 | 551 | let ip = Some(msg.create_string("1.2.3.4")); 552 | let hostname = Some(msg.create_string("www.example.com")); 553 | 554 | let origin = log_flatbuffers::Origin::create( 555 | msg, 556 | &log_flatbuffers::OriginArgs { 557 | ip, 558 | port: 8000, 559 | hostname, 560 | protocol: log_flatbuffers::Origin_Protocol::https, 561 | ..Default::default() 562 | } 563 | ); 564 | 565 | let server_ip = msg.create_string("192.168.1.1"); 566 | let server_name = msg.create_string("metal.cloudflare.com"); 567 | let remote_ip = msg.create_string("10.1.2.3"); 568 | let ray_id = msg.create_string("10c73629cce30078-LAX"); 569 | 570 | log_flatbuffers::Log::create( 571 | msg, 572 | &log_flatbuffers::LogArgs { 573 | http: Some(http), 574 | origin: Some(origin), 575 | timestamp: 2837513946597, 576 | zoneId: 123456, 577 | zonePlan: log_flatbuffers::ZonePlan::free, 578 | country: log_flatbuffers::Country::us, 579 | cacheStatus: log_flatbuffers::CacheStatus::hit, 580 | serverIp: Some(server_ip), 581 | serverName: Some(server_name), 582 | remoteIp: Some(remote_ip), 583 | bytesDlv: 123456, 584 | rayId: Some(ray_id), 585 | ..Default::default() 586 | } 587 | ) 588 | } 589 | 590 | pub fn populate_log_with_builder<'a, 'b>(msg: &'b mut flatbuffers::FlatBufferBuilder<'a>) -> flatbuffers::WIPOffset> { 591 | let content_type = msg.create_string("text/html"); 592 | let user_agent = msg.create_string("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/33.0.1750.146 Safari/537.36"); 593 | let referer = msg.create_string("https://www.cloudflare.com/"); 594 | let request_uri = msg.create_string("/cdn-cgi/trace"); 595 | 596 | let http = { 597 | let mut http_builder = log_flatbuffers::HTTPBuilder::new(msg); 598 | http_builder.add_protocol(log_flatbuffers::HTTP_Protocol::http11); 599 | http_builder.add_hostStatus(200); 600 | http_builder.add_upStatus(520); 601 | http_builder.add_method(log_flatbuffers::HTTP_Method::get); 602 | http_builder.add_contentType(content_type); 603 | http_builder.add_userAgent(user_agent); 604 | http_builder.add_referer(referer); 605 | http_builder.add_requestURI(request_uri); 606 | http_builder.finish() 607 | }; 608 | 609 | let ip = msg.create_string("1.2.3.4"); 610 | let hostname = msg.create_string("www.example.com"); 611 | 612 | let origin = { 613 | let mut origin_builder = log_flatbuffers::OriginBuilder::new(msg); 614 | origin_builder.add_ip(ip); 615 | origin_builder.add_port(8000); 616 | origin_builder.add_hostname(hostname); 617 | origin_builder.add_protocol(log_flatbuffers::Origin_Protocol::https); 618 | origin_builder.finish() 619 | }; 620 | 621 | let server_ip = msg.create_string("192.168.1.1"); 622 | let server_name = msg.create_string("metal.cloudflare.com"); 623 | let remote_ip = msg.create_string("10.1.2.3"); 624 | let ray_id = msg.create_string("10c73629cce30078-LAX"); 625 | 626 | let mut log_builder = log_flatbuffers::LogBuilder::new(msg); 627 | log_builder.add_http(http); 628 | log_builder.add_origin(origin); 629 | log_builder.add_timestamp(2837513946597); 630 | log_builder.add_zoneId(123456); 631 | log_builder.add_zonePlan(log_flatbuffers::ZonePlan::free); 632 | log_builder.add_country(log_flatbuffers::Country::us); 633 | log_builder.add_cacheStatus(log_flatbuffers::CacheStatus::hit); 634 | log_builder.add_serverIp(server_ip); 635 | log_builder.add_serverName(server_name); 636 | log_builder.add_remoteIp(remote_ip); 637 | log_builder.add_bytesDlv(123456); 638 | log_builder.add_rayId(ray_id); 639 | log_builder.finish() 640 | } 641 | } 642 | -------------------------------------------------------------------------------- /rust/src/log.capnp: -------------------------------------------------------------------------------- 1 | @0x9512134be87cbc78; 2 | 3 | struct HTTP { 4 | protocol @0 :Protocol; 5 | enum Protocol { 6 | unknown @0; 7 | http10 @1; 8 | http11 @2; 9 | max @3; 10 | } 11 | 12 | status @1 :UInt16; 13 | 14 | hostStatus @2 :UInt16; 15 | 16 | upStatus @3 :UInt16; 17 | 18 | method @4 :Method; 19 | enum Method { 20 | unknown @0; 21 | get @1; 22 | post @2; 23 | delete @3; 24 | put @4; 25 | head @5; 26 | purge @6; 27 | options @7; 28 | propfind @8; 29 | mkcol @9; 30 | patch @10; 31 | max @11; 32 | } 33 | 34 | contentType @5 :Text; 35 | userAgent @6 :Text; 36 | referer @7 :Text; 37 | requestURI @8 :Text; 38 | } 39 | 40 | enum CacheStatus { 41 | unknown @0; 42 | miss @1; 43 | expired @2; 44 | hit @3; 45 | max @4; 46 | } 47 | 48 | struct Origin { 49 | ip @0 :Text; 50 | port @1 :UInt16; 51 | hostname @2 :Text; 52 | protocol @3 :Protocol; 53 | enum Protocol { 54 | unknown @0; 55 | http @1; 56 | https @2; 57 | max @3; 58 | } 59 | } 60 | 61 | enum ZonePlan { 62 | unknown @0; 63 | free @1; 64 | pro @2; 65 | biz @3; 66 | ent @4; 67 | max @5; 68 | } 69 | 70 | enum Country { 71 | unknown @0; 72 | # Anonymous Proxy 73 | a1 @1; 74 | # Satellite Provider 75 | a2 @2; 76 | # Other Country 77 | o1 @3; 78 | # Andorra 79 | ad @4; 80 | # United Arab Emirates 81 | ae @5; 82 | # Afghanistan 83 | af @6; 84 | # Antigua and Barbuda 85 | ag @7; 86 | # Anguilla 87 | ai @8; 88 | # Albania 89 | al @9; 90 | # Armenia 91 | am @10; 92 | # Angola 93 | ao @11; 94 | # Asia/Pacific Region 95 | ap @12; 96 | # Antarctica 97 | aq @13; 98 | # Argentina 99 | ar @14; 100 | # American Samoa 101 | as @15; 102 | # Austria 103 | at @16; 104 | # Australia 105 | au @17; 106 | # Aruba 107 | aw @18; 108 | # Aland Islands 109 | ax @19; 110 | # Azerbaijan 111 | az @20; 112 | # Bosnia and Herzegovina 113 | ba @21; 114 | # Barbados 115 | bb @22; 116 | # Bangladesh 117 | bd @23; 118 | # Belgium 119 | be @24; 120 | # Burkina Faso 121 | bf @25; 122 | # Bulgaria 123 | bg @26; 124 | # Bahrain 125 | bh @27; 126 | # Burundi 127 | bi @28; 128 | # Benin 129 | bj @29; 130 | # Saint Bartelemey 131 | bl @30; 132 | # Bermuda 133 | bm @31; 134 | # Brunei Darussalam 135 | bn @32; 136 | # Bolivia 137 | bo @33; 138 | # Bonaire, Saint Eustatius and Saba 139 | bq @34; 140 | # Brazil 141 | br @35; 142 | # Bahamas 143 | bs @36; 144 | # Bhutan 145 | bt @37; 146 | # Bouvet Island 147 | bv @38; 148 | # Botswana 149 | bw @39; 150 | # Belarus 151 | by @40; 152 | # Belize 153 | bz @41; 154 | # Canada 155 | ca @42; 156 | # Cocos (Keeling) Islands 157 | cc @43; 158 | # Congo, The Democratic Republic of the 159 | cd @44; 160 | # Central African Republic 161 | cf @45; 162 | # Congo 163 | cg @46; 164 | # Switzerland 165 | ch @47; 166 | # Cote d'Ivoire 167 | ci @48; 168 | # Cook Islands 169 | ck @49; 170 | # Chile 171 | cl @50; 172 | # Cameroon 173 | cm @51; 174 | # China 175 | cn @52; 176 | # Colombia 177 | co @53; 178 | # Costa Rica 179 | cr @54; 180 | # Cuba 181 | cu @55; 182 | # Cape Verde 183 | cv @56; 184 | # Curacao 185 | cw @57; 186 | # Christmas Island 187 | cx @58; 188 | # Cyprus 189 | cy @59; 190 | # Czech Republic 191 | cz @60; 192 | # Germany 193 | de @61; 194 | # Djibouti 195 | dj @62; 196 | # Denmark 197 | dk @63; 198 | # Dominica 199 | dm @64; 200 | # Dominican Republic 201 | do @65; 202 | # Algeria 203 | dz @66; 204 | # Ecuador 205 | ec @67; 206 | # Estonia 207 | ee @68; 208 | # Egypt 209 | eg @69; 210 | # Western Sahara 211 | eh @70; 212 | # Eritrea 213 | er @71; 214 | # Spain 215 | es @72; 216 | # Ethiopia 217 | et @73; 218 | # Europe 219 | eu @74; 220 | # Finland 221 | fi @75; 222 | # Fiji 223 | fj @76; 224 | # Falkland Islands (Malvinas) 225 | fk @77; 226 | # Micronesia, Federated States of 227 | fm @78; 228 | # Faroe Islands 229 | fo @79; 230 | # France 231 | fr @80; 232 | # Gabon 233 | ga @81; 234 | # United Kingdom 235 | gb @82; 236 | # Grenada 237 | gd @83; 238 | # Georgia 239 | ge @84; 240 | # French Guiana 241 | gf @85; 242 | # Guernsey 243 | gg @86; 244 | # Ghana 245 | gh @87; 246 | # Gibraltar 247 | gi @88; 248 | # Greenland 249 | gl @89; 250 | # Gambia 251 | gm @90; 252 | # Guinea 253 | gn @91; 254 | # Guadeloupe 255 | gp @92; 256 | # Equatorial Guinea 257 | gq @93; 258 | # Greece 259 | gr @94; 260 | # South Georgia and the South Sandwich Islands 261 | gs @95; 262 | # Guatemala 263 | gt @96; 264 | # Guam 265 | gu @97; 266 | # Guinea-Bissau 267 | gw @98; 268 | # Guyana 269 | gy @99; 270 | # Hong Kong 271 | hk @100; 272 | # Heard Island and McDonald Islands 273 | hm @101; 274 | # Honduras 275 | hn @102; 276 | # Croatia 277 | hr @103; 278 | # Haiti 279 | ht @104; 280 | # Hungary 281 | hu @105; 282 | # Indonesia 283 | id @106; 284 | # Ireland 285 | ie @107; 286 | # Israel 287 | il @108; 288 | # Isle of Man 289 | im @109; 290 | # India 291 | in @110; 292 | # British Indian Ocean Territory 293 | io @111; 294 | # Iraq 295 | iq @112; 296 | # Iran, Islamic Republic of 297 | ir @113; 298 | # Iceland 299 | is @114; 300 | # Italy 301 | it @115; 302 | # Jersey 303 | je @116; 304 | # Jamaica 305 | jm @117; 306 | # Jordan 307 | jo @118; 308 | # Japan 309 | jp @119; 310 | # Kenya 311 | ke @120; 312 | # Kyrgyzstan 313 | kg @121; 314 | # Cambodia 315 | kh @122; 316 | # Kiribati 317 | ki @123; 318 | # Comoros 319 | km @124; 320 | # Saint Kitts and Nevis 321 | kn @125; 322 | # Korea, Democratic People's Republic of 323 | kp @126; 324 | # Korea, Republic of 325 | kr @127; 326 | # Kuwait 327 | kw @128; 328 | # Cayman Islands 329 | ky @129; 330 | # Kazakhstan 331 | kz @130; 332 | # Lao People's Democratic Republic 333 | la @131; 334 | # Lebanon 335 | lb @132; 336 | # Saint Lucia 337 | lc @133; 338 | # Liechtenstein 339 | li @134; 340 | # Sri Lanka 341 | lk @135; 342 | # Liberia 343 | lr @136; 344 | # Lesotho 345 | ls @137; 346 | # Lithuania 347 | lt @138; 348 | # Luxembourg 349 | lu @139; 350 | # Latvia 351 | lv @140; 352 | # Libyan Arab Jamahiriya 353 | ly @141; 354 | # Morocco 355 | ma @142; 356 | # Monaco 357 | mc @143; 358 | # Moldova, Republic of 359 | md @144; 360 | # Montenegro 361 | me @145; 362 | # Saint Martin 363 | mf @146; 364 | # Madagascar 365 | mg @147; 366 | # Marshall Islands 367 | mh @148; 368 | # Macedonia 369 | mk @149; 370 | # Mali 371 | ml @150; 372 | # Myanmar 373 | mm @151; 374 | # Mongolia 375 | mn @152; 376 | # Macao 377 | mo @153; 378 | # Northern Mariana Islands 379 | mp @154; 380 | # Martinique 381 | mq @155; 382 | # Mauritania 383 | mr @156; 384 | # Montserrat 385 | ms @157; 386 | # Malta 387 | mt @158; 388 | # Mauritius 389 | mu @159; 390 | # Maldives 391 | mv @160; 392 | # Malawi 393 | mw @161; 394 | # Mexico 395 | mx @162; 396 | # Malaysia 397 | my @163; 398 | # Mozambique 399 | mz @164; 400 | # Namibia 401 | na @165; 402 | # New Caledonia 403 | nc @166; 404 | # Niger 405 | ne @167; 406 | # Norfolk Island 407 | nf @168; 408 | # Nigeria 409 | ng @169; 410 | # Nicaragua 411 | ni @170; 412 | # Netherlands 413 | nl @171; 414 | # Norway 415 | no @172; 416 | # Nepal 417 | np @173; 418 | # Nauru 419 | nr @174; 420 | # Niue 421 | nu @175; 422 | # New Zealand 423 | nz @176; 424 | # Oman 425 | om @177; 426 | # Panama 427 | pa @178; 428 | # Peru 429 | pe @179; 430 | # French Polynesia 431 | pf @180; 432 | # Papua New Guinea 433 | pg @181; 434 | # Philippines 435 | ph @182; 436 | # Pakistan 437 | pk @183; 438 | # Poland 439 | pl @184; 440 | # Saint Pierre and Miquelon 441 | pm @185; 442 | # Pitcairn 443 | pn @186; 444 | # Puerto Rico 445 | pr @187; 446 | # Palestinian Territory 447 | ps @188; 448 | # Portugal 449 | pt @189; 450 | # Palau 451 | pw @190; 452 | # Paraguay 453 | py @191; 454 | # Qatar 455 | qa @192; 456 | # Reunion 457 | re @193; 458 | # Romania 459 | ro @194; 460 | # Serbia 461 | rs @195; 462 | # Russian Federation 463 | ru @196; 464 | # Rwanda 465 | rw @197; 466 | # Saudi Arabia 467 | sa @198; 468 | # Solomon Islands 469 | sb @199; 470 | # Seychelles 471 | sc @200; 472 | # Sudan 473 | sd @201; 474 | # Sweden 475 | se @202; 476 | # Singapore 477 | sg @203; 478 | # Saint Helena 479 | sh @204; 480 | # Slovenia 481 | si @205; 482 | # Svalbard and Jan Mayen 483 | sj @206; 484 | # Slovakia 485 | sk @207; 486 | # Sierra Leone 487 | sl @208; 488 | # San Marino 489 | sm @209; 490 | # Senegal 491 | sn @210; 492 | # Somalia 493 | so @211; 494 | # Suriname 495 | sr @212; 496 | # South Sudan 497 | ss @213; 498 | # Sao Tome and Principe 499 | st @214; 500 | # El Salvador 501 | sv @215; 502 | # Sint Maarten 503 | sx @216; 504 | # Syrian Arab Republic 505 | sy @217; 506 | # Swaziland 507 | sz @218; 508 | # Turks and Caicos Islands 509 | tc @219; 510 | # Chad 511 | td @220; 512 | # French Southern Territories 513 | tf @221; 514 | # Togo 515 | tg @222; 516 | # Thailand 517 | th @223; 518 | # Tajikistan 519 | tj @224; 520 | # Tokelau 521 | tk @225; 522 | # Timor-Leste 523 | tl @226; 524 | # Turkmenistan 525 | tm @227; 526 | # Tunisia 527 | tn @228; 528 | # Tonga 529 | to @229; 530 | # Turkey 531 | tr @230; 532 | # Trinidad and Tobago 533 | tt @231; 534 | # Tuvalu 535 | tv @232; 536 | # Taiwan 537 | tw @233; 538 | # Tanzania, United Republic of 539 | tz @234; 540 | # Ukraine 541 | ua @235; 542 | # Uganda 543 | ug @236; 544 | # United States Minor Outlying Islands 545 | um @237; 546 | # United States 547 | us @238; 548 | # Uruguay 549 | uy @239; 550 | # Uzbekistan 551 | uz @240; 552 | # Holy See (Vatican City State) 553 | va @241; 554 | # Saint Vincent and the Grenadines 555 | vc @242; 556 | # Venezuela 557 | ve @243; 558 | # Virgin Islands, British 559 | vg @244; 560 | # Virgin Islands, U.S. 561 | vi @245; 562 | # Vietnam 563 | vn @246; 564 | # Vanuatu 565 | vu @247; 566 | # Wallis and Futuna 567 | wf @248; 568 | # Samoa 569 | ws @249; 570 | # Other Countries 571 | xx @250; 572 | # Yemen 573 | ye @251; 574 | # Mayotte 575 | yt @252; 576 | # South Africa 577 | za @253; 578 | # Zambia 579 | zm @254; 580 | # Zimbabwe 581 | zw @255; 582 | max @256; 583 | } 584 | 585 | struct Log { 586 | timestamp @0 :Int64; 587 | zoneId @1 :UInt32; 588 | zonePlan @2 :ZonePlan; 589 | http @3 :HTTP; 590 | origin @4 :Origin; 591 | country @5 :Country; 592 | cacheStatus @6 :CacheStatus; 593 | serverIp @7 :Text; 594 | serverName @8 :Text; 595 | remoteIp @9 :Text; 596 | bytesDlv @10 :UInt64; 597 | rayId @11 :Text; 598 | } 599 | -------------------------------------------------------------------------------- /rust/src/log.fbs: -------------------------------------------------------------------------------- 1 | enum HTTP_Protocol:uint8 { 2 | unknown, 3 | http10, 4 | http11, 5 | } 6 | 7 | enum HTTP_Method:uint8 { 8 | unknown, 9 | get, 10 | post, 11 | delete, 12 | put, 13 | head, 14 | purge, 15 | options, 16 | propfind, 17 | mkcol, 18 | patch, 19 | } 20 | 21 | table HTTP { 22 | protocol:HTTP_Protocol; 23 | status:uint16; 24 | 25 | hostStatus:uint16; 26 | 27 | upStatus:uint16; 28 | 29 | method:HTTP_Method; 30 | contentType:string; 31 | userAgent:string; 32 | referer:string; 33 | requestURI:string; 34 | } 35 | 36 | enum CacheStatus:uint8 { 37 | unknown, 38 | miss, 39 | expired, 40 | hit, 41 | } 42 | 43 | enum Origin_Protocol:uint8 { 44 | unknown, 45 | http, 46 | https, 47 | } 48 | 49 | table Origin { 50 | ip:string; 51 | port:uint16; 52 | hostname:string; 53 | protocol:Origin_Protocol; 54 | } 55 | 56 | enum ZonePlan:uint8 { 57 | unknown, 58 | free, 59 | pro, 60 | biz, 61 | ent, 62 | } 63 | 64 | enum Country:uint16 { 65 | unknown, 66 | // Anonymous Proxy 67 | a1, 68 | // Satellite Provider 69 | a2, 70 | // Other Country 71 | o1, 72 | // Andorra 73 | ad, 74 | // United Arab Emirates 75 | ae, 76 | // Afghanistan 77 | af, 78 | // Antigua and Barbuda 79 | ag, 80 | // Anguilla 81 | ai, 82 | // Albania 83 | al, 84 | // Armenia 85 | am, 86 | // Angola 87 | ao, 88 | // Asia/Pacific Region 89 | ap, 90 | // Antarctica 91 | aq, 92 | // Argentina 93 | ar, 94 | // American Samoa 95 | as, 96 | // Austria 97 | at, 98 | // Australia 99 | au, 100 | // Aruba 101 | aw, 102 | // Aland Islands 103 | ax, 104 | // Azerbaijan 105 | az, 106 | // Bosnia and Herzegovina 107 | ba, 108 | // Barbados 109 | bb, 110 | // Bangladesh 111 | bd, 112 | // Belgium 113 | be, 114 | // Burkina Faso 115 | bf, 116 | // Bulgaria 117 | bg, 118 | // Bahrain 119 | bh, 120 | // Burundi 121 | bi, 122 | // Benin 123 | bj, 124 | // Saint Bartelemey 125 | bl, 126 | // Bermuda 127 | bm, 128 | // Brunei Darussalam 129 | bn, 130 | // Bolivia 131 | bo, 132 | // Bonaire, Saint Eustatius and Saba 133 | bq, 134 | // Brazil 135 | br, 136 | // Bahamas 137 | bs, 138 | // Bhutan 139 | bt, 140 | // Bouvet Island 141 | bv, 142 | // Botswana 143 | bw, 144 | // Belarus 145 | by, 146 | // Belize 147 | bz, 148 | // Canada 149 | ca, 150 | // Cocos (Keeling) Islands 151 | cc, 152 | // Congo, The Democratic Republic of the 153 | cd, 154 | // Central African Republic 155 | cf, 156 | // Congo 157 | cg, 158 | // Switzerland 159 | ch, 160 | // Cote d'Ivoire 161 | ci, 162 | // Cook Islands 163 | ck, 164 | // Chile 165 | cl, 166 | // Cameroon 167 | cm, 168 | // China 169 | cn, 170 | // Colombia 171 | co, 172 | // Costa Rica 173 | cr, 174 | // Cuba 175 | cu, 176 | // Cape Verde 177 | cv, 178 | // Curacao 179 | cw, 180 | // Christmas Island 181 | cx, 182 | // Cyprus 183 | cy, 184 | // Czech Republic 185 | cz, 186 | // Germany 187 | de, 188 | // Djibouti 189 | dj, 190 | // Denmark 191 | dk, 192 | // Dominica 193 | dm, 194 | // Dominican Republic 195 | do, 196 | // Algeria 197 | dz, 198 | // Ecuador 199 | ec, 200 | // Estonia 201 | ee, 202 | // Egypt 203 | eg, 204 | // Western Sahara 205 | eh, 206 | // Eritrea 207 | er, 208 | // Spain 209 | es, 210 | // Ethiopia 211 | et, 212 | // Europe 213 | eu, 214 | // Finland 215 | fi, 216 | // Fiji 217 | fj, 218 | // Falkland Islands (Malvinas) 219 | fk, 220 | // Micronesia, Federated States of 221 | fm, 222 | // Faroe Islands 223 | fo, 224 | // France 225 | fr, 226 | // Gabon 227 | ga, 228 | // United Kingdom 229 | gb, 230 | // Grenada 231 | gd, 232 | // Georgia 233 | ge, 234 | // French Guiana 235 | gf, 236 | // Guernsey 237 | gg, 238 | // Ghana 239 | gh, 240 | // Gibraltar 241 | gi, 242 | // Greenland 243 | gl, 244 | // Gambia 245 | gm, 246 | // Guinea 247 | gn, 248 | // Guadeloupe 249 | gp, 250 | // Equatorial Guinea 251 | gq, 252 | // Greece 253 | gr, 254 | // South Georgia and the South Sandwich Islands 255 | gs, 256 | // Guatemala 257 | gt, 258 | // Guam 259 | gu, 260 | // Guinea-Bissau 261 | gw, 262 | // Guyana 263 | gy, 264 | // Hong Kong 265 | hk, 266 | // Heard Island and McDonald Islands 267 | hm, 268 | // Honduras 269 | hn, 270 | // Croatia 271 | hr, 272 | // Haiti 273 | ht, 274 | // Hungary 275 | hu, 276 | // Indonesia 277 | id, 278 | // Ireland 279 | ie, 280 | // Israel 281 | il, 282 | // Isle of Man 283 | im, 284 | // India 285 | in, 286 | // British Indian Ocean Territory 287 | io, 288 | // Iraq 289 | iq, 290 | // Iran, Islamic Republic of 291 | ir, 292 | // Iceland 293 | is, 294 | // Italy 295 | it, 296 | // Jersey 297 | je, 298 | // Jamaica 299 | jm, 300 | // Jordan 301 | jo, 302 | // Japan 303 | jp, 304 | // Kenya 305 | ke, 306 | // Kyrgyzstan 307 | kg, 308 | // Cambodia 309 | kh, 310 | // Kiribati 311 | ki, 312 | // Comoros 313 | km, 314 | // Saint Kitts and Nevis 315 | kn, 316 | // Korea, Democratic People's Republic of 317 | kp, 318 | // Korea, Republic of 319 | kr, 320 | // Kuwait 321 | kw, 322 | // Cayman Islands 323 | ky, 324 | // Kazakhstan 325 | kz, 326 | // Lao People's Democratic Republic 327 | la, 328 | // Lebanon 329 | lb, 330 | // Saint Lucia 331 | lc, 332 | // Liechtenstein 333 | li, 334 | // Sri Lanka 335 | lk, 336 | // Liberia 337 | lr, 338 | // Lesotho 339 | ls, 340 | // Lithuania 341 | lt, 342 | // Luxembourg 343 | lu, 344 | // Latvia 345 | lv, 346 | // Libyan Arab Jamahiriya 347 | ly, 348 | // Morocco 349 | ma, 350 | // Monaco 351 | mc, 352 | // Moldova, Republic of 353 | md, 354 | // Montenegro 355 | me, 356 | // Saint Martin 357 | mf, 358 | // Madagascar 359 | mg, 360 | // Marshall Islands 361 | mh, 362 | // Macedonia 363 | mk, 364 | // Mali 365 | ml, 366 | // Myanmar 367 | mm, 368 | // Mongolia 369 | mn, 370 | // Macao 371 | mo, 372 | // Northern Mariana Islands 373 | mp, 374 | // Martinique 375 | mq, 376 | // Mauritania 377 | mr, 378 | // Montserrat 379 | ms, 380 | // Malta 381 | mt, 382 | // Mauritius 383 | mu, 384 | // Maldives 385 | mv, 386 | // Malawi 387 | mw, 388 | // Mexico 389 | mx, 390 | // Malaysia 391 | my, 392 | // Mozambique 393 | mz, 394 | // Namibia 395 | na, 396 | // New Caledonia 397 | nc, 398 | // Niger 399 | ne, 400 | // Norfolk Island 401 | nf, 402 | // Nigeria 403 | ng, 404 | // Nicaragua 405 | ni, 406 | // Netherlands 407 | nl, 408 | // Norway 409 | no, 410 | // Nepal 411 | np, 412 | // Nauru 413 | nr, 414 | // Niue 415 | nu, 416 | // New Zealand 417 | nz, 418 | // Oman 419 | om, 420 | // Panama 421 | pa, 422 | // Peru 423 | pe, 424 | // French Polynesia 425 | pf, 426 | // Papua New Guinea 427 | pg, 428 | // Philippines 429 | ph, 430 | // Pakistan 431 | pk, 432 | // Poland 433 | pl, 434 | // Saint Pierre and Miquelon 435 | pm, 436 | // Pitcairn 437 | pn, 438 | // Puerto Rico 439 | pr, 440 | // Palestinian Territory 441 | ps, 442 | // Portugal 443 | pt, 444 | // Palau 445 | pw, 446 | // Paraguay 447 | py, 448 | // Qatar 449 | qa, 450 | // Reunion 451 | re, 452 | // Romania 453 | ro, 454 | // Serbia 455 | rs, 456 | // Russian Federation 457 | ru, 458 | // Rwanda 459 | rw, 460 | // Saudi Arabia 461 | sa, 462 | // Solomon Islands 463 | sb, 464 | // Seychelles 465 | sc, 466 | // Sudan 467 | sd, 468 | // Sweden 469 | se, 470 | // Singapore 471 | sg, 472 | // Saint Helena 473 | sh, 474 | // Slovenia 475 | si, 476 | // Svalbard and Jan Mayen 477 | sj, 478 | // Slovakia 479 | sk, 480 | // Sierra Leone 481 | sl, 482 | // San Marino 483 | sm, 484 | // Senegal 485 | sn, 486 | // Somalia 487 | so, 488 | // Suriname 489 | sr, 490 | // South Sudan 491 | ss, 492 | // Sao Tome and Principe 493 | st, 494 | // El Salvador 495 | sv, 496 | // Sint Maarten 497 | sx, 498 | // Syrian Arab Republic 499 | sy, 500 | // Swaziland 501 | sz, 502 | // Turks and Caicos Islands 503 | tc, 504 | // Chad 505 | td, 506 | // French Southern Territories 507 | tf, 508 | // Togo 509 | tg, 510 | // Thailand 511 | th, 512 | // Tajikistan 513 | tj, 514 | // Tokelau 515 | tk, 516 | // Timor-Leste 517 | tl, 518 | // Turkmenistan 519 | tm, 520 | // Tunisia 521 | tn, 522 | // Tonga 523 | to, 524 | // Turkey 525 | tr, 526 | // Trinidad and Tobago 527 | tt, 528 | // Tuvalu 529 | tv, 530 | // Taiwan 531 | tw, 532 | // Tanzania, United Republic of 533 | tz, 534 | // Ukraine 535 | ua, 536 | // Uganda 537 | ug, 538 | // United States Minor Outlying Islands 539 | um, 540 | // United States 541 | us, 542 | // Uruguay 543 | uy, 544 | // Uzbekistan 545 | uz, 546 | // Holy See (Vatican City State) 547 | va, 548 | // Saint Vincent and the Grenadines 549 | vc, 550 | // Venezuela 551 | ve, 552 | // Virgin Islands, British 553 | vg, 554 | // Virgin Islands, U.S. 555 | vi, 556 | // Vietnam 557 | vn, 558 | // Vanuatu 559 | vu, 560 | // Wallis and Futuna 561 | wf, 562 | // Samoa 563 | ws, 564 | // Other Countries 565 | xx, 566 | // Yemen 567 | ye, 568 | // Mayotte 569 | yt, 570 | // South Africa 571 | za, 572 | // Zambia 573 | zm, 574 | // Zimbabwe 575 | zw, 576 | } 577 | 578 | table Log { 579 | timestamp:int64; 580 | zoneId:uint32; 581 | zonePlan:ZonePlan; 582 | http:HTTP; 583 | origin:Origin; 584 | country:Country; 585 | cacheStatus:CacheStatus; 586 | serverIp:string; 587 | serverName:string; 588 | remoteIp:string; 589 | bytesDlv:uint64; 590 | rayId:string; 591 | } 592 | 593 | root_type Log; 594 | -------------------------------------------------------------------------------- /rust/src/log_proto.proto: -------------------------------------------------------------------------------- 1 | syntax = "proto2"; 2 | package pb; 3 | 4 | message HTTP { 5 | enum Protocol { 6 | HTTP_PROTOCOL_UNKNOWN = 0; 7 | HTTP10 = 1; 8 | HTTP11 = 2; 9 | } 10 | optional Protocol protocol = 1; 11 | 12 | optional uint32 status = 2; 13 | optional uint32 host_status = 3; 14 | optional uint32 up_status = 4; 15 | 16 | enum Method { 17 | METHOD_UNKNOWN = 0; 18 | GET = 1; 19 | POST = 2; 20 | DELETE = 3; 21 | PUT = 4; 22 | HEAD = 5; 23 | PURGE = 6; 24 | OPTIONS = 7; 25 | PROPFIND = 8; 26 | MKCOL = 9; 27 | PATCH = 10; 28 | } 29 | optional Method method = 5; 30 | 31 | optional string content_type = 6; 32 | optional string user_agent = 7; 33 | optional string referer = 8; 34 | optional string request_uri = 9; 35 | } 36 | 37 | enum CacheStatus { 38 | CACHESTATUS_UNKNOWN = 0; 39 | MISS = 1; 40 | EXPIRED = 2; 41 | HIT = 3; 42 | } 43 | 44 | message Origin { 45 | optional bytes ip = 1; 46 | optional uint32 port = 2; 47 | optional string hostname = 3; 48 | enum Protocol { 49 | ORIGIN_PROTOCOL_UNKNOWN = 0; 50 | HTTP = 1; 51 | HTTPS = 2; 52 | } 53 | optional Protocol protocol = 4; 54 | } 55 | 56 | enum ZonePlan { 57 | ZONEPLAN_UNKNOWN = 0; 58 | FREE = 1; 59 | PRO = 2; 60 | BIZ = 3; 61 | ENT = 4; 62 | } 63 | 64 | message Log { 65 | optional sfixed64 timestamp = 1; 66 | optional uint32 zone_id = 2; 67 | optional ZonePlan zone_plan = 3; 68 | optional HTTP http = 4; 69 | optional Origin origin = 5; 70 | optional Country country = 6; 71 | optional CacheStatus cache_status = 7; 72 | optional bytes server_ip = 8; 73 | optional string server_name = 9; 74 | optional bytes remote_ip = 10; 75 | optional uint64 bytes_dlv = 11; 76 | optional string ray_id = 12; 77 | } 78 | 79 | enum Country { 80 | UNKNOWN = 0; 81 | A1 = 1; 82 | A2 = 2; 83 | O1 = 3; 84 | AD = 4; 85 | AE = 5; 86 | AF = 6; 87 | AG = 7; 88 | AI = 8; 89 | AL = 9; 90 | AM = 10; 91 | AO = 11; 92 | AP = 12; 93 | AQ = 13; 94 | AR = 14; 95 | AS = 15; 96 | AT = 16; 97 | AU = 17; 98 | AW = 18; 99 | AX = 19; 100 | AZ = 20; 101 | BA = 21; 102 | BB = 22; 103 | BD = 23; 104 | BE = 24; 105 | BF = 25; 106 | BG = 26; 107 | BH = 27; 108 | BI = 28; 109 | BJ = 29; 110 | BL = 30; 111 | BM = 31; 112 | BN = 32; 113 | BO = 33; 114 | BQ = 34; 115 | BR = 35; 116 | BS = 36; 117 | BT = 37; 118 | BV = 38; 119 | BW = 39; 120 | BY = 40; 121 | BZ = 41; 122 | CA = 42; 123 | CC = 43; 124 | CD = 44; 125 | CF = 45; 126 | CG = 46; 127 | CH = 47; 128 | CI = 48; 129 | CK = 49; 130 | CL = 50; 131 | CM = 51; 132 | CN = 52; 133 | CO = 53; 134 | CR = 54; 135 | CU = 55; 136 | CV = 56; 137 | CW = 57; 138 | CX = 58; 139 | CY = 59; 140 | CZ = 60; 141 | DE = 61; 142 | DJ = 62; 143 | DK = 63; 144 | DM = 64; 145 | DO = 65; 146 | DZ = 66; 147 | EC = 67; 148 | EE = 68; 149 | EG = 69; 150 | EH = 70; 151 | ER = 71; 152 | ES = 72; 153 | ET = 73; 154 | EU = 74; 155 | FI = 75; 156 | FJ = 76; 157 | FK = 77; 158 | FM = 78; 159 | FO = 79; 160 | FR = 80; 161 | GA = 81; 162 | GB = 82; 163 | GD = 83; 164 | GE = 84; 165 | GF = 85; 166 | GG = 86; 167 | GH = 87; 168 | GI = 88; 169 | GL = 89; 170 | GM = 90; 171 | GN = 91; 172 | GP = 92; 173 | GQ = 93; 174 | GR = 94; 175 | GS = 95; 176 | GT = 96; 177 | GU = 97; 178 | GW = 98; 179 | GY = 99; 180 | HK = 100; 181 | HM = 101; 182 | HN = 102; 183 | HR = 103; 184 | HT = 104; 185 | HU = 105; 186 | ID = 106; 187 | IE = 107; 188 | IL = 108; 189 | IM = 109; 190 | IN = 110; 191 | IO = 111; 192 | IQ = 112; 193 | IR = 113; 194 | IS = 114; 195 | IT = 115; 196 | JE = 116; 197 | JM = 117; 198 | JO = 118; 199 | JP = 119; 200 | KE = 120; 201 | KG = 121; 202 | KH = 122; 203 | KI = 123; 204 | KM = 124; 205 | KN = 125; 206 | KP = 126; 207 | KR = 127; 208 | KW = 128; 209 | KY = 129; 210 | KZ = 130; 211 | LA = 131; 212 | LB = 132; 213 | LC = 133; 214 | LI = 134; 215 | LK = 135; 216 | LR = 136; 217 | LS = 137; 218 | LT = 138; 219 | LU = 139; 220 | LV = 140; 221 | LY = 141; 222 | MA = 142; 223 | MC = 143; 224 | MD = 144; 225 | ME = 145; 226 | MF = 146; 227 | MG = 147; 228 | MH = 148; 229 | MK = 149; 230 | ML = 150; 231 | MM = 151; 232 | MN = 152; 233 | MO = 153; 234 | MP = 154; 235 | MQ = 155; 236 | MR = 156; 237 | MS = 157; 238 | MT = 158; 239 | MU = 159; 240 | MV = 160; 241 | MW = 161; 242 | MX = 162; 243 | MY = 163; 244 | MZ = 164; 245 | NA = 165; 246 | NC = 166; 247 | NE = 167; 248 | NF = 168; 249 | NG = 169; 250 | NI = 170; 251 | NL = 171; 252 | NO = 172; 253 | NP = 173; 254 | NR = 174; 255 | NU = 175; 256 | NZ = 176; 257 | OM = 177; 258 | PA = 178; 259 | PE = 179; 260 | PF = 180; 261 | PG = 181; 262 | PH = 182; 263 | PK = 183; 264 | PL = 184; 265 | PM = 185; 266 | PN = 186; 267 | PR = 187; 268 | PS = 188; 269 | PT = 189; 270 | PW = 190; 271 | PY = 191; 272 | QA = 192; 273 | RE = 193; 274 | RO = 194; 275 | RS = 195; 276 | RU = 196; 277 | RW = 197; 278 | SA = 198; 279 | SB = 199; 280 | SC = 200; 281 | SD = 201; 282 | SE = 202; 283 | SG = 203; 284 | SH = 204; 285 | SI = 205; 286 | SJ = 206; 287 | SK = 207; 288 | SL = 208; 289 | SM = 209; 290 | SN = 210; 291 | SO = 211; 292 | SR = 212; 293 | SS = 213; 294 | ST = 214; 295 | SV = 215; 296 | SX = 216; 297 | SY = 217; 298 | SZ = 218; 299 | TC = 219; 300 | TD = 220; 301 | TF = 221; 302 | TG = 222; 303 | TH = 223; 304 | TJ = 224; 305 | TK = 225; 306 | TL = 226; 307 | TM = 227; 308 | TN = 228; 309 | TO = 229; 310 | TR = 230; 311 | TT = 231; 312 | TV = 232; 313 | TW = 233; 314 | TZ = 234; 315 | UA = 235; 316 | UG = 236; 317 | UM = 237; 318 | US = 238; 319 | UY = 239; 320 | UZ = 240; 321 | VA = 241; 322 | VC = 242; 323 | VE = 243; 324 | VG = 244; 325 | VI = 245; 326 | VN = 246; 327 | VU = 247; 328 | WF = 248; 329 | WS = 249; 330 | XX = 250; 331 | YE = 251; 332 | YT = 252; 333 | ZA = 253; 334 | ZM = 254; 335 | ZW = 255; 336 | } 337 | --------------------------------------------------------------------------------