├── .gitignore ├── README.md ├── config └── config.exs ├── cpp ├── .gitignore ├── AsyncAck.cc ├── HeadRest.cc ├── Makefile ├── RequestReply.cc ├── SyncAck.cc └── ThrottleCpp.cc ├── lib ├── async_ack_handler.ex ├── async_ack_worker.ex ├── head_rest_container.ex ├── head_rest_handler.ex ├── head_rest_worker.ex ├── request_reply_handler.ex ├── request_reply_worker.ex ├── sync_ack_handler.ex ├── sync_ack_worker.ex ├── tcp_ex_playground.ex ├── throttle_ack_container.ex ├── throttle_ack_handler.ex └── throttle_ack_worker.ex ├── mix.exs ├── mix.lock └── test ├── tcp_ex_playground_test.exs └── test_helper.exs /.gitignore: -------------------------------------------------------------------------------- 1 | /_build 2 | /deps 3 | erl_crash.dump 4 | *.ez 5 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | TcpExPlayground 2 | =============== 3 | 4 | This project holds experiences in the Elixir TCP world. They all have a few corresponding blog entries and also a git branch. The git branches are then merged into master too: 5 | 6 | - [01_request_reply branch](https://github.com/dbeck/tcp_ex_playground/tree/01_request_reply) is documented [here](http://dbeck.github.io/simple-TCP-message-performance-in-Elixir/). This is a very basic TCP Request/Reply client and server example. The C++ client is [here](https://github.com/dbeck/tcp_ex_playground/blob/master/cpp/RequestReply.cc). The Elixir server is [here](https://github.com/dbeck/tcp_ex_playground/blob/master/lib/request_reply_handler.ex). The result is **22k msgs/sec** on my laptop. 7 | 8 | - [02_throttled_acks branch](https://github.com/dbeck/tcp_ex_playground/tree/02_throttled_acks) is documented [here](http://dbeck.github.io/Four-Times-Speedup-By-Throttling/). This is an attempt to improve performance by changing the messaging pattern that allows combining and delaying ACK messages. The C++ client is [here](https://github.com/dbeck/tcp_ex_playground/blob/master/cpp/ThrottleCpp.cc). The Elixir server is [here](https://github.com/dbeck/tcp_ex_playground/blob/master/lib/throttle_ack_handler.ex). The result is **90k msgs/sec** on my laptop. 9 | 10 | - [03_head_rest_pattern branch](https://github.com/dbeck/tcp_ex_playground/tree/03_head_rest_pattern) is documented [here](http://dbeck.github.io/Over-Two-Times-Speedup-By-Better-Elixir-Code/). By better using the Elixir pattern matching on binariy messages I had 2x performance gain. The C++ client is [here](https://github.com/dbeck/tcp_ex_playground/blob/master/cpp/HeadRest.cc). The Elixir server is [here](https://github.com/dbeck/tcp_ex_playground/blob/master/lib/head_rest_handler.ex). The result is **250k msgs/sec** on my laptop. 11 | 12 | - [04_synch_ack branch](https://github.com/dbeck/tcp_ex_playground/tree/04_synch_ack) is documented [here](http://dbeck.github.io/Passing-Millions-Of-Small-TCP-Messages-in-Elixir/). By removing a bottleneck from the code I achieved significant performance gain. The C++ client is [here](https://github.com/dbeck/tcp_ex_playground/blob/master/cpp/SyncAck.cc). The Elixir server is [here](https://github.com/dbeck/tcp_ex_playground/blob/master/lib/sync_ack_handler.ex). The result is over **2M msgs/sec** on my laptop. 13 | 14 | - [05_asynch_ack branch](https://github.com/dbeck/tcp_ex_playground/tree/05_asynch_ack) is documented [here](http://dbeck.github.io/Wrapping-up-my-Elixir-TCP-experiments/). In this latest experiment I retried moving the ACK sending to and independent process. With that I achieved **3M request per sec** on my MacBook Air laptop. When I tried it on an old Linux machine and an EC2 Amazon instance, this last code was slower than the previous **04_synch_ack** experiment. Looks like at this performance range one needs to consider the actual hardware too where the code is going to run. Memory and CPU speed both impacts heavily the speed of passing data between Elixir processes. The client code is available [here](https://github.com/dbeck/tcp_ex_playground/blob/master/cpp/AsyncAck.cc). The Elixir server is [here](https://github.com/dbeck/tcp_ex_playground/blob/master/lib/async_ack_handler.ex). 15 | 16 | License 17 | ======= 18 | 19 | Copyright (c) 2015 [David Beck](http://dbeck.github.io) 20 | 21 | MIT License 22 | 23 | Permission is hereby granted, free of charge, to any person obtaining 24 | a copy of this software and associated documentation files (the 25 | "Software"), to deal in the Software without restriction, including 26 | without limitation the rights to use, copy, modify, merge, publish, 27 | distribute, sublicense, and/or sell copies of the Software, and to 28 | permit persons to whom the Software is furnished to do so, subject to 29 | the following conditions: 30 | 31 | The above copyright notice and this permission notice shall be 32 | included in all copies or substantial portions of the Software. 33 | 34 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 35 | EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 36 | MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 37 | NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE 38 | LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION 39 | OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION 40 | WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. -------------------------------------------------------------------------------- /config/config.exs: -------------------------------------------------------------------------------- 1 | # This file is responsible for configuring your application 2 | # and its dependencies with the aid of the Mix.Config module. 3 | use Mix.Config 4 | 5 | # This configuration is loaded before any dependency and is restricted 6 | # to this project. If another project depends on this project, this 7 | # file won't be loaded nor affect the parent project. For this reason, 8 | # if you want to provide default values for your application for third- 9 | # party users, it should be done in your mix.exs file. 10 | 11 | # Sample configuration: 12 | # 13 | # config :logger, :console, 14 | # level: :info, 15 | # format: "$date $time [$level] $metadata$message\n", 16 | # metadata: [:user_id] 17 | 18 | # It is also possible to import configuration files, relative to this 19 | # directory. For example, you can emulate configuration per environment 20 | # by uncommenting the line below and defining dev.exs, test.exs and such. 21 | # Configuration from the imported file will override the ones defined 22 | # here (which is why it is important to import them last). 23 | # 24 | # import_config "#{Mix.env}.exs" 25 | -------------------------------------------------------------------------------- /cpp/.gitignore: -------------------------------------------------------------------------------- 1 | RequestReply 2 | ThrottleCpp 3 | HeadRest 4 | SyncAck 5 | AsyncAck 6 | -------------------------------------------------------------------------------- /cpp/AsyncAck.cc: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | #include 9 | #include 10 | #include 11 | 12 | #include 13 | #include 14 | #include 15 | #include 16 | #include 17 | #include 18 | 19 | namespace 20 | { 21 | // 22 | // to help freeing C resources 23 | // 24 | struct on_destruct 25 | { 26 | std::function fun_; 27 | on_destruct(std::function fun) : fun_(fun) {} 28 | ~on_destruct() { if( fun_ ) fun_(); } 29 | }; 30 | 31 | // 32 | // for measuring ellapsed time and print statistics 33 | // 34 | struct timer 35 | { 36 | typedef std::chrono::high_resolution_clock highres_clock; 37 | typedef std::chrono::time_point timepoint; 38 | 39 | timepoint start_; 40 | uint64_t iteration_; 41 | 42 | timer(uint64_t iter) : start_{highres_clock::now()}, iteration_{iter} {} 43 | 44 | int64_t spent_usec() 45 | { 46 | using namespace std::chrono; 47 | timepoint now{highres_clock::now()}; 48 | return duration_cast(now-start_).count(); 49 | } 50 | 51 | ~timer() 52 | { 53 | using namespace std::chrono; 54 | timepoint now{highres_clock::now()}; 55 | 56 | uint64_t usec_diff = duration_cast(now-start_).count(); 57 | double call_per_ms = iteration_*1000.0 / ((double)usec_diff); 58 | double call_per_sec = iteration_*1000000.0 / ((double)usec_diff); 59 | double us_per_call = (double)usec_diff / (double)iteration_; 60 | 61 | std::cout << "elapsed usec=" << usec_diff 62 | << " avg(usec/call)=" << std::setprecision(8) << us_per_call 63 | << " avg(call/msec)=" << std::setprecision(8) << call_per_ms 64 | << " avg(call/sec)=" << std::setprecision(8) << call_per_sec 65 | << std::endl; 66 | } 67 | }; 68 | 69 | template 70 | struct buffer 71 | { 72 | // each packet has 3 parts: 73 | // - 64 bit ID 74 | // - 32 bit size 75 | // - data 76 | struct iovec items_[MAX_ITEMS*3]; 77 | uint64_t ids_[MAX_ITEMS]; 78 | size_t n_items_; 79 | uint32_t len_; 80 | char data_[5]; 81 | 82 | buffer() : n_items_{0}, len_{5} 83 | { 84 | memcpy(data_, "hello", 5); 85 | 86 | for( size_t i=0; i= MAX_ITEMS); 113 | } 114 | 115 | void flush(int sockfd) 116 | { 117 | if( !n_items_ ) return; 118 | if( writev(sockfd, items_, (n_items_*3)) != (17*n_items_) ) 119 | { 120 | throw "failed to send data"; 121 | } 122 | n_items_ = 0; 123 | } 124 | }; 125 | } 126 | 127 | 128 | int main(int argc, char ** argv) 129 | { 130 | try 131 | { 132 | // create a TCP socket 133 | int sockfd = socket(AF_INET, SOCK_STREAM, 0); 134 | if( sockfd < 0 ) 135 | { 136 | throw "can't create socket"; 137 | } 138 | on_destruct close_sockfd( [sockfd](){ close(sockfd); } ); 139 | 140 | // server address (127.0.0.1:8005) 141 | struct sockaddr_in server_addr; 142 | ::memset(&server_addr, 0, sizeof(server_addr)); 143 | 144 | server_addr.sin_family = AF_INET; 145 | server_addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK); 146 | server_addr.sin_port = htons(8005); 147 | 148 | // connect to server 149 | if( connect(sockfd, (struct sockaddr *)&server_addr, sizeof(struct sockaddr)) == -1 ) 150 | { 151 | throw "failed to connect to server at 127.0.0.1:8005"; 152 | } 153 | 154 | { 155 | /* This hurts performance 156 | int flag = 1; 157 | if( setsockopt( sockfd, IPPROTO_TCP, TCP_NODELAY, (void *)&flag, sizeof(flag)) == -1 ) 158 | { 159 | throw "failed to set TCP_NODELAY on the socket"; 160 | } 161 | */ 162 | } 163 | 164 | // the buffer template parameter tells how many messages shall 165 | // we batch together 166 | buffer<150> data; 167 | uint64_t id = 0; 168 | int64_t last_ack = -1; 169 | 170 | // 171 | // this lambda function checks if we have received a new ACK. 172 | // if we did then it checks the content and returns the max 173 | // acknowledged ID. this supports receiving multiple ACKs in 174 | // a single transfer. 175 | // 176 | auto check_ack = [sockfd](int64_t last_ack) { 177 | int64_t ret_ack = last_ack; 178 | fd_set fdset; 179 | FD_ZERO(&fdset); 180 | FD_SET(sockfd, &fdset); 181 | 182 | // give 1 msec to the acks to arrive 183 | struct timeval tv { 0, 1000 }; 184 | int select_ret = select( sockfd+1, &fdset, NULL, NULL, &tv ); 185 | if( select_ret < 0) 186 | { 187 | throw "failed to select, socket error?"; 188 | } 189 | else if( select_ret == 0 ) 190 | { 191 | // timed out 192 | } 193 | else if( select_ret > 0 && FD_ISSET(sockfd,&fdset) ) 194 | { 195 | // max 2048 acks that we handle in one check 196 | size_t alloc_bytes = 12 * 2048; 197 | std::unique_ptr ack_data{new uint8_t[alloc_bytes]}; 198 | 199 | // 200 | // let's receive what has arrived. if there are more than 2048 201 | // ACKs waiting, then the next loop will take care of them 202 | // 203 | 204 | auto recv_ret = recv(sockfd, ack_data.get(), alloc_bytes, 0); 205 | if( recv_ret < 0 ) 206 | { 207 | throw "failed to recv, socket error?"; 208 | } 209 | else if( recv_ret < 12 ) 210 | { 211 | throw "failed to recv, partial packet?"; 212 | } 213 | else if( recv_ret > 11 ) 214 | { 215 | for( size_t pos=0; pos (checked_at_usec+3000) ) 265 | { 266 | last_ack = check_ack(last_ack); 267 | checked_at_usec = spent_usec; 268 | } 269 | } 270 | ++id; 271 | } 272 | 273 | // flush all unflushed items 274 | data.flush(sockfd); 275 | 276 | // wait for all outstanding ACKs 277 | while( last_ack < (id-1) ) 278 | { 279 | last_ack = check_ack(last_ack); 280 | // time out in 20 seconds 281 | if( t.spent_usec() > 20000000 ) 282 | { 283 | std::cerr << "last_ack=" << last_ack << " id=" << id-1 << " missing=" << (id-1)-last_ack << std::endl; 284 | throw "timed out while waiting for ACK"; 285 | } 286 | } 287 | } 288 | 289 | while( last_ack < (id-1) ) 290 | { 291 | last_ack = check_ack(last_ack); 292 | if( last_ack != id ) 293 | { 294 | std::cerr << "last_ack=" << last_ack << " id=" << id << "\n"; 295 | std::this_thread::sleep_for(std::chrono::milliseconds(1000)); 296 | } 297 | } 298 | } 299 | catch( const char * msg ) 300 | { 301 | perror(msg); 302 | } 303 | return 0; 304 | } 305 | -------------------------------------------------------------------------------- /cpp/HeadRest.cc: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | #include 9 | #include 10 | #include 11 | 12 | #include 13 | #include 14 | #include 15 | #include 16 | #include 17 | 18 | namespace 19 | { 20 | // 21 | // to help freeing C resources 22 | // 23 | struct on_destruct 24 | { 25 | std::function fun_; 26 | on_destruct(std::function fun) : fun_(fun) {} 27 | ~on_destruct() { if( fun_ ) fun_(); } 28 | }; 29 | 30 | // 31 | // for measuring ellapsed time and print statistics 32 | // 33 | struct timer 34 | { 35 | typedef std::chrono::high_resolution_clock highres_clock; 36 | typedef std::chrono::time_point timepoint; 37 | 38 | timepoint start_; 39 | uint64_t iteration_; 40 | 41 | timer(uint64_t iter) : start_{highres_clock::now()}, iteration_{iter} {} 42 | 43 | int64_t spent_usec() 44 | { 45 | using namespace std::chrono; 46 | timepoint now{highres_clock::now()}; 47 | return duration_cast(now-start_).count(); 48 | } 49 | 50 | ~timer() 51 | { 52 | using namespace std::chrono; 53 | timepoint now{highres_clock::now()}; 54 | 55 | uint64_t usec_diff = duration_cast(now-start_).count(); 56 | double call_per_ms = iteration_*1000.0 / ((double)usec_diff); 57 | double call_per_sec = iteration_*1000000.0 / ((double)usec_diff); 58 | double us_per_call = (double)usec_diff / (double)iteration_; 59 | 60 | std::cout << "elapsed usec=" << usec_diff 61 | << " avg(usec/call)=" << us_per_call 62 | << " avg(call/msec)=" << call_per_ms 63 | << " avg(call/sec)=" << call_per_sec 64 | << std::endl; 65 | } 66 | }; 67 | 68 | template 69 | struct buffer 70 | { 71 | // each packet has 3 parts: 72 | // - 64 bit ID 73 | // - 32 bit size 74 | // - data 75 | struct iovec items_[MAX_ITEMS*3]; 76 | uint64_t ids_[MAX_ITEMS]; 77 | size_t n_items_; 78 | uint32_t len_; 79 | char data_[5]; 80 | 81 | buffer() : n_items_{0}, len_{5} 82 | { 83 | memcpy(data_, "hello", 5); 84 | 85 | for( size_t i=0; i= MAX_ITEMS); 112 | } 113 | 114 | void flush(int sockfd) 115 | { 116 | if( !n_items_ ) return; 117 | if( writev(sockfd, items_, (n_items_*3)) != (17*n_items_) ) 118 | { 119 | throw "failed to send data"; 120 | } 121 | n_items_ = 0; 122 | } 123 | }; 124 | } 125 | 126 | 127 | int main(int argc, char ** argv) 128 | { 129 | try 130 | { 131 | // create a TCP socket 132 | int sockfd = socket(AF_INET, SOCK_STREAM, 0); 133 | if( sockfd < 0 ) 134 | { 135 | throw "can't create socket"; 136 | } 137 | on_destruct close_sockfd( [sockfd](){ close(sockfd); } ); 138 | 139 | // server address (127.0.0.1:8003) 140 | struct sockaddr_in server_addr; 141 | ::memset(&server_addr, 0, sizeof(server_addr)); 142 | 143 | server_addr.sin_family = AF_INET; 144 | server_addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK); 145 | server_addr.sin_port = htons(8003); 146 | 147 | // connect to server 148 | if( connect(sockfd, (struct sockaddr *)&server_addr, sizeof(struct sockaddr)) == -1 ) 149 | { 150 | throw "failed to connect to server at 127.0.0.1:8003"; 151 | } 152 | 153 | { 154 | int flag = 1; 155 | if( setsockopt( sockfd, IPPROTO_TCP, TCP_NODELAY, (void *)&flag, sizeof(flag)) == -1 ) 156 | { 157 | throw "failed to set TCP_NODELAY on the socket"; 158 | } 159 | } 160 | 161 | // the buffer template parameter tells how many messages shall 162 | // we batch together 163 | buffer<50> data; 164 | uint64_t id = 0; 165 | int64_t last_ack = -1; 166 | 167 | // 168 | // this lambda function checks if we have received a new ACK. 169 | // if we did then it checks the content and returns the max 170 | // acknowledged ID. this supports receiving multiple ACKs in 171 | // a single transfer. 172 | // 173 | auto check_ack = [sockfd](int64_t last_ack) { 174 | int64_t ret_ack = last_ack; 175 | fd_set fdset; 176 | FD_ZERO(&fdset); 177 | FD_SET(sockfd, &fdset); 178 | 179 | // give 10 msec to the acks to arrive 180 | struct timeval tv { 0, 10000 }; 181 | int select_ret = select( sockfd+1, &fdset, NULL, NULL, &tv ); 182 | if( select_ret < 0) 183 | { 184 | throw "failed to select, socket error?"; 185 | } 186 | else if( select_ret > 0 && FD_ISSET(sockfd,&fdset) ) 187 | { 188 | // max 2048 acks that we handle in one check 189 | size_t alloc_bytes = 12 * 2048; 190 | std::unique_ptr ack_data{new uint8_t[alloc_bytes]}; 191 | 192 | // 193 | // let's receive what has arrived. if there are more than 2048 194 | // ACKs waiting, then the next loop will take care of them 195 | // 196 | 197 | auto recv_ret = recv(sockfd, ack_data.get(), alloc_bytes, 0); 198 | if( recv_ret < 0 ) 199 | { 200 | throw "failed to recv, socket error?"; 201 | } 202 | if( recv_ret > 0 ) 203 | { 204 | for( size_t pos=0; pos (checked_at_usec+30000) ) 253 | { 254 | last_ack = check_ack(last_ack); 255 | checked_at_usec = spent_usec; 256 | } 257 | } 258 | ++id; 259 | } 260 | 261 | // flush all unflushed items 262 | data.flush(sockfd); 263 | 264 | // wait for all outstanding ACKs 265 | while( last_ack < (id-1) ) 266 | last_ack = check_ack(last_ack); 267 | } 268 | 269 | while( last_ack < (id-1) ) 270 | { 271 | last_ack = check_ack(last_ack); 272 | if( last_ack != id ) 273 | { 274 | std::cerr << "last_ack=" << last_ack << " id=" << id << "\n"; 275 | std::this_thread::sleep_for(std::chrono::milliseconds(1000)); 276 | } 277 | } 278 | } 279 | catch( const char * msg ) 280 | { 281 | perror(msg); 282 | } 283 | return 0; 284 | } 285 | -------------------------------------------------------------------------------- /cpp/Makefile: -------------------------------------------------------------------------------- 1 | 2 | all: RequestReply ThrottleCpp HeadRest SyncAck AsyncAck 3 | 4 | RequestReply: RequestReply.cc 5 | g++ -o RequestReply -O3 -std=c++11 -Wall RequestReply.cc 6 | 7 | ThrottleCpp: ThrottleCpp.cc 8 | g++ -o ThrottleCpp -O3 -std=c++11 -Wall ThrottleCpp.cc 9 | 10 | HeadRest: HeadRest.cc 11 | g++ -o HeadRest -O3 -std=c++11 -Wall HeadRest.cc 12 | 13 | SyncAck: SyncAck.cc 14 | g++ -o SyncAck -O3 -std=c++11 -Wall SyncAck.cc 15 | 16 | AsyncAck: AsyncAck.cc 17 | g++ -o AsyncAck -O3 -std=c++11 -Wall AsyncAck.cc 18 | 19 | clean: 20 | rm -f RequestReply ThrottleCpp HeadRest SyncAck AsyncAck 21 | -------------------------------------------------------------------------------- /cpp/RequestReply.cc: -------------------------------------------------------------------------------- 1 | // inet_addr 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | #include 9 | #include 10 | 11 | #include 12 | #include 13 | #include 14 | #include 15 | 16 | namespace 17 | { 18 | struct on_destruct 19 | { 20 | std::function fun_; 21 | on_destruct(std::function fun) : fun_(fun) {} 22 | ~on_destruct() { fun_(); } 23 | }; 24 | 25 | struct timer 26 | { 27 | typedef std::chrono::high_resolution_clock highres_clock; 28 | typedef std::chrono::time_point timepoint; 29 | 30 | timepoint start_; 31 | uint64_t iteration_; 32 | 33 | timer(uint64_t iter) : start_{highres_clock::now()}, iteration_{iter} {} 34 | 35 | ~timer() 36 | { 37 | using namespace std::chrono; 38 | timepoint now{highres_clock::now()}; 39 | 40 | uint64_t usec_diff = duration_cast(now-start_).count(); 41 | double call_per_ms = iteration_*1000.0 / ((double)usec_diff); 42 | double call_per_sec = iteration_*1000000.0 / ((double)usec_diff); 43 | double us_per_call = (double)usec_diff / (double)iteration_; 44 | 45 | std::cout << "elapsed usec=" << usec_diff 46 | << " avg(usec/call)=" << us_per_call 47 | << " avg(call/msec)=" << call_per_ms 48 | << " avg(call/sec)=" << call_per_sec 49 | << std::endl; 50 | } 51 | }; 52 | } 53 | 54 | 55 | int main(int argc, char ** argv) 56 | { 57 | try 58 | { 59 | // create a TCP socket 60 | int sockfd = socket(AF_INET, SOCK_STREAM, 0); 61 | if( sockfd < 0 ) 62 | { 63 | throw "can't create socket"; 64 | } 65 | on_destruct close_sockfd( [sockfd](){ close(sockfd); } ); 66 | 67 | // server address (127.0.0.1:8001) 68 | struct sockaddr_in server_addr; 69 | ::memset(&server_addr, 0, sizeof(server_addr)); 70 | 71 | server_addr.sin_family = AF_INET; 72 | server_addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK); 73 | server_addr.sin_port = htons(8001); 74 | 75 | // connect to server 76 | if( connect(sockfd, (struct sockaddr *)&server_addr, sizeof(struct sockaddr)) == -1 ) 77 | { 78 | throw "failed to connect to server at 127.0.0.1:8001"; 79 | } 80 | 81 | // prepare data 82 | char data[] = "Hello"; 83 | uint64_t id = 0; 84 | uint32_t len = htonl(5); 85 | 86 | struct iovec data_iov[3] = { 87 | { (char *)&id, 8 }, // id 88 | { (char *)&len, 4 }, // len 89 | { data, 5 } // data 90 | }; 91 | 92 | for( int i=0; i<100; ++i ) 93 | { 94 | timer t(10000); 95 | // send data in a loop 96 | for( id = 0; id<10000; ++id ) 97 | { 98 | if( writev(sockfd, data_iov, 3) != 17 ) throw "failed to send data"; 99 | uint64_t response = 0; 100 | if( recv(sockfd, &response, 8, 0) != 8 ) throw "failed to receive data"; 101 | if( response != id ) throw "invalid response received"; 102 | } 103 | } 104 | 105 | } 106 | catch( const char * msg ) 107 | { 108 | perror(msg); 109 | } 110 | return 0; 111 | } 112 | -------------------------------------------------------------------------------- /cpp/SyncAck.cc: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | #include 9 | #include 10 | #include 11 | 12 | #include 13 | #include 14 | #include 15 | #include 16 | #include 17 | #include 18 | 19 | namespace 20 | { 21 | // 22 | // to help freeing C resources 23 | // 24 | struct on_destruct 25 | { 26 | std::function fun_; 27 | on_destruct(std::function fun) : fun_(fun) {} 28 | ~on_destruct() { if( fun_ ) fun_(); } 29 | }; 30 | 31 | // 32 | // for measuring ellapsed time and print statistics 33 | // 34 | struct timer 35 | { 36 | typedef std::chrono::high_resolution_clock highres_clock; 37 | typedef std::chrono::time_point timepoint; 38 | 39 | timepoint start_; 40 | uint64_t iteration_; 41 | 42 | timer(uint64_t iter) : start_{highres_clock::now()}, iteration_{iter} {} 43 | 44 | int64_t spent_usec() 45 | { 46 | using namespace std::chrono; 47 | timepoint now{highres_clock::now()}; 48 | return duration_cast(now-start_).count(); 49 | } 50 | 51 | ~timer() 52 | { 53 | using namespace std::chrono; 54 | timepoint now{highres_clock::now()}; 55 | 56 | uint64_t usec_diff = duration_cast(now-start_).count(); 57 | double call_per_ms = iteration_*1000.0 / ((double)usec_diff); 58 | double call_per_sec = iteration_*1000000.0 / ((double)usec_diff); 59 | double us_per_call = (double)usec_diff / (double)iteration_; 60 | 61 | std::cout << "elapsed usec=" << usec_diff 62 | << " avg(usec/call)=" << std::setprecision(8) << us_per_call 63 | << " avg(call/msec)=" << std::setprecision(8) << call_per_ms 64 | << " avg(call/sec)=" << std::setprecision(8) << call_per_sec 65 | << std::endl; 66 | } 67 | }; 68 | 69 | template 70 | struct buffer 71 | { 72 | // each packet has 3 parts: 73 | // - 64 bit ID 74 | // - 32 bit size 75 | // - data 76 | struct iovec items_[MAX_ITEMS*3]; 77 | uint64_t ids_[MAX_ITEMS]; 78 | size_t n_items_; 79 | uint32_t len_; 80 | char data_[5]; 81 | 82 | buffer() : n_items_{0}, len_{5} 83 | { 84 | memcpy(data_, "hello", 5); 85 | 86 | for( size_t i=0; i= MAX_ITEMS); 113 | } 114 | 115 | void flush(int sockfd) 116 | { 117 | if( !n_items_ ) return; 118 | if( writev(sockfd, items_, (n_items_*3)) != (17*n_items_) ) 119 | { 120 | throw "failed to send data"; 121 | } 122 | n_items_ = 0; 123 | } 124 | }; 125 | } 126 | 127 | 128 | int main(int argc, char ** argv) 129 | { 130 | try 131 | { 132 | // create a TCP socket 133 | int sockfd = socket(AF_INET, SOCK_STREAM, 0); 134 | if( sockfd < 0 ) 135 | { 136 | throw "can't create socket"; 137 | } 138 | on_destruct close_sockfd( [sockfd](){ close(sockfd); } ); 139 | 140 | // server address (127.0.0.1:8004) 141 | struct sockaddr_in server_addr; 142 | ::memset(&server_addr, 0, sizeof(server_addr)); 143 | 144 | server_addr.sin_family = AF_INET; 145 | server_addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK); 146 | server_addr.sin_port = htons(8004); 147 | 148 | // connect to server 149 | if( connect(sockfd, (struct sockaddr *)&server_addr, sizeof(struct sockaddr)) == -1 ) 150 | { 151 | throw "failed to connect to server at 127.0.0.1:8004"; 152 | } 153 | 154 | { 155 | /* This hurts performance 156 | int flag = 1; 157 | if( setsockopt( sockfd, IPPROTO_TCP, TCP_NODELAY, (void *)&flag, sizeof(flag)) == -1 ) 158 | { 159 | throw "failed to set TCP_NODELAY on the socket"; 160 | } 161 | */ 162 | } 163 | 164 | // the buffer template parameter tells how many messages shall 165 | // we batch together 166 | buffer<50> data; 167 | uint64_t id = 0; 168 | int64_t last_ack = -1; 169 | 170 | // 171 | // this lambda function checks if we have received a new ACK. 172 | // if we did then it checks the content and returns the max 173 | // acknowledged ID. this supports receiving multiple ACKs in 174 | // a single transfer. 175 | // 176 | auto check_ack = [sockfd](int64_t last_ack) { 177 | int64_t ret_ack = last_ack; 178 | fd_set fdset; 179 | FD_ZERO(&fdset); 180 | FD_SET(sockfd, &fdset); 181 | 182 | // give 1 msec to the acks to arrive 183 | struct timeval tv { 0, 1000 }; 184 | int select_ret = select( sockfd+1, &fdset, NULL, NULL, &tv ); 185 | if( select_ret < 0) 186 | { 187 | throw "failed to select, socket error?"; 188 | } 189 | else if( select_ret > 0 && FD_ISSET(sockfd,&fdset) ) 190 | { 191 | // max 2048 acks that we handle in one check 192 | size_t alloc_bytes = 12 * 2048; 193 | std::unique_ptr ack_data{new uint8_t[alloc_bytes]}; 194 | 195 | // 196 | // let's receive what has arrived. if there are more than 2048 197 | // ACKs waiting, then the next loop will take care of them 198 | // 199 | 200 | auto recv_ret = recv(sockfd, ack_data.get(), alloc_bytes, 0); 201 | if( recv_ret < 0 ) 202 | { 203 | throw "failed to recv, socket error?"; 204 | } 205 | if( recv_ret > 0 ) 206 | { 207 | for( size_t pos=0; pos (checked_at_usec+30000) ) 256 | { 257 | last_ack = check_ack(last_ack); 258 | checked_at_usec = spent_usec; 259 | } 260 | } 261 | ++id; 262 | } 263 | 264 | // flush all unflushed items 265 | data.flush(sockfd); 266 | 267 | // wait for all outstanding ACKs 268 | while( last_ack < (id-1) ) 269 | last_ack = check_ack(last_ack); 270 | } 271 | 272 | while( last_ack < (id-1) ) 273 | { 274 | last_ack = check_ack(last_ack); 275 | if( last_ack != id ) 276 | { 277 | std::cerr << "last_ack=" << last_ack << " id=" << id << "\n"; 278 | std::this_thread::sleep_for(std::chrono::milliseconds(1000)); 279 | } 280 | } 281 | } 282 | catch( const char * msg ) 283 | { 284 | perror(msg); 285 | } 286 | return 0; 287 | } 288 | -------------------------------------------------------------------------------- /cpp/ThrottleCpp.cc: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | #include 9 | #include 10 | 11 | #include 12 | #include 13 | #include 14 | #include 15 | #include 16 | 17 | namespace 18 | { 19 | // 20 | // to help freeing C resources 21 | // 22 | struct on_destruct 23 | { 24 | std::function fun_; 25 | on_destruct(std::function fun) : fun_(fun) {} 26 | ~on_destruct() { fun_(); } 27 | }; 28 | 29 | // 30 | // for measuring ellapsed time and print statistics 31 | // 32 | struct timer 33 | { 34 | typedef std::chrono::high_resolution_clock highres_clock; 35 | typedef std::chrono::time_point timepoint; 36 | 37 | timepoint start_; 38 | uint64_t iteration_; 39 | 40 | timer(uint64_t iter) : start_{highres_clock::now()}, iteration_{iter} {} 41 | 42 | int64_t spent_usec() 43 | { 44 | using namespace std::chrono; 45 | timepoint now{highres_clock::now()}; 46 | return duration_cast(now-start_).count(); 47 | } 48 | 49 | ~timer() 50 | { 51 | using namespace std::chrono; 52 | timepoint now{highres_clock::now()}; 53 | 54 | uint64_t usec_diff = duration_cast(now-start_).count(); 55 | double call_per_ms = iteration_*1000.0 / ((double)usec_diff); 56 | double call_per_sec = iteration_*1000000.0 / ((double)usec_diff); 57 | double us_per_call = (double)usec_diff / (double)iteration_; 58 | 59 | std::cout << "elapsed usec=" << usec_diff 60 | << " avg(usec/call)=" << us_per_call 61 | << " avg(call/msec)=" << call_per_ms 62 | << " avg(call/sec)=" << call_per_sec 63 | << std::endl; 64 | } 65 | }; 66 | } 67 | 68 | 69 | int main(int argc, char ** argv) 70 | { 71 | try 72 | { 73 | // create a TCP socket 74 | int sockfd = socket(AF_INET, SOCK_STREAM, 0); 75 | if( sockfd < 0 ) 76 | { 77 | throw "can't create socket"; 78 | } 79 | on_destruct close_sockfd( [sockfd](){ close(sockfd); } ); 80 | 81 | // server address (127.0.0.1:8002) 82 | struct sockaddr_in server_addr; 83 | ::memset(&server_addr, 0, sizeof(server_addr)); 84 | 85 | server_addr.sin_family = AF_INET; 86 | server_addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK); 87 | server_addr.sin_port = htons(8002); 88 | 89 | // connect to server 90 | if( connect(sockfd, (struct sockaddr *)&server_addr, sizeof(struct sockaddr)) == -1 ) 91 | { 92 | throw "failed to connect to server at 127.0.0.1:8002"; 93 | } 94 | 95 | // prepare data 96 | char data[] = "Hello"; 97 | uint64_t id = 0; 98 | uint32_t len = 5; 99 | int64_t last_ack = -1; 100 | 101 | struct iovec data_iov[3] = { 102 | { (char *)&id, 8 }, // id 103 | { (char *)&len, 4 }, // len 104 | { data, 5 } // data 105 | }; 106 | 107 | 108 | // 109 | // this lambda function checks if we have received a new ACK. 110 | // if we did then it checks the content and returns the max 111 | // acknowledged ID. this supports receiving multiple ACKs in 112 | // a single transfer. 113 | // 114 | auto check_ack = [sockfd](int64_t last_ack) { 115 | int64_t ret_ack = last_ack; 116 | fd_set fdset; 117 | FD_ZERO(&fdset); 118 | FD_SET(sockfd, &fdset); 119 | 120 | // give 1 ms to the acks to arrive 121 | struct timeval tv { 0, 1000 }; 122 | int select_ret = select( sockfd+1, &fdset, NULL, NULL, &tv ); 123 | if( select_ret < 0) 124 | { 125 | throw "failed to select, socket error?"; 126 | } 127 | if( select_ret > 0 && FD_ISSET(sockfd,&fdset) ) 128 | { 129 | // max 2048 acks that we handle in one check 130 | size_t alloc_bytes = 12 * 2048; 131 | std::unique_ptr ack_data{new uint8_t[alloc_bytes]}; 132 | 133 | // 134 | // let's receive what has arrived. if there are more than 2048 135 | // ACKs waiting, then the next loop will take care of them 136 | // 137 | 138 | auto recv_ret = recv(sockfd, ack_data.get(), alloc_bytes, 0); 139 | if( recv_ret < 0 ) 140 | { 141 | throw "failed to recv, socket error?"; 142 | } 143 | if( recv_ret > 0 ) 144 | { 145 | for( size_t pos=0; pos (checked_at_usec+30000) ) 191 | { 192 | last_ack = check_ack(last_ack); 193 | checked_at_usec = spent_usec; 194 | } 195 | } 196 | ++id; 197 | } 198 | 199 | // wait for all outstanding ACKs 200 | while( last_ack < (id-1) ) 201 | last_ack = check_ack(last_ack); 202 | } 203 | 204 | while( last_ack < (id-1) ) 205 | { 206 | last_ack = check_ack(last_ack); 207 | if( last_ack != id ) 208 | { 209 | std::cerr << "last_ack=" << last_ack << " id=" << id << "\n"; 210 | std::this_thread::sleep_for(std::chrono::milliseconds(1000)); 211 | } 212 | } 213 | } 214 | catch( const char * msg ) 215 | { 216 | perror(msg); 217 | } 218 | return 0; 219 | } 220 | -------------------------------------------------------------------------------- /lib/async_ack_handler.ex: -------------------------------------------------------------------------------- 1 | defmodule AsyncAck.Handler do 2 | 3 | def start_link(ref, socket, transport, opts) do 4 | pid = spawn_link(__MODULE__, :init, [ref, socket, transport, opts]) 5 | {:ok, pid} 6 | end 7 | 8 | def init(ref, socket, transport, _Opts = []) do 9 | :ok = :ranch.accept_ack(ref) 10 | transport.setopts(socket, [nodelay: :true]) 11 | responder_pid = spawn_link(__MODULE__, :responder, [socket, transport, <<>>, [], 0]) 12 | Process.flag(:trap_exit, true) 13 | loop(socket, transport, responder_pid) 14 | end 15 | 16 | def calc_skipped([]) do 17 | 0 18 | end 19 | 20 | def calc_skipped([{_, skipped}]) do 21 | skipped 22 | end 23 | 24 | def calc_skipped([{_, skipped} | rest]) do 25 | 1 + skipped + calc_skipped(rest) 26 | end 27 | 28 | def flush(_, _, []) do 29 | end 30 | 31 | def flush(socket, transport, ack_list) do 32 | [{id, _ } | _ ] = ack_list 33 | skipped = calc_skipped(ack_list) 34 | packet = << id :: binary-size(8), skipped :: little-size(32) >> 35 | transport.send(socket, packet) 36 | end 37 | 38 | def responder(socket, transport, yet_to_parse, ack_list, packet_count) do 39 | receive do 40 | {:message, packet} -> 41 | case parse(yet_to_parse <> packet, << >>, 0) do 42 | {not_yet_parsed, {id, skipped} } -> 43 | new_ack_list = [{id, skipped} | ack_list] 44 | if packet_count > 20 do 45 | flush(socket, transport, new_ack_list) 46 | responder(socket, transport, not_yet_parsed, [], 0) 47 | else 48 | responder(socket, transport, not_yet_parsed, new_ack_list, packet_count+1) 49 | end 50 | {not_yet_parsed, {} } -> 51 | responder(socket, transport, not_yet_parsed, ack_list, packet_count+1) 52 | end 53 | {:stop} -> :stop 54 | after 55 | 5 -> 56 | flush(socket, transport, ack_list) 57 | responder(socket, transport, yet_to_parse, [], 0) 58 | end 59 | end 60 | 61 | def loop(socket, transport, responder_pid) do 62 | case transport.recv(socket, 0, 5000) do 63 | {:ok, packet} -> 64 | {:message_queue_len, length} = :erlang.process_info(responder_pid, :message_queue_len) 65 | if( length > 100 ) 66 | do 67 | :timer.sleep(div(length, 100)) 68 | end 69 | send responder_pid, {:message, packet} 70 | loop(socket, transport, responder_pid) 71 | {:error, :timeout} -> 72 | shutdown(socket, transport, responder_pid) 73 | _ -> 74 | shutdown(socket, transport, responder_pid) 75 | end 76 | end 77 | 78 | defp shutdown(socket, transport, responder_pid) do 79 | send responder_pid, {:stop} 80 | receive do 81 | {:EXIT, responder_pid, :normal} -> :ok 82 | end 83 | :ok = transport.close(socket) 84 | end 85 | 86 | defp parse(<< >>, << >>, _skipped ) do 87 | { << >>, {} } 88 | end 89 | 90 | defp parse(<< >>, last_id, skipped ) do 91 | { << >>, { last_id, skipped } } 92 | end 93 | 94 | defp parse(packet, << >>, 0) do 95 | case packet do 96 | # TODO : revise this 1MB safeguard against garbage here 97 | << id :: binary-size(8), sz :: little-size(32) , _data :: binary-size(sz) >> when sz < 1_000_000 -> 98 | { << >>, { id, 0 } } 99 | << id :: binary-size(8), sz :: little-size(32) , _data :: binary-size(sz) , rest :: binary >> when sz < 100 -> 100 | parse(rest, id, 0) 101 | unparsed -> 102 | { unparsed, {} } 103 | end 104 | end 105 | 106 | defp parse(packet, last_id, skipped) do 107 | case packet do 108 | # TODO : revise this 1MB safeguard against garbage here 109 | << id :: binary-size(8), sz :: little-size(32) , _data :: binary-size(sz) >> when sz < 1_000_000 -> 110 | { << >>, { id, skipped+1 } } 111 | << id :: binary-size(8), sz :: little-size(32) , _data :: binary-size(sz) , rest :: binary >> when sz < 100 -> 112 | parse(rest, id, skipped+1) 113 | unparsed -> 114 | { unparsed, {last_id, skipped} } 115 | end 116 | end 117 | end 118 | -------------------------------------------------------------------------------- /lib/async_ack_worker.ex: -------------------------------------------------------------------------------- 1 | defmodule AsyncAck.Worker do 2 | def start_link do 3 | opts = [port: 8005] 4 | {:ok, _} = :ranch.start_listener(:AsyncAck, 10, :ranch_tcp, opts, AsyncAck.Handler, []) 5 | end 6 | end 7 | -------------------------------------------------------------------------------- /lib/head_rest_container.ex: -------------------------------------------------------------------------------- 1 | defmodule HeadRest.Container do 2 | 3 | def start_link do 4 | Agent.start_link(fn -> [] end) 5 | end 6 | 7 | def stop(container) do 8 | Agent.stop(container) 9 | end 10 | 11 | def flush(container) do 12 | Agent.get_and_update(container, fn list -> {list, []} end) 13 | end 14 | 15 | def push(container, id, data) do 16 | Agent.update(container, fn list -> [{id, data}| list] end) 17 | end 18 | 19 | defp generate([]) do 20 | {} 21 | end 22 | 23 | defp generate( [{id, _}] ) do 24 | {id, 0} 25 | end 26 | 27 | defp generate( [{id, _} | tail] ) do 28 | tail_len = List.foldl(tail, 0, fn (_, acc) -> 1 + acc end) 29 | {id, tail_len} 30 | end 31 | 32 | def generate_ack(list) do 33 | generate(list) 34 | end 35 | end -------------------------------------------------------------------------------- /lib/head_rest_handler.ex: -------------------------------------------------------------------------------- 1 | defmodule HeadRest.Handler do 2 | 3 | def start_link(ref, socket, transport, opts) do 4 | pid = spawn_link(__MODULE__, :init, [ref, socket, transport, opts]) 5 | {:ok, pid} 6 | end 7 | 8 | def init(ref, socket, transport, _Opts = []) do 9 | :ok = :ranch.accept_ack(ref) 10 | {:ok, container} = HeadRest.Container.start_link 11 | timer_pid = spawn_link(__MODULE__, :timer, [socket, transport, container]) 12 | transport.setopts(socket, [nodelay: :true]) 13 | loop(socket, transport, container, timer_pid, << >>) 14 | end 15 | 16 | def flush(socket, transport, container) do 17 | list = HeadRest.Container.flush(container) 18 | case HeadRest.Container.generate_ack(list) do 19 | {id, skipped} -> 20 | packet = << id :: binary-size(8), skipped :: little-size(32) >> 21 | transport.send(socket, packet) 22 | {} -> 23 | :ok 24 | end 25 | end 26 | 27 | def timer(socket, transport, container) do 28 | flush(socket, transport, container) 29 | receive do 30 | {:stop} -> :stop 31 | after 32 | 5 -> timer(socket, transport, container) 33 | end 34 | end 35 | 36 | def loop(socket, transport, container, timer_pid, yet_to_parse) do 37 | case transport.recv(socket, 0, 5000) do 38 | {:ok, packet} -> 39 | not_yet_parsed = process(container, yet_to_parse <> packet) 40 | loop(socket, transport, container, timer_pid, not_yet_parsed) 41 | {:error, :timeout} -> 42 | flush(socket, transport, container) 43 | shutdown(socket, transport, container, timer_pid) 44 | _ -> 45 | shutdown(socket, transport, container, timer_pid) 46 | end 47 | end 48 | 49 | defp shutdown(socket, transport, container, timer_pid) do 50 | HeadRest.Container.stop(container) 51 | :ok = transport.close(socket) 52 | send timer_pid, {:stop} 53 | end 54 | 55 | defp process(_container, << >> ) do 56 | << >> 57 | end 58 | 59 | defp process(container, packet) do 60 | case packet do 61 | << id :: binary-size(8), sz :: little-size(32) , data :: binary-size(sz) >> -> 62 | HeadRest.Container.push(container, id, data) 63 | << >> 64 | << id :: binary-size(8), sz :: little-size(32) , data :: binary-size(sz) , rest :: binary >> -> 65 | HeadRest.Container.push(container, id, data) 66 | process(container, rest) 67 | unparsed -> 68 | unparsed 69 | end 70 | end 71 | end -------------------------------------------------------------------------------- /lib/head_rest_worker.ex: -------------------------------------------------------------------------------- 1 | defmodule HeadRest.Worker do 2 | def start_link do 3 | opts = [port: 8003] 4 | {:ok, _} = :ranch.start_listener(:HeadRest, 10, :ranch_tcp, opts, HeadRest.Handler, []) 5 | end 6 | end 7 | -------------------------------------------------------------------------------- /lib/request_reply_handler.ex: -------------------------------------------------------------------------------- 1 | defmodule RequestReply.Handler do 2 | 3 | def start_link(ref, socket, transport, opts) do 4 | pid = spawn_link(__MODULE__, :init, [ref, socket, transport, opts]) 5 | {:ok, pid} 6 | end 7 | 8 | def init(ref, socket, transport, _Opts = []) do 9 | :ok = :ranch.accept_ack(ref) 10 | loop(socket, transport) 11 | end 12 | 13 | def loop(socket, transport) do 14 | case transport.recv(socket, 0, 5000) do 15 | {:ok, << id :: binary-size(8), sz :: size(32), data :: binary-size(sz) >> } -> 16 | transport.send(socket, id) 17 | loop(socket, transport) 18 | _ -> 19 | :ok = transport.close(socket) 20 | end 21 | end 22 | end 23 | -------------------------------------------------------------------------------- /lib/request_reply_worker.ex: -------------------------------------------------------------------------------- 1 | defmodule RequestReply.Worker do 2 | def start_link do 3 | opts = [port: 8001] 4 | {:ok, _} = :ranch.start_listener(:RequestReply, 10, :ranch_tcp, opts, RequestReply.Handler, []) 5 | end 6 | end 7 | -------------------------------------------------------------------------------- /lib/sync_ack_handler.ex: -------------------------------------------------------------------------------- 1 | defmodule SyncAck.Handler do 2 | 3 | def start_link(ref, socket, transport, opts) do 4 | pid = spawn_link(__MODULE__, :init, [ref, socket, transport, opts]) 5 | {:ok, pid} 6 | end 7 | 8 | def init(ref, socket, transport, _Opts = []) do 9 | :ok = :ranch.accept_ack(ref) 10 | transport.setopts(socket, [nodelay: :true]) 11 | loop(socket, transport, << >>) 12 | end 13 | 14 | def loop(socket, transport, yet_to_parse) do 15 | case transport.recv(socket, 0, 5000) do 16 | {:ok, packet} -> 17 | case process(yet_to_parse <> packet, << >>, 0) do 18 | {not_yet_parsed, {id, skipped} } -> 19 | packet = << id :: binary-size(8), skipped :: little-size(32) >> 20 | transport.send(socket, packet) 21 | loop(socket, transport, not_yet_parsed) 22 | {not_yet_parsed, {} } -> 23 | loop(socket, transport, not_yet_parsed) 24 | end 25 | {:error, :timeout} -> 26 | shutdown(socket, transport) 27 | _ -> 28 | shutdown(socket, transport) 29 | end 30 | end 31 | 32 | defp shutdown(socket, transport) do 33 | :ok = transport.close(socket) 34 | end 35 | 36 | defp process(<< >>, << >>, _skipped ) do 37 | { << >>, {} } 38 | end 39 | 40 | defp process(<< >>, last_id, skipped ) do 41 | { << >>, { last_id, skipped } } 42 | end 43 | 44 | defp process(packet, << >>, 0) do 45 | case packet do 46 | << id :: binary-size(8), sz :: little-size(32) , _data :: binary-size(sz) >> -> 47 | { << >>, { id, 0 } } 48 | << id :: binary-size(8), sz :: little-size(32) , _data :: binary-size(sz) , rest :: binary >> -> 49 | process(rest, id, 0) 50 | unparsed -> 51 | { unparsed, {} } 52 | end 53 | end 54 | 55 | defp process(packet, last_id, skipped) do 56 | case packet do 57 | << id :: binary-size(8), sz :: little-size(32) , _data :: binary-size(sz) >> -> 58 | { << >>, { id, skipped+1 } } 59 | << id :: binary-size(8), sz :: little-size(32) , _data :: binary-size(sz) , rest :: binary >> -> 60 | process(rest, id, skipped+1) 61 | unparsed -> 62 | { unparsed, {last_id, skipped} } 63 | end 64 | end 65 | end -------------------------------------------------------------------------------- /lib/sync_ack_worker.ex: -------------------------------------------------------------------------------- 1 | defmodule SyncAck.Worker do 2 | def start_link do 3 | opts = [port: 8004] 4 | {:ok, _} = :ranch.start_listener(:SyncAck, 10, :ranch_tcp, opts, SyncAck.Handler, []) 5 | end 6 | end 7 | -------------------------------------------------------------------------------- /lib/tcp_ex_playground.ex: -------------------------------------------------------------------------------- 1 | defmodule TcpExPlayground do 2 | use Application 3 | 4 | def start(_type, _args) do 5 | import Supervisor.Spec, warn: false 6 | 7 | children = [ 8 | worker(RequestReply.Worker, []), 9 | worker(ThrottleAck.Worker, []), 10 | worker(HeadRest.Worker, []), 11 | worker(SyncAck.Worker, []), 12 | worker(AsyncAck.Worker, []) 13 | ] 14 | 15 | opts = [strategy: :one_for_one, name: TcpExPlayground.Supervisor] 16 | Supervisor.start_link(children, opts) 17 | end 18 | end 19 | -------------------------------------------------------------------------------- /lib/throttle_ack_container.ex: -------------------------------------------------------------------------------- 1 | defmodule ThrottleAck.Container do 2 | 3 | def start_link do 4 | Agent.start_link(fn -> [] end) 5 | end 6 | 7 | def stop(container) do 8 | Agent.stop(container) 9 | end 10 | 11 | def flush(container) do 12 | Agent.get_and_update(container, fn list -> {list, []} end) 13 | end 14 | 15 | def push(container, id, data) do 16 | Agent.update(container, fn list -> [{id, data}| list] end) 17 | end 18 | 19 | defp generate([]) do 20 | {} 21 | end 22 | 23 | defp generate( [{id, _}] ) do 24 | {id, 0} 25 | end 26 | 27 | defp generate( [{id, _} | tail] ) do 28 | tail_len = List.foldl(tail, 0, fn (_, acc) -> 1 + acc end) 29 | {id, tail_len} 30 | end 31 | 32 | def generate_ack(list) do 33 | generate(list) 34 | end 35 | end -------------------------------------------------------------------------------- /lib/throttle_ack_handler.ex: -------------------------------------------------------------------------------- 1 | defmodule ThrottleAck.Handler do 2 | 3 | def start_link(ref, socket, transport, opts) do 4 | pid = spawn_link(__MODULE__, :init, [ref, socket, transport, opts]) 5 | {:ok, pid} 6 | end 7 | 8 | def init(ref, socket, transport, _Opts = []) do 9 | :ok = :ranch.accept_ack(ref) 10 | {:ok, container} = ThrottleAck.Container.start_link 11 | timer_pid = spawn_link(__MODULE__, :timer, [socket, transport, container]) 12 | loop(socket, transport, container, timer_pid) 13 | end 14 | 15 | def flush(socket, transport, container) do 16 | list = ThrottleAck.Container.flush(container) 17 | case ThrottleAck.Container.generate_ack(list) do 18 | {id, skipped} -> 19 | packet = << id :: binary-size(8), skipped :: little-size(32) >> 20 | transport.send(socket, packet) 21 | {} -> 22 | IO.puts "empty data, everything flushed already" 23 | end 24 | end 25 | 26 | def timer(socket, transport, container) do 27 | flush(socket, transport, container) 28 | receive do 29 | {:stop} -> 30 | IO.puts "stop command arrived" 31 | :stop 32 | after 33 | 5 -> 34 | timer(socket, transport, container) 35 | end 36 | end 37 | 38 | def shutdown(socket, transport, container, timer_pid) do 39 | ThrottleAck.Container.stop(container) 40 | :ok = transport.close(socket) 41 | send timer_pid, {:stop} 42 | end 43 | 44 | def loop(socket, transport, container, timer_pid) do 45 | case transport.recv(socket, 12, 5000) do 46 | {:ok, id_sz_bin} -> 47 | << id :: binary-size(8), sz :: little-size(32) >> = id_sz_bin 48 | case transport.recv(socket, sz, 5000) do 49 | {:ok, data} -> 50 | ThrottleAck.Container.push(container, id, data) 51 | loop(socket, transport, container, timer_pid) 52 | {:error, :timeout} -> 53 | flush(socket, transport, container) 54 | shutdown(socket, transport, container, timer_pid) 55 | _ -> 56 | shutdown(socket, transport, container, timer_pid) 57 | end 58 | _ -> 59 | shutdown(socket, transport, container, timer_pid) 60 | end 61 | end 62 | end 63 | -------------------------------------------------------------------------------- /lib/throttle_ack_worker.ex: -------------------------------------------------------------------------------- 1 | defmodule ThrottleAck.Worker do 2 | def start_link do 3 | opts = [port: 8002] 4 | {:ok, _} = :ranch.start_listener(:ThrottleAck, 10, :ranch_tcp, opts, ThrottleAck.Handler, []) 5 | end 6 | end -------------------------------------------------------------------------------- /mix.exs: -------------------------------------------------------------------------------- 1 | defmodule TcpExPlayground.Mixfile do 2 | use Mix.Project 3 | 4 | def project do 5 | [app: :tcp_ex_playground, 6 | version: "0.0.1", 7 | elixir: "~> 1.0", 8 | build_embedded: Mix.env == :prod, 9 | start_permanent: Mix.env == :prod, 10 | deps: deps] 11 | end 12 | 13 | def application do 14 | [ applications: [:logger, :ranch], 15 | mod: {TcpExPlayground, []} ] 16 | end 17 | 18 | defp deps do 19 | [{:ranch, "~> 1.1"}] 20 | end 21 | end 22 | -------------------------------------------------------------------------------- /mix.lock: -------------------------------------------------------------------------------- 1 | %{"ranch": {:hex, :ranch, "1.1.0"}} 2 | -------------------------------------------------------------------------------- /test/tcp_ex_playground_test.exs: -------------------------------------------------------------------------------- 1 | defmodule TcpExPlaygroundTest do 2 | use ExUnit.Case 3 | 4 | test "the truth" do 5 | assert 1 + 1 == 2 6 | end 7 | end 8 | -------------------------------------------------------------------------------- /test/test_helper.exs: -------------------------------------------------------------------------------- 1 | ExUnit.start() 2 | --------------------------------------------------------------------------------