├── .gitattributes ├── .gitignore ├── NginxHttpChunkinModule.wiki ├── README.markdown ├── config ├── misc ├── Makefile ├── README └── chunked.rl ├── src ├── chunked_parser.c ├── chunked_parser.h ├── chunked_parser.rl ├── ddebug.h ├── ngx_http_chunkin_filter_module.c ├── ngx_http_chunkin_filter_module.h ├── ngx_http_chunkin_request_body.c ├── ngx_http_chunkin_request_body.h ├── ngx_http_chunkin_util.c └── ngx_http_chunkin_util.h ├── t ├── bug.t ├── error.t ├── ext.t ├── lib │ └── Test │ │ └── Nginx │ │ ├── LWP │ │ └── Chunkin.pm │ │ └── Socket │ │ └── Chunkin.pm ├── max_chunks.t ├── pipelined.t ├── pressure.t ├── random.t ├── sanity.t └── timeout.t ├── util ├── build.sh ├── releng ├── update-readme.sh └── wiki2pod.pl └── valgrind.suppress /.gitattributes: -------------------------------------------------------------------------------- 1 | *.t linguist-language=Text 2 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .libs 2 | *.swp 3 | *.slo 4 | *.la 5 | *.swo 6 | *.lo 7 | *~ 8 | *.o 9 | print.txt 10 | .rsync 11 | *.tar.gz 12 | dist 13 | build[78] 14 | build 15 | tags 16 | update-readme 17 | *.tmp 18 | test/Makefile 19 | test/blib 20 | test.sh 21 | t.sh 22 | t/t.sh 23 | t/servroot/ 24 | releng 25 | reset 26 | *.t_ 27 | .rsync 28 | genmobi.sh 29 | *.mobi 30 | misc/chunked 31 | reindex 32 | src/body.h 33 | src/module.c 34 | src/util.h 35 | src/module.h 36 | src/body.c 37 | src/util.c 38 | ctags 39 | go 40 | util/gprof 41 | all 42 | build1[0123] 43 | buildroot/ 44 | work/ 45 | *.plist 46 | Makefile 47 | nginx 48 | -------------------------------------------------------------------------------- /NginxHttpChunkinModule.wiki: -------------------------------------------------------------------------------- 1 | = Name = 2 | 3 | '''chunkin-nginx-module''' - HTTP 1.1 chunked-encoding request body support for Nginx. 4 | 5 | ''This module is not distributed with the Nginx source.'' See [[#Installation|the installation instructions]]. 6 | 7 | = Status = 8 | 9 | This module is considered production ready. 10 | 11 | = Version = 12 | 13 | This document describes chunkin-nginx-module [http://github.com/agentzh/chunkin-nginx-module/tarball/v0.21 v0.21] released on August 3, 2010. 14 | 15 | = Synopsis = 16 | 17 | 18 | chunkin on; 19 | 20 | error_page 411 = @my_411_error; 21 | location @my_411_error { 22 | chunkin_resume; 23 | } 24 | 25 | location /foo { 26 | # your fastcgi_pass/proxy_pass/set/if and 27 | # any other config directives go here... 28 | } 29 | ... 30 | 31 | 32 | 33 | chunkin on; 34 | 35 | error_page 411 = @my_411_error; 36 | location @my_411_error { 37 | chunkin_resume; 38 | } 39 | 40 | location /bar { 41 | chunkin_keepalive on; # WARNING: too experimental! 42 | 43 | # your fastcgi_pass/proxy_pass/set/if and 44 | # any other config directives go here... 45 | } 46 | 47 | 48 | = Description = 49 | 50 | This module adds [http://tools.ietf.org/html/rfc2616#section-3.6.1 HTTP 1.1 chunked] input support for Nginx without the need of patching the Nginx core. 51 | 52 | Behind the scene, it registers an access-phase handler that will eagerly read and decode incoming request bodies when a Transfer-Encoding: chunked header triggers a 411 error page in Nginx. For requests that are not in the chunked transfer encoding, this module is a "no-op". 53 | 54 | To enable the magic, just turn on the [[#chunkin|chunkin]] config option and define a custom 411 error_page using [[#chunkin_resume|chunkin_resume]], like this: 55 | 56 | 57 | server { 58 | chunkin on; 59 | 60 | error_page 411 = @my_411_error; 61 | location @my_411_error { 62 | chunkin_resume; 63 | } 64 | 65 | ... 66 | } 67 | 68 | 69 | No other modification is required in your nginx.conf file and everything should work out of the box including the standard [[NginxHttpProxyModule|proxy module]] (except for those [[#Known Issues|known issues]]). Note that the [[#chunkin|chunkin]] directive is not allowed in the location block while the [[#chunkin_resume|chunkin_resume]] directive is only allowed on in locations. 70 | 71 | The core module's [[NginxHttpCoreModule#client_body_buffer_size|client_body_buffer_size]], [[NginxHttpCoreModule#client_max_body_size|client_max_body_size]], and [[NginxHttpCoreModule#client_body_timeout|client_body_timeout]] directive settings are honored. Note that, the "body sizes" here always indicate chunked-encoded body, not the data that has already been decoded. Basically, the 72 | chunked-encoded body will always be slightly larger than the original data that is not encoded. 73 | 74 | The [[NginxHttpCoreModule#client_body_in_file_only|client_body_in_file_only]] and [[NginxHttpCoreModule#client_body_in_single_buffer|client_body_in_single_buffer]] settings are followed partially. See [[#Known Issues|Know Issues]]. 75 | 76 | This module is not supposed to be merged into the Nginx core because I've used [http://www.complang.org/ragel/ Ragel] to generate the chunked encoding parser for joy :) 77 | 78 | == How it works == 79 | 80 | Nginx explicitly checks chunked Transfer-Encoding headers and absent content length header in its very 81 | early phase. Well, as early as the ngx_http_process_request_header 82 | function. So this module takes a rather tricky approach. That is, use an output filter to intercept the 411 Length Required error page response issued by ngx_http_process_request_header, 83 | fix things and finally issue an internal redirect to the current location, 84 | thus starting from those phases we all know and love, this time 85 | bypassing the horrible ngx_http_process_request_header function. 86 | 87 | In the rewrite phase of the newly created request, this module eagerly reads in the chunked request body in a way similar to that of the standard ngx_http_read_client_request_body function, but using its own chunked parser generated by Ragel. The decoded request body will be put into r->request_body->bufs and a corresponding Content-Length header will be inserted into r->headers_in. 88 | 89 | Those modules using the standard ngx_http_read_client_request_body function to read the request body will just work out of box because ngx_http_read_client_request_body returns immediately when it sees r->request_body->bufs already exists. 90 | 91 | Special efforts have been made to reduce data copying and dynamic memory allocation. 92 | 93 | = Directives = 94 | 95 | == chunkin == 96 | '''syntax:''' ''chunkin on|off'' 97 | 98 | '''default:''' ''off'' 99 | 100 | '''context:''' ''http, server'' 101 | 102 | Enables or disables this module's hooks. 103 | 104 | == chunkin_resume == 105 | '''syntax:''' ''chunkin_resume'' 106 | 107 | '''default:''' ''none'' 108 | 109 | '''context:''' ''location'' 110 | 111 | This directive must be used in your custom 411 error page location to help this module work correctly. For example: 112 | 113 | 114 | error_page 411 = @my_error; 115 | location @my_error { 116 | chunkin_resume; 117 | } 118 | 119 | 120 | For the technical reason behind the necessity of this directive, please read the nginx-devel thread [http://nginx.org/pipermail/nginx-devel/2009-December/000041.html Content-Length is not ignored for chunked requests: Nginx violates RFC 2616]. 121 | 122 | This directive was first introduced in the [[#v0.17|v0.17]] release. 123 | 124 | == chunkin_max_chunks_per_buf == 125 | '''syntax:''' ''chunkin_max_chunks_per_buf '' 126 | 127 | '''default:''' ''512'' 128 | 129 | '''context:''' ''http, server, location'' 130 | 131 | Set the max chunk count threshold for the buffer determined by the [[NginxHttpCoreModule#client_body_buffer_size|client_body_buffer_size]] directive. 132 | If the average chunk size is 1 KB and your [[NginxHttpCoreModule#client_body_buffer_size|client_body_buffer_size]] setting 133 | is 1 meta bytes, then you should set this threshold to 1024 or 2048. 134 | 135 | When the raw body size is exceeding [[NginxHttpCoreModule#client_body_buffer_size|client_body_buffer_size]] ''or'' the chunk counter is exceeding this chunkin_max_chunks_per_buf setting, the decoded data will be temporarily buffered into disk files, and then the main buffer gets cleared and the chunk counter gets reset back to 0 (or 1 if there's a "pending chunk"). 136 | 137 | This directive was first introduced in the [[#v0.17|v0.17]] release. 138 | 139 | == chunkin_keepalive == 140 | '''syntax:''' ''chunkin_keepalive on|off'' 141 | 142 | '''default:''' ''off'' 143 | 144 | '''context:''' ''http, server, location, if'' 145 | 146 | Turns on or turns off HTTP 1.1 keep-alive and HTTP 1.1 pipelining support. 147 | 148 | Keep-alive without pipelining should be quite stable but pipelining support is very preliminary, limited, and almost untested. 149 | 150 | This directive was first introduced in the [[#v0.07|v0.07 release]]. 151 | 152 | '''Technical note on the HTTP 1.1 pipeling support''' 153 | 154 | The basic idea is to copy the bytes left by my chunked parser in 155 | r->request_body->buf over into r->header_in so that nginx's 156 | ngx_http_set_keepalive and ngx_http_init_request functions will pick 157 | it up for the subsequent pipelined requests. When the request body is 158 | small enough to be completely preread into the r->header_in buffer, 159 | then no data copy is needed here -- just setting r->header_in->pos 160 | correctly will suffice. 161 | 162 | The only issue that remains is how to enlarge r->header_in when the 163 | data left in r->request_body->buf is just too large to be hold in the 164 | remaining room between r->header_in->pos and r->header_in->end. For 165 | now, this module will just give up and simply turn off r->keepalive. 166 | 167 | I know we can always use exactly the remaining room in r->header_in as 168 | the buffer size when reading data from c->recv, but's suboptimal when 169 | the remaining room in r->header_in happens to be very small while 170 | r->request_body->buf is quite large. 171 | 172 | I haven't fully grokked all the details among r->header_in, c->buffer, 173 | busy/free lists and those so-called "large header buffers". Is there a 174 | clean and safe way to reallocate or extend the r->header_in buffer? 175 | 176 | = Trouble Shooting = 177 | 178 | When combining this module with ngx_proxy and ngx_fastcgi, nginx sends a "Transfer-Encoding: " header which is invalid and not being treated well by some webservers on backend, for example, riak. So a work-around for now is to use the [http://wiki.nginx.org/NginxHttpHeadersMoreModule ngx_headers_more] module to remove the Transfer-Encoding completely, as in 179 | 180 | 181 | chunkin on; 182 | 183 | error_page 411 = @my_411_error; 184 | location @my_411_error { 185 | chunkin_resume; 186 | } 187 | 188 | location / { 189 | more_clear_input_headers 'Transfer-Encoding'; 190 | proxy_pass http://riak; 191 | } 192 | 193 | 194 | Thanks [http://github.com/hoodoos hoodoos] for sharing this trick :) 195 | 196 | = Installation = 197 | 198 | Grab the nginx source code from [http://nginx.net/ nginx.net], for example, 199 | the version 0.8.41 (see [[#Compatibility|nginx compatibility]]), and then build the source with this module: 200 | 201 | 202 | $ wget 'http://sysoev.ru/nginx/nginx-0.8.41.tar.gz' 203 | $ tar -xzvf nginx-0.8.41.tar.gz 204 | $ cd nginx-0.8.41/ 205 | 206 | # Here we assume you would install you nginx under /opt/nginx/. 207 | $ ./configure --prefix=/opt/nginx \ 208 | --add-module=/path/to/chunkin-nginx-module 209 | 210 | $ make -j2 211 | $ make install 212 | 213 | 214 | Download the latest version of the release tarball of this module from [http://github.com/agentzh/chunkin-nginx-module/downloads chunkin-nginx-module file list]. 215 | 216 | == For Developers == 217 | 218 | The chunked parser is generated by [http://www.complang.org/ragel/ Ragel]. If you want to 219 | regenerate the parser's C file, i.e., [http://github.com/agentzh/chunkin-nginx-module/blob/master/src/chunked_parser.c src/chunked_parser.c], use 220 | the following command from the root of the chunkin module's source tree: 221 | 222 | 223 | $ ragel -G2 src/chunked_parser.rl 224 | 225 | 226 | = Packages from users = 227 | 228 | == Fedora 13 RPM files == 229 | 230 | The following source and binary rpm files are contributed by Ernest Folch, with nginx 0.8.54, ngx_chunkin v0.21 and ngx_headers_more v0.13: 231 | 232 | * [http://agentzh.org/misc/nginx/ernest/nginx-0.8.54-1.fc13.src.rpm nginx-0.8.54-1.fc13.src.rpm] 233 | * [http://agentzh.org/misc/nginx/ernest/nginx-0.8.54-1.fc13.x86_64.rpm nginx-0.8.54-1.fc13.x86_64.rpm] 234 | 235 | = Compatibility = 236 | 237 | The following versions of Nginx should work with this module: 238 | 239 | * '''1.0.x''' (last tested: 1.0.2) 240 | * '''0.8.x''' (last tested: 0.8.54) 241 | * '''0.7.x >= 0.7.21''' (last tested: 0.7.67) 242 | 243 | Earlier versions of Nginx like 0.6.x and 0.5.x will ''not'' work. 244 | 245 | If you find that any particular version of Nginx above 0.7.21 does not work with this module, please consider [[#Report Bugs|reporting a bug]]. 246 | 247 | = Report Bugs = 248 | 249 | Although a lot of effort has been put into testing and code tuning, there must be some serious bugs lurking somewhere in this module. So whenever you are bitten by any quirks, please don't hesitate to 250 | 251 | # send a bug report or even patches to , 252 | # or create a ticket on the [http://github.com/agentzh/chunkin-nginx-module/issues issue tracking interface] provided by GitHub. 253 | 254 | = Source Repository = 255 | 256 | Available on github at [http://github.com/agentzh/chunkin-nginx-module agentzh/chunkin-nginx-module]. 257 | 258 | = ChangeLog = 259 | 260 | == v0.21 == 261 | * applied a patch from Gong Kaihui (龚开晖) to always call post_handler in ngx_http_chunkin_read_chunked_request_body. 262 | 263 | == v0.20 == 264 | * fixed a bug that may read incomplete chunked body. thanks Gong Kaihui (龚开晖). 265 | * fixed various memory issues in the implementation which may cause nginx processes to crash. 266 | * added support for chunked PUT requests. 267 | * now we always require "error_page 411 @resume" and no default (buggy) magic any more. thanks Gong Kaihui (龚开晖). 268 | 269 | == v0.19 == 270 | * we now use ragel -G2 to generate the chunked parser and we're 36% faster. 271 | * we now eagerly read the data octets in the chunked parser and we're 43% faster. 272 | 273 | == v0.18 == 274 | * added support for chunk-extension to the chunked parser as per [http://tools.ietf.org/html/rfc2616#section-3.6.1 RFC 2616], but we just ignore them (if any) because we don't understand them. 275 | * added more diagnostic information for certian error messages. 276 | 277 | == v0.17 == 278 | * implemented the [[#chunkin_max_chunks_per_buf|chunkin_max_chunks_per_buf]] directive to allow overriding the default 512 setting. 279 | * we now bypass nginx's [http://nginx.org/pipermail/nginx-devel/2009-December/000041.html discard requesty body bug] by requiring our users to define explicit 411 error_page with [[#chunkin_resume|chunkin_resume]] in the error page location. Thanks J for reporting related bugs. 280 | * fixed r->phase_handler in our post read handler. our handler may run one more time before :P 281 | * the chunkin handler now returns NGX_DECLINED rather than NGX_OK when our ngx_http_chunkin_read_chunked_request_body function returns NGX_OK, to avoid bypassing other access-phase handlers. 282 | 283 | == v0.16 == 284 | * turned off ddebug in the previous release. thanks J for reporting it. 285 | 286 | == v0.15 == 287 | * fixed a regression that ctx->chunks_count never incremented in earlier versions. 288 | 289 | == v0.14 == 290 | * now we no longer skip those operations between the (interrupted) ngx_http_process_request_header and the server rewrite phase. this fixed the security issues regarding the [[NginxHttpCoreModule#internal|internal]] directive as well as SSL sessions. 291 | * try to ignore CR/LF/SP/HT at the begining of the chunked body. 292 | * now we allow HT as padding spaces and ignore leading CRLFs. 293 | * improved diagnostic info in the error.log messages when parsefail occurs. 294 | 295 | == v0.11 == 296 | * added a random valid-chunked-request generator in t/random.t. 297 | * fixed a new connection leak issue caught by t/random.t. 298 | 299 | == v0.10 == 300 | * fixed a serious bug in the chunked parser grammer: there would be ambiguity when CRLF appears in the chunked data sections. Thanks J for reporting it. 301 | 302 | == v0.08 == 303 | * fixed gcc compilation errors on x86_64, thanks J for reporting it. 304 | * used the latest Ragel 6.6 to generate the chunked_parser.c file in the source tree. 305 | 306 | == v0.07 == 307 | 308 | * marked the disgarded 411 error page's output chain bufs as consumed by setting buf->pos = buf->last. (See [http://nginx.org/pipermail/nginx-devel/2009-December/000025.html this nginx-devel thread] for more details.) 309 | * added the [[#chunkin_keepalive|chunkin_keepalive]] directive which can enable HTTP 1.1 keep-alive and HTTP 1.1 pipelining, and defaults to off. 310 | * fixed the alphtype bug in the Ragel parser spec; which caused rejection of non-ascii octets in the chunked data. Thanks J for his bug report. 311 | * added Test::Nginx::Socket to test our nginx module on the socket level. Thanks J for his bug report. 312 | * rewrote the bufs recycling part and preread-buf-to-rb-buf transition part, also refactored the Ragel parser spec, thus eliminating lots of serious bugs. 313 | * provided better diagnostics in the error log message for "bad chunked body" parsefails in the chunked parser. For example: 314 | 315 | 316 | 2009/12/02 17:35:52 [error] 32244#0: *1 bad chunked body (offset 7, near "4^M 317 | hell <-- HERE o^M 318 | 0^M 319 | ^M 320 | ", marked by " <-- HERE "). 321 | , client: 127.0.0.1, server: localhost, request: "POST /main 322 | HTTP/1.1", host: "localhost" 323 | 324 | 325 | * added some code to let the chunked parser handle special 0-size chunks that are not the last chunk. 326 | * fixed a connection leak bug regarding incorrect r->main->count reference counter handling for nginx 0.8.11+ (well, the ngx_http_read_client_request_body function in the nginx core also has this issue, I'll report it later.) 327 | 328 | == v0.06 == 329 | * minor optimization: we won't traverse the output chain link if the chain count is not large enough. 330 | 331 | = Test Suite = 332 | 333 | This module comes with a Perl-driven test suite. The [http://github.com/agentzh/chunkin-nginx-module/tree/master/test/t/ test cases] are 334 | [http://github.com/agentzh/chunkin-nginx-module/blob/master/test/t/sanity.t declarative] too. Thanks to the [http://search.cpan.org/perldoc?Test::Base Test::Base] module in the Perl world. 335 | 336 | To run it on your side: 337 | 338 | 339 | $ cd test 340 | $ PATH=/path/to/your/nginx-with-chunkin-module:$PATH prove -r t 341 | 342 | 343 | You need to terminate any Nginx processes before running the test suite if you have changed the Nginx server binary. 344 | 345 | At the moment, [http://search.cpan.org/perldoc?LWP::UserAgent LWP::UserAgent] is used by the [http://github.com/agentzh/chunkin-nginx-module/blob/master/test/lib/Test/Nginx/LWP.pm test scaffold] for simplicity. 346 | 347 | Because a single nginx server (by default, localhost:1984) is used across all the test scripts (.t files), it's meaningless to run the test suite in parallel by specifying -jN when invoking the prove utility. 348 | 349 | Some parts of the test suite requires modules [[NginxHttpProxyModule|proxy]] and [[NginxHttpEchoModule|echo]] to be enabled as well when building Nginx. 350 | 351 | = Known Issues = 352 | 353 | * May not work with certain 3rd party modules like the [http://www.grid.net.ru/nginx/upload.en.html upload module] because it implements its own request body reading mechanism. 354 | * "client_body_in_single_buffer on" may *not* be obeyed for short contents and fast network. 355 | * "client_body_in_file_only on" may *not* be obeyed for short contents and fast network. 356 | * HTTP 1.1 pipelining may not fully work yet. 357 | 358 | = TODO = 359 | 360 | * make the chunkin handler run at the end of the access phase rather than beginning. 361 | * add support for trailers as specified in the [http://tools.ietf.org/html/rfc2616#section-3.6.1 RFC 2616]. 362 | * fix the [[#Known Issues|known issues]]. 363 | 364 | = Getting involved = 365 | 366 | You'll be very welcomed to submit patches to the [[#Author|author]] or just ask for a commit bit to the [[#Source Repository|source repository]] on GitHub. 367 | 368 | = Author = 369 | 370 | agentzh (章亦春) '''' 371 | 372 | This wiki page is also maintained by the author himself, and everybody is encouraged to improve this page as well. 373 | 374 | = Copyright & License = 375 | 376 | The basic client request body reading code is based on the ngx_http_read_client_request_body function and its utility functions in the Nginx 0.8.20 core. This part of code is copyrighted by Igor Sysoev. 377 | 378 | Copyright (c) 2009, Taobao Inc., Alibaba Group ( http://www.taobao.com ). 379 | 380 | Copyright (c) 2009, agentzh . 381 | 382 | This module is licensed under the terms of the BSD license. 383 | 384 | Redistribution and use in source and binary forms, with or without 385 | modification, are permitted provided that the following conditions 386 | are met: 387 | 388 | * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 389 | * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 390 | * Neither the name of the Taobao Inc. nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. 391 | 392 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 393 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 394 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 395 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 396 | HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 397 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED 398 | TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR 399 | PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF 400 | LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING 401 | NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 402 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 403 | 404 | = See Also = 405 | 406 | * The original thread on the Nginx mailing list that inspires this module's development: [http://forum.nginx.org/read.php?2,4453,20543 "'Content-Length' header for POSTs"]. 407 | * The orginal announcement thread on the Nginx mailing list: [http://forum.nginx.org/read.php?2,22967 "The chunkin module: Experimental chunked input support for Nginx"]. 408 | * The original [http://agentzh.spaces.live.com/blog/cns!FF3A735632E41548!481.entry blog post] about this module's initial development. 409 | * The thread discussing chunked input support on the nginx-devel mailing list: [http://nginx.org/pipermail/nginx-devel/2009-December/000021.html "Chunked request body and HTTP header parser"]. 410 | * The [[NginxHttpEchoModule|echo module]] for Nginx module's automated testing. 411 | * [http://tools.ietf.org/html/rfc2616#section-3.6.1 RFC 2616 - Chunked Transfer Coding]. 412 | 413 | -------------------------------------------------------------------------------- /README.markdown: -------------------------------------------------------------------------------- 1 | Name 2 | ==== 3 | 4 | **ngx_chunkin** - HTTP 1.1 chunked-encoding request body support for Nginx. 5 | 6 | *This module is not distributed with the Nginx source.* See [the installation instructions](#installation). 7 | 8 | Table of Contents 9 | ================= 10 | 11 | * [Status](#status) 12 | * [Version](#version) 13 | * [Synopsis](#synopsis) 14 | * [Description](#description) 15 | * [How it works](#how-it-works) 16 | * [Directives](#directives) 17 | * [chunkin](#chunkin) 18 | * [chunkin_resume](#chunkin_resume) 19 | * [chunkin_max_chunks_per_buf](#chunkin_max_chunks_per_buf) 20 | * [chunkin_keepalive](#chunkin_keepalive) 21 | * [Installation](#installation) 22 | * [Installation on Ubuntu 10.04 LTS using apt/dpkg](#installation-on-ubuntu-1004-lts-using-aptdpkg) 23 | * [For Developers](#for-developers) 24 | * [Packages from users](#packages-from-users) 25 | * [Fedora 13 RPM files](#fedora-13-rpm-files) 26 | * [Nginx Compatibility](#nginx-compatibility) 27 | * [Community](#community) 28 | * [English Mailing List](#english-mailing-list) 29 | * [Chinese Mailing List](#chinese-mailing-list) 30 | * [Bugs and Patches](#bugs-and-patches) 31 | * [Source Repository](#source-repository) 32 | * [ChangeLog](#changelog) 33 | * [v0.22](#v022) 34 | * [v0.21](#v021) 35 | * [v0.20](#v020) 36 | * [v0.19](#v019) 37 | * [v0.18](#v018) 38 | * [v0.17](#v017) 39 | * [v0.16](#v016) 40 | * [v0.15](#v015) 41 | * [v0.14](#v014) 42 | * [v0.11](#v011) 43 | * [v0.10](#v010) 44 | * [v0.08](#v008) 45 | * [v0.07](#v007) 46 | * [v0.06](#v006) 47 | * [Test Suite](#test-suite) 48 | * [Known Issues](#known-issues) 49 | * [TODO](#todo) 50 | * [Getting involved](#getting-involved) 51 | * [Author](#author) 52 | * [Copyright & License](#copyright--license) 53 | * [See Also](#see-also) 54 | 55 | Status 56 | ====== 57 | 58 | This module is no longer needed for Nginx 1.3.9+ because since 1.3.9, the Nginx core already has built-in support for the chunked request bodies. 59 | 60 | And this module is now only maintained for Nginx versions older than 1.3.9. 61 | 62 | Version 63 | ======= 64 | 65 | This document describes ngx_chunkin [v0.23](http://github.com/agentzh/chunkin-nginx-module/tags) released on February 8, 2013. 66 | 67 | Synopsis 68 | ======== 69 | 70 | ```nginx 71 | 72 | chunkin on; 73 | 74 | error_page 411 = @my_411_error; 75 | location @my_411_error { 76 | chunkin_resume; 77 | } 78 | 79 | location /foo { 80 | # your fastcgi_pass/proxy_pass/set/if and 81 | # any other config directives go here... 82 | } 83 | ... 84 | ``` 85 | 86 | ```nginx 87 | 88 | chunkin on; 89 | 90 | error_page 411 = @my_411_error; 91 | location @my_411_error { 92 | chunkin_resume; 93 | } 94 | 95 | location /bar { 96 | chunkin_keepalive on; # WARNING: too experimental! 97 | 98 | # your fastcgi_pass/proxy_pass/set/if and 99 | # any other config directives go here... 100 | } 101 | ``` 102 | 103 | [Back to TOC](#table-of-contents) 104 | 105 | Description 106 | =========== 107 | 108 | This module adds [HTTP 1.1 chunked](http://tools.ietf.org/html/rfc2616#section-3.6.1) input support for Nginx without the need of patching the Nginx core. 109 | 110 | Behind the scene, it registers an access-phase handler that will eagerly read and decode incoming request bodies when a `Transfer-Encoding: chunked` header triggers a `411` error page in Nginx. For requests that are not in the `chunked` transfer encoding, this module is a "no-op". 111 | 112 | To enable the magic, just turn on the [chunkin](#chunkin) config option and define a custom `411 error_page` using [chunkin_resume](#chunkin_resume), like this: 113 | 114 | ```nginx 115 | 116 | server { 117 | chunkin on; 118 | 119 | error_page 411 = @my_411_error; 120 | location @my_411_error { 121 | chunkin_resume; 122 | } 123 | 124 | ... 125 | } 126 | ``` 127 | 128 | No other modification is required in your nginx.conf file and everything should work out of the box including the standard [proxy module](http://nginx.org/en/docs/http/ngx_http_proxy_module.html) (except for those [known issues](#known-issues)). Note that the [chunkin](#chunkin) directive is not allowed in the location block while the [chunkin_resume](#chunkin_resume) directive is only allowed on in `locations`. 129 | 130 | The core module's [client_body_buffer_size](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size), [client_max_body_size](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_max_body_size), and [client_body_timeout](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_timeout) directive settings are honored. Note that, the "body sizes" here always indicate chunked-encoded body, not the data that has already been decoded. Basically, the 131 | chunked-encoded body will always be slightly larger than the original data that is not encoded. 132 | 133 | The [client_body_in_file_only](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_in_file_only) and [client_body_in_single_buffer](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_in_single_buffer) settings are followed partially. See [Know Issues](#known-issues). 134 | 135 | This module is not supposed to be merged into the Nginx core because I've used [Ragel](http://www.complang.org/ragel/) to generate the chunked encoding parser for joy :) 136 | 137 | [Back to TOC](#table-of-contents) 138 | 139 | How it works 140 | ------------ 141 | 142 | Nginx explicitly checks chunked `Transfer-Encoding` headers and absent content length header in its very 143 | early phase. Well, as early as the `ngx_http_process_request_header` 144 | function. So this module takes a rather tricky approach. That is, use an output filter to intercept the `411 Length Required` error page response issued by `ngx_http_process_request_header`, 145 | fix things and finally issue an internal redirect to the current location, 146 | thus starting from those phases we all know and love, this time 147 | bypassing the horrible `ngx_http_process_request_header` function. 148 | 149 | In the `rewrite` phase of the newly created request, this module eagerly reads in the chunked request body in a way similar to that of the standard `ngx_http_read_client_request_body` function, but using its own chunked parser generated by Ragel. The decoded request body will be put into `r->request_body->bufs` and a corresponding `Content-Length` header will be inserted into `r->headers_in`. 150 | 151 | Those modules using the standard `ngx_http_read_client_request_body` function to read the request body will just work out of box because `ngx_http_read_client_request_body` returns immediately when it sees `r->request_body->bufs` already exists. 152 | 153 | Special efforts have been made to reduce data copying and dynamic memory allocation. 154 | 155 | [Back to TOC](#table-of-contents) 156 | 157 | Directives 158 | ========== 159 | 160 | [Back to TOC](#table-of-contents) 161 | 162 | chunkin 163 | ------- 164 | **syntax:** *chunkin on|off* 165 | 166 | **default:** *off* 167 | 168 | **context:** *http, server* 169 | 170 | **phase:** *access* 171 | 172 | Enables or disables this module's hooks. 173 | 174 | [Back to TOC](#table-of-contents) 175 | 176 | chunkin_resume 177 | -------------- 178 | **syntax:** *chunkin_resume* 179 | 180 | **default:** *no* 181 | 182 | **context:** *location* 183 | 184 | **phase:** *content* 185 | 186 | This directive must be used in your custom `411 error page` location to help this module work correctly. For example: 187 | 188 | ```nginx 189 | 190 | error_page 411 = @my_error; 191 | location @my_error { 192 | chunkin_resume; 193 | } 194 | ``` 195 | 196 | For the technical reason behind the necessity of this directive, please read the `nginx-devel` thread [Content-Length is not ignored for chunked requests: Nginx violates RFC 2616](http://nginx.org/pipermail/nginx-devel/2009-December/000041.html). 197 | 198 | This directive was first introduced in the [v0.17](#v017) release. 199 | 200 | [Back to TOC](#table-of-contents) 201 | 202 | chunkin_max_chunks_per_buf 203 | -------------------------- 204 | **syntax:** *chunkin_max_chunks_per_buf <number>* 205 | 206 | **default:** *512* 207 | 208 | **context:** *http, server, location* 209 | 210 | Set the max chunk count threshold for the buffer determined by the [client_body_buffer_size](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size) directive. 211 | If the average chunk size is `1 KB` and your [client_body_buffer_size](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size) setting 212 | is 1 meta bytes, then you should set this threshold to `1024` or `2048`. 213 | 214 | When the raw body size is exceeding [client_body_buffer_size](http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size) *or* the chunk counter is exceeding this `chunkin_max_chunks_per_buf` setting, the decoded data will be temporarily buffered into disk files, and then the main buffer gets cleared and the chunk counter gets reset back to 0 (or `1` if there's a "pending chunk"). 215 | 216 | This directive was first introduced in the [v0.17](#v017) release. 217 | 218 | [Back to TOC](#table-of-contents) 219 | 220 | chunkin_keepalive 221 | ----------------- 222 | **syntax:** *chunkin_keepalive on|off* 223 | 224 | **default:** *off* 225 | 226 | **context:** *http, server, location, if* 227 | 228 | Turns on or turns off HTTP 1.1 keep-alive and HTTP 1.1 pipelining support. 229 | 230 | Keep-alive without pipelining should be quite stable but pipelining support is very preliminary, limited, and almost untested. 231 | 232 | This directive was first introduced in the [v0.07 release](#v007). 233 | 234 | **Technical note on the HTTP 1.1 pipeling support** 235 | 236 | The basic idea is to copy the bytes left by my chunked parser in 237 | `r->request_body->buf` over into `r->header_in` so that nginx's 238 | `ngx_http_set_keepalive` and `ngx_http_init_request` functions will pick 239 | it up for the subsequent pipelined requests. When the request body is 240 | small enough to be completely preread into the `r->header_in` buffer, 241 | then no data copy is needed here -- just setting `r->header_in->pos` 242 | correctly will suffice. 243 | 244 | The only issue that remains is how to enlarge `r->header_in` when the 245 | data left in `r->request_body->buf` is just too large to be hold in the 246 | remaining room between `r->header_in->pos` and `r->header_in->end`. For 247 | now, this module will just give up and simply turn off `r->keepalive`. 248 | 249 | I know we can always use exactly the remaining room in `r->header_in` as 250 | the buffer size when reading data from `c->recv`, but's suboptimal when 251 | the remaining room in `r->header_in` happens to be very small while 252 | `r->request_body->buf` is quite large. 253 | 254 | I haven't fully grokked all the details among `r->header_in`, `c->buffer`, 255 | busy/free lists and those so-called "large header buffers". Is there a 256 | clean and safe way to reallocate or extend the `r->header_in` buffer? 257 | 258 | [Back to TOC](#table-of-contents) 259 | 260 | Installation 261 | ============ 262 | 263 | Grab the nginx source code from [nginx.org](http://nginx.org/), for example, 264 | the version 1.2.6 (see [nginx compatibility](#compatibility)), and then build the source with this module: 265 | 266 | ```bash 267 | 268 | wget 'http://nginx.org/download/nginx-1.2.6.tar.gz' 269 | tar -xzvf nginx-1.2.6.tar.gz 270 | cd nginx-1.2.6/ 271 | 272 | # Here we assume you would install you nginx under /opt/nginx/. 273 | ./configure --prefix=/opt/nginx \ 274 | --add-module=/path/to/chunkin-nginx-module 275 | 276 | make -j2 277 | make install 278 | ``` 279 | 280 | Download the latest version of the release tarball of this module from [chunkin-nginx-module file list](http://github.com/agentzh/chunkin-nginx-module/tags). 281 | 282 | 283 | [Back to TOC](#table-of-contents) 284 | 285 | Installation on Ubuntu 10.04 LTS using apt/dpkg 286 | ----------------------------------------------- 287 | 288 | You need to have dpkg-dev installed (apt-get install dpkg-dev) 289 | And this guide assumes we are using the official repo. 290 | ```bash 291 | 292 | sudo -s 293 | apt-get source nginx 294 | apt-get build-dep nginx 295 | wget 'https://github.com/agentzh/chunkin-nginx-module/tarball/v0.23rc2' 296 | tar -xzvf v0.23rc2 297 | 298 | #rename directory to make it easier to remember later. 299 | mv agentzh-chunkin-* chunkin 300 | 301 | #this next one of course will change depending on which repo/version you are using. 302 | cd nginx-1.0.14/ 303 | vim debian/rules 304 | 305 | #See the ./configure section (for both "override_dh_auto_build": and "configure_debug:") 306 | #At this point it's a good idea to have a idea of what you will need of modules/addons. remove any lines you don't need. 307 | #The current last item in the ./configure section needs to have '\' added. 308 | #Then add this: --add-module=../chunkin 309 | 310 | dpkg-buildpackage 311 | 312 | cd .. 313 | #the next one of course will change according to version/build 314 | dpkg -i nginx_1.0.14-1~lucid_i386.deb 315 | 316 | #verify install with nginx -V see that add-module=../chunkin is in the configurea arguemnts list. 317 | 318 | ``` 319 | 320 | [Back to TOC](#table-of-contents) 321 | 322 | For Developers 323 | -------------- 324 | 325 | The chunked parser is generated by [Ragel](http://www.complang.org/ragel/). If you want to 326 | regenerate the parser's C file, i.e., [src/chunked_parser.c](http://github.com/agentzh/chunkin-nginx-module/blob/master/src/chunked_parser.c), use 327 | the following command from the root of the chunkin module's source tree: 328 | 329 | ```bash 330 | 331 | $ ragel -G2 src/chunked_parser.rl 332 | ``` 333 | 334 | [Back to TOC](#table-of-contents) 335 | 336 | Packages from users 337 | =================== 338 | 339 | [Back to TOC](#table-of-contents) 340 | 341 | Fedora 13 RPM files 342 | ------------------- 343 | 344 | The following source and binary rpm files are contributed by Ernest Folch, with nginx 0.8.54, ngx_chunkin v0.21 and ngx_headers_more v0.13: 345 | 346 | * [nginx-0.8.54-1.fc13.src.rpm](http://agentzh.org/misc/nginx/ernest/nginx-0.8.54-1.fc13.src.rpm) 347 | * [nginx-0.8.54-1.fc13.x86_64.rpm](http://agentzh.org/misc/nginx/ernest/nginx-0.8.54-1.fc13.x86_64.rpm) 348 | 349 | [Back to TOC](#table-of-contents) 350 | 351 | Nginx Compatibility 352 | =================== 353 | 354 | The following versions of Nginx should work with this module: 355 | 356 | * **1.2.x** (last tested: 1.2.6) 357 | * **1.1.x** (last tested: 1.1.5) 358 | * **1.0.x** (last tested: 1.0.10) 359 | * **0.8.x** (last tested: 0.8.54) 360 | * **0.7.x >= 0.7.21** (last tested: 0.7.67) 361 | 362 | Earlier versions of Nginx like 0.6.x and 0.5.x will *not* work. 363 | 364 | If you find that any particular version of Nginx above 0.7.21 does not work with this module, please consider [reporting a bug](#report-bugs). 365 | 366 | [Back to TOC](#table-of-contents) 367 | 368 | Community 369 | ========= 370 | 371 | [Back to TOC](#table-of-contents) 372 | 373 | English Mailing List 374 | -------------------- 375 | 376 | The [openresty-en](https://groups.google.com/group/openresty-en) mailing list is for English speakers. 377 | 378 | [Back to TOC](#table-of-contents) 379 | 380 | Chinese Mailing List 381 | -------------------- 382 | 383 | The [openresty](https://groups.google.com/group/openresty) mailing list is for Chinese speakers. 384 | 385 | [Back to TOC](#table-of-contents) 386 | 387 | Bugs and Patches 388 | ================ 389 | 390 | Please submit bug reports, wishlists, or patches by 391 | 392 | 1. creating a ticket on the [GitHub Issue Tracker](http://github.com/agentzh/chunkin-nginx-module/issues), 393 | 1. or posting to the [OpenResty community](#community). 394 | 395 | [Back to TOC](#table-of-contents) 396 | 397 | Source Repository 398 | ================= 399 | 400 | Available on github at [agentzh/chunkin-nginx-module](http://github.com/agentzh/chunkin-nginx-module). 401 | 402 | [Back to TOC](#table-of-contents) 403 | 404 | ChangeLog 405 | ========= 406 | 407 | [Back to TOC](#table-of-contents) 408 | 409 | v0.22 410 | ----- 411 | * now we remove the request header Transfer-Encoding completely because at least Apache will complain about the empty-value `Transfer-Encoding` request header. thanks hoodoos and Sandesh Kotwal. 412 | * now we allow DELETE requests with chunked request bodies per hoodoos's request. 413 | * now we use the 2-clause BSD license. 414 | 415 | [Back to TOC](#table-of-contents) 416 | 417 | v0.21 418 | ----- 419 | * applied a patch from Gong Kaihui (龚开晖) to always call `post_handler` in `ngx_http_chunkin_read_chunked_request_body`. 420 | 421 | [Back to TOC](#table-of-contents) 422 | 423 | v0.20 424 | ----- 425 | * fixed a bug that may read incomplete chunked body. thanks Gong Kaihui (龚开晖). 426 | * fixed various memory issues in the implementation which may cause nginx processes to crash. 427 | * added support for chunked PUT requests. 428 | * now we always require "error_page 411 @resume" and no default (buggy) magic any more. thanks Gong Kaihui (龚开晖). 429 | 430 | [Back to TOC](#table-of-contents) 431 | 432 | v0.19 433 | ----- 434 | * we now use ragel -G2 to generate the chunked parser and we're 36% faster. 435 | * we now eagerly read the data octets in the chunked parser and we're 43% faster. 436 | 437 | [Back to TOC](#table-of-contents) 438 | 439 | v0.18 440 | ----- 441 | * added support for `chunk-extension` to the chunked parser as per [RFC 2616](http://tools.ietf.org/html/rfc2616#section-3.6.1), but we just ignore them (if any) because we don't understand them. 442 | * added more diagnostic information for certain error messages. 443 | 444 | [Back to TOC](#table-of-contents) 445 | 446 | v0.17 447 | ----- 448 | * implemented the [chunkin_max_chunks_per_buf](#chunkin_max_chunks_per_buf) directive to allow overriding the default `512` setting. 449 | * we now bypass nginx's [discard requesty body bug](http://nginx.org/pipermail/nginx-devel/2009-December/000041.html) by requiring our users to define explicit `411 error_page` with [chunkin_resume](#chunkin_resume) in the error page location. Thanks J for reporting related bugs. 450 | * fixed `r->phase_handler` in our post read handler. our handler may run one more time before :P 451 | * the chunkin handler now returns `NGX_DECLINED` rather than `NGX_OK` when our `ngx_http_chunkin_read_chunked_request_body` function returns `NGX_OK`, to avoid bypassing other access-phase handlers. 452 | 453 | [Back to TOC](#table-of-contents) 454 | 455 | v0.16 456 | ----- 457 | * turned off ddebug in the previous release. thanks J for reporting it. 458 | 459 | [Back to TOC](#table-of-contents) 460 | 461 | v0.15 462 | ----- 463 | * fixed a regression that ctx->chunks_count never incremented in earlier versions. 464 | 465 | [Back to TOC](#table-of-contents) 466 | 467 | v0.14 468 | ----- 469 | * now we no longer skip those operations between the (interrupted) ngx_http_process_request_header and the server rewrite phase. this fixed the security issues regarding the [internal](http://nginx.org/en/docs/http/ngx_http_core_module.html#internal) directive as well as SSL sessions. 470 | * try to ignore CR/LF/SP/HT at the begining of the chunked body. 471 | * now we allow HT as padding spaces and ignore leading CRLFs. 472 | * improved diagnostic info in the error.log messages when parsefail occurs. 473 | 474 | [Back to TOC](#table-of-contents) 475 | 476 | v0.11 477 | ----- 478 | * added a random valid-chunked-request generator in t/random.t. 479 | * fixed a new connection leak issue caught by t/random.t. 480 | 481 | [Back to TOC](#table-of-contents) 482 | 483 | v0.10 484 | ----- 485 | * fixed a serious bug in the chunked parser grammer: there would be ambiguity when CRLF appears in the chunked data sections. Thanks J for reporting it. 486 | 487 | [Back to TOC](#table-of-contents) 488 | 489 | v0.08 490 | ----- 491 | * fixed gcc compilation errors on x86_64, thanks J for reporting it. 492 | * used the latest Ragel 6.6 to generate the `chunked_parser.c` file in the source tree. 493 | 494 | [Back to TOC](#table-of-contents) 495 | 496 | v0.07 497 | ----- 498 | 499 | * marked the disgarded 411 error page's output chain bufs as consumed by setting `buf->pos = buf->last`. (See [this nginx-devel thread](http://nginx.org/pipermail/nginx-devel/2009-December/000025.html) for more details.) 500 | * added the [chunkin_keepalive](#chunkin_keepalive) directive which can enable HTTP 1.1 keep-alive and HTTP 1.1 pipelining, and defaults to `off`. 501 | * fixed the `alphtype` bug in the Ragel parser spec; which caused rejection of non-ascii octets in the chunked data. Thanks J for his bug report. 502 | * added `Test::Nginx::Socket` to test our nginx module on the socket level. Thanks J for his bug report. 503 | * rewrote the bufs recycling part and preread-buf-to-rb-buf transition part, also refactored the Ragel parser spec, thus eliminating lots of serious bugs. 504 | * provided better diagnostics in the error log message for "bad chunked body" parsefails in the chunked parser. For example: 505 | 506 | 507 | 2009/12/02 17:35:52 [error] 32244#0: *1 bad chunked body (offset 7, near "4^M 508 | hell <-- HERE o^M 509 | 0^M 510 | ^M 511 | ", marked by " <-- HERE "). 512 | , client: 127.0.0.1, server: localhost, request: "POST /main 513 | HTTP/1.1", host: "localhost" 514 | 515 | 516 | * added some code to let the chunked parser handle special 0-size chunks that are not the last chunk. 517 | * fixed a connection leak bug regarding incorrect `r->main->count` reference counter handling for nginx 0.8.11+ (well, the `ngx_http_read_client_request_body` function in the nginx core also has this issue, I'll report it later.) 518 | 519 | [Back to TOC](#table-of-contents) 520 | 521 | v0.06 522 | ----- 523 | * minor optimization: we won't traverse the output chain link if the chain count is not large enough. 524 | 525 | [Back to TOC](#table-of-contents) 526 | 527 | Test Suite 528 | ========== 529 | 530 | This module comes with a Perl-driven test suite. The [test cases](http://github.com/agentzh/chunkin-nginx-module/tree/master/test/t/) are 531 | [declarative](http://github.com/agentzh/chunkin-nginx-module/blob/master/test/t/sanity.t) too. Thanks to the [Test::Base](http://search.cpan.org/perldoc?Test::Base) module in the Perl world. 532 | 533 | To run it on your side: 534 | 535 | ```bash 536 | 537 | $ cd test 538 | $ PATH=/path/to/your/nginx-with-chunkin-module:$PATH prove -r t 539 | ``` 540 | 541 | You need to terminate any Nginx processes before running the test suite if you have changed the Nginx server binary. 542 | 543 | At the moment, [LWP::UserAgent](http://search.cpan.org/perldoc?LWP::UserAgent) is used by the [test scaffold](http://github.com/agentzh/chunkin-nginx-module/blob/master/test/lib/Test/Nginx/LWP.pm) for simplicity. 544 | 545 | Because a single nginx server (by default, `localhost:1984`) is used across all the test scripts (`.t` files), it's meaningless to run the test suite in parallel by specifying `-jN` when invoking the `prove` utility. 546 | 547 | Some parts of the test suite requires modules [proxy](http://nginx.org/en/docs/http/ngx_http_proxy_module.html) and [echo](http://github.com/openresty/echo-nginx-module) to be enabled as well when building Nginx. 548 | 549 | [Back to TOC](#table-of-contents) 550 | 551 | Known Issues 552 | ============ 553 | 554 | * May not work with certain 3rd party modules like the [upload module](http://www.grid.net.ru/nginx/upload.en.html) because it implements its own request body reading mechanism. 555 | * "client_body_in_single_buffer on" may *not* be obeyed for short contents and fast network. 556 | * "client_body_in_file_only on" may *not* be obeyed for short contents and fast network. 557 | * HTTP 1.1 pipelining may not fully work yet. 558 | 559 | [Back to TOC](#table-of-contents) 560 | 561 | TODO 562 | ==== 563 | 564 | * make the chunkin handler run at the end of the `access phase` rather than beginning. 565 | * add support for `trailers` as specified in the [RFC 2616](http://tools.ietf.org/html/rfc2616#section-3.6.1). 566 | * fix the [known issues](#known-issues). 567 | 568 | [Back to TOC](#table-of-contents) 569 | 570 | Getting involved 571 | ================ 572 | 573 | You'll be very welcomed to submit patches to the [author](#author) or just ask for a commit bit to the [source repository](#source-repository) on GitHub. 574 | 575 | [Back to TOC](#table-of-contents) 576 | 577 | Author 578 | ====== 579 | 580 | Yichun "agentzh" Zhang (章亦春) *<agentzh@gmail.com>*, OpenResty Inc. 581 | 582 | This wiki page is also maintained by the author himself, and everybody is encouraged to improve this page as well. 583 | 584 | [Back to TOC](#table-of-contents) 585 | 586 | Copyright & License 587 | =================== 588 | 589 | The basic client request body reading code is based on the `ngx_http_read_client_request_body` function and its utility functions in the Nginx 0.8.20 core. This part of code is copyrighted by Igor Sysoev. 590 | 591 | Copyright (c) 2009-2017, Yichun Zhang (agentzh), OpenResty Inc. 592 | 593 | This module is licensed under the terms of the BSD license. 594 | 595 | Redistribution and use in source and binary forms, with or without 596 | modification, are permitted provided that the following conditions 597 | are met: 598 | 599 | * Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 600 | * Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 601 | * Neither the name of the Taobao Inc. nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission. 602 | 603 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS 604 | "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT 605 | LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR 606 | A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT 607 | HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, 608 | SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED 609 | TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR 610 | PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF 611 | LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING 612 | NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 613 | SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 614 | 615 | [Back to TOC](#table-of-contents) 616 | 617 | See Also 618 | ======== 619 | 620 | * The original thread on the Nginx mailing list that inspires this module's development: ["'Content-Length' header for POSTs"](http://forum.nginx.org/read.php?2,4453,20543). 621 | * The orginal announcement thread on the Nginx mailing list: ["The chunkin module: Experimental chunked input support for Nginx"](http://forum.nginx.org/read.php?2,22967). 622 | * The original [blog post](http://agentzh.spaces.live.com/blog/cns!FF3A735632E41548!481.entry) about this module's initial development. 623 | * The thread discussing chunked input support on the nginx-devel mailing list: ["Chunked request body and HTTP header parser"](http://nginx.org/pipermail/nginx-devel/2009-December/000021.html). 624 | * The [echo module](http://github.com/openresty/echo-nginx-module) for Nginx module's automated testing. 625 | * [RFC 2616 - Chunked Transfer Coding](http://tools.ietf.org/html/rfc2616#section-3.6.1). 626 | 627 | -------------------------------------------------------------------------------- /config: -------------------------------------------------------------------------------- 1 | ngx_addon_name=ngx_http_chunkin_filter_module 2 | HTTP_AUX_FILTER_MODULES="$HTTP_AUX_FILTER_MODULES ngx_http_chunkin_filter_module" 3 | NGX_ADDON_SRCS="$NGX_ADDON_SRCS $ngx_addon_dir/src/ngx_http_chunkin_filter_module.c $ngx_addon_dir/src/ngx_http_chunkin_request_body.c $ngx_addon_dir/src/chunked_parser.c $ngx_addon_dir/src/ngx_http_chunkin_util.c" 4 | NGX_ADDON_DEPS="$NGX_ADDON_DEPS $ngx_addon_dir/src/ddebug.h $ngx_addon_dir/src/ngx_http_chunkin_request_body.h $ngx_addon_dir/src/ngx_http_chunkin_filter_module.h $ngx_addon_dir/src/chunked_parser.h $ngx_addon_dir/src/ngx_http_chunkin_util.h" 5 | 6 | -------------------------------------------------------------------------------- /misc/Makefile: -------------------------------------------------------------------------------- 1 | %: %.c 2 | gcc -Wall -O2 -o $@ $< 3 | 4 | %.c: %.rl 5 | ragel -G2 $< 6 | 7 | -------------------------------------------------------------------------------- /misc/README: -------------------------------------------------------------------------------- 1 | This is a standalone program that tests the chunked body parser. 2 | 3 | To build the "chunked" program, run the following command within this directory: 4 | 5 | $ make chunked 6 | 7 | Then invoke the program like this: 8 | 9 | $ perl -e 'print "5\r\nhello\r\n0\r\n\r\n"' | ./chunked 10 | 11 | -------------------------------------------------------------------------------- /misc/chunked.rl: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | 4 | enum { 5 | BUFSIZE = 256 6 | }; 7 | 8 | %% machine chunked; 9 | %% write data; 10 | 11 | int main (int argc, char **argv) 12 | { 13 | int cs, res = 0; 14 | char *p, *pe, *eof; 15 | char buf[BUFSIZE]; 16 | size_t len = 0; 17 | size_t bytes_read; 18 | eof = NULL; 19 | size_t data_bytes_read = 0, chunk_size = 0; 20 | int chunk_size_order = 0; 21 | 22 | %%{ 23 | action bad_last_chunk { 24 | fprintf(stderr, "bad last chunk (offset %d).\n", p - buf); 25 | } 26 | 27 | action bad_chunk_data { 28 | fprintf(stderr, "bad chunk data " 29 | "(bytes already read %d, bytes expected: %d): " 30 | "offset %d.\n", data_bytes_read, chunk_size, p - buf); 31 | } 32 | 33 | action test_len { 34 | data_bytes_read < chunk_size 35 | } 36 | 37 | action read_data_byte { 38 | data_bytes_read++; 39 | fprintf(stderr, "data bytes read: %d (char: %c)\n", 40 | data_bytes_read, *p); 41 | } 42 | 43 | action start_reading_size { 44 | data_bytes_read = 0; 45 | chunk_size = 0; 46 | chunk_size_order = 0; 47 | } 48 | 49 | action read_size { 50 | chunk_size <<= 4; 51 | chunk_size_order++; 52 | if (*p >= 'A' && *p <= 'F') { 53 | chunk_size |= 10 + *p - 'A'; 54 | } else if (*p >= 'a' && *p <= 'f') { 55 | chunk_size |= 10 + *p - 'a'; 56 | } else { 57 | chunk_size |= *p - '0'; 58 | } 59 | printf("INFO: chunk size: %d\n", chunk_size); 60 | } 61 | 62 | action verify_data { 63 | if (data_bytes_read != chunk_size) { 64 | fprintf(stderr, "ERROR: chunk size not meet: " 65 | "%d != %d\n", data_bytes_read, chunk_size); 66 | fbreak; 67 | } 68 | } 69 | 70 | CRLF = "\r\n"; 71 | 72 | chunk_size = (xdigit+ - "0") >start_reading_size $read_size; 73 | 74 | chunk_data_octet = any 75 | when test_len $err(bad_chunk_data) $read_data_byte; 76 | 77 | chunk_data = chunk_data_octet* %verify_data; 78 | 79 | chunk = chunk_size " "* CRLF 80 | chunk_data CRLF; 81 | 82 | last_chunk = "0" " "* CRLF ${ fprintf(stderr, "in last chunk %d (cs: %d)\n", p - buf, cs); }; #$err(bad_last_chunk); 83 | 84 | main := (chunk* 85 | last_chunk 86 | CRLF) %err{ fprintf(stderr, "in end %d (cs: %d)\n", p - buf, cs); }; 87 | 88 | }%% 89 | 90 | %% write init; 91 | 92 | while (len < BUFSIZE) { 93 | bytes_read = 94 | fread(buf + len, sizeof(char), BUFSIZE - len, stdin); 95 | if (bytes_read == 0) { 96 | break; 97 | } 98 | len += bytes_read; 99 | if (feof(stdin) || ferror(stdin)) { 100 | break; 101 | } 102 | } 103 | 104 | p = buf; 105 | pe = buf + len; 106 | 107 | %% write exec; 108 | 109 | printf("cs >= first_final: %d, execute = %i, p moved %d, " 110 | "p remaining: %d\n", cs >= chunked_first_final, res, p - buf, pe - p); 111 | 112 | return 0; 113 | } 114 | 115 | -------------------------------------------------------------------------------- /src/chunked_parser.c: -------------------------------------------------------------------------------- 1 | 2 | #line 1 "src/chunked_parser.rl" 3 | /* Copyright (C) agentzh */ 4 | 5 | #ifndef DDEBUG 6 | #define DDEBUG 0 7 | #endif 8 | 9 | #include "ddebug.h" 10 | 11 | #include "chunked_parser.h" 12 | #include "ngx_http_chunkin_util.h" 13 | 14 | #define ngx_chunkin_min(x, y) ((x) < (y) ? (x) : (y)) 15 | 16 | enum { 17 | PRE_TEXT_LEN = 25, 18 | POST_TEXT_LEN = 25 19 | }; 20 | 21 | 22 | #line 20 "src/chunked_parser.rl" 23 | 24 | #line 25 "src/chunked_parser.c" 25 | static const int chunked_start = 1; 26 | static const int chunked_first_final = 43; 27 | static const int chunked_error = 0; 28 | 29 | static const int chunked_en_main = 1; 30 | 31 | 32 | #line 21 "src/chunked_parser.rl" 33 | 34 | 35 | ngx_int_t 36 | ngx_http_chunkin_init_chunked_parser(ngx_http_request_t *r, 37 | ngx_http_chunkin_ctx_t *ctx) 38 | { 39 | int cs; 40 | 41 | 42 | #line 43 "src/chunked_parser.c" 43 | { 44 | cs = chunked_start; 45 | } 46 | 47 | #line 30 "src/chunked_parser.rl" 48 | 49 | ctx->chunks = NULL; 50 | ctx->next_chunk = NULL; 51 | ctx->chunk = NULL; 52 | ctx->chunk_size = 0; 53 | ctx->chunk_size_order = 0; 54 | ctx->chunk_bytes_read = 0; 55 | 56 | ctx->chunks_total_size = 0; 57 | 58 | ctx->parser_state = cs; 59 | 60 | return NGX_OK; 61 | } 62 | 63 | 64 | ngx_int_t 65 | ngx_http_chunkin_run_chunked_parser(ngx_http_request_t *r, 66 | ngx_http_chunkin_ctx_t *ctx, u_char **pos_addr, u_char *last, char *caller_info) 67 | { 68 | int cs = ctx->parser_state; 69 | ngx_connection_t *c = r->connection; 70 | signed char *pos = (signed char *) *pos_addr; 71 | signed char *p = (signed char *) *pos_addr; 72 | signed char *pe = (signed char *) last; 73 | signed char *eof = NULL; 74 | ngx_buf_t *b; 75 | ngx_flag_t done = 0; 76 | ngx_str_t pre, post; 77 | char* err_ctx = ""; 78 | ngx_str_t user_agent = ngx_null_string; 79 | ssize_t rest; 80 | 81 | 82 | #line 236 "src/chunked_parser.rl" 83 | 84 | 85 | 86 | #line 87 "src/chunked_parser.c" 87 | { 88 | short _widec; 89 | if ( p == pe ) 90 | goto _test_eof; 91 | switch ( cs ) 92 | { 93 | case 1: 94 | if ( (*p) == 48 ) 95 | goto tr0; 96 | if ( (*p) < 65 ) { 97 | if ( 49 <= (*p) && (*p) <= 57 ) 98 | goto tr2; 99 | } else if ( (*p) > 70 ) { 100 | if ( 97 <= (*p) && (*p) <= 102 ) 101 | goto tr2; 102 | } else 103 | goto tr2; 104 | goto st0; 105 | tr3: 106 | #line 227 "src/chunked_parser.rl" 107 | { err_ctx = "last_chunk"; } 108 | goto st0; 109 | tr9: 110 | #line 168 "src/chunked_parser.rl" 111 | { err_ctx = "CRLF"; } 112 | #line 227 "src/chunked_parser.rl" 113 | { err_ctx = "last_chunk"; } 114 | goto st0; 115 | tr11: 116 | #line 168 "src/chunked_parser.rl" 117 | { err_ctx = "CRLF"; } 118 | #line 227 "src/chunked_parser.rl" 119 | { err_ctx = "last_chunk"; } 120 | #line 231 "src/chunked_parser.rl" 121 | { err_ctx = "parser"; } 122 | goto st0; 123 | tr13: 124 | #line 168 "src/chunked_parser.rl" 125 | { err_ctx = "CRLF"; } 126 | #line 231 "src/chunked_parser.rl" 127 | { err_ctx = "parser"; } 128 | goto st0; 129 | tr23: 130 | #line 168 "src/chunked_parser.rl" 131 | { err_ctx = "CRLF"; } 132 | goto st0; 133 | tr29: 134 | #line 218 "src/chunked_parser.rl" 135 | { err_ctx = "chunk_size"; } 136 | #line 221 "src/chunked_parser.rl" 137 | { err_ctx = "chunk_ext"; } 138 | goto st0; 139 | tr33: 140 | #line 168 "src/chunked_parser.rl" 141 | { err_ctx = "CRLF"; } 142 | #line 221 "src/chunked_parser.rl" 143 | { err_ctx = "chunk_ext"; } 144 | goto st0; 145 | tr35: 146 | #line 168 "src/chunked_parser.rl" 147 | { err_ctx = "CRLF"; } 148 | #line 221 "src/chunked_parser.rl" 149 | { err_ctx = "chunk_ext"; } 150 | #line 177 "src/chunked_parser.rl" 151 | { err_ctx = "chunk_data"; } 152 | goto st0; 153 | tr37: 154 | #line 177 "src/chunked_parser.rl" 155 | { err_ctx = "chunk_data"; } 156 | goto st0; 157 | tr40: 158 | #line 181 "src/chunked_parser.rl" 159 | { err_ctx = "chunk_data_terminator"; } 160 | goto st0; 161 | tr43: 162 | #line 221 "src/chunked_parser.rl" 163 | { err_ctx = "chunk_ext"; } 164 | goto st0; 165 | #line 166 "src/chunked_parser.c" 166 | st0: 167 | cs = 0; 168 | goto _out; 169 | tr0: 170 | #line 98 "src/chunked_parser.rl" 171 | { 172 | ctx->chunk_bytes_read = 0; 173 | ctx->chunk_size = 0; 174 | ctx->chunk_size_order = 0; 175 | } 176 | #line 104 "src/chunked_parser.rl" 177 | { 178 | ctx->chunk_size <<= 4; 179 | ctx->chunk_size_order++; 180 | if (*p >= 'A' && *p <= 'F') { 181 | ctx->chunk_size |= 10 + *p - 'A'; 182 | } else if (*p >= 'a' && *p <= 'f') { 183 | ctx->chunk_size |= 10 + *p - 'a'; 184 | } else { 185 | ctx->chunk_size |= *p - '0'; 186 | } 187 | 188 | ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, 189 | "chunkin: chunk size: %uz\n", ctx->chunk_size); 190 | } 191 | goto st2; 192 | tr6: 193 | #line 104 "src/chunked_parser.rl" 194 | { 195 | ctx->chunk_size <<= 4; 196 | ctx->chunk_size_order++; 197 | if (*p >= 'A' && *p <= 'F') { 198 | ctx->chunk_size |= 10 + *p - 'A'; 199 | } else if (*p >= 'a' && *p <= 'f') { 200 | ctx->chunk_size |= 10 + *p - 'a'; 201 | } else { 202 | ctx->chunk_size |= *p - '0'; 203 | } 204 | 205 | ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, 206 | "chunkin: chunk size: %uz\n", ctx->chunk_size); 207 | } 208 | goto st2; 209 | st2: 210 | if ( ++p == pe ) 211 | goto _test_eof2; 212 | case 2: 213 | #line 214 "src/chunked_parser.c" 214 | switch( (*p) ) { 215 | case 9: goto st3; 216 | case 13: goto st4; 217 | case 32: goto st3; 218 | case 48: goto tr6; 219 | case 59: goto st7; 220 | } 221 | if ( (*p) < 65 ) { 222 | if ( 49 <= (*p) && (*p) <= 57 ) 223 | goto tr7; 224 | } else if ( (*p) > 70 ) { 225 | if ( 97 <= (*p) && (*p) <= 102 ) 226 | goto tr7; 227 | } else 228 | goto tr7; 229 | goto tr3; 230 | st3: 231 | if ( ++p == pe ) 232 | goto _test_eof3; 233 | case 3: 234 | switch( (*p) ) { 235 | case 9: goto st3; 236 | case 13: goto st4; 237 | case 32: goto st3; 238 | case 59: goto st7; 239 | } 240 | goto tr3; 241 | st4: 242 | if ( ++p == pe ) 243 | goto _test_eof4; 244 | case 4: 245 | if ( (*p) == 10 ) 246 | goto st5; 247 | goto tr9; 248 | st5: 249 | if ( ++p == pe ) 250 | goto _test_eof5; 251 | case 5: 252 | if ( (*p) == 13 ) 253 | goto st6; 254 | goto tr11; 255 | st6: 256 | if ( ++p == pe ) 257 | goto _test_eof6; 258 | case 6: 259 | if ( (*p) == 10 ) 260 | goto tr14; 261 | goto tr13; 262 | tr14: 263 | #line 66 "src/chunked_parser.rl" 264 | { 265 | done = 1; 266 | } 267 | goto st43; 268 | st43: 269 | if ( ++p == pe ) 270 | goto _test_eof43; 271 | case 43: 272 | #line 273 "src/chunked_parser.c" 273 | goto tr13; 274 | st7: 275 | if ( ++p == pe ) 276 | goto _test_eof7; 277 | case 7: 278 | switch( (*p) ) { 279 | case 9: goto st7; 280 | case 32: goto st7; 281 | case 34: goto st0; 282 | case 44: goto st0; 283 | case 47: goto st0; 284 | case 123: goto st0; 285 | case 125: goto st0; 286 | case 127: goto st0; 287 | } 288 | if ( (*p) < 40 ) { 289 | if ( 0 <= (*p) && (*p) <= 31 ) 290 | goto st0; 291 | } else if ( (*p) > 41 ) { 292 | if ( (*p) > 64 ) { 293 | if ( 91 <= (*p) && (*p) <= 93 ) 294 | goto st0; 295 | } else if ( (*p) >= 58 ) 296 | goto st0; 297 | } else 298 | goto st0; 299 | goto st8; 300 | st8: 301 | if ( ++p == pe ) 302 | goto _test_eof8; 303 | case 8: 304 | switch( (*p) ) { 305 | case 9: goto st9; 306 | case 13: goto st4; 307 | case 32: goto st9; 308 | case 34: goto tr3; 309 | case 44: goto tr3; 310 | case 47: goto tr3; 311 | case 59: goto st7; 312 | case 61: goto st10; 313 | case 123: goto tr3; 314 | case 125: goto tr3; 315 | case 127: goto tr3; 316 | } 317 | if ( (*p) < 40 ) { 318 | if ( 0 <= (*p) && (*p) <= 31 ) 319 | goto tr3; 320 | } else if ( (*p) > 41 ) { 321 | if ( (*p) > 64 ) { 322 | if ( 91 <= (*p) && (*p) <= 93 ) 323 | goto tr3; 324 | } else if ( (*p) >= 58 ) 325 | goto tr3; 326 | } else 327 | goto tr3; 328 | goto st8; 329 | st9: 330 | if ( ++p == pe ) 331 | goto _test_eof9; 332 | case 9: 333 | switch( (*p) ) { 334 | case 9: goto st9; 335 | case 13: goto st4; 336 | case 32: goto st9; 337 | case 59: goto st7; 338 | case 61: goto st10; 339 | } 340 | goto tr3; 341 | st10: 342 | if ( ++p == pe ) 343 | goto _test_eof10; 344 | case 10: 345 | switch( (*p) ) { 346 | case 9: goto st10; 347 | case 32: goto st10; 348 | case 34: goto st12; 349 | case 44: goto st0; 350 | case 47: goto st0; 351 | case 123: goto st0; 352 | case 125: goto st0; 353 | case 127: goto st0; 354 | } 355 | if ( (*p) < 40 ) { 356 | if ( 0 <= (*p) && (*p) <= 31 ) 357 | goto st0; 358 | } else if ( (*p) > 41 ) { 359 | if ( (*p) > 64 ) { 360 | if ( 91 <= (*p) && (*p) <= 93 ) 361 | goto st0; 362 | } else if ( (*p) >= 58 ) 363 | goto st0; 364 | } else 365 | goto st0; 366 | goto st11; 367 | st11: 368 | if ( ++p == pe ) 369 | goto _test_eof11; 370 | case 11: 371 | switch( (*p) ) { 372 | case 9: goto st3; 373 | case 13: goto st4; 374 | case 32: goto st3; 375 | case 34: goto tr3; 376 | case 44: goto tr3; 377 | case 47: goto tr3; 378 | case 59: goto st7; 379 | case 123: goto tr3; 380 | case 125: goto tr3; 381 | case 127: goto tr3; 382 | } 383 | if ( (*p) < 40 ) { 384 | if ( 0 <= (*p) && (*p) <= 31 ) 385 | goto tr3; 386 | } else if ( (*p) > 41 ) { 387 | if ( (*p) > 64 ) { 388 | if ( 91 <= (*p) && (*p) <= 93 ) 389 | goto tr3; 390 | } else if ( (*p) >= 58 ) 391 | goto tr3; 392 | } else 393 | goto tr3; 394 | goto st11; 395 | st12: 396 | if ( ++p == pe ) 397 | goto _test_eof12; 398 | case 12: 399 | switch( (*p) ) { 400 | case 34: goto st3; 401 | case 92: goto st13; 402 | case 127: goto st0; 403 | } 404 | if ( (*p) > 8 ) { 405 | if ( 10 <= (*p) && (*p) <= 31 ) 406 | goto st0; 407 | } else if ( (*p) >= 0 ) 408 | goto st0; 409 | goto st12; 410 | st13: 411 | if ( ++p == pe ) 412 | goto _test_eof13; 413 | case 13: 414 | switch( (*p) ) { 415 | case 13: goto st14; 416 | case 34: goto st15; 417 | case 92: goto st13; 418 | } 419 | goto st12; 420 | st14: 421 | if ( ++p == pe ) 422 | goto _test_eof14; 423 | case 14: 424 | switch( (*p) ) { 425 | case 34: goto st3; 426 | case 92: goto st13; 427 | case 127: goto tr23; 428 | } 429 | if ( (*p) > 8 ) { 430 | if ( 10 <= (*p) && (*p) <= 31 ) 431 | goto tr23; 432 | } else if ( (*p) >= 0 ) 433 | goto tr23; 434 | goto st12; 435 | st15: 436 | if ( ++p == pe ) 437 | goto _test_eof15; 438 | case 15: 439 | switch( (*p) ) { 440 | case 9: goto st15; 441 | case 13: goto st4; 442 | case 32: goto st15; 443 | case 34: goto st3; 444 | case 59: goto st16; 445 | case 92: goto st13; 446 | case 127: goto tr3; 447 | } 448 | if ( 0 <= (*p) && (*p) <= 31 ) 449 | goto tr3; 450 | goto st12; 451 | st16: 452 | if ( ++p == pe ) 453 | goto _test_eof16; 454 | case 16: 455 | switch( (*p) ) { 456 | case 9: goto st16; 457 | case 32: goto st16; 458 | case 34: goto st3; 459 | case 44: goto st12; 460 | case 47: goto st12; 461 | case 92: goto st13; 462 | case 123: goto st12; 463 | case 125: goto st12; 464 | case 127: goto st0; 465 | } 466 | if ( (*p) < 40 ) { 467 | if ( 0 <= (*p) && (*p) <= 31 ) 468 | goto st0; 469 | } else if ( (*p) > 41 ) { 470 | if ( (*p) > 64 ) { 471 | if ( 91 <= (*p) && (*p) <= 93 ) 472 | goto st12; 473 | } else if ( (*p) >= 58 ) 474 | goto st12; 475 | } else 476 | goto st12; 477 | goto st17; 478 | st17: 479 | if ( ++p == pe ) 480 | goto _test_eof17; 481 | case 17: 482 | switch( (*p) ) { 483 | case 9: goto st18; 484 | case 13: goto st4; 485 | case 32: goto st18; 486 | case 34: goto st3; 487 | case 44: goto st12; 488 | case 47: goto st12; 489 | case 59: goto st16; 490 | case 61: goto st19; 491 | case 92: goto st13; 492 | case 123: goto st12; 493 | case 125: goto st12; 494 | case 127: goto tr3; 495 | } 496 | if ( (*p) < 40 ) { 497 | if ( 0 <= (*p) && (*p) <= 31 ) 498 | goto tr3; 499 | } else if ( (*p) > 41 ) { 500 | if ( (*p) > 64 ) { 501 | if ( 91 <= (*p) && (*p) <= 93 ) 502 | goto st12; 503 | } else if ( (*p) >= 58 ) 504 | goto st12; 505 | } else 506 | goto st12; 507 | goto st17; 508 | st18: 509 | if ( ++p == pe ) 510 | goto _test_eof18; 511 | case 18: 512 | switch( (*p) ) { 513 | case 9: goto st18; 514 | case 13: goto st4; 515 | case 32: goto st18; 516 | case 34: goto st3; 517 | case 59: goto st16; 518 | case 61: goto st19; 519 | case 92: goto st13; 520 | case 127: goto tr3; 521 | } 522 | if ( 0 <= (*p) && (*p) <= 31 ) 523 | goto tr3; 524 | goto st12; 525 | st19: 526 | if ( ++p == pe ) 527 | goto _test_eof19; 528 | case 19: 529 | switch( (*p) ) { 530 | case 9: goto st19; 531 | case 32: goto st19; 532 | case 34: goto st15; 533 | case 44: goto st12; 534 | case 47: goto st12; 535 | case 92: goto st13; 536 | case 123: goto st12; 537 | case 125: goto st12; 538 | case 127: goto st0; 539 | } 540 | if ( (*p) < 40 ) { 541 | if ( 0 <= (*p) && (*p) <= 31 ) 542 | goto st0; 543 | } else if ( (*p) > 41 ) { 544 | if ( (*p) > 64 ) { 545 | if ( 91 <= (*p) && (*p) <= 93 ) 546 | goto st12; 547 | } else if ( (*p) >= 58 ) 548 | goto st12; 549 | } else 550 | goto st12; 551 | goto st20; 552 | st20: 553 | if ( ++p == pe ) 554 | goto _test_eof20; 555 | case 20: 556 | switch( (*p) ) { 557 | case 9: goto st15; 558 | case 13: goto st4; 559 | case 32: goto st15; 560 | case 34: goto st3; 561 | case 44: goto st12; 562 | case 47: goto st12; 563 | case 59: goto st16; 564 | case 92: goto st13; 565 | case 123: goto st12; 566 | case 125: goto st12; 567 | case 127: goto tr3; 568 | } 569 | if ( (*p) < 40 ) { 570 | if ( 0 <= (*p) && (*p) <= 31 ) 571 | goto tr3; 572 | } else if ( (*p) > 41 ) { 573 | if ( (*p) > 64 ) { 574 | if ( 91 <= (*p) && (*p) <= 93 ) 575 | goto st12; 576 | } else if ( (*p) >= 58 ) 577 | goto st12; 578 | } else 579 | goto st12; 580 | goto st20; 581 | tr2: 582 | #line 98 "src/chunked_parser.rl" 583 | { 584 | ctx->chunk_bytes_read = 0; 585 | ctx->chunk_size = 0; 586 | ctx->chunk_size_order = 0; 587 | } 588 | #line 104 "src/chunked_parser.rl" 589 | { 590 | ctx->chunk_size <<= 4; 591 | ctx->chunk_size_order++; 592 | if (*p >= 'A' && *p <= 'F') { 593 | ctx->chunk_size |= 10 + *p - 'A'; 594 | } else if (*p >= 'a' && *p <= 'f') { 595 | ctx->chunk_size |= 10 + *p - 'a'; 596 | } else { 597 | ctx->chunk_size |= *p - '0'; 598 | } 599 | 600 | ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, 601 | "chunkin: chunk size: %uz\n", ctx->chunk_size); 602 | } 603 | goto st21; 604 | tr7: 605 | #line 104 "src/chunked_parser.rl" 606 | { 607 | ctx->chunk_size <<= 4; 608 | ctx->chunk_size_order++; 609 | if (*p >= 'A' && *p <= 'F') { 610 | ctx->chunk_size |= 10 + *p - 'A'; 611 | } else if (*p >= 'a' && *p <= 'f') { 612 | ctx->chunk_size |= 10 + *p - 'a'; 613 | } else { 614 | ctx->chunk_size |= *p - '0'; 615 | } 616 | 617 | ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, 618 | "chunkin: chunk size: %uz\n", ctx->chunk_size); 619 | } 620 | goto st21; 621 | st21: 622 | if ( ++p == pe ) 623 | goto _test_eof21; 624 | case 21: 625 | #line 626 "src/chunked_parser.c" 626 | switch( (*p) ) { 627 | case 9: goto st22; 628 | case 13: goto st23; 629 | case 32: goto st22; 630 | case 59: goto st28; 631 | } 632 | if ( (*p) < 65 ) { 633 | if ( 48 <= (*p) && (*p) <= 57 ) 634 | goto tr7; 635 | } else if ( (*p) > 70 ) { 636 | if ( 97 <= (*p) && (*p) <= 102 ) 637 | goto tr7; 638 | } else 639 | goto tr7; 640 | goto tr29; 641 | st22: 642 | if ( ++p == pe ) 643 | goto _test_eof22; 644 | case 22: 645 | switch( (*p) ) { 646 | case 9: goto st22; 647 | case 13: goto st23; 648 | case 32: goto st22; 649 | case 59: goto st28; 650 | } 651 | goto tr29; 652 | st23: 653 | if ( ++p == pe ) 654 | goto _test_eof23; 655 | case 23: 656 | if ( (*p) == 10 ) 657 | goto st24; 658 | goto tr33; 659 | st24: 660 | if ( ++p == pe ) 661 | goto _test_eof24; 662 | case 24: 663 | _widec = (*p); 664 | _widec = (short)(128 + ((*p) - -128)); 665 | if ( 666 | #line 70 "src/chunked_parser.rl" 667 | 668 | ctx->chunk_bytes_read < ctx->chunk_size 669 | ) _widec += 256; 670 | if ( 384 <= _widec && _widec <= 639 ) 671 | goto tr36; 672 | goto tr35; 673 | tr36: 674 | #line 119 "src/chunked_parser.rl" 675 | { 676 | ctx->chunk = ngx_http_chunkin_get_buf(r->pool, ctx); 677 | 678 | ctx->chunks_count++; 679 | 680 | if (ctx->chunks) { 681 | *ctx->next_chunk = ctx->chunk; 682 | } else { 683 | ctx->chunks = ctx->chunk; 684 | } 685 | 686 | ctx->next_chunk = &ctx->chunk->next; 687 | 688 | b = ctx->chunk->buf; 689 | 690 | b->last = b->pos = (u_char *) p; 691 | b->memory = 1; 692 | } 693 | #line 74 "src/chunked_parser.rl" 694 | { 695 | /* optimization for buffered chunk data */ 696 | 697 | rest = ngx_chunkin_min( 698 | (ssize_t)ctx->chunk_size - (ssize_t)ctx->chunk_bytes_read, 699 | (ssize_t)(pe - p)); 700 | 701 | dd("moving %d, chunk size %d, read %d, rest %d", 702 | (int)rest, 703 | (int)ctx->chunk_size, 704 | (int)ctx->chunk_bytes_read, 705 | (int)rest); 706 | 707 | ctx->chunk_bytes_read += rest; 708 | p += rest - 1; 709 | ctx->chunk->buf->last = (u_char *)p + 1; 710 | ctx->chunks_total_size += rest; 711 | 712 | /* dd("bytes read: %d (char '%c', bytes read %d, chunk size %d)", ctx->chunk->buf->last - ctx->chunk->buf->pos, *p, ctx->chunk_bytes_read, ctx->chunk_size); */ 713 | ngx_log_debug2(NGX_LOG_DEBUG_HTTP, c->log, 0, 714 | "chunkin: data bytes read: %uz (char: \"%c\")\n", 715 | ctx->chunk_bytes_read, *p); 716 | } 717 | goto st25; 718 | tr39: 719 | #line 74 "src/chunked_parser.rl" 720 | { 721 | /* optimization for buffered chunk data */ 722 | 723 | rest = ngx_chunkin_min( 724 | (ssize_t)ctx->chunk_size - (ssize_t)ctx->chunk_bytes_read, 725 | (ssize_t)(pe - p)); 726 | 727 | dd("moving %d, chunk size %d, read %d, rest %d", 728 | (int)rest, 729 | (int)ctx->chunk_size, 730 | (int)ctx->chunk_bytes_read, 731 | (int)rest); 732 | 733 | ctx->chunk_bytes_read += rest; 734 | p += rest - 1; 735 | ctx->chunk->buf->last = (u_char *)p + 1; 736 | ctx->chunks_total_size += rest; 737 | 738 | /* dd("bytes read: %d (char '%c', bytes read %d, chunk size %d)", ctx->chunk->buf->last - ctx->chunk->buf->pos, *p, ctx->chunk_bytes_read, ctx->chunk_size); */ 739 | ngx_log_debug2(NGX_LOG_DEBUG_HTTP, c->log, 0, 740 | "chunkin: data bytes read: %uz (char: \"%c\")\n", 741 | ctx->chunk_bytes_read, *p); 742 | } 743 | goto st25; 744 | st25: 745 | if ( ++p == pe ) 746 | goto _test_eof25; 747 | case 25: 748 | #line 749 "src/chunked_parser.c" 749 | _widec = (*p); 750 | if ( (*p) < 13 ) { 751 | if ( (*p) <= 12 ) { 752 | _widec = (short)(128 + ((*p) - -128)); 753 | if ( 754 | #line 70 "src/chunked_parser.rl" 755 | 756 | ctx->chunk_bytes_read < ctx->chunk_size 757 | ) _widec += 256; 758 | } 759 | } else if ( (*p) > 13 ) { 760 | if ( 14 <= (*p) ) 761 | { _widec = (short)(128 + ((*p) - -128)); 762 | if ( 763 | #line 70 "src/chunked_parser.rl" 764 | 765 | ctx->chunk_bytes_read < ctx->chunk_size 766 | ) _widec += 256; 767 | } 768 | } else { 769 | _widec = (short)(128 + ((*p) - -128)); 770 | if ( 771 | #line 70 "src/chunked_parser.rl" 772 | 773 | ctx->chunk_bytes_read < ctx->chunk_size 774 | ) _widec += 256; 775 | } 776 | if ( _widec == 269 ) 777 | goto st26; 778 | if ( 384 <= _widec && _widec <= 639 ) 779 | goto tr39; 780 | goto tr37; 781 | st26: 782 | if ( ++p == pe ) 783 | goto _test_eof26; 784 | case 26: 785 | if ( (*p) == 10 ) 786 | goto tr41; 787 | goto tr40; 788 | tr41: 789 | #line 138 "src/chunked_parser.rl" 790 | { 791 | if (ctx->chunk_bytes_read != ctx->chunk_size) { 792 | ngx_log_error(NGX_LOG_ERR, c->log, 0, 793 | "ERROR: chunk size not met: " 794 | "%uz != %uz\n", ctx->chunk_bytes_read, 795 | ctx->chunk_size); 796 | *pos_addr = (u_char*) p; 797 | ctx->parser_state = chunked_error; 798 | return NGX_ERROR; 799 | } 800 | 801 | if (ctx->chunk_size == 0) { 802 | /* remove the current chunk */ 803 | ctx->chunk->next = ctx->free_bufs; 804 | ctx->free_bufs = ctx->chunk; 805 | ctx->chunk = ctx->last_complete_chunk; 806 | if (ctx->last_complete_chunk) { 807 | ctx->last_complete_chunk->next = NULL; 808 | } else { 809 | ctx->chunks = NULL; 810 | } 811 | } else { 812 | ctx->last_complete_chunk = ctx->chunk; 813 | } 814 | } 815 | goto st27; 816 | st27: 817 | if ( ++p == pe ) 818 | goto _test_eof27; 819 | case 27: 820 | #line 821 "src/chunked_parser.c" 821 | if ( (*p) == 48 ) 822 | goto tr0; 823 | if ( (*p) < 65 ) { 824 | if ( 49 <= (*p) && (*p) <= 57 ) 825 | goto tr2; 826 | } else if ( (*p) > 70 ) { 827 | if ( 97 <= (*p) && (*p) <= 102 ) 828 | goto tr2; 829 | } else 830 | goto tr2; 831 | goto tr40; 832 | st28: 833 | if ( ++p == pe ) 834 | goto _test_eof28; 835 | case 28: 836 | switch( (*p) ) { 837 | case 9: goto st28; 838 | case 32: goto st28; 839 | case 34: goto st0; 840 | case 44: goto st0; 841 | case 47: goto st0; 842 | case 123: goto st0; 843 | case 125: goto st0; 844 | case 127: goto st0; 845 | } 846 | if ( (*p) < 40 ) { 847 | if ( 0 <= (*p) && (*p) <= 31 ) 848 | goto st0; 849 | } else if ( (*p) > 41 ) { 850 | if ( (*p) > 64 ) { 851 | if ( 91 <= (*p) && (*p) <= 93 ) 852 | goto st0; 853 | } else if ( (*p) >= 58 ) 854 | goto st0; 855 | } else 856 | goto st0; 857 | goto st29; 858 | st29: 859 | if ( ++p == pe ) 860 | goto _test_eof29; 861 | case 29: 862 | switch( (*p) ) { 863 | case 9: goto st30; 864 | case 13: goto st23; 865 | case 32: goto st30; 866 | case 34: goto tr43; 867 | case 44: goto tr43; 868 | case 47: goto tr43; 869 | case 59: goto st28; 870 | case 61: goto st31; 871 | case 123: goto tr43; 872 | case 125: goto tr43; 873 | case 127: goto tr43; 874 | } 875 | if ( (*p) < 40 ) { 876 | if ( 0 <= (*p) && (*p) <= 31 ) 877 | goto tr43; 878 | } else if ( (*p) > 41 ) { 879 | if ( (*p) > 64 ) { 880 | if ( 91 <= (*p) && (*p) <= 93 ) 881 | goto tr43; 882 | } else if ( (*p) >= 58 ) 883 | goto tr43; 884 | } else 885 | goto tr43; 886 | goto st29; 887 | st30: 888 | if ( ++p == pe ) 889 | goto _test_eof30; 890 | case 30: 891 | switch( (*p) ) { 892 | case 9: goto st30; 893 | case 13: goto st23; 894 | case 32: goto st30; 895 | case 59: goto st28; 896 | case 61: goto st31; 897 | } 898 | goto tr43; 899 | st31: 900 | if ( ++p == pe ) 901 | goto _test_eof31; 902 | case 31: 903 | switch( (*p) ) { 904 | case 9: goto st31; 905 | case 32: goto st31; 906 | case 34: goto st34; 907 | case 44: goto st0; 908 | case 47: goto st0; 909 | case 123: goto st0; 910 | case 125: goto st0; 911 | case 127: goto st0; 912 | } 913 | if ( (*p) < 40 ) { 914 | if ( 0 <= (*p) && (*p) <= 31 ) 915 | goto st0; 916 | } else if ( (*p) > 41 ) { 917 | if ( (*p) > 64 ) { 918 | if ( 91 <= (*p) && (*p) <= 93 ) 919 | goto st0; 920 | } else if ( (*p) >= 58 ) 921 | goto st0; 922 | } else 923 | goto st0; 924 | goto st32; 925 | st32: 926 | if ( ++p == pe ) 927 | goto _test_eof32; 928 | case 32: 929 | switch( (*p) ) { 930 | case 9: goto st33; 931 | case 13: goto st23; 932 | case 32: goto st33; 933 | case 34: goto tr43; 934 | case 44: goto tr43; 935 | case 47: goto tr43; 936 | case 59: goto st28; 937 | case 123: goto tr43; 938 | case 125: goto tr43; 939 | case 127: goto tr43; 940 | } 941 | if ( (*p) < 40 ) { 942 | if ( 0 <= (*p) && (*p) <= 31 ) 943 | goto tr43; 944 | } else if ( (*p) > 41 ) { 945 | if ( (*p) > 64 ) { 946 | if ( 91 <= (*p) && (*p) <= 93 ) 947 | goto tr43; 948 | } else if ( (*p) >= 58 ) 949 | goto tr43; 950 | } else 951 | goto tr43; 952 | goto st32; 953 | st33: 954 | if ( ++p == pe ) 955 | goto _test_eof33; 956 | case 33: 957 | switch( (*p) ) { 958 | case 9: goto st33; 959 | case 13: goto st23; 960 | case 32: goto st33; 961 | case 59: goto st28; 962 | } 963 | goto tr43; 964 | st34: 965 | if ( ++p == pe ) 966 | goto _test_eof34; 967 | case 34: 968 | switch( (*p) ) { 969 | case 34: goto st33; 970 | case 92: goto st35; 971 | case 127: goto st0; 972 | } 973 | if ( (*p) > 8 ) { 974 | if ( 10 <= (*p) && (*p) <= 31 ) 975 | goto st0; 976 | } else if ( (*p) >= 0 ) 977 | goto st0; 978 | goto st34; 979 | st35: 980 | if ( ++p == pe ) 981 | goto _test_eof35; 982 | case 35: 983 | switch( (*p) ) { 984 | case 13: goto st36; 985 | case 34: goto st37; 986 | case 92: goto st35; 987 | } 988 | goto st34; 989 | st36: 990 | if ( ++p == pe ) 991 | goto _test_eof36; 992 | case 36: 993 | switch( (*p) ) { 994 | case 34: goto st33; 995 | case 92: goto st35; 996 | case 127: goto tr23; 997 | } 998 | if ( (*p) > 8 ) { 999 | if ( 10 <= (*p) && (*p) <= 31 ) 1000 | goto tr23; 1001 | } else if ( (*p) >= 0 ) 1002 | goto tr23; 1003 | goto st34; 1004 | st37: 1005 | if ( ++p == pe ) 1006 | goto _test_eof37; 1007 | case 37: 1008 | switch( (*p) ) { 1009 | case 9: goto st37; 1010 | case 13: goto st23; 1011 | case 32: goto st37; 1012 | case 34: goto st33; 1013 | case 59: goto st38; 1014 | case 92: goto st35; 1015 | case 127: goto tr43; 1016 | } 1017 | if ( 0 <= (*p) && (*p) <= 31 ) 1018 | goto tr43; 1019 | goto st34; 1020 | st38: 1021 | if ( ++p == pe ) 1022 | goto _test_eof38; 1023 | case 38: 1024 | switch( (*p) ) { 1025 | case 9: goto st38; 1026 | case 32: goto st38; 1027 | case 34: goto st33; 1028 | case 44: goto st34; 1029 | case 47: goto st34; 1030 | case 92: goto st35; 1031 | case 123: goto st34; 1032 | case 125: goto st34; 1033 | case 127: goto st0; 1034 | } 1035 | if ( (*p) < 40 ) { 1036 | if ( 0 <= (*p) && (*p) <= 31 ) 1037 | goto st0; 1038 | } else if ( (*p) > 41 ) { 1039 | if ( (*p) > 64 ) { 1040 | if ( 91 <= (*p) && (*p) <= 93 ) 1041 | goto st34; 1042 | } else if ( (*p) >= 58 ) 1043 | goto st34; 1044 | } else 1045 | goto st34; 1046 | goto st39; 1047 | st39: 1048 | if ( ++p == pe ) 1049 | goto _test_eof39; 1050 | case 39: 1051 | switch( (*p) ) { 1052 | case 9: goto st40; 1053 | case 13: goto st23; 1054 | case 32: goto st40; 1055 | case 34: goto st33; 1056 | case 44: goto st34; 1057 | case 47: goto st34; 1058 | case 59: goto st38; 1059 | case 61: goto st41; 1060 | case 92: goto st35; 1061 | case 123: goto st34; 1062 | case 125: goto st34; 1063 | case 127: goto tr43; 1064 | } 1065 | if ( (*p) < 40 ) { 1066 | if ( 0 <= (*p) && (*p) <= 31 ) 1067 | goto tr43; 1068 | } else if ( (*p) > 41 ) { 1069 | if ( (*p) > 64 ) { 1070 | if ( 91 <= (*p) && (*p) <= 93 ) 1071 | goto st34; 1072 | } else if ( (*p) >= 58 ) 1073 | goto st34; 1074 | } else 1075 | goto st34; 1076 | goto st39; 1077 | st40: 1078 | if ( ++p == pe ) 1079 | goto _test_eof40; 1080 | case 40: 1081 | switch( (*p) ) { 1082 | case 9: goto st40; 1083 | case 13: goto st23; 1084 | case 32: goto st40; 1085 | case 34: goto st33; 1086 | case 59: goto st38; 1087 | case 61: goto st41; 1088 | case 92: goto st35; 1089 | case 127: goto tr43; 1090 | } 1091 | if ( 0 <= (*p) && (*p) <= 31 ) 1092 | goto tr43; 1093 | goto st34; 1094 | st41: 1095 | if ( ++p == pe ) 1096 | goto _test_eof41; 1097 | case 41: 1098 | switch( (*p) ) { 1099 | case 9: goto st41; 1100 | case 32: goto st41; 1101 | case 34: goto st37; 1102 | case 44: goto st34; 1103 | case 47: goto st34; 1104 | case 92: goto st35; 1105 | case 123: goto st34; 1106 | case 125: goto st34; 1107 | case 127: goto st0; 1108 | } 1109 | if ( (*p) < 40 ) { 1110 | if ( 0 <= (*p) && (*p) <= 31 ) 1111 | goto st0; 1112 | } else if ( (*p) > 41 ) { 1113 | if ( (*p) > 64 ) { 1114 | if ( 91 <= (*p) && (*p) <= 93 ) 1115 | goto st34; 1116 | } else if ( (*p) >= 58 ) 1117 | goto st34; 1118 | } else 1119 | goto st34; 1120 | goto st42; 1121 | st42: 1122 | if ( ++p == pe ) 1123 | goto _test_eof42; 1124 | case 42: 1125 | switch( (*p) ) { 1126 | case 9: goto st37; 1127 | case 13: goto st23; 1128 | case 32: goto st37; 1129 | case 34: goto st33; 1130 | case 44: goto st34; 1131 | case 47: goto st34; 1132 | case 59: goto st38; 1133 | case 92: goto st35; 1134 | case 123: goto st34; 1135 | case 125: goto st34; 1136 | case 127: goto tr43; 1137 | } 1138 | if ( (*p) < 40 ) { 1139 | if ( 0 <= (*p) && (*p) <= 31 ) 1140 | goto tr43; 1141 | } else if ( (*p) > 41 ) { 1142 | if ( (*p) > 64 ) { 1143 | if ( 91 <= (*p) && (*p) <= 93 ) 1144 | goto st34; 1145 | } else if ( (*p) >= 58 ) 1146 | goto st34; 1147 | } else 1148 | goto st34; 1149 | goto st42; 1150 | } 1151 | _test_eof2: cs = 2; goto _test_eof; 1152 | _test_eof3: cs = 3; goto _test_eof; 1153 | _test_eof4: cs = 4; goto _test_eof; 1154 | _test_eof5: cs = 5; goto _test_eof; 1155 | _test_eof6: cs = 6; goto _test_eof; 1156 | _test_eof43: cs = 43; goto _test_eof; 1157 | _test_eof7: cs = 7; goto _test_eof; 1158 | _test_eof8: cs = 8; goto _test_eof; 1159 | _test_eof9: cs = 9; goto _test_eof; 1160 | _test_eof10: cs = 10; goto _test_eof; 1161 | _test_eof11: cs = 11; goto _test_eof; 1162 | _test_eof12: cs = 12; goto _test_eof; 1163 | _test_eof13: cs = 13; goto _test_eof; 1164 | _test_eof14: cs = 14; goto _test_eof; 1165 | _test_eof15: cs = 15; goto _test_eof; 1166 | _test_eof16: cs = 16; goto _test_eof; 1167 | _test_eof17: cs = 17; goto _test_eof; 1168 | _test_eof18: cs = 18; goto _test_eof; 1169 | _test_eof19: cs = 19; goto _test_eof; 1170 | _test_eof20: cs = 20; goto _test_eof; 1171 | _test_eof21: cs = 21; goto _test_eof; 1172 | _test_eof22: cs = 22; goto _test_eof; 1173 | _test_eof23: cs = 23; goto _test_eof; 1174 | _test_eof24: cs = 24; goto _test_eof; 1175 | _test_eof25: cs = 25; goto _test_eof; 1176 | _test_eof26: cs = 26; goto _test_eof; 1177 | _test_eof27: cs = 27; goto _test_eof; 1178 | _test_eof28: cs = 28; goto _test_eof; 1179 | _test_eof29: cs = 29; goto _test_eof; 1180 | _test_eof30: cs = 30; goto _test_eof; 1181 | _test_eof31: cs = 31; goto _test_eof; 1182 | _test_eof32: cs = 32; goto _test_eof; 1183 | _test_eof33: cs = 33; goto _test_eof; 1184 | _test_eof34: cs = 34; goto _test_eof; 1185 | _test_eof35: cs = 35; goto _test_eof; 1186 | _test_eof36: cs = 36; goto _test_eof; 1187 | _test_eof37: cs = 37; goto _test_eof; 1188 | _test_eof38: cs = 38; goto _test_eof; 1189 | _test_eof39: cs = 39; goto _test_eof; 1190 | _test_eof40: cs = 40; goto _test_eof; 1191 | _test_eof41: cs = 41; goto _test_eof; 1192 | _test_eof42: cs = 42; goto _test_eof; 1193 | 1194 | _test_eof: {} 1195 | if ( p == eof ) 1196 | { 1197 | switch ( cs ) { 1198 | case 14: 1199 | case 36: 1200 | #line 168 "src/chunked_parser.rl" 1201 | { err_ctx = "CRLF"; } 1202 | break; 1203 | case 25: 1204 | #line 177 "src/chunked_parser.rl" 1205 | { err_ctx = "chunk_data"; } 1206 | break; 1207 | case 26: 1208 | case 27: 1209 | #line 181 "src/chunked_parser.rl" 1210 | { err_ctx = "chunk_data_terminator"; } 1211 | break; 1212 | case 29: 1213 | case 30: 1214 | case 32: 1215 | case 33: 1216 | case 37: 1217 | case 39: 1218 | case 40: 1219 | case 42: 1220 | #line 221 "src/chunked_parser.rl" 1221 | { err_ctx = "chunk_ext"; } 1222 | break; 1223 | case 2: 1224 | case 3: 1225 | case 8: 1226 | case 9: 1227 | case 11: 1228 | case 15: 1229 | case 17: 1230 | case 18: 1231 | case 20: 1232 | #line 227 "src/chunked_parser.rl" 1233 | { err_ctx = "last_chunk"; } 1234 | break; 1235 | case 23: 1236 | #line 168 "src/chunked_parser.rl" 1237 | { err_ctx = "CRLF"; } 1238 | #line 221 "src/chunked_parser.rl" 1239 | { err_ctx = "chunk_ext"; } 1240 | break; 1241 | case 4: 1242 | #line 168 "src/chunked_parser.rl" 1243 | { err_ctx = "CRLF"; } 1244 | #line 227 "src/chunked_parser.rl" 1245 | { err_ctx = "last_chunk"; } 1246 | break; 1247 | case 6: 1248 | #line 168 "src/chunked_parser.rl" 1249 | { err_ctx = "CRLF"; } 1250 | #line 231 "src/chunked_parser.rl" 1251 | { err_ctx = "parser"; } 1252 | break; 1253 | case 21: 1254 | case 22: 1255 | #line 218 "src/chunked_parser.rl" 1256 | { err_ctx = "chunk_size"; } 1257 | #line 221 "src/chunked_parser.rl" 1258 | { err_ctx = "chunk_ext"; } 1259 | break; 1260 | case 24: 1261 | #line 168 "src/chunked_parser.rl" 1262 | { err_ctx = "CRLF"; } 1263 | #line 221 "src/chunked_parser.rl" 1264 | { err_ctx = "chunk_ext"; } 1265 | #line 177 "src/chunked_parser.rl" 1266 | { err_ctx = "chunk_data"; } 1267 | break; 1268 | case 5: 1269 | #line 168 "src/chunked_parser.rl" 1270 | { err_ctx = "CRLF"; } 1271 | #line 227 "src/chunked_parser.rl" 1272 | { err_ctx = "last_chunk"; } 1273 | #line 231 "src/chunked_parser.rl" 1274 | { err_ctx = "parser"; } 1275 | break; 1276 | #line 1277 "src/chunked_parser.c" 1277 | } 1278 | } 1279 | 1280 | _out: {} 1281 | } 1282 | 1283 | #line 239 "src/chunked_parser.rl" 1284 | 1285 | ctx->parser_state = cs; 1286 | 1287 | *pos_addr = (u_char *) p; 1288 | 1289 | if (p != pe) { 1290 | dd("ASSERTION FAILED: p != pe"); 1291 | } 1292 | 1293 | if (done) { 1294 | return NGX_OK; 1295 | } 1296 | 1297 | if (cs == chunked_error) { 1298 | 1299 | #if EXTENDED_DEBUG 1300 | 1301 | ngx_str_t headers_buf, preread_buf; 1302 | 1303 | #endif 1304 | 1305 | for (post.data = (u_char *) p, post.len = 0; 1306 | post.data + post.len != (u_char *) pe; post.len++) 1307 | { 1308 | if (post.len >= POST_TEXT_LEN) { 1309 | break; 1310 | } 1311 | } 1312 | 1313 | for (pre.data = (u_char *) p, pre.len = 0; 1314 | pre.data != (u_char *) pos; pre.data--, pre.len++) 1315 | { 1316 | if (pre.len >= PRE_TEXT_LEN) { 1317 | break; 1318 | } 1319 | } 1320 | 1321 | if (r->headers_in.user_agent) { 1322 | user_agent = r->headers_in.user_agent->value; 1323 | } 1324 | 1325 | #if EXTENDED_DEBUG 1326 | 1327 | headers_buf.data = r->header_in->start; 1328 | headers_buf.len = ctx->saved_header_in_pos - r->header_in->start; 1329 | 1330 | if (strcmp(caller_info, "preread") == 0) { 1331 | preread_buf.data = (u_char *) pos; 1332 | preread_buf.len = pe - pos; 1333 | 1334 | } else { 1335 | preread_buf.data = ctx->saved_header_in_pos; 1336 | preread_buf.len = r->header_in->pos - ctx->saved_header_in_pos; 1337 | } 1338 | 1339 | #endif 1340 | 1341 | ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, 1342 | "bad chunked body (buf size %O, buf offset %O, " 1343 | "total decoded %uz, chunks count %d, " 1344 | "chunk size %uz, chunk data read %uz, " 1345 | "total to disk %uz, " 1346 | "raw body size %O, caller \"%s\", " 1347 | "plain_http %d, " 1348 | 1349 | #if (NGX_HTTP_SSL) 1350 | 1351 | "ssl %d, " 1352 | #endif 1353 | 1354 | "keepalive %d, err ctx \"%s\", " 1355 | "ctx ref count %ud, user agent \"%V\", " 1356 | 1357 | #if EXTENDED_DEBUG 1358 | 1359 | "headers \"%V\", preread \"%V\", " 1360 | 1361 | #endif 1362 | 1363 | "at char '%c' (%d), " 1364 | "near \"%V <-- HERE %V\", marked by \" <-- HERE \").\n", 1365 | (off_t) (pe - pos), (off_t) (p - pos), 1366 | ctx->chunks_total_size, ctx->chunks_count, 1367 | ctx->chunk_size, ctx->chunk_bytes_read, 1368 | ctx->chunks_written_size, 1369 | (off_t) ctx->raw_body_size, caller_info, 1370 | (int) r->plain_http, 1371 | 1372 | #if (NGX_HTTP_SSL) 1373 | 1374 | r->connection->ssl ? 1 : 0, 1375 | 1376 | #endif 1377 | 1378 | (int) r->keepalive, err_ctx, 1379 | ctx->count, &user_agent, 1380 | 1381 | #if EXTENDED_DEBUG 1382 | 1383 | &headers_buf, &preread_buf, 1384 | 1385 | #endif 1386 | 1387 | *p, *p, 1388 | &pre, &post); 1389 | 1390 | return NGX_ERROR; 1391 | } 1392 | 1393 | return NGX_AGAIN; 1394 | } 1395 | 1396 | -------------------------------------------------------------------------------- /src/chunked_parser.h: -------------------------------------------------------------------------------- 1 | #ifndef CHUNKIN_CHUNKED_PARSER_H 2 | #define CHUNKIN_CHUNKED_PARSER_H 3 | 4 | #include "ngx_http_chunkin_filter_module.h" 5 | 6 | ngx_int_t ngx_http_chunkin_init_chunked_parser(ngx_http_request_t *r, 7 | ngx_http_chunkin_ctx_t *ctx); 8 | 9 | ngx_int_t ngx_http_chunkin_run_chunked_parser(ngx_http_request_t *r, 10 | ngx_http_chunkin_ctx_t *ctx, u_char **pos_addr, u_char *last, 11 | char *caller_info); 12 | 13 | #endif /* CHUNKIN_CHUNKED_PARSER_H */ 14 | 15 | -------------------------------------------------------------------------------- /src/chunked_parser.rl: -------------------------------------------------------------------------------- 1 | /* Copyright (C) agentzh */ 2 | 3 | #ifndef DDEBUG 4 | #define DDEBUG 0 5 | #endif 6 | 7 | #include "ddebug.h" 8 | 9 | #include "chunked_parser.h" 10 | #include "ngx_http_chunkin_util.h" 11 | 12 | #define ngx_chunkin_min(x, y) ((x) < (y) ? (x) : (y)) 13 | 14 | enum { 15 | PRE_TEXT_LEN = 25, 16 | POST_TEXT_LEN = 25 17 | }; 18 | 19 | %% machine chunked; 20 | %% write data; 21 | 22 | 23 | ngx_int_t 24 | ngx_http_chunkin_init_chunked_parser(ngx_http_request_t *r, 25 | ngx_http_chunkin_ctx_t *ctx) 26 | { 27 | int cs; 28 | 29 | %% write init; 30 | 31 | ctx->chunks = NULL; 32 | ctx->next_chunk = NULL; 33 | ctx->chunk = NULL; 34 | ctx->chunk_size = 0; 35 | ctx->chunk_size_order = 0; 36 | ctx->chunk_bytes_read = 0; 37 | 38 | ctx->chunks_total_size = 0; 39 | 40 | ctx->parser_state = cs; 41 | 42 | return NGX_OK; 43 | } 44 | 45 | 46 | ngx_int_t 47 | ngx_http_chunkin_run_chunked_parser(ngx_http_request_t *r, 48 | ngx_http_chunkin_ctx_t *ctx, u_char **pos_addr, u_char *last, char *caller_info) 49 | { 50 | int cs = ctx->parser_state; 51 | ngx_connection_t *c = r->connection; 52 | signed char *pos = (signed char *) *pos_addr; 53 | signed char *p = (signed char *) *pos_addr; 54 | signed char *pe = (signed char *) last; 55 | signed char *eof = NULL; 56 | ngx_buf_t *b; 57 | ngx_flag_t done = 0; 58 | ngx_str_t pre, post; 59 | char* err_ctx = ""; 60 | ngx_str_t user_agent = ngx_null_string; 61 | ssize_t rest; 62 | 63 | %%{ 64 | #alphtype unsigned char; 65 | 66 | action finalize { 67 | done = 1; 68 | } 69 | 70 | action test_len { 71 | ctx->chunk_bytes_read < ctx->chunk_size 72 | } 73 | 74 | action read_data_byte { 75 | /* optimization for buffered chunk data */ 76 | 77 | rest = ngx_chunkin_min( 78 | (ssize_t)ctx->chunk_size - (ssize_t)ctx->chunk_bytes_read, 79 | (ssize_t)(pe - p)); 80 | 81 | dd("moving %d, chunk size %d, read %d, rest %d", 82 | (int)rest, 83 | (int)ctx->chunk_size, 84 | (int)ctx->chunk_bytes_read, 85 | (int)rest); 86 | 87 | ctx->chunk_bytes_read += rest; 88 | p += rest - 1; 89 | ctx->chunk->buf->last = (u_char *)p + 1; 90 | ctx->chunks_total_size += rest; 91 | 92 | /* dd("bytes read: %d (char '%c', bytes read %d, chunk size %d)", ctx->chunk->buf->last - ctx->chunk->buf->pos, *p, ctx->chunk_bytes_read, ctx->chunk_size); */ 93 | ngx_log_debug2(NGX_LOG_DEBUG_HTTP, c->log, 0, 94 | "chunkin: data bytes read: %uz (char: \"%c\")\n", 95 | ctx->chunk_bytes_read, *p); 96 | } 97 | 98 | action start_reading_size { 99 | ctx->chunk_bytes_read = 0; 100 | ctx->chunk_size = 0; 101 | ctx->chunk_size_order = 0; 102 | } 103 | 104 | action read_size { 105 | ctx->chunk_size <<= 4; 106 | ctx->chunk_size_order++; 107 | if (*p >= 'A' && *p <= 'F') { 108 | ctx->chunk_size |= 10 + *p - 'A'; 109 | } else if (*p >= 'a' && *p <= 'f') { 110 | ctx->chunk_size |= 10 + *p - 'a'; 111 | } else { 112 | ctx->chunk_size |= *p - '0'; 113 | } 114 | 115 | ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, 116 | "chunkin: chunk size: %uz\n", ctx->chunk_size); 117 | } 118 | 119 | action start_reading_data { 120 | ctx->chunk = ngx_http_chunkin_get_buf(r->pool, ctx); 121 | 122 | ctx->chunks_count++; 123 | 124 | if (ctx->chunks) { 125 | *ctx->next_chunk = ctx->chunk; 126 | } else { 127 | ctx->chunks = ctx->chunk; 128 | } 129 | 130 | ctx->next_chunk = &ctx->chunk->next; 131 | 132 | b = ctx->chunk->buf; 133 | 134 | b->last = b->pos = (u_char *) p; 135 | b->memory = 1; 136 | } 137 | 138 | action verify_data { 139 | if (ctx->chunk_bytes_read != ctx->chunk_size) { 140 | ngx_log_error(NGX_LOG_ERR, c->log, 0, 141 | "ERROR: chunk size not met: " 142 | "%uz != %uz\n", ctx->chunk_bytes_read, 143 | ctx->chunk_size); 144 | *pos_addr = (u_char*) p; 145 | ctx->parser_state = chunked_error; 146 | return NGX_ERROR; 147 | } 148 | 149 | if (ctx->chunk_size == 0) { 150 | /* remove the current chunk */ 151 | ctx->chunk->next = ctx->free_bufs; 152 | ctx->free_bufs = ctx->chunk; 153 | ctx->chunk = ctx->last_complete_chunk; 154 | if (ctx->last_complete_chunk) { 155 | ctx->last_complete_chunk->next = NULL; 156 | } else { 157 | ctx->chunks = NULL; 158 | } 159 | } else { 160 | ctx->last_complete_chunk = ctx->chunk; 161 | } 162 | } 163 | 164 | CR = "\r"; 165 | 166 | LF = "\n"; 167 | 168 | CRLF = CR LF $err{ err_ctx = "CRLF"; }; 169 | 170 | chunk_size = (xdigit+ - "0"+) >start_reading_size $read_size; 171 | 172 | chunk_data_octet = any when test_len; 173 | 174 | chunk_data = (chunk_data_octet+) 175 | >start_reading_data 176 | $read_data_byte 177 | $err{ err_ctx = "chunk_data"; } 178 | ; 179 | 180 | chunk_data_terminator = CR when ! test_len LF 181 | $err{ err_ctx = "chunk_data_terminator"; } 182 | ; 183 | 184 | SP = ' '; 185 | HT = '\t'; 186 | 187 | LWS = CRLF ? ( SP | HT )+; 188 | 189 | separator = "(" | ")" | "<" | ">" | "@" 190 | | "," | ";" | ":" | "\\" | ["] 191 | | "/" | "[" | "]" | "?" | "=" 192 | | "{" | "}" | SP | HT 193 | ; 194 | 195 | CTL = 0..31 | 127; 196 | 197 | token = (any -- (CTL | separator))+; 198 | 199 | chunk_ext_name = token; 200 | 201 | TEXT = (any -- CTL) | LWS; 202 | 203 | qdtext = TEXT -- ["]; 204 | 205 | CHAR = 0..127; 206 | 207 | quoted_pair = "\\" CHAR; 208 | 209 | quoted_string = ["] ( qdtext | quoted_pair )* ["]; 210 | 211 | chunk_ext_val = token | quoted_string; 212 | 213 | chunk_extension = ( ";" LWS* chunk_ext_name LWS* 214 | ("=" LWS* chunk_ext_val LWS*) ? )* 215 | ; 216 | 217 | chunk = chunk_size (LWS* -- CRLF) 218 | $err{ err_ctx = "chunk_size"; } 219 | (chunk_extension -- CRLF) ? 220 | CRLF 221 | $err{ err_ctx = "chunk_ext"; } 222 | chunk_data chunk_data_terminator 223 | @verify_data; 224 | 225 | last_chunk = "0"+ (LWS* -- CRLF) 226 | (chunk_extension -- CRLF) ? 227 | CRLF $err{ err_ctx = "last_chunk"; } 228 | ; 229 | 230 | parser = chunk* last_chunk CRLF 231 | $err{ err_ctx = "parser"; } 232 | ; 233 | 234 | main := parser @finalize; 235 | 236 | }%% 237 | 238 | %% write exec; 239 | 240 | ctx->parser_state = cs; 241 | 242 | *pos_addr = (u_char *) p; 243 | 244 | if (p != pe) { 245 | dd("ASSERTION FAILED: p != pe"); 246 | } 247 | 248 | if (done) { 249 | return NGX_OK; 250 | } 251 | 252 | if (cs == chunked_error) { 253 | 254 | #if EXTENDED_DEBUG 255 | 256 | ngx_str_t headers_buf, preread_buf; 257 | 258 | #endif 259 | 260 | for (post.data = (u_char *) p, post.len = 0; 261 | post.data + post.len != (u_char *) pe; post.len++) 262 | { 263 | if (post.len >= POST_TEXT_LEN) { 264 | break; 265 | } 266 | } 267 | 268 | for (pre.data = (u_char *) p, pre.len = 0; 269 | pre.data != (u_char *) pos; pre.data--, pre.len++) 270 | { 271 | if (pre.len >= PRE_TEXT_LEN) { 272 | break; 273 | } 274 | } 275 | 276 | if (r->headers_in.user_agent) { 277 | user_agent = r->headers_in.user_agent->value; 278 | } 279 | 280 | #if EXTENDED_DEBUG 281 | 282 | headers_buf.data = r->header_in->start; 283 | headers_buf.len = ctx->saved_header_in_pos - r->header_in->start; 284 | 285 | if (strcmp(caller_info, "preread") == 0) { 286 | preread_buf.data = (u_char *) pos; 287 | preread_buf.len = pe - pos; 288 | 289 | } else { 290 | preread_buf.data = ctx->saved_header_in_pos; 291 | preread_buf.len = r->header_in->pos - ctx->saved_header_in_pos; 292 | } 293 | 294 | #endif 295 | 296 | ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, 297 | "bad chunked body (buf size %O, buf offset %O, " 298 | "total decoded %uz, chunks count %d, " 299 | "chunk size %uz, chunk data read %uz, " 300 | "total to disk %uz, " 301 | "raw body size %O, caller \"%s\", " 302 | "plain_http %d, " 303 | 304 | #if (NGX_HTTP_SSL) 305 | 306 | "ssl %d, " 307 | #endif 308 | 309 | "keepalive %d, err ctx \"%s\", " 310 | "ctx ref count %ud, user agent \"%V\", " 311 | 312 | #if EXTENDED_DEBUG 313 | 314 | "headers \"%V\", preread \"%V\", " 315 | 316 | #endif 317 | 318 | "at char '%c' (%d), " 319 | "near \"%V <-- HERE %V\", marked by \" <-- HERE \").\n", 320 | (off_t) (pe - pos), (off_t) (p - pos), 321 | ctx->chunks_total_size, ctx->chunks_count, 322 | ctx->chunk_size, ctx->chunk_bytes_read, 323 | ctx->chunks_written_size, 324 | (off_t) ctx->raw_body_size, caller_info, 325 | (int) r->plain_http, 326 | 327 | #if (NGX_HTTP_SSL) 328 | 329 | r->connection->ssl ? 1 : 0, 330 | 331 | #endif 332 | 333 | (int) r->keepalive, err_ctx, 334 | ctx->count, &user_agent, 335 | 336 | #if EXTENDED_DEBUG 337 | 338 | &headers_buf, &preread_buf, 339 | 340 | #endif 341 | 342 | *p, *p, 343 | &pre, &post); 344 | 345 | return NGX_ERROR; 346 | } 347 | 348 | return NGX_AGAIN; 349 | } 350 | 351 | -------------------------------------------------------------------------------- /src/ddebug.h: -------------------------------------------------------------------------------- 1 | /* Copyright (C) agentzh */ 2 | 3 | #ifndef DDEBUG_H 4 | #define DDEBUG_H 5 | 6 | #include 7 | 8 | #if defined(DDEBUG) && (DDEBUG) 9 | 10 | # if (NGX_HAVE_VARIADIC_MACROS) 11 | 12 | # define dd(...) fprintf(stderr, "chunkin *** "); \ 13 | fprintf(stderr, __VA_ARGS__); \ 14 | fprintf(stderr, " at %s line %d.\n", __FILE__, __LINE__) 15 | 16 | # else 17 | 18 | #include 19 | #include 20 | 21 | #include 22 | 23 | static void dd(const char* fmt, ...) { 24 | } 25 | 26 | # endif 27 | 28 | #else 29 | 30 | # if (NGX_HAVE_VARIADIC_MACROS) 31 | 32 | # define dd(...) 33 | 34 | # else 35 | 36 | #include 37 | 38 | static void dd(const char* fmt, ...) { 39 | } 40 | 41 | # endif 42 | 43 | #endif 44 | 45 | #if defined(DDEBUG) && (DDEBUG) 46 | 47 | #define dd_check_read_event_handler(r) \ 48 | dd("r->read_event_handler = %s", \ 49 | r->read_event_handler == ngx_http_block_reading ? \ 50 | "ngx_http_block_reading" : \ 51 | r->read_event_handler == ngx_http_test_reading ? \ 52 | "ngx_http_test_reading" : \ 53 | r->read_event_handler == ngx_http_request_empty_handler ? \ 54 | "ngx_http_request_empty_handler" : "UNKNOWN") 55 | 56 | #define dd_check_write_event_handler(r) \ 57 | dd("r->write_event_handler = %s", \ 58 | r->write_event_handler == ngx_http_handler ? \ 59 | "ngx_http_handler" : \ 60 | r->write_event_handler == ngx_http_core_run_phases ? \ 61 | "ngx_http_core_run_phases" : \ 62 | r->write_event_handler == ngx_http_request_empty_handler ? \ 63 | "ngx_http_request_empty_handler" : "UNKNOWN") 64 | 65 | #else 66 | 67 | #define dd_check_read_event_handler(r) 68 | #define dd_check_write_event_handler(r) 69 | 70 | #endif 71 | 72 | #endif /* DDEBUG_H */ 73 | 74 | -------------------------------------------------------------------------------- /src/ngx_http_chunkin_filter_module.c: -------------------------------------------------------------------------------- 1 | /* Copyright (C) agentzh */ 2 | 3 | #ifndef DDEBUG 4 | #define DDEBUG 0 5 | #endif 6 | #include "ddebug.h" 7 | 8 | #include 9 | 10 | #include "ngx_http_chunkin_util.h" 11 | #include "ngx_http_chunkin_filter_module.h" 12 | #include "ngx_http_chunkin_request_body.h" 13 | 14 | enum { 15 | DEFAULT_MAX_CHUNKS_PER_BUF = 512 16 | }; 17 | 18 | static ngx_int_t ngx_http_chunkin_resume_handler(ngx_http_request_t *r); 19 | 20 | static char* ngx_http_chunkin_resume(ngx_conf_t *cf, ngx_command_t *cmd, 21 | void *conf); 22 | 23 | static void *ngx_http_chunkin_create_conf(ngx_conf_t *cf); 24 | static char *ngx_http_chunkin_merge_conf(ngx_conf_t *cf, void *parent, 25 | void *child); 26 | 27 | static ngx_int_t ngx_http_chunkin_handler(ngx_http_request_t *r); 28 | 29 | static ngx_int_t ngx_http_chunkin_init(ngx_conf_t *cf); 30 | 31 | static void ngx_http_chunkin_post_read(ngx_http_request_t *r); 32 | 33 | 34 | static ngx_command_t ngx_http_chunkin_commands[] = { 35 | 36 | { ngx_string("chunkin"), 37 | NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_CONF_FLAG, 38 | ngx_conf_set_flag_slot, 39 | NGX_HTTP_LOC_CONF_OFFSET, 40 | offsetof(ngx_http_chunkin_conf_t, enabled), 41 | NULL }, 42 | 43 | { ngx_string("chunkin_max_chunks_per_buf"), 44 | NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF|NGX_CONF_FLAG, 45 | ngx_conf_set_num_slot, 46 | NGX_HTTP_LOC_CONF_OFFSET, 47 | offsetof(ngx_http_chunkin_conf_t, max_chunks_per_buf), 48 | NULL }, 49 | 50 | { ngx_string("chunkin_resume"), 51 | NGX_HTTP_LOC_CONF|NGX_CONF_NOARGS, 52 | ngx_http_chunkin_resume, 53 | NGX_HTTP_LOC_CONF_OFFSET, 54 | 0, 55 | NULL }, 56 | 57 | { ngx_string("chunkin_keepalive"), 58 | NGX_HTTP_MAIN_CONF|NGX_HTTP_SRV_CONF|NGX_HTTP_LOC_CONF 59 | |NGX_HTTP_LIF_CONF|NGX_CONF_FLAG, 60 | ngx_conf_set_flag_slot, 61 | NGX_HTTP_LOC_CONF_OFFSET, 62 | offsetof(ngx_http_chunkin_conf_t, keepalive), 63 | NULL }, 64 | 65 | ngx_null_command 66 | }; 67 | 68 | 69 | static ngx_http_module_t ngx_http_chunkin_filter_module_ctx = { 70 | NULL, /* preconfiguration */ 71 | ngx_http_chunkin_init, /* postconfiguration */ 72 | 73 | NULL, /* create main configuration */ 74 | NULL, /* init main configuration */ 75 | 76 | NULL, /* create server configuration */ 77 | NULL, /* merge server configuration */ 78 | 79 | ngx_http_chunkin_create_conf, /* create location configuration */ 80 | ngx_http_chunkin_merge_conf /* merge location configuration */ 81 | }; 82 | 83 | 84 | ngx_module_t ngx_http_chunkin_filter_module = { 85 | NGX_MODULE_V1, 86 | &ngx_http_chunkin_filter_module_ctx, /* module context */ 87 | ngx_http_chunkin_commands, /* module directives */ 88 | NGX_HTTP_MODULE, /* module type */ 89 | NULL, /* init master */ 90 | NULL, /* init module */ 91 | NULL, /* init process */ 92 | NULL, /* init thread */ 93 | NULL, /* exit thread */ 94 | NULL, /* exit process */ 95 | NULL, /* exit master */ 96 | NGX_MODULE_V1_PADDING 97 | }; 98 | 99 | 100 | static void * 101 | ngx_http_chunkin_create_conf(ngx_conf_t *cf) 102 | { 103 | ngx_http_chunkin_conf_t *conf; 104 | 105 | conf = ngx_palloc(cf->pool, sizeof(ngx_http_chunkin_conf_t)); 106 | if (conf == NULL) { 107 | return NULL; 108 | } 109 | 110 | conf->enabled = NGX_CONF_UNSET; 111 | conf->keepalive = NGX_CONF_UNSET; 112 | conf->max_chunks_per_buf = NGX_CONF_UNSET_UINT; 113 | 114 | return conf; 115 | } 116 | 117 | 118 | static char * 119 | ngx_http_chunkin_merge_conf(ngx_conf_t *cf, void *parent, void *child) 120 | { 121 | ngx_http_chunkin_conf_t *prev = parent; 122 | ngx_http_chunkin_conf_t *conf = child; 123 | 124 | ngx_conf_merge_value(conf->enabled, prev->enabled, 0); 125 | ngx_conf_merge_value(conf->keepalive, prev->keepalive, 0); 126 | ngx_conf_merge_uint_value(conf->max_chunks_per_buf, 127 | prev->max_chunks_per_buf, 128 | DEFAULT_MAX_CHUNKS_PER_BUF); 129 | 130 | return NGX_CONF_OK; 131 | } 132 | 133 | 134 | static ngx_flag_t 135 | ngx_http_chunkin_is_chunked_encoding(ngx_http_request_t *r) 136 | { 137 | dd("is chunked encoding..."); 138 | 139 | return r->headers_in.transfer_encoding && 140 | r->headers_in.transfer_encoding->value.len >= 7 && 141 | ngx_strcasestrn(r->headers_in.transfer_encoding->value.data, 142 | "chunked", 7 - 1); 143 | } 144 | 145 | 146 | static ngx_int_t 147 | ngx_http_chunkin_init(ngx_conf_t *cf) 148 | { 149 | ngx_http_handler_pt *h; 150 | ngx_http_core_main_conf_t *cmcf; 151 | 152 | cmcf = ngx_http_conf_get_module_main_conf(cf, ngx_http_core_module); 153 | 154 | h = ngx_array_push(&cmcf->phases[NGX_HTTP_ACCESS_PHASE].handlers); 155 | if (h == NULL) { 156 | return NGX_ERROR; 157 | } 158 | 159 | *h = ngx_http_chunkin_handler; 160 | 161 | return NGX_OK; 162 | } 163 | 164 | 165 | static ngx_int_t 166 | ngx_http_chunkin_handler(ngx_http_request_t *r) 167 | { 168 | ngx_http_chunkin_ctx_t *ctx; 169 | ngx_http_chunkin_conf_t *conf; 170 | ngx_int_t rc; 171 | 172 | dd("entered chunkin handler..."); 173 | 174 | conf = ngx_http_get_module_loc_conf(r, ngx_http_chunkin_filter_module); 175 | 176 | if (!conf->enabled || r != r->main) { 177 | dd("conf not enabled: %d", (int) conf->enabled); 178 | 179 | return NGX_DECLINED; 180 | } 181 | 182 | ctx = ngx_http_get_module_ctx(r, ngx_http_chunkin_filter_module); 183 | 184 | if (ctx == NULL) { 185 | dd("ctx not found"); 186 | 187 | return NGX_DECLINED; 188 | } 189 | 190 | if (ctx->done) { 191 | return NGX_DECLINED; 192 | } 193 | 194 | if (ctx->waiting_more_body) { 195 | return NGX_AGAIN; 196 | } 197 | 198 | dd("reading chunked input eagerly..."); 199 | 200 | if (!conf->keepalive && r->keepalive) { 201 | dd("dis-enabling r->keepalive..."); 202 | r->keepalive = 0; 203 | } 204 | 205 | dd_check_read_event_handler(r); 206 | dd_check_write_event_handler(r); 207 | 208 | /* 209 | rc = ngx_http_read_client_request_body(r, 210 | ngx_http_chunkin_post_read); 211 | */ 212 | 213 | ngx_http_chunkin_clear_transfer_encoding(r); 214 | 215 | r->header_in->pos = r->header_end + sizeof(CRLF) - 1; 216 | 217 | if (r->header_in->pos > r->header_in->last) { 218 | ngx_log_error(NGX_LOG_INFO, r->connection->log, 0, 219 | "chunkin: r->header_in->pos overflown"); 220 | 221 | return NGX_HTTP_INTERNAL_SERVER_ERROR; 222 | } 223 | 224 | if (*(r->header_in->pos - 2) != CR || *(r->header_in->pos - 1) != LF) { 225 | ngx_log_error(NGX_LOG_INFO, r->connection->log, 0, 226 | "chunkin: r->header_in->pos not lead by CRLF"); 227 | 228 | return NGX_HTTP_INTERNAL_SERVER_ERROR; 229 | } 230 | 231 | dd("chunkin handler: header_in->pos: %d", 232 | (int)(r->header_in->pos - r->header_in->start)); 233 | 234 | r->request_body_in_persistent_file = 1; 235 | 236 | rc = ngx_http_chunkin_read_chunked_request_body(r, 237 | ngx_http_chunkin_post_read); 238 | 239 | if (rc >= NGX_HTTP_SPECIAL_RESPONSE) { 240 | dd("read client request body returned special response %d", (int) rc); 241 | return rc; 242 | } 243 | 244 | dd("read client request body returned %d", (int) rc); 245 | 246 | if (rc == NGX_AGAIN) { 247 | ctx->waiting_more_body = 1; 248 | 249 | return NGX_AGAIN; 250 | } 251 | 252 | ctx->done = 1; 253 | 254 | return NGX_DECLINED; 255 | } 256 | 257 | 258 | static void 259 | ngx_http_chunkin_post_read(ngx_http_request_t *r) 260 | { 261 | ngx_http_chunkin_ctx_t *ctx; 262 | 263 | dd("post read"); 264 | 265 | dd_check_read_event_handler(r); 266 | dd_check_write_event_handler(r); 267 | 268 | r->read_event_handler = ngx_http_block_reading; 269 | 270 | ctx = ngx_http_get_module_ctx(r, ngx_http_chunkin_filter_module); 271 | 272 | ctx->done = 1; 273 | 274 | if (ctx->waiting_more_body) { 275 | ctx->waiting_more_body = 0; 276 | ngx_http_core_run_phases(r); 277 | } 278 | } 279 | 280 | 281 | static char* ngx_http_chunkin_resume(ngx_conf_t *cf, ngx_command_t *cmd, 282 | void *conf) 283 | { 284 | ngx_http_core_loc_conf_t *clcf; 285 | 286 | clcf = ngx_http_conf_get_module_loc_conf(cf, 287 | ngx_http_core_module); 288 | 289 | clcf->handler = ngx_http_chunkin_resume_handler; 290 | 291 | return NGX_CONF_OK; 292 | } 293 | 294 | 295 | static ngx_int_t 296 | ngx_http_chunkin_resume_handler(ngx_http_request_t *r) 297 | { 298 | ngx_int_t rc; 299 | ngx_http_chunkin_conf_t *conf; 300 | ngx_http_chunkin_ctx_t *ctx; 301 | 302 | conf = ngx_http_get_module_loc_conf(r, ngx_http_chunkin_filter_module); 303 | 304 | dd("method: %.*s (%d)", (int) r->method_name.len, r->method_name.data, 305 | (int) r->method); 306 | 307 | if (!conf->enabled || r != r->main 308 | || (r->method != NGX_HTTP_PUT && r->method != NGX_HTTP_POST && 309 | r->method != NGX_HTTP_DELETE)) 310 | { 311 | dd("conf not enabled or in subrequest or not POST nor PUT requests"); 312 | 313 | return NGX_HTTP_LENGTH_REQUIRED; 314 | } 315 | 316 | if (!ngx_http_chunkin_is_chunked_encoding(r->main)) { 317 | dd("found POST request, but not chunked"); 318 | return NGX_HTTP_LENGTH_REQUIRED; 319 | } 320 | 321 | dd("chunked request test passed"); 322 | 323 | /* XXX just to fool the nginx core */ 324 | r->headers_in.content_length_n = 1; 325 | 326 | ngx_http_chunkin_clear_transfer_encoding(r); 327 | 328 | ctx = ngx_http_get_module_ctx(r, ngx_http_chunkin_filter_module); 329 | 330 | if (ctx == NULL) { 331 | ctx = ngx_pcalloc(r->pool, sizeof(ngx_http_chunkin_ctx_t)); 332 | if (ctx == NULL) { 333 | return NGX_ERROR; 334 | } 335 | 336 | /* ctx->ignore_body is set to 0 */ 337 | } 338 | 339 | r->discard_body = 0; 340 | r->error_page = 0; 341 | r->err_status = 0; 342 | 343 | #if 0 344 | r->method = NGX_HTTP_PUT; 345 | r->headers_in.content_length = NULL; 346 | r->headers_in.content_length_n = -1; 347 | #endif 348 | 349 | rc = ngx_http_chunkin_process_request_header(r); 350 | if (rc != NGX_OK) { 351 | return rc; 352 | } 353 | 354 | #if 0 355 | r->plain_http = 1; 356 | #endif 357 | 358 | rc = ngx_http_chunkin_process_request(r); 359 | if (rc != NGX_OK) { 360 | return rc; 361 | } 362 | 363 | rc = ngx_http_chunkin_internal_redirect(r, &r->main->uri, &r->main->args, 364 | ctx); 365 | 366 | /* ngx_http_process_request calls this to handle subrequests, 367 | * so we need to call it here as well */ 368 | ngx_http_run_posted_requests(r->connection); 369 | 370 | return rc; 371 | } 372 | 373 | -------------------------------------------------------------------------------- /src/ngx_http_chunkin_filter_module.h: -------------------------------------------------------------------------------- 1 | #ifndef NGX_HTTP_CHUNKIN_FILTER_MODULE_H 2 | #define NGX_HTTP_CHUNKIN_FILTER_MODULE_H 3 | 4 | #include 5 | #include 6 | #include 7 | 8 | typedef struct { 9 | ngx_flag_t enabled; 10 | ngx_flag_t keepalive; 11 | ngx_uint_t max_chunks_per_buf; 12 | 13 | } ngx_http_chunkin_conf_t; 14 | 15 | 16 | typedef struct { 17 | ngx_flag_t ignore_body /* for output filters only */; 18 | int parser_state; /* current state */ 19 | 20 | size_t chunk_bytes_read; 21 | size_t chunk_size; 22 | int chunk_size_order; 23 | 24 | size_t chunks_total_size; 25 | size_t chunks_written_size; 26 | ngx_uint_t chunks_count; 27 | 28 | off_t raw_body_size; 29 | 30 | ngx_chain_t *chunks; 31 | ngx_chain_t **next_chunk; 32 | ngx_chain_t *chunk; 33 | 34 | ngx_flag_t just_after_preread; 35 | 36 | ngx_chain_t *free_bufs; 37 | ngx_chain_t *last_complete_chunk; 38 | 39 | u_char *saved_header_in_pos; 40 | 41 | ngx_uint_t count; 42 | ngx_flag_t r_discard_body:1; 43 | 44 | ngx_flag_t done:1; 45 | ngx_flag_t waiting_more_body:1; 46 | 47 | } ngx_http_chunkin_ctx_t; 48 | 49 | 50 | extern ngx_module_t ngx_http_chunkin_filter_module; 51 | 52 | #endif /* NGX_HTTP_CHUNKIN_FILTER_MODULE_H */ 53 | 54 | -------------------------------------------------------------------------------- /src/ngx_http_chunkin_request_body.c: -------------------------------------------------------------------------------- 1 | 2 | /* 3 | * Copyright (C) Yichun Zhang (agentzh) 4 | */ 5 | 6 | 7 | #ifndef DDEBUG 8 | #define DDEBUG 0 9 | #endif 10 | 11 | 12 | #include "ddebug.h" 13 | 14 | #include "ngx_http_chunkin_util.h" 15 | #include "chunked_parser.h" 16 | #include "ngx_http_chunkin_filter_module.h" 17 | #include "ngx_http_chunkin_request_body.h" 18 | 19 | 20 | enum { 21 | MAX_CHUNKS_PER_DISK_WRITE = 512 22 | }; 23 | 24 | 25 | static ngx_int_t ngx_http_test_expect(ngx_http_request_t *r); 26 | static void ngx_http_chunkin_read_chunked_request_body_handler( 27 | ngx_http_request_t *r); 28 | static ngx_int_t ngx_http_chunkin_do_read_chunked_request_body( 29 | ngx_http_request_t *r); 30 | static ngx_int_t ngx_http_write_request_body(ngx_http_request_t *r, 31 | ngx_chain_t *body, int chain_count); 32 | 33 | 34 | /* this function's implementation is borrowed from nginx 0.8.20 35 | * and modified a lot to work with the chunked encoding. 36 | * copyrighted (C) by Igor Sysoev */ 37 | ngx_int_t 38 | ngx_http_chunkin_read_chunked_request_body(ngx_http_request_t *r, 39 | ngx_http_client_body_handler_pt post_handler) 40 | { 41 | size_t preread; 42 | size_t size; 43 | ngx_http_request_body_t *rb; 44 | ngx_http_core_loc_conf_t *clcf; 45 | ngx_http_chunkin_ctx_t *ctx; 46 | ngx_int_t rc; 47 | 48 | /* r->request_body_in_single_buf = 1; */ 49 | /* r->request_body_in_file_only = 1; */ 50 | 51 | if (r->request_body || r->discard_body) { 52 | post_handler(r); 53 | return NGX_OK; 54 | } 55 | 56 | if (ngx_http_test_expect(r) != NGX_OK) { 57 | return NGX_HTTP_INTERNAL_SERVER_ERROR; 58 | } 59 | 60 | rb = ngx_pcalloc(r->pool, sizeof(ngx_http_request_body_t)); 61 | if (rb == NULL) { 62 | return NGX_HTTP_INTERNAL_SERVER_ERROR; 63 | } 64 | 65 | r->request_body = rb; 66 | 67 | clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); 68 | 69 | rb->post_handler = post_handler; 70 | 71 | ctx = ngx_http_get_module_ctx(r, ngx_http_chunkin_filter_module); 72 | if (ctx == NULL) { 73 | return NGX_HTTP_INTERNAL_SERVER_ERROR; 74 | } 75 | 76 | ctx->count++; 77 | 78 | ctx->saved_header_in_pos = r->header_in->pos; 79 | 80 | preread = r->header_in->last - r->header_in->pos; 81 | 82 | ngx_http_chunkin_init_chunked_parser(r, ctx); 83 | 84 | if (preread) { 85 | ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, 86 | "chunkin: http chunked client request body preread %uz", 87 | preread); 88 | 89 | dd("raw chunked body %.*s", preread, r->header_in->pos); 90 | 91 | rc = ngx_http_chunkin_run_chunked_parser(r, ctx, 92 | &r->header_in->pos, r->header_in->last, "preread"); 93 | 94 | if (rc >= NGX_HTTP_SPECIAL_RESPONSE) { 95 | return rc; 96 | } 97 | 98 | if (rc == NGX_ERROR) { 99 | /* chunked body parsefail */ 100 | return NGX_HTTP_BAD_REQUEST; 101 | } 102 | 103 | if (rc == NGX_OK) { 104 | /* TODO: take r->request_body_in_file_only 105 | * into account */ 106 | 107 | dd("we have the whole chunked body in 'prepread'..."); 108 | 109 | dd("keepalive? %s", r->keepalive ? "yes" : "no"); 110 | 111 | dd("chunks total size: %d", ctx->chunks_total_size); 112 | 113 | rc = ngx_http_chunkin_set_content_length_header(r, 114 | ctx->chunks_total_size); 115 | 116 | if (rc >= NGX_HTTP_SPECIAL_RESPONSE) { 117 | return rc; 118 | } 119 | 120 | rb->bufs = ctx->chunks; 121 | if (rb->bufs) { 122 | rb->buf = ctx->chunks->buf; 123 | } 124 | 125 | dd("buf len: %d", rb->buf->last - rb->buf->pos); 126 | 127 | dd("buf left (len %d): %s", 128 | (int) (r->header_in->last - r->header_in->pos), 129 | r->header_in->pos); 130 | 131 | /* r->header_in->pos = r->header_in->last; */ 132 | 133 | r->request_length += r->header_in->pos 134 | - ctx->saved_header_in_pos; 135 | 136 | ctx->raw_body_size += r->header_in->pos 137 | - ctx->saved_header_in_pos; 138 | 139 | post_handler(r); 140 | 141 | return NGX_OK; 142 | } 143 | 144 | if (rc != NGX_AGAIN) { 145 | /* this is impossible to reach... */ 146 | return NGX_HTTP_INTERNAL_SERVER_ERROR; 147 | } 148 | 149 | size = r->header_in->last - r->header_in->pos; 150 | if (size) { 151 | ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, 152 | "chunkin: internal assertion failed: %O bytes " 153 | "left in the r->header_in buffer but " 154 | "the parser returns NGX_AGAIN", 155 | (off_t) size); 156 | 157 | return NGX_HTTP_INTERNAL_SERVER_ERROR; 158 | } 159 | 160 | dd("we need to read more chunked body in addition to 'prepread'..."); 161 | 162 | ctx->just_after_preread = 1; 163 | 164 | /* r->header_in->pos = r->header_in->last; */ 165 | 166 | r->request_length += preread; 167 | 168 | ctx->raw_body_size += preread; 169 | } 170 | 171 | size = clcf->client_body_buffer_size; 172 | if (ctx->chunks_total_size > size) { 173 | size = ctx->chunks_total_size; 174 | } 175 | 176 | dd("chunks total size after preread: %d, buf size: %d", 177 | (int)ctx->chunks_total_size, (int)size); 178 | 179 | rb->buf = ngx_create_temp_buf(r->pool, size); 180 | if (rb->buf == NULL) { 181 | return NGX_HTTP_INTERNAL_SERVER_ERROR; 182 | } 183 | 184 | r->read_event_handler = ngx_http_chunkin_read_chunked_request_body_handler; 185 | 186 | rc = ngx_http_chunkin_do_read_chunked_request_body(r); 187 | 188 | return rc; 189 | } 190 | 191 | 192 | /* mostly a clone of the ngx_http_read_client_request_body_handler 193 | * function in ngx_http_request_body.c of nginx 0.8.20. 194 | * copyrighted by Igor Sysoev. */ 195 | static void 196 | ngx_http_chunkin_read_chunked_request_body_handler(ngx_http_request_t *r) 197 | { 198 | ngx_int_t rc; 199 | 200 | if (r->connection->read->timedout) { 201 | r->connection->timedout = 1; 202 | 203 | (void) ngx_http_discard_request_body(r); 204 | 205 | ngx_http_finalize_request(r, NGX_HTTP_REQUEST_TIME_OUT); 206 | return; 207 | } 208 | 209 | rc = ngx_http_chunkin_do_read_chunked_request_body(r); 210 | 211 | if (rc >= NGX_HTTP_SPECIAL_RESPONSE) { 212 | ngx_http_finalize_request(r, rc); 213 | } 214 | } 215 | 216 | 217 | /* mostly based on the ngx_http_do_read_client_request_body 218 | * function in ngx_http_request_body.c of nginx 0.8.20. 219 | * copyrighted by Igor Sysoev. */ 220 | static ngx_int_t 221 | ngx_http_chunkin_do_read_chunked_request_body(ngx_http_request_t *r) 222 | { 223 | ngx_int_t rc; 224 | size_t size; 225 | ssize_t n; 226 | ngx_buf_t *b; 227 | ngx_connection_t *c; 228 | ngx_http_request_body_t *rb; 229 | ngx_http_core_loc_conf_t *clcf; 230 | ngx_http_chunkin_ctx_t *ctx; 231 | ngx_flag_t done; 232 | ngx_chain_t *cl, *pending_chunk; 233 | u_char *p; 234 | ngx_http_chunkin_conf_t *conf; 235 | 236 | c = r->connection; 237 | rb = r->request_body; 238 | ctx = ngx_http_get_module_ctx(r, ngx_http_chunkin_filter_module); 239 | 240 | ngx_log_debug0(NGX_LOG_DEBUG_HTTP, c->log, 0, 241 | "chunkin: http chunkin read chunked client request body"); 242 | 243 | done = 0; 244 | 245 | clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); 246 | 247 | if (ctx->just_after_preread) { 248 | ctx->just_after_preread = 0; 249 | 250 | dd("Just after preread and ctx->chunks defined (bytes read: %d, " 251 | "chunk size: %d, last chars %c %c %c)", ctx->chunk_bytes_read, 252 | ctx->chunk_size, *(r->header_in->pos - 2), 253 | *(r->header_in->pos - 1), *r->header_in->pos); 254 | 255 | for (cl = ctx->chunks; cl; cl = cl->next) { 256 | b = cl->buf; 257 | 258 | dd("before ngx_copy..."); 259 | p = rb->buf->last; 260 | rb->buf->last = ngx_copy(rb->buf->last, b->pos, b->last - b->pos); 261 | dd("after ngx_copy..."); 262 | 263 | b->pos = p; 264 | b->last = rb->buf->last; 265 | } 266 | } 267 | 268 | conf = ngx_http_get_module_loc_conf(r, ngx_http_chunkin_filter_module); 269 | 270 | for ( ;; ) { 271 | for ( ;; ) { 272 | /* 273 | dd("client_max_body_size: %d, raw_body_size: %d", 274 | (int)clcf->client_max_body_size, 275 | (int)ctx->raw_body_size); 276 | */ 277 | 278 | if (clcf->client_max_body_size 279 | && clcf->client_max_body_size < ctx->raw_body_size) 280 | { 281 | ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, 282 | "chunkin: client intended to send too large body: " 283 | "%O bytes", (off_t) ctx->raw_body_size); 284 | 285 | return NGX_HTTP_REQUEST_ENTITY_TOO_LARGE; 286 | } 287 | 288 | /* dd("rb->buf pos: %d", (int) (rb->buf->last - rb->buf->pos)); */ 289 | 290 | if (rb->buf->last == rb->buf->end 291 | || ctx->chunks_count > conf->max_chunks_per_buf) 292 | { 293 | if (ctx->chunks_count > conf->max_chunks_per_buf) { 294 | dd("too many chounks already: %d (max %d, buf last %c)", 295 | (int) ctx->chunks_count, 296 | (int) conf->max_chunks_per_buf, 297 | *(rb->buf->last - 2)); 298 | } 299 | 300 | if (ctx->chunks == NULL 301 | || (ctx->chunks_total_size 302 | && ctx->chunks_total_size <= 303 | ctx->chunks_written_size)) 304 | { 305 | ngx_log_error(NGX_LOG_WARN, c->log, 0, 306 | "chunkin: the chunkin_max_chunks_per_buf or " 307 | "max_client_body_size setting seems rather small " 308 | "(chunks %snull, total decoded %d, " 309 | "total written %d)", 310 | (u_char *) (ctx->chunks ? "not " : ""), 311 | (int) ctx->chunks_total_size, 312 | (int) ctx->chunks_written_size); 313 | 314 | } else { 315 | dd("save exceeding part to disk (%d bytes), buf size: %d, " 316 | "chunks count: %d", 317 | ctx->chunks_total_size - ctx->chunks_written_size, 318 | rb->buf->end - rb->buf->start, 319 | ctx->chunks_count); 320 | 321 | rc = ngx_http_write_request_body(r, ctx->chunks, 322 | ctx->chunks_count); 323 | 324 | if (rc != NGX_OK) { 325 | return NGX_HTTP_INTERNAL_SERVER_ERROR; 326 | } 327 | } 328 | 329 | if (ctx->last_complete_chunk) { 330 | pending_chunk = ctx->last_complete_chunk->next; 331 | /* add ctx->chunks ~ ctx->last_complete_chunk 332 | * into ctx->free_bufs */ 333 | ctx->last_complete_chunk->next = ctx->free_bufs; 334 | ctx->free_bufs = ctx->chunks; 335 | ctx->last_complete_chunk = NULL; 336 | } else { 337 | pending_chunk = ctx->chunks; 338 | } 339 | 340 | if (pending_chunk) { 341 | ctx->next_chunk = &pending_chunk->next; 342 | ctx->chunks = pending_chunk; 343 | ctx->chunk = pending_chunk; 344 | 345 | ctx->chunk->buf->pos = rb->buf->start; 346 | ctx->chunk->buf->last = rb->buf->start; 347 | ctx->chunks_count = 1; 348 | 349 | ctx->chunks_written_size = ctx->chunks_total_size 350 | - ngx_buf_size(pending_chunk->buf); 351 | } else { 352 | ctx->next_chunk = NULL; 353 | 354 | ctx->chunks = NULL; 355 | ctx->chunk = NULL; 356 | 357 | ctx->chunks_count = 0; 358 | 359 | ctx->chunks_written_size = ctx->chunks_total_size; 360 | } 361 | 362 | dd("reset rb->buf"); 363 | 364 | rb->buf->last = rb->buf->start; 365 | 366 | #if 0 367 | /* XXX just for debugging... */ 368 | ngx_memzero(rb->buf->start, rb->buf->end - rb->buf->start); 369 | #endif 370 | } 371 | 372 | size = rb->buf->end - rb->buf->last; 373 | 374 | n = c->recv(c, rb->buf->last, size); 375 | ngx_log_debug1(NGX_LOG_DEBUG_HTTP, c->log, 0, 376 | "chunkin: http chunked client request body recv %z", 377 | n); 378 | 379 | if (n == NGX_AGAIN) { 380 | dd("NGX_AGAIN caught"); 381 | break; 382 | } 383 | 384 | if (n == 0) { 385 | ngx_log_error(NGX_LOG_INFO, c->log, 0, 386 | "chunkin: client closed prematurely connection"); 387 | } 388 | 389 | if (n == 0 || n == NGX_ERROR) { 390 | c->error = 1; 391 | return NGX_HTTP_BAD_REQUEST; 392 | } 393 | 394 | /* save the original pos */ 395 | p = rb->buf->last; 396 | 397 | rc = ngx_http_chunkin_run_chunked_parser(r, ctx, 398 | &rb->buf->last, rb->buf->last + n, "main handler"); 399 | 400 | if (rc >= NGX_HTTP_SPECIAL_RESPONSE) { 401 | return rc; 402 | } 403 | 404 | if (rc == NGX_ERROR) { 405 | /* chunked body parsefail */ 406 | return NGX_HTTP_BAD_REQUEST; 407 | } 408 | 409 | /* rb->buf->last += n; */ 410 | 411 | r->request_length += n; 412 | 413 | ctx->raw_body_size += n; 414 | 415 | if (rc == NGX_OK) { 416 | dd("successfully done the parsing"); 417 | 418 | dd("keepalive? %s", r->keepalive ? "yes" : "no"); 419 | 420 | if (r->keepalive) { 421 | dd("cleaning the buffers for pipelined reqeusts (if any)"); 422 | size = p + n - rb->buf->last; 423 | if (size) { 424 | dd("found remaining data for pipelined requests"); 425 | if (size > (size_t) (r->header_in->end 426 | - ctx->saved_header_in_pos)) 427 | { 428 | /* XXX enlarge the r->header_in buffer... */ 429 | r->keepalive = 0; 430 | } else { 431 | r->header_in->pos = ctx->saved_header_in_pos; 432 | r->header_in->last = ngx_copy(r->header_in->pos, 433 | rb->buf->last, size); 434 | 435 | } 436 | } 437 | } 438 | 439 | done = 1; 440 | break; 441 | } 442 | 443 | /* rc == NGX_AGAIN */ 444 | 445 | if (rb->buf->last < rb->buf->end) { 446 | break; 447 | } 448 | } 449 | 450 | if (done) { 451 | break; 452 | } 453 | 454 | if (!c->read->ready) { 455 | ngx_add_timer(c->read, clcf->client_body_timeout); 456 | 457 | if (ngx_handle_read_event(c->read, 0) != NGX_OK) { 458 | return NGX_HTTP_INTERNAL_SERVER_ERROR; 459 | } 460 | 461 | return NGX_AGAIN; 462 | } 463 | } 464 | 465 | if (!done) { 466 | return NGX_HTTP_BAD_REQUEST; 467 | } 468 | 469 | if (c->read->timer_set) { 470 | ngx_del_timer(c->read); 471 | } 472 | 473 | if (rb->temp_file || r->request_body_in_file_only) { 474 | size = ctx->chunks_total_size - ctx->chunks_written_size; 475 | dd("save the last part to disk...(%d bytes left)", size); 476 | if (size == 0) { 477 | ctx->chunks = NULL; 478 | } 479 | 480 | #if 0 481 | n = 0; 482 | for (cl = ctx->chunks; cl != NULL; cl = cl->next) { 483 | /* dd("chunks %d found buf %c", n, *cl->buf->start); */ 484 | n++; 485 | } 486 | #endif 487 | 488 | dd("for total %d chunks found", n); 489 | 490 | /* save the last part */ 491 | rc = ngx_http_write_request_body(r, ctx->chunks, ctx->chunks_count); 492 | if (rc != NGX_OK) { 493 | return NGX_HTTP_INTERNAL_SERVER_ERROR; 494 | } 495 | 496 | b = ngx_calloc_buf(r->pool); 497 | if (b == NULL) { 498 | return NGX_HTTP_INTERNAL_SERVER_ERROR; 499 | } 500 | 501 | b->in_file = 1; 502 | b->file_pos = 0; 503 | b->file_last = rb->temp_file->file.offset; 504 | b->file = &rb->temp_file->file; 505 | 506 | rb->bufs = ngx_http_chunkin_get_buf(r->pool, ctx); 507 | 508 | if (rb->bufs == NULL) { 509 | return NGX_HTTP_INTERNAL_SERVER_ERROR; 510 | } 511 | 512 | rb->bufs->buf = b; 513 | 514 | } else if (r->request_body_in_single_buf) { 515 | dd("request body in single buf"); 516 | 517 | /* XXX we may not have to allocate a big buffer here */ 518 | 519 | size = 0; 520 | for (cl = ctx->chunks; cl != NULL; cl = cl->next) { 521 | size += ngx_buf_size(cl->buf); 522 | } 523 | 524 | rb->buf = ngx_create_temp_buf(r->pool, size); 525 | 526 | if (rb->buf == NULL) { 527 | return NGX_HTTP_INTERNAL_SERVER_ERROR; 528 | } 529 | 530 | for (cl = ctx->chunks; cl != NULL; cl = cl->next) { 531 | size = ngx_buf_size(cl->buf); 532 | 533 | dd("copy buf ...(size %d)", size); 534 | 535 | rb->buf->last = 536 | ngx_cpymem(rb->buf->last, cl->buf->pos, size); 537 | } 538 | 539 | rb->bufs = ngx_http_chunkin_get_buf(r->pool, ctx); 540 | 541 | if (rb->bufs == NULL) { 542 | return NGX_HTTP_INTERNAL_SERVER_ERROR; 543 | } 544 | 545 | rb->bufs->buf = rb->buf; 546 | 547 | } else { 548 | rb->bufs = ctx->chunks; 549 | } 550 | 551 | #if 0 552 | if (rb->bufs) { 553 | int i; 554 | for (i = 0, cl = rb->bufs; cl; cl = cl->next, i++) { 555 | if (cl->buf->memory && cl->buf->pos == cl->buf->last) { 556 | dd("Found zero size buf in chain pos %d", i); 557 | } 558 | } 559 | } 560 | #endif 561 | 562 | dd("last minute, chunks count: %d, chunks_total_size: %d", 563 | ctx->chunks_count, ctx->chunks_total_size); 564 | 565 | #if 0 566 | size = 0; 567 | for (cl = rb->bufs; cl; cl = cl->next) { 568 | size += ngx_buf_size(cl->buf); 569 | } 570 | dd("data size: %d", size); 571 | #endif 572 | 573 | rc = ngx_http_chunkin_set_content_length_header(r, ctx->chunks_total_size); 574 | 575 | if (rc >= NGX_HTTP_SPECIAL_RESPONSE) { 576 | return rc; 577 | } 578 | 579 | rb->post_handler(r); 580 | 581 | return NGX_OK; 582 | } 583 | 584 | 585 | /* mostly an exact clone of the ngx_http_test_expect 586 | * function in ngx_http_request_body.c of nginx 0.8.20. 587 | * copyrighted by Igor Sysoev. */ 588 | static ngx_int_t 589 | ngx_http_test_expect(ngx_http_request_t *r) 590 | { 591 | ngx_int_t n; 592 | ngx_str_t *expect; 593 | 594 | if (r->expect_tested 595 | || r->headers_in.expect == NULL 596 | || r->http_version < NGX_HTTP_VERSION_11) 597 | { 598 | return NGX_OK; 599 | } 600 | 601 | r->expect_tested = 1; 602 | 603 | expect = &r->headers_in.expect->value; 604 | 605 | if (expect->len != sizeof("100-continue") - 1 606 | || ngx_strncasecmp(expect->data, (u_char *) "100-continue", 607 | sizeof("100-continue") - 1) 608 | != 0) 609 | { 610 | return NGX_OK; 611 | } 612 | 613 | ngx_log_debug0(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, 614 | "send 100 Continue"); 615 | 616 | n = r->connection->send(r->connection, 617 | (u_char *) "HTTP/1.1 100 Continue" CRLF CRLF, 618 | sizeof("HTTP/1.1 100 Continue" CRLF CRLF) - 1); 619 | 620 | if (n == sizeof("HTTP/1.1 100 Continue" CRLF CRLF) - 1) { 621 | return NGX_OK; 622 | } 623 | 624 | /* we assume that such small packet should be send successfully */ 625 | 626 | return NGX_ERROR; 627 | } 628 | 629 | 630 | /* mostly an exact clone of the ngx_http_write_request_body 631 | * function in ngx_http_request_body.c of nginx 0.8.20. 632 | * copyrighted by Igor Sysoev. */ 633 | static ngx_int_t 634 | ngx_http_write_request_body(ngx_http_request_t *r, ngx_chain_t *body, 635 | int chain_count) 636 | { 637 | ssize_t n; 638 | ngx_temp_file_t *tf; 639 | ngx_http_request_body_t *rb; 640 | ngx_http_core_loc_conf_t *clcf; 641 | int i; 642 | ngx_chain_t *cl, *saved_next; 643 | 644 | rb = r->request_body; 645 | 646 | if (rb->temp_file == NULL) { 647 | tf = ngx_pcalloc(r->pool, sizeof(ngx_temp_file_t)); 648 | if (tf == NULL) { 649 | return NGX_ERROR; 650 | } 651 | 652 | clcf = ngx_http_get_module_loc_conf(r, ngx_http_core_module); 653 | 654 | tf->file.fd = NGX_INVALID_FILE; 655 | tf->file.log = r->connection->log; 656 | tf->path = clcf->client_body_temp_path; 657 | tf->pool = r->pool; 658 | 659 | if (rb->buf && rb->buf->last == rb->buf->end) { 660 | tf->warn = "a client request body is buffered to a temporary file " 661 | "(exceeding client_body_buffer_size)"; 662 | } else { 663 | tf->warn = "a client request body is buffered to a temporary file " 664 | "(exceeding chunkin_max_chunks_per_buf)"; 665 | } 666 | 667 | tf->log_level = r->request_body_file_log_level; 668 | tf->persistent = r->request_body_in_persistent_file; 669 | tf->clean = r->request_body_in_clean_file; 670 | 671 | if (r->request_body_file_group_access) { 672 | tf->access = 0660; 673 | } 674 | 675 | rb->temp_file = tf; 676 | } 677 | 678 | if (body == NULL) { 679 | return NGX_OK; 680 | } 681 | 682 | /* we need this hack to work around a bug in 683 | * ngx_write_chain_to_temp_file when the chain 684 | * link is longer than 1024, we'll receive 685 | * random alert like this: 686 | * [alert] 13493#0: *1 pread() read only 1024 687 | * of 3603 from 688 | * "/opt/nginx/client_body_temp/0000000001" */ 689 | if (chain_count > MAX_CHUNKS_PER_DISK_WRITE) { 690 | i = 0; 691 | for (cl = body; cl; cl = cl->next) { 692 | if (i >= MAX_CHUNKS_PER_DISK_WRITE) { 693 | /* dd("wrote %d links first...", i+1); */ 694 | 695 | saved_next = cl->next; 696 | cl->next = NULL; 697 | n = ngx_write_chain_to_temp_file(rb->temp_file, body); 698 | /* TODO: n == 0 or not complete and level event */ 699 | 700 | if (n == NGX_ERROR) { 701 | return NGX_ERROR; 702 | } 703 | 704 | rb->temp_file->offset += n; 705 | 706 | cl->next = saved_next; 707 | body = cl->next; 708 | i = 0; 709 | } else { 710 | i++; 711 | } 712 | } 713 | } 714 | 715 | if (body) { 716 | n = ngx_write_chain_to_temp_file(rb->temp_file, body); 717 | /* TODO: n == 0 or not complete and level event */ 718 | 719 | if (n == NGX_ERROR) { 720 | return NGX_ERROR; 721 | } 722 | 723 | rb->temp_file->offset += n; 724 | } 725 | 726 | return NGX_OK; 727 | } 728 | -------------------------------------------------------------------------------- /src/ngx_http_chunkin_request_body.h: -------------------------------------------------------------------------------- 1 | #ifndef NGX_HTTP_CHUNKIN_REQUEST_BODY_H 2 | #define NGX_HTTP_CHUNKIN_REQUEST_BODY_H 3 | 4 | #include 5 | 6 | ngx_int_t ngx_http_chunkin_read_chunked_request_body(ngx_http_request_t *r, 7 | ngx_http_client_body_handler_pt post_handler); 8 | 9 | #endif /* NGX_HTTP_CHUNKIN_REQUEST_BODY_H */ 10 | 11 | -------------------------------------------------------------------------------- /src/ngx_http_chunkin_util.c: -------------------------------------------------------------------------------- 1 | /* Copyright (C) agentzh */ 2 | 3 | #ifndef DDEBUG 4 | #define DDEBUG 0 5 | #endif 6 | #include "ddebug.h" 7 | 8 | #include "ngx_http_chunkin_util.h" 9 | 10 | 11 | static ngx_int_t ngx_http_chunkin_rm_header(ngx_list_t *l, ngx_table_elt_t *h); 12 | static ngx_int_t ngx_http_chunkin_rm_header_helper(ngx_list_t *l, 13 | ngx_list_part_t *cur, ngx_uint_t i); 14 | 15 | 16 | static ngx_str_t ngx_http_chunkin_content_length_header_key = 17 | ngx_string("Content-Length"); 18 | 19 | 20 | void 21 | ngx_http_chunkin_clear_transfer_encoding(ngx_http_request_t *r) 22 | { 23 | ngx_int_t rc; 24 | 25 | if (r->headers_in.transfer_encoding) { 26 | rc = ngx_http_chunkin_rm_header(&r->headers_in.headers, 27 | r->headers_in.transfer_encoding); 28 | 29 | if (rc == NGX_OK) { 30 | r->headers_in.transfer_encoding = NULL; 31 | } else { 32 | /* not found */ 33 | 34 | r->headers_in.transfer_encoding->value.len = 0; 35 | r->headers_in.transfer_encoding->value.data = (u_char *) ""; 36 | r->headers_in.transfer_encoding->hash = 0; 37 | r->headers_in.transfer_encoding = NULL; 38 | } 39 | } 40 | } 41 | 42 | 43 | ngx_int_t 44 | ngx_http_chunkin_set_content_length_header(ngx_http_request_t *r, size_t len) 45 | { 46 | ngx_table_elt_t *h; 47 | ngx_list_part_t *part; 48 | ngx_uint_t i; 49 | 50 | r->headers_in.content_length_n = len; 51 | 52 | part = &r->headers_in.headers.part; 53 | h = part->elts; 54 | 55 | for (i = 0; /* void */; i++) { 56 | 57 | if (i >= part->nelts) { 58 | if (part->next == NULL) { 59 | break; 60 | } 61 | 62 | part = part->next; 63 | h = part->elts; 64 | i = 0; 65 | } 66 | 67 | /* 68 | ngx_log_debug1(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, 69 | "header value [%V]", &h[i].value); 70 | dd("header value (raw): [%s]", h[i].value.data); 71 | */ 72 | 73 | if (h[i].key.len == ngx_http_chunkin_content_length_header_key.len 74 | && ngx_strncasecmp(h[i].key.data, 75 | ngx_http_chunkin_content_length_header_key.data, 76 | h[i].key.len) == 0) 77 | { 78 | dd("Found existing content-length header."); 79 | 80 | h[i].value.data = ngx_palloc(r->pool, NGX_OFF_T_LEN); 81 | 82 | if (h[i].value.data == NULL) { 83 | return NGX_HTTP_INTERNAL_SERVER_ERROR; 84 | } 85 | 86 | h[i].value.len = ngx_sprintf(h[i].value.data, "%O", 87 | r->headers_in.content_length_n) - h[i].value.data; 88 | 89 | return NGX_OK; 90 | } 91 | } 92 | 93 | h = ngx_list_push(&r->headers_in.headers); 94 | 95 | if (h == NULL) { 96 | return NGX_HTTP_INTERNAL_SERVER_ERROR; 97 | } 98 | 99 | h->hash = r->header_hash; 100 | 101 | h->key = ngx_http_chunkin_content_length_header_key; 102 | 103 | h->value.data = ngx_palloc(r->pool, NGX_OFF_T_LEN); 104 | 105 | if (h->value.data == NULL) { 106 | return NGX_HTTP_INTERNAL_SERVER_ERROR; 107 | } 108 | 109 | h->value.len = ngx_sprintf(h->value.data, "%O", 110 | r->headers_in.content_length_n) - h->value.data; 111 | 112 | h->lowcase_key = ngx_pnalloc(r->pool, h->key.len); 113 | if (h->lowcase_key == NULL) { 114 | return NGX_HTTP_INTERNAL_SERVER_ERROR; 115 | } 116 | 117 | ngx_strlow(h->lowcase_key, h->key.data, h->key.len); 118 | 119 | r->headers_in.content_length = h; 120 | 121 | return NGX_OK; 122 | } 123 | 124 | 125 | ngx_chain_t * 126 | ngx_http_chunkin_get_buf(ngx_pool_t *pool, ngx_http_chunkin_ctx_t *ctx) 127 | { 128 | ngx_chain_t *cl; 129 | ngx_chain_t **ll; 130 | ngx_uint_t i; 131 | 132 | cl = ctx->free_bufs; 133 | 134 | if (cl == NULL) { 135 | ll = &ctx->free_bufs; 136 | for (i = 0; i < 4; i++) { 137 | cl = ngx_alloc_chain_link(pool); 138 | if (cl == NULL) { 139 | return NULL; 140 | } 141 | 142 | cl->buf = ngx_calloc_buf(pool); 143 | if (cl->buf == NULL) { 144 | return NULL; 145 | } 146 | 147 | cl->next = NULL; 148 | *ll = cl; 149 | ll = &cl->next; 150 | } 151 | cl = ctx->free_bufs; 152 | } 153 | 154 | if (cl) { 155 | ctx->free_bufs = cl->next; 156 | 157 | cl->buf->shadow = NULL; 158 | cl->next = NULL; 159 | 160 | dd("returned free buf"); 161 | return cl; 162 | } 163 | 164 | dd("allocate new buf"); 165 | 166 | cl = ngx_alloc_chain_link(pool); 167 | if (cl == NULL) { 168 | return NULL; 169 | } 170 | 171 | cl->buf = ngx_calloc_buf(pool); 172 | if (cl->buf == NULL) { 173 | return NULL; 174 | } 175 | 176 | cl->next = NULL; 177 | 178 | /* cl->buf->tag = (ngx_buf_tag_t) &ngx_http_chunkin_filter_module; */ 179 | 180 | return cl; 181 | } 182 | 183 | 184 | /* modified from the ngx_http_internal_redirect function 185 | * in ngx_http_core_module.c of nginx 0.8.29. 186 | * copyrighted by Igor Sysoev. */ 187 | ngx_int_t 188 | ngx_http_chunkin_restart_request(ngx_http_request_t *r, 189 | ngx_http_chunkin_ctx_t *ctx) 190 | { 191 | ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, 192 | "chunkin: restart request: \"%V?%V\"", 193 | &r->uri, &r->args); 194 | 195 | ngx_memzero(r->ctx, sizeof(void *) * ngx_http_max_module); 196 | 197 | ngx_http_set_ctx(r, ctx, ngx_http_chunkin_filter_module); 198 | 199 | r->internal = 0; 200 | /* r->phase_handler = 0; */ 201 | 202 | ngx_http_handler(r); 203 | 204 | #if defined(nginx_version) && nginx_version >= 8011 205 | 206 | dd("DISCARD BODY: %d", (int)r->discard_body); 207 | 208 | if (!ctx->r_discard_body) { 209 | r->main->count++; 210 | } 211 | 212 | #endif 213 | 214 | return NGX_DONE; 215 | } 216 | 217 | 218 | /* mostly a clone of the ngx_http_process_request_header function 219 | * in ngx_http_request.c of nginx 0.8.29. 220 | * copyrighted by Igor Sysoev. */ 221 | ngx_int_t 222 | ngx_http_chunkin_process_request_header(ngx_http_request_t *r) 223 | { 224 | dd("entered process_request_header"); 225 | 226 | if (r->headers_in.host == NULL && r->http_version > NGX_HTTP_VERSION_10) { 227 | ngx_log_error(NGX_LOG_INFO, r->connection->log, 0, 228 | "client sent HTTP/1.1 request without \"Host\" header"); 229 | return NGX_HTTP_BAD_REQUEST; 230 | } 231 | 232 | if (r->headers_in.content_length) { 233 | r->headers_in.content_length_n = 234 | ngx_atoof(r->headers_in.content_length->value.data, 235 | r->headers_in.content_length->value.len); 236 | 237 | if (r->headers_in.content_length_n == NGX_ERROR) { 238 | ngx_log_error(NGX_LOG_INFO, r->connection->log, 0, 239 | "client sent invalid \"Content-Length\" header"); 240 | return NGX_HTTP_LENGTH_REQUIRED; 241 | } 242 | } 243 | 244 | dd("method: %d (%d)", (int)r->method, (int)NGX_HTTP_PUT); 245 | dd("content_length_n: %d (%d)", (int)r->headers_in.content_length_n, -1); 246 | 247 | if (r->method & NGX_HTTP_PUT && r->headers_in.content_length_n == -1) { 248 | ngx_log_error(NGX_LOG_INFO, r->connection->log, 0, 249 | "client sent %V method without \"Content-Length\" header", 250 | &r->method_name); 251 | return NGX_HTTP_LENGTH_REQUIRED; 252 | } 253 | 254 | if (r->method & NGX_HTTP_TRACE) { 255 | ngx_log_error(NGX_LOG_INFO, r->connection->log, 0, 256 | "client sent TRACE method"); 257 | return NGX_HTTP_NOT_ALLOWED; 258 | } 259 | 260 | if (r->headers_in.transfer_encoding 261 | && ngx_strcasestrn(r->headers_in.transfer_encoding->value.data, 262 | "chunked", 7 - 1)) 263 | { 264 | ngx_log_error(NGX_LOG_INFO, r->connection->log, 0, 265 | "client sent \"Transfer-Encoding: chunked\" header"); 266 | return NGX_HTTP_LENGTH_REQUIRED; 267 | } 268 | 269 | if (r->headers_in.connection_type == NGX_HTTP_CONNECTION_KEEP_ALIVE) { 270 | if (r->headers_in.keep_alive) { 271 | r->headers_in.keep_alive_n = 272 | ngx_atotm(r->headers_in.keep_alive->value.data, 273 | r->headers_in.keep_alive->value.len); 274 | } 275 | } 276 | 277 | return NGX_OK; 278 | } 279 | 280 | 281 | /* mostly a clone of the ngx_http_process_request function 282 | * in ngx_http_request.c of nginx 0.8.29. 283 | * copyrighted by Igor Sysoev. */ 284 | ngx_int_t 285 | ngx_http_chunkin_process_request(ngx_http_request_t *r) 286 | { 287 | ngx_connection_t *c; 288 | 289 | c = r->connection; 290 | 291 | if (r->plain_http) { 292 | ngx_log_error(NGX_LOG_INFO, c->log, 0, 293 | "client sent plain HTTP request to HTTPS port"); 294 | /* ngx_http_finalize_request(r, NGX_HTTP_TO_HTTPS); */ 295 | return NGX_HTTP_TO_HTTPS; 296 | } 297 | 298 | #if (NGX_HTTP_SSL) 299 | 300 | if (c->ssl) { 301 | long rc; 302 | X509 *cert; 303 | ngx_http_ssl_srv_conf_t *sscf; 304 | 305 | sscf = ngx_http_get_module_srv_conf(r, ngx_http_ssl_module); 306 | 307 | if (sscf->verify) { 308 | rc = SSL_get_verify_result(c->ssl->connection); 309 | 310 | if (rc != X509_V_OK) { 311 | ngx_log_error(NGX_LOG_INFO, c->log, 0, 312 | "client SSL certificate verify error: (%l:%s)", 313 | rc, X509_verify_cert_error_string(rc)); 314 | 315 | ngx_ssl_remove_cached_session(sscf->ssl.ctx, 316 | (SSL_get0_session(c->ssl->connection))); 317 | 318 | /* ngx_http_finalize_request(r, NGX_HTTPS_CERT_ERROR); */ 319 | return NGX_HTTPS_CERT_ERROR; 320 | } 321 | 322 | if (sscf->verify == 1) { 323 | cert = SSL_get_peer_certificate(c->ssl->connection); 324 | 325 | if (cert == NULL) { 326 | ngx_log_error(NGX_LOG_INFO, c->log, 0, 327 | "client sent no required SSL certificate"); 328 | 329 | ngx_ssl_remove_cached_session(sscf->ssl.ctx, 330 | (SSL_get0_session(c->ssl->connection))); 331 | 332 | /* ngx_http_finalize_request(r, NGX_HTTPS_NO_CERT); */ 333 | return NGX_HTTPS_NO_CERT; 334 | } 335 | 336 | X509_free(cert); 337 | } 338 | } 339 | } 340 | 341 | #endif 342 | 343 | if (c->read->timer_set) { 344 | ngx_del_timer(c->read); 345 | } 346 | 347 | #if (NGX_STAT_STUB) 348 | (void) ngx_atomic_fetch_add(ngx_stat_reading, -1); 349 | r->stat_reading = 0; 350 | (void) ngx_atomic_fetch_add(ngx_stat_writing, 1); 351 | r->stat_writing = 1; 352 | #endif 353 | 354 | return NGX_OK; 355 | } 356 | 357 | 358 | ngx_int_t 359 | ngx_http_chunkin_internal_redirect(ngx_http_request_t *r, 360 | ngx_str_t *uri, ngx_str_t *args, ngx_http_chunkin_ctx_t *ctx) 361 | { 362 | ngx_http_core_srv_conf_t *cscf; 363 | 364 | r->uri_changes--; 365 | 366 | if (r->uri_changes == 0) { 367 | ngx_log_error(NGX_LOG_ERR, r->connection->log, 0, 368 | "rewrite or internal redirection cycle " 369 | "while internal redirect to \"%V\"", uri); 370 | 371 | #if defined(nginx_version) && nginx_version >= 8011 372 | 373 | r->main->count++; 374 | 375 | #endif 376 | 377 | ngx_http_finalize_request(r, NGX_HTTP_INTERNAL_SERVER_ERROR); 378 | return NGX_DONE; 379 | } 380 | 381 | r->uri = *uri; 382 | 383 | if (args) { 384 | r->args = *args; 385 | 386 | } else { 387 | r->args.len = 0; 388 | r->args.data = NULL; 389 | } 390 | 391 | ngx_log_debug2(NGX_LOG_DEBUG_HTTP, r->connection->log, 0, 392 | "internal redirect: \"%V?%V\"", uri, &r->args); 393 | 394 | ngx_http_set_exten(r); 395 | 396 | /* clear the modules contexts */ 397 | ngx_memzero(r->ctx, sizeof(void *) * ngx_http_max_module); 398 | 399 | ngx_http_set_ctx(r, ctx, ngx_http_chunkin_filter_module); 400 | 401 | cscf = ngx_http_get_module_srv_conf(r, ngx_http_core_module); 402 | r->loc_conf = cscf->ctx->loc_conf; 403 | 404 | ngx_http_update_location_config(r); 405 | 406 | #if (NGX_HTTP_CACHE) 407 | r->cache = NULL; 408 | #endif 409 | 410 | r->internal = 0; 411 | 412 | #if defined(nginx_version) && nginx_version >= 8011 413 | 414 | dd("DISCARD BODY: %d", (int)r->discard_body); 415 | 416 | if (!ctx->r_discard_body) { 417 | r->main->count++; 418 | } 419 | 420 | #endif 421 | 422 | ngx_http_handler(r); 423 | 424 | return NGX_DONE; 425 | } 426 | 427 | 428 | static ngx_int_t 429 | ngx_http_chunkin_rm_header(ngx_list_t *l, ngx_table_elt_t *h) 430 | { 431 | ngx_uint_t i; 432 | ngx_list_part_t *part; 433 | ngx_table_elt_t *data; 434 | 435 | part = &l->part; 436 | data = part->elts; 437 | 438 | for (i = 0; /* void */; i++) { 439 | dd("i: %d, part: %p", (int) i, part); 440 | 441 | if (i >= part->nelts) { 442 | if (part->next == NULL) { 443 | break; 444 | } 445 | 446 | part = part->next; 447 | data = part->elts; 448 | i = 0; 449 | } 450 | 451 | if (&data[i] == h) { 452 | dd("found header"); 453 | 454 | return ngx_http_chunkin_rm_header_helper(l, part, i); 455 | } 456 | } 457 | 458 | return NGX_ERROR; 459 | } 460 | 461 | 462 | static ngx_int_t 463 | ngx_http_chunkin_rm_header_helper(ngx_list_t *l, ngx_list_part_t *cur, 464 | ngx_uint_t i) 465 | { 466 | ngx_table_elt_t *data; 467 | ngx_list_part_t *new, *part; 468 | 469 | dd("list rm item: part %p, i %d, nalloc %d", cur, (int) i, 470 | (int) l->nalloc); 471 | 472 | data = cur->elts; 473 | 474 | dd("cur: nelts %d, nalloc %d", (int) cur->nelts, 475 | (int) l->nalloc); 476 | 477 | if (i == 0) { 478 | cur->elts = (char *) cur->elts + l->size; 479 | cur->nelts--; 480 | 481 | if (cur == l->last) { 482 | if (l->nalloc > 1) { 483 | l->nalloc--; 484 | return NGX_OK; 485 | } 486 | 487 | /* l->nalloc == 1 */ 488 | 489 | part = &l->part; 490 | while (part->next != cur) { 491 | if (part->next == NULL) { 492 | return NGX_ERROR; 493 | } 494 | part = part->next; 495 | } 496 | 497 | part->next = NULL; 498 | l->last = part; 499 | 500 | return NGX_OK; 501 | } 502 | 503 | if (cur->nelts == 0) { 504 | part = &l->part; 505 | while (part->next != cur) { 506 | if (part->next == NULL) { 507 | return NGX_ERROR; 508 | } 509 | part = part->next; 510 | } 511 | 512 | part->next = cur->next; 513 | 514 | return NGX_OK; 515 | } 516 | 517 | return NGX_OK; 518 | } 519 | 520 | if (i == cur->nelts - 1) { 521 | cur->nelts--; 522 | 523 | if (cur == l->last) { 524 | l->nalloc--; 525 | } 526 | 527 | return NGX_OK; 528 | } 529 | 530 | new = ngx_palloc(l->pool, sizeof(ngx_list_part_t)); 531 | if (new == NULL) { 532 | return NGX_ERROR; 533 | } 534 | 535 | new->elts = &data[i + 1]; 536 | new->nelts = cur->nelts - i - 1; 537 | new->next = cur->next; 538 | 539 | l->nalloc = new->nelts; 540 | 541 | cur->nelts = i; 542 | cur->next = new; 543 | if (cur == l->last) { 544 | l->last = new; 545 | } 546 | 547 | return NGX_OK; 548 | } 549 | 550 | -------------------------------------------------------------------------------- /src/ngx_http_chunkin_util.h: -------------------------------------------------------------------------------- 1 | #ifndef NGX_HTTP_CHUNKIN_UTIL_H 2 | #define NGX_HTTP_CHUNKIN_UTIL_H 3 | 4 | #include 5 | #include "ngx_http_chunkin_filter_module.h" 6 | 7 | void ngx_http_chunkin_clear_transfer_encoding(ngx_http_request_t *r); 8 | 9 | ngx_int_t ngx_http_chunkin_set_content_length_header(ngx_http_request_t *r, 10 | size_t len); 11 | 12 | ngx_chain_t * ngx_http_chunkin_get_buf(ngx_pool_t *pool, 13 | ngx_http_chunkin_ctx_t *ctx); 14 | 15 | ngx_int_t ngx_http_chunkin_restart_request(ngx_http_request_t *r, 16 | ngx_http_chunkin_ctx_t *ctx); 17 | 18 | ngx_int_t ngx_http_chunkin_process_request_header(ngx_http_request_t *r); 19 | 20 | ngx_int_t ngx_http_chunkin_process_request(ngx_http_request_t *r); 21 | 22 | ngx_int_t ngx_http_chunkin_internal_redirect(ngx_http_request_t *r, 23 | ngx_str_t *uri, ngx_str_t *args, ngx_http_chunkin_ctx_t *ctx); 24 | 25 | #endif /* NGX_HTTP_CHUNKIN_UTIL_H */ 26 | 27 | -------------------------------------------------------------------------------- /t/bug.t: -------------------------------------------------------------------------------- 1 | # vi:filetype= 2 | 3 | use lib 't/lib'; 4 | use Test::Nginx::Socket::Chunkin; 5 | 6 | repeat_each(3); 7 | #warn 'repeat each: ', repeat_each, "\n"; 8 | 9 | plan tests => repeat_each() * 2 * blocks(); 10 | 11 | no_diff; 12 | 13 | run_tests(); 14 | 15 | __DATA__ 16 | 17 | === TEST 1: binary in data 18 | --- config 19 | chunkin on; 20 | location /ar.do { 21 | echo_request_body; 22 | } 23 | --- more_headers 24 | Transfer-Encoding: chunked 25 | --- request eval 26 | "POST /ar.do 27 | 1b9\r 28 | ".'%Q`3�BBT0123456789AF%Q_�k%Q_�kwtk-emulatorSunMicrosystems_wtk AX7`encoding=pcm encoding=pcm&rate=8000&bits=8&channels=1 encoding=pcm&rate=22050&bits=16&chanZach-Laptop-W1.01.0en-US1.01.11.11.0SunMicrosystems_wtMIDP-2.11.0.10H,1Haudio/x-wavtruetruetruetruenencoding=rgb565&width=160&height=120 encoding=rgb565&width=320&height=240 encoding=rgb565&width=120&height=160encoding=jpeg encoding=png+12345678900http://mmsc.127.0.0.01'."\r 29 | 0\r 30 | \r 31 | " 32 | --- response_body eval 33 | '%Q`3�BBT0123456789AF%Q_�k%Q_�kwtk-emulatorSunMicrosystems_wtk AX7`encoding=pcm encoding=pcm&rate=8000&bits=8&channels=1 encoding=pcm&rate=22050&bits=16&chanZach-Laptop-W1.01.0en-US1.01.11.11.0SunMicrosystems_wtMIDP-2.11.0.10H,1Haudio/x-wavtruetruetruetruenencoding=rgb565&width=160&height=120 encoding=rgb565&width=320&height=240 encoding=rgb565&width=120&height=160encoding=jpeg encoding=png+12345678900http://mmsc.127.0.0.01' 34 | 35 | 36 | 37 | === TEST 2: CRLF in data (missing trailing CRLF) 38 | --- config 39 | chunkin on; 40 | location /ar.do { 41 | echo_request_body; 42 | } 43 | --- more_headers 44 | Transfer-Encoding: chunked 45 | --- request eval 46 | "POST /ar.do 47 | 2\r 48 | \r 49 | 0\r 50 | \r 51 | " 52 | --- response_body_like: 400 Bad Request 53 | --- error_code: 400 54 | 55 | 56 | 57 | === TEST 3: CRLF in data 58 | --- config 59 | chunkin on; 60 | location /ar.do { 61 | echo_request_body; 62 | } 63 | --- more_headers 64 | Transfer-Encoding: chunked 65 | --- request eval 66 | "POST /ar.do 67 | 2\r 68 | \r 69 | \r 70 | 0\r 71 | \r 72 | " 73 | --- response_body eval 74 | "\r\n" 75 | 76 | 77 | 78 | === TEST 4: leading CRLF in data 79 | --- config 80 | chunkin on; 81 | location /ar.do { 82 | echo_request_body; 83 | } 84 | --- more_headers 85 | Transfer-Encoding: chunked 86 | --- request eval 87 | "POST /ar.do 88 | 4\r 89 | \r 90 | ab\r 91 | 0\r 92 | \r 93 | " 94 | --- response_body eval 95 | "\r\nab" 96 | 97 | 98 | 99 | === TEST 5: trailing CRLF in data 100 | --- config 101 | chunkin on; 102 | location /ar.do { 103 | echo_request_body; 104 | } 105 | --- more_headers 106 | Transfer-Encoding: chunked 107 | --- request eval 108 | "POST /ar.do 109 | 4\r 110 | ab\r 111 | \r 112 | 0\r 113 | \r 114 | " 115 | --- response_body eval 116 | "ab\r\n" 117 | 118 | 119 | 120 | === TEST 6: embedded CRLF in data 121 | --- config 122 | chunkin on; 123 | location /ar.do { 124 | echo_request_body; 125 | } 126 | --- more_headers 127 | Transfer-Encoding: chunked 128 | --- request eval 129 | "POST /ar.do 130 | 6\r 131 | ab\r 132 | cd\r 133 | 0\r 134 | \r 135 | " 136 | --- response_body eval 137 | "ab\r\ncd" 138 | 139 | 140 | 141 | === TEST 7: 413 in proxy 142 | --- config 143 | chunkin on; 144 | location /main { 145 | proxy_pass $scheme://127.0.0.1:$server_port/proxy; 146 | } 147 | location /proxy { 148 | return 411; 149 | } 150 | --- more_headers 151 | Transfer-Encoding: chunked 152 | --- request eval 153 | "POST /main 154 | 2\r 155 | ab\r 156 | 0\r 157 | \r 158 | " 159 | --- response_body_like: 411 Length Required 160 | --- error_code: 411 161 | 162 | 163 | 164 | === TEST 8: padding spaces 165 | --- config 166 | chunkin on; 167 | location /ar.do { 168 | echo_request_body; 169 | } 170 | --- more_headers 171 | Transfer-Encoding: chunked 172 | --- request eval 173 | "POST /ar.do 174 | 6 \r 175 | ab\r 176 | cd\r 177 | 0\r 178 | \r 179 | " 180 | --- response_body eval 181 | "ab\r\ncd" 182 | 183 | 184 | 185 | === TEST 9: padding spaces (using HT) 186 | --- config 187 | chunkin on; 188 | location /ar.do { 189 | echo_request_body; 190 | } 191 | --- more_headers 192 | Transfer-Encoding: chunked 193 | --- request eval 194 | "POST /ar.do 195 | 6 \t\r 196 | ab\r 197 | cd\r 198 | 0\r 199 | \r 200 | " 201 | --- response_body eval 202 | "ab\r\ncd" 203 | 204 | 205 | 206 | === TEST 10: padding spaces (using \t and ' ' in last chunk) 207 | --- config 208 | chunkin on; 209 | location /ar.do { 210 | echo_request_body; 211 | } 212 | --- more_headers 213 | Transfer-Encoding: chunked 214 | --- request eval 215 | "POST /ar.do 216 | 6 \t\r 217 | ab\r 218 | cd\r 219 | 0\t \r 220 | \r 221 | " 222 | --- response_body eval 223 | "ab\r\ncd" 224 | 225 | 226 | 227 | === TEST 11: padding LWS (using \t and ' ' with a leading CRLF in last chunk) 228 | --- config 229 | chunkin on; 230 | location /ar.do { 231 | echo_request_body; 232 | } 233 | --- more_headers 234 | Transfer-Encoding: chunked 235 | --- request eval 236 | "POST /ar.do 237 | 6\r 238 | \r 239 | \t\r 240 | ab\r 241 | cd\r 242 | 0\t\r 243 | \r 244 | \r 245 | \r 246 | " 247 | --- response_body_like: 400 Bad Request 248 | --- error_code: 400 249 | 250 | 251 | 252 | === TEST 12: leading CRLF 253 | --- config 254 | chunkin on; 255 | location /ar.do { 256 | echo_request_body; 257 | } 258 | --- more_headers 259 | Transfer-Encoding: chunked 260 | --- request eval 261 | "POST /ar.do 262 | \r 263 | 6\r 264 | ab\r 265 | cd\r 266 | 0\r 267 | \r 268 | " 269 | --- response_body_like: 400 Bad Request 270 | --- error_code: 400 271 | 272 | 273 | 274 | === TEST 13: zero-padding 275 | --- config 276 | chunkin on; 277 | location /ar.do { 278 | echo_request_body; 279 | } 280 | --- more_headers 281 | Transfer-Encoding: chunked 282 | --- request eval 283 | "POST /ar.do 284 | 0006\r 285 | ab\r 286 | cd\r 287 | 0\r 288 | \r 289 | " 290 | --- response_body eval 291 | "ab\r\ncd" 292 | 293 | 294 | 295 | === TEST 14: leading new lines 296 | --- config 297 | chunkin on; 298 | location /ar.do { 299 | echo_request_body; 300 | } 301 | --- more_headers 302 | Transfer-Encoding: chunked 303 | --- request eval 304 | "POST /ar.do 305 | \n \11\r 306 | 0006\r 307 | ab\r 308 | cd\r 309 | 0\r 310 | \r 311 | " 312 | --- response_body_like: 400 Bad Request 313 | --- error_code: 400 314 | 315 | 316 | 317 | === TEST 15: internal guard 318 | --- config 319 | chunkin on; 320 | location /ar.do { 321 | internal; 322 | echo_request_body; 323 | } 324 | --- more_headers 325 | Transfer-Encoding: chunked 326 | --- request eval 327 | "POST /ar.do 328 | \n \11\r 329 | 0006\r 330 | ab\r 331 | cd\r 332 | 0\r 333 | \r 334 | " 335 | --- response_body_like: 404 Not Found 336 | --- error_code: 404 337 | 338 | 339 | 340 | === TEST 16: phase issue 341 | --- config 342 | chunkin on; 343 | location /ar.do { 344 | deny all; 345 | echo_request_body; 346 | } 347 | --- more_headers 348 | Transfer-Encoding: chunked 349 | --- request eval 350 | "POST /ar.do 351 | 1\r 352 | a\r 353 | 0\r 354 | \r 355 | " 356 | --- response_body_like: 403 Forbidden 357 | --- error_code: 403 358 | 359 | 360 | 361 | === TEST 17: contenth-length AND chunked 362 | --- config 363 | chunkin on; 364 | location /aar.do { 365 | echo_request_body; 366 | } 367 | --- more_headers 368 | Transfer-Encoding: chunked 369 | --- raw_request eval 370 | "POST /aar.do HTTP/1.1\r 371 | Host: data.test.com\r 372 | Content-Type:application/octet-stream\r 373 | transfer-encoding:chunked\r 374 | User-Agent: SEC-SGHD840/1.0 NetFront/3.2 Profile/MIDP-2.0 Configuration/CLDC-1.1 UNTRUSTED/1.0\r 375 | Content-Length: 6263\r 376 | Via: ZXWAP GateWay,ZTE Technologies\r 377 | x-up-calling-line-id: 841223657459\r 378 | Connection: close\r 379 | \r 380 | 5\r 381 | hello\r 382 | 0\r 383 | \r 384 | " 385 | --- response_body: hello 386 | --- timeout: 1 387 | 388 | 389 | 390 | === TEST 18: Content-length AND chunked 391 | --- config 392 | chunkin on; 393 | location /aar.do { 394 | echo_request_body; 395 | } 396 | --- raw_request eval 397 | ["POST /aar.do HTTP/1.1\r 398 | Host: data.test.com\r 399 | Content-Type:application/octet-stream\r 400 | transfer-encoding:chunked\r 401 | User-Agent: SEC-SGHD840/1.0 NetFront/3.2 Profile/MIDP-2.0 Configuration/CLDC-1.1 UNTRUSTED/1.0\r 402 | Content-Length: 6263\r 403 | Via: ZXWAP GateWay,ZTE Technologies\r 404 | x-up-calling-line-id: 841223657459\r 405 | Connection: close\r 406 | \r 407 | 5\r 408 | ", "hell","o\r 409 | ", "0\r 410 | \r 411 | "] 412 | --- raw_request_middle_delay: 0.001 413 | --- response_body: hello 414 | --- timeout: 4 415 | 416 | 417 | 418 | === TEST 19: Content-length AND chunked (ready for the read_discard_request_body to work) 419 | --- config 420 | chunkin on; 421 | location /aar.do { 422 | echo_request_body; 423 | } 424 | --- raw_request eval 425 | ["POST /aar.do HTTP/1.1\r 426 | Host: data.test.com\r 427 | Content-Type:application/octet-stream\r 428 | transfer-encoding:chunked\r 429 | User-Agent: SEC-SGHD840/1.0 NetFront/3.2 Profile/MIDP-2.0 Configuration/CLDC-1.1 UNTRUSTED/1.0\r 430 | Content-Length: 6263\r 431 | Via: ZXWAP GateWay,ZTE Technologies\r 432 | x-up-calling-line-id: 841223657459\r 433 | Connection: close\r 434 | \r 435 | 5\r 436 | ", "hell","o\r 437 | ", "0\r 438 | \r 439 | "] 440 | --- raw_request_middle_delay: 0 441 | --- response_body: hello 442 | --- timeout: 1 443 | 444 | 445 | 446 | === TEST 20: packets in a single buf 447 | --- config 448 | chunkin on; 449 | location /aar.do { 450 | client_body_buffer_size 1m; 451 | echo_request_body; 452 | } 453 | --- raw_request eval 454 | ["POST /aar.do HTTP/1.1\r 455 | Content-Type: application/octet-stream\r 456 | Connection: close\r 457 | Host: data.test.com\r 458 | Transfer-Encoding: chunked\r 459 | User-Agent: SonyEricssonW395/R1BA010 Profile/MIDP-2.1 Configuration/CLDC-1.1 UNTRUSTED/1.0\r 460 | \r 461 | ", 462 | "535\r 463 | ".('a' x 1228), 464 | ('a' x 107), "\r 465 | ", "0\r 466 | \r 467 | "] 468 | --- raw_request_middle_delay: 0.002 469 | --- response_body: hello 470 | --- timeout: 2 471 | --- SKIP 472 | 473 | 474 | 475 | === TEST 21: packets in a single buf 476 | --- config 477 | chunkin on; 478 | location /aar.do { 479 | client_body_buffer_size 1m; 480 | echo_request_body; 481 | } 482 | --- raw_request eval 483 | ["POST /aar.do HTTP/1.1\r 484 | Content-Type: application/octet-stream\r 485 | Connection: close\r 486 | Host: data.test.com\r 487 | Transfer-Encoding: chunked\r 488 | User-Agent: SonyEricssonW395/R1BA010 Profile/MIDP-2.1 Configuration/CLDC-1.1 UNTRUSTED/1.0\r 489 | \r 490 | ", 491 | "535\r 492 | ".('a' x 1228), 493 | ('a' x 105) . "\r 494 | ", "0\r 495 | \r 496 | "] 497 | --- raw_request_middle_delay: 0.002 498 | --- response_body eval 499 | 'a' x 0x535 500 | --- timeout: 5 501 | 502 | 503 | 504 | === TEST 22: packets in a single buf 505 | --- config 506 | chunkin on; 507 | location /aar.do { 508 | client_body_buffer_size 1m; 509 | echo_request_body; 510 | } 511 | --- raw_request eval 512 | ["POST /aar.do HTTP/1.1\r 513 | Content-Type: application/octet-stream\r 514 | Connection: close\r 515 | Host: data.test.com\r 516 | Transfer-Encoding: chunked\r 517 | User-Agent: SonyEricssonW395/R1BA010 Profile/MIDP-2.1 Configuration/CLDC-1.1 UNTRUSTED/1.0\r 518 | \r 519 | ", 520 | "7e0\r 521 | ".('a' x 2016)."\r", 522 | "aaaa 523 | ", "0\r 524 | \r 525 | "] 526 | --- raw_request_middle_delay: 0.002 527 | --- response_body eval 528 | 'a' x 0x7e0 529 | --- timeout: 2 530 | --- SKIP 531 | 532 | 533 | 534 | === TEST 23: not exceeding max body limit (chunk spanning preread and rb->buf) 535 | --- config 536 | chunkin on; 537 | location /main { 538 | client_body_buffer_size 10m; 539 | client_max_body_size 10m; 540 | chunkin_max_chunks_per_buf 2048; 541 | 542 | echo_read_request_body; 543 | 544 | echo_request_body; 545 | echo; 546 | } 547 | --- more_headers 548 | #Transfer-Encoding: chunked 549 | --- request eval 550 | "POST /main 551 | ".("a" x (1 * 1024 * 1024)) 552 | --- response_body eval 553 | "a" x (1 * 1024 * 1024) 554 | --- timeout: 60 555 | --- SKIP 556 | 557 | 558 | 559 | === TEST 24: binary in data 560 | --- config 561 | chunkin on; 562 | location /ar.do { 563 | echo $request_method; 564 | echo_request_body; 565 | } 566 | --- more_headers 567 | Transfer-Encoding: chunked 568 | --- request eval 569 | "DELETE /ar.do 570 | 5\r 571 | hello\r 572 | 0\r 573 | \r 574 | " 575 | --- response_body chop 576 | DELETE 577 | hello 578 | 579 | 580 | 581 | === TEST 25: CR LF across the boundary 582 | --- config 583 | chunkin on; 584 | location /ar.do { 585 | echo $request_method; 586 | echo_request_body; 587 | } 588 | --- more_headers 589 | Transfer-Encoding: chunked 590 | --- request eval 591 | "DELETE /ar.do 592 | 5\r 593 | hell\r\r 594 | 0\r 595 | \r 596 | " 597 | --- response_body eval 598 | "DELETE 599 | hell\r" 600 | 601 | 602 | 603 | === TEST 26: no contenth-length nor chunked (PUT) 604 | --- config 605 | chunkin on; 606 | location /aar.do { 607 | echo_request_body; 608 | } 609 | --- raw_request eval 610 | "PUT /aar.do HTTP/1.1\r 611 | Host: data.test.com\r 612 | Content-Type:application/octet-stream\r 613 | User-Agent: SEC-SGHD840/1.0 NetFront/3.2 Profile/MIDP-2.0 Configuration/CLDC-1.1 UNTRUSTED/1.0\r 614 | Via: ZXWAP GateWay,ZTE Technologies\r 615 | x-up-calling-line-id: 841223657459\r 616 | Connection: close\r 617 | \r 618 | 5\r 619 | hello\r 620 | 0\r 621 | \r 622 | " 623 | --- response_body_like: 411 Length Required 624 | --- error_code: 411 625 | 626 | 627 | 628 | === TEST 27: no contenth-length nor chunked (POST) 629 | --- config 630 | chunkin on; 631 | location /aar.do { 632 | echo_request_body; 633 | } 634 | --- raw_request eval 635 | "POST /aar.do HTTP/1.1\r 636 | Host: data.test.com\r 637 | Content-Type:application/octet-stream\r 638 | User-Agent: SEC-SGHD840/1.0 NetFront/3.2 Profile/MIDP-2.0 Configuration/CLDC-1.1 UNTRUSTED/1.0\r 639 | Via: ZXWAP GateWay,ZTE Technologies\r 640 | x-up-calling-line-id: 841223657459\r 641 | Connection: close\r 642 | \r 643 | 5\r 644 | hello\r 645 | 0\r 646 | \r 647 | " 648 | --- response_body_like: 411 Length Required 649 | --- error_code: 411 650 | --- SKIP 651 | 652 | 653 | 654 | === TEST 28: did not work with subrequests 655 | --- config 656 | chunkin on; 657 | 658 | error_page 411 = @my_411_error; 659 | location @my_411_error { 660 | chunkin_resume; 661 | } 662 | 663 | location = /t { 664 | content_by_lua_file html/a.lua; 665 | } 666 | 667 | location = /sub { 668 | echo hello world; 669 | } 670 | --- user_files 671 | >>> a.lua 672 | ngx.req.read_body() 673 | local res = ngx.location.capture("/sub") 674 | ngx.say("sr: ", res.status) 675 | --- raw_request eval 676 | "POST /t HTTP/1.1\r 677 | Host: localhost\r 678 | Transfer-Encoding: chunked\r 679 | \r 680 | 5\r 681 | hello\r 682 | 0\r 683 | \r 684 | " 685 | --- more_headers 686 | Transfer-Encoding: chunked 687 | --- stap2 688 | F(ngx_http_finalize_request) { 689 | if ($r->main->count >= 3) { 690 | printf("============ %d\n", $r->main->count) 691 | print_ubacktrace() 692 | } 693 | } 694 | --- response_body_body 695 | sr: 200 696 | --- no_error_log 697 | [error] 698 | 699 | -------------------------------------------------------------------------------- /t/error.t: -------------------------------------------------------------------------------- 1 | # vi:filetype= 2 | 3 | use lib 't/lib'; 4 | use Test::Nginx::Socket::Chunkin; 5 | 6 | plan tests => repeat_each() * 2 * blocks(); 7 | 8 | no_diff; 9 | 10 | run_tests(); 11 | 12 | __DATA__ 13 | 14 | === TEST 1: sanity 15 | --- config 16 | location /main { 17 | echo hi; 18 | } 19 | --- request 20 | GET /main 21 | --- response_body 22 | hi 23 | 24 | 25 | 26 | === TEST 2: good chunked body 27 | --- config 28 | chunkin on; 29 | location /main { 30 | echo_request_body; 31 | } 32 | --- more_headers 33 | Transfer-Encoding: chunked 34 | --- request eval 35 | "POST /main 36 | 5\r 37 | hello\r 38 | 0\r 39 | \r 40 | " 41 | --- response_body chomp 42 | hello 43 | 44 | 45 | 46 | === TEST 3: chunk size too small 47 | --- config 48 | chunkin on; 49 | location /main { 50 | echo_request_body; 51 | } 52 | --- more_headers 53 | Transfer-Encoding: chunked 54 | --- request eval 55 | "POST /main 56 | 4\r 57 | hello\r 58 | 0\r 59 | \r 60 | " 61 | --- error_code: 400 62 | --- response_body_like: 400 Bad Request 63 | 64 | 65 | 66 | === TEST 4: chunk size too big 67 | --- config 68 | chunkin on; 69 | location /main { 70 | echo_request_body; 71 | } 72 | --- more_headers 73 | Transfer-Encoding: chunked 74 | --- request eval 75 | "POST /main 76 | 6\r 77 | hello\r 78 | 0\r 79 | \r 80 | " 81 | --- response_body_like: 400 Bad Request 82 | --- error_code: 400 83 | 84 | 85 | 86 | === TEST 5: chunk size even bigger 87 | --- config 88 | chunkin on; 89 | location /main { 90 | echo_request_body; 91 | } 92 | --- more_headers 93 | Transfer-Encoding: chunked 94 | --- request eval 95 | "POST /main 96 | 7\r 97 | hello\r 98 | 0\r 99 | \r 100 | " 101 | --- response_body_like: 400 Bad Request 102 | --- error_code: 400 103 | 104 | 105 | 106 | === TEST 6: chunk size WAY too big and rejected by ragel DFA 107 | --- config 108 | chunkin on; 109 | location /main { 110 | echo_request_body; 111 | } 112 | --- more_headers 113 | Transfer-Encoding: chunked 114 | --- request eval 115 | "POST /main 116 | 8\r 117 | hello\r 118 | 0\r 119 | \r 120 | " 121 | --- error_code: 400 122 | --- response_body_like: 400 Bad Request 123 | 124 | 125 | 126 | === TEST 7: missing LF after data chunk 127 | --- config 128 | chunkin on; 129 | location /main { 130 | echo_request_body; 131 | } 132 | --- more_headers 133 | Transfer-Encoding: chunked 134 | --- request eval 135 | "POST /main 136 | 5\r 137 | hello\r0\r 138 | \r 139 | " 140 | --- error_code: 400 141 | --- response_body_like: 400 Bad Request 142 | 143 | 144 | 145 | === TEST 8: missing CR after data chunk 146 | --- config 147 | chunkin on; 148 | location /main { 149 | echo_request_body; 150 | } 151 | --- more_headers 152 | Transfer-Encoding: chunked 153 | --- request eval 154 | "POST /main 155 | 5\r 156 | hello 157 | 0\r 158 | \r 159 | " 160 | --- error_code: 400 161 | --- response_body_like: 400 Bad Request 162 | 163 | 164 | 165 | === TEST 9: missing CRLF after data chunk 166 | --- config 167 | chunkin on; 168 | location /main { 169 | echo_request_body; 170 | } 171 | --- more_headers 172 | Transfer-Encoding: chunked 173 | --- request eval 174 | "POST /main 175 | 5\r 176 | hello0\r 177 | \r 178 | " 179 | --- error_code: 400 180 | --- response_body_like: 400 Bad Request 181 | 182 | 183 | 184 | === TEST 10: 2 zero chunks 185 | --- config 186 | chunkin on; 187 | location /main { 188 | echo_request_body; 189 | } 190 | --- more_headers 191 | Transfer-Encoding: chunked 192 | --- request eval 193 | "POST /main 194 | 0\r 195 | \r 196 | 0\r 197 | \r 198 | " 199 | --- response_body 200 | 201 | 202 | 203 | === TEST 11: 1 00 chunk and 1 zero chunk 204 | --- config 205 | chunkin on; 206 | location /main { 207 | #echo "length: $http_content_length"; 208 | echo_request_body; 209 | } 210 | --- more_headers 211 | Transfer-Encoding: chunked 212 | --- request eval 213 | "POST /main 214 | 00\r 215 | \r 216 | 0\r 217 | \r 218 | " 219 | --- response_body 220 | 221 | 222 | 223 | === TEST 12: 1 00 chunk and 1 zero chunk 224 | --- config 225 | chunkin on; 226 | location /main { 227 | echo_request_body; 228 | } 229 | --- more_headers 230 | Transfer-Encoding: chunked 231 | --- request eval 232 | "POST /main 233 | 10\r 234 | helloworld,hello\r 235 | 00\r 236 | \r 237 | 0\r 238 | \r 239 | " 240 | --- response_body: helloworld,hello 241 | 242 | 243 | 244 | === TEST 13: bad chunk size 245 | --- config 246 | chunkin on; 247 | location /main { 248 | echo_request_body; 249 | } 250 | --- more_headers 251 | Transfer-Encoding: chunked 252 | --- request eval 253 | "POST /main 254 | zc\r 255 | hello\r 256 | 0\r 257 | \r 258 | " 259 | --- response_body_like: 400 Bad Request 260 | --- error_code: 400 261 | 262 | 263 | 264 | === TEST 14: bad chunk size in the 2nd chunk 265 | --- config 266 | chunkin on; 267 | location /main { 268 | echo_request_body; 269 | } 270 | --- more_headers 271 | Transfer-Encoding: chunked 272 | User-Agent: Java Browser 273 | --- request eval 274 | "POST /main 275 | 1\r 276 | a\r 277 | zc\r 278 | hello\r 279 | 0\r 280 | \r 281 | " 282 | --- response_body_like: 400 Bad Request 283 | --- error_code: 400 284 | 285 | 286 | 287 | === TEST 15: error near the end of big chunks 288 | --- config 289 | chunkin on; 290 | location /main { 291 | echo_request_body; 292 | } 293 | --- more_headers 294 | Transfer-Encoding: chunked 295 | --- request eval 296 | "POST /main 297 | 800\r 298 | ".('a'x2047)."\r 299 | 0\r 300 | \r 301 | " 302 | --- response_body_like: 400 Bad Request 303 | --- error_code: 400 304 | --- timeout: 10 305 | 306 | -------------------------------------------------------------------------------- /t/ext.t: -------------------------------------------------------------------------------- 1 | # vi:filetype= 2 | 3 | use lib 't/lib'; 4 | use Test::Nginx::Socket::Chunkin; 5 | 6 | plan tests => repeat_each() * 2 * blocks(); 7 | 8 | #no_diff; 9 | 10 | run_tests(); 11 | 12 | __DATA__ 13 | 14 | === TEST 1: bad ext (missing leading ;) 15 | --- config 16 | chunkin on; 17 | location /main { 18 | echo_request_body; 19 | } 20 | --- more_headers 21 | Transfer-Encoding: chunked 22 | --- request eval 23 | "POST /main 24 | 3 a=b\r 25 | abc\r 26 | 0\r 27 | \r 28 | " 29 | --- response_body_like: 400 Bad Request 30 | --- error_code: 400 31 | 32 | 33 | 34 | === TEST 2: sanity 35 | --- config 36 | chunkin on; 37 | location /main { 38 | echo_request_body; 39 | } 40 | --- more_headers 41 | Transfer-Encoding: chunked 42 | --- request eval 43 | "POST /main 44 | 3;a=b\r 45 | abc\r 46 | 0\r 47 | \r 48 | " 49 | --- response_body: abc 50 | 51 | 52 | 53 | === TEST 3: with spaces 54 | --- config 55 | chunkin on; 56 | location /main { 57 | echo_request_body; 58 | } 59 | --- more_headers 60 | Transfer-Encoding: chunked 61 | --- request eval 62 | "POST /main 63 | 3 ;\t a\t = b\r 64 | abc\r 65 | 0\r 66 | \r 67 | " 68 | --- response_body: abc 69 | 70 | 71 | 72 | === TEST 4: ext with out val 73 | --- config 74 | chunkin on; 75 | location /main { 76 | echo_request_body; 77 | } 78 | --- more_headers 79 | Transfer-Encoding: chunked 80 | --- request eval 81 | "POST /main 82 | 3 ;\t foo\t \r 83 | abc\r 84 | 0\r 85 | \r 86 | " 87 | --- response_body: abc 88 | 89 | 90 | 91 | === TEST 5: multiple exts 92 | --- config 93 | chunkin on; 94 | location /main { 95 | echo_request_body; 96 | } 97 | --- more_headers 98 | Transfer-Encoding: chunked 99 | --- request eval 100 | "POST /main 101 | 3 ;\t foo = bar; blah = ".'"hello\\\"!"'."\t \r 102 | abc\r 103 | 0\r 104 | \r 105 | " 106 | --- response_body: abc 107 | 108 | -------------------------------------------------------------------------------- /t/lib/Test/Nginx/LWP/Chunkin.pm: -------------------------------------------------------------------------------- 1 | package Test::Nginx::LWP::Chunkin; 2 | 3 | use lib 'lib'; 4 | use lib 'inc'; 5 | use Test::Nginx::LWP -Base; 6 | 7 | config_preamble(<<'_EOC_'); 8 | error_page 411 = @chunkin_error; 9 | location @chunkin_error { 10 | chunkin_resume; 11 | } 12 | _EOC_ 13 | 14 | 1; 15 | -------------------------------------------------------------------------------- /t/lib/Test/Nginx/Socket/Chunkin.pm: -------------------------------------------------------------------------------- 1 | package Test::Nginx::Socket::Chunkin; 2 | 3 | use lib 'lib'; 4 | use lib 'inc'; 5 | use Test::Nginx::Socket -Base; 6 | 7 | config_preamble(<<'_EOC_'); 8 | error_page 411 = @chunkin_error; 9 | location @chunkin_error { 10 | chunkin_resume; 11 | } 12 | _EOC_ 13 | 14 | 1; 15 | -------------------------------------------------------------------------------- /t/max_chunks.t: -------------------------------------------------------------------------------- 1 | # vi:filetype= 2 | 3 | use lib 't/lib'; 4 | use Test::Nginx::LWP::Chunkin; 5 | 6 | plan tests => repeat_each() * 2 * blocks(); 7 | 8 | no_diff; 9 | 10 | run_tests(); 11 | 12 | __DATA__ 13 | 14 | === TEST 1: exceeding the default max chunks per buf setting (512) 15 | --- config 16 | chunkin on; 17 | location /main { 18 | client_body_buffer_size 1m; 19 | echo_request_body; 20 | } 21 | --- request 22 | POST /main 23 | --- chunked_body eval 24 | [split //, 'a' x 1024] 25 | --- response_body eval 26 | 'a' x 1024 27 | 28 | 29 | 30 | === TEST 2: NOT exceeding the custom max chunks per buf setting 31 | --- config 32 | chunkin on; 33 | location /main { 34 | client_body_buffer_size 1m; 35 | chunkin_max_chunks_per_buf 1024; 36 | echo_request_body; 37 | } 38 | --- request 39 | POST /main 40 | --- chunked_body eval 41 | [split //, 'a' x 1024] 42 | --- response_body eval 43 | 'a' x 1024 44 | 45 | 46 | 47 | === TEST 3: JUST exceeding the custom max chunks per buf setting 48 | --- config 49 | chunkin on; 50 | location /main { 51 | client_body_buffer_size 1m; 52 | chunkin_max_chunks_per_buf 1024; 53 | echo_request_body; 54 | } 55 | --- request 56 | POST /main 57 | --- chunked_body eval 58 | [split //, 'a' x 1025] 59 | --- response_body eval 60 | 'a' x 1025 61 | 62 | -------------------------------------------------------------------------------- /t/pipelined.t: -------------------------------------------------------------------------------- 1 | # vi:filetype= 2 | 3 | use lib 't/lib'; 4 | 5 | my $skip_all; 6 | 7 | BEGIN { $skip_all = 1; } 8 | 9 | use Test::Nginx::Socket::Chunkin $skip_all ? 10 | (skip_all => 'too experimental to run the tests properly :P') 11 | : (); 12 | 13 | plan tests => repeat_each() * 2 * blocks(); 14 | 15 | no_diff; 16 | 17 | run_tests(); 18 | 19 | __DATA__ 20 | 21 | === TEST 1: sanity 22 | --- config 23 | chunkin on; 24 | location /main { 25 | chunkin_keepalive on; 26 | client_body_buffer_size 4; 27 | echo_request_body; 28 | } 29 | location /main2 { 30 | chunkin_keepalive on; 31 | client_body_buffer_size 4; 32 | echo_request_body; 33 | } 34 | 35 | --- more_headers 36 | Transfer-Encoding: chunked 37 | --- pipelined_requests eval 38 | [ 39 | "POST /main 40 | 5\r 41 | hello\r 42 | 0\r 43 | \r 44 | ", 45 | "POST /main2 46 | 6\r 47 | ,world\r 48 | 0\r 49 | \r 50 | "] 51 | --- response_body: hello,world 52 | 53 | 54 | 55 | === TEST 2: standard body read 56 | --- config 57 | chunkin on; 58 | location /main { 59 | client_body_buffer_size 4; 60 | echo_read_request_body; 61 | echo_request_body; 62 | } 63 | --- more_headers 64 | Content-Length: 5 65 | --- pipelined_requests eval 66 | [ 67 | "POST /main 68 | hello", 69 | "POST /main 70 | world"] 71 | --- response_body: helloworld 72 | 73 | -------------------------------------------------------------------------------- /t/pressure.t: -------------------------------------------------------------------------------- 1 | # vi:filetype= 2 | 3 | use lib 't/lib'; 4 | use Test::Nginx::LWP::Chunkin; 5 | 6 | plan tests => repeat_each() * 2 * blocks(); 7 | 8 | #no_diff; 9 | 10 | run_tests(); 11 | 12 | __DATA__ 13 | 14 | === TEST 1: many little chunks 15 | --- config 16 | chunkin on; 17 | location /main { 18 | client_body_buffer_size 4; 19 | echo_request_body; 20 | } 21 | --- request 22 | POST /main 23 | --- chunked_body eval 24 | [split //, 25 | "hello world blah blah blah! oh, yay! end"] 26 | --- response_body eval 27 | "hello world blah blah blah! oh, yay! end" 28 | 29 | 30 | 31 | === TEST 2: many little chunks (more!) 32 | --- config 33 | chunkin on; 34 | location /main { 35 | client_body_buffer_size 1k; 36 | #echo_sleep 500; 37 | echo_request_body; 38 | } 39 | --- request 40 | POST /main 41 | --- chunked_body eval 42 | [split //, 43 | ("hello world blah blah blah! o, yah!" x 4) . 'end'] 44 | --- response_body eval 45 | ("hello world blah blah blah! o, yah!" x 4) . 'end' 46 | 47 | 48 | 49 | === TEST 3: many little chunks (more!) 50 | --- config 51 | chunkin on; 52 | location /main { 53 | client_body_buffer_size 60; 54 | #echo_sleep 500; 55 | echo_request_body; 56 | } 57 | --- request 58 | POST /main 59 | --- chunked_body eval 60 | [split //, 61 | ("hello world blah blah blah! oh, yah!" x 100) . 'end'] 62 | --- response_body eval 63 | ("hello world blah blah blah! oh, yah!" x 100) . 'end' 64 | 65 | 66 | 67 | === TEST 4: exceeding max body limit (this test may fail randomly with the error "500 write failed: Connection reset by peer", which is considered OK). 68 | --- config 69 | chunkin on; 70 | location /main { 71 | client_body_buffer_size 512; 72 | client_max_body_size 1024; 73 | 74 | echo_request_body; 75 | } 76 | --- request 77 | POST /main 78 | --- chunked_body eval 79 | [split //, 80 | ("a" x 1024) . 'e'] 81 | --- response_body_like: 413 Request Entity Too Large 82 | --- error_code: 413 83 | 84 | 85 | 86 | === TEST 5: not exceeding max body limit (chunk spanning preread and rb->buf) 87 | --- config 88 | chunkin on; 89 | location /main { 90 | client_body_buffer_size 512; 91 | client_max_body_size 1048; 92 | 93 | echo_request_body; 94 | } 95 | --- request 96 | POST /main 97 | --- chunked_body eval 98 | ["a" x 1024] 99 | --- response_body eval 100 | "a" x 1024 101 | 102 | 103 | 104 | === TEST 6: next chunk reset bug 105 | --- config 106 | chunkin on; 107 | location /main { 108 | client_body_buffer_size 600; 109 | client_max_body_size 8k; 110 | 111 | echo_request_body; 112 | } 113 | --- request 114 | POST /main 115 | --- middle_chunk_delay: 0.001 116 | --- chunked_body eval 117 | [split //, 118 | ("a" x 700) . 'e'] 119 | --- response_body eval 120 | "a" x 700 . 'e' 121 | 122 | 123 | 124 | === TEST 7: next chunk reset bug (too many chunks) 125 | --- config 126 | chunkin on; 127 | location /main { 128 | client_body_buffer_size 8k; 129 | client_max_body_size 8k; 130 | 131 | echo_request_body; 132 | } 133 | --- request 134 | POST /main 135 | --- chunked_body eval 136 | [split //, 137 | ("a" x 700) . 'e'] 138 | --- response_body eval 139 | "a" x 700 . 'e' 140 | 141 | 142 | 143 | === TEST 8: normal POST with chunkin on 144 | --- config 145 | chunkin on; 146 | location /main { 147 | client_body_buffer_size 600; 148 | client_max_body_size 8k; 149 | 150 | echo_read_request_body; 151 | echo_request_body; 152 | } 153 | --- request 154 | POST /main 155 | hello, world 156 | --- response_body chomp 157 | hello, world 158 | 159 | 160 | 161 | === TEST 9: not exceeding max body limit (chunk spanning preread and rb->buf) 162 | --- config 163 | chunkin on; 164 | location /main { 165 | client_body_buffer_size 10m; 166 | client_max_body_size 10m; 167 | 168 | echo_request_body; 169 | echo; 170 | } 171 | --- request 172 | POST /main 173 | --- chunked_body eval 174 | [split //, "a" x (500 * 1024)] 175 | --- middle_chunk_delay: 0 176 | --- response_body eval 177 | "a" x (500 * 1024) 178 | --- quit 179 | --- SKIP 180 | 181 | -------------------------------------------------------------------------------- /t/random.t: -------------------------------------------------------------------------------- 1 | # vi:filetype= 2 | 3 | use lib 't/lib'; 4 | use Test::Nginx::Socket::Chunkin; 5 | 6 | #worker_connections(1024); 7 | master_process_enabled(1); 8 | 9 | our $data; 10 | 11 | repeat_each(500); 12 | 13 | plan tests => repeat_each() * 2 * blocks(); 14 | 15 | no_diff; 16 | 17 | run_tests(); 18 | 19 | __DATA__ 20 | 21 | === TEST 1: single chunk 22 | --- config 23 | chunkin on; 24 | location /ar.do { 25 | client_max_body_size 1m; 26 | client_body_buffer_size 512; 27 | echo_request_body; 28 | } 29 | --- more_headers 30 | Transfer-Encoding: chunked 31 | --- request eval 32 | $::data = ''; 33 | my $count = (int rand 32766) + 1; 34 | for (my $i = 0; $i < $count; $i++) { 35 | my $c = chr int rand 128; 36 | $::data .= $c; 37 | } 38 | #warn $::data; 39 | my $s = "POST /ar.do 40 | ". 41 | sprintf("%x\r\n", length $::data). 42 | $::data 43 | ."\r 44 | 0\r 45 | \r 46 | "; 47 | open my $out, '>/tmp/out.txt' or die $!; 48 | print $out $s; 49 | close $out; 50 | $s 51 | --- response_body eval 52 | $::data 53 | --- timeout: 10 54 | 55 | 56 | 57 | === TEST 2: single chunk 58 | --- config 59 | chunkin on; 60 | location /ar.do { 61 | client_max_body_size 1m; 62 | client_body_buffer_size 1k; 63 | echo_request_body; 64 | } 65 | --- more_headers 66 | Transfer-Encoding: chunked 67 | --- request eval 68 | $::data = ''; 69 | my $count = 32766; 70 | for (my $i = 0; $i < $count; $i++) { 71 | my $c = chr int rand 256; 72 | $::data .= $c; 73 | } 74 | #warn $::data; 75 | my $s = "POST /ar.do 76 | ". 77 | sprintf("%x\r\n", length $::data). 78 | $::data 79 | ."\r 80 | 0\r 81 | \r 82 | "; 83 | open my $out, '>/tmp/out.txt' or die $!; 84 | print $out $s; 85 | close $out; 86 | $s 87 | --- response_body eval 88 | $::data 89 | --- timeout: 10 90 | 91 | -------------------------------------------------------------------------------- /t/sanity.t: -------------------------------------------------------------------------------- 1 | # vi:ft= 2 | 3 | use lib 't/lib'; 4 | use Test::Nginx::LWP::Chunkin; 5 | 6 | plan tests => repeat_each() * 2 * blocks(); 7 | 8 | no_diff; 9 | 10 | run_tests(); 11 | 12 | __DATA__ 13 | 14 | === TEST 1: off & POST 15 | --- config 16 | chunkin off; 17 | location /main { 18 | echo hi; 19 | } 20 | --- request 21 | POST /main 22 | --- chunked_body eval 23 | ["hello", "world"] 24 | --- error_code: 411 25 | --- response_body_like: 411 Length Required 26 | 27 | 28 | 29 | === TEST 2: default (off) & POST 30 | --- config 31 | location /main { 32 | } 33 | --- request 34 | POST /main 35 | --- chunked_body eval 36 | ["hello", "world"] 37 | --- error_code: 411 38 | --- response_body_like: 411 Length Required 39 | 40 | 41 | 42 | === TEST 3: off & GET 43 | --- config 44 | chunkin off; 45 | location /main { 46 | echo hi; 47 | } 48 | --- request 49 | GET /main 50 | --- response_body 51 | hi 52 | --- error_code: 200 53 | 54 | 55 | 56 | === TEST 4: on & GET 57 | --- config 58 | chunkin on; 59 | location /main { 60 | echo hi; 61 | } 62 | --- request 63 | GET /main 64 | --- response_body 65 | hi 66 | --- error_code: 200 67 | 68 | 69 | 70 | === TEST 5: on & POST 71 | --- config 72 | chunkin on; 73 | location /main { 74 | echo $request_method; 75 | #echo $echo_request_body; 76 | } 77 | --- request 78 | POST /main 79 | --- chunked_body eval 80 | ["hello", "world"] 81 | --- error_code: 200 82 | --- response_body 83 | POST 84 | 85 | 86 | 87 | === TEST 6: raw request headers (indeed chunked) 88 | --- config 89 | chunkin on; 90 | location /main { 91 | echo 'headers:'; 92 | echo -n $echo_client_request_headers; 93 | } 94 | --- request 95 | POST /main 96 | --- chunked_body eval 97 | ["hello", "world"] 98 | --- error_code: 200 99 | --- response_body eval 100 | "headers: 101 | POST /main HTTP/1.1\r 102 | Host: localhost:\$ServerPortForClient\r 103 | User-Agent: Test::Nginx::LWP\r 104 | Content-Type: text/plain\r 105 | Transfer-Encoding: chunked\r 106 | \r 107 | " 108 | 109 | 110 | 111 | === TEST 7: request headers filtered by chunkin 112 | This test passes only for nginx versions 113 | * 0.7.x >= 0.7.21 114 | * 0.8.x >= 0.8.10 115 | --- config 116 | chunkin on; 117 | location /main { 118 | proxy_pass_request_headers on; 119 | proxy_pass $scheme://127.0.0.1:$server_port/proxy; 120 | } 121 | 122 | location /proxy { 123 | echo -n $echo_client_request_headers; 124 | } 125 | --- request 126 | POST /main 127 | --- chunked_body eval 128 | ["hello", "world"] 129 | --- error_code: 200 130 | --- response_body eval 131 | "POST /proxy HTTP/1.0\r 132 | Host: 127.0.0.1:\$ServerPort\r 133 | Connection: close\r 134 | User-Agent: Test::Nginx::LWP\r 135 | Content-Type: text/plain\r 136 | Content-Length: 10\r 137 | \r 138 | " 139 | 140 | 141 | 142 | === TEST 8: 0 chunk body 143 | --- config 144 | chunkin on; 145 | location /main { 146 | echo "body:"; 147 | echo $echo_request_body; 148 | } 149 | --- request 150 | POST /main 151 | --- chunked_body eval 152 | [""] 153 | --- error_code: 200 154 | --- response_body eval 155 | "body: 156 | 157 | " 158 | 159 | 160 | 161 | === TEST 9: 0 chunk body via proxy (header okay) 162 | --- config 163 | chunkin on; 164 | location /main { 165 | proxy_pass $scheme://127.0.0.1:$server_port/proxy; 166 | } 167 | location /proxy { 168 | echo -n $echo_client_request_headers; 169 | } 170 | --- request 171 | POST /main 172 | --- chunked_body eval 173 | [""] 174 | --- error_code: 200 175 | --- response_body eval 176 | "POST /proxy HTTP/1.0\r 177 | Host: 127.0.0.1:\$ServerPort\r 178 | Connection: close\r 179 | User-Agent: Test::Nginx::LWP\r 180 | Content-Type: text/plain\r 181 | Content-Length: 0\r 182 | \r 183 | " 184 | 185 | 186 | 187 | === TEST 10: single char in preread 188 | --- config 189 | chunkin on; 190 | location /main { 191 | proxy_pass $scheme://127.0.0.1:$server_port/proxy; 192 | } 193 | location /proxy { 194 | echo -n $echo_client_request_headers; 195 | } 196 | --- request 197 | POST /main 198 | --- chunked_body eval 199 | ["a"] 200 | --- error_code: 200 201 | --- response_body eval 202 | "POST /proxy HTTP/1.0\r 203 | Host: 127.0.0.1:\$ServerPort\r 204 | Connection: close\r 205 | User-Agent: Test::Nginx::LWP\r 206 | Content-Type: text/plain\r 207 | Content-Length: 1\r 208 | \r 209 | " 210 | 211 | 212 | 213 | === TEST 11: single char in preread (headers okay) 214 | --- config 215 | chunkin on; 216 | location /main { 217 | echo "body:"; 218 | echo $echo_request_body; 219 | } 220 | --- request 221 | POST /main 222 | --- chunked_body eval 223 | ["a"] 224 | --- error_code: 200 225 | --- response_body 226 | body: 227 | a 228 | 229 | 230 | 231 | === TEST 12: on & POST & read body & no single buf 232 | --- config 233 | chunkin on; 234 | location /main { 235 | echo "body:"; 236 | echo $echo_request_body; 237 | } 238 | --- request 239 | POST /main 240 | --- middle_chunk_delay: 0.01 241 | --- chunked_body eval 242 | ["hello", "world"] 243 | --- error_code: 200 244 | --- response_body 245 | body: 246 | helloworld 247 | 248 | 249 | 250 | === TEST 13: on & POST & read body & single buf 251 | --- config 252 | chunkin on; 253 | location /main { 254 | client_body_in_single_buffer on; 255 | echo "body:"; 256 | echo $echo_request_body; 257 | } 258 | --- request 259 | POST /main 260 | --- middle_chunk_delay: 0.01 261 | --- chunked_body eval 262 | ["hello", "world"] 263 | --- error_code: 200 264 | --- response_body 265 | body: 266 | helloworld 267 | --- skip_nginx: 2: < 0.7.58 268 | 269 | 270 | 271 | === TEST 14: on & POST & read body & no single buf & echo_request_body 272 | --- config 273 | chunkin on; 274 | location /main { 275 | client_body_in_single_buffer on; 276 | echo "body:"; 277 | echo_request_body; 278 | echo; 279 | } 280 | --- request 281 | POST /main 282 | --- middle_chunk_delay: 0.01 283 | --- chunked_body eval 284 | ["hello", "world"] 285 | --- error_code: 200 286 | --- response_body 287 | body: 288 | helloworld 289 | --- skip_nginx: 2: < 0.7.58 290 | 291 | 292 | 293 | === TEST 15: request headers filtered by chunkin (with delay) 294 | This test passes only for nginx versions 295 | * 0.7.x >= 0.7.21 296 | * 0.8.x >= 0.8.10 297 | --- middle_chunk_delay: 0.01 298 | --- config 299 | chunkin on; 300 | location /main { 301 | proxy_pass_request_headers on; 302 | proxy_pass $scheme://127.0.0.1:$server_port/proxy; 303 | } 304 | 305 | location /proxy { 306 | echo -n $echo_client_request_headers; 307 | } 308 | --- request 309 | POST /main 310 | --- chunked_body eval 311 | ["hello", "world"] 312 | --- error_code: 200 313 | --- response_body eval 314 | "POST /proxy HTTP/1.0\r 315 | Host: 127.0.0.1:\$ServerPort\r 316 | Connection: close\r 317 | User-Agent: Test::Nginx::LWP\r 318 | Content-Type: text/plain\r 319 | Content-Length: 10\r 320 | \r 321 | " 322 | 323 | 324 | 325 | === TEST 16: small buf (using 2-byte buf) 326 | --- config 327 | chunkin on; 328 | location /main { 329 | client_body_buffer_size 2; 330 | echo "body:"; 331 | echo $echo_request_body; 332 | echo_request_body; 333 | } 334 | --- request 335 | POST /main 336 | --- start_chunk_delay: 0.01 337 | --- middle_chunk_delay: 0.01 338 | --- chunked_body eval 339 | ["hello", "world"] 340 | --- error_code: 200 341 | --- response_body eval 342 | "body: 343 | 344 | helloworld" 345 | 346 | 347 | 348 | === TEST 17: small buf (using 1-byte buf) 349 | --- config 350 | chunkin on; 351 | location /main { 352 | client_body_buffer_size 1; 353 | echo "body:"; 354 | echo $echo_request_body; 355 | echo_request_body; 356 | } 357 | --- request 358 | POST /main 359 | --- start_chunk_delay: 0.01 360 | --- middle_chunk_delay: 0.01 361 | --- chunked_body eval 362 | ["hello", "world"] 363 | --- error_code: 200 364 | --- response_body eval 365 | "body: 366 | 367 | helloworld" 368 | 369 | 370 | 371 | === TEST 18: small buf (using 3-byte buf) 372 | --- config 373 | chunkin on; 374 | location /main { 375 | client_body_buffer_size 3; 376 | echo "body:"; 377 | echo $echo_request_body; 378 | echo_request_body; 379 | } 380 | --- request 381 | POST /main 382 | --- start_chunk_delay: 0.01 383 | --- middle_chunk_delay: 0.01 384 | --- chunked_body eval 385 | ["hello", "world"] 386 | --- error_code: 200 387 | --- response_body eval 388 | "body: 389 | 390 | helloworld" 391 | 392 | 393 | 394 | === TEST 19: big chunk 395 | --- config 396 | chunkin on; 397 | location /main { 398 | client_body_buffer_size 3; 399 | echo "body:"; 400 | echo $echo_request_body; 401 | echo_request_body; 402 | } 403 | --- request 404 | POST /main 405 | --- start_chunk_delay: 0.01 406 | --- middle_chunk_delay: 0.01 407 | --- chunked_body eval 408 | ["hello", "world" x 1024, "!" x 1024] 409 | --- error_code: 200 410 | --- response_body eval 411 | "body: 412 | 413 | hello" . ("world" x 1024) . ('!' x 1024) 414 | 415 | 416 | 417 | === TEST 20: in memory 418 | --- config 419 | chunkin on; 420 | location /main { 421 | client_body_buffer_size 4k; 422 | echo "body:"; 423 | echo $echo_request_body; 424 | } 425 | --- request 426 | POST /main 427 | --- start_chunk_delay: 0.01 428 | --- middle_chunk_delay: 0.01 429 | --- chunked_body eval 430 | ["hello", "world"] 431 | --- error_code: 200 432 | --- response_body 433 | body: 434 | helloworld 435 | 436 | 437 | 438 | === TEST 21: on & PUT 439 | --- config 440 | chunkin on; 441 | location /main { 442 | #echo_read_request_body; 443 | echo_request_body; 444 | } 445 | --- request 446 | PUT /main 447 | --- chunked_body eval 448 | ["hello", "world"] 449 | --- response_body chomp 450 | helloworld 451 | --- error_code: 200 452 | 453 | -------------------------------------------------------------------------------- /t/timeout.t: -------------------------------------------------------------------------------- 1 | # vi:filetype= 2 | 3 | use lib 't/lib'; 4 | use Test::Nginx::Socket::Chunkin; 5 | 6 | plan tests => repeat_each() * 2 * blocks(); 7 | 8 | no_diff; 9 | 10 | run_tests(); 11 | 12 | __DATA__ 13 | 14 | === TEST 1: bad chunk size in the 2nd chunk 15 | --- config 16 | chunkin on; 17 | location /main { 18 | client_body_timeout 1; 19 | echo_read_request_body; 20 | echo_request_body; 21 | } 22 | --- more_headers 23 | Transfer-Encoding: chunked 24 | User-Agent: Java Browser 25 | --- timeout: 2 26 | --- request eval 27 | "POST /main 28 | 0\r 29 | " 30 | --- response_body: 31 | --- error_code: 32 | 33 | 34 | 35 | === TEST 2: bad chunk size in the 2nd chunk (using standard client body reader) 36 | --- config 37 | chunkin on; 38 | location /main { 39 | client_body_timeout 1; 40 | echo_read_request_body; 41 | echo_request_body; 42 | } 43 | --- more_headers 44 | Content-Length: 5 45 | --- timeout: 3 46 | --- request eval 47 | "POST /main 48 | he" 49 | --- response_body: 50 | --- error_code: 51 | --- SKIP 52 | 53 | -------------------------------------------------------------------------------- /util/build.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # this file is mostly meant to be used by the author himself. 4 | 5 | ragel -G2 src/chunked_parser.rl 6 | if [ $? != 0 ]; then 7 | echo 'Failed to generate the chunked parser.' 1>&2 8 | exit 1; 9 | fi 10 | 11 | root=`pwd` 12 | version=$1 13 | force=$2 14 | 15 | ngx-build $force $version \ 16 | --with-cc-opt="-funsigned-char" \ 17 | --with-ld-opt="-L$PCRE_LIB -Wl,-rpath,$PCRE_LIB:$LUAJIT_LIB:/usr/local/lib" \ 18 | --add-module=$root/../echo-nginx-module \ 19 | --add-module=$root/../lua-nginx-module \ 20 | --add-module=$root $opts \ 21 | --with-debug 22 | #\ 23 | #--with-http_ssl_module #\ 24 | #--with-cc-opt="-pg" --with-ld-opt="-pg" \ 25 | #--without-http_ssi_module # we cannot disable ssi because echo_location_async depends on it (i dunno why?!) 26 | 27 | -------------------------------------------------------------------------------- /util/releng: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ./update-readme 4 | ack '(?<=\#define)\s*DDEBUG\s*1' src 5 | ack '(?<=This document describes chunkin-nginx-module v)\d+\.\d+' README 6 | 7 | -------------------------------------------------------------------------------- /util/update-readme.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | perl util/wiki2pod.pl doc/readme.wiki > /tmp/a.pod && pod2text /tmp/a.pod > README 4 | 5 | -------------------------------------------------------------------------------- /util/wiki2pod.pl: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env perl 2 | 3 | use strict; 4 | use warnings; 5 | use bytes; 6 | 7 | my @nl_counts; 8 | my $last_nl_count_level; 9 | 10 | my @bl_counts; 11 | my $last_bl_count_level; 12 | 13 | sub fmt_pos ($) { 14 | (my $s = $_[0]) =~ s{\#(.*)}{/"$1"}; 15 | $s; 16 | } 17 | 18 | sub fmt_mark ($$) { 19 | my ($tag, $s) = @_; 20 | my $max_level = 0; 21 | while ($s =~ /([<>])\1*/g) { 22 | my $level = length $&; 23 | if ($level > $max_level) { 24 | $max_level = $level; 25 | } 26 | } 27 | 28 | my $times = $max_level + 1; 29 | if ($times > 1) { 30 | $s = " $s "; 31 | } 32 | return $tag . ('<' x $times) . $s . ('>' x $times); 33 | } 34 | 35 | print "=encoding utf-8\n\n"; 36 | 37 | while (<>) { 38 | if ($. == 1) { 39 | # strip the leading U+FEFF byte in MS-DOS text files 40 | my $first = ord(substr($_, 0, 1)); 41 | #printf STDERR "0x%x", $first; 42 | #my $second = ord(substr($_, 2, 1)); 43 | #printf STDERR "0x%x", $second; 44 | if ($first == 0xEF) { 45 | substr($_, 0, 1, ''); 46 | #warn "Hit!"; 47 | } 48 | } 49 | s{\[(http[^ \]]+) ([^\]]*)\]}{$2 (L<$1>)}gi; 50 | s{ \[\[ ( [^\]\|]+ ) \| ([^\]]*) \]\] }{"L<$2|" . fmt_pos($1) . ">"}gixe; 51 | s{(.*?)}{fmt_mark('C', $1)}gie; 52 | s{'''(.*?)'''}{fmt_mark('B', $1)}ge; 53 | s{''(.*?)''}{fmt_mark('I', $1)}ge; 54 | if (s{^\s*<[^>]+>\s*$}{}) { 55 | next; 56 | } 57 | 58 | if (/^\s*$/) { 59 | print "\n"; 60 | next; 61 | } 62 | 63 | =begin cmt 64 | 65 | if ($. == 1) { 66 | warn $_; 67 | for my $i (0..length($_) - 1) { 68 | my $chr = substr($_, $i, 1); 69 | warn "chr ord($i): ".ord($chr)." \"$chr\"\n"; 70 | } 71 | } 72 | 73 | =end cmt 74 | =cut 75 | 76 | if (/(=+) (.*) \1$/) { 77 | #warn "HERE! $_" if $. == 1; 78 | my ($level, $title) = (length $1, $2); 79 | collapse_lists(); 80 | 81 | print "\n=head$level $title\n\n"; 82 | } elsif (/^(\#+) (.*)/) { 83 | my ($level, $txt) = (length($1) - 1, $2); 84 | if (defined $last_nl_count_level && $level != $last_nl_count_level) { 85 | print "\n=back\n\n"; 86 | } 87 | $last_nl_count_level = $level; 88 | $nl_counts[$level] ||= 0; 89 | if ($nl_counts[$level] == 0) { 90 | print "\n=over\n\n"; 91 | } 92 | $nl_counts[$level]++; 93 | print "\n=item $nl_counts[$level].\n\n"; 94 | print "$txt\n"; 95 | } elsif (/^(\*+) (.*)/) { 96 | my ($level, $txt) = (length($1) - 1, $2); 97 | if (defined $last_bl_count_level && $level != $last_bl_count_level) { 98 | print "\n=back\n\n"; 99 | } 100 | $last_bl_count_level = $level; 101 | $bl_counts[$level] ||= 0; 102 | if ($bl_counts[$level] == 0) { 103 | print "\n=over\n\n"; 104 | } 105 | $bl_counts[$level]++; 106 | print "\n=item *\n\n"; 107 | print "$txt\n"; 108 | } else { 109 | collapse_lists(); 110 | print; 111 | } 112 | } 113 | 114 | collapse_lists(); 115 | 116 | sub collapse_lists { 117 | while (defined $last_nl_count_level && $last_nl_count_level >= 0) { 118 | print "\n=back\n\n"; 119 | $last_nl_count_level--; 120 | } 121 | undef $last_nl_count_level; 122 | undef @nl_counts; 123 | 124 | while (defined $last_bl_count_level && $last_bl_count_level >= 0) { 125 | print "\n=back\n\n"; 126 | $last_bl_count_level--; 127 | } 128 | undef $last_bl_count_level; 129 | undef @bl_counts; 130 | } 131 | 132 | -------------------------------------------------------------------------------- /valgrind.suppress: -------------------------------------------------------------------------------- 1 | { 2 | 3 | Memcheck:Leak 4 | fun:malloc 5 | fun:ngx_alloc 6 | fun:ngx_create_pool 7 | } 8 | { 9 | 10 | Memcheck:Leak 11 | fun:malloc 12 | fun:ngx_alloc 13 | fun:ngx_malloc 14 | } 15 | { 16 | 17 | Memcheck:Leak 18 | fun:memalign 19 | fun:posix_memalign 20 | fun:ngx_memalign 21 | fun:ngx_palloc_block 22 | fun:ngx_palloc 23 | } 24 | 25 | { 26 | 27 | Memcheck:Leak 28 | fun:malloc 29 | fun:ngx_alloc 30 | fun:ngx_palloc_large 31 | fun:ngx_palloc 32 | fun:ngx_create_temp_buf 33 | fun:ngx_http_chunkin_read_chunked_request_body 34 | fun:ngx_http_chunkin_handler 35 | fun:ngx_http_core_access_phase 36 | fun:ngx_http_core_run_phases 37 | fun:ngx_http_handler 38 | fun:ngx_http_chunkin_internal_redirect 39 | fun:ngx_http_chunkin_resume_handler 40 | } 41 | { 42 | 43 | Memcheck:Leak 44 | fun:malloc 45 | fun:ngx_alloc 46 | fun:ngx_event_process_init 47 | fun:ngx_worker_process_init 48 | fun:ngx_worker_process_cycle 49 | fun:ngx_spawn_process 50 | fun:ngx_start_worker_processes 51 | fun:ngx_master_process_cycle 52 | fun:main 53 | } 54 | { 55 | 56 | Memcheck:Param 57 | socketcall.sendmsg(msg.msg_iov[i]) 58 | fun:__sendmsg_nocancel 59 | fun:ngx_write_channel 60 | fun:ngx_signal_worker_processes 61 | fun:ngx_master_process_cycle 62 | fun:main 63 | } 64 | { 65 | nginx-core-process-init 66 | Memcheck:Leak 67 | fun:malloc 68 | fun:ngx_alloc 69 | fun:ngx_event_process_init 70 | fun:ngx_single_process_cycle 71 | fun:main 72 | } 73 | { 74 | nginx-core-crc32-init 75 | Memcheck:Leak 76 | fun:malloc 77 | fun:ngx_alloc 78 | fun:ngx_crc32_table_init 79 | fun:main 80 | } 81 | { 82 | palloc_large_for_init_request 83 | Memcheck:Leak 84 | fun:malloc 85 | fun:ngx_alloc 86 | fun:ngx_palloc_large 87 | fun:ngx_palloc 88 | fun:ngx_pcalloc 89 | fun:ngx_http_init_request 90 | fun:ngx_epoll_process_events 91 | fun:ngx_process_events_and_timers 92 | fun:ngx_single_process_cycle 93 | fun:main 94 | } 95 | { 96 | palloc_large_for_create_temp_buf 97 | Memcheck:Leak 98 | fun:malloc 99 | fun:ngx_alloc 100 | fun:ngx_palloc_large 101 | fun:ngx_palloc 102 | fun:ngx_create_temp_buf 103 | fun:ngx_http_init_request 104 | fun:ngx_epoll_process_events 105 | fun:ngx_process_events_and_timers 106 | fun:ngx_single_process_cycle 107 | fun:main 108 | } 109 | { 110 | accept_create_pool 111 | Memcheck:Leak 112 | fun:memalign 113 | fun:posix_memalign 114 | fun:ngx_memalign 115 | fun:ngx_create_pool 116 | fun:ngx_event_accept 117 | fun:ngx_epoll_process_events 118 | fun:ngx_process_events_and_timers 119 | fun:ngx_single_process_cycle 120 | fun:main 121 | } 122 | { 123 | create_pool_for_init_req 124 | Memcheck:Leak 125 | fun:memalign 126 | fun:posix_memalign 127 | fun:ngx_memalign 128 | fun:ngx_create_pool 129 | fun:ngx_http_init_request 130 | fun:ngx_epoll_process_events 131 | fun:ngx_process_events_and_timers 132 | fun:ngx_single_process_cycle 133 | fun:main 134 | } 135 | 136 | --------------------------------------------------------------------------------