├── .gitignore ├── .travis.yml ├── CHANGES.rst ├── LICENSE ├── Makefile.am ├── README.rst ├── autogen.sh ├── configure.ac ├── m4 └── curl.m4 ├── man └── vmod_curl.rst └── src ├── Makefile.am ├── debug_flags.h ├── tests ├── test01.vtc ├── test02.vtc ├── test03.vtc ├── test04.vtc ├── test05.vtc ├── test06.vtc ├── test07.vtc ├── test08.vtc ├── test09.vtc ├── test10.vtc ├── test11.vtc ├── test12.vtc ├── test13.vtc └── test14.vtc ├── vmod_curl.c └── vmod_curl.vcc /.gitignore: -------------------------------------------------------------------------------- 1 | Makefile 2 | Makefile.in 3 | .deps/ 4 | .libs/ 5 | *.o 6 | *.lo 7 | *.la 8 | *~ 9 | 10 | /aclocal.m4 11 | /autom4te.cache/ 12 | /compile 13 | /config.guess 14 | /config.h 15 | /config.h.in 16 | /config.log 17 | /config.status 18 | /config.sub 19 | /configure 20 | /depcomp 21 | /install-sh 22 | /libtool 23 | /ltmain.sh 24 | /missing 25 | /stamp-h1 26 | /m4/ 27 | !/m4/curl.m4 28 | test-driver 29 | src/test-suite.log 30 | src/tests/*.log 31 | src/tests/*.trs 32 | 33 | /src/vcc_curl_if.* 34 | /src/*.rst 35 | /src/*.3 36 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: c 2 | 3 | before_install: 4 | - sudo apt-get update -q 5 | - sudo apt-get install -qq apt-transport-https python-docutils 6 | - curl -s https://packagecloud.io/install/repositories/varnishcache/varnish41/script.deb.sh | sudo bash 7 | - sudo apt-get -q update 8 | - sudo apt-get install varnish varnish-dev 9 | 10 | 11 | before_script: 12 | - ./autogen.sh 13 | - ./configure --prefix=/usr 14 | - make 15 | 16 | script: 17 | - make check -j4 VERBOSE=1 18 | 19 | compiler: 20 | - clang 21 | - gcc 22 | -------------------------------------------------------------------------------- /CHANGES.rst: -------------------------------------------------------------------------------- 1 | 2 | This is a running log of changes to libvmod-curl. 3 | 4 | libvmod-curl 1.0.4 (2018-03-07) 5 | ------------------------------- 6 | 7 | * Add unix domain socket support 8 | 9 | * Skip Content-Length when calling header_add_all() 10 | 11 | * Skip Transfer-Encoding 12 | 13 | * More debug flags: logging headers and bodies 14 | 15 | * Added compatibility with Varnish Cache 6.0 16 | 17 | This release was tested with Varnish Cache 4.1.9 and trunk (2018-03-07) 18 | 19 | 20 | libvmod-curl 1.0.3 (2016-04-27) 21 | ------------------------------- 22 | 23 | * [autoconf] README and CHANGES files are now installed. 24 | 25 | Bugfix: 26 | 27 | * Use VTAILQ_REMOVE_SAFE when removing headers. 28 | 29 | This release was tested with Varnish Cache 4.1.2. 30 | 31 | 32 | libvmod-curl 1.0.2 (2016-03-14) 33 | ------------------------------- 34 | 35 | * Code was ported to Varnish Cache 4.1 format. 36 | 37 | * Minor improvements to the documentation. 38 | 39 | Bugfixes: 40 | 41 | * Add all headers from a req or bereq object (Issue #33) 42 | 43 | This release was tested with Varnish Cache 4.1.2. 44 | 45 | 46 | libvmod-curl 1.0.1 (2015-04-21) 47 | ------------------------------- 48 | 49 | This release was made before the introduction of the changes file. 50 | 51 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2011-2016 Varnish Software 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without 5 | modification, are permitted provided that the following conditions 6 | are met: 7 | 1. Redistributions of source code must retain the above copyright 8 | notice, this list of conditions and the following disclaimer. 9 | 2. Redistributions in binary form must reproduce the above copyright 10 | notice, this list of conditions and the following disclaimer in the 11 | documentation and/or other materials provided with the distribution. 12 | 13 | THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 14 | ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 15 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 16 | ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE 17 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 18 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 19 | OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 20 | HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 21 | LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 22 | OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 23 | SUCH DAMAGE. 24 | -------------------------------------------------------------------------------- /Makefile.am: -------------------------------------------------------------------------------- 1 | ACLOCAL_AMFLAGS = -I m4 -I @VARNISHAPI_DATAROOTDIR@/aclocal 2 | 3 | DISTCHECK_CONFIGURE_FLAGS = RST2MAN=: 4 | 5 | SUBDIRS = src 6 | 7 | EXTRA_DIST = README.rst LICENSE CHANGES.rst 8 | 9 | doc_DATA = README.rst LICENSE CHANGES.rst 10 | -------------------------------------------------------------------------------- /README.rst: -------------------------------------------------------------------------------- 1 | 2 | .. image:: https://travis-ci.org/varnish/libvmod-curl.svg?branch=master 3 | :alt: Travis CI badge 4 | :target: https://travis-ci.org/varnish/libvmod-curl/ 5 | 6 | This vmod provides cURL bindings for Varnish so you can use Varnish 7 | as an HTTP client and fetch headers and bodies from backends. 8 | 9 | WARNING: Using vmod-curl to connect to HTTPS sites is currently unsupported 10 | and may lead to segmentation faults on VCL load/unload. (openssl library 11 | intricacies) 12 | 13 | Installation 14 | ============ 15 | 16 | Source releases can be downloaded from: 17 | 18 | https://download.varnish-software.com/libvmod-curl/ 19 | 20 | Installation requires an installed version of Varnish Cache, including the 21 | development files. Requirements can be found in the `Varnish documentation`_. 22 | 23 | .. _`Varnish documentation`: https://www.varnish-cache.org/docs/4.1/installation/install.html#compiling-varnish-from-source 24 | .. _`Varnish Project packages`: https://www.varnish-cache.org/releases/index.html 25 | 26 | Source code is built with autotools, you need to install the correct 27 | development packages first. 28 | If you are using the official `Varnish Project packages`_:: 29 | 30 | sudo apt install varnish-dev || sudo yum install varnish-devel 31 | 32 | If you are using the distro provided packages:: 33 | 34 | sudo apt install libvarnishapi-dev || sudo yum install varnish-libs-devel 35 | 36 | In both cases, you also need the libcurl development package:: 37 | 38 | sudo apt install libcurl4-openssl-dev || sudo yum install libcurl-devel 39 | 40 | Then proceed to the configure and build:: 41 | 42 | ./configure 43 | make 44 | make check # optional 45 | sudo make install 46 | 47 | The resulting loadable modules (``libvmod_*.so`` files) will be installed to 48 | the Varnish module directory. (default `/usr/lib/varnish/vmods/`) 49 | 50 | Usage 51 | ===== 52 | 53 | To use the vmod do something along the lines of:: 54 | 55 | import curl; 56 | 57 | sub vcl_recv { 58 | curl.get("http://example.com/test"); 59 | if (curl.header("X-Foo") == "bar") { 60 | ... 61 | } 62 | 63 | curl.free(); 64 | } 65 | 66 | 67 | See src/vmod_curl.vcc for the rest of the callable functions. 68 | 69 | Development 70 | =========== 71 | 72 | The source git tree lives on Github: https://github.com/varnish/libvmod-curl 73 | 74 | All source code is placed in the master git branch. Pull requests and issue 75 | reporting are appreciated. 76 | 77 | Unlike building from releases, you need to first bootstrap the build system 78 | when you work from git. In addition to the dependencies mentioned in the 79 | installation section, you also need to install the build tools:: 80 | 81 | sudo apt-get automake autotools-dev python-docutils 82 | 83 | Then build the vmod:: 84 | 85 | ./autogen.sh 86 | ./configure 87 | make 88 | make check # recommended 89 | 90 | If the ``configure`` step succeeds but the ``make`` step fails, check for 91 | warnings in the ``./configure`` output or the ``config.log`` file. You may be 92 | missing bootstrap dependencies not required by release archives. 93 | 94 | If you have installed Varnish to a non-standard directory, call ``autogen.sh`` 95 | and ``configure`` with ``PKG_CONFIG_PATH`` and ``ACLOCAL_PATH`` pointing to 96 | the appropriate path. For instance, when varnishd configure was called with 97 | ``--prefix=$PREFIX``, use:: 98 | 99 | export PKG_CONFIG_PATH=$PREFIX/lib/pkgconfig 100 | export ACLOCAL_PATH=$PREFIX/share/aclocal 101 | 102 | -- 103 | 104 | Development of this VMOD has been sponsored by the Norwegian company 105 | Aspiro Music AS for usage on their WiMP music streaming service. 106 | 107 | .. _`Varnish Project packages`: https://www.varnish-cache.org/releases/index.html 108 | -------------------------------------------------------------------------------- /autogen.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | warn() { 4 | echo "WARNING: $@" 1>&2 5 | } 6 | 7 | case `uname -s` in 8 | Darwin) 9 | LIBTOOLIZE=glibtoolize 10 | ;; 11 | FreeBSD) 12 | LIBTOOLIZE=libtoolize 13 | ;; 14 | OpenBSD) 15 | LIBTOOLIZE=libtoolize 16 | ;; 17 | Linux) 18 | LIBTOOLIZE=libtoolize 19 | ;; 20 | SunOS) 21 | LIBTOOLIZE=libtoolize 22 | ;; 23 | *) 24 | warn "unrecognized platform:" `uname -s` 25 | LIBTOOLIZE=libtoolize 26 | esac 27 | 28 | automake_version=`automake --version | tr ' ' '\n' | egrep '^[0-9]\.[0-9a-z.-]+'` 29 | if [ -z "$automake_version" ] ; then 30 | warn "unable to determine automake version" 31 | else 32 | case $automake_version in 33 | 0.*|1.[0-8]|1.[0-8][.-]*) 34 | warn "automake ($automake_version) detected; 1.9 or newer recommended" 35 | ;; 36 | *) 37 | ;; 38 | esac 39 | fi 40 | 41 | # check for varnishapi.m4 in custom paths 42 | dataroot=$(pkg-config --variable=datarootdir varnishapi 2>/dev/null) 43 | if [ -z "$dataroot" ] ; then 44 | cat >&2 <<'EOF' 45 | Package varnishapi was not found in the pkg-config search path. 46 | Perhaps you should add the directory containing `varnishapi.pc' 47 | to the PKG_CONFIG_PATH environment variable 48 | EOF 49 | exit 1 50 | fi 51 | set -ex 52 | 53 | $LIBTOOLIZE --copy --force 54 | aclocal -I m4 -I ${dataroot}/aclocal 55 | autoheader 56 | automake --add-missing --copy --foreign 57 | autoconf 58 | -------------------------------------------------------------------------------- /configure.ac: -------------------------------------------------------------------------------- 1 | AC_PREREQ(2.64) 2 | AC_COPYRIGHT([Copyright (c) 2011-2016 Varnish Software]) 3 | AC_INIT([libvmod-curl], [1.0.4]) 4 | AC_CONFIG_MACRO_DIR([m4]) 5 | AC_CONFIG_SRCDIR(src/vmod_curl.vcc) 6 | AM_CONFIG_HEADER(config.h) 7 | 8 | AC_CANONICAL_SYSTEM 9 | AC_LANG(C) 10 | 11 | AM_INIT_AUTOMAKE([foreign color-tests parallel-tests]) 12 | 13 | AC_GNU_SOURCE 14 | AC_PROG_CC 15 | AC_PROG_CC_STDC 16 | if test "x$ac_cv_prog_cc_c99" = xno; then 17 | AC_MSG_ERROR([Could not find a C99 compatible compiler]) 18 | fi 19 | AC_PROG_CPP 20 | 21 | AC_PROG_INSTALL 22 | AC_PROG_LIBTOOL 23 | AC_PROG_MAKE_SET 24 | 25 | # Check for rst utilities 26 | AC_CHECK_PROGS(RST2MAN, [rst2man rst2man.py], "no") 27 | if test "x$RST2MAN" = "xno"; then 28 | AC_MSG_WARN([rst2man not found - not building man pages]) 29 | fi 30 | AM_CONDITIONAL(HAVE_RST2MAN, [test "x$RST2MAN" != "xno"]) 31 | 32 | # Check for pkg-config 33 | PKG_PROG_PKG_CONFIG 34 | PKG_CHECK_MODULES([CURL], [libcurl]) 35 | 36 | # Checks for header files. 37 | AC_HEADER_STDC 38 | AC_CHECK_HEADERS([sys/stdlib.h]) 39 | 40 | # Check for python 41 | AC_CHECK_PROGS(PYTHON, [python3 python3.1 python3.2 python2.7 python2.6 python2.5 python2 python], "no") 42 | if test "x$PYTHON" = "xno"; then 43 | AC_MSG_ERROR([Python is needed to build this vmod, please install python.]) 44 | fi 45 | 46 | VARNISH_PREREQ([7.3.0]) 47 | VARNISH_VMODS([curl]) 48 | 49 | AX_CURLOPT_CHECK([CURLOPT_TIMEOUT_MS]) 50 | AX_CURLOPT_CHECK([CURLOPT_CONNECTTIMEOUT_MS]) 51 | AX_CURLOPT_CHECK([CURLOPT_UNIX_SOCKET_PATH]) 52 | 53 | VMOD_TESTS="$(cd $srcdir/src && echo tests/*.vtc)" 54 | AC_SUBST(VMOD_TESTS) 55 | 56 | AC_CONFIG_FILES([ 57 | Makefile 58 | src/Makefile 59 | ]) 60 | AC_OUTPUT 61 | 62 | AS_ECHO(" 63 | ==== $PACKAGE_STRING ==== 64 | 65 | varnish: $VARNISH_VERSION 66 | prefix: $prefix 67 | vmoddir: $vmoddir 68 | vcldir: $vcldir 69 | pkgvcldir: $pkgvcldir 70 | 71 | compiler: $CC 72 | cflags: $CFLAGS 73 | ldflags: $LDFLAGS 74 | ") 75 | -------------------------------------------------------------------------------- /m4/curl.m4: -------------------------------------------------------------------------------- 1 | # curl.m4 2 | # 3 | # serial 1 4 | 5 | AC_DEFUN([AX_CURLOPT_CHECK], [ 6 | save_CFLAGS="${CFLAGS}" 7 | save_LIBS="${LIBS}" 8 | CFLAGS="${CFLAGS} ${CURL_CFLAGS}" 9 | LIBS="${LIBS} ${CURL_LIBS}" 10 | AC_CACHE_CHECK([if curl supports $1], 11 | [ax_cv_curl_$1], 12 | [AC_COMPILE_IFELSE( 13 | [AC_LANG_PROGRAM([[ 14 | #include 15 | ]],[[ 16 | CURL *curl; 17 | curl = curl_easy_init(); 18 | curl_easy_setopt(curl, $1, 10); 19 | ]])], 20 | [ax_cv_curl_$1=yes], 21 | [ax_cv_curl_$1=no]) 22 | ]) 23 | if test "$ax_cv_curl_$1" = "yes"; then 24 | AC_DEFINE([HAVE_$1], [1], [Define if curl supports $1]) 25 | fi 26 | CFLAGS="${save_CFLAGS}" 27 | LIBS="${save_LIBS}" 28 | ]) 29 | -------------------------------------------------------------------------------- /man/vmod_curl.rst: -------------------------------------------------------------------------------- 1 | ========= 2 | vmod_curl 3 | ========= 4 | 5 | ------------------- 6 | Varnish cURL Module 7 | ------------------- 8 | 9 | SYNOPSIS 10 | ======== 11 | 12 | import curl; 13 | 14 | DESCRIPTION 15 | =========== 16 | 17 | Varnish Module that provides cURL bindings for Varnish so you can use 18 | Varnish as an HTTP client and fetch headers and bodies from backends. 19 | 20 | FUNCTIONS 21 | ========= 22 | 23 | get 24 | --- 25 | 26 | Prototype 27 | :: 28 | 29 | get(STRING) 30 | Return value 31 | VOID 32 | Description 33 | Performs a GET request to the given URL. A deprecated alias 34 | for this function is `fetch`. 35 | Example 36 | :: 37 | 38 | curl.get("http://example.com/test"); 39 | curl.free(); 40 | 41 | head 42 | ---- 43 | 44 | Prototype 45 | :: 46 | 47 | head(STRING) 48 | Return value 49 | VOID 50 | Description 51 | Performs a HEAD request to the given URL. 52 | Example 53 | :: 54 | 55 | curl.head("http://example.com/test"); 56 | curl.free(); 57 | 58 | post 59 | ---- 60 | 61 | Prototype 62 | :: 63 | 64 | post(STRING, STRING) 65 | Return value 66 | VOID 67 | Description 68 | Performs a POST request to the given URL. The second 69 | parameter are the POST parameters. 70 | Example 71 | :: 72 | 73 | curl.post("http://example.com/test", "a=b"); 74 | curl.free(); 75 | 76 | header 77 | ------ 78 | 79 | Prototype 80 | :: 81 | 82 | header(STRING) 83 | Return value 84 | STRING 85 | Description 86 | Returns the header named in the first argument. 87 | Example 88 | :: 89 | 90 | curl.get("http://example.com/test"); 91 | if (curl.header("X-Foo") == "bar") { 92 | ... 93 | } 94 | curl.free(); 95 | 96 | free 97 | ---- 98 | 99 | Prototype 100 | :: 101 | 102 | free() 103 | Return value 104 | VOID 105 | Description 106 | Free the memory used by headers. 107 | Not needed, will be handled automatically if it's not called. 108 | 109 | status 110 | ------ 111 | 112 | Prototype 113 | :: 114 | 115 | status() 116 | Return value 117 | INT 118 | Description 119 | Returns the HTTP status code. 120 | Example 121 | :: 122 | 123 | curl.get("http://example.com/test"); 124 | if (curl.status() == 404) { 125 | ... 126 | } 127 | curl.free(); 128 | 129 | error 130 | ----- 131 | 132 | Prototype 133 | :: 134 | 135 | error() 136 | Return value 137 | STRING 138 | Description 139 | Returns the HTTP error. 140 | 141 | body 142 | ---- 143 | 144 | Prototype 145 | :: 146 | 147 | body() 148 | Return value 149 | STRING 150 | Description 151 | Returns the HTTP body content. 152 | 153 | set_timeout 154 | ----------- 155 | 156 | Prototype 157 | :: 158 | 159 | set_timeout(INT) 160 | Return value 161 | VOID 162 | Description 163 | Sets the CURLOPT_TIMEOUT_MS option to the value of the first argument. 164 | 165 | set_connect_timeout 166 | ------------------- 167 | 168 | Prototype 169 | :: 170 | 171 | set_connect_timeout(INT) 172 | Return value 173 | VOID 174 | Description 175 | Sets the CURLOPT_CONNECTTIMEOUT_MS option to the value of the first argument. 176 | 177 | set_ssl_verify_peer 178 | ------------------- 179 | 180 | Prototype 181 | :: 182 | 183 | set_ssl_verify_peer(INT) 184 | Return value 185 | VOID 186 | Description 187 | Sets the CURLOPT_SSL_VERIFYPEER option to either 0L or 1L, depending on the boolean value of the first argument. 188 | 189 | set_ssl_verify_host 190 | ------------------- 191 | 192 | Prototype 193 | :: 194 | 195 | set_ssl_verify_host(INT) 196 | Return value 197 | VOID 198 | Description 199 | Sets the CURLOPT_SSL_VERIFYHOST option to either 0L or 1L, depending on the boolean value of the first argument. 200 | 201 | set_ssl_cafile 202 | -------------- 203 | 204 | Prototype 205 | :: 206 | 207 | set_ssl_cafile(STRING) 208 | Return value 209 | VOID 210 | Description 211 | Sets the CURLOPT_CAINFO option to the value of the first argument. 212 | 213 | set_ssl_capath 214 | -------------- 215 | 216 | Prototype 217 | :: 218 | 219 | set_ssl_capath(STRING) 220 | Return value 221 | VOID 222 | Description 223 | Sets the CURLOPT_CAPATH option to the value of the first argument. 224 | 225 | header_add 226 | ---------- 227 | 228 | Prototype 229 | :: 230 | 231 | header_add(STRING) 232 | Return value 233 | VOID 234 | Description 235 | Adds a custom request header. 236 | If you add a header that is otherwise generated and used by libcurl 237 | internally, your added one will be used instead. If you add a header 238 | with no content as in "Accept:" (no data on the right side of the 239 | colon), the internally used header will get disabled. Thus, using this 240 | option you can add new headers, replace internal headers and remove 241 | internal headers. To add a header with no content, make the content be 242 | two quotes: "" 243 | Example 244 | :: 245 | 246 | // copy Host: header from request 247 | curl.header_add("Host: " + req.http.Host); 248 | // disable Accept header generated by libcurl 249 | curl.header_add("Accept:"); 250 | // add X-curl-Request header with no content 251 | curl.header_add("X-curl-Request: " + curl.unescape("%22%22")); 252 | // alternative using long string syntax 253 | curl.header_add({"X-curl-Request: """}); 254 | 255 | header_add_all 256 | -------------- 257 | 258 | Prototype 259 | :: 260 | 261 | header_add_all() 262 | Return value 263 | VOID 264 | Description 265 | Adds all headers in the req object to the request. 266 | If used from a backend thread, the bereq object is used. 267 | Can be used in combination with header_add() and header_remove(). 268 | The rules in header_add() apply to header_add_all(). 269 | Example 270 | :: 271 | 272 | // copy all headers from the req object in context 273 | curl.header_add_all(); 274 | 275 | header_remove 276 | ------------- 277 | 278 | Prototype 279 | :: 280 | 281 | header_remove(STRING) 282 | Return value 283 | VOID 284 | Description 285 | Removes all custom request header fields matching the given header name. 286 | Only headers added by header_add() can be removed. To disable headers 287 | generated internally by libcurl *add* the header with no content. 288 | Example 289 | :: 290 | 291 | curl.header_remove("Host"); 292 | 293 | escape 294 | ------ 295 | 296 | Prototype 297 | :: 298 | 299 | escape(STRING) 300 | Return value 301 | STRING 302 | Description 303 | URL encodes the given string. 304 | 305 | unescape 306 | -------- 307 | 308 | Prototype 309 | :: 310 | 311 | unescape(STRING) 312 | Return value 313 | STRING 314 | Description 315 | URL decodes the given string. 316 | 317 | set_proxy 318 | --------- 319 | 320 | Prototype 321 | :: 322 | 323 | set_proxy(STRING) 324 | Return value 325 | VOID 326 | Description 327 | Set the proxy to use. A deprecated alias for this function is `proxy`. 328 | Example 329 | :: 330 | 331 | curl.set_proxy("http://user:secret@some.server.dom:8080/"); 332 | 333 | set_method 334 | ---------- 335 | 336 | Prototype 337 | :: 338 | 339 | set_method(STRING) 340 | Return value 341 | VOID 342 | Description 343 | Set a custom string for this request. 344 | This doesn't actually change how libcurl behaves or acts in 345 | regards to the particular request method, it will only change 346 | the actual string sent in the request. 347 | Example 348 | :: 349 | 350 | curl.set_method("PURGE"); 351 | curl.head("http://example.com/test"); 352 | 353 | BUGS 354 | ==== 355 | 356 | None. 357 | 358 | COPYRIGHT 359 | ========= 360 | 361 | Development of this VMOD has been sponsored by the Norwegian company 362 | Aspiro Music AS for usage on their WiMP music streaming service. 363 | 364 | This document is licensed under the same license as the 365 | libvmod-curl project. See LICENSE for details. 366 | 367 | * Copyright (c) 2011-2014 Varnish Software 368 | -------------------------------------------------------------------------------- /src/Makefile.am: -------------------------------------------------------------------------------- 1 | AM_CFLAGS = $(VARNISHAPI_CFLAGS) 2 | 3 | # Modules 4 | 5 | vmod_LTLIBRARIES = \ 6 | libvmod_curl.la 7 | 8 | libvmod_curl_la_CFLAGS = $(AM_CFLAGS) $(CURL_CFLAGS) 9 | libvmod_curl_la_LDFLAGS = $(VMOD_LDFLAGS) $(CURL_LIBS) 10 | libvmod_curl_la_SOURCES = \ 11 | vmod_curl.c \ 12 | debug_flags.h 13 | nodist_libvmod_curl_la_SOURCES = \ 14 | vcc_curl_if.c \ 15 | vcc_curl_if.h 16 | 17 | @BUILD_VMOD_CURL@ 18 | 19 | # Test suite 20 | 21 | AM_TESTS_ENVIRONMENT = \ 22 | PATH="$(abs_builddir):$(VARNISH_TEST_PATH):$(PATH)" \ 23 | LD_LIBRARY_PATH="$(VARNISH_LIBRARY_PATH)" 24 | TEST_EXTENSIONS = .vtc 25 | VTC_LOG_COMPILER = varnishtest -lv 26 | AM_VTC_LOG_FLAGS = \ 27 | -p vcl_path="$(abs_top_srcdir)/vcl:$(VARNISHAPI_VCLDIR)" \ 28 | -p vmod_path="$(abs_builddir)/.libs:$(vmoddir):$(VARNISHAPI_VMODDIR)" 29 | TESTS = @VMOD_TESTS@ 30 | 31 | # Documentation 32 | 33 | dist_doc_DATA = \ 34 | vmod_curl.vcc \ 35 | $(TESTS) 36 | 37 | dist_man_MANS = \ 38 | vmod_curl.3 39 | 40 | 41 | .rst.1: 42 | $(AM_V_GEN) $(RST2MAN) $< $@ 43 | -------------------------------------------------------------------------------- /src/debug_flags.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2011-2016 Varnish Software 3 | * All rights reserved. 4 | * 5 | * Redistribution and use in source and binary forms, with or without 6 | * modification, are permitted provided that the following conditions 7 | * are met: 8 | * 1. Redistributions of source code must retain the above copyright 9 | * notice, this list of conditions and the following disclaimer. 10 | * 2. Redistributions in binary form must reproduce the above copyright 11 | * notice, this list of conditions and the following disclaimer in the 12 | * documentation and/or other materials provided with the distribution. 13 | * 14 | * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND 15 | * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 16 | * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 17 | * ARE DISCLAIMED. IN NO EVENT SHALL AUTHOR OR CONTRIBUTORS BE LIABLE 18 | * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 19 | * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS 20 | * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) 21 | * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 22 | * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY 23 | * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF 24 | * SUCH DAMAGE. 25 | * 26 | * N.B. This should be kept in sync with set_debug definition in .vcc 27 | */ 28 | 29 | DBG(NONE, none, ( 0)) 30 | DBG(TEXT, text, (1<<0)) 31 | DBG(HEADER_IN, header_in, (1<<1)) 32 | DBG(HEADER_OUT, header_out, (1<<2)) 33 | DBG(DATA_IN, data_in, (1<<3)) 34 | DBG(DATA_OUT, data_out, (1<<4)) 35 | -------------------------------------------------------------------------------- /src/tests/test01.vtc: -------------------------------------------------------------------------------- 1 | varnishtest "Test curl vmod" 2 | 3 | server s1 { 4 | rxreq 5 | expect req.method == GET 6 | txresp -hdr "Foo: bar" -body "Test" 7 | accept 8 | rxreq 9 | expect req.method == GET 10 | txresp -hdr "Foo: bar" 11 | accept 12 | rxreq 13 | expect req.method == HEAD 14 | expect req.bodylen == 0 15 | txresp -hdr "Foo: bar" 16 | accept 17 | rxreq 18 | expect req.method == POST 19 | expect req.bodylen == 3 20 | txresp -hdr "Foo: bar" 21 | accept 22 | rxreq 23 | expect req.method == GET 24 | expect req.url == "http://example.com/" 25 | expect req.http.proxy-authorization ~ "Basic" 26 | txresp -hdr "Foo: bar" 27 | accept 28 | rxreq 29 | expect req.method == GET 30 | expect req.url == "http://example.com/" 31 | expect req.http.proxy-authorization ~ "Basic" 32 | txresp -hdr "Foo: bar" 33 | } -start 34 | 35 | varnish v1 -vcl+backend { 36 | import curl; 37 | sub vcl_recv { 38 | if (req.http.func == "GET") { 39 | curl.get("http://${s1_addr}:${s1_port}"); 40 | return (synth(404)); 41 | } elsif (req.http.func == "FETCH") { 42 | curl.fetch("http://${s1_addr}:${s1_port}"); 43 | return (synth(404)); 44 | } elsif (req.http.func == "HEAD") { 45 | curl.head("http://${s1_addr}:${s1_port}"); 46 | return (synth(404)); 47 | } elsif (req.http.func == "POST") { 48 | curl.post("http://${s1_addr}:${s1_port}", "a=b"); 49 | return (synth(404)); 50 | } elsif (req.http.func == "SET_PROXY") { 51 | curl.set_proxy("http://user:secret@${s1_addr}:${s1_port}"); 52 | curl.get("http://example.com/"); 53 | return (synth(404)); 54 | } elsif (req.http.func == "PROXY") { 55 | curl.proxy("http://user:secret@${s1_addr}:${s1_port}"); 56 | curl.get("http://example.com/"); 57 | return (synth(404)); 58 | } 59 | } 60 | 61 | sub vcl_synth { 62 | set resp.status = curl.status(); 63 | set resp.http.foo = curl.header("foo"); 64 | set resp.http.cl = curl.header("content-length"); 65 | set resp.http.body = curl.body(); 66 | } 67 | } -start 68 | 69 | client c1 { 70 | txreq -url "/" -hdr "func: GET" 71 | rxresp 72 | expect resp.http.foo == "bar" 73 | expect resp.http.cl == "4" 74 | expect resp.status == 200 75 | expect resp.http.body == "Test" 76 | } -run 77 | 78 | client c2 { 79 | txreq -url "/" -hdr "func: FETCH" 80 | rxresp 81 | expect resp.http.foo == "bar" 82 | expect resp.http.cl == "0" 83 | expect resp.status == 200 84 | expect resp.http.body == "" 85 | } -run 86 | 87 | client c3 { 88 | txreq -url "/" -hdr "func: HEAD" 89 | rxresp 90 | expect resp.http.foo == "bar" 91 | expect resp.http.cl == "0" 92 | expect resp.status == 200 93 | expect resp.http.body == "" 94 | } -run 95 | 96 | client c4 { 97 | txreq -url "/" -hdr "func: POST" 98 | rxresp 99 | expect resp.http.foo == "bar" 100 | expect resp.http.cl == "0" 101 | expect resp.status == 200 102 | expect resp.http.body == "" 103 | } -run 104 | 105 | client c5 { 106 | txreq -url "/" -hdr "func: SET_PROXY" 107 | rxresp 108 | expect resp.http.foo == "bar" 109 | expect resp.http.cl == "0" 110 | expect resp.status == 200 111 | expect resp.http.body == "" 112 | } -run 113 | 114 | client c6 { 115 | txreq -url "/" -hdr "func: PROXY" 116 | rxresp 117 | expect resp.http.foo == "bar" 118 | expect resp.http.cl == "0" 119 | expect resp.status == 200 120 | expect resp.http.body == "" 121 | } -run 122 | -------------------------------------------------------------------------------- /src/tests/test02.vtc: -------------------------------------------------------------------------------- 1 | varnishtest "Test data timeout" 2 | 3 | server s1 { 4 | rxreq 5 | delay 2 6 | txresp -hdr "Foo: bar" 7 | } -start 8 | 9 | varnish v1 -vcl+backend { 10 | import curl; 11 | import std; 12 | 13 | sub vcl_recv { 14 | curl.set_timeout(1000); 15 | curl.set_connect_timeout(1000); 16 | std.timestamp("curl:start"); 17 | curl.get("http://${s1_addr}:${s1_port}"); 18 | return (synth(200)); 19 | } 20 | 21 | sub vcl_synth { 22 | std.timestamp("curl:end"); 23 | set resp.http.x-status = curl.status(); 24 | set resp.http.x-error = curl.error(); 25 | 26 | # Fix for RHEL5 old curl lib 27 | if (resp.http.x-error == "a timeout was reached") { 28 | set resp.http.x-error = "Timeout was reached"; 29 | } 30 | } 31 | } -start 32 | 33 | logexpect l1 -v v1 -g request { 34 | expect * 1001 Timestamp {curl:start: \S+ 0\.\d+ 0\.\d+} 35 | expect * = Timestamp {curl:end: \S+ 1\.0\d+ 1\.0\d+} 36 | } -start 37 | 38 | client c1 { 39 | txreq 40 | rxresp 41 | expect resp.http.x-status == 0 42 | expect resp.http.x-error == "Timeout was reached" 43 | } -run 44 | 45 | logexpect l1 -wait 46 | -------------------------------------------------------------------------------- /src/tests/test03.vtc: -------------------------------------------------------------------------------- 1 | varnishtest "Test objects are cleared out on get when restarting" 2 | 3 | server s1 { 4 | rxreq 5 | txresp -hdr "Foo: bar" 6 | rxreq 7 | txresp -hdr "Foo: bar" 8 | } -start 9 | 10 | varnish v1 -vcl+backend { 11 | import curl; 12 | sub vcl_deliver { 13 | curl.get("http://localhost:65500"); 14 | if (req.restarts < 1) { 15 | return (restart); 16 | } 17 | set resp.http.x-error = curl.error(); 18 | 19 | # Fix for RHEL5 old curl lib 20 | if (resp.http.x-error == "couldn't connect to server") { 21 | set resp.http.x-error = "Couldn't connect to server"; 22 | } 23 | } 24 | } -start 25 | 26 | client c1 { 27 | txreq -url "/" 28 | rxresp 29 | expect resp.status == 200 30 | expect resp.http.x-error == "Couldn't connect to server" 31 | } 32 | 33 | client c1 -run 34 | client c1 -run 35 | -------------------------------------------------------------------------------- /src/tests/test04.vtc: -------------------------------------------------------------------------------- 1 | varnishtest "Test strange headers" 2 | 3 | server s1 { 4 | rxreq 5 | txresp -hdr "Foo: asdf " 6 | 7 | accept 8 | rxreq 9 | txresp -hdr "Foo: " 10 | 11 | accept 12 | rxreq 13 | txresp -hdr "Foo:" 14 | 15 | accept 16 | rxreq 17 | txresp -hdr ":" 18 | 19 | accept 20 | rxreq 21 | txresp -hdr "Foo" 22 | } -start 23 | 24 | varnish v1 -vcl+backend { 25 | import curl; 26 | sub vcl_recv { 27 | curl.get("http://${s1_addr}:${s1_port}"); 28 | return (synth(200)); 29 | } 30 | 31 | sub vcl_synth { 32 | set resp.http.x-status = curl.status(); 33 | if (curl.error()) { 34 | set resp.http.x-error = curl.error(); 35 | } else { 36 | set resp.http.x-error = ""; 37 | } 38 | if (curl.header("foo")) { 39 | set resp.http.x-foo = curl.header("foo"); 40 | } else { 41 | set resp.http.x-foo = ""; 42 | } 43 | } 44 | } -start 45 | 46 | client c1 { 47 | txreq -url "/test1" 48 | rxresp 49 | expect resp.http.x-status == 200 50 | expect resp.http.x-error == "" 51 | expect resp.http.x-foo == "asdf" 52 | } -run 53 | 54 | client c2 { 55 | txreq -url "/test2" 56 | rxresp 57 | expect resp.http.x-status == 200 58 | expect resp.http.x-error == "" 59 | expect resp.http.x-foo == "" 60 | } -run 61 | 62 | client c3 { 63 | txreq -url "/test3" 64 | rxresp 65 | expect resp.http.x-status == 200 66 | expect resp.http.x-error == "" 67 | expect resp.http.x-foo == "" 68 | } -run 69 | 70 | client c4 { 71 | txreq -url "/test4" 72 | rxresp 73 | expect resp.http.x-status == 200 74 | expect resp.http.x-error == "" 75 | expect resp.http.x-foo == "" 76 | } -run 77 | 78 | client c5 { 79 | txreq -url "/test5" 80 | rxresp 81 | expect resp.http.x-status == 200 82 | expect resp.http.x-error == "" 83 | expect resp.http.x-foo == "" 84 | } -run 85 | -------------------------------------------------------------------------------- /src/tests/test05.vtc: -------------------------------------------------------------------------------- 1 | varnishtest "Test string escape" 2 | 3 | server s1 { 4 | rxreq 5 | txresp 6 | } -start 7 | 8 | varnish v1 -vcl+backend { 9 | import curl; 10 | sub vcl_deliver { 11 | set resp.http.x-escaped = curl.escape({"abcd%#=/[]"}); 12 | set resp.http.x-unescaped = curl.unescape(resp.http.x-escaped); 13 | } 14 | } -start 15 | 16 | client c1 { 17 | txreq 18 | rxresp 19 | expect resp.status == 200 20 | expect resp.http.x-escaped == "abcd%25%23%3D%2F%5B%5D" 21 | expect resp.http.x-unescaped == "abcd%#=/[]" 22 | } -run 23 | -------------------------------------------------------------------------------- /src/tests/test06.vtc: -------------------------------------------------------------------------------- 1 | varnishtest "Test header_add" 2 | 3 | server s1 { 4 | rxreq 5 | expect req.http.foo == "bar" 6 | txresp 7 | accept 8 | rxreq 9 | expect req.http.extra == "1" 10 | expect req.http.foo != "bar" 11 | txresp 12 | } -start 13 | 14 | varnish v1 -vcl+backend { 15 | import curl; 16 | sub vcl_recv { 17 | if (req.http.extra) { 18 | curl.header_add("extra: 1"); 19 | } else { 20 | curl.header_add("foo: bar"); 21 | } 22 | curl.get("http://${s1_addr}:${s1_port}"); 23 | return (synth(200)); 24 | } 25 | } -start 26 | 27 | client c1 { 28 | txreq -url "/" 29 | rxresp 30 | } -run 31 | 32 | client c2 { 33 | txreq -url "/" -hdr "Extra: 1" 34 | rxresp 35 | } -run 36 | -------------------------------------------------------------------------------- /src/tests/test07.vtc: -------------------------------------------------------------------------------- 1 | varnishtest "Test header_remove" 2 | 3 | server s1 { 4 | rxreq 5 | expect req.http.foo == "bar" 6 | expect req.http.foobar != "bar" 7 | txresp 8 | } -start 9 | 10 | varnish v1 -vcl+backend { 11 | import curl; 12 | sub vcl_recv { 13 | curl.header_add("foobar: bar"); 14 | curl.header_add("foo: bar"); 15 | curl.header_add("foobar: bar"); 16 | curl.header_remove("foobar"); 17 | curl.get("http://${s1_addr}:${s1_port}"); 18 | return (synth(200)); 19 | } 20 | } -start 21 | 22 | client c1 { 23 | txreq -url "/" 24 | rxresp 25 | } -run 26 | -------------------------------------------------------------------------------- /src/tests/test08.vtc: -------------------------------------------------------------------------------- 1 | varnishtest "Test set_method" 2 | 3 | server s1 { 4 | rxreq 5 | expect req.method == PURGE 6 | txresp 7 | accept 8 | rxreq 9 | expect req.method == HEAD 10 | txresp 11 | } -start 12 | 13 | varnish v1 -vcl+backend { 14 | import curl; 15 | sub vcl_recv { 16 | curl.set_method("PURGE"); 17 | curl.head("http://${s1_addr}:${s1_port}"); 18 | curl.head("http://${s1_addr}:${s1_port}"); 19 | return (synth(200)); 20 | } 21 | } -start 22 | 23 | client c1 { 24 | txreq -url "/" 25 | rxresp 26 | } -run 27 | -------------------------------------------------------------------------------- /src/tests/test09.vtc: -------------------------------------------------------------------------------- 1 | varnishtest "Test curl vmod" 2 | 3 | server s1 { 4 | rxreq 5 | expect req.method == GET 6 | txresp -hdr "Foo: bar" -body "Test" 7 | accept 8 | rxreq 9 | expect req.method == GET 10 | txresp -hdr "Foo: bar" 11 | accept 12 | rxreq 13 | expect req.method == HEAD 14 | expect req.bodylen == 0 15 | txresp -hdr "Foo: bar" 16 | accept 17 | rxreq 18 | expect req.method == POST 19 | expect req.bodylen == 3 20 | txresp -hdr "Foo: bar" 21 | accept 22 | rxreq 23 | expect req.method == GET 24 | expect req.url == "http://example.com/" 25 | expect req.http.proxy-authorization ~ "Basic" 26 | txresp -hdr "Foo: bar" 27 | accept 28 | rxreq 29 | expect req.method == GET 30 | expect req.url == "http://example.com/" 31 | expect req.http.proxy-authorization ~ "Basic" 32 | txresp -hdr "Foo: bar" 33 | } -start 34 | 35 | server s2 -repeat 6 { 36 | rxreq 37 | txresp 38 | } -start 39 | 40 | varnish v1 -vcl+backend { 41 | import curl; 42 | 43 | sub vcl_recv { 44 | return (pass); 45 | } 46 | 47 | sub vcl_backend_fetch { 48 | set bereq.backend = s2; 49 | } 50 | 51 | sub vcl_backend_response { 52 | if (bereq.http.func == "GET") { 53 | curl.get("http://${s1_addr}:${s1_port}"); 54 | } elsif (bereq.http.func == "FETCH") { 55 | curl.fetch("http://${s1_addr}:${s1_port}"); 56 | } elsif (bereq.http.func == "HEAD") { 57 | curl.head("http://${s1_addr}:${s1_port}"); 58 | } elsif (bereq.http.func == "POST") { 59 | curl.post("http://${s1_addr}:${s1_port}", "a=b"); 60 | } elsif (bereq.http.func == "SET_PROXY") { 61 | curl.set_proxy("http://user:secret@${s1_addr}:${s1_port}"); 62 | curl.get("http://example.com/"); 63 | } elsif (bereq.http.func == "PROXY") { 64 | curl.proxy("http://user:secret@${s1_addr}:${s1_port}"); 65 | curl.get("http://example.com/"); 66 | } 67 | 68 | set beresp.status = curl.status(); 69 | set beresp.http.foo = curl.header("foo"); 70 | set beresp.http.cl = curl.header("content-length"); 71 | set beresp.http.body = curl.body(); 72 | } 73 | } -start 74 | 75 | client c1 { 76 | txreq -url "/" -hdr "func: GET" 77 | rxresp 78 | expect resp.http.foo == "bar" 79 | expect resp.http.cl == "4" 80 | expect resp.status == 200 81 | expect resp.http.body == "Test" 82 | } -run 83 | 84 | client c2 { 85 | txreq -url "/" -hdr "func: FETCH" 86 | rxresp 87 | expect resp.http.foo == "bar" 88 | expect resp.http.cl == "0" 89 | expect resp.status == 200 90 | expect resp.http.body == "" 91 | } -run 92 | 93 | client c3 { 94 | txreq -url "/" -hdr "func: HEAD" 95 | rxresp 96 | expect resp.http.foo == "bar" 97 | expect resp.http.cl == "0" 98 | expect resp.status == 200 99 | expect resp.http.body == "" 100 | } -run 101 | 102 | client c4 { 103 | txreq -url "/" -hdr "func: POST" 104 | rxresp 105 | expect resp.http.foo == "bar" 106 | expect resp.http.cl == "0" 107 | expect resp.status == 200 108 | expect resp.http.body == "" 109 | } -run 110 | 111 | client c5 { 112 | txreq -url "/" -hdr "func: SET_PROXY" 113 | rxresp 114 | expect resp.http.foo == "bar" 115 | expect resp.http.cl == "0" 116 | expect resp.status == 200 117 | expect resp.http.body == "" 118 | } -run 119 | 120 | client c6 { 121 | txreq -url "/" -hdr "func: PROXY" 122 | rxresp 123 | expect resp.http.foo == "bar" 124 | expect resp.http.cl == "0" 125 | expect resp.status == 200 126 | expect resp.http.body == "" 127 | } -run 128 | -------------------------------------------------------------------------------- /src/tests/test10.vtc: -------------------------------------------------------------------------------- 1 | varnishtest "Test header_add_all" 2 | 3 | server s1 { 4 | rxreq 5 | expect req.http.foo == "bar" 6 | expect req.http.be == "0" 7 | txresp 8 | accept 9 | rxreq 10 | expect req.http.foobe == "barbe" 11 | expect req.http.be == "2" 12 | txresp 13 | } -start 14 | 15 | varnish v1 -vcl+backend { 16 | import curl; 17 | sub vcl_recv { 18 | if (req.http.be == "0") { 19 | curl.header_add_all(); 20 | curl.get("http://${s1_addr}:${s1_port}/"); 21 | return (synth(200)); 22 | } 23 | 24 | set req.http.be = "2"; 25 | } 26 | 27 | sub vcl_backend_fetch { 28 | set bereq.http.foobe = "barbe"; 29 | curl.header_add_all(); 30 | curl.get("http://${s1_addr}:${s1_port}/"); 31 | return (abandon); 32 | } 33 | } -start 34 | 35 | client c1 { 36 | txreq -url "/" -hdr "be: 0" -hdr "foo: bar" 37 | rxresp 38 | } -run 39 | 40 | client c2 { 41 | txreq -url "/" -hdr "be: 1" 42 | rxresp 43 | } -run 44 | -------------------------------------------------------------------------------- /src/tests/test11.vtc: -------------------------------------------------------------------------------- 1 | varnishtest "Test sub-second resolution timeouts" 2 | 3 | server s1 -repeat 2 { 4 | rxreq 5 | delay 2 6 | txresp -hdr "Foo: bar" 7 | } -start 8 | 9 | varnish v1 -vcl+backend { 10 | import curl; 11 | import std; 12 | 13 | sub vcl_recv { 14 | if (req.url == "/") { 15 | curl.set_timeout(200); 16 | curl.set_connect_timeout(200); 17 | } else { 18 | curl.set_timeout(1310); 19 | curl.set_connect_timeout(1310); 20 | } 21 | std.timestamp("curl:start"); 22 | curl.get("http://${s1_addr}:${s1_port}"); 23 | return (synth(200)); 24 | } 25 | 26 | sub vcl_synth { 27 | std.timestamp("curl:end"); 28 | set resp.http.x-status = curl.status(); 29 | set resp.http.x-error = curl.error(); 30 | 31 | # Fix for RHEL5 old curl lib 32 | if (resp.http.x-error == "a timeout was reached") { 33 | set resp.http.x-error = "Timeout was reached"; 34 | } 35 | } 36 | } -start 37 | 38 | logexpect l1 -v v1 -g request { 39 | expect * 1001 Timestamp {curl:start: \S+ 0\.0\d+ 0\.\d+} 40 | expect * = Timestamp {curl:end: \S+ 0\.2\d+ 0\.2\d+} 41 | 42 | expect * 1002 Timestamp {curl:start: \S+ 0\.\d+ 0\.\d+} 43 | expect * = Timestamp {curl:end: \S+ 1\.3\d+ 1\.3\d+} 44 | } -start 45 | 46 | client c1 { 47 | txreq -url "/" 48 | rxresp 49 | expect resp.http.x-status == 0 50 | expect resp.http.x-error == "Timeout was reached" 51 | 52 | txreq -url "/other" 53 | rxresp 54 | expect resp.http.x-status == 0 55 | expect resp.http.x-error == "Timeout was reached" 56 | } -run 57 | 58 | logexpect l1 -wait 59 | -------------------------------------------------------------------------------- /src/tests/test12.vtc: -------------------------------------------------------------------------------- 1 | varnishtest "Test the body() function behaviour via synthetic and via header assignment" 2 | 3 | server s1 { 4 | rxreq 5 | expect req.method == GET 6 | # The resp has a body containing special char not allowed in headers 7 | txresp -body "Te\r\nst" 8 | } -start 9 | 10 | # validate_headers: this test deliberately tests returning line breaks 11 | # in a header. Do not use in production. 12 | varnish v1 -arg "-p feature=-validate_headers" -vcl+backend { 13 | import curl; 14 | 15 | sub vcl_recv { 16 | if (req.http.func == "GET") { 17 | curl.get("http://${s1_addr}:${s1_port}"); 18 | return (synth(404)); 19 | } 20 | } 21 | 22 | sub vcl_synth { 23 | set resp.status = curl.status(); 24 | set resp.http.cl = curl.header("content-length"); 25 | 26 | # Headers can't contain special chars, therefore we will get a truncated resp body [1] 27 | set resp.http.body = curl.body(); 28 | 29 | # curl.body() via synthetic will return the complete response body [2] 30 | synthetic(curl.body()); 31 | 32 | return(deliver); 33 | } 34 | } -start 35 | 36 | client c1 { 37 | txreq -url "/" -hdr "func: GET" 38 | rxresp 39 | expect resp.http.cl == "6" 40 | expect resp.status == 200 41 | # [1] 42 | expect resp.http.body == "Te" 43 | # [2] 44 | expect resp.body == "Te\r\nst" 45 | } -run 46 | -------------------------------------------------------------------------------- /src/tests/test13.vtc: -------------------------------------------------------------------------------- 1 | varnishtest "Skip the C-L when copying the headers" 2 | 3 | server s1 { 4 | rxreq 5 | expect req.http.content-length == 6 | txresp 7 | accept 8 | rxreq 9 | expect req.http.transfer-encoding == 10 | txresp 11 | accept 12 | rxreq 13 | expect req.http.content-length-custom != 14 | txresp 15 | } -start 16 | 17 | varnish v1 -vcl+backend { 18 | import curl; 19 | 20 | sub vcl_recv { 21 | curl.header_add_all(); 22 | curl.get("http://${s1_addr}:${s1_port}/"); 23 | return (synth(curl.status())); 24 | } 25 | } -start 26 | 27 | client c1 { 28 | txreq -req POST -body foo 29 | rxresp 30 | expect resp.status == 200 31 | send "POST / HTTP/1.1\r\n" 32 | send "Transfer-Encoding: chunked\r\n" 33 | send "\r\n" 34 | send "3\r\n" 35 | send "foo\r\n" 36 | send "0\r\n" 37 | send "\r\n" 38 | rxresp 39 | expect resp.status == 200 40 | txreq -hdr "Content-Length-Custom: 1234" 41 | rxresp 42 | expect resp.status == 200 43 | } -run 44 | -------------------------------------------------------------------------------- /src/tests/test14.vtc: -------------------------------------------------------------------------------- 1 | varnishtest "Test set_debug" 2 | 3 | server s1 { 4 | rxreq 5 | expect req.bodylen == 7 6 | txresp -body "foo" 7 | accept 8 | rxreq 9 | expect req.bodylen == 7 10 | txresp -body "foo" 11 | accept 12 | rxreq 13 | expect req.bodylen == 8 14 | txresp -body "bar" 15 | } -start 16 | 17 | varnish v1 -vcl+backend { 18 | import curl; 19 | 20 | sub vcl_recv { 21 | curl.set_debug(text); 22 | curl.set_debug(header_in); 23 | curl.set_debug(header_out); 24 | curl.set_debug(data_in); 25 | curl.set_debug(data_out); 26 | curl.post("http://${s1_addr}:${s1_port}/", "foo=bar"); 27 | curl.set_debug(none); 28 | curl.post("http://${s1_addr}:${s1_port}/", "foo=bar"); 29 | curl.set_debug(text); 30 | curl.post("http://${s1_addr}:${s1_port}/", "baz=quux"); 31 | return (synth(curl.status())); 32 | } 33 | } -start 34 | 35 | logexpect l1 -v v1 -g request { 36 | expect * 1001 Debug {text: .* ${s1_addr}} 37 | expect * = Debug {header_out: POST} 38 | expect * = Debug {data_out: foo=bar} 39 | expect * = Debug {header_in: HTTP/1.1} 40 | expect * = Debug {data_in: foo} 41 | expect * = Debug {text: Connection} 42 | expect * = Debug {data_out: baz=quux} 43 | expect * = Debug {data_in: bar} 44 | } -start 45 | 46 | client c1 { 47 | txreq 48 | rxresp 49 | } -run 50 | 51 | logexpect l1 -wait 52 | -------------------------------------------------------------------------------- /src/vmod_curl.c: -------------------------------------------------------------------------------- 1 | #include "config.h" 2 | 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | 9 | #include 10 | #include 11 | 12 | #ifndef VRT_H_INCLUDED 13 | # include 14 | #endif 15 | 16 | #ifndef VDEF_H_INCLUDED 17 | # include 18 | #endif 19 | 20 | #include "vsb.h" 21 | #include "vcc_curl_if.h" 22 | 23 | #ifndef VRT_CTX 24 | #define VRT_CTX const struct vrt_ctx *ctx 25 | #endif 26 | 27 | struct hdr { 28 | char *key; 29 | char *value; 30 | VTAILQ_ENTRY(hdr) list; 31 | }; 32 | 33 | struct req_hdr { 34 | char *value; 35 | VTAILQ_ENTRY(req_hdr) list; 36 | }; 37 | 38 | enum debug_flags { 39 | #define DBG(u,l,v) DBG_##u = v, 40 | #include "debug_flags.h" 41 | #undef DBG 42 | }; 43 | 44 | struct vmod_curl { 45 | unsigned magic; 46 | #define VMOD_CURL_MAGIC 0xBBB0C87C 47 | unsigned vxid; 48 | long status; 49 | long timeout; 50 | long connect_timeout; 51 | char flags; 52 | #define F_SSL_VERIFY_PEER (1 << 0) 53 | #define F_SSL_VERIFY_HOST (1 << 1) 54 | #define F_METHOD_GET (1 << 2) 55 | #define F_METHOD_HEAD (1 << 3) 56 | #define F_METHOD_POST (1 << 4) 57 | const char *url; 58 | const char *method; 59 | const char *postfields; 60 | const char *error; 61 | const char *cafile; 62 | const char *capath; 63 | #ifdef HAVE_CURLOPT_UNIX_SOCKET_PATH 64 | const char *sun_path; 65 | #endif 66 | VTAILQ_HEAD(, hdr) headers; 67 | VTAILQ_HEAD(, req_hdr) req_headers; 68 | const char *proxy; 69 | struct vsb *body; 70 | const struct vrt_ctx *vctx; 71 | unsigned debug_flags; 72 | }; 73 | 74 | static void cm_clear(struct vmod_curl *c); 75 | 76 | int 77 | event_function(VRT_CTX, struct vmod_priv *priv, enum vcl_event_e e) 78 | { 79 | (void)ctx; 80 | (void)priv; 81 | if (e != VCL_EVENT_LOAD) 82 | return (0); 83 | curl_global_init(CURL_GLOBAL_ALL); 84 | return (0); 85 | } 86 | 87 | int 88 | vmod_event_function(VRT_CTX, struct vmod_priv *priv, enum vcl_event_e e) 89 | { 90 | return (event_function(ctx, priv, e)); 91 | } 92 | 93 | static void 94 | cm_init(struct vmod_curl *c) 95 | { 96 | c->magic = VMOD_CURL_MAGIC; 97 | VTAILQ_INIT(&c->headers); 98 | VTAILQ_INIT(&c->req_headers); 99 | c->body = VSB_new_auto(); 100 | cm_clear(c); 101 | } 102 | 103 | static void 104 | cm_clear_body(struct vmod_curl *c) 105 | { 106 | CHECK_OBJ_NOTNULL(c, VMOD_CURL_MAGIC); 107 | 108 | VSB_clear(c->body); 109 | } 110 | 111 | static void 112 | cm_clear_headers(struct vmod_curl *c) 113 | { 114 | struct hdr *h, *h2; 115 | 116 | CHECK_OBJ_NOTNULL(c, VMOD_CURL_MAGIC); 117 | 118 | VTAILQ_FOREACH_SAFE(h, &c->headers, list, h2) { 119 | VTAILQ_REMOVE(&c->headers, h, list); 120 | free(h->key); 121 | free(h->value); 122 | free(h); 123 | } 124 | } 125 | 126 | static void 127 | cm_clear_req_headers(struct vmod_curl *c) 128 | { 129 | struct req_hdr *rh, *rh2; 130 | 131 | CHECK_OBJ_NOTNULL(c, VMOD_CURL_MAGIC); 132 | 133 | VTAILQ_FOREACH_SAFE(rh, &c->req_headers, list, rh2) { 134 | VTAILQ_REMOVE(&c->req_headers, rh, list); 135 | free(rh->value); 136 | free(rh); 137 | } 138 | } 139 | 140 | static void 141 | cm_clear_fetch_state(struct vmod_curl *c) 142 | { 143 | CHECK_OBJ_NOTNULL(c, VMOD_CURL_MAGIC); 144 | 145 | c->flags &= ~(F_METHOD_GET | F_METHOD_HEAD | F_METHOD_POST); 146 | cm_clear_body(c); 147 | cm_clear_headers(c); 148 | } 149 | 150 | static void 151 | cm_clear(struct vmod_curl *c) 152 | { 153 | CHECK_OBJ_NOTNULL(c, VMOD_CURL_MAGIC); 154 | 155 | cm_clear_fetch_state(c); 156 | 157 | c->connect_timeout = -1; 158 | c->timeout = -1; 159 | c->cafile = NULL; 160 | c->capath = NULL; 161 | #ifdef HAVE_CURLOPT_UNIX_SOCKET_PATH 162 | c->sun_path = NULL; 163 | #endif 164 | c->error = NULL; 165 | c->flags = 0; 166 | c->method = NULL; 167 | c->proxy = NULL; 168 | c->status = 0; 169 | c->vxid = 0; 170 | c->vctx = NULL; 171 | c->debug_flags = 0; 172 | } 173 | 174 | static int 175 | cm_debug(CURL *handle, curl_infotype type, char *data, size_t size, 176 | void *userdata) 177 | { 178 | struct vmod_curl *c; 179 | 180 | (void)handle; 181 | 182 | CAST_OBJ_NOTNULL(c, userdata, VMOD_CURL_MAGIC); 183 | CHECK_OBJ_NOTNULL(c->vctx, VRT_CTX_MAGIC); 184 | AN(c->debug_flags); 185 | #define DBG(u,l,v) \ 186 | if ((unsigned)type == CURLINFO_##u && \ 187 | (c->debug_flags & v) != 0) \ 188 | VSLb(c->vctx->vsl, SLT_Debug, #l ": %.*s", \ 189 | (int)size, data); 190 | #include "debug_flags.h" 191 | #undef DBG 192 | return (0); 193 | } 194 | 195 | void 196 | free_func(VRT_CTX, void *p) 197 | { 198 | struct vmod_curl *c; 199 | 200 | CAST_OBJ_NOTNULL(c, p, VMOD_CURL_MAGIC); 201 | 202 | cm_clear_req_headers(c); 203 | cm_clear(c); 204 | VSB_destroy(&c->body); 205 | 206 | FREE_OBJ(c); 207 | } 208 | 209 | static const struct vmod_priv_methods priv_curl_methods[1] = {{ 210 | .magic = VMOD_PRIV_METHODS_MAGIC, 211 | .type = "cURL", 212 | .fini = free_func 213 | }}; 214 | 215 | static struct vmod_curl * 216 | cm_get(struct vmod_priv *priv) 217 | { 218 | struct vmod_curl *cm; 219 | 220 | if (priv->priv == NULL) { 221 | ALLOC_OBJ(cm, VMOD_CURL_MAGIC); 222 | cm_init(cm); 223 | priv->priv = cm; 224 | priv->methods = priv_curl_methods; 225 | } else 226 | CAST_OBJ_NOTNULL(cm, priv->priv, VMOD_CURL_MAGIC); 227 | 228 | return (cm); 229 | } 230 | 231 | static size_t 232 | recv_data(void *ptr, size_t size, size_t nmemb, void *s) 233 | { 234 | struct vmod_curl *vc; 235 | 236 | CAST_OBJ_NOTNULL(vc, s, VMOD_CURL_MAGIC); 237 | 238 | VSB_bcat(vc->body, ptr, size * nmemb); 239 | return (size * nmemb); 240 | } 241 | 242 | static size_t 243 | recv_hdrs(void *ptr, size_t size, size_t nmemb, void *s) 244 | { 245 | struct vmod_curl *vc; 246 | struct hdr *h; 247 | char *split; 248 | ptrdiff_t keylen, vallen; 249 | 250 | CAST_OBJ_NOTNULL(vc, s, VMOD_CURL_MAGIC); 251 | 252 | split = memchr(ptr, ':', size * nmemb); 253 | if (split == NULL) 254 | return (size * nmemb); 255 | 256 | keylen = split - (char *)ptr; 257 | assert(keylen >= 0); 258 | if (keylen == 0) 259 | return (size * nmemb); 260 | 261 | h = calloc(1, sizeof(struct hdr)); 262 | AN(h); 263 | h->key = strndup(ptr, keylen); 264 | AN(h->key); 265 | 266 | vallen = size * nmemb - keylen; 267 | assert(vallen > 0); /* Counts ':' so always larger than 0 */ 268 | split++; /* Drop ':' */ 269 | vallen--; 270 | while (vallen > 0 && isspace(*split)) { 271 | split++; 272 | vallen--; 273 | } 274 | while (vallen > 0 && isspace(*(split + vallen - 1))) 275 | vallen--; 276 | h->value = strndup(split, vallen); 277 | AN(h->value); 278 | 279 | VTAILQ_INSERT_HEAD(&vc->headers, h, list); 280 | 281 | return (size * nmemb); 282 | } 283 | 284 | static void 285 | cm_perform(struct vmod_curl *c) 286 | { 287 | CURL *curl_handle; 288 | CURLcode cr; 289 | struct curl_slist *req_headers = NULL; 290 | struct req_hdr *rh; 291 | 292 | curl_handle = curl_easy_init(); 293 | AN(curl_handle); 294 | 295 | VTAILQ_FOREACH(rh, &c->req_headers, list) 296 | req_headers = curl_slist_append(req_headers, rh->value); 297 | 298 | if (c->flags & F_METHOD_POST) { 299 | curl_easy_setopt(curl_handle, CURLOPT_POST, 1L); 300 | curl_easy_setopt(curl_handle, CURLOPT_POSTFIELDS, 301 | c->postfields); 302 | } else if (c->flags & F_METHOD_HEAD) 303 | curl_easy_setopt(curl_handle, CURLOPT_NOBODY, 1L); 304 | else if (c->flags & F_METHOD_GET) 305 | curl_easy_setopt(curl_handle, CURLOPT_HTTPGET, 1L); 306 | 307 | if (req_headers) 308 | curl_easy_setopt(curl_handle, CURLOPT_HTTPHEADER, 309 | req_headers); 310 | 311 | curl_easy_setopt(curl_handle, CURLOPT_URL, c->url); 312 | curl_easy_setopt(curl_handle, CURLOPT_NOSIGNAL, 1L); 313 | curl_easy_setopt(curl_handle, CURLOPT_NOPROGRESS, 1L); 314 | curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, recv_data); 315 | curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, c); 316 | curl_easy_setopt(curl_handle, CURLOPT_HEADERFUNCTION, recv_hdrs); 317 | curl_easy_setopt(curl_handle, CURLOPT_HEADERDATA, c); 318 | 319 | if (c->proxy) 320 | curl_easy_setopt(curl_handle, CURLOPT_PROXY, c->proxy); 321 | 322 | if (c->timeout > 0) { 323 | #ifdef HAVE_CURLOPT_TIMEOUT_MS 324 | curl_easy_setopt(curl_handle, CURLOPT_TIMEOUT_MS, 325 | c->timeout); 326 | #else 327 | curl_easy_setopt(curl_handle, CURLOPT_TIMEOUT, 328 | c->timeout / 1000); 329 | #endif 330 | } 331 | 332 | if (c->connect_timeout > 0) { 333 | #ifdef HAVE_CURLOPT_CONNECTTIMEOUT_MS 334 | curl_easy_setopt(curl_handle, CURLOPT_CONNECTTIMEOUT_MS, 335 | c->connect_timeout); 336 | #else 337 | curl_easy_setopt(curl_handle, CURLOPT_CONNECTTIMEOUT, 338 | c->connect_timeout / 1000); 339 | #endif 340 | } 341 | 342 | if (c->flags & F_SSL_VERIFY_PEER) 343 | curl_easy_setopt(curl_handle, CURLOPT_SSL_VERIFYPEER, 1L); 344 | else 345 | curl_easy_setopt(curl_handle, CURLOPT_SSL_VERIFYPEER, 0L); 346 | 347 | if (c->flags & F_SSL_VERIFY_HOST) 348 | curl_easy_setopt(curl_handle, CURLOPT_SSL_VERIFYHOST, 1L); 349 | else 350 | curl_easy_setopt(curl_handle, CURLOPT_SSL_VERIFYHOST, 0L); 351 | 352 | if (c->cafile) 353 | curl_easy_setopt(curl_handle, CURLOPT_CAINFO, c->cafile); 354 | 355 | if (c->capath) 356 | curl_easy_setopt(curl_handle, CURLOPT_CAPATH, c->capath); 357 | 358 | #ifdef HAVE_CURLOPT_UNIX_SOCKET_PATH 359 | if (c->sun_path) 360 | curl_easy_setopt(curl_handle, CURLOPT_UNIX_SOCKET_PATH, 361 | c->sun_path); 362 | #endif 363 | 364 | if (c->debug_flags != 0) { 365 | curl_easy_setopt(curl_handle, CURLOPT_DEBUGFUNCTION, 366 | cm_debug); 367 | curl_easy_setopt(curl_handle, CURLOPT_DEBUGDATA, c); 368 | curl_easy_setopt(curl_handle, CURLOPT_VERBOSE, 1L); 369 | } else 370 | curl_easy_setopt(curl_handle, CURLOPT_VERBOSE, 0L); 371 | 372 | curl_easy_setopt(curl_handle, CURLOPT_CUSTOMREQUEST, c->method); 373 | 374 | cr = curl_easy_perform(curl_handle); 375 | 376 | if (cr != 0) 377 | c->error = curl_easy_strerror(cr); 378 | 379 | curl_easy_getinfo(curl_handle, CURLINFO_RESPONSE_CODE, &c->status); 380 | 381 | if (req_headers) 382 | curl_slist_free_all(req_headers); 383 | 384 | c->method = NULL; 385 | 386 | cm_clear_req_headers(c); 387 | curl_easy_cleanup(curl_handle); 388 | VSB_finish(c->body); 389 | } 390 | 391 | VCL_VOID 392 | vmod_fetch(VRT_CTX, struct vmod_priv *priv, VCL_STRING url) 393 | { 394 | vmod_get(ctx, priv, url); 395 | } 396 | 397 | VCL_VOID 398 | vmod_get(VRT_CTX, struct vmod_priv *priv, VCL_STRING url) 399 | { 400 | struct vmod_curl *c; 401 | 402 | (void)ctx; 403 | 404 | c = cm_get(priv); 405 | cm_clear_fetch_state(c); 406 | c->url = url; 407 | c->flags |= F_METHOD_GET; 408 | cm_perform(c); 409 | } 410 | 411 | VCL_VOID 412 | vmod_head(VRT_CTX, struct vmod_priv *priv, VCL_STRING url) 413 | { 414 | struct vmod_curl *c; 415 | 416 | (void)ctx; 417 | 418 | c = cm_get(priv); 419 | cm_clear_fetch_state(c); 420 | c->url = url; 421 | c->flags |= F_METHOD_HEAD; 422 | cm_perform(c); 423 | } 424 | 425 | VCL_VOID 426 | vmod_post(VRT_CTX, struct vmod_priv *priv, VCL_STRING url, 427 | VCL_STRING postfields) 428 | { 429 | struct vmod_curl *c; 430 | 431 | (void)ctx; 432 | 433 | c = cm_get(priv); 434 | cm_clear_fetch_state(c); 435 | c->url = url; 436 | c->flags |= F_METHOD_POST; 437 | c->postfields = postfields; 438 | cm_perform(c); 439 | } 440 | 441 | VCL_INT 442 | vmod_status(VRT_CTX, struct vmod_priv *priv) 443 | { 444 | (void)ctx; 445 | return (cm_get(priv)->status); 446 | } 447 | 448 | VCL_VOID 449 | vmod_free(VRT_CTX, struct vmod_priv *priv) 450 | { 451 | (void)ctx; 452 | cm_clear(cm_get(priv)); 453 | } 454 | 455 | VCL_STRING 456 | vmod_error(VRT_CTX, struct vmod_priv *priv) 457 | { 458 | struct vmod_curl *c; 459 | 460 | (void)ctx; 461 | 462 | c = cm_get(priv); 463 | if (c->status != 0) 464 | return (NULL); 465 | return (c->error); 466 | } 467 | 468 | VCL_STRING 469 | vmod_header(VRT_CTX, struct vmod_priv *priv, VCL_STRING header) 470 | { 471 | struct hdr *h; 472 | const char *r = NULL; 473 | struct vmod_curl *c; 474 | 475 | (void)ctx; 476 | 477 | c = cm_get(priv); 478 | 479 | VTAILQ_FOREACH(h, &c->headers, list) { 480 | if (strcasecmp(h->key, header) == 0) { 481 | r = h->value; 482 | break; 483 | } 484 | } 485 | return (r); 486 | } 487 | 488 | VCL_STRING 489 | vmod_body(VRT_CTX, struct vmod_priv *priv) 490 | { 491 | (void)ctx; 492 | return (VSB_data(cm_get(priv)->body)); 493 | } 494 | 495 | VCL_VOID 496 | vmod_set_timeout(VRT_CTX, struct vmod_priv *priv, VCL_INT timeout) 497 | { 498 | (void)ctx; 499 | cm_get(priv)->timeout = timeout; 500 | } 501 | 502 | VCL_VOID 503 | vmod_set_connect_timeout(VRT_CTX, struct vmod_priv *priv, VCL_INT timeout) 504 | { 505 | (void)ctx; 506 | cm_get(priv)->connect_timeout = timeout; 507 | } 508 | 509 | VCL_VOID 510 | vmod_set_ssl_verify_peer(VRT_CTX, struct vmod_priv *priv, VCL_INT verify) 511 | { 512 | (void)ctx; 513 | 514 | if (verify) 515 | cm_get(priv)->flags |= F_SSL_VERIFY_PEER; 516 | else 517 | cm_get(priv)->flags &= ~F_SSL_VERIFY_PEER; 518 | } 519 | 520 | VCL_VOID 521 | vmod_set_ssl_verify_host(VRT_CTX, struct vmod_priv *priv, VCL_INT verify) 522 | { 523 | (void)ctx; 524 | 525 | if (verify) 526 | cm_get(priv)->flags |= F_SSL_VERIFY_HOST; 527 | else 528 | cm_get(priv)->flags &= ~F_SSL_VERIFY_HOST; 529 | } 530 | 531 | VCL_VOID 532 | vmod_set_ssl_cafile(VRT_CTX, struct vmod_priv *priv, VCL_STRING path) 533 | { 534 | (void)ctx; 535 | cm_get(priv)->cafile = path; 536 | } 537 | 538 | VCL_VOID 539 | vmod_set_ssl_capath(VRT_CTX, struct vmod_priv *priv, VCL_STRING path) 540 | { 541 | (void)ctx; 542 | cm_get(priv)->capath = path; 543 | } 544 | 545 | VCL_VOID 546 | vmod_set_unix_path(VRT_CTX, struct vmod_priv *priv, VCL_STRING path) 547 | { 548 | #ifdef HAVE_CURLOPT_UNIX_SOCKET_PATH 549 | (void)ctx; 550 | cm_get(priv)->sun_path = path; 551 | #else 552 | (void)path; 553 | (void)priv; 554 | VSLb(ctx->vsl, SLT_Error, 555 | "Unix domain socket support not available"); 556 | #endif 557 | } 558 | 559 | VCL_VOID 560 | vmod_header_add(VRT_CTX, struct vmod_priv *priv, VCL_STRING value) 561 | { 562 | struct vmod_curl *c; 563 | struct req_hdr *rh; 564 | 565 | (void)ctx; 566 | 567 | c = cm_get(priv); 568 | rh = calloc(1, sizeof(struct req_hdr)); 569 | AN(rh); 570 | rh->value = strdup(value); 571 | AN(rh->value); 572 | VTAILQ_INSERT_HEAD(&c->req_headers, rh, list); 573 | } 574 | 575 | VCL_VOID 576 | vmod_header_add_all(VRT_CTX, struct vmod_priv *priv) 577 | { 578 | struct http *hp; 579 | unsigned u; 580 | 581 | CHECK_OBJ_NOTNULL(ctx, VRT_CTX_MAGIC); 582 | 583 | if (VALID_OBJ(ctx->http_bereq, HTTP_MAGIC)) 584 | hp = ctx->http_bereq; 585 | else if (VALID_OBJ(ctx->http_req, HTTP_MAGIC)) 586 | hp = ctx->http_req; 587 | else 588 | return; 589 | 590 | for (u = HTTP_HDR_FIRST; u < hp->nhd; u++) { 591 | Tcheck(hp->hd[u]); 592 | #define SKIPHDR(n,t) \ 593 | if ((size_t)((t).e - (t).b) > strlen(n) && \ 594 | !strncasecmp((n), (t).b, strlen((n)))) \ 595 | continue; 596 | SKIPHDR("Content-Length:", hp->hd[u]); 597 | SKIPHDR("Transfer-Encoding:", hp->hd[u]); 598 | #undef SKIPHDR 599 | vmod_header_add(ctx, priv, hp->hd[u].b); 600 | } 601 | } 602 | 603 | VCL_VOID 604 | vmod_header_remove(VRT_CTX, struct vmod_priv *priv, VCL_STRING header) 605 | { 606 | struct vmod_curl *c; 607 | struct req_hdr *rh, *rh2; 608 | char *split, *s; 609 | 610 | (void)ctx; 611 | 612 | c = cm_get(priv); 613 | 614 | VTAILQ_FOREACH_SAFE(rh, &c->req_headers, list, rh2) { 615 | s = strdup(rh->value); 616 | AN(s); 617 | if ((split = strchr(s, ':')) != NULL) 618 | *split = '\x0'; 619 | if (strcasecmp(s, header) == 0) { 620 | VTAILQ_REMOVE(&c->req_headers, rh, list); 621 | free(rh->value); 622 | free(rh); 623 | } 624 | free(s); 625 | } 626 | } 627 | 628 | VCL_STRING 629 | vmod_escape(VRT_CTX, VCL_STRING str) 630 | { 631 | CURL *curl_handle; 632 | char *esc, *r; 633 | 634 | curl_handle = curl_easy_init(); 635 | AN(curl_handle); 636 | esc = curl_easy_escape(curl_handle, str, 0); 637 | AN(esc); 638 | r = WS_Copy(ctx->ws, esc, -1); 639 | curl_free(esc); 640 | curl_easy_cleanup(curl_handle); 641 | return (r); 642 | } 643 | 644 | VCL_STRING 645 | vmod_unescape(VRT_CTX, VCL_STRING str) 646 | { 647 | CURL *curl_handle; 648 | char *tmp, *r; 649 | 650 | curl_handle = curl_easy_init(); 651 | AN(curl_handle); 652 | tmp = curl_easy_unescape(curl_handle, str, 0, NULL); 653 | AN(tmp); 654 | r = WS_Copy(ctx->ws, tmp, -1); 655 | curl_free(tmp); 656 | curl_easy_cleanup(curl_handle); 657 | return (r); 658 | } 659 | 660 | VCL_VOID 661 | vmod_proxy(VRT_CTX, struct vmod_priv *priv, VCL_STRING proxy) 662 | { 663 | (void)ctx; 664 | vmod_set_proxy(ctx, priv, proxy); 665 | } 666 | 667 | VCL_VOID 668 | vmod_set_proxy(VRT_CTX, struct vmod_priv *priv, VCL_STRING proxy) 669 | { 670 | (void)ctx; 671 | cm_get(priv)->proxy = proxy; 672 | } 673 | 674 | VCL_VOID 675 | vmod_set_method(VRT_CTX, struct vmod_priv *priv, VCL_STRING name) 676 | { 677 | (void)ctx; 678 | cm_get(priv)->method = name; 679 | } 680 | 681 | VCL_VOID 682 | vmod_set_debug(VRT_CTX, struct vmod_priv *priv, VCL_ENUM what) 683 | { 684 | struct vmod_curl *c; 685 | 686 | c = cm_get(priv); 687 | c->vctx = ctx; 688 | #define DBG(u,l,v) if (!strcmp(what, #l)) c->debug_flags |= v; 689 | #include "debug_flags.h" 690 | #undef DBG 691 | } 692 | -------------------------------------------------------------------------------- /src/vmod_curl.vcc: -------------------------------------------------------------------------------- 1 | $Module curl 3 "cURL bindings for Varnish" 2 | $Event event_function 3 | $ABI vrt 4 | 5 | # GET the URL in the first parameter 6 | $Function VOID get(PRIV_TASK, STRING) 7 | 8 | # HEAD the URL in the first parameter 9 | $Function VOID head(PRIV_TASK, STRING) 10 | 11 | # Compatibility name for get 12 | $Function VOID fetch(PRIV_TASK, STRING) 13 | 14 | # POST the URL in the first parameter with the body fields given in 15 | # the second 16 | $Function VOID post(PRIV_TASK, STRING, STRING) 17 | 18 | # Return the header named in the first argument 19 | $Function STRING header(PRIV_TASK, STRING) 20 | 21 | # Free the memory used by headers. Not needed, will be handled 22 | # automatically if it's not called. 23 | $Function VOID free(PRIV_TASK) 24 | 25 | # The HTTP status code 26 | $Function INT status(PRIV_TASK) 27 | 28 | $Function STRING error(PRIV_TASK) 29 | 30 | # A response body can contain chars that are not allowed into headers, 31 | # e.g. CRLF. If the response body is a binary and/or it contains any 32 | # special chars, then this funtion MUST be used via synthetic: 33 | # synthetic(curl.body()). Otherwise it can be assigned to a header 34 | # resp.http.x-body = curl.body(); 35 | # Test 12 for a complete example. 36 | $Function STRING body(PRIV_TASK) 37 | 38 | # set_timeout and set_connect_timeout are not 39 | # global, but per request functions, therefore 40 | # they can't be used in vcl_init. 41 | $Function VOID set_timeout(PRIV_TASK, INT) 42 | $Function VOID set_connect_timeout(PRIV_TASK, INT) 43 | 44 | $Function VOID set_ssl_verify_peer(PRIV_TASK, INT) 45 | $Function VOID set_ssl_verify_host(PRIV_TASK, INT) 46 | $Function VOID set_ssl_cafile(PRIV_TASK, STRING) 47 | $Function VOID set_ssl_capath(PRIV_TASK, STRING) 48 | 49 | $Function STRING escape(STRING) 50 | $Function STRING unescape(STRING) 51 | 52 | # Add / Remove request headers 53 | $Function VOID header_add(PRIV_TASK, STRING) 54 | $Function VOID header_remove(PRIV_TASK, STRING) 55 | 56 | # Add all request headers from the req (or bereq) object 57 | $Function VOID header_add_all(PRIV_TASK) 58 | 59 | $Function VOID proxy(PRIV_TASK, STRING) 60 | 61 | $Function VOID set_proxy(PRIV_TASK, STRING) 62 | $Function VOID set_method(PRIV_TASK, STRING) 63 | 64 | $Function VOID set_unix_path(PRIV_TASK, STRING) 65 | 66 | $Function VOID set_debug(PRIV_TASK, ENUM { none, text, header_in, header_out, data_in, data_out }) 67 | --------------------------------------------------------------------------------