├── ccan ├── list │ ├── LICENSE │ ├── list.c │ └── _info ├── str │ ├── LICENSE │ ├── str.c │ ├── str_debug.h │ └── debug.c ├── check_type │ ├── LICENSE │ ├── _info │ └── check_type.h ├── build_assert │ ├── LICENSE │ ├── build_assert.h │ └── _info ├── container_of │ ├── LICENSE │ ├── _info │ └── container_of.h ├── Makefile.am ├── LICENSE.MIT └── LICENSE.CC0 ├── src ├── mkkvstore │ └── .gitignore ├── liburdma │ ├── urdma.driver │ └── verbs.h ├── .gitignore ├── kvstore_client │ ├── .gitignore │ ├── options.h │ └── options.c ├── kvstore_server │ ├── .gitignore │ ├── options.h │ ├── nvm.h │ ├── kvstore.h │ ├── options.c │ └── nvm.c ├── udp_pingpong │ ├── .gitignore │ └── README ├── verbs_pingpong │ └── .gitignore ├── urdmad │ ├── .gitignore │ ├── urdmad.service.in │ ├── kni.h │ └── interface.h ├── kmod │ ├── .gitignore │ ├── Kbuild │ ├── debug.h │ ├── obj.h │ ├── ae.c │ ├── Makefile.in │ ├── backports.h │ ├── cm.h │ └── verbs.h ├── util │ ├── binheap.h │ ├── config_file.h │ ├── list.h │ ├── util.h │ └── binheap.c └── tests │ └── test_parse.c ├── tests ├── .gitignore ├── list_test.c └── binheap.c ├── scripts ├── .gitignore ├── dpdk_nic_bind.service ├── run_real.sh ├── run_real_with_perf.sh ├── setup_dpdk_node.sh ├── make_memcached_workflow.py ├── plot.py ├── perf_regather.py ├── perf_run_ssh_helper.py ├── deploy_test.bash └── perf_run_tmux_helper.py ├── NEWS ├── configure.developer ├── .gitignore ├── config ├── .gitignore ├── profiles │ ├── kvstore_bobbio_este.sh │ ├── verbs_pingpong_bobbio_este.sh │ ├── verbs_pingpong_bobbio_este.json │ ├── udp_pingpong_bobbio_este.json │ └── udp_pingpong_bobbio_este.sh ├── version.sh └── rdma_core.m4 ├── TODO ├── include ├── kvstore_limits.h ├── proto_trp.h ├── urdma_kabi.h ├── proto_memcached.h ├── kvstore_storage.h └── urdmad_private.h └── doc ├── checksum-offload.txt ├── urdma-schema.json ├── send-recv.txt ├── protocol.txt ├── flow-director.txt └── implementation.txt /ccan/list/LICENSE: -------------------------------------------------------------------------------- 1 | ../LICENSE.MIT -------------------------------------------------------------------------------- /ccan/str/LICENSE: -------------------------------------------------------------------------------- 1 | ../LICENSE.CC0 -------------------------------------------------------------------------------- /ccan/check_type/LICENSE: -------------------------------------------------------------------------------- 1 | ../LICENSE.CC0 -------------------------------------------------------------------------------- /src/mkkvstore/.gitignore: -------------------------------------------------------------------------------- 1 | mkkvstore 2 | -------------------------------------------------------------------------------- /ccan/build_assert/LICENSE: -------------------------------------------------------------------------------- 1 | ../LICENSE.CC0 -------------------------------------------------------------------------------- /ccan/container_of/LICENSE: -------------------------------------------------------------------------------- 1 | ../LICENSE.CC0 -------------------------------------------------------------------------------- /src/liburdma/urdma.driver: -------------------------------------------------------------------------------- 1 | driver urdma 2 | -------------------------------------------------------------------------------- /tests/.gitignore: -------------------------------------------------------------------------------- 1 | binheap 2 | test_list 3 | -------------------------------------------------------------------------------- /scripts/.gitignore: -------------------------------------------------------------------------------- 1 | __pycache__ 2 | *.pyc 3 | -------------------------------------------------------------------------------- /src/.gitignore: -------------------------------------------------------------------------------- 1 | dpdk_build/ 2 | write_client 3 | -------------------------------------------------------------------------------- /src/kvstore_client/.gitignore: -------------------------------------------------------------------------------- 1 | kvstore_client 2 | -------------------------------------------------------------------------------- /src/kvstore_server/.gitignore: -------------------------------------------------------------------------------- 1 | kvstore_server 2 | -------------------------------------------------------------------------------- /src/udp_pingpong/.gitignore: -------------------------------------------------------------------------------- 1 | udp_pingpong 2 | -------------------------------------------------------------------------------- /src/verbs_pingpong/.gitignore: -------------------------------------------------------------------------------- 1 | verbs_pingpong 2 | -------------------------------------------------------------------------------- /src/urdmad/.gitignore: -------------------------------------------------------------------------------- 1 | urdmad.service 2 | urdmad 3 | -------------------------------------------------------------------------------- /NEWS: -------------------------------------------------------------------------------- 1 | USIW 0.0.2 2 | ---------- 3 | 4 | - Initial open-source version of USIW driver 5 | -------------------------------------------------------------------------------- /src/kmod/.gitignore: -------------------------------------------------------------------------------- 1 | .cache.mk 2 | *.cmd 3 | *.ko 4 | *.mod.h 5 | *.mod.c 6 | *.o.ur-safe 7 | proto_trp.h 8 | urdma_kabi.h 9 | Module.symvers 10 | modules.order 11 | .tmp_versions 12 | -------------------------------------------------------------------------------- /ccan/Makefile.am: -------------------------------------------------------------------------------- 1 | noinst_LIBRARIES = libccan.a 2 | libccan_a_SOURCES = \ 3 | ccan/container_of/container_of.h \ 4 | ccan/check_type/check_type.h \ 5 | ccan/build_assert/build_assert.h \ 6 | ccan/list/list.c \ 7 | ccan/list/list.h 8 | -------------------------------------------------------------------------------- /configure.developer: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | ./configure CFLAGS="-g -O2 -Wall -Wextra -Wno-sign-compare -Wno-unused-parameter -Wmissing-prototypes -Wmissing-declarations -Wwrite-strings -Wformat=2 -Wshadow -Wstrict-prototypes -Wold-style-definition -Wredundant-decls" "$@" 4 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | aclocal.m4 2 | autom4te.cache 3 | config.h* 4 | config.log 5 | config.status 6 | configure 7 | libtool 8 | Makefile 9 | /Makefile.in 10 | stamp-h1 11 | urdma*.tar* 12 | 13 | .deps 14 | .dirstamp 15 | .libs 16 | *.la 17 | *.lo 18 | *.log 19 | *.o 20 | *.o.d 21 | *.trs 22 | -------------------------------------------------------------------------------- /config/.gitignore: -------------------------------------------------------------------------------- 1 | # configury 2 | ar-lib 3 | config.guess 4 | config.sub 5 | compile 6 | depcomp 7 | install-sh 8 | ltmain.sh 9 | missing 10 | test-driver 11 | 12 | # libtool m4 stuff 13 | libtool.m4 14 | ltoptions.m4 15 | ltsugar.m4 16 | ltversion.m4 17 | lt~obsolete.m4 18 | 19 | # Experimental setups 20 | workloads 21 | -------------------------------------------------------------------------------- /ccan/str/str.c: -------------------------------------------------------------------------------- 1 | /* CC0 (Public domain) - see LICENSE file for details */ 2 | #include 3 | 4 | size_t strcount(const char *haystack, const char *needle) 5 | { 6 | size_t i = 0, nlen = strlen(needle); 7 | 8 | while ((haystack = strstr(haystack, needle)) != NULL) { 9 | i++; 10 | haystack += nlen; 11 | } 12 | return i; 13 | } 14 | -------------------------------------------------------------------------------- /src/urdmad/urdmad.service.in: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=Create and manage virtual urdma devices 3 | 4 | [Service] 5 | Type=simple 6 | ExecStart=@bindir@/urdmad --file-prefix=%H --proc-type=primary -l0-1 --log-level=8 --base-virtaddr=0x700000000000 -- --systemd 7 | LimitMEMLOCK=infinity 8 | RuntimeDirectory=urdma 9 | 10 | [Install] 11 | WantedBy=multi-user.target 12 | -------------------------------------------------------------------------------- /TODO: -------------------------------------------------------------------------------- 1 | build system 2 | - Explicitly link in DPDK libraries instead of using -ldpdk as a 3 | shortcut, to avoid interference from distributions which use 4 | --as-needed by default. 5 | 6 | liburdma 7 | - Choose rte_mempool sizes dynamically, and choose maximum number of queue 8 | pairs and tx/rx ring size according to the rte_mempool size 9 | - Add static inlines to wrap all container_of()'s 10 | - Update flow director rules to match source *and* destination addresses 11 | - fclose(conf_file) in parse_config() error cleanup 12 | - Sleep on KNI interfaces until we actually get a packet 13 | - Always tell kernel about qp_state transitions 14 | 15 | kmod 16 | - Dynamically register/unregister interfaces instead of using a static list 17 | - Clean up socket callbacks 18 | -------------------------------------------------------------------------------- /config/profiles/kvstore_bobbio_este.sh: -------------------------------------------------------------------------------- 1 | # Top source directory on local system---assumed to be git clone 2 | TOP_SRCDIR=${HOME}/src/usiw 3 | 4 | # Directory to deploy to 5 | DEPLOY_DIR=src/usiw 6 | 7 | # App executable 8 | SERVER_APP=kvstore_server 9 | CLIENT_APP=kvstore_client 10 | 11 | # Management IP addresses 12 | SERVER_NODE=bobbio 13 | CLIENT_NODE=este 14 | 15 | # DPDK IP address 16 | SERVER_DPDK_IP=10.0.0.2 17 | SERVER_DPDK_IP_PREFIX=24 18 | CLIENT_DPDK_IP=10.0.0.1 19 | CLIENT_DPDK_IP_PREFIX=24 20 | 21 | # Show only INFO or higher priority log messages 22 | DPDK_LOG_LEVEL=7 23 | SERVER_LCORE_LAYOUT="0,1" 24 | CLIENT_LCORE_LAYOUT="0,1" 25 | SERVER_PORT="01:00.0" 26 | CLIENT_PORT="01:00.0" 27 | 28 | # Other arguments --- passed to client, *not* to EAL 29 | SERVER_EXTRA_ARGS= 30 | CLIENT_EXTRA_ARGS="--input client_workload.txt" 31 | -------------------------------------------------------------------------------- /config/profiles/verbs_pingpong_bobbio_este.sh: -------------------------------------------------------------------------------- 1 | # Top source directory on local system---assumed to be git clone 2 | TOP_SRCDIR=${HOME}/src/usiw 3 | 4 | # Directory to deploy to 5 | DEPLOY_DIR=usiw 6 | 7 | # App executable 8 | SERVER_APP=verbs_pingpong 9 | CLIENT_APP=verbs_pingpong 10 | 11 | # Management IP addresses 12 | SERVER_NODE=bobbio 13 | CLIENT_NODE=este 14 | 15 | # DPDK IP address 16 | SERVER_DPDK_IP=10.0.0.2 17 | SERVER_DPDK_IP_PREFIX=24 18 | CLIENT_DPDK_IP=10.0.0.1 19 | CLIENT_DPDK_IP_PREFIX=24 20 | 21 | # Show only INFO or higher priority log messages 22 | DPDK_LOG_LEVEL=7 23 | SERVER_LCORE_LAYOUT="0,1,2" 24 | CLIENT_LCORE_LAYOUT="0,1,2" 25 | SERVER_PORT="01:00.0" 26 | CLIENT_PORT="01:00.0" 27 | 28 | # Other arguments --- passed to client, *not* to EAL 29 | SERVER_EXTRA_ARGS="-s 1024 -b 16 -c 5000000" 30 | CLIENT_EXTRA_ARGS="-s 1024 -b 16 -c 5000000" 31 | -------------------------------------------------------------------------------- /config/profiles/verbs_pingpong_bobbio_este.json: -------------------------------------------------------------------------------- 1 | { 2 | "server": { 3 | "app": "verbs_pingpong", 4 | "node": "bobbio", 5 | "dpdk_interfaces": [ 6 | { "port": "01:00.0", "ipv4_address": "10.0.0.2/24" } 7 | ], 8 | "lcore_layout": "0" 9 | }, 10 | "client": { 11 | "app": "verbs_pingpong", 12 | "node": "este", 13 | "dpdk_interfaces": [ 14 | { "port": "01:00.0", "ipv4_address": "10.0.0.1/24" } 15 | ], 16 | "lcore_layout": "0", 17 | "server_ip_address": "10.0.0.2" 18 | }, 19 | "common": { 20 | "top_srcdir": "~/src/usiw", 21 | "deploy_dir": "src/usiw", 22 | "app_args": ["-b", "16", "-c", "2500000"], 23 | "eal_log_level": 7 24 | }, 25 | "experiment": { 26 | "run_count": 3, 27 | "parameters": { 28 | "packet_size": [64, 128, 192, 256, 384, 512, 640, 768, 896, 1024, 1152, 1280, 1408, 1514] 29 | } 30 | } 31 | } 32 | -------------------------------------------------------------------------------- /ccan/str/str_debug.h: -------------------------------------------------------------------------------- 1 | /* CC0 (Public domain) - see LICENSE file for details */ 2 | #ifndef CCAN_STR_DEBUG_H 3 | #define CCAN_STR_DEBUG_H 4 | 5 | /* #define CCAN_STR_DEBUG 1 */ 6 | 7 | #ifdef CCAN_STR_DEBUG 8 | /* Because we mug the real ones with macros, we need our own wrappers. */ 9 | int str_isalnum(int i); 10 | int str_isalpha(int i); 11 | int str_isascii(int i); 12 | #if HAVE_ISBLANK 13 | int str_isblank(int i); 14 | #endif 15 | int str_iscntrl(int i); 16 | int str_isdigit(int i); 17 | int str_isgraph(int i); 18 | int str_islower(int i); 19 | int str_isprint(int i); 20 | int str_ispunct(int i); 21 | int str_isspace(int i); 22 | int str_isupper(int i); 23 | int str_isxdigit(int i); 24 | 25 | char *str_strstr(const char *haystack, const char *needle); 26 | char *str_strchr(const char *s, int c); 27 | char *str_strrchr(const char *s, int c); 28 | #endif /* CCAN_STR_DEBUG */ 29 | 30 | #endif /* CCAN_STR_DEBUG_H */ 31 | -------------------------------------------------------------------------------- /config/profiles/udp_pingpong_bobbio_este.json: -------------------------------------------------------------------------------- 1 | { 2 | "server": { 3 | "app": "udp_pingpong", 4 | "node": "bobbio", 5 | "dpdk_interfaces": [ 6 | { "port": "01:00.0", "ipv4_address": "10.0.0.2/24" } 7 | ], 8 | "lcore_layout": "0,1" 9 | }, 10 | "client": { 11 | "app": "udp_pingpong", 12 | "node": "este", 13 | "dpdk_interfaces": [ 14 | { "port": "01:00.0", "ipv4_address": "10.0.0.1/24" } 15 | ], 16 | "lcore_layout": "0,1", 17 | "server_ip_address": "10.0.0.2" 18 | }, 19 | "common": { 20 | "top_srcdir": "~/src/usiw", 21 | "deploy_dir": "src/usiw", 22 | "app_args": ["-b", "16", "-c", "2500000"], 23 | "eal_log_level": 7 24 | }, 25 | "experiment": { 26 | "run_count": 3, 27 | "parameters": { 28 | "packet_size": [64, 128, 192, 256, 384, 512, 640, 768, 896, 1024, 1152, 1280, 1408, 1514], 29 | "checksum_offload": [true, false] 30 | } 31 | } 32 | } 33 | -------------------------------------------------------------------------------- /ccan/check_type/_info: -------------------------------------------------------------------------------- 1 | #include "config.h" 2 | #include 3 | #include 4 | 5 | /** 6 | * check_type - routines for compile time type checking 7 | * 8 | * C has fairly weak typing: ints get automatically converted to longs, signed 9 | * to unsigned, etc. There are some cases where this is best avoided, and 10 | * these macros provide methods for evoking warnings (or build errors) when 11 | * a precise type isn't used. 12 | * 13 | * On compilers which don't support typeof() these routines are less effective, 14 | * since they have to use sizeof() which can only distiguish between types of 15 | * different size. 16 | * 17 | * License: CC0 (Public domain) 18 | * Author: Rusty Russell 19 | */ 20 | int main(int argc, char *argv[]) 21 | { 22 | if (argc != 2) 23 | return 1; 24 | 25 | if (strcmp(argv[1], "depends") == 0) { 26 | #if !HAVE_TYPEOF 27 | printf("ccan/build_assert\n"); 28 | #endif 29 | return 0; 30 | } 31 | 32 | return 1; 33 | } 34 | -------------------------------------------------------------------------------- /ccan/LICENSE.MIT: -------------------------------------------------------------------------------- 1 | Permission is hereby granted, free of charge, to any person obtaining a copy 2 | of this software and associated documentation files (the "Software"), to deal 3 | in the Software without restriction, including without limitation the rights 4 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 5 | copies of the Software, and to permit persons to whom the Software is 6 | furnished to do so, subject to the following conditions: 7 | 8 | The above copyright notice and this permission notice shall be included in 9 | all copies or substantial portions of the Software. 10 | 11 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 12 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 13 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 14 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 15 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 16 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN 17 | THE SOFTWARE. 18 | -------------------------------------------------------------------------------- /config/profiles/udp_pingpong_bobbio_este.sh: -------------------------------------------------------------------------------- 1 | # Top source directory on local system---assumed to be git clone 2 | TOP_SRCDIR=${HOME}/src/usiw 3 | 4 | # Directory to deploy to 5 | DEPLOY_DIR=src/usiw 6 | 7 | # App executable 8 | SERVER_APP=udp_pingpong 9 | CLIENT_APP=udp_pingpong 10 | 11 | # Management IP addresses 12 | SERVER_NODE=bobbio 13 | CLIENT_NODE=este 14 | 15 | # DPDK IP address 16 | SERVER_DPDK_IP=10.0.0.2 17 | SERVER_DPDK_IP_PREFIX=24 18 | CLIENT_DPDK_IP=10.0.0.1 19 | CLIENT_DPDK_IP_PREFIX=24 20 | 21 | # Show only INFO or higher priority log messages 22 | DPDK_LOG_LEVEL=7 23 | SERVER_LCORE_LAYOUT="0,1" 24 | CLIENT_LCORE_LAYOUT="0,1" 25 | SERVER_PORT="01:00.0" 26 | CLIENT_PORT="01:00.0" 27 | 28 | # Other arguments 29 | SERVER_EXTRA_ARGS="-w ${SERVER_PORT} -l ${SERVER_LCORE_LAYOUT} --log-level ${DPDK_LOG_LEVEL} -d /usr/lib64/librte_pmd_i40e.so -- --disable-checksum-offload -s 1024 -b 16 -c 25000000 ${SERVER_DPDK_IP}/${SERVER_DPDK_IP_PREFIX}" 30 | CLIENT_EXTRA_ARGS="-w ${CLIENT_PORT} -l ${CLIENT_LCORE_LAYOUT} --log-level ${DPDK_LOG_LEVEL} -d /usr/lib64/librte_pmd_i40e.so -- --disable-checksum-offload -s 1024 -b 16 -c 25000000 ${CLIENT_DPDK_IP}/${CLIENT_DPDK_IP_PREFIX}" 31 | -------------------------------------------------------------------------------- /ccan/list/list.c: -------------------------------------------------------------------------------- 1 | /* Licensed under BSD-MIT - see LICENSE file for details */ 2 | #include 3 | #include 4 | #include "list.h" 5 | 6 | static void *corrupt(const char *abortstr, 7 | const struct list_node *head, 8 | const struct list_node *node, 9 | unsigned int count) 10 | { 11 | if (abortstr) { 12 | fprintf(stderr, 13 | "%s: prev corrupt in node %p (%u) of %p\n", 14 | abortstr, node, count, head); 15 | abort(); 16 | } 17 | return NULL; 18 | } 19 | 20 | struct list_node *list_check_node(const struct list_node *node, 21 | const char *abortstr) 22 | { 23 | const struct list_node *p, *n; 24 | int count = 0; 25 | 26 | for (p = node, n = node->next; n != node; p = n, n = n->next) { 27 | count++; 28 | if (n->prev != p) 29 | return corrupt(abortstr, node, n, count); 30 | } 31 | /* Check prev on head node. */ 32 | if (node->prev != p) 33 | return corrupt(abortstr, node, node, 0); 34 | 35 | return (struct list_node *)node; 36 | } 37 | 38 | struct list_head *list_check(const struct list_head *h, const char *abortstr) 39 | { 40 | if (!list_check_node(&h->n, abortstr)) 41 | return NULL; 42 | return (struct list_head *)h; 43 | } 44 | -------------------------------------------------------------------------------- /ccan/build_assert/build_assert.h: -------------------------------------------------------------------------------- 1 | /* CC0 (Public domain) - see LICENSE file for details */ 2 | #ifndef CCAN_BUILD_ASSERT_H 3 | #define CCAN_BUILD_ASSERT_H 4 | 5 | /** 6 | * BUILD_ASSERT - assert a build-time dependency. 7 | * @cond: the compile-time condition which must be true. 8 | * 9 | * Your compile will fail if the condition isn't true, or can't be evaluated 10 | * by the compiler. This can only be used within a function. 11 | * 12 | * Example: 13 | * #include 14 | * ... 15 | * static char *foo_to_char(struct foo *foo) 16 | * { 17 | * // This code needs string to be at start of foo. 18 | * BUILD_ASSERT(offsetof(struct foo, string) == 0); 19 | * return (char *)foo; 20 | * } 21 | */ 22 | #define BUILD_ASSERT(cond) \ 23 | do { (void) sizeof(char [1 - 2*!(cond)]); } while(0) 24 | 25 | /** 26 | * BUILD_ASSERT_OR_ZERO - assert a build-time dependency, as an expression. 27 | * @cond: the compile-time condition which must be true. 28 | * 29 | * Your compile will fail if the condition isn't true, or can't be evaluated 30 | * by the compiler. This can be used in an expression: its value is "0". 31 | * 32 | * Example: 33 | * #define foo_to_char(foo) \ 34 | * ((char *)(foo) \ 35 | * + BUILD_ASSERT_OR_ZERO(offsetof(struct foo, string) == 0)) 36 | */ 37 | #define BUILD_ASSERT_OR_ZERO(cond) \ 38 | (sizeof(char [1 - 2*!(cond)]) - 1) 39 | 40 | #endif /* CCAN_BUILD_ASSERT_H */ 41 | -------------------------------------------------------------------------------- /ccan/build_assert/_info: -------------------------------------------------------------------------------- 1 | #include "config.h" 2 | #include 3 | #include 4 | 5 | /** 6 | * build_assert - routines for build-time assertions 7 | * 8 | * This code provides routines which will cause compilation to fail should some 9 | * assertion be untrue: such failures are preferable to run-time assertions, 10 | * but much more limited since they can only depends on compile-time constants. 11 | * 12 | * These assertions are most useful when two parts of the code must be kept in 13 | * sync: it is better to avoid such cases if possible, but seconds best is to 14 | * detect invalid changes at build time. 15 | * 16 | * For example, a tricky piece of code might rely on a certain element being at 17 | * the start of the structure. To ensure that future changes don't break it, 18 | * you would catch such changes in your code like so: 19 | * 20 | * Example: 21 | * #include 22 | * #include 23 | * 24 | * struct foo { 25 | * char string[5]; 26 | * int x; 27 | * }; 28 | * 29 | * static char *foo_string(struct foo *foo) 30 | * { 31 | * // This trick requires that the string be first in the structure 32 | * BUILD_ASSERT(offsetof(struct foo, string) == 0); 33 | * return (char *)foo; 34 | * } 35 | * 36 | * License: CC0 (Public domain) 37 | * Author: Rusty Russell 38 | */ 39 | int main(int argc, char *argv[]) 40 | { 41 | if (argc != 2) 42 | return 1; 43 | 44 | if (strcmp(argv[1], "depends") == 0) 45 | /* Nothing. */ 46 | return 0; 47 | 48 | return 1; 49 | } 50 | -------------------------------------------------------------------------------- /ccan/container_of/_info: -------------------------------------------------------------------------------- 1 | #include "config.h" 2 | #include 3 | #include 4 | 5 | /** 6 | * container_of - routine for upcasting 7 | * 8 | * It is often convenient to create code where the caller registers a pointer 9 | * to a generic structure and a callback. The callback might know that the 10 | * pointer points to within a larger structure, and container_of gives a 11 | * convenient and fairly type-safe way of returning to the enclosing structure. 12 | * 13 | * This idiom is an alternative to providing a void * pointer for every 14 | * callback. 15 | * 16 | * Example: 17 | * #include 18 | * #include 19 | * 20 | * struct timer { 21 | * void *members; 22 | * }; 23 | * 24 | * struct info { 25 | * int my_stuff; 26 | * struct timer timer; 27 | * }; 28 | * 29 | * static void my_timer_callback(struct timer *timer) 30 | * { 31 | * struct info *info = container_of(timer, struct info, timer); 32 | * printf("my_stuff is %u\n", info->my_stuff); 33 | * } 34 | * 35 | * static void register_timer(struct timer *timer) 36 | * { 37 | * (void)timer; 38 | * (void)my_timer_callback; 39 | * //... 40 | * } 41 | * 42 | * int main(void) 43 | * { 44 | * struct info info = { .my_stuff = 1 }; 45 | * 46 | * register_timer(&info.timer); 47 | * // ... 48 | * return 0; 49 | * } 50 | * 51 | * License: CC0 (Public domain) 52 | * Author: Rusty Russell 53 | */ 54 | int main(int argc, char *argv[]) 55 | { 56 | if (argc != 2) 57 | return 1; 58 | 59 | if (strcmp(argv[1], "depends") == 0) { 60 | printf("ccan/check_type\n"); 61 | return 0; 62 | } 63 | 64 | return 1; 65 | } 66 | -------------------------------------------------------------------------------- /ccan/list/_info: -------------------------------------------------------------------------------- 1 | #include "config.h" 2 | #include 3 | #include 4 | 5 | /** 6 | * list - double linked list routines 7 | * 8 | * The list header contains routines for manipulating double linked lists. 9 | * It defines two types: struct list_head used for anchoring lists, and 10 | * struct list_node which is usually embedded in the structure which is placed 11 | * in the list. 12 | * 13 | * Example: 14 | * #include 15 | * #include 16 | * #include 17 | * #include 18 | * 19 | * struct parent { 20 | * const char *name; 21 | * struct list_head children; 22 | * unsigned int num_children; 23 | * }; 24 | * 25 | * struct child { 26 | * const char *name; 27 | * struct list_node list; 28 | * }; 29 | * 30 | * int main(int argc, char *argv[]) 31 | * { 32 | * struct parent p; 33 | * struct child *c; 34 | * int i; 35 | * 36 | * if (argc < 2) 37 | * errx(1, "Usage: %s parent children...", argv[0]); 38 | * 39 | * p.name = argv[1]; 40 | * list_head_init(&p.children); 41 | * p.num_children = 0; 42 | * for (i = 2; i < argc; i++) { 43 | * c = malloc(sizeof(*c)); 44 | * c->name = argv[i]; 45 | * list_add(&p.children, &c->list); 46 | * p.num_children++; 47 | * } 48 | * 49 | * printf("%s has %u children:", p.name, p.num_children); 50 | * list_for_each(&p.children, c, list) 51 | * printf("%s ", c->name); 52 | * printf("\n"); 53 | * return 0; 54 | * } 55 | * 56 | * License: BSD-MIT 57 | * Author: Rusty Russell 58 | */ 59 | int main(int argc, char *argv[]) 60 | { 61 | if (argc != 2) 62 | return 1; 63 | 64 | if (strcmp(argv[1], "depends") == 0) { 65 | printf("ccan/str\n"); 66 | printf("ccan/container_of\n"); 67 | printf("ccan/check_type\n"); 68 | return 0; 69 | } 70 | 71 | return 1; 72 | } 73 | -------------------------------------------------------------------------------- /src/kvstore_server/options.h: -------------------------------------------------------------------------------- 1 | /* options.h */ 2 | 3 | /* 4 | * Userspace Software iWARP library for DPDK 5 | * 6 | * Authors: Patrick MacArthur 7 | * 8 | * Copyright (c) 2016, IBM Corporation 9 | * 10 | * This software is available to you under a choice of one of two 11 | * licenses. You may choose to be licensed under the terms of the GNU 12 | * General Public License (GPL) Version 2, available from the file 13 | * COPYING in the main directory of this source tree, or the 14 | * BSD license below: 15 | * 16 | * Redistribution and use in source and binary forms, with or 17 | * without modification, are permitted provided that the following 18 | * conditions are met: 19 | * 20 | * - Redistributions of source code must retain the above copyright notice, 21 | * this list of conditions and the following disclaimer. 22 | * 23 | * - Redistributions in binary form must reproduce the above copyright 24 | * notice, this list of conditions and the following disclaimer in the 25 | * documentation and/or other materials provided with the distribution. 26 | * 27 | * - Neither the name of IBM nor the names of its contributors may be 28 | * used to endorse or promote products derived from this software without 29 | * specific prior written permission. 30 | * 31 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | * SOFTWARE. 39 | */ 40 | 41 | #ifndef CLIENT_OPTIONS_H 42 | #define CLIENT_OPTIONS_H 43 | 44 | struct server_options { 45 | char *nvm_fn; 46 | }; 47 | 48 | const char * 49 | option_string(void); 50 | 51 | int 52 | parse_options(int argc, char **argv, struct server_options *options); 53 | 54 | #endif 55 | -------------------------------------------------------------------------------- /scripts/dpdk_nic_bind.service: -------------------------------------------------------------------------------- 1 | # 2 | # Userspace Software iWARP library for DPDK 3 | # 4 | # Authors: Patrick MacArthur 5 | # 6 | # Copyright (c) 2016, IBM Corporation 7 | # 8 | # This software is available to you under a choice of one of two 9 | # licenses. You may choose to be licensed under the terms of the GNU 10 | # General Public License (GPL) Version 2, available from the file 11 | # COPYING in the main directory of this source tree, or the 12 | # BSD license below: 13 | # 14 | # Redistribution and use in source and binary forms, with or 15 | # without modification, are permitted provided that the following 16 | # conditions are met: 17 | # 18 | # - Redistributions of source code must retain the above copyright notice, 19 | # this list of conditions and the following disclaimer. 20 | # 21 | # - Redistributions in binary form must reproduce the above copyright 22 | # notice, this list of conditions and the following disclaimer in the 23 | # documentation and/or other materials provided with the distribution. 24 | # 25 | # - Neither the name of IBM nor the names of its contributors may be 26 | # used to endorse or promote products derived from this software without 27 | # specific prior written permission. 28 | # 29 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 30 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 31 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 32 | # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 33 | # BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 34 | # ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 35 | # CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 36 | # SOFTWARE. 37 | 38 | [Unit] 39 | Description=Configure network port for use by DPDK 40 | Wants=systemd-modules-load.service 41 | 42 | [Service] 43 | Type=oneshot 44 | RemainAfterExit=yes 45 | ExecStart=/usr/sbin/dpdk_nic_bind -b vfio-pci 01:00.0 01:00.1 46 | ExecStop=/usr/sbin/dpdk_nic_bind -b i40e 01:00.0 01:00.1 47 | 48 | [Install] 49 | WantedBy=network.target 50 | -------------------------------------------------------------------------------- /src/kmod/Kbuild: -------------------------------------------------------------------------------- 1 | # Kbuild 2 | # 3 | # Software iWARP device driver for Linux 4 | # 5 | # Authors: Bernard Metzler 6 | # Patrick MacArthur 7 | # 8 | # Copyright (c) 2008-2016, IBM Corporation 9 | # 10 | # This software is available to you under a choice of one of two 11 | # licenses. You may choose to be licensed under the terms of the GNU 12 | # General Public License (GPL) Version 2, available from the file 13 | # COPYING in the main directory of this source tree, or the 14 | # BSD license below: 15 | # 16 | # Redistribution and use in source and binary forms, with or 17 | # without modification, are permitted provided that the following 18 | # conditions are met: 19 | # 20 | # - Redistributions of source code must retain the above copyright notice, 21 | # this list of conditions and the following disclaimer. 22 | # 23 | # - Redistributions in binary form must reproduce the above copyright 24 | # notice, this list of conditions and the following disclaimer in the 25 | # documentation and/or other materials provided with the distribution. 26 | # 27 | # - Neither the name of IBM nor the names of its contributors may be 28 | # used to endorse or promote products derived from this software without 29 | # specific prior written permission. 30 | # 31 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | # BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | # ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | # CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | # SOFTWARE. 39 | 40 | # Define to y to enable debug messages by default 41 | CONFIG_URDMA_DEBUG ?= n 42 | 43 | ccflags-y += -g -DPACKAGE_VERSION=$(PACKAGE_VERSION) 44 | ccflags-$(CONFIG_URDMA_DEBUG) += -DDEBUG 45 | 46 | obj-m += urdma.o 47 | 48 | urdma-objs := ae.o cm.o debug.o main.o mem.o obj.o qp.o verbs.o 49 | -------------------------------------------------------------------------------- /config/version.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # 3 | # Userspace Software iWARP library for DPDK 4 | # 5 | # Authors: Patrick MacArthur 6 | # 7 | # Copyright (c) 2016, IBM Corporation 8 | # Copyright (c) 2016, University of New Hampshire 9 | # 10 | # This software is available to you under a choice of one of two 11 | # licenses. You may choose to be licensed under the terms of the GNU 12 | # General Public License (GPL) Version 2, available from the file 13 | # COPYING in the main directory of this source tree, or the 14 | # BSD license below: 15 | # 16 | # Redistribution and use in source and binary forms, with or 17 | # without modification, are permitted provided that the following 18 | # conditions are met: 19 | # 20 | # - Redistributions of source code must retain the above copyright notice, 21 | # this list of conditions and the following disclaimer. 22 | # 23 | # - Redistributions in binary form must reproduce the above copyright 24 | # notice, this list of conditions and the following disclaimer in the 25 | # documentation and/or other materials provided with the distribution. 26 | # 27 | # - Neither the name of IBM nor the names of its contributors may be 28 | # used to endorse or promote products derived from this software without 29 | # specific prior written permission. 30 | # 31 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | # BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | # ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | # CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | # SOFTWARE. 39 | 40 | # This script is called by the build system to determine the current version. 41 | 42 | if [ $# -eq 1 ]; then 43 | cd $1 44 | fi 45 | 46 | if [ -f .version ]; then 47 | cat .version 48 | elif [ -d .git ]; then 49 | echo -n $(git describe --abbrev=12 | sed -e 's/^v//') 50 | else 51 | printf 'unknown\n' >&2 52 | exit 1 53 | fi 54 | -------------------------------------------------------------------------------- /src/kvstore_client/options.h: -------------------------------------------------------------------------------- 1 | /* options.h */ 2 | 3 | /* 4 | * Userspace Software iWARP library for DPDK 5 | * 6 | * Authors: Patrick MacArthur 7 | * 8 | * Copyright (c) 2016, IBM Corporation 9 | * 10 | * This software is available to you under a choice of one of two 11 | * licenses. You may choose to be licensed under the terms of the GNU 12 | * General Public License (GPL) Version 2, available from the file 13 | * COPYING in the main directory of this source tree, or the 14 | * BSD license below: 15 | * 16 | * Redistribution and use in source and binary forms, with or 17 | * without modification, are permitted provided that the following 18 | * conditions are met: 19 | * 20 | * - Redistributions of source code must retain the above copyright notice, 21 | * this list of conditions and the following disclaimer. 22 | * 23 | * - Redistributions in binary form must reproduce the above copyright 24 | * notice, this list of conditions and the following disclaimer in the 25 | * documentation and/or other materials provided with the distribution. 26 | * 27 | * - Neither the name of IBM nor the names of its contributors may be 28 | * used to endorse or promote products derived from this software without 29 | * specific prior written permission. 30 | * 31 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | * SOFTWARE. 39 | */ 40 | 41 | #ifndef CLIENT_OPTIONS_H 42 | #define CLIENT_OPTIONS_H 43 | 44 | struct client_options { 45 | char *command_fn; 46 | FILE *command_fp; 47 | char *output_fn; 48 | FILE *output_fp; 49 | }; 50 | 51 | const char * 52 | option_string(void); 53 | 54 | int 55 | parse_options(int argc, char **argv, struct client_options *options); 56 | 57 | #endif 58 | -------------------------------------------------------------------------------- /ccan/str/debug.c: -------------------------------------------------------------------------------- 1 | /* CC0 (Public domain) - see LICENSE file for details */ 2 | #include "config.h" 3 | #include 4 | #include 5 | #include 6 | #include 7 | 8 | #ifdef CCAN_STR_DEBUG 9 | /* Because we mug the real ones with macros, we need our own wrappers. */ 10 | int str_isalnum(int i) 11 | { 12 | assert(i >= -1 && i < 256); 13 | return isalnum(i); 14 | } 15 | 16 | int str_isalpha(int i) 17 | { 18 | assert(i >= -1 && i < 256); 19 | return isalpha(i); 20 | } 21 | 22 | int str_isascii(int i) 23 | { 24 | assert(i >= -1 && i < 256); 25 | return isascii(i); 26 | } 27 | 28 | #if HAVE_ISBLANK 29 | int str_isblank(int i) 30 | { 31 | assert(i >= -1 && i < 256); 32 | return isblank(i); 33 | } 34 | #endif 35 | 36 | int str_iscntrl(int i) 37 | { 38 | assert(i >= -1 && i < 256); 39 | return iscntrl(i); 40 | } 41 | 42 | int str_isdigit(int i) 43 | { 44 | assert(i >= -1 && i < 256); 45 | return isdigit(i); 46 | } 47 | 48 | int str_isgraph(int i) 49 | { 50 | assert(i >= -1 && i < 256); 51 | return isgraph(i); 52 | } 53 | 54 | int str_islower(int i) 55 | { 56 | assert(i >= -1 && i < 256); 57 | return islower(i); 58 | } 59 | 60 | int str_isprint(int i) 61 | { 62 | assert(i >= -1 && i < 256); 63 | return isprint(i); 64 | } 65 | 66 | int str_ispunct(int i) 67 | { 68 | assert(i >= -1 && i < 256); 69 | return ispunct(i); 70 | } 71 | 72 | int str_isspace(int i) 73 | { 74 | assert(i >= -1 && i < 256); 75 | return isspace(i); 76 | } 77 | 78 | int str_isupper(int i) 79 | { 80 | assert(i >= -1 && i < 256); 81 | return isupper(i); 82 | } 83 | 84 | int str_isxdigit(int i) 85 | { 86 | assert(i >= -1 && i < 256); 87 | return isxdigit(i); 88 | } 89 | 90 | #undef strstr 91 | #undef strchr 92 | #undef strrchr 93 | 94 | char *str_strstr(const char *haystack, const char *needle) 95 | { 96 | return strstr(haystack, needle); 97 | } 98 | 99 | char *str_strchr(const char *haystack, int c) 100 | { 101 | return strchr(haystack, c); 102 | } 103 | 104 | char *str_strrchr(const char *haystack, int c) 105 | { 106 | return strrchr(haystack, c); 107 | } 108 | #endif 109 | -------------------------------------------------------------------------------- /include/kvstore_limits.h: -------------------------------------------------------------------------------- 1 | /** @file kvstore_limits.h 2 | * 3 | * Limits for kvstore. To be included by applications that do not need to know 4 | * kvstore's persistent data structures (e.g., clients which will access the 5 | * kvstore over a network). */ 6 | 7 | /* 8 | * Userspace Software iWARP library for DPDK 9 | * 10 | * Authors: Patrick MacArthur 11 | * 12 | * Copyright (c) 2016, IBM Corporation 13 | * 14 | * This software is available to you under a choice of one of two 15 | * licenses. You may choose to be licensed under the terms of the GNU 16 | * General Public License (GPL) Version 2, available from the file 17 | * COPYING in the main directory of this source tree, or the 18 | * BSD license below: 19 | * 20 | * Redistribution and use in source and binary forms, with or 21 | * without modification, are permitted provided that the following 22 | * conditions are met: 23 | * 24 | * - Redistributions of source code must retain the above copyright notice, 25 | * this list of conditions and the following disclaimer. 26 | * 27 | * - Redistributions in binary form must reproduce the above copyright 28 | * notice, this list of conditions and the following disclaimer in the 29 | * documentation and/or other materials provided with the distribution. 30 | * 31 | * - Neither the name of IBM nor the names of its contributors may be 32 | * used to endorse or promote products derived from this software without 33 | * specific prior written permission. 34 | * 35 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 36 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 37 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 38 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 39 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 40 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 41 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 42 | * SOFTWARE. 43 | */ 44 | 45 | #ifndef KVSTORE_LIMITS_H 46 | #define KVSTORE_LIMITS_H 47 | 48 | enum { KVSTORE_KEY_LEN_MAX = 40 }; 49 | 50 | enum { KVSTORE_VALUE_LEN_MAX = 20480 }; 51 | 52 | #endif 53 | -------------------------------------------------------------------------------- /src/kvstore_server/nvm.h: -------------------------------------------------------------------------------- 1 | /* nvm.h */ 2 | 3 | /* 4 | * Userspace Software iWARP library for DPDK 5 | * 6 | * Authors: Patrick MacArthur 7 | * 8 | * Copyright (c) 2016, IBM Corporation 9 | * 10 | * This software is available to you under a choice of one of two 11 | * licenses. You may choose to be licensed under the terms of the GNU 12 | * General Public License (GPL) Version 2, available from the file 13 | * COPYING in the main directory of this source tree, or the 14 | * BSD license below: 15 | * 16 | * Redistribution and use in source and binary forms, with or 17 | * without modification, are permitted provided that the following 18 | * conditions are met: 19 | * 20 | * - Redistributions of source code must retain the above copyright notice, 21 | * this list of conditions and the following disclaimer. 22 | * 23 | * - Redistributions in binary form must reproduce the above copyright 24 | * notice, this list of conditions and the following disclaimer in the 25 | * documentation and/or other materials provided with the distribution. 26 | * 27 | * - Neither the name of IBM nor the names of its contributors may be 28 | * used to endorse or promote products derived from this software without 29 | * specific prior written permission. 30 | * 31 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | * SOFTWARE. 39 | */ 40 | 41 | #ifndef SERVER_NVM_H 42 | #define SERVER_NVM_H 43 | 44 | struct nvm_context { 45 | void *addr; 46 | size_t size; 47 | }; 48 | 49 | struct nvm_context * 50 | nvm_open(const char *partition_name); 51 | 52 | struct nvm_context * 53 | nvm_open_anonymous(size_t storage_size); 54 | 55 | void 56 | nvm_close(struct nvm_context *ctx); 57 | 58 | unsigned long 59 | nvm_get_pagesize(void); 60 | 61 | void 62 | nvm_flush(struct nvm_context *ctx, void *addr, size_t length); 63 | 64 | #endif 65 | -------------------------------------------------------------------------------- /src/urdmad/kni.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Userspace Software iWARP library for DPDK 3 | * 4 | * Authors: Patrick MacArthur 5 | * 6 | * Copyright (c) 2016, IBM Corporation 7 | * Copyright (c) 2016, University of New Hampshire 8 | * 9 | * This software is available to you under a choice of one of two 10 | * licenses. You may choose to be licensed under the terms of the GNU 11 | * General Public License (GPL) Version 2, available from the file 12 | * COPYING in the main directory of this source tree, or the 13 | * BSD license below: 14 | * 15 | * Redistribution and use in source and binary forms, with or 16 | * without modification, are permitted provided that the following 17 | * conditions are met: 18 | * 19 | * - Redistributions of source code must retain the above copyright notice, 20 | * this list of conditions and the following disclaimer. 21 | * 22 | * - Redistributions in binary form must reproduce the above copyright 23 | * notice, this list of conditions and the following disclaimer in the 24 | * documentation and/or other materials provided with the distribution. 25 | * 26 | * - Neither the name of IBM nor the names of its contributors may be 27 | * used to endorse or promote products derived from this software without 28 | * specific prior written permission. 29 | * 30 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 31 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 32 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 33 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 34 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 35 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 36 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 37 | * SOFTWARE. 38 | */ 39 | 40 | #ifndef IWARP_KNI_H 41 | #define IWARP_KNI_H 42 | 43 | #include 44 | 45 | struct usiw_driver; 46 | struct usiw_port_config; 47 | struct usiw_port; 48 | 49 | int 50 | usiw_set_ipv4_addr(struct usiw_driver *driver, struct usiw_port *port, 51 | struct usiw_port_config *config); 52 | 53 | int 54 | usiw_port_setup_kni(struct usiw_port *port); 55 | 56 | int 57 | usiw_driver_setup_netlink(struct usiw_driver *driver); 58 | 59 | #endif 60 | -------------------------------------------------------------------------------- /src/kmod/debug.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Software iWARP device driver for Linux 3 | * 4 | * Authors: Bernard Metzler 5 | * Patrick MacArthur 6 | * Fredy Neeser 7 | * 8 | * Copyright (c) 2008-2016, IBM Corporation 9 | * 10 | * This software is available to you under a choice of one of two 11 | * licenses. You may choose to be licensed under the terms of the GNU 12 | * General Public License (GPL) Version 2, available from the file 13 | * COPYING in the main directory of this source tree, or the 14 | * BSD license below: 15 | * 16 | * Redistribution and use in source and binary forms, with or 17 | * without modification, are permitted provided that the following 18 | * conditions are met: 19 | * 20 | * - Redistributions of source code must retain the above copyright notice, 21 | * this list of conditions and the following disclaimer. 22 | * 23 | * - Redistributions in binary form must reproduce the above copyright 24 | * notice, this list of conditions and the following disclaimer in the 25 | * documentation and/or other materials provided with the distribution. 26 | * 27 | * - Neither the name of IBM nor the names of its contributors may be 28 | * used to endorse or promote products derived from this software without 29 | * specific prior written permission. 30 | * 31 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | * SOFTWARE. 39 | */ 40 | 41 | #ifndef _SIW_DEBUG_H 42 | #define _SIW_DEBUG_H 43 | 44 | #define DBG_OBJ " siw_obj" 45 | #define DBG_MM " siw_mm" 46 | #define DBG_EH " siw_eh" 47 | #define DBG_CM " siw_cm" 48 | #define DBG_KT " siw_kt" 49 | #define DBG_DM " siw_dm" 50 | #define DBG_CQ " siw_cq" 51 | 52 | extern void siw_debug_init(void); 53 | extern void siw_debugfs_add_device(struct siw_dev *); 54 | extern void siw_debugfs_del_device(struct siw_dev *); 55 | extern void siw_debugfs_delete(void); 56 | 57 | #endif 58 | -------------------------------------------------------------------------------- /src/util/binheap.h: -------------------------------------------------------------------------------- 1 | /* binheap.h */ 2 | 3 | /* 4 | * Userspace Software iWARP library for DPDK 5 | * 6 | * Authors: Patrick MacArthur 7 | * 8 | * Copyright (c) 2017, University of New Hampshire 9 | * 10 | * This software is available to you under a choice of one of two 11 | * licenses. You may choose to be licensed under the terms of the GNU 12 | * General Public License (GPL) Version 2, available from the file 13 | * COPYING in the main directory of this source tree, or the 14 | * BSD license below: 15 | * 16 | * Redistribution and use in source and binary forms, with or 17 | * without modification, are permitted provided that the following 18 | * conditions are met: 19 | * 20 | * - Redistributions of source code must retain the above copyright notice, 21 | * this list of conditions and the following disclaimer. 22 | * 23 | * - Redistributions in binary form must reproduce the above copyright 24 | * notice, this list of conditions and the following disclaimer in the 25 | * documentation and/or other materials provided with the distribution. 26 | * 27 | * - Neither the name of IBM nor the names of its contributors may be 28 | * used to endorse or promote products derived from this software without 29 | * specific prior written permission. 30 | * 31 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | * SOFTWARE. 39 | */ 40 | 41 | #ifndef BINHEAP_H 42 | #define BINHEAP_H 43 | 44 | #include 45 | #include 46 | #include 47 | 48 | struct binheap { 49 | size_t size; 50 | size_t capacity; 51 | uint32_t arr[]; 52 | }; 53 | 54 | struct binheap *binheap_new(size_t capacity); 55 | int binheap_insert(struct binheap *binheap, uint32_t v); 56 | int binheap_peek(struct binheap *binheap, uint32_t *v); 57 | int binheap_pop(struct binheap *binheap); 58 | 59 | static inline bool 60 | binheap_empty(struct binheap *binheap) 61 | { 62 | return binheap->size == 0; 63 | } 64 | 65 | #endif 66 | -------------------------------------------------------------------------------- /src/tests/test_parse.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Userspace Software iWARP library for DPDK 3 | * 4 | * Authors: Patrick MacArthur 5 | * 6 | * Copyright (c) 2016, IBM Corporation 7 | * 8 | * This software is available to you under a choice of one of two 9 | * licenses. You may choose to be licensed under the terms of the GNU 10 | * General Public License (GPL) Version 2, available from the file 11 | * COPYING in the main directory of this source tree, or the 12 | * BSD license below: 13 | * 14 | * Redistribution and use in source and binary forms, with or 15 | * without modification, are permitted provided that the following 16 | * conditions are met: 17 | * 18 | * - Redistributions of source code must retain the above copyright notice, 19 | * this list of conditions and the following disclaimer. 20 | * 21 | * - Redistributions in binary form must reproduce the above copyright 22 | * notice, this list of conditions and the following disclaimer in the 23 | * documentation and/or other materials provided with the distribution. 24 | * 25 | * - Neither the name of IBM nor the names of its contributors may be 26 | * used to endorse or promote products derived from this software without 27 | * specific prior written permission. 28 | * 29 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 30 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 31 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 32 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 33 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 34 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 35 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 36 | * SOFTWARE. 37 | */ 38 | 39 | #include 40 | 41 | int main(void) 42 | { 43 | uint32_t address; 44 | int prefix_len; 45 | int ret; 46 | 47 | ret = parse_ipv4_address("10.0.0.1/24", &address, &prefix_len); 48 | assert(ret == 0); 49 | assert(rte_be_to_cpu_32(address) == 0x0a000001); 50 | assert(prefix_len == 24); 51 | 52 | ret = parse_ipv4_address("255.255.255.255/32", &address, &prefix_len); 53 | assert(ret == 0); 54 | assert(rte_be_to_cpu_32(address) == 0xffffffff); 55 | assert(prefix_len == 32); 56 | 57 | address = 0x12345678; 58 | prefix_len = 100; 59 | ret = parse_ipv4_address("127.0.0.1", &address, &prefix_len); 60 | assert(ret == -EINVAL); 61 | assert(address == 0x12345678); 62 | assert(prefix_len == 100); 63 | } 64 | -------------------------------------------------------------------------------- /doc/checksum-offload.txt: -------------------------------------------------------------------------------- 1 | DPDK support computing or offloading checksum computation for TCP, UDP, SCTP, 2 | and IP. 3 | 4 | To enable checksum offloading: 5 | 6 | (1) Ensure that the TX queue has offload support enabled. This requires 7 | explicitly passing the rte_eth_txconf structure to rte_eth_tx_queue_setup() 8 | (rather than setting it to NULL and allowing the driver to pick default 9 | parameters). What I do is call rte_eth_dev_info_get(), which returns the 10 | defaults as part of the info structure, and then modify that. 11 | 12 | To have UDP checksum offloading work, the ETH_TXQ_FLAGS_NOXSUMUDP flag MUST NOT 13 | be set in txconf->txq_flags. The Programmer's Guide does *not* document this 14 | and led me on a wild goose chase for several hours trying to figure out 15 | why this wasn't working. The issue is that the i40e driver will set this flag 16 | by default, to allow it to use a more efficient transmit path. 17 | 18 | I presume you could have checksum offload support enabled on some TX queues but 19 | not others on the same port, but I have not (yet) tested this. 20 | 21 | (2) As documented in the Programmer's Guide, you must set up some metadata in 22 | the rte_mbuf structure, *and* seed the checksum field with a specific value: 23 | 24 | (a) For IPv4 checksum offload: 25 | 26 | mbuf->l2_len = sizeof(struct ether_hdr); 27 | mbuf->l3_len = sizeof(struct ipv4_hdr); 28 | mbuf->ol_flags |= PKT_TX_IPV4|PKT_TX_IP_CKSUM; 29 | ip->hdr_checksum = 0; 30 | 31 | Here we just seed the checksum field with 0 (which is natural since I 32 | believe a 0 byte does not change the checksum). 33 | 34 | 35 | (b) For UDP checksum offload: 36 | mbuf->l2_len = sizeof(struct ether_hdr); 37 | mbuf->l3_len = sizeof(struct ipv4_hdr); 38 | mbuf->ol_flags |= PKT_TX_IPV4|PKT_TX_UDP_CKSUM; 39 | udp->dgram_cksum = rte_ipv4_phdr_cksum(ip, mbuf->ol_flags); 40 | 41 | Here we must pre-compute the IP psuedo-header checksum that is part of 42 | the UDP checksum (which is a minor requirement compared to checksumming 43 | the entire packet, but I imagine is a bit harder to do in hardware, since 44 | it's not just a linear scan). 45 | 46 | The other important thing to note is the l2_len and l3_len fields, which are 47 | used by the hardware to locate the region of the packet to compute the 48 | checksum for (this allows, e.g., hardware offloading of checksums in 49 | tunnelled packets by adjusting these values). Although they must be 2 byte 50 | and 4 byte aligned, respectively (a requirement I noticed by browsing the 51 | i40e driver source code while trying to figure out why the offload wasn't 52 | working). 53 | -------------------------------------------------------------------------------- /ccan/check_type/check_type.h: -------------------------------------------------------------------------------- 1 | /* CC0 (Public domain) - see LICENSE file for details */ 2 | #ifndef CCAN_CHECK_TYPE_H 3 | #define CCAN_CHECK_TYPE_H 4 | #ifdef HAVE_CONFIG_H 5 | #include "config.h" 6 | #endif 7 | 8 | /** 9 | * check_type - issue a warning or build failure if type is not correct. 10 | * @expr: the expression whose type we should check (not evaluated). 11 | * @type: the exact type we expect the expression to be. 12 | * 13 | * This macro is usually used within other macros to try to ensure that a macro 14 | * argument is of the expected type. No type promotion of the expression is 15 | * done: an unsigned int is not the same as an int! 16 | * 17 | * check_type() always evaluates to 0. 18 | * 19 | * If your compiler does not support typeof, then the best we can do is fail 20 | * to compile if the sizes of the types are unequal (a less complete check). 21 | * 22 | * Example: 23 | * // They should always pass a 64-bit value to _set_some_value! 24 | * #define set_some_value(expr) \ 25 | * _set_some_value((check_type((expr), uint64_t), (expr))) 26 | */ 27 | 28 | /** 29 | * check_types_match - issue a warning or build failure if types are not same. 30 | * @expr1: the first expression (not evaluated). 31 | * @expr2: the second expression (not evaluated). 32 | * 33 | * This macro is usually used within other macros to try to ensure that 34 | * arguments are of identical types. No type promotion of the expressions is 35 | * done: an unsigned int is not the same as an int! 36 | * 37 | * check_types_match() always evaluates to 0. 38 | * 39 | * If your compiler does not support typeof, then the best we can do is fail 40 | * to compile if the sizes of the types are unequal (a less complete check). 41 | * 42 | * Example: 43 | * // Do subtraction to get to enclosing type, but make sure that 44 | * // pointer is of correct type for that member. 45 | * #define container_of(mbr_ptr, encl_type, mbr) \ 46 | * (check_types_match((mbr_ptr), &((encl_type *)0)->mbr), \ 47 | * ((encl_type *) \ 48 | * ((char *)(mbr_ptr) - offsetof(encl_type, mbr)))) 49 | */ 50 | #if HAVE_TYPEOF 51 | #define check_type(expr, type) \ 52 | ((typeof(expr) *)0 != (type *)0) 53 | 54 | #define check_types_match(expr1, expr2) \ 55 | ((typeof(expr1) *)0 != (typeof(expr2) *)0) 56 | #else 57 | #include 58 | /* Without typeof, we can only test the sizes. */ 59 | #define check_type(expr, type) \ 60 | BUILD_ASSERT_OR_ZERO(sizeof(expr) == sizeof(type)) 61 | 62 | #define check_types_match(expr1, expr2) \ 63 | BUILD_ASSERT_OR_ZERO(sizeof(expr1) == sizeof(expr2)) 64 | #endif /* HAVE_TYPEOF */ 65 | 66 | #endif /* CCAN_CHECK_TYPE_H */ 67 | -------------------------------------------------------------------------------- /scripts/run_real.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # Userspace Software iWARP library for DPDK 4 | # 5 | # Authors: Patrick MacArthur 6 | # 7 | # Copyright (c) 2016, IBM Corporation 8 | # Copyright (c) 2016-2018, University of New Hampshire 9 | # 10 | # This software is available to you under a choice of one of two 11 | # licenses. You may choose to be licensed under the terms of the GNU 12 | # General Public License (GPL) Version 2, available from the file 13 | # COPYING in the main directory of this source tree, or the 14 | # BSD license below: 15 | # 16 | # Redistribution and use in source and binary forms, with or 17 | # without modification, are permitted provided that the following 18 | # conditions are met: 19 | # 20 | # - Redistributions of source code must retain the above copyright notice, 21 | # this list of conditions and the following disclaimer. 22 | # 23 | # - Redistributions in binary form must reproduce the above copyright 24 | # notice, this list of conditions and the following disclaimer in the 25 | # documentation and/or other materials provided with the distribution. 26 | # 27 | # - Neither the name of IBM nor the names of its contributors may be 28 | # used to endorse or promote products derived from this software without 29 | # specific prior written permission. 30 | # 31 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | # BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | # ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | # CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | # SOFTWARE. 39 | 40 | TOP_SRCDIR=$1 41 | TESTAPP=$2 42 | LOGID=$3 43 | shift 3 44 | 45 | LOGFILE=${HOME}/log/${TESTAPP}/${LOGID}.log 46 | 47 | readonly TOP_SRCDIR TESTIP LOGFILE 48 | export LOGFILE 49 | 50 | set -ex 51 | mkdir -p $(dirname ${LOGFILE}) 52 | exec > >(tee ${LOGFILE}) 2>&1 53 | 54 | cd ${TOP_SRCDIR} 55 | . /usr/share/dpdk/dpdk-sdk-env.sh 56 | make distclean || true 57 | ./configure --sysconfdir=/etc 58 | make 59 | sudo make install 60 | sudo ldconfig 61 | sudo depmod -A 62 | sudo chown root:dpdk /usr/local/bin/urdmad 63 | sudo chmod 0750 /usr/local/bin/urdmad 64 | sudo setcap cap_sys_admin,cap_net_admin+ep /usr/local/bin/urdmad 65 | systemctl --user daemon-reload 66 | 67 | systemctl --user stop urdmad || true 68 | sudo modprobe -r urdma || true 69 | sudo modprobe rte_kni 70 | sudo modprobe urdma 71 | 72 | systemctl --user start urdmad 73 | sleep 5 74 | ${TESTAPP} "$@" 75 | systemctl --user stop urdmad 76 | -------------------------------------------------------------------------------- /scripts/run_real_with_perf.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # Userspace Software iWARP library for DPDK 4 | # 5 | # Authors: Patrick MacArthur 6 | # 7 | # Copyright (c) 2016, IBM Corporation 8 | # Copyright (c) 2016-2018, University of New Hampshire 9 | # 10 | # This software is available to you under a choice of one of two 11 | # licenses. You may choose to be licensed under the terms of the GNU 12 | # General Public License (GPL) Version 2, available from the file 13 | # COPYING in the main directory of this source tree, or the 14 | # BSD license below: 15 | # 16 | # Redistribution and use in source and binary forms, with or 17 | # without modification, are permitted provided that the following 18 | # conditions are met: 19 | # 20 | # - Redistributions of source code must retain the above copyright notice, 21 | # this list of conditions and the following disclaimer. 22 | # 23 | # - Redistributions in binary form must reproduce the above copyright 24 | # notice, this list of conditions and the following disclaimer in the 25 | # documentation and/or other materials provided with the distribution. 26 | # 27 | # - Neither the name of IBM nor the names of its contributors may be 28 | # used to endorse or promote products derived from this software without 29 | # specific prior written permission. 30 | # 31 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | # BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | # ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | # CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | # SOFTWARE. 39 | 40 | TOP_SRCDIR=$1 41 | TESTAPP=$2 42 | LOGID=$3 43 | shift 3 44 | 45 | LOGFILE=${HOME}/log/${TESTAPP}/${LOGID}.log 46 | 47 | readonly TOP_SRCDIR TESTIP LOGFILE 48 | export LOGFILE 49 | 50 | set -ex 51 | mkdir -p $(dirname ${LOGFILE}) 52 | exec > >(tee ${LOGFILE}) 2>&1 53 | 54 | cd ${TOP_SRCDIR} 55 | . /usr/share/dpdk/dpdk-sdk-env.sh 56 | make distclean || true 57 | ./configure --sysconfdir=/etc 58 | make 59 | sudo make install 60 | sudo ldconfig 61 | sudo depmod -A 62 | sudo chown root:dpdk /usr/local/bin/urdmad 63 | sudo chmod 0750 /usr/local/bin/urdmad 64 | sudo setcap cap_sys_admin,cap_net_admin+ep /usr/local/bin/urdmad 65 | systemctl --user daemon-reload 66 | 67 | systemctl --user stop urdmad || true 68 | sudo modprobe -r urdma || true 69 | sudo modprobe rte_kni 70 | sudo modprobe urdma 71 | 72 | systemctl --user start urdmad 73 | sleep 5 74 | perf record --call-graph dwarf -o ${HOME}/perf.${LOGID}.data -- ${TESTAPP} "$@" 75 | systemctl --user stop urdmad 76 | -------------------------------------------------------------------------------- /doc/urdma-schema.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "http://json-schema.org/draft-04/schema#", 3 | "type": "object", 4 | "properties": { 5 | "ports": { 6 | "type": "array", 7 | "description": "NIC ports to enable", 8 | "minItems": 1, 9 | "items": { "$ref": "#/definitions/port" }, 10 | "uniqueItems": true 11 | }, 12 | "eal_args": { 13 | "type": "object", 14 | "description": "EAL arguments for subprocesses", 15 | "properties": { 16 | "log-level": { 17 | "type": "number", 18 | "description": "Maximum log level to display" 19 | } 20 | } 21 | }, 22 | "stats_timer_interval": { 23 | "type": "integer", 24 | "description": "Interval for urdmad to dump dropped packet statistics" 25 | }, 26 | "socket": { 27 | "type": "string", 28 | "description": "The location of the socket file for urdmad" 29 | } 30 | }, 31 | "additionalProperties": false, 32 | "required": [ "ports" ], 33 | "definitions": { 34 | "port": { 35 | "type": "object", 36 | "description": "Settings for the given Ethernet port", 37 | "properties": { 38 | "pci_address": { 39 | "type": "string", 40 | "description": "PCI address of port to match" 41 | }, 42 | "ipv4_address": { 43 | "type": "string", 44 | "description": "IPv4 address to assign to the port" 45 | }, 46 | "mtu": { 47 | "type": "number", 48 | "description": "MTU for the Ethernet port (1500 or 9000)" 49 | }, 50 | "max_qp": { 51 | "type": "number", 52 | "description": "Maximum number of queue pairs to allocate" 53 | }, 54 | "rx_desc_count": { 55 | "type": "number", 56 | "description": "Descriptors per receive queue" 57 | }, 58 | "tx_desc_count": { 59 | "type": "number", 60 | "description": "Descriptors per transmit queue" 61 | }, 62 | "rx_burst_size": { 63 | "type": "number", 64 | "description": "Number of packets to receive at once" 65 | }, 66 | "tx_burst_size": { 67 | "type": "number", 68 | "description": "Number of packets to send at once" 69 | } 70 | }, 71 | "additionalProperties": false 72 | } 73 | } 74 | } 75 | -------------------------------------------------------------------------------- /scripts/setup_dpdk_node.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # setup_dpdk_node.sh 3 | # 4 | # Userspace Software iWARP library for DPDK 5 | # 6 | # Authors: Patrick MacArthur 7 | # 8 | # Copyright (c) 2016, IBM Corporation 9 | # Copyright (c) 2016-2018, University of New Hampshire 10 | # 11 | # This software is available to you under a choice of one of two 12 | # licenses. You may choose to be licensed under the terms of the GNU 13 | # General Public License (GPL) Version 2, available from the file 14 | # COPYING in the main directory of this source tree, or the 15 | # BSD license below: 16 | # 17 | # Redistribution and use in source and binary forms, with or 18 | # without modification, are permitted provided that the following 19 | # conditions are met: 20 | # 21 | # - Redistributions of source code must retain the above copyright notice, 22 | # this list of conditions and the following disclaimer. 23 | # 24 | # - Redistributions in binary form must reproduce the above copyright 25 | # notice, this list of conditions and the following disclaimer in the 26 | # documentation and/or other materials provided with the distribution. 27 | # 28 | # - Neither the name of IBM nor the names of its contributors may be 29 | # used to endorse or promote products derived from this software without 30 | # specific prior written permission. 31 | # 32 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 33 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 34 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 35 | # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 36 | # BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 37 | # ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 38 | # CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 39 | # SOFTWARE. 40 | 41 | # Sets up DPDK on a node, along with everything needed to run as non-root 42 | # on Ubuntu 17.10. 43 | 44 | dpdk_group_users=(patrick) 45 | 46 | sudo yum install bc libpcap-devel dpdk-dev \ 47 | linux-image-generic linux-headers-generic \ 48 | libibverbs-dev librdmacm-dev 49 | 50 | if [[ -d /etc/modules-load.d ]]; then 51 | sudo tee /etc/modules-load.d/90-dpdk.conf < 5 | * 6 | * Copyright (c) 2016, IBM Corporation 7 | * Copyright (c) 2016, University of New Hampshire 8 | * 9 | * This software is available to you under a choice of one of two 10 | * licenses. You may choose to be licensed under the terms of the GNU 11 | * General Public License (GPL) Version 2, available from the file 12 | * COPYING in the main directory of this source tree, or the 13 | * BSD license below: 14 | * 15 | * Redistribution and use in source and binary forms, with or 16 | * without modification, are permitted provided that the following 17 | * conditions are met: 18 | * 19 | * - Redistributions of source code must retain the above copyright notice, 20 | * this list of conditions and the following disclaimer. 21 | * 22 | * - Redistributions in binary form must reproduce the above copyright 23 | * notice, this list of conditions and the following disclaimer in the 24 | * documentation and/or other materials provided with the distribution. 25 | * 26 | * - Neither the name of IBM nor the names of its contributors may be 27 | * used to endorse or promote products derived from this software without 28 | * specific prior written permission. 29 | * 30 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 31 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 32 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 33 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 34 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 35 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 36 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 37 | * SOFTWARE. 38 | */ 39 | 40 | #include 41 | #include 42 | 43 | enum { ipv4_addr_len_max = 20 }; 44 | 45 | enum urdma_port_id_type { 46 | urdma_port_id_index = 0, 47 | urdma_port_id_pci = 1, 48 | }; 49 | 50 | struct json_object; 51 | 52 | struct usiw_port_config { 53 | int id_type; 54 | struct rte_pci_addr pci_address; 55 | unsigned int mtu; 56 | unsigned int rx_desc_count; 57 | unsigned int tx_desc_count; 58 | unsigned int rx_burst_size; 59 | unsigned int tx_burst_size; 60 | int max_qp; 61 | char ipv4_address[ipv4_addr_len_max]; 62 | }; 63 | 64 | struct usiw_config { 65 | struct json_object *root; 66 | }; 67 | 68 | int 69 | urdma__config_file_get_ports(struct usiw_config *config, 70 | struct usiw_port_config **port_config); 71 | 72 | int 73 | urdma__config_file_get_eal_args(struct usiw_config *config, char **argv); 74 | 75 | char * 76 | urdma__config_file_get_sock_name(struct usiw_config *config); 77 | 78 | int 79 | urdma__config_file_get_timer_interval(struct usiw_config *config); 80 | 81 | int 82 | urdma__config_file_open(struct usiw_config *config); 83 | 84 | void 85 | urdma__config_file_close(struct usiw_config *config); 86 | -------------------------------------------------------------------------------- /doc/send-recv.txt: -------------------------------------------------------------------------------- 1 | DPDK Receive Path 2 | ----------------- 3 | 4 | The DPDK receive path attempts to optimize the message rate and support 5 | placement into userspace memory, but does not provide the user with direct 6 | control over the placement of received data. Rather, messages are received 7 | into packet memory buffer pools (struct mbuf_pool) which are managed by the 8 | DPDK libraries. This allows DPDK to work with NICs that do not have built-in 9 | MMU hardware to support RDMA-style direct data placement. 10 | 11 | When an interface port is initialized, the receive queues are each initialized 12 | with a memory buffer and a specified number of receive descriptors. Each 13 | active receive descriptor consumes a memory buffer from the memory pool. If a 14 | message arrives to the queue but all descriptors are consumed, the message is 15 | silently dropped (an interface counter will be incremented). 16 | 17 | Note that messages are generally received in one contiguous buffer, unless the 18 | port has been configured in advance to segment received packets (note that I 19 | have not tested this behavior). 20 | 21 | To retrieve a message, use the rte_eth_recv_burst() function: 22 | 23 | int rte_eth_rx_burst(uint16_t port, uint16_t queue, 24 | struct dpdk_mbuf **mbuf, const uint16_t count); 25 | 26 | This function returns the number of packets retrieved from the queue. It is 27 | intended to be used in burst mode to decrease the amount of I/O between the 28 | application and the NIC. 29 | 30 | Verbs Receive Path 31 | ------------------ 32 | 33 | Verbs has two types of data transfer operations: send/receive channel 34 | semantics, and RDMA READ/RDMA WRITE memory semantics. Verbs gives the user 35 | much more control over where data is placed than DPDK. However, with most 36 | verbs providers, the application is expected to register in advance all memory 37 | to be used in data transfers, which populates the corresponding physical 38 | addresses in the HCA's onboard MMU. 39 | 40 | A verbs queue pair is initialized with a send and receive queue size. No 41 | memory is allocated for receives automatically. Instead, the user must in 42 | advance post receive requests using the aptly named ibv_post_recv() verb, 43 | which defines the memory area in which the next message(s) should be received. 44 | The verbs interface also allows a scatter-gather list to be specified so that 45 | the received packet may be scattered to multiple memory areas; for example, 46 | to separate a header from the corresponding data. However, the HCA will 47 | completely fill each scatter-gather element in order until the message has been 48 | fully placed or the message is larger than the total space allocated for it. 49 | 50 | In InfiniBand, if a message arrives to an empty receive queue, or a message is 51 | too large, the HCA responds with a Receive Not Ready NAK, with no notice given 52 | to the receiving application. The sender can retransmit for a given number of 53 | times, and if the receive queue is still empty, the queue pair transitions to 54 | the error state and is then unusable. 55 | -------------------------------------------------------------------------------- /src/kvstore_server/kvstore.h: -------------------------------------------------------------------------------- 1 | /* kvstore.h */ 2 | 3 | /* 4 | * Userspace Software iWARP library for DPDK 5 | * 6 | * Authors: Patrick MacArthur 7 | * 8 | * Copyright (c) 2016, IBM Corporation 9 | * 10 | * This software is available to you under a choice of one of two 11 | * licenses. You may choose to be licensed under the terms of the GNU 12 | * General Public License (GPL) Version 2, available from the file 13 | * COPYING in the main directory of this source tree, or the 14 | * BSD license below: 15 | * 16 | * Redistribution and use in source and binary forms, with or 17 | * without modification, are permitted provided that the following 18 | * conditions are met: 19 | * 20 | * - Redistributions of source code must retain the above copyright notice, 21 | * this list of conditions and the following disclaimer. 22 | * 23 | * - Redistributions in binary form must reproduce the above copyright 24 | * notice, this list of conditions and the following disclaimer in the 25 | * documentation and/or other materials provided with the distribution. 26 | * 27 | * - Neither the name of IBM nor the names of its contributors may be 28 | * used to endorse or promote products derived from this software without 29 | * specific prior written permission. 30 | * 31 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | * SOFTWARE. 39 | */ 40 | 41 | #ifndef KVSTORE_H 42 | #define KVSTORE_H 43 | 44 | #include "kvstore_limits.h" 45 | #include "verbs.h" 46 | 47 | struct kvstore; 48 | struct kvstore_elem; 49 | 50 | struct kv_handle { 51 | void *value; 52 | struct ibv_mr *mr; 53 | size_t length; 54 | }; 55 | 56 | enum /*kv_open_flags*/ { 57 | kv_must_exist = 1, 58 | }; 59 | 60 | struct kvstore * 61 | kvstore_new(const char *partition_name, size_t cache_capacity, 62 | struct ibv_pd *pd); 63 | 64 | void 65 | kvstore_free(struct kvstore *store); 66 | 67 | struct kv_handle * 68 | kvstore_object_create(struct kvstore *store, const char *key, 69 | void *new_value, size_t value_len); 70 | 71 | struct kv_handle * 72 | kvstore_object_get(struct kvstore *store, const char *key); 73 | 74 | struct kv_handle * 75 | kvstore_object_set(struct kvstore *store, const char *key, 76 | void *new_value, size_t value_len); 77 | 78 | struct kv_handle * 79 | kvstore_object_replace(struct kvstore *store, const char *key, 80 | void *new_value, size_t value_len); 81 | 82 | int 83 | kvstore_object_flush(struct kvstore *store, const char *key); 84 | 85 | int 86 | kvstore_object_delete(struct kvstore *store, const char *key); 87 | 88 | uint64_t 89 | kvstore_cas_version(const struct kv_handle *h); 90 | 91 | #endif 92 | -------------------------------------------------------------------------------- /src/kvstore_server/options.c: -------------------------------------------------------------------------------- 1 | /* options.c */ 2 | 3 | /* 4 | * Userspace Software iWARP library for DPDK 5 | * 6 | * Authors: Patrick MacArthur 7 | * 8 | * Copyright (c) 2016, IBM Corporation 9 | * 10 | * This software is available to you under a choice of one of two 11 | * licenses. You may choose to be licensed under the terms of the GNU 12 | * General Public License (GPL) Version 2, available from the file 13 | * COPYING in the main directory of this source tree, or the 14 | * BSD license below: 15 | * 16 | * Redistribution and use in source and binary forms, with or 17 | * without modification, are permitted provided that the following 18 | * conditions are met: 19 | * 20 | * - Redistributions of source code must retain the above copyright notice, 21 | * this list of conditions and the following disclaimer. 22 | * 23 | * - Redistributions in binary form must reproduce the above copyright 24 | * notice, this list of conditions and the following disclaimer in the 25 | * documentation and/or other materials provided with the distribution. 26 | * 27 | * - Neither the name of IBM nor the names of its contributors may be 28 | * used to endorse or promote products derived from this software without 29 | * specific prior written permission. 30 | * 31 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | * SOFTWARE. 39 | */ 40 | 41 | #include 42 | #include 43 | #include 44 | #include 45 | #include 46 | 47 | #include 48 | #include 49 | 50 | #include "options.h" 51 | 52 | const char * 53 | option_string(void) 54 | { 55 | return "[-f|--nvm-file ] "; 56 | } 57 | 58 | int 59 | parse_options(int argc, char **argv, struct server_options *options) 60 | { 61 | static const struct option longopts[] = { 62 | { .name = "help", .has_arg = no_argument, 63 | .flag = NULL, .val = 'h' }, 64 | { .name = "nvm-file", .has_arg = required_argument, 65 | .flag = NULL, .val = 'f' }, 66 | { 0 }, 67 | }; 68 | 69 | int ch; 70 | 71 | memset(options, 0, sizeof(*options)); 72 | while ((ch = getopt_long(argc, argv, 73 | "h" /* --help */ 74 | "f:" /* --nvm-file */ 75 | , longopts, NULL)) != -1) { 76 | switch (ch) { 77 | case 'h': 78 | fprintf(stderr, "-h, --help: Print this help message\n"); 79 | fprintf(stderr, "-f, --nvm-file: Back storage with this file\n"); 80 | exit(EXIT_SUCCESS); 81 | break; 82 | case 'f': 83 | options->nvm_fn = optarg; 84 | break; 85 | default: 86 | rte_exit(EXIT_FAILURE, "Unexpected option -%c\n", ch); 87 | } 88 | } 89 | 90 | return optind - 1; 91 | } 92 | -------------------------------------------------------------------------------- /scripts/make_memcached_workflow.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # 3 | # Userspace Software iWARP library for DPDK 4 | # 5 | # Authors: Patrick MacArthur 6 | # 7 | # Copyright (c) 2016, IBM Corporation 8 | # 9 | # This software is available to you under a choice of one of two 10 | # licenses. You may choose to be licensed under the terms of the GNU 11 | # General Public License (GPL) Version 2, available from the file 12 | # COPYING in the main directory of this source tree, or the 13 | # BSD license below: 14 | # 15 | # Redistribution and use in source and binary forms, with or 16 | # without modification, are permitted provided that the following 17 | # conditions are met: 18 | # 19 | # - Redistributions of source code must retain the above copyright notice, 20 | # this list of conditions and the following disclaimer. 21 | # 22 | # - Redistributions in binary form must reproduce the above copyright 23 | # notice, this list of conditions and the following disclaimer in the 24 | # documentation and/or other materials provided with the distribution. 25 | # 26 | # - Neither the name of IBM nor the names of its contributors may be 27 | # used to endorse or promote products derived from this software without 28 | # specific prior written permission. 29 | # 30 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 31 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 32 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 33 | # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 34 | # BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 35 | # ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 36 | # CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 37 | # SOFTWARE. 38 | 39 | import random 40 | 41 | key_count = 10000 42 | op_count = 250000 43 | value_size_min = 128 44 | value_size_max = 20480 45 | 46 | char_range = range(ord('a'), ord('z')) 47 | 48 | def random_string(size_range, char_range): 49 | size = random.choice(size_range) 50 | key = "" 51 | for x in range(size): 52 | key += chr(random.choice(char_range)) 53 | return key 54 | 55 | def make_set_request(key): 56 | value = random_string(range(value_size_min, value_size_max), char_range) 57 | print("set", key, 0, 0, len(value), end='\r\n') 58 | print(value, end='\r\n') 59 | 60 | def make_get_request(key): 61 | print("get", key, end='\r\n') 62 | 63 | def main(): 64 | random.seed() 65 | key_list = [] 66 | for x in range(key_count): 67 | key = random_string(range(5, 20), char_range) 68 | key_list.append(key) 69 | 70 | # Seed phase 71 | for key in key_list: 72 | value = random_string(range(value_size_min, value_size_max), char_range) 73 | print("add", key, 0, 0, len(value), end='\r\n') 74 | print(value, end='\r\n') 75 | 76 | # Request phase 77 | for x in range(op_count): 78 | fn = random.choice([make_set_request, make_get_request]) 79 | key = random.choice(key_list) 80 | fn(key) 81 | 82 | if __name__ == "__main__": 83 | main() 84 | -------------------------------------------------------------------------------- /doc/protocol.txt: -------------------------------------------------------------------------------- 1 | This provides an iWARP implementation over UDP, with the following differences 2 | from RFC 5040, RFC 5041: 3 | 4 | - UDP preserves message boundaries, we intentionally fit one RDMAP segment 5 | per UDP datagram, and we do not produce packets larger than 1 MTU. As a 6 | result, we do not run over MPA. 7 | 8 | - We use our own thin shim protocol between UDP and DDP called TRP (trivial 9 | reliability protocol), with the following packet format: 10 | 11 | 0 1 2 3 12 | 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 13 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 14 | | Packet Sequence Number (PSN) | 15 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 16 | | Acknowledgement PSN | 17 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 18 | |I|A|F|R| Credits | 19 | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ 20 | 21 | Every packet is assigned a monotonically increasing packet sequence number 22 | by its sender, which is reflected when the packet is acknowledged. 23 | 24 | Credits is the number of packets beyond the acknowledgement PSN that the 25 | receiver is allowed to send. Put another way, the maximum PSN that the 26 | receiver may send is ACK PSN + Credits. 27 | 28 | There are four flag bits, documented in the source code: 29 | 30 | * I (Init) 31 | * A (Ack/Sack) 32 | * F (FIN) 33 | * R (reserved for future use) 34 | 35 | - Terminate messages can be divided into two broad categories: fatal and 36 | non-fatal. Non-fatal Terminate messages are those that correspond to a 37 | single request and are due to user error, e.g., making an RDMA READ or RDMA 38 | WRITE request using an invalid stag. Fatal Terminate messages are due to 39 | protocol errors; these will not be delivered to the user but will terminate 40 | the end-to-end context. 41 | 42 | - Fatal Terminate messages will behave as follows: the bottom 13 reserved bits 43 | will contain a timeout in milliseconds. The sender will ignore all packets 44 | from the recipient for that duration, and will not send any further messages 45 | to the sender for the duration. A Terminate message is sent only once. The 46 | only exception to silent ignoring is that any received Terminate messages 47 | will refresh the duration. 48 | 49 | Because Terminate messages can be lost, a sender may wish to delay sending 50 | packets even after the duration runs out. 51 | 52 | The intent is to flush out any messages from the erroneous operation, and 53 | then allow further operations. 54 | 55 | When irrecoverable packet loss occurs, a best-effort Terminate message will 56 | be sent. 57 | 58 | Any time a Terminate message is sent or received, a marker will be inserted 59 | into the send queue. Any operations targeted at the "terminated" endpoint 60 | prior to the marker will be rejected with an appropriate error code. This 61 | ensures that any operations that were causally dependent on the terminated 62 | operation are not started (e.g., SEND after WRITE to signal completion). 63 | -------------------------------------------------------------------------------- /src/util/list.h: -------------------------------------------------------------------------------- 1 | /* list.h */ 2 | 3 | /* 4 | * Userspace Software iWARP library for DPDK 5 | * 6 | * Authors: Patrick MacArthur 7 | * 8 | * Copyright (c) 2016, IBM Corporation 9 | * Copyright (c) 2016, University of New Hampshire 10 | * 11 | * This software is available to you under a choice of one of two 12 | * licenses. You may choose to be licensed under the terms of the GNU 13 | * General Public License (GPL) Version 2, available from the file 14 | * COPYING in the main directory of this source tree, or the 15 | * BSD license below: 16 | * 17 | * Redistribution and use in source and binary forms, with or 18 | * without modification, are permitted provided that the following 19 | * conditions are met: 20 | * 21 | * - Redistributions of source code must retain the above copyright notice, 22 | * this list of conditions and the following disclaimer. 23 | * 24 | * - Redistributions in binary form must reproduce the above copyright 25 | * notice, this list of conditions and the following disclaimer in the 26 | * documentation and/or other materials provided with the distribution. 27 | * 28 | * - Neither the name of IBM nor the names of its contributors may be 29 | * used to endorse or promote products derived from this software without 30 | * specific prior written permission. 31 | * 32 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 33 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 34 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 35 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 36 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 37 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 38 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 39 | * SOFTWARE. 40 | */ 41 | 42 | /* Macros to extend the BSD queue data structures. */ 43 | 44 | #ifndef LIST_H 45 | #define LIST_H 46 | 47 | #include 48 | #include 49 | 50 | /** Declares a new tailq head type named _tailq_head. */ 51 | #define DECLARE_TAILQ_HEAD(type) \ 52 | TAILQ_HEAD(type ##_tailq_head, type) 53 | 54 | /** Iterates through all elements of the list. lptr will contain each element 55 | * in sequence. prev is used internally to keep a pointer to the previous 56 | * element, so that the current element may be deleted inside the loop. 57 | * Deletion of any element except the current is undefined. */ 58 | #define LIST_FOR_EACH(lptr, head, name, prev) \ 59 | for ((prev) = &(head)->lh_first, (lptr) = *(prev); (lptr) != NULL; \ 60 | (prev) = ((lptr) == (*prev)) \ 61 | ? &(lptr)-> name .le_next : (prev), (lptr) = *(prev)) 62 | 63 | /** Iterates through all elements of the tailq. lptr will contain each element 64 | * in sequence. prev is used internally to keep a pointer to the previous 65 | * element, so that the current element may be deleted inside the loop. 66 | * Deletion of any element except the current is undefined. */ 67 | #define TAILQ_FOR_EACH(lptr, head, name, prev) \ 68 | for ((prev) = &(head)->tqh_first, (lptr) = *(prev); (lptr) != NULL; \ 69 | (prev) = ((lptr) == (*prev)) \ 70 | ? &(lptr)-> name .tqe_next : (prev), (lptr) = *(prev)) 71 | 72 | #endif 73 | -------------------------------------------------------------------------------- /tests/list_test.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Userspace Software iWARP library for DPDK 3 | * 4 | * Authors: Patrick MacArthur 5 | * 6 | * Copyright (c) 2016, IBM Corporation 7 | * 8 | * This software is available to you under a choice of one of two 9 | * licenses. You may choose to be licensed under the terms of the GNU 10 | * General Public License (GPL) Version 2, available from the file 11 | * COPYING in the main directory of this source tree, or the 12 | * BSD license below: 13 | * 14 | * Redistribution and use in source and binary forms, with or 15 | * without modification, are permitted provided that the following 16 | * conditions are met: 17 | * 18 | * - Redistributions of source code must retain the above copyright notice, 19 | * this list of conditions and the following disclaimer. 20 | * 21 | * - Redistributions in binary form must reproduce the above copyright 22 | * notice, this list of conditions and the following disclaimer in the 23 | * documentation and/or other materials provided with the distribution. 24 | * 25 | * - Neither the name of IBM nor the names of its contributors may be 26 | * used to endorse or promote products derived from this software without 27 | * specific prior written permission. 28 | * 29 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 30 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 31 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 32 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 33 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 34 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 35 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 36 | * SOFTWARE. 37 | */ 38 | 39 | #include 40 | #include 41 | 42 | #include 43 | 44 | struct list_head head = LIST_HEAD_INIT(head); 45 | 46 | struct entry { 47 | int value; 48 | struct list_node entries; 49 | }; 50 | 51 | void 52 | print_list(FILE *stream, struct list_head *head) 53 | { 54 | struct entry *e, *next; 55 | fprintf(stream, "< "); 56 | list_for_each_safe(head, e, next, entries) { 57 | fprintf(stream, "%d ", e->value); 58 | } 59 | fprintf(stream, ">\n"); 60 | } 61 | 62 | int 63 | main(void) 64 | { 65 | struct entry *e, *e2, *next; 66 | int x; 67 | 68 | printf("Empty list\n"); 69 | print_list(stdout, &head); 70 | 71 | printf("Inserting 10 values\n"); 72 | for (x = 0; x < 10; ++x) { 73 | e = malloc(sizeof(*e)); 74 | e->value = x; 75 | list_add_tail(&head, &e->entries); 76 | } 77 | print_list(stdout, &head); 78 | 79 | printf("Removing 7\n"); 80 | list_for_each_safe(&head, e, next, entries) { 81 | if (e->value == 7) { 82 | list_del(&e->entries); 83 | } 84 | } 85 | print_list(stdout, &head); 86 | 87 | printf("Adding 14\n"); 88 | list_for_each_safe(&head, e, next, entries) { 89 | if (e->value == 6) { 90 | e2 = malloc(sizeof(*e)); 91 | e2->value = 14; 92 | list_add_after(&head, &e->entries, &e2->entries); 93 | } 94 | } 95 | print_list(stdout, &head); 96 | 97 | printf("Removing all elements\n"); 98 | list_for_each_safe(&head, e, next, entries) { 99 | list_del(&e->entries); 100 | } 101 | print_list(stdout, &head); 102 | } 103 | -------------------------------------------------------------------------------- /src/udp_pingpong/README: -------------------------------------------------------------------------------- 1 | udp_pingpong 2 | ============ 3 | 4 | The udp_pingpong microbenchmark sends bursts of messages back and forth using 5 | UDP over IPv4. Neither IPv6 nor VLANs are supported (yet). 6 | 7 | At least *two* logical cores (lcores) are required. The master lcore is 8 | reserved for control operations. It will sleep as often as possible, waking up 9 | at least once every second to respond to ARP requests and notice that the 10 | worker lcores have finished. (NOTE: currently not used due to lack of 11 | available cores on my test systems) 12 | 13 | Command-line options: 14 | 15 | > --packet-count=COUNT / -c COUNT 16 | 17 | The total number of messages to send in each direction. 18 | 19 | > --packet-size=SIZE / -s SIZE 20 | 21 | The size of each message. The size must be at least large enough to hold the 22 | UDP, IPv4, and Ethernet headers (42 bytes), and must be no larger than a single 23 | MTU. 24 | 25 | > --burst-size=COUNT / -b COUNT 26 | 27 | The number of messages in each burst. The application will send and retrieve 28 | up to this many messages to/from the NIC at once. 29 | 30 | > --output=FILE / -o FILE 31 | 32 | Outputs the performance results, as a single object in JSON format, to the 33 | specified file. 34 | 35 | Additionally, for each configured Ethernet port a command line argument is 36 | expected giving the IP address and prefix to bind on that port, i.e., 37 | "10.0.0.1/24". 38 | 39 | Finally, to start in client mode, a final command line argument is needed to 40 | specify the IP address of a running server instance. The client will initiate 41 | an ARP to that IP address, and begin the pingpong once it receives an ARP 42 | reply. 43 | 44 | The JSON performance output looks like: 45 | 46 | { 47 | "requested_packet_count": 1000000, 48 | "packet_size": 512, 49 | "burst_size": 64, 50 | "slave_lcore_count": 2, 51 | "packet_count": 2000000, 52 | "physical_time": 0.556628099, 53 | "cpu_time": 1.112983712, 54 | "recv_poll_time": 0.900035843, 55 | "max_recv_poll_time": 0.000002229, 56 | "message_rate": 3593063.311945, 57 | "throughput": 14717.187326, 58 | "throughput_unit": "Mbps", 59 | "latency": 1.456789, 60 | "latency_unit": "microsecond", 61 | "recv_count_per_burst_histo": [0, 330715, 364485, 238409, 51517, 3139, 316, 55, 33, 21, 12, 8, 10, 5, 3, 3, 2, 0, 2, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0] 62 | } 63 | 64 | requested_packet_count, packet_size, burst_size are the input parameters above. 65 | 66 | slave_lcore_count is the number of lcores that participated in the pingpong. 67 | 68 | physical_time is the amount of real time during which data transfer was taking 69 | place (time the first thread started sending to the time that the last thread 70 | finished). 71 | 72 | cpu_time is the total amount of time that all threads spent in the data 73 | transfer phase. 74 | 75 | recv_poll_time is the amount of time that all threads spent polling for 76 | receives. cpu_time - recv_poll_time is the amount of time that all threads 77 | spent processing the transfers. 78 | 79 | max_recv_poll_time is the maximum number of times that it took to poll for the 80 | next burst of received messages, across all worker threads. 81 | 82 | message_rate is the number of messages sent per second. 83 | 84 | throughput and throughput_unit are the number of bytes sent per second. 85 | 86 | recv_count_per_burst_histo is an array describing the histogram of the number 87 | of messages received in each burst. 88 | -------------------------------------------------------------------------------- /src/kmod/obj.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Software iWARP device driver for Linux 3 | * 4 | * Authors: Bernard Metzler 5 | * Patrick MacArthur 6 | * 7 | * Copyright (c) 2008-2016, IBM Corporation 8 | * 9 | * This software is available to you under a choice of one of two 10 | * licenses. You may choose to be licensed under the terms of the GNU 11 | * General Public License (GPL) Version 2, available from the file 12 | * COPYING in the main directory of this source tree, or the 13 | * BSD license below: 14 | * 15 | * Redistribution and use in source and binary forms, with or 16 | * without modification, are permitted provided that the following 17 | * conditions are met: 18 | * 19 | * - Redistributions of source code must retain the above copyright notice, 20 | * this list of conditions and the following disclaimer. 21 | * 22 | * - Redistributions in binary form must reproduce the above copyright 23 | * notice, this list of conditions and the following disclaimer in the 24 | * documentation and/or other materials provided with the distribution. 25 | * 26 | * - Neither the name of IBM nor the names of its contributors may be 27 | * used to endorse or promote products derived from this software without 28 | * specific prior written permission. 29 | * 30 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 31 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 32 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 33 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 34 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 35 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 36 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 37 | * SOFTWARE. 38 | */ 39 | 40 | #ifndef _SIW_OBJ_H 41 | #define _SIW_OBJ_H 42 | 43 | #include 44 | #include 45 | #include 46 | #include 47 | #include 48 | 49 | #include 50 | 51 | #include "debug.h" 52 | 53 | 54 | static inline struct siw_dev *siw_dev_ofa2siw(struct ib_device *ofa_dev) 55 | { 56 | return container_of(ofa_dev, struct siw_dev, ofa_dev); 57 | } 58 | 59 | static inline void siw_cq_get(struct siw_cq *cq) 60 | { 61 | kref_get(&cq->hdr.ref); 62 | pr_debug(DBG_OBJ "(CQ%d): New refcount: %d\n", 63 | OBJ_ID(cq), kref_read(&cq->hdr.ref)); 64 | } 65 | static inline void siw_qp_get(struct siw_qp *qp) 66 | { 67 | kref_get(&qp->hdr.ref); 68 | pr_debug(DBG_OBJ "(QP%d): New refcount: %d\n", 69 | OBJ_ID(qp), kref_read(&qp->hdr.ref)); 70 | } 71 | static inline void siw_pd_get(struct siw_pd *pd) 72 | { 73 | kref_get(&pd->hdr.ref); 74 | pr_debug(DBG_OBJ "(PD%d): New refcount: %d\n", 75 | OBJ_ID(pd), kref_read(&pd->hdr.ref)); 76 | } 77 | 78 | extern void siw_remove_obj(spinlock_t *lock, struct idr *idr, 79 | struct siw_objhdr *hdr); 80 | 81 | extern void siw_objhdr_init(struct siw_objhdr *); 82 | extern void siw_idr_init(struct siw_dev *); 83 | extern void siw_idr_release(struct siw_dev *); 84 | 85 | extern struct siw_cq *siw_cq_id2obj(struct siw_dev *, int); 86 | extern struct siw_qp *siw_qp_id2obj(struct siw_dev *, int); 87 | 88 | extern int siw_qp_add(struct siw_dev *, struct siw_qp *); 89 | extern int siw_cq_add(struct siw_dev *, struct siw_cq *); 90 | extern int siw_pd_add(struct siw_dev *, struct siw_pd *); 91 | 92 | extern void siw_cq_put(struct siw_cq *); 93 | extern void siw_qp_put(struct siw_qp *); 94 | extern void siw_pd_put(struct siw_pd *); 95 | 96 | #endif 97 | -------------------------------------------------------------------------------- /src/kmod/ae.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Software iWARP device driver for Linux 3 | * 4 | * Authors: Bernard Metzler 5 | * Patrick MacArthur 6 | * 7 | * Copyright (c) 2008-2016, IBM Corporation 8 | * Copyright (c) 2016, University of New Hampshire 9 | * 10 | * This software is available to you under a choice of one of two 11 | * licenses. You may choose to be licensed under the terms of the GNU 12 | * General Public License (GPL) Version 2, available from the file 13 | * COPYING in the main directory of this source tree, or the 14 | * BSD license below: 15 | * 16 | * Redistribution and use in source and binary forms, with or 17 | * without modification, are permitted provided that the following 18 | * conditions are met: 19 | * 20 | * - Redistributions of source code must retain the above copyright notice, 21 | * this list of conditions and the following disclaimer. 22 | * 23 | * - Redistributions in binary form must reproduce the above copyright 24 | * notice, this list of conditions and the following disclaimer in the 25 | * documentation and/or other materials provided with the distribution. 26 | * 27 | * - Neither the name of IBM nor the names of its contributors may be 28 | * used to endorse or promote products derived from this software without 29 | * specific prior written permission. 30 | * 31 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | * SOFTWARE. 39 | */ 40 | 41 | #include 42 | #include 43 | #include 44 | #include 45 | #include 46 | #include 47 | #include 48 | #include 49 | 50 | #include 51 | #include 52 | #include 53 | #include 54 | 55 | #include "urdma.h" 56 | #include "obj.h" 57 | #include "cm.h" 58 | 59 | 60 | void siw_qp_event(struct siw_qp *qp, enum ib_event_type etype) 61 | { 62 | struct ib_event event; 63 | struct ib_qp *ofa_qp = &qp->ofa_qp; 64 | 65 | event.event = etype; 66 | event.device = ofa_qp->device; 67 | event.element.qp = ofa_qp; 68 | 69 | if (!(qp->attrs.flags & SIW_QP_IN_DESTROY) && ofa_qp->event_handler) { 70 | pr_debug(DBG_EH ": reporting %d\n", etype); 71 | (*ofa_qp->event_handler)(&event, ofa_qp->qp_context); 72 | } 73 | } 74 | 75 | void siw_cq_event(struct siw_cq *cq, enum ib_event_type etype) 76 | { 77 | struct ib_event event; 78 | struct ib_cq *ofa_cq = &cq->ofa_cq; 79 | 80 | event.event = etype; 81 | event.device = ofa_cq->device; 82 | event.element.cq = ofa_cq; 83 | 84 | if (ofa_cq->event_handler) { 85 | pr_debug(DBG_EH ": reporting %d\n", etype); 86 | (*ofa_cq->event_handler)(&event, ofa_cq->cq_context); 87 | } 88 | } 89 | 90 | void siw_port_event(struct siw_dev *sdev, u8 port, enum ib_event_type etype) 91 | { 92 | struct ib_event event; 93 | 94 | event.event = etype; 95 | event.device = &sdev->ofa_dev; 96 | event.element.port_num = port; 97 | 98 | pr_debug(DBG_EH ": reporting %d\n", etype); 99 | ib_dispatch_event(&event); 100 | } 101 | -------------------------------------------------------------------------------- /scripts/plot.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # 3 | # Userspace Software iWARP library for DPDK 4 | # 5 | # Authors: Patrick MacArthur 6 | # 7 | # Copyright (c) 2016, IBM Corporation 8 | # 9 | # This software is available to you under a choice of one of two 10 | # licenses. You may choose to be licensed under the terms of the GNU 11 | # General Public License (GPL) Version 2, available from the file 12 | # COPYING in the main directory of this source tree, or the 13 | # BSD license below: 14 | # 15 | # Redistribution and use in source and binary forms, with or 16 | # without modification, are permitted provided that the following 17 | # conditions are met: 18 | # 19 | # - Redistributions of source code must retain the above copyright notice, 20 | # this list of conditions and the following disclaimer. 21 | # 22 | # - Redistributions in binary form must reproduce the above copyright 23 | # notice, this list of conditions and the following disclaimer in the 24 | # documentation and/or other materials provided with the distribution. 25 | # 26 | # - Neither the name of IBM nor the names of its contributors may be 27 | # used to endorse or promote products derived from this software without 28 | # specific prior written permission. 29 | # 30 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 31 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 32 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 33 | # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 34 | # BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 35 | # ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 36 | # CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 37 | # SOFTWARE. 38 | 39 | import numpy as np 40 | import pandas as pd 41 | from pandas.io.json import json_normalize 42 | import matplotlib.pyplot as plt 43 | import json 44 | import sys 45 | 46 | def make_plot(data, x_axis, y_axis, category): 47 | '''Makes a plot for the given data set (as a list of hash objects) using 48 | matplotlib.''' 49 | grouped = data.groupby([x_axis, category]) 50 | y_values = grouped[y_axis].mean().unstack() 51 | errors = grouped[y_axis].std().unstack() 52 | fig, ax = plt.subplots() 53 | y_values.plot(yerr=errors, ax=ax, kind='line', 54 | xlim=[0, data[x_axis].max() * 1.05], 55 | ylim=[0, data[y_axis].max() * 1.05]) 56 | ax.yaxis.set_label_text(y_axis) 57 | plt.savefig("{}-vs-{}.pdf".format(y_axis, x_axis)) 58 | 59 | def make_hist(data, x_axis, x_value, y_axis): 60 | grouped = data.groupby((x_axis, 'run_id')) 61 | group = grouped.get_group((x_value, 0)) 62 | plt.figure("{} = {}".format(x_axis, x_value)) 63 | group[y_axis].apply(lambda v: plt.hist(range(0, len(v)), len(v), weights=v, 64 | log=True)) 65 | plt.xlabel(y_axis) 66 | plt.ylabel('Count') 67 | plt.title("{} = {}; burst_size = 96".format(x_axis, x_value)) 68 | plt.savefig("hist-{}-{}-{}.pdf".format(y_axis, x_axis, x_value)) 69 | 70 | def make_data_frame(filename): 71 | with open(filename, 'r') as fp: 72 | jdata = json.load(fp) 73 | return json_normalize(jdata) 74 | 75 | def main(): 76 | json_in_file = sys.argv[1] 77 | x_axis = sys.argv[2] 78 | print("input file is", json_in_file) 79 | print("x axis label is", x_axis) 80 | data = make_data_frame(json_in_file) 81 | 82 | make_plot(data, x_axis, 'message_rate', 'app') 83 | make_plot(data, x_axis, 'latency', 'app') 84 | make_plot(data, x_axis, 'throughput', 'app') 85 | 86 | if __name__ == "__main__": 87 | main() 88 | -------------------------------------------------------------------------------- /include/proto_trp.h: -------------------------------------------------------------------------------- 1 | /* proto.h */ 2 | 3 | /* 4 | * Userspace Software iWARP library for DPDK 5 | * 6 | * Authors: Patrick MacArthur 7 | * 8 | * Copyright (c) 2016, IBM Corporation 9 | * 10 | * This software is available to you under a choice of one of two 11 | * licenses. You may choose to be licensed under the terms of the GNU 12 | * General Public License (GPL) Version 2, available from the file 13 | * COPYING in the main directory of this source tree, or the 14 | * BSD license below: 15 | * 16 | * Redistribution and use in source and binary forms, with or 17 | * without modification, are permitted provided that the following 18 | * conditions are met: 19 | * 20 | * - Redistributions of source code must retain the above copyright notice, 21 | * this list of conditions and the following disclaimer. 22 | * 23 | * - Redistributions in binary form must reproduce the above copyright 24 | * notice, this list of conditions and the following disclaimer in the 25 | * documentation and/or other materials provided with the distribution. 26 | * 27 | * - Neither the name of IBM nor the names of its contributors may be 28 | * used to endorse or promote products derived from this software without 29 | * specific prior written permission. 30 | * 31 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | * SOFTWARE. 39 | */ 40 | 41 | #ifndef PROTO_TRP_H 42 | #define PROTO_TRP_H 43 | 44 | #ifdef __KERNEL__ 45 | #include 46 | #else 47 | #include 48 | #endif 49 | 50 | enum { 51 | trp_req = 0x1000, 52 | /**< Initial request from the client. Any data in the packet 53 | * is considered to be part of the RDMA CM private data 54 | * exchange. */ 55 | trp_accept = 0x2000, 56 | /**< Accepting a connection request from the client. Any data 57 | * in the packet is passed as private data to the client 58 | * application. */ 59 | trp_reject = 0x3000, 60 | /**< Rejecting a connection request from the client. Any data 61 | * in the packet is passed as private data to the client 62 | * application. */ 63 | trp_fin = 0x4000, 64 | /**< Indicates that the sender wishes to close the connection. 65 | * The connection is destroyed as soon as this message is sent; 66 | * no response from the receiver is necessary nor expected. */ 67 | trp_sack = 0x5000, 68 | /**< This packet is a selective acknowledgement that contains 69 | * no data. Rather, the psn and ack_psn fields indicate the 70 | * minimum and (maximum + 1) sequence numbers, respectively, in 71 | * a contiguous range that have been received. */ 72 | trp_opcode_mask = 0xf000, 73 | /**< Mask of all bits used for opcode. */ 74 | trp_reserved_mask = 0x0fff, 75 | /**< Mask of all bits not currently used. */ 76 | trp_opcode_shift = 12, 77 | /**< Number of bits that opcode is shifted by. */ 78 | }; 79 | 80 | struct trp_hdr { 81 | uint32_t psn; 82 | uint32_t ack_psn; 83 | uint16_t opcode; 84 | } __attribute__((__packed__)); 85 | 86 | struct trp_rr_params { 87 | uint16_t pd_len; 88 | uint16_t ird; 89 | uint16_t ord; 90 | } __attribute__((__packed__)); 91 | 92 | struct trp_rr { 93 | struct trp_hdr hdr; 94 | struct trp_rr_params params; 95 | } __attribute__((__packed__)); 96 | 97 | #endif 98 | -------------------------------------------------------------------------------- /scripts/perf_regather.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # 3 | # Userspace Software iWARP library for DPDK 4 | # 5 | # Authors: Patrick MacArthur 6 | # 7 | # Copyright (c) 2016, IBM Corporation 8 | # 9 | # This software is available to you under a choice of one of two 10 | # licenses. You may choose to be licensed under the terms of the GNU 11 | # General Public License (GPL) Version 2, available from the file 12 | # COPYING in the main directory of this source tree, or the 13 | # BSD license below: 14 | # 15 | # Redistribution and use in source and binary forms, with or 16 | # without modification, are permitted provided that the following 17 | # conditions are met: 18 | # 19 | # - Redistributions of source code must retain the above copyright notice, 20 | # this list of conditions and the following disclaimer. 21 | # 22 | # - Redistributions in binary form must reproduce the above copyright 23 | # notice, this list of conditions and the following disclaimer in the 24 | # documentation and/or other materials provided with the distribution. 25 | # 26 | # - Neither the name of IBM nor the names of its contributors may be 27 | # used to endorse or promote products derived from this software without 28 | # specific prior written permission. 29 | # 30 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 31 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 32 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 33 | # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 34 | # BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 35 | # ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 36 | # CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 37 | # SOFTWARE. 38 | 39 | import copy 40 | import datetime 41 | import json 42 | import os 43 | import os.path 44 | import subprocess as subp 45 | import sys 46 | import tempfile 47 | import time 48 | 49 | def load_config(conf_file): 50 | with open(conf_file, 'r') as fp: 51 | return json.load(fp) 52 | 53 | def add_result(config, results, current_config): 54 | node = config['client']['node'] 55 | resultsdir = config['common']['resultsdir'] 56 | results_file = current_config['results_file'] 57 | with open(results_file, 'r') as fp: 58 | next_result = json.load(fp) 59 | next_result['run_id'] = current_config['runid'] 60 | next_result['app'] = config['client']['app'] 61 | results.append(next_result) 62 | 63 | def main(): 64 | config = load_config(sys.argv[1]) 65 | 66 | config['common']['logid'] = sys.argv[2] 67 | config['common']['resultsdir'] = os.path.join(os.environ['HOME'], 68 | 'results', config['client']['app'], config['common']['logid']) 69 | 70 | experiment = config['experiment'] 71 | results = [] 72 | for runid in range(experiment['run_count']): 73 | for value in experiment['values']: 74 | var_opt = experiment['var_opt'] 75 | current_config = dict(var_opt=var_opt, value=value, runid=runid) 76 | current_config['results_file'] = os.path.join( 77 | os.environ['HOME'], 'results', config['client']['app'], 78 | config['common']['logid'], 79 | '{}.{}.{}.client.log'.format(var_opt, str(value), 80 | str(runid))) 81 | add_result(config, results, current_config) 82 | 83 | with open(os.path.join(config['common']['resultsdir'], 84 | 'results.json'), 'w') as fp: 85 | json.dump(results, fp, indent=2) 86 | 87 | if __name__ == '__main__': 88 | main() 89 | -------------------------------------------------------------------------------- /src/kmod/Makefile.in: -------------------------------------------------------------------------------- 1 | # @configure_input@ 2 | 3 | # Userspace Software iWARP library for DPDK 4 | # 5 | # Authors: Patrick MacArthur 6 | # 7 | # Copyright (c) 2016-2018, University of New Hampshire 8 | # 9 | # This software is available to you under a choice of one of two 10 | # licenses. You may choose to be licensed under the terms of the GNU 11 | # General Public License (GPL) Version 2, available from the file 12 | # COPYING in the main directory of this source tree, or the 13 | # BSD license below: 14 | # 15 | # Redistribution and use in source and binary forms, with or 16 | # without modification, are permitted provided that the following 17 | # conditions are met: 18 | # 19 | # - Redistributions of source code must retain the above copyright notice, 20 | # this list of conditions and the following disclaimer. 21 | # 22 | # - Redistributions in binary form must reproduce the above copyright 23 | # notice, this list of conditions and the following disclaimer in the 24 | # documentation and/or other materials provided with the distribution. 25 | # 26 | # - Neither the name of IBM nor the names of its contributors may be 27 | # used to endorse or promote products derived from this software without 28 | # specific prior written permission. 29 | # 30 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 31 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 32 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 33 | # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 34 | # BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 35 | # ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 36 | # CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 37 | # SOFTWARE. 38 | 39 | 40 | SOURCES = ae.c backports.h cm.c cm.h debug.c debug.h main.c mem.c \ 41 | obj.c obj.h qp.c urdma.h verbs.c verbs.h Kbuild 42 | 43 | ifeq ($(WANT_DEBUG),yes) 44 | export CONFIG_URDMA_DEBUG = y 45 | endif 46 | 47 | kmod_prefix=@kmod_prefix@ 48 | 49 | .PHONY: all 50 | all: 51 | for i in $(SOURCES); do if [ ! -e $${i} ]; then ln -s @srcdir@/$${i} $${i}; fi; done 52 | if [ ! -e urdma_kabi.h ]; then ln -s @top_srcdir@/include/urdma_kabi.h .; fi 53 | if [ ! -e proto_trp.h ]; then ln -s @top_srcdir@/include/proto_trp.h .; fi 54 | make -C @KERNELDIR@ M=`pwd` top_srcdir=$(realpath @top_srcdir@) PACKAGE_VERSION='\"@PACKAGE_VERSION@\"' modules 55 | 56 | Makefile: @srcdir@/Makefile.in 57 | cd @top_builddir@; ./config.status 58 | 59 | .PHONY: distdir 60 | distdir: 61 | for i in $(SOURCES); do cp @srcdir@/$${i} $(distdir)/$${i}; done 62 | 63 | .PHONY: install 64 | install: install-exec install-data 65 | 66 | .PHONY: install-exec 67 | install-exec: 68 | make -C @KERNELDIR@ M=`pwd` INSTALL_MOD_PATH=$(DESTDIR)/$(kmod_prefix) modules_install 69 | 70 | install-data: 71 | 72 | uninstall: 73 | 74 | install-dvi: 75 | 76 | install-html: 77 | 78 | install-info: 79 | 80 | install-ps: 81 | 82 | install-pdf: 83 | 84 | installdirs: 85 | 86 | check: 87 | 88 | installcheck: 89 | 90 | .PHONY: mostlyclean 91 | mostlyclean: 92 | make -C @KERNELDIR@ M=`pwd` clean 93 | $(RM) *.o.ur-safe 94 | 95 | .PHONY: clean 96 | clean: mostlyclean 97 | for i in $(SOURCES); do if [ -h $${i} ]; then $(RM) $${i}; fi; done 98 | $(RM) urdma_kabi.h 99 | $(RM) .cache.mk 100 | 101 | .PHONY: distclean 102 | distclean: clean 103 | $(RM) Makefile 104 | 105 | .PHONY: maintainer-clean 106 | maintainer-clean: distclean 107 | 108 | dvi: 109 | 110 | pdf: 111 | 112 | ps: 113 | 114 | info: 115 | 116 | html: 117 | 118 | tags: 119 | 120 | .PHONY: ctags 121 | ctags: Makefile 122 | ctags $(SOURCES) 123 | -------------------------------------------------------------------------------- /src/urdmad/interface.h: -------------------------------------------------------------------------------- 1 | /* src/urdmad/interface.h */ 2 | 3 | /* 4 | * Userspace Software iWARP library for DPDK 5 | * 6 | * Author: Patrick MacArthur 7 | * 8 | * Copyright (c) 2016, University of New Hampshire 9 | * 10 | * This software is available to you under a choice of one of two 11 | * licenses. You may choose to be licensed under the terms of the GNU 12 | * General Public License (GPL) Version 2, available from the file 13 | * COPYING in the main directory of this source tree, or the 14 | * BSD license below: 15 | * 16 | * Redistribution and use in source and binary forms, with or 17 | * without modification, are permitted provided that the following 18 | * conditions are met: 19 | * 20 | * - Redistributions of source code must retain the above copyright notice, 21 | * this list of conditions and the following disclaimer. 22 | * 23 | * - Redistributions in binary form must reproduce the above copyright 24 | * notice, this list of conditions and the following disclaimer in the 25 | * documentation and/or other materials provided with the distribution. 26 | * 27 | * - Neither the names of IBM, UNH, nor the names of its contributors may be 28 | * used to endorse or promote products derived from this software without 29 | * specific prior written permission. 30 | * 31 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | * SOFTWARE. 39 | */ 40 | 41 | #ifndef URDMAD_INTERFACE_H 42 | #define URDMAD_INTERFACE_H 43 | 44 | #include 45 | 46 | #include 47 | #include 48 | #include 49 | #include 50 | 51 | #define PENDING_DATAGRAM_INFO_SIZE 64 52 | 53 | #ifndef container_of 54 | #define container_of(ptr, type, field) \ 55 | ((type *)((uint8_t *)(ptr) - offsetof(type, field))) 56 | #endif 57 | 58 | enum usiw_port_flags { 59 | port_checksum_offload = 1, 60 | port_fdir = 2, 61 | port_2tuple = 4, 62 | port_5tuple = 8, 63 | }; 64 | 65 | struct usiw_port { 66 | int portid; 67 | uint64_t timer_freq; 68 | 69 | struct rte_mempool *rx_mempool; 70 | struct rte_mempool *tx_ddp_mempool; 71 | struct rte_mempool *tx_hdr_mempool; 72 | 73 | uint16_t rx_desc_count; 74 | uint16_t tx_desc_count; 75 | uint16_t rx_burst_size; 76 | uint16_t tx_burst_size; 77 | uint16_t max_qp; 78 | struct list_head avail_qp; 79 | struct urdmad_qp *qp; 80 | 81 | uint64_t flags; 82 | 83 | struct rte_kni *kni; 84 | struct ether_addr ether_addr; 85 | uint32_t ipv4_addr; 86 | int ipv4_prefix_len; 87 | uint16_t mtu; 88 | 89 | char kni_name[RTE_KNI_NAMESIZE]; 90 | struct rte_eth_dev_info dev_info; 91 | }; 92 | 93 | struct urdma_fd { 94 | int fd; 95 | void (*data_ready)(struct urdma_fd *fd); 96 | }; 97 | 98 | struct urdma_process { 99 | struct urdma_fd fd; 100 | struct list_node entry; 101 | struct list_head owned_qps; 102 | uint32_t core_mask[RTE_MAX_LCORE / 32]; 103 | }; 104 | 105 | struct usiw_driver { 106 | struct nl_sock *nl_sock; 107 | struct nl_cache *nl_link_cache; 108 | 109 | struct urdma_fd chardev; 110 | struct urdma_fd listen; 111 | struct urdma_fd timer; 112 | struct list_head processes; 113 | int epoll_fd; 114 | int port_count; 115 | uint16_t progress_lcore; 116 | struct usiw_port ports[]; 117 | }; 118 | 119 | #endif 120 | -------------------------------------------------------------------------------- /scripts/perf_run_ssh_helper.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # 3 | # Userspace Software iWARP library for DPDK 4 | # 5 | # Authors: Patrick MacArthur 6 | # 7 | # Copyright (c) 2016, IBM Corporation 8 | # 9 | # This software is available to you under a choice of one of two 10 | # licenses. You may choose to be licensed under the terms of the GNU 11 | # General Public License (GPL) Version 2, available from the file 12 | # COPYING in the main directory of this source tree, or the 13 | # BSD license below: 14 | # 15 | # Redistribution and use in source and binary forms, with or 16 | # without modification, are permitted provided that the following 17 | # conditions are met: 18 | # 19 | # - Redistributions of source code must retain the above copyright notice, 20 | # this list of conditions and the following disclaimer. 21 | # 22 | # - Redistributions in binary form must reproduce the above copyright 23 | # notice, this list of conditions and the following disclaimer in the 24 | # documentation and/or other materials provided with the distribution. 25 | # 26 | # - Neither the name of IBM nor the names of its contributors may be 27 | # used to endorse or promote products derived from this software without 28 | # specific prior written permission. 29 | # 30 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 31 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 32 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 33 | # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 34 | # BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 35 | # ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 36 | # CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 37 | # SOFTWARE. 38 | 39 | import json 40 | import os 41 | import os.path 42 | import subprocess as subp 43 | import sys 44 | import tempfile 45 | 46 | def load_modules(*args): 47 | if 'MODULESHOME' not in os.environ: 48 | environ['MODULESHOME'] = '/usr/share/Modules' 49 | environ['LOADEDMODULES'] = '' 50 | environ['MODULESPATH'] = '/usr/share/Modules/modulefiles:/etc/modulefiles' 51 | for arg in args: 52 | out = subp.check_output(['modulecmd', 'python', 'load', arg]) 53 | exec(out) 54 | 55 | def get_var(config, varname): 56 | if varname in config['local']: 57 | return config['local'][varname] 58 | elif varname in config['common']: 59 | return config['common'][varname] 60 | else: 61 | return None 62 | 63 | def run_logged(args): 64 | print("+", str(args)) 65 | sys.stdout.flush() 66 | return subp.call(args) 67 | 68 | 69 | def main(): 70 | try: 71 | load_modules('use.own', 'dpdk') 72 | 73 | config = json.load(sys.stdin) 74 | top_srcdir = get_var(config, 'deploy_dir') 75 | app = get_var(config, 'app') 76 | 77 | run_logged(['make', '-C', top_srcdir, 'O=build', 78 | 'EXTRA_CFLAGS=-DNDEBUG']) 79 | run_logged(['pkill', app]) 80 | 81 | eal_args = ['-l', config['local']['lcore_layout']] 82 | app_args = config['common']['app_args'] 83 | app_args += ['-o', config['current']['results_file']] 84 | for iface in get_var(config, 'dpdk_interfaces'): 85 | eal_args += ['-w', iface['port']] 86 | app_args.append(iface['ipv4_address']) 87 | eal_log_level = get_var(config, 'eal_log_level') 88 | if eal_log_level is not None: 89 | eal_args += ['--log-level', str(eal_log_level)] 90 | if 'server_ip_address' in config['local']: 91 | app_args.append(config['local']['server_ip_address']) 92 | 93 | app_path = os.path.join(top_srcdir, 'build', 'src', app, 94 | os.environ['RTE_TARGET'], 'app', app) 95 | run_logged([app_path] + eal_args + ['--'] + app_args) 96 | finally: 97 | pass 98 | 99 | if __name__ == '__main__': 100 | main() 101 | -------------------------------------------------------------------------------- /scripts/deploy_test.bash: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # Userspace Software iWARP library for DPDK 4 | # 5 | # Authors: Patrick MacArthur 6 | # 7 | # Copyright (c) 2016, IBM Corporation 8 | # Copyright (c) 2018, University of New Hampshire 9 | # 10 | # This software is available to you under a choice of one of two 11 | # licenses. You may choose to be licensed under the terms of the GNU 12 | # General Public License (GPL) Version 2, available from the file 13 | # COPYING in the main directory of this source tree, or the 14 | # BSD license below: 15 | # 16 | # Redistribution and use in source and binary forms, with or 17 | # without modification, are permitted provided that the following 18 | # conditions are met: 19 | # 20 | # - Redistributions of source code must retain the above copyright notice, 21 | # this list of conditions and the following disclaimer. 22 | # 23 | # - Redistributions in binary form must reproduce the above copyright 24 | # notice, this list of conditions and the following disclaimer in the 25 | # documentation and/or other materials provided with the distribution. 26 | # 27 | # - Neither the name of IBM nor the names of its contributors may be 28 | # used to endorse or promote products derived from this software without 29 | # specific prior written permission. 30 | # 31 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | # BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | # ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | # CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | # SOFTWARE. 39 | 40 | if [[ $# -ne 1 ]]; then 41 | printf "Usage: %s \n" "$0" >&2 42 | printf "\nExample profiles are stored in config/profiles\n" 43 | exit 1 44 | fi 45 | 46 | profile=$1 47 | source "${profile}" 48 | 49 | # Make input variables read-only 50 | readonly TOP_SRCDIR DEPLOY_DIR SERVER_NODE CLIENT_NODE 51 | readonly SERVER_DPDK_IP 52 | readonly CLIENT_DPDK_IP 53 | readonly SERVER_EXTRA_ARGS CLIENT_EXTRA_ARGS 54 | 55 | # Log ID 56 | LOGID=$(date +%Y%m%d-%H%M) 57 | readonly LOGID 58 | 59 | readonly our_tmpdir=$(mktemp -d) 60 | trap "rm -rf ${our_tmpdir}" EXIT 61 | mkfifo ${our_tmpdir}/server_fifo 62 | mkfifo ${our_tmpdir}/client_fifo 63 | 64 | cd ${TOP_SRCDIR} 65 | ssh ${SERVER_NODE} mkdir -p ${DEPLOY_DIR} 66 | ssh ${CLIENT_NODE} mkdir -p ${DEPLOY_DIR} 67 | rm -f urdma-*.tar.gz 68 | make distcheck || exit 1 69 | tarname=urdma-*.tar.gz 70 | tarbn=$(basename ${tarname} .tar.gz) 71 | ssh ${SERVER_NODE} tar -C ${DEPLOY_DIR} -xz <${tarname} 72 | ssh ${CLIENT_NODE} tar -C ${DEPLOY_DIR} -xz <${tarname} 73 | tmux neww -n dpdk-server \ 74 | "ret=127 75 | trap 'exec 3>${our_tmpdir}/server_fifo; echo \${ret} >&3' EXIT 76 | ssh -t ${SERVER_NODE} ${DEPLOY_DIR}/${tarbn}/scripts/run_real.sh \ 77 | ${DEPLOY_DIR}/${tarbn} ${SERVER_APP} ${LOGID} \ 78 | ${SERVER_EXTRA_ARGS} 79 | ret=\$?" 80 | sleep 5 81 | tmux neww -n dpdk-client \ 82 | "ret=127 83 | trap 'exec 3>${our_tmpdir}/client_fifo; echo \${ret} >&3' EXIT 84 | ssh -t ${CLIENT_NODE} ${DEPLOY_DIR}/${tarbn}/scripts/run_real.sh \ 85 | ${DEPLOY_DIR}/${tarbn} ${CLIENT_APP} ${LOGID} \ 86 | ${CLIENT_EXTRA_ARGS} \ 87 | -a ${SERVER_DPDK_IP} 88 | ret=\$?" 89 | 90 | exec 3<${our_tmpdir}/server_fifo 91 | read server_ret <&3 92 | exec 3<&- 93 | 94 | exec 3<${our_tmpdir}/client_fifo 95 | read client_ret <&3 96 | exec 3<&- 97 | 98 | rsync -a ${SERVER_NODE}:log/ ${HOME}/log/${SERVER_NODE} 99 | rsync -a ${CLIENT_NODE}:log/ ${HOME}/log/${CLIENT_NODE} 100 | 101 | printf "Output is in the following log files:\n" 102 | printf " %s/log/%s/%s/%s.log\n" \ 103 | "${HOME}" "${SERVER_NODE}" "${SERVER_APP}" "${LOGID}" 104 | printf " %s/log/%s/%s/%s.log\n" \ 105 | "${HOME}" "${CLIENT_NODE}" "${CLIENT_APP}" "${LOGID}" 106 | 107 | if [[ ${server_ret} -ne 0 ]]; then 108 | exit ${server_ret} 109 | elif [[ ${client_ret} -ne 0 ]]; then 110 | exit ${client_ret} 111 | fi 112 | -------------------------------------------------------------------------------- /src/kvstore_client/options.c: -------------------------------------------------------------------------------- 1 | /* options.c */ 2 | 3 | /* 4 | * Userspace Software iWARP library for DPDK 5 | * 6 | * Authors: Patrick MacArthur 7 | * 8 | * Copyright (c) 2016, IBM Corporation 9 | * Copyright (c) 2016, University of New Hampshire 10 | * 11 | * This software is available to you under a choice of one of two 12 | * licenses. You may choose to be licensed under the terms of the GNU 13 | * General Public License (GPL) Version 2, available from the file 14 | * COPYING in the main directory of this source tree, or the 15 | * BSD license below: 16 | * 17 | * Redistribution and use in source and binary forms, with or 18 | * without modification, are permitted provided that the following 19 | * conditions are met: 20 | * 21 | * - Redistributions of source code must retain the above copyright notice, 22 | * this list of conditions and the following disclaimer. 23 | * 24 | * - Redistributions in binary form must reproduce the above copyright 25 | * notice, this list of conditions and the following disclaimer in the 26 | * documentation and/or other materials provided with the distribution. 27 | * 28 | * - Neither the name of IBM nor the names of its contributors may be 29 | * used to endorse or promote products derived from this software without 30 | * specific prior written permission. 31 | * 32 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 33 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 34 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 35 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 36 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 37 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 38 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 39 | * SOFTWARE. 40 | */ 41 | 42 | #ifdef HAVE_CONFIG_H 43 | #include 44 | #endif 45 | 46 | #include 47 | #include 48 | #include 49 | #include 50 | #include 51 | 52 | #include 53 | #include 54 | 55 | #include "options.h" 56 | 57 | const char * 58 | option_string(void) 59 | { 60 | return "[-i input|--input ] [-o output|--output ] "; 61 | } 62 | 63 | int 64 | parse_options(int argc, char **argv, struct client_options *options) 65 | { 66 | static const struct option longopts[] = { 67 | { .name = "help", .has_arg = no_argument, 68 | .flag = NULL, .val = 'h' }, 69 | { .name = "input", .has_arg = required_argument, 70 | .flag = NULL, .val = 'i' }, 71 | { .name = "output", .has_arg = required_argument, 72 | .flag = NULL, .val = 'o' }, 73 | { 0 }, 74 | }; 75 | 76 | int ch; 77 | 78 | memset(options, 0, sizeof(*options)); 79 | while ((ch = getopt_long(argc, argv, 80 | "h" /* --help */ 81 | "i:" /* --input */ 82 | "o:" /* --output */ 83 | , longopts, NULL)) != -1) { 84 | switch (ch) { 85 | case 'h': 86 | fprintf(stderr, "-h, --help: Print this help message\n"); 87 | fprintf(stderr, "-i, --input: File with memcached commands\n"); 88 | fprintf(stderr, "-o, --output: File with memcached responses\n"); 89 | break; 90 | case 'i': 91 | options->command_fn = optarg; 92 | options->command_fp = fopen(options->command_fn, "r"); 93 | if (!options->command_fp) { 94 | rte_exit(EXIT_FAILURE, "Open %s for reading: %s\n", 95 | options->command_fn, 96 | strerror(errno)); 97 | } 98 | break; 99 | case 'o': 100 | options->output_fn = optarg; 101 | if (strcmp(options->output_fn, "-") == 0) { 102 | options->output_fp = stdout; 103 | } else { 104 | options->output_fp = fopen(options->output_fn, "w"); 105 | } 106 | if (!options->output_fp) { 107 | rte_exit(EXIT_FAILURE, "Open %s for writing: %s\n", 108 | options->output_fn, 109 | strerror(errno)); 110 | } 111 | break; 112 | default: 113 | rte_exit(EXIT_FAILURE, "Unexpected option -%c\n", ch); 114 | } 115 | } 116 | 117 | if (!options->command_fp) { 118 | fprintf(stderr, "No input file specified.\n"); 119 | return -1; 120 | } 121 | 122 | return optind - 1; 123 | } 124 | -------------------------------------------------------------------------------- /scripts/perf_run_tmux_helper.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # 3 | # Userspace Software iWARP library for DPDK 4 | # 5 | # Authors: Patrick MacArthur 6 | # 7 | # Copyright (c) 2016, IBM Corporation 8 | # 9 | # This software is available to you under a choice of one of two 10 | # licenses. You may choose to be licensed under the terms of the GNU 11 | # General Public License (GPL) Version 2, available from the file 12 | # COPYING in the main directory of this source tree, or the 13 | # BSD license below: 14 | # 15 | # Redistribution and use in source and binary forms, with or 16 | # without modification, are permitted provided that the following 17 | # conditions are met: 18 | # 19 | # - Redistributions of source code must retain the above copyright notice, 20 | # this list of conditions and the following disclaimer. 21 | # 22 | # - Redistributions in binary form must reproduce the above copyright 23 | # notice, this list of conditions and the following disclaimer in the 24 | # documentation and/or other materials provided with the distribution. 25 | # 26 | # - Neither the name of IBM nor the names of its contributors may be 27 | # used to endorse or promote products derived from this software without 28 | # specific prior written permission. 29 | # 30 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 31 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 32 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 33 | # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 34 | # BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 35 | # ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 36 | # CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 37 | # SOFTWARE. 38 | 39 | import json 40 | import io 41 | import os 42 | import os.path 43 | import subprocess as subp 44 | import sys 45 | import tempfile 46 | 47 | def get_var(config, varname): 48 | if varname in config['local']: 49 | return config['local'][varname] 50 | elif varname in config['common']: 51 | return config['common'][varname] 52 | else: 53 | return None 54 | 55 | def make_log_dir(node, app): 56 | log_dir = os.path.join(os.environ['HOME'], 'log', node, app) 57 | os.makedirs(log_dir, exist_ok=True) 58 | return log_dir 59 | 60 | def main(): 61 | try: 62 | with open(sys.argv[1], 'r') as conf_fp: 63 | try: 64 | config = json.load(conf_fp) 65 | logid = get_var(config, 'logid') 66 | node = get_var(config, 'node') 67 | app = get_var(config, 'app') 68 | log_file = os.path.join(make_log_dir(node, app), 69 | '{}.log'.format(logid)) 70 | conf_fp.seek(0, io.SEEK_SET) 71 | print("Starting processes") 72 | sys.stdout.flush() 73 | p1 = subp.Popen(['ssh', '-t', config['local']['node'], 74 | os.path.join( 75 | config['common']['deploy_dir'], 76 | 'scripts', 'perf_run_ssh_helper.py')], 77 | stdin=conf_fp, stdout=subp.PIPE, 78 | stderr=subp.STDOUT, bufsize=0) 79 | p2 = subp.Popen(['tee', log_file], stdin=p1.stdout) 80 | print("Waiting for processes to finish") 81 | sys.stdout.flush() 82 | p1.stdout.close() 83 | try: 84 | p2.communicate() 85 | except KeyboardInterrupt: 86 | p1.kill() 87 | p2.kill() 88 | p2.communicate() 89 | except ValueError: 90 | print('Could not parse JSON object from', sys.argv[1]) 91 | conf_fp.seek(0, io.SEEK_SET) 92 | print(conf_fp.read()) 93 | raise 94 | finally: 95 | if len(sys.argv) > 2: 96 | with open(sys.argv[2], 'w') as signal_fp: 97 | signal_fp.write('done\n') 98 | 99 | if __name__ == '__main__': 100 | main() 101 | -------------------------------------------------------------------------------- /src/util/util.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Userspace Software iWARP library for DPDK 3 | * 4 | * Authors: Patrick MacArthur 5 | * 6 | * Copyright (c) 2016, IBM Corporation 7 | * 8 | * This software is available to you under a choice of one of two 9 | * licenses. You may choose to be licensed under the terms of the GNU 10 | * General Public License (GPL) Version 2, available from the file 11 | * COPYING in the main directory of this source tree, or the 12 | * BSD license below: 13 | * 14 | * Redistribution and use in source and binary forms, with or 15 | * without modification, are permitted provided that the following 16 | * conditions are met: 17 | * 18 | * - Redistributions of source code must retain the above copyright notice, 19 | * this list of conditions and the following disclaimer. 20 | * 21 | * - Redistributions in binary form must reproduce the above copyright 22 | * notice, this list of conditions and the following disclaimer in the 23 | * documentation and/or other materials provided with the distribution. 24 | * 25 | * - Neither the name of IBM nor the names of its contributors may be 26 | * used to endorse or promote products derived from this software without 27 | * specific prior written permission. 28 | * 29 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 30 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 31 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 32 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 33 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 34 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 35 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 36 | * SOFTWARE. 37 | */ 38 | 39 | #ifndef UTIL_H 40 | #define UTIL_H 41 | 42 | #include 43 | #include 44 | #include 45 | 46 | #include 47 | #include 48 | 49 | #ifdef HAVE_UINT16_T_PORT_ID 50 | typedef uint16_t dpdk_port_id_type; 51 | #else 52 | typedef uint8_t dpdk_port_id_type; 53 | #endif 54 | 55 | #ifdef RTE_ETH_NAME_MAX_LEN 56 | static const size_t pci_addr_namebuf_size = RTE_ETH_NAME_MAX_LEN; 57 | #else 58 | /* maximum size of buffer to hold "0000:00:00.0" */ 59 | static const size_t pci_addr_namebuf_size = 13; 60 | #endif 61 | 62 | #ifdef NDEBUG 63 | #define NDEBUG_UNUSED __attribute__((unused)) 64 | #else 65 | #define NDEBUG_UNUSED 66 | #endif 67 | 68 | #define DO_WARN_ONCE(cond, var, format, ...) ({ \ 69 | static bool var = false; \ 70 | if ((cond) && !var) { \ 71 | var = true; \ 72 | RTE_LOG(WARNING, USER1, format, ##__VA_ARGS__); \ 73 | }; cond; }) 74 | 75 | #define PASTE(x, y) x##y 76 | 77 | /** Prints the given warning message (like fprintf(stderr, format, ...)) if cond 78 | * is true, but only once per execution. */ 79 | #define WARN_ONCE(cond, format, ...) \ 80 | DO_WARN_ONCE(cond, PASTE(xyzwarn_, __LINE__), format, ##__VA_ARGS__) 81 | 82 | int 83 | parse_ipv4_address(const char *str, uint32_t *address, int *prefix_len); 84 | 85 | void 86 | port_dump_info(FILE *stream, uint16_t port_id); 87 | 88 | #ifndef HAVE_RTE_ETH_DEV_GET_NAME_BY_PORT 89 | static inline int 90 | rte_eth_dev_get_name_by_port(uint16_t port_id, char *name) 91 | { 92 | struct rte_eth_dev_info info; 93 | int ret; 94 | if ((ret = rte_eth_dev_info_get(port_id, &info))) 95 | return ret; 96 | return rte_pci_device_name(&info->pci_dev->addr, name, 97 | pci_addr_namebuf_size); 98 | } 99 | #endif 100 | 101 | #ifdef HAVE_RTE_ETH_DEV_GET_PORT_BY_NAME 102 | static inline int 103 | lookup_ethdev_by_pci_addr(struct rte_pci_addr *addr) 104 | { 105 | char namebuf[pci_addr_namebuf_size]; 106 | dpdk_port_id_type port_id; 107 | int ret; 108 | 109 | rte_pci_device_name(addr, namebuf, pci_addr_namebuf_size); 110 | ret = rte_eth_dev_get_port_by_name(namebuf, &port_id); 111 | if (!ret) 112 | return port_id; 113 | else 114 | return ret; 115 | } 116 | #else 117 | static inline int 118 | lookup_ethdev_by_pci_addr(struct rte_pci_addr *addr) 119 | { 120 | struct rte_eth_dev_info info; 121 | int x, count; 122 | 123 | count = rte_eth_dev_count(); 124 | for (x = 0; x < count; ++x) { 125 | if (!rte_eth_dev_is_valid_port(x)) 126 | continue; 127 | rte_eth_dev_info_get(x, &info); 128 | if (!rte_eal_compare_pci_addr(addr, &info.pci_dev->addr)) { 129 | return x; 130 | } 131 | } 132 | return -ENODEV; 133 | } 134 | #endif 135 | 136 | #endif 137 | -------------------------------------------------------------------------------- /include/urdma_kabi.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Software iWARP device driver for Linux 3 | * 4 | * Authors: Bernard Metzler 5 | * Patrick MacArthur 6 | * 7 | * Copyright (c) 2008-2016, IBM Corporation 8 | * Copyright (c) 2016, University of New Hampshire 9 | * 10 | * This software is available to you under a choice of one of two 11 | * licenses. You may choose to be licensed under the terms of the GNU 12 | * General Public License (GPL) Version 2, available from the file 13 | * COPYING in the main directory of this source tree, or the 14 | * BSD license below: 15 | * 16 | * Redistribution and use in source and binary forms, with or 17 | * without modification, are permitted provided that the following 18 | * conditions are met: 19 | * 20 | * - Redistributions of source code must retain the above copyright notice, 21 | * this list of conditions and the following disclaimer. 22 | * 23 | * - Redistributions in binary form must reproduce the above copyright 24 | * notice, this list of conditions and the following disclaimer in the 25 | * documentation and/or other materials provided with the distribution. 26 | * 27 | * - Neither the name of IBM nor the names of its contributors may be 28 | * used to endorse or promote products derived from this software without 29 | * specific prior written permission. 30 | * 31 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | * SOFTWARE. 39 | */ 40 | 41 | #ifndef URDMA_KABI_H 42 | #define URDMA_KABI_H 43 | 44 | #include 45 | 46 | /* 47 | * user commands/command responses must correlate with the siw_abi 48 | * in user land. 49 | */ 50 | 51 | #define URDMA_ABI_VERSION_MIN 0 52 | #define URDMA_ABI_VERSION_MAX 0 53 | 54 | #define VERSION_ID_URDMA 3 55 | #define URDMA_VENDOR_ID 0x626d74 /* ascii 'bmt' for now */ 56 | #define URDMA_VENDOR_PART_ID 0x0816 57 | 58 | #define URDMA_NODE_DESC "Userspace RDMA" 59 | #define URDMA_DEV_PREFIX "urdma_" 60 | 61 | struct urdma_uresp_create_cq { 62 | uint32_t cq_id; 63 | }; 64 | 65 | struct urdma_udata_create_qp { 66 | uint32_t ord_max; 67 | uint32_t ird_max; 68 | uint16_t urdmad_dev_id; 69 | uint16_t urdmad_qp_id; 70 | uint16_t rxq; 71 | uint16_t txq; 72 | }; 73 | 74 | struct urdma_uresp_create_qp { 75 | uint32_t kmod_qp_id; 76 | }; 77 | 78 | struct urdma_uresp_alloc_ctx { 79 | uint32_t dev_id; 80 | uint32_t event_fd; 81 | }; 82 | 83 | struct urdma_cq_event { 84 | uint32_t event_type; 85 | uint32_t cq_id; 86 | }; 87 | 88 | struct urdma_qp_connected_event { 89 | uint32_t event_type; 90 | uint32_t kmod_qp_id; 91 | uint16_t urdmad_dev_id; 92 | uint16_t urdmad_qp_id; 93 | uint32_t src_ipv4; 94 | uint16_t src_port; 95 | uint32_t dst_ipv4; 96 | uint8_t dst_ether[6]; 97 | uint16_t dst_port; 98 | uint8_t ord_max; 99 | uint8_t ird_max; 100 | uint16_t rxq; 101 | uint16_t txq; 102 | }; 103 | 104 | struct urdma_qp_disconnected_event { 105 | uint32_t event_type; 106 | uint16_t urdmad_dev_id; 107 | uint16_t urdmad_qp_id; 108 | }; 109 | 110 | struct urdma_qp_rtr_event { 111 | uint32_t event_type; 112 | uint32_t kmod_qp_id; 113 | }; 114 | 115 | struct urdma_event_storage { 116 | uint32_t event_type; 117 | char buf[sizeof(struct urdma_qp_connected_event) - 4]; 118 | }; 119 | 120 | enum urdma_event_type { 121 | SIW_EVENT_COMP_POSTED = 0, 122 | /**< Sent from userspace to kernel to indicate that the 123 | * associated completion channel should be woken up. */ 124 | SIW_EVENT_QP_CONNECTED = 1, 125 | /**< Sent from the kernel to userspace to indicate that a 126 | * connection is ready. */ 127 | SIW_EVENT_QP_RTR = 2, 128 | /**< Sent from userspace to the kernel in response to a 129 | * QP_CONNECTED event to indicate that we are ready to receive 130 | * messages on the queue pair (flow director filters have been 131 | * set up and DPDK queues enabled). */ 132 | SIW_EVENT_QP_DISCONNECTED = 3, 133 | /**< Sent from the kernel to userspace to indicate that a 134 | * connection has been torn down. */ 135 | }; 136 | 137 | #endif 138 | -------------------------------------------------------------------------------- /include/proto_memcached.h: -------------------------------------------------------------------------------- 1 | /* proto_memcached.h */ 2 | 3 | /* 4 | * Userspace Software iWARP library for DPDK 5 | * 6 | * Authors: Patrick MacArthur 7 | * 8 | * Copyright (c) 2016, IBM Corporation 9 | * 10 | * This software is available to you under a choice of one of two 11 | * licenses. You may choose to be licensed under the terms of the GNU 12 | * General Public License (GPL) Version 2, available from the file 13 | * COPYING in the main directory of this source tree, or the 14 | * BSD license below: 15 | * 16 | * Redistribution and use in source and binary forms, with or 17 | * without modification, are permitted provided that the following 18 | * conditions are met: 19 | * 20 | * - Redistributions of source code must retain the above copyright notice, 21 | * this list of conditions and the following disclaimer. 22 | * 23 | * - Redistributions in binary form must reproduce the above copyright 24 | * notice, this list of conditions and the following disclaimer in the 25 | * documentation and/or other materials provided with the distribution. 26 | * 27 | * - Neither the name of IBM nor the names of its contributors may be 28 | * used to endorse or promote products derived from this software without 29 | * specific prior written permission. 30 | * 31 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | * SOFTWARE. 39 | */ 40 | 41 | #ifndef PROTO_MEMCACHED_H 42 | #define PROTO_MEMCACHED_H 43 | 44 | enum memcached_magic { 45 | memcached_magic_request = 0x80, 46 | memcached_magic_response = 0x81, 47 | }; 48 | 49 | enum memcached_response_status { 50 | memcached_no_error = 0, 51 | memcached_key_not_found = 1, 52 | memcached_key_exists = 2, 53 | memcached_value_too_large = 3, 54 | memcached_invalid_arguments = 4, 55 | memcached_item_not_stored = 5, 56 | memcached_non_numeric_value = 6, 57 | memcached_unknown_command = 0x81, 58 | memcached_out_of_memory = 0x82, 59 | }; 60 | 61 | enum memcached_opcode { 62 | memcached_opcode_get = 0x00, 63 | memcached_opcode_set = 0x01, 64 | memcached_opcode_add = 0x02, 65 | memcached_opcode_replace = 0x03, 66 | memcached_opcode_delete = 0x04, 67 | }; 68 | 69 | struct memcached_header { 70 | uint8_t magic; 71 | uint8_t opcode; 72 | uint16_t key_length; 73 | uint8_t extras_length; 74 | uint8_t data_type; 75 | uint16_t status; /* reserved for requests */ 76 | uint32_t total_body_length; 77 | uint32_t opaque; 78 | uint64_t cas_version; 79 | 80 | /* Added for RDMA protocol capability */ 81 | uint32_t rdma_stag; 82 | uint32_t rdma_length; 83 | uint64_t rdma_offset; 84 | }; 85 | 86 | struct memcached_set_req_header { 87 | struct memcached_header head; 88 | uint32_t flags; 89 | uint32_t expire; 90 | }; 91 | 92 | struct memcached_get_resp_header { 93 | struct memcached_header head; 94 | uint32_t flags; 95 | uint32_t value_len; 96 | }; 97 | 98 | /** Returns a pointer to the key inside of a memcached header. 99 | * 100 | * Note 1: This will return a pointer even if there is no key in the packet, 101 | * this should be verified using the key_length field beforehand. 102 | * 103 | * Note 2: The key that is returned is directly from the packet and is NOT 104 | * null-terminated. */ 105 | static inline char * 106 | memcached_header_key(struct memcached_header *head) 107 | { 108 | return (char *)((uintptr_t)head + sizeof(*head) + head->extras_length); 109 | } /* memcached_header_key */ 110 | 111 | /** Returns a pointer to the value inside of a memcached header. 112 | * 113 | * Note 1: This will return a pointer even if there is no value in the packet, 114 | * this should be verified using the length fields beforehand. 115 | * 116 | * Note 2: The value that is returned is directly from the packet. It may be 117 | * binary data and is NOT null-terminated. The length can be calculated from 118 | * other fields in the packet. */ 119 | static inline void * 120 | memcached_header_value(struct memcached_header *head) 121 | { 122 | return (void *)((uintptr_t)head + sizeof(*head) + head->extras_length 123 | + rte_be_to_cpu_16(head->key_length)); 124 | } /* memcached_header_key */ 125 | 126 | #endif 127 | -------------------------------------------------------------------------------- /src/util/binheap.c: -------------------------------------------------------------------------------- 1 | /* binheap.c */ 2 | 3 | /* 4 | * Userspace Software iWARP library for DPDK 5 | * 6 | * Authors: Patrick MacArthur 7 | * 8 | * Copyright (c) 2017, University of New Hampshire 9 | * 10 | * This software is available to you under a choice of one of two 11 | * licenses. You may choose to be licensed under the terms of the GNU 12 | * General Public License (GPL) Version 2, available from the file 13 | * COPYING in the main directory of this source tree, or the 14 | * BSD license below: 15 | * 16 | * Redistribution and use in source and binary forms, with or 17 | * without modification, are permitted provided that the following 18 | * conditions are met: 19 | * 20 | * - Redistributions of source code must retain the above copyright notice, 21 | * this list of conditions and the following disclaimer. 22 | * 23 | * - Redistributions in binary form must reproduce the above copyright 24 | * notice, this list of conditions and the following disclaimer in the 25 | * documentation and/or other materials provided with the distribution. 26 | * 27 | * - Neither the name of IBM nor the names of its contributors may be 28 | * used to endorse or promote products derived from this software without 29 | * specific prior written permission. 30 | * 31 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | * SOFTWARE. 39 | */ 40 | 41 | #include "binheap.h" 42 | #include 43 | #include 44 | 45 | #define SWAP(a, b) { \ 46 | typeof(a) tmp = a; \ 47 | a = b; \ 48 | b = tmp; \ 49 | } 50 | 51 | static int array_min(uint32_t *arr, int i, int j) 52 | { 53 | if (arr[j] < arr[i]) 54 | return j; 55 | else 56 | return i; 57 | } 58 | 59 | static void array_swap(uint32_t *arr, int i, int j) 60 | { 61 | SWAP(arr[i], arr[j]); 62 | } 63 | 64 | static void push_down(struct binheap *binheap, int index) 65 | { 66 | int next; 67 | 68 | while (index != 0) { 69 | next = (index - 1) / 2; 70 | if (binheap->arr[index] < binheap->arr[next]) { 71 | array_swap(binheap->arr, index, next); 72 | index = next; 73 | } else { 74 | return; 75 | } 76 | } 77 | } 78 | 79 | /* Insert element v into binheap while preserving the heap invariant. The 80 | * caller is responsible for ensuring the heap remains within its capacity. */ 81 | int binheap_insert(struct binheap *binheap, uint32_t v) 82 | { 83 | assert(binheap->size <= binheap->capacity); 84 | if (binheap->size == binheap->capacity) { 85 | return -1; 86 | } 87 | binheap->arr[binheap->size] = v; 88 | push_down(binheap, binheap->size++); 89 | return 0; 90 | } /* binheap_insert */ 91 | 92 | /* Returns the minimum element of the heap without removing it. */ 93 | int binheap_peek(struct binheap *binheap, uint32_t *v) 94 | { 95 | if (binheap->size == 0) { 96 | return -1; 97 | } 98 | 99 | *v = binheap->arr[0]; 100 | return 0; 101 | } /* binheap_peek */ 102 | 103 | /* Removes the minimum element of the heap while preserving the heap 104 | * invariant. */ 105 | int binheap_pop(struct binheap *binheap) 106 | { 107 | int index = 0, min; 108 | unsigned int left, right; 109 | 110 | if (binheap->size == 0) { 111 | return -1; 112 | } 113 | array_swap(binheap->arr, 0, --binheap->size); 114 | while ((left = index * 2 + 1) < binheap->size) { 115 | min = array_min(binheap->arr, index, left); 116 | right = left + 1; 117 | if (right < binheap->size) { 118 | min = array_min(binheap->arr, min, right); 119 | } 120 | 121 | if (min != index) { 122 | array_swap(binheap->arr, index, min); 123 | index = min; 124 | } else { 125 | break; 126 | } 127 | } 128 | return 0; 129 | } /* binheap_pop */ 130 | 131 | /* Creates a binheap with the given capacity >= 0. The binheap will be able to 132 | * hold elements of type uint32_t and calling minheap_peek() will return the 133 | * minimum element placed into it. */ 134 | struct binheap * 135 | binheap_new(size_t capacity) 136 | { 137 | struct binheap *heap; 138 | 139 | assert(capacity != 0); 140 | 141 | heap = malloc(sizeof(*heap) + capacity * sizeof(uint32_t)); 142 | if (heap) { 143 | heap->capacity = capacity; 144 | heap->size = 0; 145 | } 146 | return heap; 147 | } /* binheap_new */ 148 | -------------------------------------------------------------------------------- /src/liburdma/verbs.h: -------------------------------------------------------------------------------- 1 | /* verbs.h */ 2 | 3 | /* 4 | * Userspace Software iWARP library for DPDK 5 | * 6 | * Authors: Patrick MacArthur 7 | * 8 | * Copyright (c) 2016, IBM Corporation 9 | * Copyright (c) 2016, University of New Hampshire 10 | * 11 | * This software is available to you under a choice of one of two 12 | * licenses. You may choose to be licensed under the terms of the GNU 13 | * General Public License (GPL) Version 2, available from the file 14 | * COPYING in the main directory of this source tree, or the 15 | * BSD license below: 16 | * 17 | * Redistribution and use in source and binary forms, with or 18 | * without modification, are permitted provided that the following 19 | * conditions are met: 20 | * 21 | * - Redistributions of source code must retain the above copyright notice, 22 | * this list of conditions and the following disclaimer. 23 | * 24 | * - Redistributions in binary form must reproduce the above copyright 25 | * notice, this list of conditions and the following disclaimer in the 26 | * documentation and/or other materials provided with the distribution. 27 | * 28 | * - Neither the name of IBM nor the names of its contributors may be 29 | * used to endorse or promote products derived from this software without 30 | * specific prior written permission. 31 | * 32 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 33 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 34 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 35 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 36 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 37 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 38 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 39 | * SOFTWARE. 40 | */ 41 | 42 | #ifndef VERBS_H 43 | #define VERBS_H 44 | 45 | #include 46 | #include 47 | #include 48 | #include 49 | 50 | #include 51 | 52 | #include 53 | #include 54 | 55 | #define URDMA_DEVICE_VENDOR_ID 0x626d74 56 | #define URDMA_DEVICE_VENDOR_PART_ID 0x0816 57 | 58 | struct urdma_ah { 59 | struct ether_addr ether_addr; 60 | uint16_t udp_port; 61 | uint32_t ipv4_addr; 62 | }; 63 | 64 | struct urdma_qp_stats { 65 | uintmax_t *recv_count_histo; 66 | /**< An array of recv_max_burst_size + 1 elements. The 67 | * element at index X corresponds to the number of times that a 68 | * burst of X messages was received by a call to 69 | * rte_eth_rx_burst(). */ 70 | size_t recv_max_burst_size; 71 | /**< The maximum burst size that usiw requests from DPDK. */ 72 | }; 73 | 74 | struct urdma_qp_stats_ex { 75 | size_t size; 76 | /**< The size of this structure filled in by urdma. Any fields 77 | * beyond this size are undefined. */ 78 | struct urdma_qp_stats base; 79 | /**< Base stats from initial (non-versioned) stats 80 | * structure. */ 81 | uintmax_t recv_psn_gap_count; 82 | /**< The number of times a packet was received with a PSN that 83 | * was not the next expected PSN. */ 84 | uintmax_t recv_retransmit_count; 85 | /**< The number of received retransmissions. */ 86 | uintmax_t recv_sack_count; 87 | /**< The number of received SACK packets. */ 88 | }; 89 | 90 | struct ibv_mr * 91 | urdma_reg_mr_with_rkey(struct ibv_pd *pd, void *addr, size_t len, int access, 92 | uint32_t rkey); 93 | 94 | int 95 | urdma_accl_post_recv(struct ibv_qp *qp, void *addr, size_t length, 96 | void *context); 97 | 98 | int 99 | urdma_accl_post_recvv(struct ibv_qp *qp, const struct iovec *iov, 100 | size_t iov_size, void *context); 101 | 102 | int 103 | urdma_accl_post_send(struct ibv_qp *qp, void *addr, size_t length, 104 | struct urdma_ah *ah, void *context); 105 | 106 | int 107 | urdma_accl_post_sendv(struct ibv_qp *qp, struct iovec *iov, size_t iov_size, 108 | struct urdma_ah *ah, void *context); 109 | 110 | int 111 | urdma_accl_post_write(struct ibv_qp *qp, void *addr, size_t length, 112 | struct urdma_ah *ah, uint64_t remote_addr, 113 | uint32_t rkey, void *context); 114 | 115 | int 116 | urdma_accl_post_read(struct ibv_qp *qp, void *addr, size_t length, 117 | struct urdma_ah *ah, uint64_t remote_addr, 118 | uint32_t rkey, void *context); 119 | 120 | void 121 | urdma_query_qp_stats(const struct ibv_qp *restrict qp, 122 | struct urdma_qp_stats *restrict stats); 123 | 124 | void 125 | urdma_free_qp_stats_ex(struct urdma_qp_stats_ex *stats); 126 | 127 | struct urdma_qp_stats_ex * 128 | urdma_query_qp_stats_ex(const struct ibv_qp *qp); 129 | 130 | #endif 131 | -------------------------------------------------------------------------------- /src/kmod/backports.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Software iWARP device driver for Linux 3 | * 4 | * Authors: Bernard Metzler 5 | * Patrick MacArthur 6 | * 7 | * Copyright (c) 2008-2016, IBM Corporation 8 | * Copyright (c) 2016-2017, University of New Hampshire 9 | * 10 | * This software is available to you under a choice of one of two 11 | * licenses. You may choose to be licensed under the terms of the GNU 12 | * General Public License (GPL) Version 2, available from the file 13 | * COPYING in the main directory of this source tree, or the 14 | * BSD license below: 15 | * 16 | * Redistribution and use in source and binary forms, with or 17 | * without modification, are permitted provided that the following 18 | * conditions are met: 19 | * 20 | * - Redistributions of source code must retain the above copyright notice, 21 | * this list of conditions and the following disclaimer. 22 | * 23 | * - Redistributions in binary form must reproduce the above copyright 24 | * notice, this list of conditions and the following disclaimer in the 25 | * documentation and/or other materials provided with the distribution. 26 | * 27 | * - Neither the name of IBM nor the names of its contributors may be 28 | * used to endorse or promote products derived from this software without 29 | * specific prior written permission. 30 | * 31 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | * SOFTWARE. 39 | */ 40 | 41 | #ifndef USIW_BACKPORTS_H 42 | #define USIW_BACKPORTS_H 43 | 44 | #include 45 | 46 | #if LINUX_VERSION_CODE >= KERNEL_VERSION(3, 2, 0) 47 | /* Introduced in Linux commit 3ebeebc38b4b and first appeared in 48 | * Linux 3.2-rc1. */ 49 | #define HAVE_RFC_6581 1 50 | #endif 51 | 52 | #if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 2, 0) 53 | /* Introduced in Linux commit 94dcba3309d97 and first appeared in 54 | * Linux 4.2-rc1. */ 55 | #define HAVE_STRUCT_IB_CQ_INIT_ATTR 1 56 | /* Introduced in Linux commit 2528e33e68092 and first appeared in 57 | * Linux 4.2-rc1. */ 58 | #define HAVE_IB_QUERY_DEVICE_UDATA 1 59 | /* Introduced in Linux commits a97e2d86a9b88 and 4cd7c9479aff3, which first 60 | * appeared in Linux 4.2-rc1. */ 61 | #define HAVE_IB_PROCESS_MAD_SIZES 1 62 | /* Introduced in Linux commit 7738613e7cb41 and first appeared in 63 | * Linux 4.2-rc1. */ 64 | #define HAVE_IB_GET_PORT_IMMUTABLE 1 65 | #endif 66 | 67 | #if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 3, 0) 68 | /* Introduced in Linux commit 036b10635739f and first appeared in 69 | * Linux 4.3-rc1. */ 70 | #define HAVE_IB_DISASSOCIATE_CONTEXT 1 71 | #endif 72 | 73 | #if LINUX_VERSION_CODE < KERNEL_VERSION(4, 5, 0) 74 | /* Removed in Linux commit b7d3e0a94fe1, which first appeared in 75 | * Linux 4.5-rc1. */ 76 | #define HAVE_IB_REG_PHYS_MR 1 77 | /* Removed in Linux commit feb7c1e38bcc, which first appeared in 78 | * Linux 4.5-rc1. */ 79 | #define HAVE_IB_BIND_MW 1 80 | #endif 81 | 82 | #if LINUX_VERSION_CODE < KERNEL_VERSION(4, 6, 0) 83 | /* The iWARP port mapper was added to core in Linux commit b493d91d333e, which 84 | * first appeared in Linux 4.6-rc1. After this commit, we need to use the 85 | * mapped local and remote addresses rather than the user-requested ones. */ 86 | #define m_local_addr local_addr 87 | #define m_remote_addr remote_addr 88 | #endif 89 | 90 | #if LINUX_VERSION_CODE < KERNEL_VERSION(4, 8, 0) 91 | /* struct dma_attrs was removed in Linux commit 00085f1efa38 ("dma-mapping: use 92 | * unsigned long for dma_attrs") which first appeared in Linux 4.8-rc1. */ 93 | typedef struct dma_attrs *dma_attrs_t; 94 | #else 95 | typedef unsigned long dma_attrs_t; 96 | #endif 97 | 98 | #if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 10, 0) 99 | #define HAVE_CREATE_AH_UDATA 1 100 | #endif 101 | 102 | #if LINUX_VERSION_CODE < KERNEL_VERSION(4, 11, 0) 103 | #define HAVE_DEVICE_ARCHDATA_DMA_OPS 1 104 | #define HAVE_IB_DMA_MAPPING_OPS 1 105 | #define kref_read(ref) (atomic_read(&((ref)->refcount))) 106 | #define ib_dma_device(rdma_dev) ((rdma_dev).dma_device) 107 | #else 108 | #define ib_dma_device(rdma_dev) ((rdma_dev).dev.parent) 109 | #endif 110 | 111 | #if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 12, 0) 112 | #define HAVE_STRUCT_RDMA_AH_ATTR 1 113 | #endif 114 | 115 | #if LINUX_VERSION_CODE < KERNEL_VERSION(4, 13, 0) 116 | #define HAVE_DMA_MAP_OPS_SET_DMA_MASK 1 117 | #endif 118 | 119 | #if LINUX_VERSION_CODE >= KERNEL_VERSION(4, 16, 0) 120 | #define GETNAME_RETURN_SIZE 1 121 | #endif 122 | 123 | #endif 124 | -------------------------------------------------------------------------------- /doc/flow-director.txt: -------------------------------------------------------------------------------- 1 | This works for the i40e driver. 2 | 3 | The whole filtering interface in DPDK is inspired by ioctl(2) (the poster child 4 | of how not to design an API), which makes figuring out how to use Flow Director 5 | even harder than it already is. However, once you actually figure out what the 6 | driver is actually doing, then it becomes easier to figure out. 7 | 8 | There are three steps to successfully enabling Flow Director and adding rules: 9 | 10 | 1. Enable perfect match mode in the eth_conf when you call 11 | rte_eth_dev_configure(). 12 | 13 | 2. For every flow type you want to create rules for, set the fields to match 14 | against (this set must be the same for every rule). 15 | 16 | 3. Add the rules. 17 | 18 | 19 | [1] ENABLE PERFECT MATCH MODE 20 | 21 | The first step is easy. You just have to set the following two fields in the 22 | rte_eth_conf structure: 23 | 24 | eth_conf.fdir_conf.mode = RTE_FDIR_MODE_PERFECT; 25 | eth_conf.fdir_conf.pballoc = RTE_FDIR_PBALLOC_64K; 26 | 27 | The pballoc field seems to not be used by the i40e driver, but I left it in 28 | since older hardware needs to reserve a specific amount of NIC memory for the 29 | rule table. 30 | 31 | The fdir_conf structure also contains a mask field that lets you set masks. As 32 | far as I can tell, the i40e driver completely ignores the mask field (even if 33 | you set fields here, if you then query it, it shows up as all zeros). I think 34 | older hardware may require this (and not use the below mechanism) but I haven't 35 | tested it due to lack of hardware. 36 | 37 | Everything else that you do with fdir is done using the ioctl(2)-like call, 38 | rte_eth_dev_filter_ctrl(). It has a number of subcommands below but you really 39 | only need to use 2 of them, described below in order: 40 | 41 | [2] SELECT FIELDS 42 | 43 | For every type of flow that you want to create a rule for, you need to tell it 44 | what fields to select: 45 | 46 | struct rte_eth_fdir_filter_info filter_info; 47 | memset(&filter_info, 0, sizeof(filter_info)); 48 | filter_info.info_type = RTE_ETH_FDIR_FILTER_INPUT_SET_SELECT; 49 | filter_info.info.input_set_conf.flow_type = RTE_ETH_FLOW_NONFRAG_IPV4_UDP; 50 | filter_info.info.input_set_conf.inset_size = 2; 51 | filter_info.info.input_set_conf.field[0] 52 | = RTE_ETH_INPUT_SET_L3_DST_IP4; 53 | filter_info.info.input_set_conf.field[1] 54 | = RTE_ETH_INPUT_SET_L4_UDP_DST_PORT; 55 | filter_info.info.input_set_conf.op = RTE_ETH_INPUT_SET_SELECT; 56 | retval = rte_eth_dev_filter_ctrl(portid, RTE_ETH_FILTER_FDIR, 57 | RTE_ETH_FILTER_SET, &filter_info); 58 | if (retval != 0) { 59 | rte_exit(EXIT_FAILURE, "Could not set fdir info: %s\n", 60 | strerror(-retval)); 61 | } 62 | 63 | This will make all UDP/IPv4 rules match the destination IPv4 address and 64 | destination port number. 65 | 66 | The inset_size is the size of the field array. The field array consists of 67 | enum values each of which specifies a field in the packet (which has to 68 | correspond to the flow_type). And the op in most cases should be 69 | RTE_ETH_INPUT_SET_SELECT. There is also RTE_ETH_INPUT_SET_ADD, but I have not 70 | tested it and don't really see a use case for it. 71 | 72 | [3] ADD RULES 73 | 74 | Finally, adding rules: 75 | 76 | memset(&fdirf, 0, sizeof(fdirf)); 77 | fdirf.soft_id = 1; 78 | fdirf.input.flow_type = RTE_ETH_FLOW_NONFRAG_IPV4_UDP; 79 | fdirf.input.flow.udp4_flow.dst_port = rte_cpu_to_be_16(10000); 80 | fdirf.input.flow.udp4_flow.ip.dst_ip = IPv4(10, 0, 0, 1); 81 | fdirf.action.rx_queue = 2; 82 | fdirf.action.behavior = RTE_ETH_FDIR_ACCEPT; 83 | fdirf.action.report_status = RTE_ETH_FDIR_NO_REPORT_STATUS; 84 | retval = rte_eth_dev_filter_ctrl(iface->portid, RTE_ETH_FILTER_FDIR, 85 | RTE_ETH_FILTER_ADD, &fdirf); 86 | if (retval != 0) { 87 | rte_exit(EXIT_FAILURE, "Could not add fdir UDP filter: %s\n", 88 | strerror(-retval)); 89 | } 90 | 91 | This adds a rule which assigns inbound packets destined for 10.0.0.1:10000 to 92 | receive queue 2. The soft_id must be unique for every rule that you add (I 93 | believe you can use it in a RTE_ETH_FILTER_UPDATE operation to update the rule 94 | later, but haven't tested that). The behavior can be RTE_ETH_FDIR_ACCEPT or 95 | even RTE_ETH_FDIR_DROP if you know you don't want to receive certain flows. 96 | The report_status field is not important for what we want to do at the moment; 97 | I think it is more relevant if you have rules based on arbitrary offsets in the 98 | packet (called "flexible" in Flow Director terms). 99 | 100 | Rules can be deleted with the analogous RTE_ETH_FILTER_DELETE operation. 101 | 102 | You can use the RTE_ETH_FILTER_INFO operation to query the status of the flow 103 | director. It returns back a "struct rte_eth_fdir_info" (not to be confused 104 | with the similarly named "struct rte_eth_fdir_filter_info" that you used 105 | above). In particular, you can find out the maximum number of rules you can add. 106 | -------------------------------------------------------------------------------- /src/kvstore_server/nvm.c: -------------------------------------------------------------------------------- 1 | /* server/nvm.c */ 2 | 3 | /* 4 | * Userspace Software iWARP library for DPDK 5 | * 6 | * Authors: Patrick MacArthur 7 | * 8 | * Copyright (c) 2016, IBM Corporation 9 | * Copyright (c) 2016, University of New Hampshire 10 | * 11 | * This software is available to you under a choice of one of two 12 | * licenses. You may choose to be licensed under the terms of the GNU 13 | * General Public License (GPL) Version 2, available from the file 14 | * COPYING in the main directory of this source tree, or the 15 | * BSD license below: 16 | * 17 | * Redistribution and use in source and binary forms, with or 18 | * without modification, are permitted provided that the following 19 | * conditions are met: 20 | * 21 | * - Redistributions of source code must retain the above copyright notice, 22 | * this list of conditions and the following disclaimer. 23 | * 24 | * - Redistributions in binary form must reproduce the above copyright 25 | * notice, this list of conditions and the following disclaimer in the 26 | * documentation and/or other materials provided with the distribution. 27 | * 28 | * - Neither the name of IBM nor the names of its contributors may be 29 | * used to endorse or promote products derived from this software without 30 | * specific prior written permission. 31 | * 32 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 33 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 34 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 35 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 36 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 37 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 38 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 39 | * SOFTWARE. 40 | */ 41 | 42 | #ifdef HAVE_CONFIG_H 43 | #include 44 | #endif 45 | 46 | #include 47 | #include 48 | #include 49 | #include 50 | #include 51 | #include 52 | #include 53 | #include 54 | #include 55 | 56 | #include 57 | #include 58 | 59 | #include "nvm.h" 60 | 61 | #if defined(PAGESIZE) && PAGESIZE >= 0 62 | #define DEFAULT_PAGE_SIZE PAGESIZE 63 | #else 64 | #define DEFAULT_PAGE_SIZE 4096 65 | #endif 66 | 67 | struct nvm_context * 68 | nvm_open(const char *partition_name) 69 | { 70 | struct nvm_context *ctx; 71 | struct stat st; 72 | int fd; 73 | 74 | fd = open(partition_name, O_RDWR); 75 | if (fd < 0) { 76 | return NULL; 77 | } 78 | 79 | if (fstat(fd, &st) < 0) { 80 | goto close_fd; 81 | } 82 | 83 | ctx = malloc(sizeof(*ctx)); 84 | if (!ctx) { 85 | goto close_fd; 86 | } 87 | 88 | ctx->size = st.st_size; 89 | ctx->addr = mmap(NULL, ctx->size, PROT_READ|PROT_WRITE, MAP_SHARED, 90 | fd, 0); 91 | if (ctx->addr == MAP_FAILED) { 92 | goto free_ctx; 93 | } 94 | 95 | if (close(fd) < 0) { 96 | nvm_close(ctx); 97 | goto out; 98 | } 99 | return ctx; 100 | 101 | free_ctx: 102 | free(ctx); 103 | 104 | close_fd: 105 | close(fd); 106 | 107 | out: 108 | return NULL; 109 | } /* nvm_open */ 110 | 111 | 112 | struct nvm_context * 113 | nvm_open_anonymous(size_t storage_size) 114 | { 115 | struct nvm_context *ctx; 116 | 117 | ctx = malloc(sizeof(*ctx)); 118 | if (!ctx) { 119 | goto err_out; 120 | } 121 | 122 | ctx->size = storage_size; 123 | ctx->addr = mmap(NULL, storage_size, PROT_READ|PROT_WRITE, 124 | MAP_SHARED|MAP_ANONYMOUS, 0, 0); 125 | if (ctx->addr == MAP_FAILED) { 126 | goto free_ctx; 127 | } 128 | 129 | return ctx; 130 | 131 | free_ctx: 132 | free(ctx); 133 | err_out: 134 | return NULL; 135 | } /* nvm_open_anonymous */ 136 | 137 | void 138 | nvm_close(struct nvm_context *ctx) 139 | { 140 | if (munmap(ctx->addr, ctx->size) < 0) { 141 | RTE_LOG(DEBUG, USER2, "Error unmapping fake NVM partition at %p", 142 | ctx->addr); 143 | } 144 | free(ctx); 145 | } /* nvm_close */ 146 | 147 | unsigned long 148 | nvm_get_pagesize(void) 149 | { 150 | long pagesize; 151 | 152 | pagesize = sysconf(_SC_PAGESIZE); 153 | if (pagesize < 0) { 154 | pagesize = DEFAULT_PAGE_SIZE; 155 | } 156 | return pagesize; 157 | } /* nvm_get_pagesize */ 158 | 159 | void 160 | nvm_flush(__attribute__((unused)) struct nvm_context *ctx, 161 | void *addr, size_t length) 162 | { 163 | unsigned long mask; 164 | uintptr_t aligned_addr; 165 | uintptr_t diff; 166 | 167 | mask = nvm_get_pagesize() - 1; 168 | 169 | aligned_addr = (uintptr_t)addr; 170 | diff = aligned_addr & mask; 171 | aligned_addr = aligned_addr - diff; 172 | length = length + diff; 173 | if (msync((void *)aligned_addr, length, MS_SYNC) < 0) { 174 | RTE_LOG(ERR, USER2, "Error flushing NVM partition: mapped address %" PRIxPTR " length %zu: %s\n", 175 | aligned_addr, length, strerror(errno)); 176 | } 177 | } /* nvm_flush */ 178 | -------------------------------------------------------------------------------- /config/rdma_core.m4: -------------------------------------------------------------------------------- 1 | # Userspace Software iWARP library for DPDK 2 | # 3 | # Authors: Patrick MacArthur 4 | # 5 | # Copyright (c) 2018, University of New Hampshire 6 | # 7 | # This software is available to you under a choice of one of two 8 | # licenses. You may choose to be licensed under the terms of the GNU 9 | # General Public License (GPL) Version 2, available from the file 10 | # COPYING in the main directory of this source tree, or the 11 | # BSD license below: 12 | # 13 | # Redistribution and use in source and binary forms, with or 14 | # without modification, are permitted provided that the following 15 | # conditions are met: 16 | # 17 | # - Redistributions of source code must retain the above copyright notice, 18 | # this list of conditions and the following disclaimer. 19 | # 20 | # - Redistributions in binary form must reproduce the above copyright 21 | # notice, this list of conditions and the following disclaimer in the 22 | # documentation and/or other materials provided with the distribution. 23 | # 24 | # - Neither the name of IBM nor the names of its contributors may be 25 | # used to endorse or promote products derived from this software without 26 | # specific prior written permission. 27 | # 28 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 29 | # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 30 | # MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 31 | # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 32 | # BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 33 | # ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 34 | # CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 35 | # SOFTWARE. 36 | 37 | # _URDMA_FUNC_VERBS_INIT_AND_ALLOC_CONTEXT() 38 | # ----------------------------------------- 39 | # Checks how many arguments the verbs_init_and_alloc_context macro 40 | # takes. Defines HAVE_VERBS_INIT_AND_ALLOC_CONTEXT to the number of 41 | # arguments that the macro takes if it exists; otherwise, aborts. 42 | AC_DEFUN([_URDMA_FUNC_VERBS_INIT_AND_ALLOC_CONTEXT], 43 | [ 44 | AC_CACHE_CHECK([number of arguments verbs_init_and_alloc_context takes], 45 | [dpdk_cv_func_verbs_init_and_alloc_context], 46 | [old_CPPFLAGS="${CPPFLAGS}" 47 | CPPFLAGS="-I${srcdir} -I${srcdir}/rdma-core/${ibverbs_pabi_version} ${CPPFLAGS}" 48 | AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include ]], 49 | [verbs_init_and_alloc_context@{:@@:}@;])], 50 | [[# No-argument case is invalid and means we didn't find a prototype]] 51 | [dpdk_cv_func_verbs_init_and_alloc_context=no], 52 | [AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include ]], 53 | [[struct foo { struct verbs_context bar; } *baz; verbs_init_and_alloc_context@{:@NULL, NULL, baz, bar, 0@:}@;]])], 54 | [dpdk_cv_func_verbs_init_and_alloc_context=5], 55 | [AC_LINK_IFELSE([AC_LANG_PROGRAM([[#include ]], 56 | [[struct foo { struct verbs_context bar; } *baz; verbs_init_and_alloc_context@{:@NULL, NULL, baz, bar@:}@;]])], 57 | [dpdk_cv_func_verbs_init_and_alloc_context=4], 58 | [dpdk_cv_func_verbs_init_and_alloc_context=no])])])]) 59 | if test "x$dpdk_cv_func_verbs_init_and_alloc_context" = "xno"; then 60 | AC_MSG_ERROR([Colud not determine; check your rdma-core installation]) 61 | else 62 | AC_DEFINE_UNQUOTED([HAVE_VERBS_INIT_AND_ALLOC_CONTEXT], 63 | [${dpdk_cv_func_verbs_init_and_alloc_context}], 64 | [Define to number of arguments taken by verbs_init_and_alloc_context]) 65 | CPPFLAGS="${old_CPPFLAGS}" 66 | fi 67 | ]) # _URDMA_FUNC_VERBS_INIT_AND_ALLOC_CONTEXT 68 | 69 | # URDMA_LIB_IBVERBS_PROV([MIN_VERSION], [MAX_VERSION]) 70 | # ------------------------- 71 | # Checks that we have rdma-core, and checks that the version is within 72 | # the supported range. 73 | # Note that this macro will abort if a suitable libibverbs is 74 | # not found; fixing this would take more effort than it is worth. 75 | # Defines the C preprocessor symbol IBVERBS_PABI_VERSION to the version 76 | # of rdma-core that was found. 77 | AC_DEFUN([URDMA_LIB_IBVERBS_PROV], 78 | [ 79 | ibverbs_pabi_version= 80 | min=$1 81 | max=$2 82 | for pabi_version in `seq "${max}" -1 "${min}"` 83 | do 84 | AC_SEARCH_LIBS([verbs_register_driver_${pabi_version}], 85 | [ibverbs], 86 | [ibverbs_pabi_version=${pabi_version}; break], []) 87 | done 88 | 89 | if test "x$ibverbs_pabi_version" = x; then 90 | AC_MSG_ERROR([urdma requires rdma-core >= MIN_VERSION]) 91 | fi 92 | 93 | AC_DEFINE_UNQUOTED([IBVERBS_PABI_VERSION], [${ibverbs_pabi_version}], 94 | [Define to rdma-core private ABI version]) 95 | AC_SUBST([ibverbs_pabi_version]) 96 | 97 | if test $ibverbs_pabi_version -ge 17; then 98 | _URDMA_FUNC_VERBS_INIT_AND_ALLOC_CONTEXT 99 | fi 100 | 101 | IBV_DEVICE_LIBRARY_EXTENSION=rdmav${ibverbs_pabi_version} 102 | AC_SUBST([IBV_DEVICE_LIBRARY_EXTENSION]) 103 | ]) # URDMA_LIB_IBVERBS_PROV 104 | -------------------------------------------------------------------------------- /ccan/container_of/container_of.h: -------------------------------------------------------------------------------- 1 | /* CC0 (Public domain) - see LICENSE file for details */ 2 | #ifndef CCAN_CONTAINER_OF_H 3 | #define CCAN_CONTAINER_OF_H 4 | #include 5 | 6 | #ifdef HAVE_CONFIG_H 7 | #include "config.h" 8 | #endif 9 | #include 10 | 11 | /** 12 | * container_of - get pointer to enclosing structure 13 | * @member_ptr: pointer to the structure member 14 | * @containing_type: the type this member is within 15 | * @member: the name of this member within the structure. 16 | * 17 | * Given a pointer to a member of a structure, this macro does pointer 18 | * subtraction to return the pointer to the enclosing type. 19 | * 20 | * Example: 21 | * struct foo { 22 | * int fielda, fieldb; 23 | * // ... 24 | * }; 25 | * struct info { 26 | * int some_other_field; 27 | * struct foo my_foo; 28 | * }; 29 | * 30 | * static struct info *foo_to_info(struct foo *foo) 31 | * { 32 | * return container_of(foo, struct info, my_foo); 33 | * } 34 | */ 35 | #define container_of(member_ptr, containing_type, member) \ 36 | ((containing_type *) \ 37 | ((char *)(member_ptr) \ 38 | - container_off(containing_type, member)) \ 39 | + check_types_match(*(member_ptr), ((containing_type *)0)->member)) 40 | 41 | 42 | /** 43 | * container_of_or_null - get pointer to enclosing structure, or NULL 44 | * @member_ptr: pointer to the structure member 45 | * @containing_type: the type this member is within 46 | * @member: the name of this member within the structure. 47 | * 48 | * Given a pointer to a member of a structure, this macro does pointer 49 | * subtraction to return the pointer to the enclosing type, unless it 50 | * is given NULL, in which case it also returns NULL. 51 | * 52 | * Example: 53 | * struct foo { 54 | * int fielda, fieldb; 55 | * // ... 56 | * }; 57 | * struct info { 58 | * int some_other_field; 59 | * struct foo my_foo; 60 | * }; 61 | * 62 | * static struct info *foo_to_info_allowing_null(struct foo *foo) 63 | * { 64 | * return container_of_or_null(foo, struct info, my_foo); 65 | * } 66 | */ 67 | static inline char *container_of_or_null_(void *member_ptr, size_t offset) 68 | { 69 | return member_ptr ? (char *)member_ptr - offset : NULL; 70 | } 71 | #define container_of_or_null(member_ptr, containing_type, member) \ 72 | ((containing_type *) \ 73 | container_of_or_null_(member_ptr, \ 74 | container_off(containing_type, member)) \ 75 | + check_types_match(*(member_ptr), ((containing_type *)0)->member)) 76 | 77 | /** 78 | * container_off - get offset to enclosing structure 79 | * @containing_type: the type this member is within 80 | * @member: the name of this member within the structure. 81 | * 82 | * Given a pointer to a member of a structure, this macro does 83 | * typechecking and figures out the offset to the enclosing type. 84 | * 85 | * Example: 86 | * struct foo { 87 | * int fielda, fieldb; 88 | * // ... 89 | * }; 90 | * struct info { 91 | * int some_other_field; 92 | * struct foo my_foo; 93 | * }; 94 | * 95 | * static struct info *foo_to_info(struct foo *foo) 96 | * { 97 | * size_t off = container_off(struct info, my_foo); 98 | * return (void *)((char *)foo - off); 99 | * } 100 | */ 101 | #define container_off(containing_type, member) \ 102 | offsetof(containing_type, member) 103 | 104 | /** 105 | * container_of_var - get pointer to enclosing structure using a variable 106 | * @member_ptr: pointer to the structure member 107 | * @container_var: a pointer of same type as this member's container 108 | * @member: the name of this member within the structure. 109 | * 110 | * Given a pointer to a member of a structure, this macro does pointer 111 | * subtraction to return the pointer to the enclosing type. 112 | * 113 | * Example: 114 | * static struct info *foo_to_i(struct foo *foo) 115 | * { 116 | * struct info *i = container_of_var(foo, i, my_foo); 117 | * return i; 118 | * } 119 | */ 120 | #if HAVE_TYPEOF 121 | #define container_of_var(member_ptr, container_var, member) \ 122 | container_of(member_ptr, typeof(*container_var), member) 123 | #else 124 | #define container_of_var(member_ptr, container_var, member) \ 125 | ((void *)((char *)(member_ptr) - \ 126 | container_off_var(container_var, member))) 127 | #endif 128 | 129 | /** 130 | * container_off_var - get offset of a field in enclosing structure 131 | * @container_var: a pointer to a container structure 132 | * @member: the name of a member within the structure. 133 | * 134 | * Given (any) pointer to a structure and a its member name, this 135 | * macro does pointer subtraction to return offset of member in a 136 | * structure memory layout. 137 | * 138 | */ 139 | #if HAVE_TYPEOF 140 | #define container_off_var(var, member) \ 141 | container_off(typeof(*var), member) 142 | #else 143 | #define container_off_var(var, member) \ 144 | ((const char *)&(var)->member - (const char *)(var)) 145 | #endif 146 | 147 | #endif /* CCAN_CONTAINER_OF_H */ 148 | -------------------------------------------------------------------------------- /include/kvstore_storage.h: -------------------------------------------------------------------------------- 1 | /** @file kvstore_storage.h 2 | * 3 | * The overall layout is as follows: 4 | * 5 | * HEADER (PAGESIZE bytes, or 4096 if PAGESIZE and _SC_PAGESIZE not defined) 6 | * BUCKETS (main_bucket_count * sizeof(struct store_bucket) bytes) 7 | * BITMASK (slot_count / 8 bytes) 8 | * DATA (slot_count * KVSTORE_VALUE_LEN_MAX) 9 | */ 10 | 11 | /* 12 | * Userspace Software iWARP library for DPDK 13 | * 14 | * Authors: Patrick MacArthur 15 | * 16 | * Copyright (c) 2016, IBM Corporation 17 | * 18 | * This software is available to you under a choice of one of two 19 | * licenses. You may choose to be licensed under the terms of the GNU 20 | * General Public License (GPL) Version 2, available from the file 21 | * COPYING in the main directory of this source tree, or the 22 | * BSD license below: 23 | * 24 | * Redistribution and use in source and binary forms, with or 25 | * without modification, are permitted provided that the following 26 | * conditions are met: 27 | * 28 | * - Redistributions of source code must retain the above copyright notice, 29 | * this list of conditions and the following disclaimer. 30 | * 31 | * - Redistributions in binary form must reproduce the above copyright 32 | * notice, this list of conditions and the following disclaimer in the 33 | * documentation and/or other materials provided with the distribution. 34 | * 35 | * - Neither the name of IBM nor the names of its contributors may be 36 | * used to endorse or promote products derived from this software without 37 | * specific prior written permission. 38 | * 39 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 40 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 41 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 42 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 43 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 44 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 45 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 46 | * SOFTWARE. 47 | */ 48 | 49 | /* These are the persistent data structures. */ 50 | 51 | #ifndef KVSTORE_STORAGE_H 52 | #define KVSTORE_STORAGE_H 53 | 54 | #include "kvstore_limits.h" 55 | 56 | enum { ENTRIES_PER_BUCKET = 8 }; 57 | 58 | enum { KVSTORE_MAGIC = UINT16_C(0x1B4D) }; 59 | 60 | /** The format of the header stored in the first page of the file. */ 61 | struct store_header { 62 | uint16_t magic; 63 | /**< 1B4D, in machine endian format */ 64 | uint16_t version; 65 | /**< version of the header format (0x00) */ 66 | uint32_t main_bucket_count; 67 | /**< Total number of buckets in hash table. */ 68 | uint32_t page_size; 69 | /**< Alignment of each section. */ 70 | uint32_t reserved_12; 71 | /**< Reserved for future use; MUST be set to 0. */ 72 | uint64_t slot_count; 73 | /**< Maximum number of data items that can be stored. */ 74 | uint64_t value_max_size; 75 | /**< Maximum size of a value in the store. Must be less than 76 | * or equal to KVSTORE_VALUE_LEN_MAX. */ 77 | }; 78 | 79 | /** The format of each entry within a bucket. */ 80 | struct store_bucket_entry { 81 | uint64_t offset; 82 | /**< The offset of the entry, in bytes, from the start of the 83 | * data section. */ 84 | uint64_t cas_version; 85 | /**< The version of this item, used for CAS operations. */ 86 | uint32_t value_size; 87 | /**< The length of the current value. */ 88 | char key[KVSTORE_KEY_LEN_MAX]; 89 | /**< The key, which is an ASCII string which may not contain 90 | * null characters. If the key is less than the maximum size, 91 | * the entry will be padded with null characters. If the key 92 | * is the maximum size, there will be no terminating null 93 | * character. 94 | * 95 | * If the first byte of the key is a null character, than the 96 | * bucket is not in use. */ 97 | }; 98 | 99 | /** The store bucket. The bucket consists of a small number entries, to allow 100 | * for quick lookup even with hash collisions. */ 101 | struct store_bucket { 102 | uint32_t version; 103 | /**< Currently unused. */ 104 | struct store_bucket_entry entries[ENTRIES_PER_BUCKET]; 105 | /**< An array of entries. */ 106 | }; 107 | 108 | /** Given an initial offset, return the offset of the main bucket array. */ 109 | static inline uintptr_t 110 | kvstore_main_bucket_offset(uintptr_t start, struct store_header *header) 111 | { 112 | return start + header->page_size; 113 | } /* kvstore_main_bucket_offset */ 114 | 115 | /** Given the main bucket offset, return the bitmask offset. */ 116 | static inline uintptr_t 117 | kvstore_bitmask_offset(uintptr_t mb_offset, struct store_header *header) 118 | { 119 | return RTE_ALIGN_CEIL(mb_offset + header->main_bucket_count 120 | * sizeof(struct store_bucket), header->page_size); 121 | } /* kvstore_bitmask_offset */ 122 | 123 | /** Given the bitmask offset, return the slot offset. */ 124 | static inline uintptr_t 125 | kvstore_slot_offset(uintptr_t bitmask_offset, struct store_header *header) 126 | { 127 | return RTE_ALIGN_CEIL(bitmask_offset + header->slot_count / 8, 128 | header->page_size); 129 | } /* kvstore_slot_offset */ 130 | 131 | #endif 132 | -------------------------------------------------------------------------------- /doc/implementation.txt: -------------------------------------------------------------------------------- 1 | Queue Pairs and Completion Queues 2 | --------------------------------- 3 | 4 | The queue consists of a ring, a hash table, and a linked list. The ring 5 | contains all queued but inactive operations, while the hash table contains all 6 | active operations. The list contains all elements inserted into the hash table 7 | for quick iteration. Note that a tree structure would be more memory-efficient 8 | but DPDK doesn't provide one, so we would have to write our own. 9 | 10 | The storage for all queue entries is kept in an array at the tail of the data 11 | structure. Attached to it is a bitmask. The bits are 0 if the queue entry is 12 | currently in use (in the ring or in the hash table), or 1 if the queue entry is 13 | free. We use our own bitmap implementation instead of the one provided by DPDK 14 | since ours is more optimized for smaller queue sizes, as it does not do an 15 | indirect lookup (which DPDK's does for cache efficiency). 16 | 17 | Completion Queues 18 | ----------------- 19 | 20 | comp_vector currently has one entry for each socket on the system and is 21 | (ab)used to allocate the CQEs on the closest memory bank for NUMA purposes. 22 | 23 | Verbs/Kernel Interaction 24 | ------------------------ 25 | 26 | Our verbs kernel driver creates a fixed number of uverbs devices which are 27 | initially not associated with any net device and have a random GUID. This 28 | is for two reasons: 29 | 30 | (1) The uverbs device must exist before any verbs function is called, or verbs 31 | will not ever call our driver initialization function. 32 | 33 | (2) DPDK functions, including rte_eal_init(), may not be called before main() 34 | is called, because they rely on their gcc constructors having been called 35 | first. There is no guaranteed ordering here, so we must defer any 36 | initialization that we do until after main() is called. 37 | 38 | Our driver initialization function configures and starts all DPDK ports, and 39 | configures and starts a KNI interface per DPDK port. When the kernel driver is 40 | notified of the netdev registration event for an interface named "kni", it 41 | will associate the interface with the corresponding verbs device, and change 42 | its GUID to match the MAC address of the corresponding NIC. This ensures that 43 | by the time that any user verbs code is run, traffic may be passed on these 44 | interfaces and will reach the kernel, which is needed for ARP and connection 45 | management to work. 46 | 47 | Connection management is done in the kernel driver, since the rdma_cm is in the 48 | kernel. Once a connection is established, the flow director rule will be set 49 | up so the kernel will no longer have any visibility into the traffic for that 50 | connection and normal traffic on that queue pair will be entirely managed in 51 | user space. The only other thing that the kernel driver will be responsible 52 | for is handling the disconnect event from rdma_cm, which userspace is notified 53 | of via the RTS->SQD state transition. 54 | 55 | Event File 56 | ---------- 57 | 58 | The completion channel in verbs cannot be signaled from userspace. To get 59 | around this limitation, the kernel verbs driver creates an event file which is 60 | used by the userspace verbs library to communicate when the completion channel 61 | must be notified of a completion event. This is returned in the 'event_fd' 62 | field of the alloc_ucontext() udata response. 63 | 64 | This fd is opened in non-blocking mode so that it can be used by the progress 65 | thread, but there should be no reason why the file could not also be used in 66 | blocking mode. 67 | 68 | There are currently 2 events passed from userspace to kernel space and 1 event 69 | passed from kernel space to userspace over this file: 70 | 71 | - SIW_EVENT_COMP_POSTED: supplied via a write() from userspace. Indicates 72 | that the kernel should signal the completion channel associated with the 73 | given completion queue 74 | 75 | - SIW_EVENT_QP_CONNECTED: supplied by the kernel and picked up via a read() 76 | from userspace. Givves the source and destination addresses as well as 77 | other connection parameters, such as the negotiated ORD/IRD values used to 78 | determine the maximum number of simultaneous RDMA READ operations. 79 | 80 | - SIW_EVENT_QP_RTR: supplied via a write() from userspace once it has 81 | processed the information in the SIW_EVENT_QP_CONNECTED event and is ready 82 | to receive data on the connection. The ESTABLISHED CM event must not be 83 | delivered by the kernel CM until the kernel has received this event. 84 | 85 | Incoming Messages 86 | ----------------- 87 | 88 | We use flow director to direct each incoming message to the appropriate queue 89 | pair based on its destination UDP port. 90 | 91 | For tagged messages, we immediately place the data based on its tag 92 | information. For RDMA READ Response messages, we then associate the message 93 | with the RDMA READ operation to update the progress and notify the user when 94 | the READ has completed. 95 | 96 | For untagged SEND messages, we must associate the incoming message with a 97 | receive immediately, which we do using the receive queue hash table and the 98 | message sequence number. Because we are built on top of UDP, which is 99 | unreliable, we must keep track of any messages which we did not receive. To do 100 | this, we keep a small array of ranges which indicates which byte ranges of the 101 | message we have received. We automatically collapse ranges when they become 102 | contiguous (due to receiving the intervening message). This allows us to deal 103 | with a small random distribution of missing messages reasonably efficiently. 104 | -------------------------------------------------------------------------------- /src/kmod/cm.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Software iWARP device driver for Linux 3 | * 4 | * Authors: Bernard Metzler 5 | * Patrick MacArthur 6 | * 7 | * Copyright (c) 2008-2016, IBM Corporation 8 | * Copyright (c) 2016, University of New Hampshire 9 | * 10 | * This software is available to you under a choice of one of two 11 | * licenses. You may choose to be licensed under the terms of the GNU 12 | * General Public License (GPL) Version 2, available from the file 13 | * COPYING in the main directory of this source tree, or the 14 | * BSD license below: 15 | * 16 | * Redistribution and use in source and binary forms, with or 17 | * without modification, are permitted provided that the following 18 | * conditions are met: 19 | * 20 | * - Redistributions of source code must retain the above copyright notice, 21 | * this list of conditions and the following disclaimer. 22 | * 23 | * - Redistributions in binary form must reproduce the above copyright 24 | * notice, this list of conditions and the following disclaimer in the 25 | * documentation and/or other materials provided with the distribution. 26 | * 27 | * - Neither the name of IBM nor the names of its contributors may be 28 | * used to endorse or promote products derived from this software without 29 | * specific prior written permission. 30 | * 31 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | * SOFTWARE. 39 | */ 40 | 41 | #ifndef _SIW_CM_H 42 | #define _SIW_CM_H 43 | 44 | #include 45 | #include 46 | 47 | #include 48 | 49 | #include "proto_trp.h" 50 | 51 | #define URDMA_PDATA_LEN_MAX 512 52 | 53 | enum siw_cep_state { 54 | SIW_EPSTATE_IDLE = 1, 55 | SIW_EPSTATE_LISTENING, 56 | SIW_EPSTATE_CONNECTING, 57 | SIW_EPSTATE_AWAIT_MPAREQ, 58 | SIW_EPSTATE_RECVD_MPAREQ, 59 | SIW_EPSTATE_AWAIT_MPAREP, 60 | SIW_EPSTATE_RECVD_MPAREP, 61 | SIW_EPSTATE_ACCEPTING, 62 | SIW_EPSTATE_RDMA_MODE, 63 | SIW_EPSTATE_CLOSED 64 | }; 65 | 66 | struct siw_mpa_info { 67 | struct trp_rr hdr; /* peer mpa hdr in host byte order */ 68 | char pdata[URDMA_PDATA_LEN_MAX]; 69 | /* private data, plus up to four pad bytes */ 70 | int bytes_rcvd; 71 | char *send_pdata; 72 | int send_pdata_size; 73 | }; 74 | 75 | struct siw_llp_info { 76 | struct socket *sock; 77 | struct sockaddr_in laddr; /* redundant with socket info above */ 78 | struct sockaddr_in raddr; /* dito, consider removal */ 79 | struct siw_sk_upcalls sk_def_upcalls; 80 | }; 81 | 82 | struct siw_dev; 83 | 84 | struct siw_cep { 85 | struct iw_cm_id *cm_id; 86 | struct siw_dev *sdev; 87 | 88 | struct list_head devq; 89 | /* 90 | * The provider_data element of a listener IWCM ID 91 | * refers to a list of one or more listener CEPs 92 | */ 93 | struct list_head listenq; 94 | 95 | /* The list of connections that have been established but we have not 96 | * yet notified userspace. */ 97 | struct list_head established_entry; 98 | 99 | /* The list of connections that have been established, and userspace 100 | * has been notified, but has not yet informed us that they have 101 | * finished setup. */ 102 | struct list_head rtr_wait_entry; 103 | 104 | struct siw_cep *listen_cep; 105 | struct siw_qp *qp; 106 | spinlock_t lock; 107 | wait_queue_head_t waitq; 108 | struct kref ref; 109 | enum siw_cep_state state; 110 | short in_use; 111 | struct siw_cm_work *timer; 112 | struct list_head work_freelist; 113 | struct siw_llp_info llp; 114 | struct siw_mpa_info mpa; 115 | uint16_t urdmad_dev_id; 116 | uint16_t urdmad_qp_id; 117 | uint16_t ord; 118 | uint16_t ird; 119 | int sk_error; /* not (yet) used XXX */ 120 | }; 121 | 122 | #define MPAREQ_TIMEOUT (HZ*10) 123 | #define MPAREP_TIMEOUT (HZ*5) 124 | 125 | enum siw_work_type { 126 | SIW_CM_WORK_ACCEPT = 1, 127 | SIW_CM_WORK_READ_MPAHDR, 128 | SIW_CM_WORK_CLOSE_LLP, /* close socket */ 129 | SIW_CM_WORK_PEER_CLOSE, /* socket indicated peer close */ 130 | SIW_CM_WORK_TIMEOUT 131 | }; 132 | 133 | struct siw_cm_work { 134 | struct delayed_work work; 135 | struct list_head list; 136 | enum siw_work_type type; 137 | struct siw_cep *cep; 138 | }; 139 | 140 | /* 141 | * With kernel 3.12, OFA ddressing changed from sockaddr_in to 142 | * sockaddr_storage 143 | */ 144 | #define to_sockaddr_in(a) (*(struct sockaddr_in *)(&(a))) 145 | 146 | extern int siw_connect(struct iw_cm_id *, struct iw_cm_conn_param *); 147 | extern int siw_accept(struct iw_cm_id *, struct iw_cm_conn_param *); 148 | extern int siw_reject(struct iw_cm_id *, const void *, u8); 149 | extern int siw_create_listen(struct iw_cm_id *, int); 150 | extern int siw_destroy_listen(struct iw_cm_id *); 151 | 152 | extern void siw_cep_get(struct siw_cep *); 153 | extern void siw_cep_put(struct siw_cep *); 154 | extern int siw_qp_rtr_fail(struct siw_cep *); 155 | extern int siw_qp_rtr(struct siw_cep *); 156 | extern int siw_cm_queue_work(struct siw_cep *, enum siw_work_type); 157 | 158 | extern int siw_cm_init(void); 159 | extern void siw_cm_exit(void); 160 | 161 | /* 162 | * TCP socket interface 163 | */ 164 | #define sk_to_qp(sk) (((struct siw_cep *)((sk)->sk_user_data))->qp) 165 | #define sk_to_cep(sk) ((struct siw_cep *)((sk)->sk_user_data)) 166 | 167 | #endif 168 | -------------------------------------------------------------------------------- /src/kmod/verbs.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Software iWARP device driver for Linux 3 | * 4 | * Authors: Bernard Metzler 5 | * Patrick MacArthur 6 | * 7 | * Copyright (c) 2008-2016, IBM Corporation 8 | * Copyright (c) 2016, University of New Hampshire 9 | * 10 | * This software is available to you under a choice of one of two 11 | * licenses. You may choose to be licensed under the terms of the GNU 12 | * General Public License (GPL) Version 2, available from the file 13 | * COPYING in the main directory of this source tree, or the 14 | * BSD license below: 15 | * 16 | * Redistribution and use in source and binary forms, with or 17 | * without modification, are permitted provided that the following 18 | * conditions are met: 19 | * 20 | * - Redistributions of source code must retain the above copyright notice, 21 | * this list of conditions and the following disclaimer. 22 | * 23 | * - Redistributions in binary form must reproduce the above copyright 24 | * notice, this list of conditions and the following disclaimer in the 25 | * documentation and/or other materials provided with the distribution. 26 | * 27 | * - Neither the name of IBM nor the names of its contributors may be 28 | * used to endorse or promote products derived from this software without 29 | * specific prior written permission. 30 | * 31 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | * SOFTWARE. 39 | */ 40 | 41 | #ifndef _SIW_VERBS_H 42 | #define _SIW_VERBS_H 43 | 44 | #include 45 | 46 | #include 47 | #include 48 | #include 49 | #include 50 | 51 | #include "urdma.h" 52 | #include "cm.h" 53 | 54 | #ifndef HAVE_IB_QUERY_DEVICE_UDATA 55 | extern int siw_query_device(struct ib_device *, struct ib_device_attr *); 56 | #else 57 | extern int siw_query_device(struct ib_device *, struct ib_device_attr *, 58 | struct ib_udata *); 59 | #endif 60 | 61 | extern void urdma_disassociate_ucontext(struct ib_ucontext *ctx); 62 | extern struct ib_ucontext *siw_alloc_ucontext(struct ib_device *, 63 | struct ib_udata *); 64 | extern int siw_dealloc_ucontext(struct ib_ucontext *); 65 | extern int siw_query_port(struct ib_device *, u8, struct ib_port_attr *); 66 | extern int siw_query_pkey(struct ib_device *, u8, u16, u16 *); 67 | extern int siw_query_gid(struct ib_device *, u8, int, union ib_gid *); 68 | 69 | extern struct ib_pd *siw_alloc_pd(struct ib_device *, struct ib_ucontext *, 70 | struct ib_udata *); 71 | extern int siw_dealloc_pd(struct ib_pd *); 72 | #ifdef HAVE_STRUCT_RDMA_AH_ATTR 73 | extern struct ib_ah *siw_create_ah(struct ib_pd *, struct rdma_ah_attr *, 74 | struct ib_udata *data); 75 | #else 76 | extern struct ib_ah *siw_create_ah(struct ib_pd *, struct ib_ah_attr * 77 | #ifdef HAVE_CREATE_AH_UDATA 78 | , struct ib_udata *data 79 | #endif 80 | ); 81 | #endif 82 | extern int siw_destroy_ah(struct ib_ah *); 83 | extern struct ib_qp *siw_create_qp(struct ib_pd *, struct ib_qp_init_attr *, 84 | struct ib_udata *); 85 | extern int siw_query_qp(struct ib_qp *, struct ib_qp_attr *, int, 86 | struct ib_qp_init_attr *); 87 | extern int siw_ofed_modify_qp(struct ib_qp *, struct ib_qp_attr *, int, 88 | struct ib_udata *); 89 | extern int siw_destroy_qp(struct ib_qp *); 90 | extern int siw_post_send(struct ib_qp *, struct ib_send_wr *, 91 | struct ib_send_wr **); 92 | extern int siw_post_receive(struct ib_qp *, struct ib_recv_wr *, 93 | struct ib_recv_wr **); 94 | #ifndef HAVE_STRUCT_IB_CQ_INIT_ATTR 95 | extern struct ib_cq *siw_create_cq(struct ib_device *, int, int, 96 | struct ib_ucontext *, struct ib_udata *); 97 | #else 98 | extern struct ib_cq *siw_create_cq(struct ib_device *, 99 | const struct ib_cq_init_attr *, 100 | struct ib_ucontext *, struct ib_udata *); 101 | #endif 102 | extern int siw_destroy_cq(struct ib_cq *); 103 | extern int siw_poll_cq(struct ib_cq *, int num_entries, struct ib_wc *); 104 | extern int siw_req_notify_cq(struct ib_cq *, enum ib_cq_notify_flags); 105 | extern struct ib_mr *siw_reg_user_mr(struct ib_pd *, u64, u64, u64, int, 106 | struct ib_udata *); 107 | extern struct ib_mr *siw_get_dma_mr(struct ib_pd *, int); 108 | extern int siw_dereg_mr(struct ib_mr *); 109 | extern struct ib_srq *siw_create_srq(struct ib_pd *, struct ib_srq_init_attr *, 110 | struct ib_udata *); 111 | extern int siw_modify_srq(struct ib_srq *, struct ib_srq_attr *, 112 | enum ib_srq_attr_mask, struct ib_udata *); 113 | extern int siw_query_srq(struct ib_srq *, struct ib_srq_attr *); 114 | extern int siw_destroy_srq(struct ib_srq *); 115 | extern int siw_post_srq_recv(struct ib_srq *, struct ib_recv_wr *, 116 | struct ib_recv_wr **); 117 | 118 | #ifndef HAVE_IB_PROCESS_MAD_SIZES 119 | extern int siw_no_mad(struct ib_device *, int, u8, struct ib_wc *, struct ib_grh *, 120 | struct ib_mad *, struct ib_mad *); 121 | #else 122 | extern int siw_no_mad(struct ib_device *ofa_dev, int flags, u8 port, 123 | const struct ib_wc *wc, const struct ib_grh *grh, 124 | const struct ib_mad_hdr *in_mad, size_t in_mad_size, 125 | struct ib_mad_hdr *out_mad, size_t *out_mad_size, 126 | u16 *out_mad_pkey_index); 127 | #endif 128 | 129 | #ifdef HAVE_IB_GET_PORT_IMMUTABLE 130 | extern int urdma_port_immutable(struct ib_device *ibdev, u8 port_num, 131 | struct ib_port_immutable *immutable); 132 | #endif 133 | 134 | #endif 135 | -------------------------------------------------------------------------------- /include/urdmad_private.h: -------------------------------------------------------------------------------- 1 | /* include/urdmad_private.h */ 2 | 3 | #ifndef URDMAD_PRIVATE_H 4 | #define URDMAD_PRIVATE_H 5 | 6 | /* 7 | * Userspace Software iWARP library for DPDK 8 | * 9 | * Internal socket protocol and ABI structures for urdmad 10 | * 11 | * Author: Patrick MacArthur 12 | * 13 | * Copyright (c) 2016, University of New Hampshire 14 | * 15 | * This software is available to you under a choice of one of two 16 | * licenses. You may choose to be licensed under the terms of the GNU 17 | * General Public License (GPL) Version 2, available from the file 18 | * COPYING in the main directory of this source tree, or the 19 | * BSD license below: 20 | * 21 | * Redistribution and use in source and binary forms, with or 22 | * without modification, are permitted provided that the following 23 | * conditions are met: 24 | * 25 | * - Redistributions of source code must retain the above copyright notice, 26 | * this list of conditions and the following disclaimer. 27 | * 28 | * - Redistributions in binary form must reproduce the above copyright 29 | * notice, this list of conditions and the following disclaimer in the 30 | * documentation and/or other materials provided with the distribution. 31 | * 32 | * - Neither the names of IBM, UNH, nor the names of its contributors may be 33 | * used to endorse or promote products derived from this software without 34 | * specific prior written permission. 35 | * 36 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 37 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 38 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 39 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 40 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 41 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 42 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 43 | * SOFTWARE. 44 | */ 45 | 46 | #include 47 | #include 48 | #include 49 | #include 50 | 51 | #include 52 | 53 | #define URDMA_SOCK_PROTO_VERSION 1 54 | 55 | /** Internal state machine of the queue pair. */ 56 | enum urdma_qp_state { 57 | usiw_qp_unbound = 0, 58 | /**< Queue pair not yet bound to remote endpoint. */ 59 | usiw_qp_connected = 1, 60 | /**< Connection established by kernel CM. */ 61 | usiw_qp_running = 2, 62 | /**< Userspace QP is connected to remote endpoint. */ 63 | usiw_qp_shutdown = 3, 64 | /**< Shutdown has been triggered by RDMA CM. */ 65 | usiw_qp_error = 4, 66 | /**< Queue pair has been invalidated. */ 67 | }; 68 | 69 | /** Fields of the queue pair that must be accessible from urdmad and verbs 70 | * processes. */ 71 | struct urdmad_qp { 72 | atomic_uint conn_state; 73 | /**< The state of the queue pair's connection. */ 74 | uint8_t ord_max; 75 | /**< Negotiated maximum number of outstanding RDMA READ 76 | * requests to the remote endpoint. */ 77 | uint8_t ird_max; 78 | /**< Negotiated maximum number of incoming RDMA READ 79 | * requests. */ 80 | pthread_mutex_t conn_event_lock; 81 | /**< Protects conn_state and related variables. */ 82 | 83 | uint16_t dev_id; 84 | /**< DPDK device/port index. */ 85 | uint16_t qp_id; 86 | /**< Index into urdmad queue pair array. */ 87 | uint16_t rx_queue; 88 | /**< Hardware receive queue to use for this queue pair. */ 89 | uint16_t tx_queue; 90 | /**< Hardware transmit queue to use for this queue pair. */ 91 | uint16_t local_udp_port; 92 | /**< UDP port assigned to local endpoint. */ 93 | uint16_t remote_udp_port; 94 | /**< UDP port assigned to remote endpoint. */ 95 | uint32_t local_ipv4_addr; 96 | /**< IPv4 address of local endpoint. */ 97 | uint32_t remote_ipv4_addr; 98 | /**< IPv4 address of remote endpoint. */ 99 | struct ether_addr remote_ether_addr; 100 | /**< MAC address of remote endpoint. */ 101 | 102 | uint16_t rx_desc_count; 103 | /**< Hardware receive descriptors on this RX queue. */ 104 | uint16_t tx_desc_count; 105 | /**< Hardware receive descriptors on this TX queue. */ 106 | uint16_t rx_burst_size; 107 | /**< Size of array passed to rte_eth_rx_burst(). */ 108 | uint16_t tx_burst_size; 109 | /**< Size of array passed to rte_eth_tx_burst(). */ 110 | uint16_t mtu; 111 | /**< Device MTU. */ 112 | 113 | struct list_node urdmad__entry; 114 | /**< Private field used only by urdmad to thread onto list. */ 115 | }; 116 | 117 | /* The messages defined for the sockets protocol. */ 118 | enum urdmad_sock_msg_op { 119 | urdma_sock_create_qp_req = 1, 120 | urdma_sock_create_qp_resp = 2, 121 | urdma_sock_destroy_qp_req = 3, 122 | urdma_sock_hello_req = 4, 123 | urdma_sock_hello_resp = 5, 124 | }; 125 | 126 | struct urdmad_sock_msg { 127 | uint32_t opcode; 128 | uint16_t dev_id; 129 | uint16_t qp_id; 130 | }; 131 | 132 | struct urdmad_sock_qp_msg { 133 | struct urdmad_sock_msg hdr; 134 | uint64_t ptr; 135 | }; 136 | 137 | struct urdmad_sock_hello_req { 138 | struct urdmad_sock_msg hdr; 139 | uint8_t proto_version; 140 | uint8_t reserved9; 141 | uint16_t req_lcore_count; 142 | }; 143 | static_assert(offsetof(struct urdmad_sock_hello_req, reserved9) == 9, 144 | "hello_req reserved9 field is at wrong offset"); 145 | 146 | struct urdmad_sock_hello_resp { 147 | struct urdmad_sock_msg hdr; 148 | uint8_t proto_version; 149 | uint8_t max_lcore; 150 | uint16_t device_count; 151 | uint32_t reserved12; 152 | uint64_t rdma_atomic_mutex_addr; 153 | uint32_t lcore_mask[RTE_MAX_LCORE / 32]; 154 | uint16_t max_qp[]; 155 | }; 156 | static_assert(offsetof(struct urdmad_sock_hello_resp, reserved12) == 12, 157 | "reserved12 field is at wrong offset"); 158 | 159 | union urdmad_sock_any_msg { 160 | struct urdmad_sock_msg hdr; 161 | struct urdmad_sock_qp_msg qp; 162 | struct urdmad_sock_hello_req hello_req; 163 | struct urdmad_sock_hello_resp hello_resp; 164 | }; 165 | 166 | #endif 167 | -------------------------------------------------------------------------------- /tests/binheap.c: -------------------------------------------------------------------------------- 1 | /* tests/binheap.c */ 2 | 3 | /* 4 | * Userspace Software iWARP library for DPDK 5 | * 6 | * Authors: Patrick MacArthur 7 | * 8 | * Copyright (c) 2017, University of New Hampshire 9 | * 10 | * This software is available to you under a choice of one of two 11 | * licenses. You may choose to be licensed under the terms of the GNU 12 | * General Public License (GPL) Version 2, available from the file 13 | * COPYING in the main directory of this source tree, or the 14 | * BSD license below: 15 | * 16 | * Redistribution and use in source and binary forms, with or 17 | * without modification, are permitted provided that the following 18 | * conditions are met: 19 | * 20 | * - Redistributions of source code must retain the above copyright notice, 21 | * this list of conditions and the following disclaimer. 22 | * 23 | * - Redistributions in binary form must reproduce the above copyright 24 | * notice, this list of conditions and the following disclaimer in the 25 | * documentation and/or other materials provided with the distribution. 26 | * 27 | * - Neither the name of IBM nor the names of its contributors may be 28 | * used to endorse or promote products derived from this software without 29 | * specific prior written permission. 30 | * 31 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 32 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 33 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 34 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 35 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 36 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 37 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 38 | * SOFTWARE. 39 | */ 40 | 41 | /* This file contains tests for our open-coded binary heap code */ 42 | 43 | #include "binheap.h" 44 | #include 45 | #include 46 | #include 47 | #include 48 | #include 49 | 50 | static const uint32_t magic = 0xA1C; 51 | 52 | #define array_size(arr) (sizeof(arr) / sizeof(*arr)) 53 | 54 | #define FAIL(format, ...) \ 55 | do { \ 56 | do_fail("%s: " format, __func__, ##__VA_ARGS__); \ 57 | } while (0); 58 | 59 | static void do_fail(const char *format, ...) 60 | { 61 | va_list va; 62 | 63 | va_start(va, format); 64 | vfprintf(stderr, format, va); 65 | va_end(va); 66 | 67 | exit(EXIT_FAILURE); 68 | } 69 | 70 | static void test_peek_empty(void) 71 | { 72 | struct binheap *h = binheap_new(1); 73 | uint32_t v; 74 | int ret; 75 | 76 | v = magic; 77 | ret = binheap_peek(h, &v); 78 | if (!ret) { 79 | FAIL("binheap_peek returned %d on empty heap\n", ret); 80 | } 81 | if (v != magic) { 82 | FAIL("binheap_peek set v=%d expected unchanged\n", v); 83 | } 84 | 85 | free(h); 86 | } 87 | 88 | static void test_insert_one(void) 89 | { 90 | struct binheap *h = binheap_new(1); 91 | uint32_t v; 92 | int ret; 93 | 94 | binheap_insert(h, magic); 95 | 96 | ret = binheap_peek(h, &v); 97 | if (ret) { 98 | FAIL("binheap_peek returned %d after element inserted\n", ret); 99 | } 100 | if (v != magic) { 101 | FAIL("binheap_peek set v=%d expected %d\n", v, magic); 102 | } 103 | 104 | ret = binheap_pop(h); 105 | if (ret != 0) { 106 | FAIL("binheap_pop returned %d expected 0\n", v); 107 | } 108 | 109 | free(h); 110 | } 111 | 112 | static void test_insert_inorder(void) 113 | { 114 | struct binheap *h = binheap_new(8); 115 | uint32_t i, v; 116 | 117 | binheap_insert(h, 1); 118 | binheap_insert(h, 2); 119 | binheap_insert(h, 3); 120 | binheap_insert(h, 4); 121 | binheap_insert(h, 5); 122 | binheap_insert(h, 6); 123 | binheap_insert(h, 7); 124 | binheap_insert(h, 8); 125 | 126 | for (i = 1; i <= 8; ++i) { 127 | binheap_peek(h, &v); 128 | if (v != i) { 129 | FAIL("binheap_peek set v=%d expected %d\n", v, i); 130 | } 131 | binheap_pop(h); 132 | } 133 | 134 | free(h); 135 | } 136 | 137 | static void test_insert_reverse(void) 138 | { 139 | struct binheap *h = binheap_new(8); 140 | uint32_t i, v; 141 | 142 | binheap_insert(h, 8); 143 | binheap_insert(h, 7); 144 | binheap_insert(h, 6); 145 | binheap_insert(h, 5); 146 | binheap_insert(h, 4); 147 | binheap_insert(h, 3); 148 | binheap_insert(h, 2); 149 | binheap_insert(h, 1); 150 | 151 | for (i = 1; i <= 8; ++i) { 152 | binheap_peek(h, &v); 153 | if (v != i) { 154 | FAIL("binheap_peek set v=%d expected %d\n", v, i); 155 | } 156 | binheap_pop(h); 157 | } 158 | 159 | free(h); 160 | } 161 | 162 | static void test_pop_left_smallest(void) 163 | { 164 | struct binheap *h = binheap_new(4); 165 | uint32_t expected[] = {1, 2, 3}; 166 | static_assert(array_size(expected) == 3, "array_size"); 167 | uint32_t v, *p, *end; 168 | 169 | binheap_insert(h, 2); 170 | binheap_insert(h, 1); 171 | binheap_insert(h, 3); 172 | 173 | for (p = expected, end = expected + array_size(expected); 174 | p != end; ++p) { 175 | binheap_peek(h, &v); 176 | if (v != *p) { 177 | FAIL("binheap_peek set v=%" PRIu32 " expected %" PRIu32 "\n", 178 | v, *p); 179 | } 180 | binheap_pop(h); 181 | } 182 | 183 | free(h); 184 | } 185 | 186 | static void test_pop_right_smallest(void) 187 | { 188 | struct binheap *h = binheap_new(4); 189 | uint32_t expected[] = {1, 2, 3}; 190 | uint32_t v, *p, *end; 191 | 192 | binheap_insert(h, 2); 193 | binheap_insert(h, 3); 194 | binheap_insert(h, 1); 195 | 196 | for (p = expected, end = expected + array_size(expected); 197 | p != end; ++p) { 198 | binheap_peek(h, &v); 199 | if (v != *p) { 200 | FAIL("binheap_peek set v=%d expected %d\n", 201 | v, *p); 202 | } 203 | binheap_pop(h); 204 | } 205 | 206 | free(h); 207 | } 208 | 209 | int main(__attribute__((__unused__)) int argc, 210 | __attribute__((__unused__)) char *argv[]) 211 | { 212 | test_peek_empty(); 213 | test_insert_one(); 214 | test_insert_inorder(); 215 | test_insert_reverse(); 216 | test_pop_left_smallest(); 217 | test_pop_right_smallest(); 218 | } 219 | -------------------------------------------------------------------------------- /ccan/LICENSE.CC0: -------------------------------------------------------------------------------- 1 | Statement of Purpose 2 | 3 | The laws of most jurisdictions throughout the world automatically confer exclusive Copyright and Related Rights (defined below) upon the creator and subsequent owner(s) (each and all, an "owner") of an original work of authorship and/or a database (each, a "Work"). 4 | 5 | Certain owners wish to permanently relinquish those rights to a Work for the purpose of contributing to a commons of creative, cultural and scientific works ("Commons") that the public can reliably and without fear of later claims of infringement build upon, modify, incorporate in other works, reuse and redistribute as freely as possible in any form whatsoever and for any purposes, including without limitation commercial purposes. These owners may contribute to the Commons to promote the ideal of a free culture and the further production of creative, cultural and scientific works, or to gain reputation or greater distribution for their Work in part through the use and efforts of others. 6 | 7 | For these and/or other purposes and motivations, and without any expectation of additional consideration or compensation, the person associating CC0 with a Work (the "Affirmer"), to the extent that he or she is an owner of Copyright and Related Rights in the Work, voluntarily elects to apply CC0 to the Work and publicly distribute the Work under its terms, with knowledge of his or her Copyright and Related Rights in the Work and the meaning and intended legal effect of CC0 on those rights. 8 | 9 | 1. Copyright and Related Rights. A Work made available under CC0 may be protected by copyright and related or neighboring rights ("Copyright and Related Rights"). Copyright and Related Rights include, but are not limited to, the following: 10 | 11 | the right to reproduce, adapt, distribute, perform, display, communicate, and translate a Work; 12 | moral rights retained by the original author(s) and/or performer(s); 13 | publicity and privacy rights pertaining to a person's image or likeness depicted in a Work; 14 | rights protecting against unfair competition in regards to a Work, subject to the limitations in paragraph 4(a), below; 15 | rights protecting the extraction, dissemination, use and reuse of data in a Work; 16 | database rights (such as those arising under Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protection of databases, and under any national implementation thereof, including any amended or successor version of such directive); and 17 | other similar, equivalent or corresponding rights throughout the world based on applicable law or treaty, and any national implementations thereof. 18 | 19 | 2. Waiver. To the greatest extent permitted by, but not in contravention of, applicable law, Affirmer hereby overtly, fully, permanently, irrevocably and unconditionally waives, abandons, and surrenders all of Affirmer's Copyright and Related Rights and associated claims and causes of action, whether now known or unknown (including existing as well as future claims and causes of action), in the Work (i) in all territories worldwide, (ii) for the maximum duration provided by applicable law or treaty (including future time extensions), (iii) in any current or future medium and for any number of copies, and (iv) for any purpose whatsoever, including without limitation commercial, advertising or promotional purposes (the "Waiver"). Affirmer makes the Waiver for the benefit of each member of the public at large and to the detriment of Affirmer's heirs and successors, fully intending that such Waiver shall not be subject to revocation, rescission, cancellation, termination, or any other legal or equitable action to disrupt the quiet enjoyment of the Work by the public as contemplated by Affirmer's express Statement of Purpose. 20 | 21 | 3. Public License Fallback. Should any part of the Waiver for any reason be judged legally invalid or ineffective under applicable law, then the Waiver shall be preserved to the maximum extent permitted taking into account Affirmer's express Statement of Purpose. In addition, to the extent the Waiver is so judged Affirmer hereby grants to each affected person a royalty-free, non transferable, non sublicensable, non exclusive, irrevocable and unconditional license to exercise Affirmer's Copyright and Related Rights in the Work (i) in all territories worldwide, (ii) for the maximum duration provided by applicable law or treaty (including future time extensions), (iii) in any current or future medium and for any number of copies, and (iv) for any purpose whatsoever, including without limitation commercial, advertising or promotional purposes (the "License"). The License shall be deemed effective as of the date CC0 was applied by Affirmer to the Work. Should any part of the License for any reason be judged legally invalid or ineffective under applicable law, such partial invalidity or ineffectiveness shall not invalidate the remainder of the License, and in such case Affirmer hereby affirms that he or she will not (i) exercise any of his or her remaining Copyright and Related Rights in the Work or (ii) assert any associated claims and causes of action with respect to the Work, in either case contrary to Affirmer's express Statement of Purpose. 22 | 23 | 4. Limitations and Disclaimers. 24 | 25 | No trademark or patent rights held by Affirmer are waived, abandoned, surrendered, licensed or otherwise affected by this document. 26 | Affirmer offers the Work as-is and makes no representations or warranties of any kind concerning the Work, express, implied, statutory or otherwise, including without limitation warranties of title, merchantability, fitness for a particular purpose, non infringement, or the absence of latent or other defects, accuracy, or the present or absence of errors, whether or not discoverable, all to the greatest extent permissible under applicable law. 27 | Affirmer disclaims responsibility for clearing rights of other persons that may apply to the Work or any use thereof, including without limitation any person's Copyright and Related Rights in the Work. Further, Affirmer disclaims responsibility for obtaining any necessary consents, permissions or other rights required for any use of the Work. 28 | Affirmer understands and acknowledges that Creative Commons is not a party to this document and has no duty or obligation with respect to this CC0 or use of the Work. 29 | --------------------------------------------------------------------------------