├── LICENSE ├── Makefile ├── Readme-cn.md ├── Readme.md ├── bpf ├── bpf.c ├── bpf.h └── bpf_helpers.h ├── common ├── ifapi.c ├── ifapi.h ├── log.c └── log.h ├── group ├── channel.c ├── dump.c ├── group.c ├── group.h └── xudp_dump.c ├── include ├── channel.h ├── common.h ├── config.h ├── dump.h ├── group_api.h ├── ip6.h ├── list.h ├── packet_parse.h ├── queue.h └── xudp.h ├── kern ├── Makefile ├── bpf_endian.h ├── bpf_helper_defs.h ├── dispatch_dict.c ├── dispatch_hash.c ├── dispatch_rr.c ├── kern.h └── kern_core.c ├── replace.sh ├── set_up.sh ├── set_up_vm2.sh ├── test ├── Makefile ├── Readme.md ├── auto │ ├── conftest.py │ ├── test_00_zerocopy.py │ ├── test_01_echo.py │ ├── test_02_rr.py │ ├── test_03_dict.py │ ├── test_04_setuid_cap.py │ ├── test_10_fork.py │ └── xudp.py ├── case │ ├── lib.c │ ├── lib.h │ ├── test_check_umem.c │ ├── test_clear_xdp.c │ ├── test_dict.c │ ├── test_dict_map_auto_release.c │ ├── test_echo.c │ ├── test_fork.c │ ├── test_group.c │ ├── test_log_debug.c │ ├── test_reload.c │ ├── test_rr.c │ └── test_setuid_cap.c ├── netns.sh ├── route6.py └── route6.sh ├── tools ├── Makefile ├── xudp-stats ├── xudp_echo_server.c └── xudpperf.c └── xudp ├── bind.c ├── checksum.h ├── group_api.c ├── kern_ops.c ├── kern_ops.h ├── neigh.c ├── neigh.h ├── packet.c ├── packet.h ├── ping.c ├── route.c ├── route.h ├── route6.c ├── route6.h ├── tx.c ├── types.h ├── xsk.c ├── xsk.h ├── xudp.c ├── xudp_map.c ├── xudp_route.c ├── xudp_route6.c └── xudp_types.h /LICENSE: -------------------------------------------------------------------------------- 1 | 木兰宽松许可证， 第2版 2 | 3 | 2020年1月 http://license.coscl.org.cn/MulanPSL2 4 | 5 | 您对“软件”的复制、使用、修改及分发受木兰宽松许可证，第2版（“本许可证”）的如下条款的约束： 6 | 7 | 0. 定义 8 | 9 | “软件” 是指由“贡献”构成的许可在“本许可证”下的程序和相关文档的集合。 10 | 11 | “贡献” 是指由任一“贡献者”许可在“本许可证”下的受版权法保护的作品。 12 | 13 | “贡献者” 是指将受版权法保护的作品许可在“本许可证”下的自然人或“法人实体”。 14 | 15 | “法人实体” 是指提交贡献的机构及其“关联实体”。 16 | 17 | “关联实体” 是指，对“本许可证”下的行为方而言，控制、受控制或与其共同受控制的机构，此处的控制是指有受控方或共同受控方至少50%直接或间接的投票权、资金或其他有价证券。 18 | 19 | 1. 授予版权许可 20 | 21 | 每个“贡献者”根据“本许可证”授予您永久性的、全球性的、免费的、非独占的、不可撤销的版权许可，您可以复制、使用、修改、分发其“贡献”，不论修改与否。 22 | 23 | 2. 授予专利许可 24 | 25 | 每个“贡献者”根据“本许可证”授予您永久性的、全球性的、免费的、非独占的、不可撤销的（根据本条规定撤销除外）专利许可，供您制造、委托制造、使用、许诺销售、销售、进口其“贡献”或以其他方式转移其“贡献”。前述专利许可仅限于“贡献者”现在或将来拥有或控制的其“贡献”本身或其“贡献”与许可“贡献”时的“软件”结合而将必然会侵犯的专利权利要求，不包括对“贡献”的修改或包含“贡献”的其他结合。如果您或您的“关联实体”直接或间接地，就“软件”或其中的“贡献”对任何人发起专利侵权诉讼（包括反诉或交叉诉讼）或其他专利维权行动，指控其侵犯专利权，则“本许可证”授予您对“软件”的专利许可自您提起诉讼或发起维权行动之日终止。 26 | 27 | 3. 无商标许可 28 | 29 | “本许可证”不提供对“贡献者”的商品名称、商标、服务标志或产品名称的商标许可，但您为满足第4条规定的声明义务而必须使用除外。 30 | 31 | 4. 分发限制 32 | 33 | 您可以在任何媒介中将“软件”以源程序形式或可执行形式重新分发，不论修改与否，但您必须向接收者提供“本许可证”的副本，并保留“软件”中的版权、商标、专利及免责声明。 34 | 35 | 5. 免责声明与责任限制 36 | 37 | “软件”及其中的“贡献”在提供时不带任何明示或默示的担保。在任何情况下，“贡献者”或版权所有者不对任何人因使用“软件”或其中的“贡献”而引发的任何直接或间接损失承担责任，不论因何种原因导致或者基于何种法律理论,即使其曾被建议有此种损失的可能性。 38 | 39 | 6. 语言 40 | 41 | “本许可证”以中英文双语表述，中英文版本具有同等法律效力。如果中英文版本存在任何冲突不一致，以中文版为准。 42 | 43 | 条款结束 44 | 45 | 如何将木兰宽松许可证，第2版，应用到您的软件 46 | 47 | 如果您希望将木兰宽松许可证，第2版，应用到您的新软件，为了方便接收者查阅，建议您完成如下三步： 48 | 49 | 1， 请您补充如下声明中的空白，包括软件名、软件的首次发表年份以及您作为版权人的名字； 50 | 51 | 2， 请您在软件包的一级目录下创建以“LICENSE”为名的文件，将整个许可证文本放入该文件中； 52 | 53 | 3， 请将如下声明文本放入每个源文件的头部注释中。 54 | 55 | Copyright (c) 2021 Alibaba Group Holding Limited 56 | Express UDP is licensed under Mulan PSL v2. 57 | You can use this software according to the terms and conditions of the Mulan PSL v2. 58 | You may obtain a copy of Mulan PSL v2 at: 59 | http://license.coscl.org.cn/MulanPSL2 60 | THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 61 | EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 62 | MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 63 | See the Mulan PSL v2 for more details. 64 | Mulan Permissive Software License，Version 2 65 | Mulan Permissive Software License，Version 2 (Mulan PSL v2) 66 | 67 | January 2020 http://license.coscl.org.cn/MulanPSL2 68 | 69 | Your reproduction, use, modification and distribution of the Software shall be subject to Mulan PSL v2 (this License) with the following terms and conditions: 70 | 71 | 0. Definition 72 | 73 | Software means the program and related documents which are licensed under this License and comprise all Contribution(s). 74 | 75 | Contribution means the copyrightable work licensed by a particular Contributor under this License. 76 | 77 | Contributor means the Individual or Legal Entity who licenses its copyrightable work under this License. 78 | 79 | Legal Entity means the entity making a Contribution and all its Affiliates. 80 | 81 | Affiliates means entities that control, are controlled by, or are under common control with the acting entity under this License, ‘control’ means direct or indirect ownership of at least fifty percent (50%) of the voting power, capital or other securities of controlled or commonly controlled entity. 82 | 83 | 1. Grant of Copyright License 84 | 85 | Subject to the terms and conditions of this License, each Contributor hereby grants to you a perpetual, worldwide, royalty-free, non-exclusive, irrevocable copyright license to reproduce, use, modify, or distribute its Contribution, with modification or not. 86 | 87 | 2. Grant of Patent License 88 | 89 | Subject to the terms and conditions of this License, each Contributor hereby grants to you a perpetual, worldwide, royalty-free, non-exclusive, irrevocable (except for revocation under this Section) patent license to make, have made, use, offer for sale, sell, import or otherwise transfer its Contribution, where such patent license is only limited to the patent claims owned or controlled by such Contributor now or in future which will be necessarily infringed by its Contribution alone, or by combination of the Contribution with the Software to which the Contribution was contributed. The patent license shall not apply to any modification of the Contribution, and any other combination which includes the Contribution. If you or your Affiliates directly or indirectly institute patent litigation (including a cross claim or counterclaim in a litigation) or other patent enforcement activities against any individual or entity by alleging that the Software or any Contribution in it infringes patents, then any patent license granted to you under this License for the Software shall terminate as of the date such litigation or activity is filed or taken. 90 | 91 | 3. No Trademark License 92 | 93 | No trademark license is granted to use the trade names, trademarks, service marks, or product names of Contributor, except as required to fulfill notice requirements in section 4. 94 | 95 | 4. Distribution Restriction 96 | 97 | You may distribute the Software in any medium with or without modification, whether in source or executable forms, provided that you provide recipients with a copy of this License and retain copyright, patent, trademark and disclaimer statements in the Software. 98 | 99 | 5. Disclaimer of Warranty and Limitation of Liability 100 | 101 | THE SOFTWARE AND CONTRIBUTION IN IT ARE PROVIDED WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED. IN NO EVENT SHALL ANY CONTRIBUTOR OR COPYRIGHT HOLDER BE LIABLE TO YOU FOR ANY DAMAGES, INCLUDING, BUT NOT LIMITED TO ANY DIRECT, OR INDIRECT, SPECIAL OR CONSEQUENTIAL DAMAGES ARISING FROM YOUR USE OR INABILITY TO USE THE SOFTWARE OR THE CONTRIBUTION IN IT, NO MATTER HOW IT’S CAUSED OR BASED ON WHICH LEGAL THEORY, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. 102 | 103 | 6. Language 104 | 105 | THIS LICENSE IS WRITTEN IN BOTH CHINESE AND ENGLISH, AND THE CHINESE VERSION AND ENGLISH VERSION SHALL HAVE THE SAME LEGAL EFFECT. IN THE CASE OF DIVERGENCE BETWEEN THE CHINESE AND ENGLISH VERSIONS, THE CHINESE VERSION SHALL PREVAIL. 106 | 107 | END OF THE TERMS AND CONDITIONS 108 | 109 | How to Apply the Mulan Permissive Software License，Version 2 (Mulan PSL v2) to Your Software 110 | 111 | To apply the Mulan PSL v2 to your work, for easy identification by recipients, you are suggested to complete following three steps: 112 | 113 | Fill in the blanks in following statement, including insert your software name, the year of the first publication of your software, and your name identified as the copyright owner; 114 | Create a file named "LICENSE" which contains the whole context of this License in the first directory of your software package; 115 | Attach the statement to the appropriate annotated syntax at the beginning of each source file. 116 | 117 | Copyright (c) 2021 Alibaba Group Holding Limited 118 | Express UDP is licensed under Mulan PSL v2. 119 | You can use this software according to the terms and conditions of the Mulan PSL v2. 120 | You may obtain a copy of Mulan PSL v2 at: 121 | http://license.coscl.org.cn/MulanPSL2 122 | THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 123 | EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 124 | MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 125 | See the Mulan PSL v2 for more details. 126 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # 2 | # Copyright (c) 2021 Alibaba Group Holding Limited 3 | # Express UDP is licensed under Mulan PSL v2. 4 | # You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | # You may obtain a copy of Mulan PSL v2 at: 6 | # http://license.coscl.org.cn/MulanPSL2 7 | # THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | # EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | # MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | # See the Mulan PSL v2 for more details. 11 | # 12 | 13 | ifdef XUDP_DEBUG 14 | CFLAGS += -DXUDP_DEBUG=1 15 | CFLAGS += -g -Wall -I /usr/include/libnl3 16 | else 17 | CFLAGS += -O2 -g -Wall -I /usr/include/libnl3 18 | endif 19 | 20 | 21 | CFLAGS_common=$(CFLAGS) -I include 22 | CFLAGS_bpf= $(CFLAGS) -I common 23 | CFLAGS_xudp= $(CFLAGS) -I common -I include -I kern -I bpf 24 | CFLAGS_group= $(CFLAGS) -I common -I include -I kern -I bpf 25 | 26 | ################################################################################ 27 | 28 | ALL += objs/libxudp.so.$(VERSION) 29 | ALL += objs/libxudp.a 30 | ALL += objs/xudp-map 31 | ALL += objs/xudp-dump 32 | ALL += objs/xudp-route 33 | ALL += objs/xudp-route6 34 | 35 | LIBXUDP += objs/common/ifapi.o 36 | LIBXUDP += objs/common/log.o 37 | LIBXUDP += objs/bpf/bpf.o 38 | 39 | LIBXUDP += objs/xudp/xudp.o 40 | LIBXUDP += objs/xudp/bind.o 41 | LIBXUDP += objs/xudp/packet.o 42 | LIBXUDP += objs/xudp/route.o 43 | LIBXUDP += objs/xudp/route6.o 44 | LIBXUDP += objs/xudp/neigh.o 45 | LIBXUDP += objs/xudp/ping.o 46 | LIBXUDP += objs/xudp/xsk.o 47 | LIBXUDP += objs/xudp/tx.o 48 | LIBXUDP += objs/xudp/group_api.o 49 | 50 | LIBXUDP += objs/group/channel.o 51 | LIBXUDP += objs/group/group.o 52 | LIBXUDP += objs/group/dump.o 53 | 54 | LIBXUDP += objs/kern/em_kern_hash.o 55 | LIBXUDP += objs/kern/em_kern_dict.o 56 | LIBXUDP += objs/kern/em_kern_rr.o 57 | 58 | LIBXUDP += objs/xudp/kern_ops.o 59 | 60 | VER_X=1 61 | VER_Y=0 62 | VER_Z=2 63 | 64 | SONAME=libxudp.so.1 65 | VERSION=$(VER_X).$(VER_Y).$(VER_Z) 66 | VERSION_INT=1000000000 67 | 68 | prefix = /usr/local 69 | 70 | 71 | ################################################################################ 72 | 73 | .PHONY: all clean test kern 74 | 75 | all: clean $(ALL) 76 | make -C tools SO=$(SO) 77 | 78 | objs/%.o: %.c 79 | @echo CC $@ 80 | @mkdir $(shell dirname $@) 2>/dev/null || true 81 | @gcc $< $(CFLAGS_$(shell dirname $<)) -fPIC -c -o $@ 82 | 83 | objs/kern/em_kern_%.o: 84 | make -C kern XUDP_DEBUG=$(XUDP_DEBUG) ../$@ 85 | 86 | # this for custom xdp bpf. 87 | # $(srcdir) is the source dirname 88 | # $(target) 89 | kern: 90 | make -C kern XUDP_DEBUG=$(XUDP_DEBUG) \ 91 | build=$(srcdir) \ 92 | srcdir=$(srcdir) \ 93 | $(srcdir)/$(target) 94 | 95 | objs/libxudp.so.$(VERSION): $(LIBXUDP) 96 | @echo LD $@ 97 | @gcc $^ -lelf -shared -o $@ -Wl,-soname,$(SONAME) -l pthread 98 | @rm objs/xudp.h 2>/dev/null || true 99 | @cp include/xudp.h objs 100 | @echo '#define XUDP_HASH "'$(shell git rev-parse --short HEAD)'"' >> objs/xudp.h 101 | @echo '#define XUDP_VERSION "'$(VERSION)'"' >> objs/xudp.h 102 | @echo '#define XUDP_VERSION_INT '$(VERSION_INT) >> objs/xudp.h 103 | @ln -s libxudp.so.$(VERSION) objs/libxudp.so 104 | @ln -s libxudp.so.$(VERSION) objs/libxudp.so.$(VER_X) 105 | 106 | objs/libxudp.a: $(LIBXUDP) 107 | @echo AR $@ 108 | @ar rcs $@ $^ 109 | 110 | objs/xudp-map: objs/xudp/xudp_map.o 111 | @gcc $^ $(LDFLAGS) -o $@ -l pthread 112 | 113 | objs/xudp-dump: objs/group/xudp_dump.o 114 | @gcc $^ $(LDFLAGS) -o $@ -l pthread 115 | 116 | objs/xudp-route: objs/xudp/route.o objs/xudp/xudp_route.o objs/common/ifapi.o objs/common/log.o 117 | @gcc $^ $(LDFLAGS) -o $@ 118 | 119 | objs/xudp-route6: objs/xudp/route6.o objs/xudp/xudp_route6.o objs/common/ifapi.o objs/common/log.o 120 | @gcc $^ $(LDFLAGS) -o $@ 121 | 122 | test: 123 | make -C test 124 | 125 | test_run: 126 | make -C test run pytest="$(pytest)" 127 | 128 | clean: 129 | -mkdir objs 2>/dev/null 130 | rm -rf objs/* 131 | 132 | install: 133 | cp objs/xudp.h $(prefix)/include/ 134 | cp -P objs/libxudp.so $(prefix)/lib64/ 135 | cp -P objs/libxudp.so.$(VER_X) $(prefix)/lib64/ 136 | cp objs/libxudp.so.$(VERSION) $(prefix)/lib64/ 137 | cp objs/libxudp.a $(prefix)/lib64/ 138 | cp objs/xudpperf $(prefix)/bin/ 139 | cp objs/xudp-echo-server $(prefix)/bin/ 140 | cp objs/xudp-map $(prefix)/bin/ 141 | cp objs/xudp-dump $(prefix)/bin/ 142 | cp tools/xudp-stats $(prefix)/bin/ 143 | cp objs/xudp-route $(prefix)/bin/ 144 | cp objs/xudp-route6 $(prefix)/bin/ 145 | 146 | uninstall: 147 | rm $(prefix)/include/xudp.h 148 | rm $(prefix)/lib64/libxudp.so 149 | rm $(prefix)/lib64/libxudp.a 150 | rm $(prefix)/bin/xudpperf 151 | rm $(prefix)/bin/xudp-echo-server 152 | rm $(prefix)/bin/xudp-map 153 | rm $(prefix)/bin/xudp-stats 154 | -------------------------------------------------------------------------------- /Readme-cn.md: -------------------------------------------------------------------------------- 1 | ## Express UDP 2 | 3 | Express UDP 是基于 XDP Socket 实现的 UDP 通信软件库. 4 | 5 | ### 收包 6 | 7 | 基于 XDP 在网卡驱动层对包进行过滤, 把指定 ip:port 的 udp 包传递给 8 | XSK, 应用层 Express UDP 从 XSK 中收包。 9 | 10 | ### 发包 11 | Express UDP 传递包给 XSK, XSK 把包构建 skb 并放入网卡队列中进行发包。XSK ZEROCOPY 12 | 模式下可以跳过构建 skb 的过程直接把包传递给网卡, 这样性能会更好。 13 | 14 | ## 构建与安装 15 | 16 | ``` 17 | yum install libnl3-devel elfutils-libelf-devel clang llvm libcap-devel 18 | make 19 | ``` 20 | 21 | 所有的输出在 objs 目录下面。 22 | 同时输出有 libxudp.a 和 libxudp.so 文件以及头文件 xudp.h。 23 | 24 | ## 体验一下 25 | tools/xudp\_echo\_server.c 是一个简单的 echo 服务实现。编译后的文件在在 objs 下面, 命令格式: 26 | 27 | ``` 28 | ./xudp-echo-server 29 | ``` 30 | ip 为要 bind 的本地地址, 如果使用 0.0.0.0 会监听所有的网卡, 详细后面还会介绍。 31 | 启动之后可以使用如下命令进行测试: 32 | 33 | ``` 34 | nc -u 35 | ``` 36 | 发送 udp 消息给服务端, server 会打印消息并 echo 给客户端。 37 | 38 | 更多实践信息可见: [阿里云实践](https://openanolis.cn/sig/high-perf-network/doc/411142504638253189?preview=null) 39 | 40 | ## bind 41 | 这个过程主要是把 ebpf 与网卡进行绑定, 并且创建 xsk 再与和 ebpf 进行绑定。 42 | 43 | 注意, 一些网卡并不支持 xdp 如 lo, 所以目前 xudp 不能工作于这些网卡。 44 | 45 | ### 指定 ip 绑定 46 | bind 过程中, Express UDP 会过滤所有的网卡, 对所有的符合 ip 指定的网卡都进行 47 | bind 操作. 比如 0.0.0.0, xudp 会尝试 bind 所有的非 lo 网卡, 如果有一个网卡 48 | bind 失败, 整个操作都视为失败。 49 | bind 可以指定多个 struct sockaddr, 由参数 num 控制, 目前最多只支持 10 个。 50 | 51 | ``` 52 | int xudp_bind(xudp *x, struct sockaddr *a, socklen_t addrlen, int num); 53 | ``` 54 | 55 | ## 多线程与多进程 56 | 由于现在一般网卡都是支持多队列的, 最好可以使用多线程或多进程进程编程。 57 | recv/send 分别有两个接口, 默认情况下会轮询地对 channel 进行 recv/send 58 | 操作。带有 channel 后缀的接口, 只对指定的 channel 进行 recv/send 操作。 59 | 60 | 多线程或多进程的情况下, 一个线程或进程操作一个 channel, 会获得比较好的性能。 61 | 62 | ## epoll/poll/select 63 | 每一个 channel 可以基于 xudp\_channel\_getfd 获得对应的 xsk 的 fd, 64 | 这个 fd 支持epoll 操作, 可以用于 epoll/poll/select 对读写事件进行监听。 65 | 66 | tools/xudp\_echo\_server.c 代码里有 epoll 的例子。 67 | 68 | ## 查看 XDP 的情况 69 | 新的版本的 iproute 可以直接查看, 配置网卡的 XDP. 70 | 笔者本地 iproute2-ss170501 已经支持这个功能了. 71 | 72 | 如果网卡有配置 xdp, 在 ip link 输出中有显示 xdp 字样。 73 | 74 | 命令 `ip link set xdp off` 可以关闭指定网卡的 XDP. 75 | 76 | ## 路由 77 | 目前 libxudp 已经支持路由, 也就是 client 和 server 可以不在同个网段, 也可以工作 78 | 于公网了。 79 | 80 | Express UDP 在启动的时候获取本机的路由, 并保存在内存里面, 目前并不支持的路由修改和 81 | 同步。 82 | 83 | 路由的查找基于 hash 实现, 速度很快, 但是代价是要占用多一些内存空间。 84 | 85 | ## ARP 86 | 87 | 目前有一个简陋的 ARP 实现, 由于阿里云有 ARP Proxy 支持, 所以打开 libxudp 88 | noarp 选项可以正常工作于阿里云, 并且没有 arp 的相关开销。 89 | 90 | 目前推荐在阿里云上打开 noarp 选项使用。 91 | 92 | -------------------------------------------------------------------------------- /Readme.md: -------------------------------------------------------------------------------- 1 | ## Express UDP 2 | 3 | Express UDP is a high-performance UDP communication software library based on 4 | the xdp socket technology introduced in kernel 4.18. 5 | 6 | [Chinese Readme](Readme-cn.md) 7 | 8 | ## make & install 9 | 10 | ``` 11 | yum install libnl3-devel elfutils-libelf-devel clang llvm libcap-devel 12 | make 13 | ``` 14 | 15 | All output is in the objs directory. At the same time, there are libxudp.a and 16 | libxudp.so files and the header file xudp.h. 17 | 18 | ## Experience It 19 | 20 | tools/xudp\_echo\_server.c is a simple echo service implementation. The compiled 21 | file is under objs, command format: 22 | 23 | ``` 24 | ./xudp-echo-server 25 | ``` 26 | 27 | ip is the local address to be bound, if you use 0.0.0.0, all network cards will 28 | be monitored. The details will be introduced later. After startup, you can use 29 | the following command to test: 30 | 31 | ``` 32 | nc -u 33 | ``` 34 | 35 | 36 | Send udp message to the server, the server will print the message and echo to 37 | the client. 38 | 39 | ## ARP 40 | 41 | There is currently a rudimentary ARP implementation. Since Alibaba Cloud has ARP 42 | Proxy support, opening the libxudp noarp option can work in Alibaba Cloud 43 | without the related overhead of arp. 44 | 45 | It is currently recommended to turn on the noarp option on Alibaba Cloud. 46 | 47 | -------------------------------------------------------------------------------- /bpf/bpf.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __BPF_H__ 15 | #define __BPF_H__ 16 | 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include 24 | 25 | #define _GNU_SOURCE /* See feature_test_macros(7) */ 26 | #include 27 | #include /* For SYS_xxx definitions */ 28 | #include "log.h" 29 | 30 | #include "bpf_helpers.h" 31 | 32 | /* 33 | * When building perf, unistd.h is overridden. __NR_bpf is 34 | * required to be defined explicitly. 35 | */ 36 | #ifndef __NR_bpf 37 | # if defined(__i386__) 38 | # define __NR_bpf 357 39 | # elif defined(__x86_64__) 40 | # define __NR_bpf 321 41 | # elif defined(__aarch64__) 42 | # define __NR_bpf 280 43 | # elif defined(__sparc__) 44 | # define __NR_bpf 349 45 | # elif defined(__s390__) 46 | # define __NR_bpf 351 47 | # elif defined(__arc__) 48 | # define __NR_bpf 280 49 | # else 50 | # error __NR_bpf not defined. libbpf does not support your arch. 51 | # endif 52 | #endif 53 | 54 | 55 | #define ptr_to_u64(x) ((uint64_t)x) 56 | 57 | #ifndef SOL_XDP 58 | #define SOL_XDP 283 59 | #endif 60 | 61 | #ifndef AF_XDP 62 | #define AF_XDP 44 63 | #endif 64 | 65 | #ifndef PF_XDP 66 | #define PF_XDP AF_XDP 67 | #endif 68 | 69 | struct str{ 70 | char *p; 71 | int size; 72 | }; 73 | 74 | struct map; 75 | 76 | struct rel_sec{ 77 | GElf_Shdr sh; 78 | Elf_Data *data; 79 | }; 80 | 81 | struct bpf{ 82 | int maps_idx; 83 | 84 | 85 | int (*map_filter)(struct bpf_map_def *, void *); 86 | void *map_filter_data; 87 | 88 | struct map *maps; 89 | int maps_n; 90 | 91 | Elf *elf; 92 | GElf_Ehdr ehdr; 93 | 94 | struct str ins; 95 | 96 | char *target_proc; 97 | 98 | struct str license; 99 | struct str version; 100 | 101 | enum bpf_prog_type type; 102 | 103 | Elf_Data *symbols; 104 | int symbols_shndx; 105 | int prog_fd; 106 | 107 | size_t strtabidx; 108 | 109 | Elf_Scn *rel_scn; 110 | 111 | struct log *log; 112 | }; 113 | 114 | static inline int sys_bpf(enum bpf_cmd cmd, union bpf_attr *attr, 115 | unsigned int size) 116 | { 117 | return syscall(__NR_bpf, cmd, attr, size); 118 | } 119 | int bpf_load(struct bpf *b, char *proc, 120 | enum bpf_prog_type type, char *elf, int size); 121 | int bpf_map_get(struct bpf *b, const char *name); 122 | void bpf_close(struct bpf *b); 123 | 124 | static inline int bpf_map_update_elem(int fd, const void *key, const void *value, 125 | __u64 flags) 126 | { 127 | union bpf_attr attr; 128 | 129 | memset(&attr, 0, sizeof(attr)); 130 | attr.map_fd = fd; 131 | attr.key = ptr_to_u64(key); 132 | attr.value = ptr_to_u64(value); 133 | attr.flags = flags; 134 | 135 | return sys_bpf(BPF_MAP_UPDATE_ELEM, &attr, sizeof(attr)); 136 | } 137 | 138 | static inline int bpf_map_lookup_elem(int fd, const void *key, void *value) 139 | { 140 | union bpf_attr attr; 141 | 142 | memset(&attr, 0, sizeof(attr)); 143 | attr.map_fd = fd; 144 | attr.key = ptr_to_u64(key); 145 | attr.value = ptr_to_u64(value); 146 | 147 | return sys_bpf(BPF_MAP_LOOKUP_ELEM, &attr, sizeof(attr)); 148 | } 149 | 150 | static inline int bpf_map_delete_elem(int fd, const void *key) 151 | { 152 | union bpf_attr attr; 153 | 154 | memset(&attr, 0, sizeof(attr)); 155 | attr.map_fd = fd; 156 | attr.key = ptr_to_u64(key); 157 | 158 | return sys_bpf(BPF_MAP_DELETE_ELEM, &attr, sizeof(attr)); 159 | } 160 | 161 | #ifdef BPF_XDP 162 | static inline int bpf_xdp_link_create(int prog_fd, int ifindex) 163 | { 164 | union bpf_attr attr; 165 | 166 | memset(&attr, 0, sizeof(attr)); 167 | 168 | attr.link_create.prog_fd = prog_fd; 169 | attr.link_create.target_ifindex = ifindex; 170 | attr.link_create.attach_type = BPF_XDP; 171 | 172 | return sys_bpf(BPF_LINK_CREATE, &attr, sizeof(attr)); 173 | } 174 | #endif 175 | 176 | static inline int bpf_next_id(int id) 177 | { 178 | union bpf_attr attr; 179 | int err; 180 | 181 | memset(&attr, 0, sizeof(attr)); 182 | 183 | attr.start_id = id; 184 | 185 | err = sys_bpf(BPF_MAP_GET_NEXT_ID, &attr, sizeof(attr)); 186 | if (err) 187 | return err; 188 | 189 | return attr.next_id; 190 | } 191 | 192 | static inline int bpf_get_fd(int id) 193 | { 194 | union bpf_attr attr; 195 | int err; 196 | 197 | memset(&attr, 0, sizeof(attr)); 198 | 199 | attr.map_id = id; 200 | 201 | err = sys_bpf(BPF_MAP_GET_FD_BY_ID, &attr, sizeof(attr)); 202 | if (err) 203 | return err; 204 | 205 | return attr.map_fd; 206 | } 207 | 208 | static inline int bpf_get_info_by_fd(int fd, struct bpf_map_info *info) 209 | { 210 | union bpf_attr attr; 211 | int err; 212 | 213 | memset(&attr, 0, sizeof(attr)); 214 | attr.info.bpf_fd = fd; 215 | attr.info.info = ptr_to_u64(info); 216 | attr.info.info_len = sizeof(*info); 217 | 218 | err = sys_bpf(BPF_OBJ_GET_INFO_BY_FD, &attr, sizeof(attr)); 219 | 220 | return err; 221 | } 222 | 223 | static inline int bpf_get_info(int id, struct bpf_map_info *info) 224 | { 225 | union bpf_attr attr; 226 | int fd, err; 227 | 228 | fd = bpf_get_fd(id); 229 | if (fd < 0) 230 | return fd; 231 | 232 | memset(&attr, 0, sizeof(attr)); 233 | attr.info.bpf_fd = fd; 234 | attr.info.info = ptr_to_u64(info); 235 | attr.info.info_len = sizeof(*info); 236 | 237 | err = sys_bpf(BPF_OBJ_GET_INFO_BY_FD, &attr, sizeof(attr)); 238 | 239 | close(fd); 240 | 241 | return err; 242 | } 243 | 244 | static inline int bpf_lookup_map(char *name) 245 | { 246 | struct bpf_map_info info; 247 | int id = 0; 248 | 249 | while (true) { 250 | id = bpf_next_id(id); 251 | if (id < 0) 252 | break; 253 | 254 | if (bpf_get_info(id, &info)) 255 | return -1; 256 | 257 | if (strcmp(name, info.name)) 258 | continue; 259 | 260 | return id; 261 | 262 | } 263 | 264 | return -1; 265 | } 266 | 267 | static inline unsigned int net_namespace_id(void) 268 | { 269 | struct stat buf; 270 | 271 | stat("/proc/self/ns/net", &buf); 272 | 273 | return buf.st_ino; 274 | } 275 | 276 | int bpf_map_get_idx(struct bpf *b, unsigned int i, int *fd); 277 | #endif 278 | 279 | 280 | -------------------------------------------------------------------------------- /bpf/bpf_helpers.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __BPF_HELPERS__ 15 | #define __BPF_HELPERS__ 16 | 17 | /* 18 | * Helper structure used by eBPF C program 19 | * to describe BPF map attributes to libbpf loader 20 | */ 21 | struct bpf_map_def { 22 | char *name; 23 | unsigned int type; 24 | unsigned int key_size; 25 | unsigned int value_size; 26 | unsigned int max_entries; 27 | unsigned int map_flags; 28 | unsigned int inner_map_fd; 29 | }; 30 | #endif 31 | -------------------------------------------------------------------------------- /common/ifapi.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __IFAPI_H__ 15 | #define __IFAPI_H__ 16 | 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include 24 | #include 25 | 26 | #include 27 | #include 28 | #include 29 | #include 30 | #include 31 | #include 32 | #include 33 | #include "log.h" 34 | #include "list.h" 35 | 36 | int ifgetchannels(const char *name, int *rx, int *tx); 37 | int ifgetringsize(const char *name, int *rx, int *tx); 38 | 39 | int nl_xdp_set(int ifindex, int fd, __u32 flags, struct log *log); 40 | void nl_xdp_off(int ifindex, struct log *log); 41 | 42 | typedef __u8 u8; 43 | 44 | struct nl_addr { 45 | struct nl_addr *next; 46 | unsigned char prefixlen; 47 | int family; 48 | union { 49 | __be32 addr; 50 | struct in6_addr addr6; 51 | }; 52 | }; 53 | 54 | struct nl_info{ 55 | struct nl_info *next; 56 | 57 | struct nl_info *master; 58 | bool isbond; 59 | bool ismaster; 60 | 61 | int ref; 62 | 63 | char ifname[IFNAMSIZ]; 64 | int ifindex; 65 | unsigned char mac[6]; 66 | unsigned int mtu; 67 | unsigned int xdp_prog_id; 68 | 69 | struct nl_addr *addr; 70 | 71 | unsigned short ifi_type; /* Device type */ 72 | unsigned int ifi_flags; /* Device flags */ 73 | }; 74 | 75 | typedef __u32 u32; 76 | 77 | struct nl_route { 78 | struct list_head list; 79 | 80 | u32 mask; 81 | u32 dst_h; 82 | 83 | union { 84 | __be32 dst; 85 | struct in6_addr dst6; 86 | }; 87 | 88 | union { 89 | __be32 pref_src; 90 | struct in6_addr pref_src6; 91 | }; 92 | 93 | union { 94 | __be32 next_hop; 95 | struct in6_addr next_hop6; 96 | }; 97 | 98 | char dst_len; 99 | int ifid; 100 | unsigned char index; 101 | }; 102 | 103 | int nl_link(struct nl_info **ni, int ifindex, struct log *log); 104 | int nl_link_byname(struct nl_info **_ni, const char *name, struct log *log); 105 | int nl_neigh(__be32 addr, unsigned char *mac, struct log *log); 106 | int nl_neigh6(struct in6_addr *addr, unsigned char *mac, struct log *log); 107 | int __nl_route(struct list_head *head, u8 family, struct log *log); 108 | #define nl_route(head, log) __nl_route(head, AF_INET, log) 109 | #define nl_route6(head, log) __nl_route(head, AF_INET6, log) 110 | 111 | void nl_route_free(struct list_head *head); 112 | /* free nl_info 113 | * 114 | * force true: free all nl_info 115 | * false: free all no ref nl_info, and return the left item 116 | * 117 | * 118 | * */ 119 | struct nl_info *nl_info_free(struct nl_info *ia, bool force); 120 | #endif 121 | 122 | 123 | -------------------------------------------------------------------------------- /common/log.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include 15 | #include 16 | #include 17 | #include 18 | 19 | #include "log.h" 20 | 21 | int logcore(struct log *log, char *fmt, ...) 22 | { 23 | char buf[1024], *pos; 24 | int size, ret, n; 25 | va_list ap; 26 | time_t t; 27 | struct tm* tm_info; 28 | 29 | size = sizeof(buf); 30 | pos = buf; 31 | 32 | if (log->time) { 33 | t = time(NULL); 34 | tm_info = localtime(&t); 35 | n = strftime(pos, size, "[%Y-%m-%d %H:%M:%S] ", tm_info); 36 | size -= n; 37 | pos += n; 38 | } else if (log->time_us) { 39 | struct timespec tp; 40 | 41 | clock_gettime(CLOCK_MONOTONIC, &tp); 42 | n = snprintf(pos, size, "[%ldus] ", 43 | tp.tv_sec * 1000 * 1000 + tp.tv_nsec / 1000); 44 | size -= n; 45 | pos += n; 46 | } 47 | 48 | if (log->prefix_len) { 49 | memcpy(pos, log->prefix, log->prefix_len); 50 | pos += log->prefix_len; 51 | size -= log->prefix_len; 52 | } 53 | 54 | va_start(ap, fmt); 55 | pos += vsnprintf(pos, size, fmt, ap); 56 | va_end(ap); 57 | 58 | if (log->cb) { 59 | if (*(pos - 1) == '\n') 60 | --pos; 61 | 62 | *pos = 0; 63 | 64 | ret = log->cb(buf, pos - buf, log->data); 65 | } else { 66 | ret = write(1, buf, pos - buf); 67 | } 68 | 69 | return ret; 70 | } 71 | -------------------------------------------------------------------------------- /common/log.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __LOG_H__ 15 | #define __LOG_H__ 16 | #include 17 | #include 18 | 19 | enum { 20 | LOG_LEVEL_ERR, 21 | LOG_LEVEL_WARN, 22 | LOG_LEVEL_INFO, 23 | LOG_LEVEL_DEBUG, 24 | }; 25 | 26 | typedef int (*log_cb)(char *buf, int size, void *data); 27 | 28 | struct log { 29 | int level; 30 | void *data; 31 | log_cb cb; 32 | char *prefix; 33 | int prefix_len; 34 | bool time; 35 | bool time_us; 36 | }; 37 | 38 | 39 | int logcore(struct log *log, char *fmt, ...); 40 | 41 | #define logerr(log, fmt, ...) \ 42 | if (LOG_LEVEL_ERR <= log->level) \ 43 | logcore(log, fmt, ##__VA_ARGS__) 44 | 45 | #define logwrn(log, fmt, ...) \ 46 | if (LOG_LEVEL_WARN <= log->level) \ 47 | logcore(log, fmt, ##__VA_ARGS__) 48 | 49 | #define loginfo(log, fmt, ...) \ 50 | if (LOG_LEVEL_INFO <= log->level) \ 51 | logcore(log, fmt, ##__VA_ARGS__) 52 | 53 | #define logdebug(log, fmt, ...) \ 54 | if (LOG_LEVEL_DEBUG <= log->level) \ 55 | logcore(log, fmt, ##__VA_ARGS__) 56 | 57 | 58 | #define log_enable(log, l) (LOG_LEVEL_##l <= log->level) 59 | 60 | 61 | #endif 62 | 63 | 64 | -------------------------------------------------------------------------------- /group/channel.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include 15 | #include "group.h" 16 | #include "ip6.h" 17 | #include "kern.h" 18 | #include "bpf.h" 19 | #include "packet_parse.h" 20 | 21 | static inline void *xq_get_data(struct xdpsock *xsk, u64 addr) 22 | { 23 | #if XDP_UMEM_UNALIGNED_CHUNK_FLAG 24 | struct rxch *rxch; 25 | int offset; 26 | 27 | rxch = container_of(xsk, struct rxch, xsk); 28 | 29 | if (!rxch->unaligned) 30 | return &xsk->frames[addr]; 31 | 32 | offset = addr >> XSK_UNALIGNED_BUF_OFFSET_SHIFT; 33 | addr = addr & XSK_UNALIGNED_BUF_ADDR_MASK; 34 | 35 | addr = addr + offset; 36 | 37 | return &xsk->frames[addr]; 38 | #else 39 | return &xsk->frames[addr]; 40 | #endif 41 | } 42 | 43 | static inline void umem_fq_put(struct xdpsock *xsk, 44 | struct xdp_desc *d, 45 | size_t nb) 46 | { 47 | fq_enq(xsk->fq, d, nb, xsk->headroom); 48 | } 49 | 50 | int xudp_channel_get_fd(xudp_channel *ch) 51 | { 52 | if (ch->istx) 53 | ch = ch->tx_xsk; 54 | return ch->sfd; 55 | } 56 | 57 | int xudp_channel_get_groupid(xudp_channel *ch) 58 | { 59 | if (ch->istx) 60 | ch = ch->tx_xsk; 61 | return ch->gid; 62 | } 63 | 64 | int xudp_channel_is_tx(xudp_channel *ch) 65 | { 66 | return ch->istx; 67 | } 68 | 69 | static int xudp_fill_msg(struct xdpsock *xsk, char *pkt, 70 | struct xdp_desc *desc, struct pkthdrs *hdrs, 71 | xudp_msg *m, int flags) 72 | { 73 | struct sockaddr_in6 *in6; 74 | struct sockaddr_in *in; 75 | struct iphdr *iph; 76 | struct ipv6hdr *iph6; 77 | struct udphdr *udp; 78 | char *body; 79 | int len; 80 | 81 | iph = hdrs->iph; 82 | iph6 = hdrs->iph6; 83 | udp = hdrs->udp; 84 | 85 | body = (void *)(udp + 1); 86 | len = htons(udp->len) - sizeof(*udp); 87 | 88 | memset(&m->peer_addr, 0, sizeof(m->peer_addr)); 89 | memset(&m->local_addr, 0, sizeof(m->local_addr)); 90 | 91 | if (hdrs->family == AF_INET) { 92 | in = (struct sockaddr_in *)&m->peer_addr; 93 | in->sin_family = AF_INET; 94 | in->sin_port = udp->source; 95 | in->sin_addr.s_addr = iph->saddr; 96 | 97 | in = (struct sockaddr_in *)&m->local_addr; 98 | in->sin_family = AF_INET; 99 | in->sin_port = udp->dest; 100 | in->sin_addr.s_addr = iph->daddr; 101 | } else { 102 | in6 = (struct sockaddr_in6 *)&m->peer_addr; 103 | in6->sin6_family = AF_INET6; 104 | in6->sin6_port = udp->source; 105 | in6->sin6_addr = iph6->saddr; 106 | 107 | in6 = (struct sockaddr_in6 *)&m->local_addr; 108 | in6->sin6_family = AF_INET6; 109 | in6->sin6_port = udp->dest; 110 | in6->sin6_addr = iph6->daddr; 111 | } 112 | 113 | m->frame = pkt; 114 | m->headroom = xsk->headroom; 115 | 116 | if (flags & XUDP_FLAG_COPY) { 117 | if (len < m->size) 118 | m->size = len; 119 | 120 | memcpy(m->p, body, m->size); 121 | } else { 122 | m->recycle1 = desc->addr; 123 | m->recycle2 = (u64)xsk; 124 | m->p = body; 125 | m->size = len; 126 | } 127 | 128 | return 0; 129 | } 130 | 131 | static int xudp_group_stats(struct xdpsock *tx_xsk, 132 | struct xudp_group *g, int fd) 133 | { 134 | struct xsk_stats_output stats, *s; 135 | struct xdp_statistics xdp_stats; 136 | struct xdpsock *xsk; 137 | socklen_t len; 138 | int ret; 139 | 140 | s = &stats; 141 | 142 | s->g_id = g->gid; 143 | s->ch_id = 0; 144 | s->group_num = __xudp_group_num(tx_xsk->x); 145 | 146 | xudp_group_channel_foreach(xsk, g) 147 | { 148 | s->is_tx = false; 149 | 150 | if (xudp_channel_is_tx(xsk)) { 151 | xsk = xsk->tx_xsk; 152 | s->is_tx = true; 153 | } 154 | 155 | memcpy(&s->stats, &xsk->stats, sizeof(xsk->stats)); 156 | memset(&xdp_stats, 0, sizeof(xdp_stats)); 157 | 158 | len = sizeof(xdp_stats); 159 | getsockopt(xsk->sfd, SOL_XDP, XDP_STATISTICS, &xdp_stats, &len); 160 | 161 | s->xsk_rx_dropped = xdp_stats.rx_dropped; 162 | s->xsk_rx_invalid_descs = xdp_stats.rx_invalid_descs; 163 | s->xsk_tx_invalid_descs = xdp_stats.tx_invalid_descs; 164 | 165 | s->kern_tx_num = 0; 166 | if (s->is_tx) { 167 | s->kern_tx_num = *xsk->ring.ring.producer - *xsk->ring.ring.consumer; 168 | } 169 | 170 | ret = send(fd, (void *)s, sizeof(*s), 0); 171 | if (ret < 0) { 172 | logerr(tx_xsk->x, "stats send. %s\n", strerror(errno)); 173 | break; 174 | } 175 | 176 | ++s->ch_id; 177 | } 178 | 179 | return 0; 180 | } 181 | 182 | static int xudp_stats_req_check(struct xdpsock *xsk, 183 | struct iphdr *iph, 184 | struct udphdr *udp) 185 | { 186 | struct sockaddr_in addr; 187 | int fd, err; 188 | 189 | if (iph->saddr != iph->daddr) 190 | return 0; 191 | 192 | fd = socket(AF_INET, SOCK_DGRAM, 0); 193 | if (fd < 0) 194 | return 1; 195 | 196 | addr.sin_family = AF_INET; 197 | addr.sin_addr.s_addr = inet_addr("127.0.0.1"); 198 | addr.sin_port = udp->source; 199 | 200 | err = connect(fd, (struct sockaddr *)&addr, (socklen_t)sizeof(addr)); 201 | if (err) { 202 | close(fd); 203 | return 1; 204 | } 205 | 206 | xudp_group_stats(xsk, xsk->group, fd); 207 | close(fd); 208 | return 1; 209 | } 210 | 211 | static int xudp_nic_recv_channel(struct xdpsock *xsk, xudp_msghdr *hdr, 212 | int flags) 213 | { 214 | struct xdp_desc descs[BATCH_SIZE]; 215 | unsigned int rcvd, i, bytes; 216 | struct pkthdrs hdrs = {}; 217 | int num, ret; 218 | xudp_msg *m; 219 | char *pkt; 220 | 221 | bytes = 0; 222 | 223 | num = MIN(sizeof(descs)/sizeof(descs[0]), hdr->total - hdr->used); 224 | 225 | rcvd = xq_deq(&xsk->ring, descs, num); 226 | if (!rcvd) { 227 | errno = EAGAIN; 228 | return -1; 229 | } 230 | 231 | for (i = 0; i < rcvd; i++) { 232 | if (i + 1 < rcvd) { 233 | pkt = xq_get_data(xsk, descs[i + 1].addr); 234 | __builtin_prefetch(pkt, 0, 3); 235 | } 236 | 237 | pkt = xq_get_data(xsk, descs[i].addr); 238 | 239 | ret = packet_parse(&hdrs, pkt, pkt + descs[i].len); 240 | 241 | if (ret < 0 || xudp_stats_req_check(xsk, hdrs.iph, hdrs.udp)) { 242 | if (!(flags & XUDP_FLAG_COPY)) 243 | umem_fq_put(xsk, &descs[i], 1); 244 | 245 | continue; 246 | } 247 | 248 | m = hdr->msg + hdr->used++; 249 | 250 | xudp_fill_msg(xsk, pkt, &descs[i], &hdrs, m, flags); 251 | 252 | dump_check(xsk->x, xsk->group, pkt, descs[i].len); 253 | 254 | bytes += m->size; 255 | } 256 | 257 | if (flags & XUDP_FLAG_COPY) 258 | umem_fq_put(xsk, descs, rcvd); 259 | 260 | xsk->stats.rx_npkts += rcvd; 261 | 262 | logdebug(xsk->x, 263 | "recv from %p %d patckets. bytes: %d\n", 264 | xsk, rcvd, bytes); 265 | 266 | return bytes; 267 | } 268 | 269 | int xudp_recv_channel(xudp_channel *ch, xudp_msghdr *hdr, int flags) 270 | { 271 | if (ch->istx) { 272 | errno = EAGAIN; 273 | return -1; 274 | } 275 | 276 | hdr->used = 0; 277 | return xudp_nic_recv_channel(ch, hdr, flags); 278 | } 279 | 280 | int xudp_recycle(xudp_msghdr *hdr) 281 | { 282 | struct xdpsock *xsk; 283 | xudp_msg *m; 284 | int i; 285 | struct xdp_desc d; 286 | 287 | for (i = 0; i < hdr->used; ++i) { 288 | m = hdr->msg + i; 289 | xsk = (void *)m->recycle2; 290 | 291 | d.addr = m->recycle1; 292 | umem_fq_put(xsk, &d, 1); 293 | } 294 | return 0; 295 | } 296 | 297 | 298 | -------------------------------------------------------------------------------- /group/dump.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include 15 | #include 16 | #include 17 | #include "group.h" 18 | #include "dump.h" 19 | 20 | 21 | static struct dump_ring *get_dump_ring(struct dump *d, struct xudp_group *g) 22 | { 23 | struct dump_ring *r; 24 | void *p; 25 | 26 | if (d->prepare) { 27 | if (g->dump) 28 | return g->dump; 29 | 30 | p = ((void *)d) + d->prepare; 31 | 32 | g->dump = p; 33 | 34 | return g->dump; 35 | } 36 | 37 | if (!g->dump) { 38 | at: 39 | p = shmat(d->shmid, NULL, 0); 40 | if ((void *)-1 == p) 41 | return NULL; 42 | 43 | g->dump = p; 44 | r = g->dump; 45 | } else { 46 | r = g->dump; 47 | 48 | if (r->shmid != d->shmid) { 49 | shmdt(g->dump); 50 | goto at; 51 | } 52 | } 53 | 54 | return r; 55 | } 56 | 57 | static void dump_pkt(struct dump_ring *r, char *pkt, int len) 58 | { 59 | struct dump_header header; 60 | struct timeval tv; 61 | int pkt_size, left; 62 | u64 pos; 63 | void *p; 64 | 65 | pkt_size = sizeof(struct dump_header) + len; 66 | pos = r->prod; 67 | 68 | if (pos + pkt_size - r->cons > r->size) { 69 | ++r->drop; 70 | return; 71 | } 72 | 73 | gettimeofday(&tv, NULL); 74 | 75 | header.len = len; 76 | header.tv_sec = tv.tv_sec; 77 | header.tv_usec = tv.tv_usec; 78 | 79 | pos = pos % r->size; 80 | p = r->pkt + pos; 81 | left = r->size - pos; 82 | 83 | if (left >= sizeof(header) + len) { 84 | memcpy(p, &header, sizeof(header)); 85 | memcpy(p + sizeof(header), pkt, len); 86 | 87 | } else if (left >= sizeof(header)) { 88 | memcpy(p, &header, sizeof(header)); 89 | 90 | left -= sizeof(header); 91 | 92 | memcpy(p + sizeof(header), pkt, left); 93 | memcpy(r->pkt, pkt + left, len - left); 94 | 95 | } else { 96 | memcpy(p, &header, left); 97 | 98 | memcpy(r->pkt, ((void *)&header) + left, sizeof(header) - left); 99 | memcpy(r->pkt + sizeof(header) - left, pkt, len); 100 | } 101 | 102 | u_smp_wmb(); 103 | 104 | r->prod += pkt_size; 105 | } 106 | 107 | void dump_free(struct dump *d, struct xudp_group *g) 108 | { 109 | struct dump_ring *r; 110 | 111 | if (d->prepare) { 112 | r = g->dump; 113 | madvise(r->pkt, r->size, MADV_DONTNEED); 114 | 115 | } else { 116 | shmdt(g->dump); 117 | } 118 | 119 | g->dump = NULL; 120 | } 121 | 122 | void dump(struct dump *d, struct xudp_group *g, char *pkt, int len) 123 | { 124 | struct dump_ring *r; 125 | 126 | r = get_dump_ring(d, g); 127 | if (!r) 128 | return; 129 | 130 | pthread_spin_lock(&r->lock); 131 | dump_pkt(r, pkt, len); 132 | pthread_spin_unlock(&r->lock); 133 | } 134 | 135 | -------------------------------------------------------------------------------- /group/group.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __GROUP_H__ 15 | #define __GROUP_H__ 16 | 17 | #include 18 | #include 19 | 20 | #include 21 | #include 22 | #include 23 | #include 24 | 25 | #include 26 | #include 27 | #include 28 | #include 29 | #include 30 | #include 31 | 32 | #include 33 | #include 34 | #include 35 | 36 | #include 37 | #include 38 | #include 39 | 40 | #include 41 | #include 42 | #include 43 | 44 | #include "list.h" 45 | #include "queue.h" 46 | #include "common.h" 47 | #include "channel.h" 48 | #include "config.h" 49 | #include "group_api.h" 50 | 51 | #undef logerr 52 | #define logerr(x, fmt, ...) \ 53 | if (LOG_LEVEL_ERR <= __xudp_log(x)->level) \ 54 | logcore(__xudp_log(x), fmt, ##__VA_ARGS__) 55 | 56 | #undef loginfo 57 | #define loginfo(x, fmt, ...) \ 58 | if (LOG_LEVEL_INFO <= __xudp_log(x)->level) \ 59 | logcore(__xudp_log(x), fmt, ##__VA_ARGS__) 60 | 61 | #undef logdebug 62 | #define logdebug(x, fmt, ...) \ 63 | if (LOG_LEVEL_DEBUG <= __xudp_log(x)->level) \ 64 | logcore(__xudp_log(x), fmt, ##__VA_ARGS__) 65 | 66 | struct xudp_group; 67 | 68 | struct rxch { 69 | struct xdpsock xsk; 70 | struct xudp_group *group; 71 | bool unaligned; 72 | }; 73 | 74 | struct xudp_group_nic { 75 | struct list_head list; 76 | int ifindex; 77 | int xsk_n; 78 | struct rxch rxch[0]; 79 | }; 80 | 81 | #endif 82 | 83 | 84 | -------------------------------------------------------------------------------- /include/channel.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __CHANNEL_H__ 15 | #define __CHANNEL_H__ 16 | 17 | #include "xudp.h" 18 | #include "list.h" 19 | #include "queue.h" 20 | #include "dump.h" 21 | 22 | struct xsk_stats { 23 | u64 send_ebusy; 24 | u64 send_again; 25 | u64 send_err; 26 | u64 send_success; 27 | 28 | u64 no_cq; 29 | u64 no_tx; 30 | 31 | u64 rx_npkts; 32 | u64 tx_npkts; 33 | }; 34 | 35 | struct xsk_stats_output { 36 | /* normal vals */ 37 | u64 nanosecond; 38 | 39 | int ch_id; 40 | int g_id; 41 | int group_num; 42 | int is_tx; 43 | 44 | /* stats */ 45 | struct xsk_stats stats; 46 | 47 | u64 xsk_rx_dropped; 48 | u64 xsk_rx_invalid_descs; 49 | u64 xsk_tx_invalid_descs; 50 | 51 | int kern_tx_num; 52 | }; 53 | 54 | struct xdpsock { 55 | struct list_head list; 56 | 57 | int sfd; 58 | 59 | /* group id*/ 60 | int gid; 61 | 62 | /* id of queue of nic */ 63 | int ifindex; 64 | int queue_id; 65 | bool istx; 66 | 67 | struct xdp_uqueue ring; 68 | 69 | struct xsk_stats stats; 70 | char *frames; 71 | 72 | struct xdp_umem_uqueue *fq; // used by rx 73 | 74 | int headroom; 75 | 76 | struct xdpsock *tx_xsk; 77 | 78 | xudp *x; 79 | 80 | struct xudp_group *group; 81 | }; 82 | 83 | struct xudp_group { 84 | xudp *x; 85 | int gid; 86 | struct list_head nics; 87 | int xsk_n; 88 | struct xdpsock *tx_xsk; 89 | 90 | int map_offset; 91 | void *dump; 92 | }; 93 | 94 | void dump(struct dump *d, struct xudp_group *g, char *pkt, int len); 95 | void dump_free(struct dump *d, struct xudp_group *g); 96 | 97 | static inline void dump_check(xudp *x, struct xudp_group *g, char *pkt, int len) 98 | { 99 | struct dump *d = (struct dump *)x; 100 | 101 | if (d->active == XUDP_DUMP_ACTIVE) { 102 | dump(d, g, pkt, len); 103 | } else { 104 | if (g->dump) 105 | dump_free(d, g); 106 | } 107 | } 108 | #endif 109 | 110 | 111 | -------------------------------------------------------------------------------- /include/common.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __COMMON_H__ 15 | #define __COMMON_H__ 16 | 17 | #include 18 | #include 19 | 20 | #define barrier() __asm__ __volatile__("": : :"memory") 21 | #ifdef __aarch64__ 22 | #define u_smp_rmb() __asm__ __volatile__("dmb ishld": : :"memory") 23 | #define u_smp_wmb() __asm__ __volatile__("dmb ishst": : :"memory") 24 | #else 25 | #define u_smp_rmb() barrier() 26 | #define u_smp_wmb() barrier() 27 | #endif 28 | 29 | #define WRITE_ONCE(x, val) (*((volatile typeof(x) *)(&(x))) = (val)); 30 | 31 | #define zobj(x) {\ 32 | x = malloc(sizeof(*x)); \ 33 | if (x) \ 34 | memset(x, 0, sizeof(*x)); \ 35 | } 36 | 37 | #define anon_map(size) mmap(NULL, size, \ 38 | PROT_READ|PROT_WRITE, \ 39 | MAP_SHARED | MAP_ANONYMOUS | \ 40 | MAP_POPULATE | MAP_LOCKED , \ 41 | 0, 0); 42 | 43 | #define ARRAY_SIZE(a) (sizeof(a)/sizeof(a[0])) 44 | 45 | #define likely(x) __builtin_expect(!!(x), 1) 46 | #define unlikely(x) __builtin_expect(!!(x), 0) 47 | 48 | #define MIN(a, b) (a > b ? b: a) 49 | 50 | #define MTU 1500 51 | #define PAGE_SIZE (4 * 1024) 52 | 53 | #define atomic_inc(v) __sync_add_and_fetch(v, 1) 54 | #define atomic_dec(v) __sync_sub_and_fetch(v, 1) 55 | #define xchg(p, o, n) __sync_bool_compare_and_swap(p, o, n) 56 | 57 | #define BUILD_BUG_ON(condition) ((void)sizeof(char[1 - 2*!!(condition)])) 58 | 59 | static inline __attribute__((const)) bool is_power_of_2(unsigned long n) 60 | { 61 | return (n != 0 && ((n & (n - 1)) == 0)); 62 | } 63 | 64 | static inline unsigned int get_power_of_2(unsigned int n) 65 | { 66 | unsigned int i, t; 67 | if (!is_power_of_2(n)) { 68 | for (i = 1; i < 31; ++i) { 69 | t = ~((1 << i) - 1); 70 | if ((t & n) == 0) { 71 | return 1 << i; 72 | } 73 | } 74 | } 75 | return n; 76 | } 77 | 78 | static inline int align(int value, int ag) 79 | { 80 | if (value % ag) 81 | return (value / ag + 1) * ag; 82 | 83 | return value; 84 | } 85 | 86 | #endif 87 | 88 | 89 | -------------------------------------------------------------------------------- /include/config.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __CONFIG_H__ 15 | #define __CONFIG_H__ 16 | 17 | #ifdef BPF_XDP 18 | #define CONF_BPF_LINK 1 19 | #else 20 | #define CONF_BPF_LINK 0 21 | #endif 22 | 23 | #define BATCH_SIZE 1024 24 | 25 | #endif 26 | 27 | 28 | -------------------------------------------------------------------------------- /include/dump.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __DUMP_H__ 15 | #define __DUMP_H__ 16 | 17 | #include 18 | #include 19 | #include 20 | 21 | #define XUDP_SHM_OFFSET 64 22 | #define XUDP_SHM_MAGIC "libxudp-shm-magic" 23 | 24 | typedef __u64 u64; 25 | typedef __u32 u32; 26 | typedef __u16 u16; 27 | typedef __u8 u8; 28 | 29 | enum { 30 | XUDP_DUMP_NOACTIVE, 31 | XUDP_DUMP_ACTIVE, 32 | }; 33 | 34 | struct dump { 35 | u64 active; 36 | u64 shmid; 37 | u64 prepare; 38 | }; 39 | 40 | struct dump_ring { 41 | pthread_spinlock_t lock; 42 | u64 size; 43 | u64 shmid; 44 | 45 | u64 prod; 46 | u64 cons; 47 | 48 | u64 drop; 49 | 50 | char pkt[]; 51 | }; 52 | 53 | struct dump_header { 54 | int len; 55 | u32 tv_sec; 56 | u32 tv_usec; 57 | }; 58 | 59 | #endif 60 | 61 | 62 | -------------------------------------------------------------------------------- /include/group_api.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __GROUP_API_H__ 15 | #define __GROUP_API_H__ 16 | 17 | #include "log.h" 18 | 19 | int __xudp_group_num(xudp *x); 20 | int __xudp_group_unaligned(xudp *x); 21 | int __xudp_nic_num(xudp *x); 22 | int __xudp_nic_queue_num(xudp *x, int nicid); 23 | struct log *__xudp_log(xudp *x); 24 | 25 | int __xudp_xsk_ring_setup(xudp *x, struct ring *r, int sfd); 26 | struct kern_info *__xudp_kern_info(xudp *x); 27 | int __xudp_nic_ifindex(xudp *x, int nicid); 28 | int __xudp_nic_queue_num(xudp *x, int ifindex); 29 | int __xudp_xsk_bind(xudp *x, int ifindex, int queue_id, int sfd); 30 | char *__xudp_umem_get_frames(xudp *x, int ifindex, int queue_id); 31 | struct xdp_umem_uqueue *__xudp_umem_get_fq(xudp *x, int ifindex, int queue_id); 32 | int __xudp_umem_get_headroom(xudp *x, int ifindex, int queue_id); 33 | int __xudp_get_bpf_map(xudp *x, char *name); 34 | struct log *__xudp_log(xudp *x); 35 | int __xudp_kern_xsk_alloc(xudp *x, int num, int *offset); 36 | int __xudp_kern_xsk_set(xudp *x, int offset, int sfd, 37 | int ifindex, int queue_id, int gid); 38 | int __xudp_dict_active(xudp *x); 39 | 40 | void __xudp_xsk_free(struct xdpsock *xsk); 41 | 42 | xudp_channel *xudp_group_get_tx(xudp_group *g); 43 | struct xdpsock *__xudp_rx_ref_tx(struct xdpsock *tx); 44 | #endif 45 | 46 | 47 | -------------------------------------------------------------------------------- /include/ip6.h: -------------------------------------------------------------------------------- 1 | #ifndef __IP6_H__ 2 | #define __IP6_H__ 3 | 4 | #include 5 | #include 6 | 7 | #define IPV6_SIZE 16 8 | 9 | typedef __u16 u16; 10 | typedef __u32 u32; 11 | typedef __u64 u64; 12 | 13 | typedef unsigned char u8; 14 | 15 | /* 16 | * pl: prefix len 17 | * size: sub size 18 | * 19 | * pl, size MUST bee the multiples of 8 20 | * */ 21 | static inline u32 ip6_slice(struct in6_addr *ip6, u8 pl, u8 size) 22 | { 23 | u32 v = 0; 24 | u8 *ip; 25 | 26 | ip = ip6->s6_addr + (pl >> 3); 27 | 28 | for (; size; size = size - 8) { 29 | v = (v << 8) + *ip; 30 | ++ip; 31 | } 32 | 33 | return v; 34 | } 35 | 36 | static inline void ip6_cpy(struct in6_addr *dst, struct in6_addr *src) 37 | { 38 | memcpy(dst->s6_addr, src->s6_addr, IPV6_SIZE); 39 | } 40 | 41 | static inline void ip6_copy_prefix(struct in6_addr *dst, 42 | struct in6_addr *src, u8 size) 43 | { 44 | u8 off, l, m; 45 | 46 | off = size / 8; 47 | l = size % 8; 48 | 49 | memcpy(dst->s6_addr, src->s6_addr, off); 50 | 51 | if (l) { 52 | m = 0xFF; 53 | m = ~(m >> l); 54 | dst->s6_addr[off + 1] = src->s6_addr[off + 1] & m; 55 | } 56 | } 57 | 58 | static inline bool ip6_cmp(struct in6_addr *dst, struct in6_addr *src, u8 size) 59 | { 60 | u8 *d1, *d2; 61 | u8 t1, t2; 62 | u8 s; 63 | 64 | d1 = dst->s6_addr; 65 | d2 = src->s6_addr; 66 | 67 | #define _v6_prefix_cmp(s) { \ 68 | if (*(u##s *)d1 != *(u##s *)d2) \ 69 | return false; \ 70 | d1 += s / 8; \ 71 | d2 += s / 8; \ 72 | } 73 | 74 | #define _cmp8() _v6_prefix_cmp(8) 75 | #define _cmp16() _v6_prefix_cmp(16) 76 | #define _cmp24() _cmp16();_cmp8() 77 | #define _cmp32() _v6_prefix_cmp(32) 78 | #define _cmp64() _v6_prefix_cmp(64) 79 | 80 | s = size >> 3; 81 | 82 | switch(s) { 83 | case 0: break; 84 | case 1: _cmp8(); break; 85 | case 2: _cmp16(); break; 86 | case 3: _cmp24(); break; 87 | case 4: _cmp32(); break; 88 | case 5: _cmp32(); _cmp8(); break; 89 | case 6: _cmp32(); _cmp16(); break; 90 | case 7: _cmp32(); _cmp24(); break; 91 | case 8: _cmp64(); break; 92 | case 9: _cmp64(); _cmp8(); break; 93 | case 10: _cmp64(); _cmp16(); break; 94 | case 11: _cmp64(); _cmp24(); break; 95 | case 12: _cmp64(); _cmp32(); break; 96 | case 13: _cmp64(); _cmp32(); _cmp8(); break; 97 | case 14: _cmp64(); _cmp32(); _cmp16(); break; 98 | case 15: _cmp64(); _cmp32(); _cmp24(); break; 99 | case 16: _cmp64(); _cmp64(); return true; 100 | }; 101 | 102 | s = size & 0x7; 103 | 104 | if (s) { 105 | t1 = (*d1) >> (8 - s); 106 | t2 = (*d2) >> (8 - s); 107 | return t1 == t2; 108 | } 109 | 110 | return true; 111 | } 112 | 113 | static inline bool ip6_eq(struct in6_addr *dst, struct in6_addr *src) 114 | { 115 | return ip6_cmp(dst, src, 128); 116 | } 117 | 118 | static inline bool ip6_is_zero(struct in6_addr *dst) 119 | { 120 | struct in6_addr p = {}; 121 | 122 | return ip6_eq(dst, &p); 123 | } 124 | 125 | #endif 126 | 127 | 128 | -------------------------------------------------------------------------------- /include/list.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef _LIST_H 15 | #define _LIST_H 16 | 17 | #include 18 | #include "common.h" 19 | 20 | struct list_head { 21 | struct list_head *next; 22 | struct list_head *prev; 23 | }; 24 | 25 | #define LIST_POISON1 NULL 26 | #define LIST_POISON2 NULL 27 | 28 | #ifndef offsetof 29 | #ifdef __compiler_offsetof 30 | #define offsetof(TYPE, MEMBER) __compiler_offsetof(TYPE, MEMBER) 31 | #else 32 | #define offsetof(TYPE, MEMBER) ((size_t)&((TYPE *)0)->MEMBER) 33 | #endif 34 | #endif 35 | 36 | #define container_of(ptr, type, member) ({ \ 37 | void *__mptr = (void *)(ptr); \ 38 | ((type *)(__mptr - offsetof(type, member))); }) 39 | 40 | static inline void INIT_LIST_HEAD(struct list_head *list) 41 | { 42 | WRITE_ONCE(list->next, list); 43 | list->prev = list; 44 | } 45 | 46 | static inline void list_insert_between(struct list_head *new, 47 | struct list_head *prev, 48 | struct list_head *next) 49 | { 50 | next->prev = new; 51 | new->next = next; 52 | new->prev = prev; 53 | WRITE_ONCE(prev->next, new); 54 | } 55 | 56 | /* insert as the first item of the list */ 57 | static inline void list_insert(struct list_head *head, struct list_head *new) 58 | { 59 | list_insert_between(new, head, head->next); 60 | } 61 | 62 | /* insert before the item */ 63 | static inline void list_insert_before(struct list_head *item, struct list_head *new) 64 | { 65 | list_insert_between(new, item->prev, item); 66 | } 67 | 68 | /* insert as the last item of the list */ 69 | static inline void list_append(struct list_head *head, struct list_head *new) 70 | { 71 | list_insert_between(new, head->prev, head); 72 | } 73 | 74 | static inline void __list_del(struct list_head *prev, struct list_head *next) 75 | { 76 | next->prev = prev; 77 | WRITE_ONCE(prev->next, next); 78 | } 79 | 80 | static inline void __list_del_entry(struct list_head *entry) 81 | { 82 | __list_del(entry->prev, entry->next); 83 | } 84 | 85 | static inline void list_del(struct list_head *entry) 86 | { 87 | __list_del_entry(entry); 88 | entry->next = LIST_POISON1; 89 | entry->prev = LIST_POISON2; 90 | } 91 | 92 | #define list_entry(ptr, type, member) container_of(ptr, type, member) 93 | 94 | #define list_first_entry(ptr, type, member) \ 95 | list_entry((ptr)->next, type, member) 96 | 97 | #define list_next_entry(pos, member) \ 98 | list_entry((pos)->member.next, typeof(*(pos)), member) 99 | 100 | #define list_for_each_entry(pos, head, member) \ 101 | for (pos = list_first_entry(head, typeof(*pos), member); \ 102 | &pos->member != (head); \ 103 | pos = list_next_entry(pos, member)) 104 | 105 | #define list_for_each_entry_safe(pos, n, head, member) \ 106 | for (pos = list_first_entry(head, typeof(*pos), member), \ 107 | n = list_next_entry(pos, member); \ 108 | &pos->member != (head); \ 109 | pos = n, n = list_next_entry(n, member)) 110 | 111 | #endif 112 | -------------------------------------------------------------------------------- /include/packet_parse.h: -------------------------------------------------------------------------------- 1 | #ifndef __PACKET_PARSE_H__ 2 | #define __PACKET_PARSE_H__ 3 | 4 | // this cannot be too big, too big will make unroll fail 5 | #define MAX_IPV6_OPT 8 6 | 7 | struct pkthdrs { 8 | int family; 9 | struct ethhdr *eth; 10 | union { 11 | struct iphdr *iph; 12 | struct ipv6hdr *iph6; 13 | }; 14 | struct udphdr *udp; 15 | }; 16 | 17 | #define access_obj_ok(obj, end) ((void *)(obj + 1) <= end) 18 | 19 | #define NEXTHDR_HOP 0 /* Hop-by-hop option header. */ 20 | #define NEXTHDR_TCP 6 /* TCP segment. */ 21 | #define NEXTHDR_UDP 17 /* UDP message. */ 22 | #define NEXTHDR_IPV6 41 /* IPv6 in IPv6 */ 23 | #define NEXTHDR_ROUTING 43 /* Routing header. */ 24 | #define NEXTHDR_FRAGMENT 44 /* Fragmentation/reassembly header. */ 25 | #define NEXTHDR_GRE 47 /* GRE header. */ 26 | #define NEXTHDR_ESP 50 /* Encapsulating security payload. */ 27 | #define NEXTHDR_AUTH 51 /* Authentication header. */ 28 | #define NEXTHDR_ICMP 58 /* ICMP for IPv6. */ 29 | #define NEXTHDR_NONE 59 /* No next header */ 30 | #define NEXTHDR_DEST 60 /* Destination options header. */ 31 | #define NEXTHDR_SCTP 132 /* SCTP message. */ 32 | #define NEXTHDR_MOBILITY 135 /* Mobility header. */ 33 | 34 | #define NEXTHDR_MAX 255 35 | 36 | #define ipv6_optlen(p) (((p)->hdrlen+1) << 3) 37 | #define ipv6_authlen(p) (((p)->hdrlen+2) << 2) 38 | 39 | #define check_opt_hdr() \ 40 | do { \ 41 | \ 42 | } while(0); \ 43 | 44 | static inline struct udphdr *parse_ipv6(struct ipv6hdr *iph6, void *end) 45 | { 46 | struct ipv6_opt_hdr *oh; 47 | u8 nexthdr; 48 | void *pos; 49 | int ol, i; 50 | 51 | pos = (void *)(iph6 + 1); 52 | nexthdr = iph6->nexthdr; 53 | 54 | #if defined(__clang__) 55 | #pragma unroll 56 | #endif 57 | for (i = 0; i < MAX_IPV6_OPT; ++i) { 58 | oh = (struct ipv6_opt_hdr *)pos; 59 | 60 | if (!access_obj_ok(oh, end)) 61 | return NULL; 62 | 63 | switch(nexthdr) { 64 | case NEXTHDR_SCTP: 65 | case NEXTHDR_ICMP: 66 | case NEXTHDR_NONE: 67 | case NEXTHDR_TCP: 68 | case NEXTHDR_IPV6: 69 | return NULL; 70 | 71 | case NEXTHDR_UDP: 72 | return (void *)(iph6 + 1); 73 | 74 | case NEXTHDR_AUTH: 75 | ol = ipv6_authlen(oh); 76 | break; 77 | 78 | case NEXTHDR_FRAGMENT: 79 | ol = 8; 80 | break; 81 | 82 | case NEXTHDR_HOP: 83 | case NEXTHDR_ROUTING: 84 | case NEXTHDR_GRE: 85 | case NEXTHDR_ESP: 86 | case NEXTHDR_DEST: 87 | case NEXTHDR_MOBILITY: 88 | ol = ipv6_optlen(oh); 89 | break; 90 | default: 91 | return NULL; 92 | } 93 | 94 | nexthdr = oh->nexthdr; 95 | pos += ol; 96 | } 97 | 98 | return NULL; 99 | } 100 | 101 | static inline int packet_parse(struct pkthdrs *hdrs, void *pkt, void *end) 102 | { 103 | struct ethhdr *eth; 104 | struct iphdr *iph; 105 | struct ipv6hdr *iph6; 106 | struct udphdr *udp; 107 | u8 *p; 108 | 109 | eth = pkt; 110 | 111 | if (!access_obj_ok(eth, end)) 112 | return 0; 113 | 114 | p = (u8 *)ð->h_proto; 115 | 116 | // ipv4 117 | if (p[0] == 0x08 || p[1] == 0x0) { 118 | iph = pkt + sizeof(*eth); 119 | if (!access_obj_ok(iph, end)) 120 | return 0; 121 | 122 | if (iph->protocol != IPPROTO_UDP) 123 | return 0; 124 | 125 | if (5 == iph->ihl) { 126 | udp = (void *)iph + 20; 127 | } else { 128 | udp = (void*)iph + (iph->ihl << 2); 129 | 130 | if ((void *)(udp + 1) > end) 131 | return 0; 132 | } 133 | 134 | if (!access_obj_ok(udp, end)) 135 | return 0; 136 | 137 | hdrs->eth = eth; 138 | hdrs->iph = iph; 139 | hdrs->udp = udp; 140 | hdrs->family = AF_INET; 141 | return 1; 142 | } 143 | 144 | // ipv6 145 | if (p[0] == 0x86 && p[1] == 0xDD) { 146 | iph6 = pkt + sizeof(*eth); 147 | if (!access_obj_ok(iph6, end)) 148 | return 0; 149 | 150 | udp = parse_ipv6(iph6, end); 151 | if (!udp) 152 | return 0; 153 | 154 | if (!access_obj_ok(udp, end)) 155 | return 0; 156 | 157 | hdrs->eth = eth; 158 | hdrs->iph6 = iph6; 159 | hdrs->udp = udp; 160 | hdrs->family = AF_INET6; 161 | return 1; 162 | } 163 | 164 | return 0; 165 | } 166 | 167 | #endif 168 | 169 | 170 | -------------------------------------------------------------------------------- /include/queue.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __QUEUE_H__ 15 | #define __QUEUE_H__ 16 | 17 | #include 18 | #include 19 | 20 | #define umem_lock(x) pthread_spin_lock(&(x)->lock) 21 | #define umem_unlock(x) pthread_spin_unlock(&(x)->lock) 22 | 23 | typedef __u64 u64; 24 | typedef __u32 u32; 25 | typedef __u16 u16; 26 | typedef __u8 u8; 27 | 28 | struct ring { 29 | u32 mask; 30 | u32 size; 31 | 32 | u32 cached_prod; 33 | u32 cached_cons; 34 | 35 | u32 *producer; 36 | u32 *consumer; 37 | u64 *flags; 38 | 39 | void *map; 40 | u64 map_size; 41 | 42 | union { 43 | u64 *addr; 44 | struct xdp_desc *desc; 45 | }; 46 | }; 47 | 48 | struct xdp_umem_uqueue { 49 | struct ring ring; 50 | 51 | pthread_spinlock_t lock; 52 | }; 53 | 54 | struct xdp_uqueue { 55 | struct ring ring; 56 | }; 57 | 58 | /* umem fq, cq options */ 59 | 60 | static inline u32 ring_free(struct ring *r, u32 n) 61 | { 62 | u32 free_entries = r->cached_cons - r->cached_prod; 63 | 64 | if (free_entries >= n) 65 | return free_entries; 66 | 67 | /* Refresh the local tail pointer */ 68 | r->cached_cons = *r->consumer + r->size; 69 | 70 | return r->cached_cons - r->cached_prod; 71 | } 72 | 73 | static inline u32 ring_avali(struct ring *r, u32 n) 74 | { 75 | u32 entries = r->cached_prod - r->cached_cons; 76 | 77 | if (entries == 0) { 78 | r->cached_prod = *r->producer; 79 | entries = r->cached_prod - r->cached_cons; 80 | } 81 | 82 | return (entries > n) ? n : entries; 83 | } 84 | 85 | static inline u32 ring_left(struct ring *r) 86 | { 87 | return *r->producer - *r->consumer; 88 | } 89 | 90 | static inline void ring_update_produce(struct ring *r) 91 | { 92 | u_smp_wmb(); 93 | *r->producer = r->cached_prod; 94 | } 95 | 96 | static inline void ring_update_consume(struct ring *r) 97 | { 98 | u_smp_wmb(); 99 | *r->consumer = r->cached_cons; 100 | } 101 | 102 | static inline u32 ring_next_con(struct ring *r) 103 | { 104 | return r->cached_cons++ & r->mask; 105 | } 106 | 107 | static inline u32 ring_next_pro(struct ring *r) 108 | { 109 | return r->cached_prod++ & r->mask; 110 | } 111 | 112 | static inline int fq_enq_nolock(struct xdp_umem_uqueue *fq, 113 | struct xdp_desc *d, 114 | size_t nb, u32 headroom) 115 | { 116 | u32 i, idx; 117 | struct ring *r = &fq->ring; 118 | 119 | if (ring_free(r, nb) < nb) 120 | /* that will not happen */ 121 | return -ENOSPC; 122 | 123 | for (i = 0; i < nb; i++) { 124 | idx = ring_next_pro(r); 125 | fq->ring.addr[idx] = d[i].addr - headroom; 126 | } 127 | 128 | ring_update_produce(r); 129 | 130 | return 0; 131 | } 132 | 133 | static inline int fq_enq(struct xdp_umem_uqueue *fq, 134 | struct xdp_desc *d, 135 | size_t nb, u32 headroom) 136 | { 137 | int ret; 138 | umem_lock(fq); 139 | ret = fq_enq_nolock(fq, d, nb, headroom); 140 | umem_unlock(fq); 141 | return ret; 142 | } 143 | 144 | static inline size_t cq_deq(struct xdp_umem_uqueue *cq, 145 | u64 *d, size_t nb) 146 | { 147 | u32 idx, i, entries; 148 | struct ring *r = &cq->ring; 149 | 150 | umem_lock(cq); 151 | 152 | entries = ring_avali(r, nb); 153 | 154 | u_smp_rmb(); 155 | 156 | if (entries > 0) { 157 | for (i = 0; i < entries; i++) { 158 | idx = ring_next_con(r); 159 | d[i] = cq->ring.addr[idx]; 160 | } 161 | 162 | ring_update_consume(r); 163 | } 164 | 165 | umem_unlock(cq); 166 | 167 | return entries; 168 | } 169 | 170 | static inline int xq_enq(struct xdp_uqueue *uq, 171 | const struct xdp_desc *descs, 172 | unsigned int ndescs) 173 | { 174 | struct ring *r = &uq->ring; 175 | unsigned int i, idx; 176 | 177 | if (ring_free(r, ndescs) < ndescs) 178 | return -ENOSPC; 179 | 180 | for (i = 0; i < ndescs; i++) { 181 | idx = ring_next_pro(r); 182 | 183 | r->desc[idx].addr = descs[i].addr; 184 | r->desc[idx].len = descs[i].len; 185 | } 186 | 187 | ring_update_produce(r); 188 | return 0; 189 | } 190 | 191 | static inline int xq_deq(struct xdp_uqueue *uq, 192 | struct xdp_desc *descs, 193 | int ndescs) 194 | { 195 | struct ring *r = &uq->ring; 196 | unsigned int idx; 197 | int i, entries; 198 | 199 | entries = ring_avali(r, ndescs); 200 | 201 | u_smp_rmb(); 202 | 203 | for (i = 0; i < entries; i++) { 204 | idx = ring_next_con(r); 205 | descs[i] = r->desc[idx]; 206 | } 207 | 208 | if (entries > 0) 209 | ring_update_consume(r); 210 | 211 | return entries; 212 | } 213 | 214 | #endif 215 | 216 | 217 | -------------------------------------------------------------------------------- /kern/Makefile: -------------------------------------------------------------------------------- 1 | BPF_CFLAGS += -I.. 2 | BPF_CFLAGS += -I../include 3 | BPF_CFLAGS += -I. 4 | BPF_CFLAGS += -D__KERNEL__ -D__BPF_TRACING__ 5 | BPF_CFLAGS += -emit-llvm -c 6 | BPF_CFLAGS += -O2 -g -Wall 7 | 8 | LLC_FLAGS += -O2 9 | 10 | #BPF_CFLAGS += -Wno-unused-value 11 | BPF_CFLAGS += -Wno-unused-function 12 | #BPF_CFLAGS += -Wno-pointer-sign 13 | #BPF_CFLAGS += -Wno-compare-distinct-pointer-types 14 | #BPF_CFLAGS += -Wno-gnu-variable-sized-type-not-at-end 15 | #BPF_CFLAGS += -Wno-address-of-packed-member 16 | #BPF_CFLAGS += -Wno-tautological-compare 17 | #BPF_CFLAGS += -Wno-unknown-warning-option 18 | 19 | ifeq ($(XUDP_DEBUG),1) 20 | BPF_CFLAGS += -DXUDP_DEBUG=1 21 | endif 22 | 23 | build=../objs/kern 24 | srcdir=./ 25 | 26 | .PRECIOUS: $(build)/kern_%.o 27 | 28 | $(build)/kern_%.o: $(srcdir)/dispatch_%.c 29 | @mkdir $(build) 2>/dev/null || true 30 | @echo clang $(shell basename $@.bc) 31 | @clang $(BPF_CFLAGS) $^ -o $@.bc 32 | @echo 'llc ' $(shell basename $@) 33 | @llc -march=bpf $(LLC_FLAGS) -filetype=obj $@.bc -o $@ 34 | 35 | $(build)/em_kern_%.o: $(build)/kern_%.o 36 | @echo 'LD ' $(shell basename $@) 37 | @cd $(build); ld -r -b binary $(shell basename $<) -o $(shell basename $@) 38 | 39 | 40 | clean: 41 | rm -rf $(build) 42 | -------------------------------------------------------------------------------- /kern/bpf_endian.h: -------------------------------------------------------------------------------- 1 | /* SPDX-License-Identifier: (LGPL-2.1 OR BSD-2-Clause) */ 2 | #ifndef __BPF_ENDIAN__ 3 | #define __BPF_ENDIAN__ 4 | 5 | /* 6 | * Isolate byte #n and put it into byte #m, for __u##b type. 7 | * E.g., moving byte #6 (nnnnnnnn) into byte #1 (mmmmmmmm) for __u64: 8 | * 1) xxxxxxxx nnnnnnnn xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx mmmmmmmm xxxxxxxx 9 | * 2) nnnnnnnn xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx mmmmmmmm xxxxxxxx 00000000 10 | * 3) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 nnnnnnnn 11 | * 4) 00000000 00000000 00000000 00000000 00000000 00000000 nnnnnnnn 00000000 12 | */ 13 | #define ___bpf_mvb(x, b, n, m) ((__u##b)(x) << (b-(n+1)*8) >> (b-8) << (m*8)) 14 | 15 | #define ___bpf_swab16(x) ((__u16)( \ 16 | ___bpf_mvb(x, 16, 0, 1) | \ 17 | ___bpf_mvb(x, 16, 1, 0))) 18 | 19 | #define ___bpf_swab32(x) ((__u32)( \ 20 | ___bpf_mvb(x, 32, 0, 3) | \ 21 | ___bpf_mvb(x, 32, 1, 2) | \ 22 | ___bpf_mvb(x, 32, 2, 1) | \ 23 | ___bpf_mvb(x, 32, 3, 0))) 24 | 25 | #define ___bpf_swab64(x) ((__u64)( \ 26 | ___bpf_mvb(x, 64, 0, 7) | \ 27 | ___bpf_mvb(x, 64, 1, 6) | \ 28 | ___bpf_mvb(x, 64, 2, 5) | \ 29 | ___bpf_mvb(x, 64, 3, 4) | \ 30 | ___bpf_mvb(x, 64, 4, 3) | \ 31 | ___bpf_mvb(x, 64, 5, 2) | \ 32 | ___bpf_mvb(x, 64, 6, 1) | \ 33 | ___bpf_mvb(x, 64, 7, 0))) 34 | 35 | /* LLVM's BPF target selects the endianness of the CPU 36 | * it compiles on, or the user specifies (bpfel/bpfeb), 37 | * respectively. The used __BYTE_ORDER__ is defined by 38 | * the compiler, we cannot rely on __BYTE_ORDER from 39 | * libc headers, since it doesn't reflect the actual 40 | * requested byte order. 41 | * 42 | * Note, LLVM's BPF target has different __builtin_bswapX() 43 | * semantics. It does map to BPF_ALU | BPF_END | BPF_TO_BE 44 | * in bpfel and bpfeb case, which means below, that we map 45 | * to cpu_to_be16(). We could use it unconditionally in BPF 46 | * case, but better not rely on it, so that this header here 47 | * can be used from application and BPF program side, which 48 | * use different targets. 49 | */ 50 | #if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ 51 | # define __bpf_ntohs(x) __builtin_bswap16(x) 52 | # define __bpf_htons(x) __builtin_bswap16(x) 53 | # define __bpf_constant_ntohs(x) ___bpf_swab16(x) 54 | # define __bpf_constant_htons(x) ___bpf_swab16(x) 55 | # define __bpf_ntohl(x) __builtin_bswap32(x) 56 | # define __bpf_htonl(x) __builtin_bswap32(x) 57 | # define __bpf_constant_ntohl(x) ___bpf_swab32(x) 58 | # define __bpf_constant_htonl(x) ___bpf_swab32(x) 59 | # define __bpf_be64_to_cpu(x) __builtin_bswap64(x) 60 | # define __bpf_cpu_to_be64(x) __builtin_bswap64(x) 61 | # define __bpf_constant_be64_to_cpu(x) ___bpf_swab64(x) 62 | # define __bpf_constant_cpu_to_be64(x) ___bpf_swab64(x) 63 | #elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ 64 | # define __bpf_ntohs(x) (x) 65 | # define __bpf_htons(x) (x) 66 | # define __bpf_constant_ntohs(x) (x) 67 | # define __bpf_constant_htons(x) (x) 68 | # define __bpf_ntohl(x) (x) 69 | # define __bpf_htonl(x) (x) 70 | # define __bpf_constant_ntohl(x) (x) 71 | # define __bpf_constant_htonl(x) (x) 72 | # define __bpf_be64_to_cpu(x) (x) 73 | # define __bpf_cpu_to_be64(x) (x) 74 | # define __bpf_constant_be64_to_cpu(x) (x) 75 | # define __bpf_constant_cpu_to_be64(x) (x) 76 | #else 77 | # error "Fix your compiler's __BYTE_ORDER__?!" 78 | #endif 79 | 80 | #define bpf_htons(x) \ 81 | (__builtin_constant_p(x) ? \ 82 | __bpf_constant_htons(x) : __bpf_htons(x)) 83 | #define bpf_ntohs(x) \ 84 | (__builtin_constant_p(x) ? \ 85 | __bpf_constant_ntohs(x) : __bpf_ntohs(x)) 86 | #define bpf_htonl(x) \ 87 | (__builtin_constant_p(x) ? \ 88 | __bpf_constant_htonl(x) : __bpf_htonl(x)) 89 | #define bpf_ntohl(x) \ 90 | (__builtin_constant_p(x) ? \ 91 | __bpf_constant_ntohl(x) : __bpf_ntohl(x)) 92 | #define bpf_cpu_to_be64(x) \ 93 | (__builtin_constant_p(x) ? \ 94 | __bpf_constant_cpu_to_be64(x) : __bpf_cpu_to_be64(x)) 95 | #define bpf_be64_to_cpu(x) \ 96 | (__builtin_constant_p(x) ? \ 97 | __bpf_constant_be64_to_cpu(x) : __bpf_be64_to_cpu(x)) 98 | 99 | #endif /* __BPF_ENDIAN__ */ 100 | #ifndef __BPF_ENDIAN__ 101 | #define __BPF_ENDIAN__ 102 | 103 | /* 104 | * Isolate byte #n and put it into byte #m, for __u##b type. 105 | * E.g., moving byte #6 (nnnnnnnn) into byte #1 (mmmmmmmm) for __u64: 106 | * 1) xxxxxxxx nnnnnnnn xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx mmmmmmmm xxxxxxxx 107 | * 2) nnnnnnnn xxxxxxxx xxxxxxxx xxxxxxxx xxxxxxxx mmmmmmmm xxxxxxxx 00000000 108 | * 3) 00000000 00000000 00000000 00000000 00000000 00000000 00000000 nnnnnnnn 109 | * 4) 00000000 00000000 00000000 00000000 00000000 00000000 nnnnnnnn 00000000 110 | */ 111 | #define ___bpf_mvb(x, b, n, m) ((__u##b)(x) << (b-(n+1)*8) >> (b-8) << (m*8)) 112 | 113 | #define ___bpf_swab16(x) ((__u16)( \ 114 | ___bpf_mvb(x, 16, 0, 1) | \ 115 | ___bpf_mvb(x, 16, 1, 0))) 116 | 117 | #define ___bpf_swab32(x) ((__u32)( \ 118 | ___bpf_mvb(x, 32, 0, 3) | \ 119 | ___bpf_mvb(x, 32, 1, 2) | \ 120 | ___bpf_mvb(x, 32, 2, 1) | \ 121 | ___bpf_mvb(x, 32, 3, 0))) 122 | 123 | #define ___bpf_swab64(x) ((__u64)( \ 124 | ___bpf_mvb(x, 64, 0, 7) | \ 125 | ___bpf_mvb(x, 64, 1, 6) | \ 126 | ___bpf_mvb(x, 64, 2, 5) | \ 127 | ___bpf_mvb(x, 64, 3, 4) | \ 128 | ___bpf_mvb(x, 64, 4, 3) | \ 129 | ___bpf_mvb(x, 64, 5, 2) | \ 130 | ___bpf_mvb(x, 64, 6, 1) | \ 131 | ___bpf_mvb(x, 64, 7, 0))) 132 | 133 | /* LLVM's BPF target selects the endianness of the CPU 134 | * it compiles on, or the user specifies (bpfel/bpfeb), 135 | * respectively. The used __BYTE_ORDER__ is defined by 136 | * the compiler, we cannot rely on __BYTE_ORDER from 137 | * libc headers, since it doesn't reflect the actual 138 | * requested byte order. 139 | * 140 | * Note, LLVM's BPF target has different __builtin_bswapX() 141 | * semantics. It does map to BPF_ALU | BPF_END | BPF_TO_BE 142 | * in bpfel and bpfeb case, which means below, that we map 143 | * to cpu_to_be16(). We could use it unconditionally in BPF 144 | * case, but better not rely on it, so that this header here 145 | * can be used from application and BPF program side, which 146 | * use different targets. 147 | */ 148 | #if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__ 149 | # define __bpf_ntohs(x) __builtin_bswap16(x) 150 | # define __bpf_htons(x) __builtin_bswap16(x) 151 | # define __bpf_constant_ntohs(x) ___bpf_swab16(x) 152 | # define __bpf_constant_htons(x) ___bpf_swab16(x) 153 | # define __bpf_ntohl(x) __builtin_bswap32(x) 154 | # define __bpf_htonl(x) __builtin_bswap32(x) 155 | # define __bpf_constant_ntohl(x) ___bpf_swab32(x) 156 | # define __bpf_constant_htonl(x) ___bpf_swab32(x) 157 | # define __bpf_be64_to_cpu(x) __builtin_bswap64(x) 158 | # define __bpf_cpu_to_be64(x) __builtin_bswap64(x) 159 | # define __bpf_constant_be64_to_cpu(x) ___bpf_swab64(x) 160 | # define __bpf_constant_cpu_to_be64(x) ___bpf_swab64(x) 161 | #elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__ 162 | # define __bpf_ntohs(x) (x) 163 | # define __bpf_htons(x) (x) 164 | # define __bpf_constant_ntohs(x) (x) 165 | # define __bpf_constant_htons(x) (x) 166 | # define __bpf_ntohl(x) (x) 167 | # define __bpf_htonl(x) (x) 168 | # define __bpf_constant_ntohl(x) (x) 169 | # define __bpf_constant_htonl(x) (x) 170 | # define __bpf_be64_to_cpu(x) (x) 171 | # define __bpf_cpu_to_be64(x) (x) 172 | # define __bpf_constant_be64_to_cpu(x) (x) 173 | # define __bpf_constant_cpu_to_be64(x) (x) 174 | #else 175 | # error "Fix your compiler's __BYTE_ORDER__?!" 176 | #endif 177 | 178 | #define bpf_htons(x) \ 179 | (__builtin_constant_p(x) ? \ 180 | __bpf_constant_htons(x) : __bpf_htons(x)) 181 | #define bpf_ntohs(x) \ 182 | (__builtin_constant_p(x) ? \ 183 | __bpf_constant_ntohs(x) : __bpf_ntohs(x)) 184 | #define bpf_htonl(x) \ 185 | (__builtin_constant_p(x) ? \ 186 | __bpf_constant_htonl(x) : __bpf_htonl(x)) 187 | #define bpf_ntohl(x) \ 188 | (__builtin_constant_p(x) ? \ 189 | __bpf_constant_ntohl(x) : __bpf_ntohl(x)) 190 | #define bpf_cpu_to_be64(x) \ 191 | (__builtin_constant_p(x) ? \ 192 | __bpf_constant_cpu_to_be64(x) : __bpf_cpu_to_be64(x)) 193 | #define bpf_be64_to_cpu(x) \ 194 | (__builtin_constant_p(x) ? \ 195 | __bpf_constant_be64_to_cpu(x) : __bpf_be64_to_cpu(x)) 196 | 197 | #endif /* __BPF_ENDIAN__ */ 198 | 199 | -------------------------------------------------------------------------------- /kern/dispatch_dict.c: -------------------------------------------------------------------------------- 1 | /* 2 | * BSD 2-Clause License 3 | * 4 | * Copyright (c) 2021, Alibaba Group Holding Limited 5 | * All rights reserved. 6 | * 7 | * Redistribution and use in source and binary forms, with or without 8 | * modification, are permitted provided that the following conditions are met: 9 | * 10 | * 1. Redistributions of source code must retain the above copyright notice, this 11 | * list of conditions and the following disclaimer. 12 | * 13 | * 2. Redistributions in binary form must reproduce the above copyright notice, 14 | * this list of conditions and the following disclaimer in the documentation 15 | * and/or other materials provided with the distribution. 16 | * 17 | * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 18 | * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 19 | * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 20 | * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 21 | * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 22 | * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 23 | * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 24 | * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 25 | * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 26 | * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 27 | * 28 | */ 29 | 30 | 31 | 32 | #include "kern_core.c" 33 | 34 | static int xskmap_dispatch(struct xudp_ctx *ctx) 35 | { 36 | u32 key, *p, hash; 37 | int ret; 38 | 39 | p = (u32 *)(ctx->hdrs.udp + 1); 40 | 41 | if (!access_ok(ctx, p)) 42 | return XDP_PASS; 43 | 44 | key = bpf_ntohl(*p); 45 | 46 | ret = xskmap_dict_go(ctx, key); 47 | if (ret > 0) 48 | return ret; 49 | 50 | hash = xudp_hash(ctx); 51 | 52 | return xskmap_go(ctx, hash); 53 | } 54 | -------------------------------------------------------------------------------- /kern/dispatch_hash.c: -------------------------------------------------------------------------------- 1 | /* 2 | * BSD 2-Clause License 3 | * 4 | * Copyright (c) 2021, Alibaba Group Holding Limited 5 | * All rights reserved. 6 | * 7 | * Redistribution and use in source and binary forms, with or without 8 | * modification, are permitted provided that the following conditions are met: 9 | * 10 | * 1. Redistributions of source code must retain the above copyright notice, this 11 | * list of conditions and the following disclaimer. 12 | * 13 | * 2. Redistributions in binary form must reproduce the above copyright notice, 14 | * this list of conditions and the following disclaimer in the documentation 15 | * and/or other materials provided with the distribution. 16 | * 17 | * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 18 | * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 19 | * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 20 | * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 21 | * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 22 | * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 23 | * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 24 | * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 25 | * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 26 | * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 27 | * 28 | */ 29 | 30 | 31 | #include "kern_core.c" 32 | 33 | static int xskmap_dispatch(struct xudp_ctx *ctx) 34 | { 35 | int key; 36 | 37 | key = xudp_hash(ctx); 38 | 39 | return xskmap_go(ctx, key); 40 | } 41 | -------------------------------------------------------------------------------- /kern/dispatch_rr.c: -------------------------------------------------------------------------------- 1 | /* 2 | * BSD 2-Clause License 3 | * 4 | * Copyright (c) 2021, Alibaba Group Holding Limited 5 | * All rights reserved. 6 | * 7 | * Redistribution and use in source and binary forms, with or without 8 | * modification, are permitted provided that the following conditions are met: 9 | * 10 | * 1. Redistributions of source code must retain the above copyright notice, this 11 | * list of conditions and the following disclaimer. 12 | * 13 | * 2. Redistributions in binary form must reproduce the above copyright notice, 14 | * this list of conditions and the following disclaimer in the documentation 15 | * and/or other materials provided with the distribution. 16 | * 17 | * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 18 | * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 19 | * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 20 | * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 21 | * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 22 | * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 23 | * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 24 | * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 25 | * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 26 | * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 27 | * 28 | */ 29 | 30 | 31 | #include "kern_core.c" 32 | 33 | static int xskmap_dispatch(struct xudp_ctx *ctx) 34 | { 35 | return xskmap_rr(ctx); 36 | } 37 | -------------------------------------------------------------------------------- /kern/kern.h: -------------------------------------------------------------------------------- 1 | /* 2 | * BSD 2-Clause License 3 | * 4 | * Copyright (c) 2021, Alibaba Group Holding Limited 5 | * All rights reserved. 6 | * 7 | * Redistribution and use in source and binary forms, with or without 8 | * modification, are permitted provided that the following conditions are met: 9 | * 10 | * 1. Redistributions of source code must retain the above copyright notice, this 11 | * list of conditions and the following disclaimer. 12 | * 13 | * 2. Redistributions in binary form must reproduce the above copyright notice, 14 | * this list of conditions and the following disclaimer in the documentation 15 | * and/or other materials provided with the distribution. 16 | * 17 | * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 18 | * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 19 | * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 20 | * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 21 | * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 22 | * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 23 | * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 24 | * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 25 | * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 26 | * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 27 | * 28 | */ 29 | 30 | #ifndef __KERN_H__ 31 | #define __KERN_H__ 32 | #include 33 | 34 | /* this is for user space c code. ebpf code should include helpers.h */ 35 | 36 | #define MAX_NIC_INDEX 400 37 | #define MAX_IPPORT_NUM 10 38 | #define MAX_CLUSTER_SLOT_NUM 256 39 | 40 | typedef unsigned char u8; 41 | typedef unsigned short u16; 42 | typedef unsigned int u32; 43 | typedef __u64 u64; 44 | 45 | struct kern_info { 46 | u16 reuse; 47 | u32 offset; 48 | 49 | int group_num; 50 | int nic_xskmap_offset[MAX_NIC_INDEX]; 51 | int nic_xskmap_set_offset[MAX_NIC_INDEX]; 52 | }; 53 | 54 | struct kern_dict_item { 55 | u8 active; 56 | u16 reuse; 57 | u32 offset; 58 | }; 59 | 60 | struct kern_ipport { 61 | __be32 addr[MAX_IPPORT_NUM]; 62 | __be16 port[MAX_IPPORT_NUM]; 63 | 64 | int ipport_n; 65 | 66 | struct in6_addr addr6[MAX_IPPORT_NUM]; 67 | __be16 port6[MAX_IPPORT_NUM]; 68 | 69 | int ipport6_n; 70 | }; 71 | 72 | #define XUDP_MAP_STATS 0 73 | #define XUDP_MAP_STATS_NUM 1 74 | #define XUDP_MAP_NS 2 75 | #define XUDP_MAP_NUM 3 76 | #define XUDP_MAP_TS 4 77 | #define XUDP_MAP_ID 50 78 | 79 | #define MAP_XSKMAP "map_xskmap" 80 | #define MAP_STATS "map_stats" 81 | #define MAP_XSKMAP_SET "map_xskmap_set" 82 | #define MAP_INFO "map_info" 83 | #define MAP_DICT "map_dict" 84 | #define MAP_IPPORT "map_ipport" 85 | 86 | #ifdef XUDP_DEBUG 87 | // cat /sys/kernel/debug/tracing/trace_pipe 88 | #define printk(fmt, ...) \ 89 | {\ 90 | const char f[] = fmt;\ 91 | bpf_trace_printk(f, sizeof(f), ##__VA_ARGS__);\ 92 | } 93 | #else 94 | #define printk(fmt, ...) 95 | #endif 96 | 97 | #endif 98 | -------------------------------------------------------------------------------- /kern/kern_core.c: -------------------------------------------------------------------------------- 1 | /* 2 | * BSD 2-Clause License 3 | * 4 | * Copyright (c) 2021, Alibaba Group Holding Limited 5 | * All rights reserved. 6 | * 7 | * Redistribution and use in source and binary forms, with or without 8 | * modification, are permitted provided that the following conditions are met: 9 | * 10 | * 1. Redistributions of source code must retain the above copyright notice, this 11 | * list of conditions and the following disclaimer. 12 | * 13 | * 2. Redistributions in binary form must reproduce the above copyright notice, 14 | * this list of conditions and the following disclaimer in the documentation 15 | * and/or other materials provided with the distribution. 16 | * 17 | * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 18 | * AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 19 | * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 20 | * DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 21 | * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 22 | * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 23 | * SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 24 | * CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 25 | * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 26 | * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 27 | * 28 | */ 29 | 30 | #include 31 | #include 32 | #include 33 | #include 34 | #include 35 | #include 36 | #include 37 | #include 38 | #include 39 | #include 40 | #include 41 | #include 42 | 43 | #include "bpf/bpf_helpers.h" 44 | 45 | #include "bpf_endian.h" 46 | #include "bpf_helper_defs.h" 47 | 48 | #include "kern.h" 49 | #include "ip6.h" 50 | #include "packet_parse.h" 51 | 52 | #define SEC(NAME) __attribute__((section(NAME), used)) 53 | 54 | char _license[] SEC("license") = "Dual BSD/GPL"; 55 | 56 | #define bpf_map struct bpf_map_def SEC("maps") 57 | 58 | /* these will defined by libxudp */ 59 | bpf_map map_dict; 60 | /* current xsk fd */ 61 | bpf_map map_xskmap; 62 | /* all xsk fd */ 63 | bpf_map map_xskmap_set; 64 | 65 | bpf_map map_info = { 66 | .type = BPF_MAP_TYPE_ARRAY, 67 | .key_size = sizeof(int), 68 | .value_size = sizeof(struct kern_info), 69 | .max_entries = 1, 70 | }; 71 | 72 | /* this used by xskmap_rr */ 73 | bpf_map map_rr = { 74 | .type = BPF_MAP_TYPE_PERCPU_ARRAY, 75 | .key_size = sizeof(int), 76 | .value_size = sizeof(unsigned int), 77 | .max_entries = 1, 78 | }; 79 | 80 | /* that will be defined by libxudp */ 81 | bpf_map map_ipport = { 82 | .type = BPF_MAP_TYPE_ARRAY, 83 | .key_size = sizeof(int), 84 | .value_size = sizeof(struct kern_ipport), 85 | .max_entries = 1, 86 | }; 87 | 88 | 89 | /* 0: stats switch 90 | * 1: stats packet count 91 | * 2: net namespace id 92 | * 3: xudp map num 93 | * 4-50: map id 94 | * 95 | * */ 96 | 97 | bpf_map map_stats = { 98 | .type = BPF_MAP_TYPE_ARRAY, 99 | .key_size = sizeof(int), 100 | .value_size = sizeof(int), 101 | .max_entries = 100, 102 | }; 103 | 104 | struct xudp_ctx { 105 | struct xdp_md *xdp; 106 | 107 | struct pkthdrs hdrs; 108 | 109 | struct kern_info *info; 110 | 111 | u32 nic_offset; 112 | u32 rx_queue_index; 113 | }; 114 | 115 | static int xskmap_dispatch(struct xudp_ctx *ctx); 116 | 117 | #define access_ok(ctx, p) ((void *)(p + 1) <= (void *)(long)ctx->xdp->data_end) 118 | 119 | static int __parse_packet(struct xudp_ctx *ctx) 120 | { 121 | void *pkt, *end; 122 | 123 | pkt = (void *)(long)ctx->xdp->data; 124 | end = (void *)(long)ctx->xdp->data_end; 125 | 126 | return packet_parse(&ctx->hdrs, pkt, end); 127 | } 128 | 129 | static bool check_ipport(struct xudp_ctx *ctx) 130 | { 131 | struct kern_ipport *ipport; 132 | struct pkthdrs *hdrs; 133 | int key = 0, i = 0; 134 | 135 | ipport = bpf_map_lookup_elem(&map_ipport, &key); 136 | if (!ipport) 137 | return false; 138 | 139 | hdrs = &ctx->hdrs; 140 | 141 | if (hdrs->family == AF_INET) { 142 | #pragma unroll 143 | for (i = 0; i < MAX_IPPORT_NUM; ++i) { 144 | if (i >= ipport->ipport_n) 145 | return false; 146 | 147 | if (ipport->port[i] == ctx->hdrs.udp->dest) { 148 | if (ipport->addr[i] == ctx->hdrs.iph->daddr) 149 | return true; 150 | if (!ipport->addr[i]) 151 | return true; 152 | } 153 | } 154 | } else { 155 | if (!access_ok(ctx, &ctx->hdrs.iph6->daddr)) 156 | return false; 157 | #pragma unroll 158 | for (i = 0; i < MAX_IPPORT_NUM; ++i) { 159 | if (i >= ipport->ipport6_n) 160 | return false; 161 | 162 | if (ipport->port6[i] == ctx->hdrs.udp->dest) { 163 | if (ip6_eq(&ipport->addr6[i], &ctx->hdrs.iph6->daddr)) 164 | return true; 165 | if (ip6_is_zero(&ipport->addr6[i])) 166 | return true; 167 | } 168 | } 169 | } 170 | 171 | return false; 172 | } 173 | 174 | static int xudp_hash(struct xudp_ctx *ctx) 175 | { 176 | int hash, i; 177 | 178 | if (ctx->hdrs.family == AF_INET) 179 | return (ctx->hdrs.iph->saddr >> 16) + 180 | (ctx->hdrs.iph->saddr & 0xffff) + 181 | ctx->hdrs.udp->source; 182 | 183 | hash = ctx->hdrs.udp->source; 184 | 185 | #pragma unroll 186 | for (i = 0; i < sizeof(ctx->hdrs.iph6->saddr)/4; ++i) 187 | hash += *((int *)&(ctx->hdrs.iph6->saddr) + i); 188 | 189 | return hash; 190 | } 191 | 192 | static int xskmap_go(struct xudp_ctx *ctx, unsigned int group_id) 193 | { 194 | int key; 195 | 196 | group_id = group_id % ctx->info->group_num; 197 | 198 | key = ctx->nic_offset; 199 | key += ctx->rx_queue_index * ctx->info->group_num; 200 | key += group_id; 201 | 202 | /* when xsk is null, return XDP_PASS */ 203 | return bpf_redirect_map(&map_xskmap, key, XDP_PASS); 204 | } 205 | 206 | static int stats_go(struct xudp_ctx *ctx) 207 | { 208 | u32 key, *p, *value; 209 | 210 | key = 0; 211 | value = bpf_map_lookup_elem(&map_stats, &key); 212 | if (!value || !*value) 213 | return XDP_ABORTED; 214 | 215 | p = (u32 *)(ctx->hdrs.udp + 1); 216 | 217 | if (!access_ok(ctx, p)) 218 | return XDP_DROP; 219 | 220 | key = 1; 221 | value = bpf_map_lookup_elem(&map_stats, &key); 222 | if (!value) 223 | return XDP_ABORTED; 224 | 225 | /* record stats request. */ 226 | *value += 1; 227 | 228 | key = bpf_ntohl(*p); 229 | 230 | return xskmap_go(ctx, key); 231 | } 232 | 233 | static int xskmap_dict_go(struct xudp_ctx *ctx, u32 key) 234 | { 235 | struct kern_dict_item *item; 236 | int ret; 237 | 238 | item = bpf_map_lookup_elem(&map_dict, &key); 239 | if (!item || !item->active) 240 | return -1; 241 | 242 | if (item->offset >= ctx->info->offset) { 243 | if (item->reuse != ctx->info->reuse - 1) { 244 | item->active = 0; 245 | return -1; 246 | } 247 | } else { 248 | if (item->reuse != ctx->info->reuse) { 249 | item->active = 0; 250 | return -1; 251 | } 252 | } 253 | 254 | key = ctx->nic_offset; 255 | key += item->offset; 256 | key += ctx->rx_queue_index; 257 | 258 | /* when xsk is null, return XDP_PASS */ 259 | ret= bpf_redirect_map(&map_xskmap_set, key, XDP_PASS); 260 | 261 | /* that mean the xsk has been released, so that item should been set to 262 | * noactive. */ 263 | if (ret == XDP_PASS) { 264 | item->active = 0; 265 | return -1; 266 | } 267 | return ret; 268 | } 269 | 270 | static int xskmap_rr(struct xudp_ctx *ctx) 271 | { 272 | unsigned int *rr; 273 | int key = 0; 274 | 275 | rr = bpf_map_lookup_elem(&map_rr, &key); 276 | if (!rr) 277 | return XDP_PASS; 278 | 279 | *rr = *rr + 1; 280 | 281 | return xskmap_go(ctx, *rr); 282 | } 283 | 284 | SEC("xdp_sock") 285 | static int xdp_sock_prog(struct xdp_md *xdp) 286 | { 287 | struct kern_info *info; 288 | struct xudp_ctx ctx; 289 | int ret, key; 290 | 291 | key = 0; 292 | info = bpf_map_lookup_elem(&map_info, &key); 293 | if (!info) 294 | return XDP_PASS; 295 | 296 | if (xdp->ingress_ifindex >= MAX_NIC_INDEX) 297 | return XDP_PASS; 298 | 299 | ctx.nic_offset = info->nic_xskmap_offset[xdp->ingress_ifindex]; 300 | 301 | ctx.xdp = xdp; 302 | ctx.info = info; 303 | ctx.rx_queue_index = xdp->rx_queue_index; 304 | 305 | if (!__parse_packet(&ctx)) 306 | return XDP_PASS; 307 | 308 | if (!check_ipport(&ctx)) 309 | return XDP_PASS; 310 | 311 | if (ctx.hdrs.family == AF_INET && ctx.hdrs.iph->daddr == ctx.hdrs.iph->saddr) 312 | return stats_go(&ctx); 313 | 314 | ret = xskmap_dispatch(&ctx); 315 | 316 | return ret; 317 | } 318 | 319 | -------------------------------------------------------------------------------- /replace.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | ip link set veth-br1 down 4 | ip link set veth-br2 down 5 | ip -n ns1 link set veth-ns1 down 6 | ip -n ns2 link set veth-ns2 down 7 | ip link set virtual-bridge down 8 | 9 | brctl delif virtual-bridge veth-br2 10 | brctl delif virtual-bridge veth-br2 11 | 12 | ip -n ns2 link del veth-ns2 13 | ip link del veth-br1 14 | ip -n ns1 link del veth-ns1 15 | ip link del veth-br2 16 | 17 | rmmod veth 18 | #insmod /mnt/nfs/bare_metal_kernel/byteatom-kernel/drivers/net/veth.ko 19 | insmod /mnt/nfs/bare_metal_kernel/af_xdp_5_10_bsk/linux-image-bsk/linux-image-bsk/drivers/net/veth.ko 20 | 21 | sleep 1 22 | 23 | ip link add veth-ns1 type veth peer name veth-br1 24 | ip link set veth-ns1 netns ns1 25 | brctl addif virtual-bridge veth-br1 26 | 27 | ip link add veth-ns2 type veth peer name veth-br2 28 | ip link set veth-ns2 netns ns2 29 | brctl addif virtual-bridge veth-br2 30 | 31 | ip -n ns1 addr add local 192.168.1.2/24 dev veth-ns1 32 | ip -n ns2 addr add local 192.168.1.3/24 dev veth-ns2 33 | ip addr add local 192.168.1.1/24 dev virtual-bridge 34 | 35 | 36 | ip link set virtual-bridge up 37 | ip link set veth-br1 up 38 | ip link set veth-br2 up 39 | ip -n ns1 link set veth-ns1 up 40 | ip -n ns2 link set veth-ns2 up 41 | 42 | ip netns exec ns1 ip route add default via 192.168.1.1 43 | ip netns exec ns2 ip route add default via 192.168.1.1 44 | 45 | echo 1 > /proc/sys/net/ipv4/ip_forward 46 | 47 | iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -o eth0 -j MASQUERADE 48 | -------------------------------------------------------------------------------- /set_up.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | #创建2个namespace 3 | #insmod /mnt/nfs/bare_metal_kernel/byteatom-kernel/drivers/net/veth.ko 4 | ip netns add ns1 5 | ip netns add ns2 6 | #创建一个linux bridge 7 | brctl addbr virtual-bridge 8 | 9 | ip link add veth-ns1 type veth peer name veth-br1 10 | ip link set veth-ns1 netns ns1 11 | brctl addif virtual-bridge veth-br1 12 | 13 | ip link add veth-ns2 type veth peer name veth-br2 14 | ip link set veth-ns2 netns ns2 15 | brctl addif virtual-bridge veth-br2 16 | 17 | ip -n ns1 addr add local 192.168.1.2/24 dev veth-ns1 18 | ip -n ns2 addr add local 192.168.1.3/24 dev veth-ns2 19 | ip addr add local 192.168.1.1/24 dev virtual-bridge 20 | 21 | ip link set virtual-bridge up 22 | ip link set veth-br1 up 23 | ip link set veth-br2 up 24 | ip -n ns1 link set veth-ns1 up 25 | ip -n ns2 link set veth-ns2 up 26 | 27 | #set default route 28 | ip netns exec ns1 ip route add default via 192.168.1.1 29 | ip netns exec ns2 ip route add default via 192.168.1.1 30 | 31 | echo 1 > /proc/sys/net/ipv4/ip_forward 32 | 33 | iptables -t nat -A POSTROUTING -s 192.168.1.0/24 -o enp0s8 -j MASQUERADE 34 | -------------------------------------------------------------------------------- /set_up_vm2.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | #创建2个namespace 3 | insmod /mnt/nfs/bare_metal_kernel/linux-ste-kernel/linux-image-bsk/drivers/net/veth.ko 4 | ip netns add ns1 5 | ip netns add ns2 6 | #创建一个linux bridge 7 | brctl addbr virtual-bridge 8 | 9 | ip link add veth-ns1 type veth peer name veth-br1 10 | ip link set veth-ns1 netns ns1 11 | brctl addif virtual-bridge veth-br1 12 | 13 | ip link add veth-ns2 type veth peer name veth-br2 14 | ip link set veth-ns2 netns ns2 15 | brctl addif virtual-bridge veth-br2 16 | 17 | ip -n ns1 addr add local 192.168.2.2/24 dev veth-ns1 18 | ip -n ns2 addr add local 192.168.2.3/24 dev veth-ns2 19 | ip addr add local 192.168.2.1/24 dev virtual-bridge 20 | 21 | ip link set virtual-bridge up 22 | ip link set veth-br1 up 23 | ip link set veth-br2 up 24 | ip -n ns1 link set veth-ns1 up 25 | ip -n ns2 link set veth-ns2 up 26 | 27 | #set default route 28 | ip netns exec ns1 ip route add default via 192.168.2.1 29 | ip netns exec ns2 ip route add default via 192.168.2.1 30 | 31 | echo 1 > /proc/sys/net/ipv4/ip_forward 32 | 33 | iptables -t nat -A POSTROUTING -s 192.168.2.0/24 -o eth0 -j MASQUERADE 34 | -------------------------------------------------------------------------------- /test/Makefile: -------------------------------------------------------------------------------- 1 | 2 | ALL := $(filter test%.c, $(shell ls case)) 3 | ALL := $(ALL:%.c=bin/%) 4 | ALL += bin/lib.o 5 | 6 | all: clean $(ALL) 7 | 8 | run: $(ALL) 9 | sh route6.sh 10 | python2 route6.py 11 | sh netns.sh 12 | echo "test for ipv4" 13 | python3 -m pytest --durations=0 auto/ $(pytest) 14 | echo "test for ipv6" 15 | python3 -m pytest --durations=0 auto/ $(pytest) --ipv6 16 | 17 | bin/lib.o: case/lib.c 18 | @echo CC lib.o 19 | @mkdir bin 2>/dev/null || true 20 | @gcc $^ -I ../objs/ -l elf -l pthread -fPIC -c -o $@ -g 21 | 22 | bin/test_%: case/test_%.c bin/lib.o 23 | @echo CC $@ 24 | @mkdir bin 2>/dev/null || true 25 | @gcc $^ -o $@ -I ../objs/ ../objs//libxudp.a -fPIC -l elf -l pthread -g 26 | 27 | 28 | clean: 29 | rm -rf bin 30 | 31 | 32 | 33 | -------------------------------------------------------------------------------- /test/Readme.md: -------------------------------------------------------------------------------- 1 | ## run test 2 | 3 | make test 4 | 5 | 6 | ## pass args to pytest 7 | 8 | make test pytest=-s 9 | make test pytest='-k echo' 10 | 11 | 12 | ## just run test without process 13 | 14 | 1. run process in netns 15 | 16 | ``` 17 | ip netns exec xudp bash 18 | ./test/bin/test_echo 19 | ``` 20 | 21 | 2. run pytest without process 22 | 23 | ``` 24 | make test pytest='-k echo --noproc' 25 | ``` 26 | 27 | -------------------------------------------------------------------------------- /test/auto/conftest.py: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | # 3 | # Copyright (c) 2021 Alibaba Group Holding Limited 4 | # Express UDP is licensed under Mulan PSL v2. 5 | # You can use this software according to the terms and conditions of the Mulan PSL v2. 6 | # You may obtain a copy of Mulan PSL v2 at: 7 | # http://license.coscl.org.cn/MulanPSL2 8 | # THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 9 | # EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 10 | # MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 11 | # See the Mulan PSL v2 for more details. 12 | # 13 | 14 | import pytest 15 | import subprocess 16 | import os 17 | 18 | import sys 19 | import signal 20 | import time 21 | import xudp 22 | 23 | p = os.path.realpath(__file__) 24 | p = os.path.dirname(p) 25 | 26 | sys.path.append(p) 27 | 28 | def pytest_addoption(parser): 29 | parser.addoption("--noproc", action="store_true") 30 | parser.addoption("--ipv6", action="store_true") 31 | 32 | @pytest.fixture(scope='function') 33 | def proc(request): 34 | noproc = request.config.getoption("--noproc") 35 | ipv6 = request.config.getoption("--ipv6") 36 | if ipv6: 37 | xudp.conf.ipv6 = True 38 | 39 | wait_init = False 40 | 41 | if noproc: 42 | proc = 'true' 43 | cmd = 'true with --noproc' 44 | else: 45 | cmd = None 46 | wait_init = getattr(request.module, 'wait_init', True) 47 | 48 | proc = getattr(request.module, 'proc', None) 49 | if not proc: 50 | proc = request.node.name 51 | 52 | 53 | p = xudp.proc_start(proc, wait_init = wait_init, cmd = cmd) 54 | 55 | yield p 56 | 57 | killall = getattr(request.module, 'killall', None) 58 | if killall: 59 | killall = proc 60 | 61 | xudp.proc_stop(p, killall) 62 | 63 | -------------------------------------------------------------------------------- /test/auto/test_00_zerocopy.py: -------------------------------------------------------------------------------- 1 | # 2 | # Copyright (c) 2021 Alibaba Group Holding Limited 3 | # Express UDP is licensed under Mulan PSL v2. 4 | # You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | # You may obtain a copy of Mulan PSL v2 at: 6 | # http://license.coscl.org.cn/MulanPSL2 7 | # THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | # EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | # MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | # See the Mulan PSL v2 for more details. 11 | # 12 | 13 | import xudp 14 | import time 15 | import os 16 | 17 | 18 | proc = 'test_log_debug' 19 | wait_init = False 20 | 21 | def test_check_zc(proc): 22 | zerocpy = False 23 | 24 | 25 | s = time.time() 26 | while True: 27 | if time.time() - s > 5: 28 | raise Exception("wait too long.") 29 | 30 | line = proc.stdout.readline() 31 | line = line.strip() 32 | 33 | if not line: 34 | continue 35 | 36 | print(line) 37 | 38 | if line.endswith('Success zero copy: true'): 39 | zerocpy = True 40 | 41 | if line.find('bind success.') > -1: 42 | break 43 | 44 | 45 | 46 | assert zerocpy 47 | 48 | 49 | def test_eth_channel_num(proc): 50 | cmd = "ip netns exec xudp ethtool -l eth1 |grep Combined | tail -n 1 | cut -d : -f 2" 51 | n = os.popen(cmd).read().strip() 52 | n = int(n) 53 | 54 | """ 55 | limit the eth1 channel num < 5 56 | some test cases depend this 57 | """ 58 | assert(n < 5) 59 | 60 | -------------------------------------------------------------------------------- /test/auto/test_01_echo.py: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | # 3 | # Copyright (c) 2021 Alibaba Group Holding Limited 4 | # Express UDP is licensed under Mulan PSL v2. 5 | # You can use this software according to the terms and conditions of the Mulan PSL v2. 6 | # You may obtain a copy of Mulan PSL v2 at: 7 | # http://license.coscl.org.cn/MulanPSL2 8 | # THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 9 | # EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 10 | # MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 11 | # See the Mulan PSL v2 for more details. 12 | # 13 | 14 | 15 | import xudp 16 | 17 | proc = 'test_echo' 18 | 19 | def echo(): 20 | udp = xudp.get_con() 21 | 22 | msg0 = 'abcdef' 23 | udp.send(msg0) 24 | msg1 = udp.recv(100) 25 | 26 | assert msg0 == msg1 27 | 28 | def test_echo(proc): 29 | echo() 30 | 31 | 32 | def test_echo_size(proc): 33 | udp = xudp.get_con() 34 | 35 | for i in range(0, 1400): 36 | msg0 = 'x' * i 37 | udp.send(msg0) 38 | msg1 = udp.recv(1500) 39 | 40 | assert msg0 == msg1 41 | 42 | -------------------------------------------------------------------------------- /test/auto/test_02_rr.py: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | # 3 | # Copyright (c) 2021 Alibaba Group Holding Limited 4 | # Express UDP is licensed under Mulan PSL v2. 5 | # You can use this software according to the terms and conditions of the Mulan PSL v2. 6 | # You may obtain a copy of Mulan PSL v2 at: 7 | # http://license.coscl.org.cn/MulanPSL2 8 | # THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 9 | # EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 10 | # MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 11 | # See the Mulan PSL v2 for more details. 12 | # 13 | 14 | 15 | 16 | import socket 17 | import xudp 18 | from collections import defaultdict 19 | 20 | def test_rr(proc): 21 | pids = defaultdict(int) 22 | 23 | udp = xudp.get_con() 24 | 25 | for i in range(10000): 26 | udp.send('pid') 27 | pid = udp.recv(100) 28 | pids[pid] += 1 29 | 30 | vs = pids.values() 31 | 32 | for v in vs: 33 | assert abs(v - 1000) < 10 34 | -------------------------------------------------------------------------------- /test/auto/test_03_dict.py: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | # 3 | # Copyright (c) 2021 Alibaba Group Holding Limited 4 | # Express UDP is licensed under Mulan PSL v2. 5 | # You can use this software according to the terms and conditions of the Mulan PSL v2. 6 | # You may obtain a copy of Mulan PSL v2 at: 7 | # http://license.coscl.org.cn/MulanPSL2 8 | # THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 9 | # EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 10 | # MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 11 | # See the Mulan PSL v2 for more details. 12 | # 13 | 14 | 15 | 16 | import xudp 17 | from collections import defaultdict 18 | import struct 19 | 20 | def check(udp, key): 21 | udp.sendto(key, '') 22 | buf = udp.recv(1024) 23 | 24 | i, total, tid = buf.split('/') 25 | 26 | assert int(i) == key 27 | 28 | return int(total) 29 | 30 | def test_dict(proc): 31 | pids = defaultdict(int) 32 | 33 | udp = xudp.get_con() 34 | 35 | total = check(udp, 0) 36 | 37 | for i in range(total): 38 | for n in range(100): 39 | check(udp, i) 40 | 41 | 42 | -------------------------------------------------------------------------------- /test/auto/test_04_setuid_cap.py: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | # 3 | # Copyright (c) 2021 Alibaba Group Holding Limited 4 | # Express UDP is licensed under Mulan PSL v2. 5 | # You can use this software according to the terms and conditions of the Mulan PSL v2. 6 | # You may obtain a copy of Mulan PSL v2 at: 7 | # http://license.coscl.org.cn/MulanPSL2 8 | # THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 9 | # EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 10 | # MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 11 | # See the Mulan PSL v2 for more details. 12 | # 13 | 14 | 15 | import os 16 | import time 17 | import xudp 18 | 19 | proc = "test_setuid_cap" 20 | killall = True 21 | 22 | class Test_with_keep_cap(object): 23 | keep_cap = '1' 24 | def setup(self): 25 | os.environ['keep_cap'] = self.keep_cap 26 | self.p = xudp.proc_start(proc) 27 | 28 | def teardown(self): 29 | xudp.proc_stop(self.p, killall = proc) 30 | 31 | def test_1(self): 32 | xudp.check_proc_wakeup() 33 | 34 | udp = xudp.get_con() 35 | 36 | udp.send('') 37 | msg1 = udp.recv(100) 38 | assert 3 == len(msg1.split('/')) 39 | 40 | 41 | class Test_with_no_keep_cap(Test_with_keep_cap): 42 | keep_cap = '0' 43 | 44 | def setup(self): 45 | os.environ['keep_cap'] = self.keep_cap 46 | self.p = xudp.proc_start(proc, wait_init = False) 47 | 48 | def test_1(self): 49 | 50 | s = time.time() 51 | while True: 52 | if time.time() - s > 4: 53 | raise Exception("wait log too log") 54 | 55 | try: 56 | line = self.p.stdout.readline() 57 | except IOError: 58 | continue 59 | 60 | line = line.strip() 61 | if not line: 62 | continue 63 | 64 | print(line) 65 | 66 | if line == 'xudp: create AF_XDP fail. Operation not permitted.': 67 | return 68 | 69 | if line == 'xudp: txch ref shmat fail. Permission denied.': 70 | return 71 | 72 | 73 | 74 | -------------------------------------------------------------------------------- /test/auto/test_10_fork.py: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | # 3 | # Copyright (c) 2021 Alibaba Group Holding Limited 4 | # Express UDP is licensed under Mulan PSL v2. 5 | # You can use this software according to the terms and conditions of the Mulan PSL v2. 6 | # You may obtain a copy of Mulan PSL v2 at: 7 | # http://license.coscl.org.cn/MulanPSL2 8 | # THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 9 | # EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 10 | # MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 11 | # See the Mulan PSL v2 for more details. 12 | # 13 | 14 | 15 | import xudp 16 | import struct 17 | import time 18 | 19 | proc = "test_fork" 20 | killall = True 21 | 22 | def req(key): 23 | fd = xudp.get_con() 24 | 25 | fd.sendto(key, '') 26 | buf = fd.recv(1024) 27 | 28 | i, total, tid = buf.split('/') 29 | return int(i) 30 | 31 | def check(key): 32 | fd = xudp.get_con() 33 | 34 | fd.sendto(key, '') 35 | buf = fd.recv(1024) 36 | 37 | i, total, tid = buf.split('/') 38 | 39 | assert int(i) == key 40 | 41 | return int(total) 42 | 43 | def try_group_restart(gid): 44 | fd = xudp.get_con() 45 | fd.sendto(gid, '') 46 | buf = fd.recv(1024) 47 | 48 | pid = buf.split('/')[2] 49 | 50 | xudp.get_con().sendto(gid, "AGAIN") 51 | xudp.check_group_wakeup(gid, int(pid)) 52 | 53 | def dict_check(): 54 | total = check(0) 55 | 56 | for i in range(total): 57 | for n in range(100): 58 | check(i) 59 | 60 | ids = [] 61 | for i in range(10000, 10050): 62 | for n in range(10): 63 | gid = req(i) 64 | ids.append(gid) 65 | 66 | ids = set(ids) 67 | 68 | assert len(ids) > 3 69 | 70 | def test_dict_check(proc): 71 | dict_check() 72 | 73 | def test_again(proc): 74 | xsk_n = xudp.xsk_num() 75 | 76 | try_group_restart(5) 77 | check(5) 78 | 79 | assert xsk_n == xudp.xsk_num() 80 | 81 | def test_exit(proc): 82 | gid = 6 83 | udp = xudp.get_con() 84 | 85 | udp.sendto(gid, "EXIT") 86 | 87 | time.sleep(0.5) 88 | 89 | for n in range(200): 90 | udp.sendto(gid, '') 91 | buf = udp.recv(1024) 92 | i, total, tid = buf.split('/') 93 | assert int(i) != gid 94 | 95 | 96 | def test_again1(proc): 97 | xsk_n = xudp.xsk_num() 98 | 99 | for i in range(0, 10): 100 | try_group_restart(5) 101 | check(5) 102 | assert xsk_n == xudp.xsk_num() 103 | 104 | 105 | -------------------------------------------------------------------------------- /test/auto/xudp.py: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | # 3 | # Copyright (c) 2021 Alibaba Group Holding Limited 4 | # Express UDP is licensed under Mulan PSL v2. 5 | # You can use this software according to the terms and conditions of the Mulan PSL v2. 6 | # You may obtain a copy of Mulan PSL v2 at: 7 | # http://license.coscl.org.cn/MulanPSL2 8 | # THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 9 | # EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 10 | # MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 11 | # See the Mulan PSL v2 for more details. 12 | # 13 | 14 | 15 | import os 16 | import subprocess 17 | import time 18 | import socket 19 | import struct 20 | import signal 21 | 22 | 23 | def getid(gid): 24 | return struct.pack(">I", gid).decode('ascii') 25 | class g: 26 | udp = None 27 | udp_noblock = None 28 | 29 | class conf: 30 | addr = ("10.0.35.2", 3486) 31 | addr6 = ("1000:2000:3000:4000::2", 3487) 32 | ipv6 = False 33 | 34 | def proc_start(name, wait_init = True, cmd = None): 35 | open("test.log", 'w') 36 | 37 | if not cmd: 38 | cmd = 'ip netns exec xudp ./bin/%s > test.log 2>&1' % name 39 | 40 | print ("\n$ %s" % cmd) 41 | 42 | p = subprocess.Popen(cmd, shell = True, stdout=subprocess.PIPE, 43 | stderr=subprocess.STDOUT) 44 | 45 | fd = open("test.log") 46 | p.stdout = fd 47 | 48 | if wait_init: 49 | import fcntl 50 | fl = fcntl.fcntl(p.stdout, fcntl.F_GETFL) 51 | fcntl.fcntl(p.stdout, fcntl.F_SETFL, fl | os.O_NONBLOCK) 52 | 53 | start = time.time() 54 | while True: 55 | if time.time() - start > 3: 56 | raise Exception("process init wait too long.") 57 | 58 | code = p.poll() 59 | if code != None: 60 | if code: 61 | raise Exception("process exit code: %s" % code ) 62 | else: 63 | raise Exception("process exit") 64 | 65 | try: 66 | line = p.stdout.readline() 67 | except IOError: 68 | continue 69 | 70 | if not line: 71 | continue 72 | 73 | print(line[0: -1]) 74 | if line.find('service ok.') > -1: 75 | break 76 | 77 | p.name = name 78 | return p 79 | 80 | def proc_stop(p, killall = None): 81 | cmd = 'killall %s' % p.name 82 | os.system(cmd) 83 | 84 | s = time.time() 85 | while p.poll() == None: 86 | if time.time() - s > 10: 87 | raise Exception("process exit wait too long.") 88 | time.sleep(0.01) 89 | 90 | 91 | class UDP(object): 92 | def __init__(self): 93 | if conf.ipv6: 94 | udp = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM) 95 | udp.bind(('::', 0)) 96 | udp.connect(conf.addr6) 97 | else: 98 | udp = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) 99 | udp.bind(('0.0.0.0', 0)) 100 | udp.connect(conf.addr) 101 | 102 | udp.setblocking(0) 103 | 104 | self.udp = udp 105 | 106 | def send(self, msg): 107 | self.udp.send(bytes(msg, encoding='ascii')) 108 | 109 | def recv(self, s): 110 | # udp recvfrom may sleep, cannot wakeup 111 | while True: 112 | try: 113 | buf = self.udp.recv(s).decode('ascii') 114 | return buf 115 | except IOError: 116 | continue 117 | 118 | def sendto(self, gid, msg): 119 | m = struct.pack(">I", gid).decode('ascii') + msg 120 | self.send(m) 121 | 122 | def get_con(): 123 | return UDP() 124 | 125 | def check_proc_wakeup(): 126 | udp = UDP() 127 | 128 | msg = getid(0) 129 | 130 | while True: 131 | try: 132 | udp.udp.send(bytes(msg, encoding='ascii')) 133 | buf = udp.udp.recv(1024) 134 | except IOError: 135 | continue 136 | 137 | return True 138 | 139 | 140 | def check_group_wakeup(gid, pid = None): 141 | udp = UDP() 142 | 143 | msg = bytes(getid(gid), encoding='ascii') 144 | 145 | while True: 146 | try: 147 | udp.udp.send(msg) 148 | buf = udp.udp.recv(1024).decode('ascii') 149 | except IOError: 150 | continue 151 | 152 | t = buf.split('/') 153 | if pid: 154 | if int(t[2]) == pid: 155 | continue 156 | if int(t[0]) == gid: 157 | return True 158 | 159 | def check_group_exit(gid): 160 | udp = UDP() 161 | 162 | msg = getid(gid) 163 | 164 | while True: 165 | try: 166 | udp.udp.send(msg) 167 | buf = udp.udp.recv(1024) 168 | except IOError: 169 | continue 170 | 171 | t = buf.split('/') 172 | if int(t[0]) == gid: 173 | continue 174 | 175 | break 176 | 177 | 178 | 179 | def xsk_num(): 180 | cmd = "lsof |grep 'protocol: XDP' -c" 181 | n = os.popen(cmd).read().strip() 182 | return int(n) 183 | 184 | 185 | -------------------------------------------------------------------------------- /test/case/lib.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include 15 | #include 16 | #include 17 | #include 18 | 19 | #include "lib.h" 20 | 21 | static struct libconf *global_libconf; 22 | static struct th *th_head; 23 | static struct th *th_tail; 24 | static bool loop = true; 25 | 26 | int mkkey(struct th *th) 27 | { 28 | int key; 29 | 30 | key = th->id + global_libconf->dict_key_offset; 31 | 32 | return key; 33 | } 34 | 35 | static void int_exit(int sig) 36 | { 37 | printf("recv sig %d. exit\n", sig); 38 | (void)sig; 39 | loop = 0; 40 | } 41 | 42 | static int epoll_add(xudp *x, int efd, struct th *th) 43 | { 44 | struct ch *ch; 45 | xudp_channel *xh; 46 | xudp_group *g; 47 | struct epoll_event e; 48 | int cn, i; 49 | 50 | e.events = EPOLLIN | EPOLLOUT | EPOLLET; 51 | 52 | if (global_libconf->conf.isolate_group) 53 | g = xudp_group_new(x, th->id); 54 | else 55 | g = xudp_group_get(x, th->id); 56 | 57 | if (!g) { 58 | printf("create group fail\n"); 59 | return -1; 60 | } 61 | 62 | xudp_group_channel_foreach(xh, g) { 63 | 64 | ch = malloc(sizeof(*ch)); 65 | 66 | ch->ch = xh; 67 | ch->fd = xudp_channel_get_fd(ch->ch); 68 | ch->th = th; 69 | ch->msghdr = xudp_alloc_msg(100); 70 | ch->libconf = global_libconf; 71 | 72 | e.data.ptr = ch; 73 | 74 | /* this just work when flow dispatch == dict */ 75 | xudp_dict_set_group_key(g, mkkey(th)); 76 | 77 | epoll_ctl(efd, EPOLL_CTL_ADD, ch->fd, &e); 78 | } 79 | 80 | return 0; 81 | } 82 | 83 | static void handler_msg(struct ch *ch, xudp_msg *m) 84 | { 85 | int ret, n; 86 | char buf[100]; 87 | 88 | n = snprintf(buf, sizeof(buf), "%d/%d/%d", 89 | ch->th->id, 90 | ch->libconf->conf.group_num, 91 | ch->libconf->conf.group_num); 92 | 93 | ret = xudp_send_channel(ch->ch, buf, n, (struct sockaddr *)&m->peer_addr, 0); 94 | 95 | if (ret < 0) { 96 | printf("xudp_send_one fail. %d\n", ret); 97 | } 98 | } 99 | 100 | static void handler_recv(struct ch *ch) 101 | { 102 | int n, i, ret; 103 | xudp_msg *m; 104 | 105 | xudp_def_msg(hdr, 100); 106 | 107 | while (true) { 108 | n = xudp_recv_channel(ch->ch, hdr, 0); 109 | if (n < 0) 110 | break; 111 | 112 | for (i = 0; i < hdr->used; ++i) { 113 | m = hdr->msg + i; 114 | ch->libconf->handler_msg(ch, m); 115 | } 116 | 117 | xudp_recycle(hdr); 118 | xudp_commit_channel(ch->ch); 119 | } 120 | } 121 | 122 | static int epoll_loop(int efd) 123 | { 124 | struct ch *ch; 125 | struct epoll_event e[1024]; 126 | int n, i; 127 | 128 | while (loop) { 129 | n = epoll_wait(efd, e, sizeof(e)/sizeof(e[0]), 100); 130 | 131 | if (n == 0) 132 | continue; 133 | 134 | if (n < 0) { 135 | //printf("epoll wait error: %s\n", strerror(errno)); 136 | continue; 137 | } 138 | 139 | for (i = 0; i < n; ++i) { 140 | ch = e[i].data.ptr; 141 | ch->libconf->handler(ch); 142 | } 143 | } 144 | } 145 | 146 | static void *handler(void *_) 147 | { 148 | int efd; 149 | struct th *th = _; 150 | xudp *x; 151 | 152 | x = th->x; 153 | 154 | efd = epoll_create(1024 * 2); 155 | 156 | epoll_add(x, efd, th); 157 | 158 | th->ready = true; 159 | 160 | epoll_loop(efd); 161 | 162 | return NULL; 163 | } 164 | 165 | static int fork_process(struct th *t) 166 | { 167 | int pid; 168 | 169 | pid = fork(); 170 | if (0 == pid) { 171 | if (global_libconf->nobody) { 172 | struct passwd *pw; 173 | pw = getpwnam("nobody"); 174 | setuid(pw->pw_uid); 175 | } 176 | 177 | printf("start new process pid: %d gid: %d\n", getpid(), t->id); 178 | handler(t); 179 | exit(0); 180 | } else { 181 | t->pid = pid; 182 | } 183 | 184 | } 185 | 186 | /* start fork process or thread */ 187 | int process(void) 188 | { 189 | struct th *t; 190 | int thread_n; 191 | int i; 192 | bool ready = false; 193 | 194 | xudp *x = global_libconf->x; 195 | 196 | thread_n = global_libconf->conf.group_num; 197 | 198 | for (i = 0; i < thread_n; ++i) { 199 | t = mmap(NULL, sizeof(*t), \ 200 | PROT_READ|PROT_WRITE, \ 201 | MAP_ANONYMOUS | MAP_SHARED | MAP_POPULATE, 202 | -1, 0); 203 | t->id = i; 204 | t->x = x; 205 | t->next = NULL; 206 | t->ready = false; 207 | 208 | if (!th_head) 209 | th_head = t; 210 | else 211 | th_tail->next = t; 212 | 213 | th_tail = t; 214 | 215 | if (global_libconf->fork) { 216 | fork_process(t); 217 | 218 | } else { 219 | pthread_create(&t->thread, 0, handler, t); 220 | } 221 | } 222 | 223 | while(!ready) { 224 | ready = true; 225 | 226 | for (t = th_head; t; t = t->next) { 227 | if (!t->ready) { 228 | ready = false; 229 | break; 230 | } 231 | } 232 | } 233 | 234 | printf("service ok.\n"); // notify pytest case service ok 235 | } 236 | 237 | int stdinit(struct libconf *libconf) 238 | { 239 | int ret = 0, i; 240 | xudp_conf *conf; 241 | struct sockaddr_storage addr[2] = {}; 242 | struct sockaddr_in *s4; 243 | struct sockaddr_in6 *s6; 244 | xudp *x; 245 | 246 | conf = &libconf->conf; 247 | 248 | signal(SIGINT, int_exit); 249 | signal(SIGTERM, int_exit); 250 | signal(SIGABRT, int_exit); 251 | signal(SIGKILL, int_exit); 252 | 253 | x = xudp_init(conf, sizeof(*conf)); 254 | if (!x) { 255 | fprintf(stderr, "xudp_init %d\n", ret); 256 | exit(-1); 257 | } 258 | 259 | libconf->x = x; 260 | global_libconf = libconf; 261 | 262 | s4 = (struct sockaddr_in *)&addr[0]; 263 | s4->sin_family = AF_INET; 264 | s4->sin_addr.s_addr = inet_addr("0.0.0.0"); 265 | s4->sin_port = htons(3486); 266 | 267 | s6 = (struct sockaddr_in6 *)&addr[1]; 268 | s6->sin6_family = AF_INET6; 269 | s6->sin6_port = htons(3487); 270 | 271 | ret = inet_pton(AF_INET6, "::", &s6->sin6_addr); 272 | if (!ret) { 273 | fprintf(stderr, "ipv6 addr error %d\n", ret); 274 | exit(-1); 275 | } 276 | 277 | ret = xudp_bind(x, (struct sockaddr *)addr, sizeof(addr[0]), 2); 278 | if (ret) { 279 | fprintf(stderr, "xudp_bind %d\n", ret); 280 | exit(-1); 281 | } 282 | } 283 | 284 | static struct th *get_th_by_pid(int pid) 285 | { 286 | struct th *t; 287 | for (t = th_head; t; t = t->next) { 288 | if (t->pid == pid) 289 | return t; 290 | } 291 | 292 | return NULL; 293 | } 294 | 295 | static void stdwait_fork(struct libconf *libconf) 296 | { 297 | int pid, status; 298 | struct th *t; 299 | 300 | while (loop) { 301 | 302 | pid = wait(&status); 303 | if (pid < 0) 304 | continue; 305 | 306 | status = WEXITSTATUS(status); 307 | //if (status == FORK_STATUS_EXIT) { 308 | // for (t = th_head; t; t = t->next) { 309 | // if (t->pid == pid) 310 | // continue; 311 | 312 | // kill(t->pid, SIGINT); 313 | // } 314 | // exit(0); 315 | //} 316 | 317 | if (status == FORK_STATUS_EXIT) { 318 | printf("fork process exit with FORK_STATUS_EXIT. %d\n", pid); 319 | //pass 320 | } 321 | 322 | if (status == FORK_STATUS_AGAIN) { 323 | printf("fork process exit with FORK_STATUS_AGAIN. %d\n", pid); 324 | t = get_th_by_pid(pid); 325 | fork_process(t); 326 | } 327 | } 328 | } 329 | 330 | void stdwait() 331 | { 332 | struct libconf *libconf; 333 | struct th *t; 334 | 335 | libconf = global_libconf; 336 | 337 | if (libconf->exit_directly) { 338 | xudp_free(libconf->x); 339 | return; 340 | } 341 | 342 | if (libconf->fork) { 343 | stdwait_fork(libconf); 344 | } else { 345 | for (t = th_head; t; t = t->next) 346 | pthread_join(t->thread, NULL); 347 | } 348 | 349 | xudp_free(libconf->x); 350 | } 351 | 352 | int stdmain(struct libconf *libconf) 353 | { 354 | int err; 355 | 356 | // close libc printf buf 357 | setbuf(stdout, NULL); 358 | 359 | err = stdinit(libconf); 360 | if (err) 361 | return err; 362 | 363 | if (!libconf->handler) 364 | libconf->handler = handler_recv; 365 | 366 | if (!libconf->handler_msg) 367 | libconf->handler_msg = handler_msg; 368 | 369 | libconf->def_handler_msg = handler_msg; 370 | 371 | process(); 372 | 373 | if (!libconf->nowait) 374 | stdwait(); 375 | 376 | return 0; 377 | } 378 | -------------------------------------------------------------------------------- /test/case/lib.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __LIB_H__ 15 | #define __LIB_H__ 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include 24 | #include 25 | #include 26 | #include 27 | #include 28 | 29 | #include "xudp.h" 30 | 31 | struct ch { 32 | struct th *th; 33 | int fd; 34 | xudp_channel *ch; 35 | xudp_msghdr *msghdr; 36 | struct libconf *libconf; 37 | }; 38 | 39 | struct libconf { 40 | xudp_conf conf; 41 | int cluster_id; 42 | xudp *x; 43 | int fork; 44 | int fork_loop; 45 | int group; 46 | int exit_directly; 47 | int nobody; 48 | int nowait; 49 | int dict_key_offset; 50 | 51 | void (*handler)(struct ch *ch); 52 | void (*handler_msg)(struct ch *ch, xudp_msg *m); 53 | void (*def_handler_msg)(struct ch *ch, xudp_msg *m); 54 | }; 55 | 56 | struct th { 57 | struct th *next; 58 | pthread_t thread; 59 | pid_t pid; 60 | xudp *x; 61 | xudp_channel *ch; 62 | int id; 63 | struct th *default_thch; 64 | int cluster_id; 65 | bool ready; 66 | }; 67 | 68 | #define FORK_STATUS_EXIT 100 69 | #define FORK_STATUS_AGAIN 101 70 | 71 | 72 | 73 | int process(void); 74 | int mkkey(struct th *th); 75 | int stdinit(struct libconf *libconf); 76 | void stdwait(); 77 | int stdmain(struct libconf *libconf); 78 | #endif 79 | 80 | 81 | -------------------------------------------------------------------------------- /test/case/test_check_umem.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include "lib.h" 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | 21 | 22 | static void handler_msg(struct ch *ch, xudp_msg *m) 23 | { 24 | int ret, n; 25 | char buf[100]; 26 | char cmd[100] = {0}; 27 | 28 | if (m->size > 4) { 29 | memcpy(cmd, m->p + 4, m->size - 4); 30 | cmd[m->size - 4] = 0; 31 | printf("cmd: %s\n", cmd); 32 | 33 | if (!strcmp(cmd, "AGAIN")) { 34 | exit(FORK_STATUS_AGAIN); 35 | } 36 | 37 | if (!strcmp(cmd, "EXIT")) { 38 | exit(FORK_STATUS_EXIT); 39 | } 40 | } 41 | 42 | n = snprintf(buf, sizeof(buf), "%d/%d/%d", 43 | ch->th->id, 44 | ch->libconf->conf.group_num, 45 | getpid()); 46 | 47 | ret = xudp_send_channel(ch->ch, buf, n, (struct sockaddr *)&m->peer_addr, 0); 48 | 49 | if (ret < 0) { 50 | printf("xudp_send_one fail. %d\n", ret); 51 | } 52 | } 53 | 54 | int main(int argc, char **argv) 55 | { 56 | struct libconf libconf = {}; 57 | int pid; 58 | 59 | /* group num < nic channels. 60 | * umem check will called */ 61 | libconf.conf.group_num = 1; 62 | libconf.conf.force_xdp = true; 63 | libconf.conf.isolate_group = true; 64 | libconf.conf.flow_dispatch = XUDP_FLOW_DISPATCH_TYPE_DICT; 65 | 66 | libconf.handler_msg = handler_msg; 67 | libconf.conf.map_dict_n = 10; 68 | libconf.fork = 1; 69 | libconf.fork_loop = 1; 70 | 71 | stdmain(&libconf); 72 | 73 | return 0; 74 | } 75 | -------------------------------------------------------------------------------- /test/case/test_clear_xdp.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include "lib.h" 15 | int main(int argc, char **argv) 16 | { 17 | xudp_xdp_clear(); 18 | return 0; 19 | } 20 | -------------------------------------------------------------------------------- /test/case/test_dict.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include "lib.h" 15 | #include 16 | 17 | static void handler_msg(struct ch *ch, xudp_msg *m) 18 | { 19 | int ret, n; 20 | char buf[100]; 21 | 22 | n = snprintf(buf, sizeof(buf), "%d/%d/%d", 23 | ch->th->id, 24 | ch->libconf->conf.group_num, 25 | syscall(__NR_gettid)); 26 | 27 | ret = xudp_send_channel(ch->ch, buf, n, (struct sockaddr *)&m->peer_addr, 0); 28 | 29 | if (ret < 0) { 30 | printf("xudp_send_one fail. %d\n", ret); 31 | } 32 | } 33 | 34 | int main(int argc, char **argv) 35 | { 36 | struct libconf libconf = {}; 37 | 38 | libconf.conf.group_num = 10; 39 | libconf.conf.force_xdp = true; 40 | libconf.conf.flow_dispatch = XUDP_FLOW_DISPATCH_TYPE_DICT; 41 | libconf.conf.map_dict_n = 10; 42 | libconf.handler_msg = handler_msg; 43 | 44 | return stdmain(&libconf); 45 | } 46 | -------------------------------------------------------------------------------- /test/case/test_dict_map_auto_release.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include "lib.h" 15 | #include 16 | 17 | static int test_cluster_id; 18 | static struct ch *tx_ch; 19 | 20 | int xudp_dict_get_group_key(xudp *x, int key, int *group_id); 21 | 22 | static int handler_msg(struct ch *ch, xudp_msg *m, char *buf) 23 | { 24 | int ret, id, key, gid; 25 | 26 | if (4 == m->size) { 27 | return sprintf(buf, "%d/%d/%d/%d/%d", 28 | mkkey(ch->th), 29 | ch->th->cluster_id, 30 | ch->th->id, 31 | ch->libconf->conf.group_num, 32 | syscall(__NR_gettid)); 33 | } 34 | 35 | if (0 == strncmp("exit", m->p + 4, m->size)) { 36 | exit(0); 37 | }; 38 | if (0 == strncmp("cover", m->p, m->size)) { 39 | return sprintf(buf, "%d/%d/%d/%d/%d", 40 | mkkey(ch->th), 41 | ch->th->cluster_id, 42 | ch->th->id, 43 | ch->libconf->conf.group_num, 44 | syscall(__NR_gettid)); 45 | }; 46 | 47 | if (0 == strncmp("new for xsk release", m->p, m->size)) { 48 | #if 0 49 | struct xudp_cluster_conf conf = {}; 50 | int pid; 51 | 52 | conf.group_num = ch->libconf->conf.group_num; 53 | 54 | ret = xudp_cluster_new(ch->libconf->x, &conf); 55 | 56 | if (ret < 0) 57 | return sprintf(buf, "-1"); 58 | 59 | pid = fork(); 60 | if (pid == 0) { 61 | cluster_work(ret); 62 | sleep(3600); 63 | } 64 | 65 | sleep(1); 66 | 67 | 68 | test_cluster_id = ret; 69 | 70 | ret = xudp_cluster_new(ch->libconf->x, &conf); 71 | 72 | if (ret >= 0) 73 | cluster_work(ret); 74 | 75 | tx_ch = ch; 76 | xudp_cluster_detach(ch->libconf->x, test_cluster_id); 77 | 78 | return sprintf(buf, "%d,%d", test_cluster_id, ret); 79 | #endif 80 | } 81 | 82 | if (0 == strncmp("check", m->p + 4, 17)) { 83 | key = ntohl(*(u32*)m->p); 84 | // xudp_dict_get_group_key(ch->libconf->x, key, &gid); 85 | return sprintf(buf, "%d/%d/%d/%d/%d/%d", 86 | mkkey(ch->th), 87 | ch->th->cluster_id, 88 | ch->th->id, 89 | ch->libconf->conf.group_num, 90 | syscall(__NR_gettid), 91 | gid); 92 | } 93 | } 94 | 95 | static void handler_recv(struct ch *ch) 96 | { 97 | int n, i, ret, id, *p; 98 | xudp_msg *m; 99 | char buf[100]; 100 | 101 | xudp_def_msg(hdr, 100); 102 | 103 | while (true) { 104 | n = xudp_recv_channel(ch->ch, hdr, 0); 105 | if (n < 0) 106 | break; 107 | 108 | for (i = 0; i < hdr->used; ++i) { 109 | m = hdr->msg + i; 110 | n = handler_msg(ch, m, buf); 111 | 112 | ret = xudp_send_channel(ch->ch, buf, n, (struct sockaddr *)&m->peer_addr, 0); 113 | 114 | if (ret < 0) { 115 | printf("xudp_send_one fail. %d\n", ret); 116 | } 117 | } 118 | 119 | xudp_recycle(hdr); 120 | xudp_commit_channel(ch->ch); 121 | } 122 | } 123 | 124 | int main(int argc, char **argv) 125 | { 126 | struct libconf libconf = {}; 127 | 128 | libconf.conf.group_num = 10; 129 | libconf.conf.force_xdp = true; 130 | libconf.conf.flow_dispatch = XUDP_FLOW_DISPATCH_TYPE_DICT; 131 | libconf.conf.map_dict_n = 100000; 132 | libconf.handler = handler_recv; 133 | //libconf.conf.log_level = XUDP_LOG_DEBUG; 134 | 135 | return stdmain(&libconf); 136 | } 137 | -------------------------------------------------------------------------------- /test/case/test_echo.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include "lib.h" 15 | #include 16 | 17 | 18 | static void handler_msg(struct ch *ch, xudp_msg *m) 19 | { 20 | int ret; 21 | 22 | ret = xudp_send_channel(ch->ch, m->p, m->size, (struct sockaddr *)&m->peer_addr, 0); 23 | 24 | if (ret < 0) { 25 | printf("xudp_send_one fail. %d\n", ret); 26 | } 27 | } 28 | 29 | int main(int argc, char **argv) 30 | { 31 | struct libconf libconf = {}; 32 | 33 | libconf.conf.group_num = 1; 34 | libconf.conf.force_xdp = true; 35 | libconf.handler_msg = handler_msg; 36 | 37 | return stdmain(&libconf); 38 | } 39 | -------------------------------------------------------------------------------- /test/case/test_fork.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include "lib.h" 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | 21 | 22 | static void handler_msg(struct ch *ch, xudp_msg *m) 23 | { 24 | int ret, n; 25 | char buf[100]; 26 | char cmd[100] = {0}; 27 | 28 | if (m->size > 4) { 29 | memcpy(cmd, m->p + 4, m->size - 4); 30 | cmd[m->size - 4] = 0; 31 | printf("cmd: %s\n", cmd); 32 | 33 | if (!strcmp(cmd, "AGAIN")) { 34 | exit(FORK_STATUS_AGAIN); 35 | } 36 | 37 | if (!strcmp(cmd, "EXIT")) { 38 | exit(FORK_STATUS_EXIT); 39 | } 40 | } 41 | 42 | n = snprintf(buf, sizeof(buf), "%d/%d/%d", 43 | ch->th->id, 44 | ch->libconf->conf.group_num, 45 | getpid()); 46 | 47 | ret = xudp_send_channel(ch->ch, buf, n, (struct sockaddr *)&m->peer_addr, 0); 48 | 49 | if (ret < 0) { 50 | printf("xudp_send_one fail. %d\n", ret); 51 | } 52 | } 53 | 54 | int main(int argc, char **argv) 55 | { 56 | struct libconf libconf = {}; 57 | int pid; 58 | 59 | libconf.conf.group_num = 10; 60 | libconf.conf.force_xdp = true; 61 | libconf.conf.isolate_group = true; 62 | libconf.conf.flow_dispatch = XUDP_FLOW_DISPATCH_TYPE_DICT; 63 | 64 | libconf.handler_msg = handler_msg; 65 | libconf.conf.map_dict_n = 10; 66 | libconf.fork = 1; 67 | libconf.fork_loop = 1; 68 | 69 | stdmain(&libconf); 70 | 71 | return 0; 72 | } 73 | -------------------------------------------------------------------------------- /test/case/test_group.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include "lib.h" 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | 21 | int get_res(int pid) 22 | { 23 | char buf[1024]; 24 | char *p; 25 | int fd; 26 | 27 | if (pid == -1) 28 | pid = getpid(); 29 | 30 | sprintf(buf, "/proc/%d/statm", pid); 31 | 32 | fd = open(buf, O_RDONLY); 33 | read(fd, buf, 1024); 34 | 35 | p = buf; 36 | while (*p++ != ' '); 37 | 38 | close(fd); 39 | 40 | return atoi(p) * 4 * 1024; 41 | } 42 | 43 | static void handler_recv(struct ch *ch) 44 | { 45 | int n, i, ret; 46 | xudp_msg *m; 47 | 48 | xudp_def_msg(hdr, 100); 49 | 50 | while (true) { 51 | n = xudp_recv_channel(ch->ch, hdr, 0); 52 | if (n < 0) 53 | break; 54 | 55 | for (i = 0; i < hdr->used; ++i) { 56 | m = hdr->msg + i; 57 | 58 | ret = xudp_send_channel(ch->ch, m->p, m->size, (struct sockaddr *)&m->peer_addr, 0); 59 | 60 | if (ret < 0) { 61 | printf("xudp_send_one fail. %d\n", ret); 62 | } 63 | } 64 | 65 | xudp_recycle(hdr); 66 | xudp_commit_channel(ch->ch); 67 | } 68 | } 69 | 70 | int main(int argc, char **argv) 71 | { 72 | struct libconf libconf = {}; 73 | int pid; 74 | 75 | libconf.conf.group_num = 1; 76 | libconf.conf.force_xdp = true; 77 | libconf.handler = handler_recv; 78 | libconf.fork = 1; 79 | 80 | 81 | return 0; 82 | } 83 | -------------------------------------------------------------------------------- /test/case/test_log_debug.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include "lib.h" 15 | #include 16 | 17 | 18 | static void handler_recv(struct ch *ch) 19 | { 20 | int n, i, ret; 21 | xudp_msg *m; 22 | 23 | xudp_def_msg(hdr, 100); 24 | 25 | while (true) { 26 | n = xudp_recv_channel(ch->ch, hdr, 0); 27 | if (n < 0) 28 | break; 29 | 30 | for (i = 0; i < hdr->used; ++i) { 31 | m = hdr->msg + i; 32 | 33 | ret = xudp_send_channel(ch->ch, m->p, m->size, (struct sockaddr *)&m->peer_addr, 0); 34 | 35 | if (ret < 0) { 36 | printf("xudp_send_one fail. %d\n", ret); 37 | } 38 | } 39 | 40 | xudp_recycle(hdr); 41 | xudp_commit_channel(ch->ch); 42 | } 43 | } 44 | 45 | int main(int argc, char **argv) 46 | { 47 | struct libconf libconf = {}; 48 | 49 | libconf.conf.group_num = 1; 50 | libconf.conf.force_xdp = true; 51 | libconf.conf.log_level = XUDP_LOG_DEBUG; 52 | libconf.handler = handler_recv; 53 | libconf.exit_directly = 1; 54 | return stdmain(&libconf); 55 | } 56 | -------------------------------------------------------------------------------- /test/case/test_reload.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include "lib.h" 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | 21 | // Simulate nginx reload scenarios where multiple groups of workers exist at the 22 | // same time 23 | 24 | static void handler_msg(struct ch *ch, xudp_msg *m) 25 | { 26 | int ret, n; 27 | char buf[100]; 28 | char cmd[100] = {0}; 29 | 30 | if (m->size > 4) { 31 | memcpy(cmd, m->p + 4, m->size - 4); 32 | cmd[m->size - 4] = 0; 33 | printf("cmd: %s\n", cmd); 34 | 35 | if (!strcmp(cmd, "AGAIN")) { 36 | exit(FORK_STATUS_AGAIN); 37 | } 38 | 39 | if (!strcmp(cmd, "EXIT")) { 40 | exit(FORK_STATUS_EXIT); 41 | } 42 | } 43 | 44 | n = snprintf(buf, sizeof(buf), "%d/%d/%d", 45 | ch->th->id, 46 | ch->libconf->conf.group_num, 47 | getpid()); 48 | 49 | ret = xudp_send_channel(ch->ch, buf, n, (struct sockaddr *)&m->peer_addr, 0); 50 | 51 | if (ret < 0) { 52 | printf("xudp_send_one fail. %d\n", ret); 53 | } 54 | } 55 | 56 | int main(int argc, char **argv) 57 | { 58 | struct libconf libconf = {}; 59 | int pid; 60 | 61 | libconf.conf.group_num = 10; 62 | libconf.conf.force_xdp = true; 63 | libconf.conf.isolate_group = true; 64 | libconf.conf.flow_dispatch = XUDP_FLOW_DISPATCH_TYPE_DICT; 65 | 66 | libconf.handler_msg = handler_msg; 67 | libconf.conf.map_dict_n = 200; 68 | libconf.fork = 1; 69 | libconf.fork_loop = 1; 70 | libconf.nowait = 1; 71 | 72 | stdmain(&libconf); 73 | 74 | libconf.dict_key_offset = 100; 75 | 76 | process(); 77 | 78 | stdwait(); 79 | 80 | return 0; 81 | } 82 | -------------------------------------------------------------------------------- /test/case/test_rr.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include "lib.h" 15 | #include 16 | 17 | 18 | static void handler_recv(struct ch *ch) 19 | { 20 | int n, i, ret; 21 | xudp_msg *m; 22 | char buf[10]; 23 | 24 | xudp_def_msg(hdr, 100); 25 | 26 | while (true) { 27 | n = xudp_recv_channel(ch->ch, hdr, 0); 28 | if (n < 0) 29 | break; 30 | 31 | for (i = 0; i < hdr->used; ++i) { 32 | m = hdr->msg + i; 33 | 34 | n = snprintf(buf, sizeof(buf), "%d", syscall(__NR_gettid)); 35 | 36 | ret = xudp_send_channel(ch->ch, buf, n, (struct sockaddr *)&m->peer_addr, 0); 37 | 38 | if (ret < 0) { 39 | printf("xudp_send_one fail. %d\n", ret); 40 | } 41 | } 42 | 43 | xudp_recycle(hdr); 44 | xudp_commit_channel(ch->ch); 45 | } 46 | } 47 | 48 | int main(int argc, char **argv) 49 | { 50 | struct libconf libconf = {}; 51 | 52 | libconf.conf.group_num = 10; 53 | libconf.conf.force_xdp = true; 54 | libconf.conf.flow_dispatch = XUDP_FLOW_DISPATCH_TYPE_RR; 55 | libconf.handler = handler_recv; 56 | 57 | return stdmain(&libconf); 58 | } 59 | -------------------------------------------------------------------------------- /test/case/test_setuid_cap.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include "lib.h" 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | 21 | 22 | int main(int argc, char **argv) 23 | { 24 | struct libconf libconf = {}; 25 | char *keep_cap; 26 | int pid; 27 | 28 | keep_cap = getenv("keep_cap"); 29 | 30 | if (keep_cap && *keep_cap == '1') 31 | libconf.conf.keep_cap = true; 32 | 33 | libconf.conf.group_num = 10; 34 | libconf.conf.force_xdp = true; 35 | libconf.conf.isolate_group = true; 36 | 37 | libconf.conf.map_dict_n = 10; 38 | libconf.fork = 1; 39 | libconf.fork_loop = 1; 40 | 41 | /* setuid to nobody */ 42 | libconf.nobody = 1; 43 | 44 | stdmain(&libconf); 45 | 46 | return 0; 47 | } 48 | -------------------------------------------------------------------------------- /test/netns.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env sh 2 | 3 | ETH0=eth0 4 | ETH1=eth1 5 | 6 | ip netns exec xudp ip link > /dev/null 2>/dev/null 7 | ret=$? 8 | 9 | if (( $ret != 0 )) 10 | then 11 | echo "create netns xudp" 12 | 13 | ip netns add xudp 14 | ip link set $ETH1 netns xudp 15 | 16 | ip addr add 10.0.35.1/24 dev $ETH0 17 | ip -6 addr add 1000:2000:3000:4000::1/64 dev $ETH0 18 | 19 | ip netns exec xudp ip addr add 10.0.35.2/24 dev $ETH1 20 | ip netns exec xudp ip -6 addr add 1000:2000:3000:4000::2/64 dev $ETH1 21 | ip netns exec xudp ip link set $ETH1 up 22 | ip netns exec xudp ip link set lo up 23 | fi 24 | -------------------------------------------------------------------------------- /test/route6.py: -------------------------------------------------------------------------------- 1 | # -*- coding:utf-8 -*- 2 | 3 | import os 4 | 5 | class g(object): 6 | nic = 'dummy0' 7 | netns = "xudp-ipv6-route" 8 | setup_dump = False 9 | xudp_dump = False 10 | check_dump = True 11 | 12 | def shell(cmd, prefix = '> ', pipe = False, echo = True, dump = False): 13 | cmd = "ip netns exec %s %s" % (g.netns, cmd) 14 | 15 | if echo: 16 | print "> ", cmd 17 | 18 | lines = os.popen(cmd).readlines() 19 | 20 | if pipe: 21 | return lines 22 | 23 | for l in lines: 24 | print "%s%s" % (prefix, l[0:-1]) 25 | 26 | if dump: 27 | _dump() 28 | 29 | 30 | def addr_add(addr, dev = 'dummy0'): 31 | shell("ip -6 addr add %s dev dummy0" % addr) 32 | g.check_dump = True 33 | 34 | def route_add(dst, via = None, dev = 'dummy0'): 35 | shell("ip -6 route add %s dev %s" % (dst, dev)) 36 | g.check_dump = True 37 | 38 | return dst 39 | 40 | 41 | def dump(): 42 | print "> DUMP OS route:" 43 | shell('ip -6 route', echo = False) 44 | print "> DUMP XUDP route:" 45 | shell('../objs/xudp-route6', echo = False) 46 | print "" 47 | 48 | def _dump(): 49 | dump() 50 | 51 | 52 | class check(object): 53 | def __init__(self, target, index = None, 54 | ifid = None, dstlen = 0, dst = "::", via = "::", src = "::"): 55 | 56 | if g.check_dump: 57 | g.check_dump = False 58 | dump() 59 | 60 | self.target = target 61 | self.index = index 62 | self.ifid = ifid 63 | self.dst = dst 64 | self.dstlen = dstlen 65 | self.via = via 66 | self.src = src 67 | 68 | self.check_done = 0 69 | 70 | if self.dst.find('/') > -1: 71 | self.dst, self.dstlen = self.dst.split('/') 72 | self.dstlen = int(self.dstlen) 73 | 74 | print("check %s index: %s dst: %s/%2s via: %s src: %s" % (self.target, 75 | self.index, self.dst, self.dstlen, self.via, self.src)) 76 | 77 | self.test() 78 | 79 | def check(self, key, value): 80 | try: 81 | if key == 'target': 82 | assert self.target == value 83 | self.check_done += 1 84 | 85 | if key == 'index' and self.index != None: 86 | assert self.index == int(value) 87 | self.check_done += 1 88 | 89 | if key == 'ret': 90 | assert int(value) == 1 91 | self.check_done += 1 92 | 93 | #if key == 'ifid': 94 | # assert self.ifid == int(value) 95 | 96 | if key == 'dstlen': 97 | assert self.dstlen == int(value) 98 | self.check_done += 1 99 | 100 | if key == 'dst': 101 | assert self.dst == value 102 | self.check_done += 1 103 | 104 | if key == 'via': 105 | assert self.via == value 106 | self.check_done += 1 107 | 108 | if key == 'src': 109 | assert self.src == value 110 | self.check_done += 1 111 | 112 | except Exception as e: 113 | raise(Exception("key: %s value: %s" % (key, value))) 114 | 115 | def test(self): 116 | 117 | 118 | if g.xudp_dump: 119 | cmd = 'ip -6 route get %s' % self.target 120 | shell(cmd) 121 | 122 | cmd = "../objs/xudp-route6 %s" % self.target 123 | 124 | lines = shell(cmd, pipe = True, echo = False) 125 | 126 | for line in lines: 127 | if g.xudp_dump: 128 | print line, 129 | 130 | if not line: 131 | continue 132 | 133 | line = line.strip() 134 | 135 | t = line.split(":", 1) 136 | if len(t) != 2: 137 | continue 138 | 139 | key, value = t 140 | self.check(key.strip(), value.strip()) 141 | 142 | assert self.check_done > 5 143 | 144 | if g.xudp_dump: 145 | print "" 146 | 147 | def run_test(): 148 | import argparse 149 | 150 | parser = argparse.ArgumentParser() 151 | parser.add_argument("--xudp-dump", action='store_true') 152 | args = parser.parse_args() 153 | 154 | g.xudp_dump = args.xudp_dump 155 | 156 | keys = globals().keys() 157 | keys.sort() 158 | for name in keys: 159 | if name.startswith('test_'): 160 | fun = globals()[name] 161 | setup(name) 162 | fun() 163 | 164 | 165 | def setup(name): 166 | print "" 167 | print "> ======= [%s] - setup ============" % name 168 | shell("ip link del %s" % g.nic, echo = False) 169 | shell("ip link set dev lo up", echo = False) 170 | shell("ip link add dummy0 type dummy", echo = False) 171 | shell("ip link set dev dummy0 up", echo = False) 172 | if g.setup_dump: 173 | dump() 174 | 175 | 176 | # check for not default 177 | def test_10(): 178 | addr_add( "1234:5678:abcd:1234::100/64") 179 | R1 = route_add("1234::/16") 180 | 181 | check("1234:F000::", dst = R1) 182 | check("123F::") # not default 183 | 184 | def test_11(): 185 | addr_add( "1234:5678:abcd:1234::100/64") 186 | R1 = route_add("1234::/16") 187 | R2 = route_add("1234:5600::/24") 188 | 189 | check("1234:56FF::", dst = R2) 190 | check("123F::") # not default 191 | 192 | def test_12(): 193 | addr_add( "1234:5678:abcd:1234::100/64") 194 | R1 = route_add("1234:5678::/32") 195 | 196 | check("1234:5678:9::", dst = R1) 197 | check("123F::") # not default 198 | 199 | def test_13(): 200 | addr_add( "1234:5678:abcd:1234::100/64") 201 | R1 = route_add("1234:5600::/24") 202 | R2 = route_add("1234:5678::/32") 203 | R3 = route_add("1234:5678:9000::/40") 204 | R4 = route_add("1234:5678:9012::/48") 205 | R5 = route_add("1234:5678:9012:3400::/56") 206 | R6 = route_add("1234:5678:9012:3456::/64") 207 | 208 | check("1234:56FF::", dst = R1) 209 | check("1234:5678:1::", dst = R2) 210 | check("1234:5678:9014::", dst = R3) 211 | check("1234:5678:9012::", dst = R4) 212 | check("1234:5678:9012:3412::", dst = R5) 213 | check("1234:5678:9012:3456:FF00::", dst = R6) 214 | 215 | def test_14(): 216 | addr_add( "1234:5678:abcd:1234::100/64") 217 | R6 = route_add("1234:5678:9012:3456::/64") 218 | 219 | check( "1234:5678:9012:3456:FF00::", dst = R6) 220 | 221 | def test_21(): 222 | addr_add( "1234:5678:abcd:1234::100/64") 223 | 224 | R1 = route_add("1234:100::/24") 225 | R2 = route_add("1234:200::/24") 226 | R3 = route_add("1234:400::/24") 227 | 228 | R4 = route_add("1234:1234:1234:1234:100::/72") 229 | R5 = route_add("1234:1234:1234:1234:200::/72") 230 | R6 = route_add("1234:1234:1234:1234:400::/72") 231 | 232 | R7 = route_add("1234:A30F::/30") 233 | 234 | check("1234:101::", dst = R1) 235 | check("1234:201::", dst = R2) 236 | check("1234:401::", dst = R3) 237 | 238 | check("1234:1234:1234:1234:101::", dst = R4) 239 | check("1234:1234:1234:1234:201::", dst = R5) 240 | check("1234:1234:1234:1234:401::", dst = R6) 241 | 242 | def test_20(): 243 | addr_add( "1234:5678:abcd:1234::100/64") 244 | 245 | R1 = route_add("1234:a000::/20") 246 | R2 = route_add("1234:a300::/24") 247 | R3 = route_add("1234:a3a0::/28") 248 | R4 = route_add("1234:a3a8::/32") 249 | 250 | check("1234:AF00::", dst = R1) 251 | check("1234:A3F0::", dst = R2) 252 | check("1234:A3A1::", dst = R3) 253 | check("1234:A3A8:1::", dst = R4) 254 | 255 | 256 | def test_30(): 257 | addr_add( "1234:5678:abcd:1234::100/64") 258 | 259 | R1 = route_add("1234:a300::/24") 260 | R2 = route_add("1234:a3a8::/32") 261 | R3 = route_add("1234:a3a8:1000::/40") 262 | 263 | check("1234:A3A9::", dst = R1) 264 | check("1234:A3A8:2000::", dst = R2) 265 | check("1234:A3A8:1010::", dst = R3) 266 | 267 | run_test() 268 | 269 | 270 | -------------------------------------------------------------------------------- /test/route6.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env sh 2 | 3 | ip netns exec xudp-ipv6-route ip link > /dev/null 2>/dev/null 4 | ret=$? 5 | 6 | if (( $ret != 0 )) 7 | then 8 | echo "create netns xudp-ipv6-route" 9 | ip netns add xudp-ipv6-route 10 | ip netns exec xudp-ipv6-route ip link set lo up 11 | fi 12 | 13 | -------------------------------------------------------------------------------- /tools/Makefile: -------------------------------------------------------------------------------- 1 | build=../objs 2 | 3 | ifdef SO 4 | LDFLAGS += -L $(build) -l xudp 5 | else 6 | LDFLAGS += $(build)/libxudp.a -l elf -l pthread 7 | endif 8 | 9 | CFLAGS += -g -O2 -Wall -I $(build) 10 | 11 | ALL += $(build)/xudpperf 12 | ALL += $(build)/xudp-echo-server 13 | 14 | all: $(ALL) 15 | 16 | $(build)/tools/%.o: %.c 17 | @echo PC $@ 18 | @mkdir $(shell dirname $@) 2>/dev/null || true 19 | @clang $< $(CFLAGS) -fPIC -c -o $@ 20 | 21 | $(build)/xudpperf: $(build)/tools/xudpperf.o 22 | @echo LD $@ 23 | @clang $^ $(LDFLAGS) -o $@ -l pthread 24 | 25 | $(build)/xudp-echo-server: $(build)/tools/xudp_echo_server.o 26 | @echo LD $@ 27 | @clang $^ $(LDFLAGS) -o $@ 28 | 29 | -------------------------------------------------------------------------------- /tools/xudp_echo_server.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include "xudp.h" 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | 24 | struct connect{ 25 | xudp *x; 26 | xudp_channel *ch; 27 | void (*handler)(struct connect *); 28 | }; 29 | 30 | static bool gloop = true; 31 | 32 | void handler(struct connect *c) 33 | { 34 | xudp_msg *m; 35 | xudp_channel *ch = c->ch; 36 | int n, i, ret; 37 | 38 | xudp_def_msg(hdr, 100); 39 | 40 | while (true) { 41 | hdr->used = 0; 42 | 43 | n = xudp_recv_channel(ch, hdr, 0); 44 | if (n < 0) 45 | break; 46 | 47 | for (i = 0; i < hdr->used; ++i) { 48 | m = hdr->msg + i; 49 | 50 | printf("recv msg: %.*s", m->size, m->p); 51 | 52 | ret = xudp_send_channel(ch, m->p, m->size, (struct sockaddr *)&m->peer_addr, 0); 53 | 54 | if (ret < 0) { 55 | printf("xudp_send_one fail. %d\n", ret); 56 | } 57 | } 58 | 59 | 60 | xudp_recycle(hdr); 61 | 62 | xudp_commit_channel(ch); 63 | } 64 | } 65 | 66 | static int epoll_add(xudp *x, int efd) 67 | { 68 | struct epoll_event e; 69 | struct connect *c; 70 | xudp_channel *ch; 71 | xudp_group *g; 72 | int fd; 73 | 74 | e.events = EPOLLIN; 75 | 76 | g = xudp_group_get(x, 0); 77 | 78 | xudp_group_channel_foreach(ch, g) { 79 | 80 | fd = xudp_channel_get_fd(ch); 81 | 82 | c = malloc(sizeof(*c)); 83 | c->ch = ch; 84 | c->x = x; 85 | c->handler = handler; 86 | 87 | e.data.ptr = c; 88 | 89 | epoll_ctl(efd, EPOLL_CTL_ADD, fd, &e); 90 | } 91 | 92 | return 0; 93 | } 94 | 95 | static int loop(int efd) 96 | { 97 | struct connect *c; 98 | struct epoll_event e[1024]; 99 | int n, i; 100 | 101 | while (gloop) { 102 | n = epoll_wait(efd, e, sizeof(e)/sizeof(e[0]), -1); 103 | 104 | if (n == 0) 105 | continue; 106 | 107 | if (n < 0) { 108 | //printf("epoll wait error: %s\n", strerror(errno)); 109 | continue; 110 | } 111 | 112 | for (i = 0; i < n; ++i) { 113 | c = e[i].data.ptr; 114 | c->handler(c); 115 | } 116 | } 117 | return 0; 118 | } 119 | 120 | static void int_exit(int sig) 121 | { 122 | (void)sig; 123 | gloop = 0; 124 | } 125 | 126 | int main(int argc, char *argv[]) 127 | { 128 | xudp *x; 129 | int efd, ret; 130 | char *addr, *port; 131 | int size; 132 | struct addrinfo *info; 133 | 134 | xudp_conf conf = {}; 135 | 136 | if (argc != 3) { 137 | addr = "0.0.0.0"; 138 | port = "8080"; 139 | } else { 140 | addr = argv[1]; 141 | port = argv[2]; 142 | } 143 | 144 | ret = getaddrinfo(addr, port, NULL, &info); 145 | if (ret) { 146 | printf("addr format err\n"); 147 | return -1; 148 | } 149 | 150 | conf.group_num = 1; 151 | conf.log_with_time = true; 152 | conf.log_level = XUDP_LOG_WARN; 153 | 154 | x = xudp_init(&conf, sizeof(conf)); 155 | if (!x) { 156 | printf("xudp init fail\n"); 157 | return -1; 158 | } 159 | 160 | if (info->ai_family == AF_INET) { 161 | printf("AF_INET addr.\n"); 162 | size = sizeof(struct sockaddr_in); 163 | } else { 164 | printf("AF_INET6 addr.\n"); 165 | size = sizeof(struct sockaddr_in6); 166 | } 167 | 168 | ret = xudp_bind(x, (struct sockaddr *)info->ai_addr, size, 1); 169 | if (ret) { 170 | xudp_free(x); 171 | printf("xudp bind fail %d\n", ret); 172 | return -1; 173 | } 174 | 175 | efd = epoll_create(1024); 176 | 177 | epoll_add(x, efd); 178 | 179 | signal(SIGINT, int_exit); 180 | signal(SIGTERM, int_exit); 181 | signal(SIGABRT, int_exit); 182 | 183 | loop(efd); 184 | xudp_free(x); 185 | } 186 | -------------------------------------------------------------------------------- /xudp/checksum.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __CHECKSUM_H_ 15 | #define __CHECKSUM_H_ 16 | 17 | # define likely(x) __builtin_expect(!!(x), 1) 18 | # define unlikely(x) __builtin_expect(!!(x), 0) 19 | 20 | #define __force 21 | #define X86 1 22 | 23 | #ifdef X86 24 | typedef unsigned int u32; 25 | 26 | 27 | // │ ip_fast_csum(): 28 | // 0.14 │ mov (%rax),%esi 29 | // 28.82 │ sub $0x4,%ecx 30 | // │ ↓ jbe cb 31 | // 0.01 │ add 0x4(%rax),%esi 32 | // 0.47 │ adc 0x8(%rax),%esi 33 | // 0.60 │ adc 0xc(%rax),%esi 34 | // 0.56 │ b0: adc 0x10(%rax),%esi 35 | // 0.54 │ lea 0x4(%rax),%rax 36 | // 0.00 │ dec %ecx 37 | // │ ↑ jne b0 38 | // │ adc $0x0,%esi 39 | // 0.49 │ mov %esi,%ecx 40 | // 0.00 │ shr $0x10,%esi 41 | // 0.58 │ add %cx,%si 42 | // 0.59 │ adc $0x0,%esi 43 | // 0.54 │ not %esi 44 | 45 | /** 46 | * ip_fast_csum - Compute the IPv4 header checksum efficiently. 47 | * iph: ipv4 header 48 | * ihl: length of header / 4 49 | */ 50 | static inline __sum16 ip_fast_csum(const void *iph, unsigned int ihl) 51 | { 52 | unsigned int sum; 53 | 54 | asm( " movl (%1), %0\n" 55 | " subl $4, %2\n" 56 | " jbe 2f\n" 57 | " addl 4(%1), %0\n" 58 | " adcl 8(%1), %0\n" 59 | " adcl 12(%1), %0\n" 60 | "1: adcl 16(%1), %0\n" 61 | " lea 4(%1), %1\n" 62 | " decl %2\n" 63 | " jne 1b\n" 64 | " adcl $0, %0\n" 65 | " movl %0, %2\n" 66 | " shrl $16, %0\n" 67 | " addw %w2, %w0\n" 68 | " adcl $0, %0\n" 69 | " notl %0\n" 70 | "2:" 71 | /* Since the input registers which are loaded with iph and ipl 72 | are modified, we must also specify them as outputs, or gcc 73 | will assume they contain their original values. */ 74 | : "=r" (sum), "=r" (iph), "=r" (ihl) 75 | : "1" (iph), "2" (ihl) 76 | : "memory"); 77 | return (__force __sum16)sum; 78 | } 79 | #endif 80 | 81 | typedef unsigned char u8; 82 | typedef unsigned short u16; 83 | 84 | static inline u16 checksum(u8 *p, u32 num, u32 sum) 85 | { 86 | int i; 87 | 88 | if (num % 2 == 1) 89 | { 90 | --num; 91 | sum += p[num] << 8; 92 | } 93 | 94 | for (i = 0; i < num; i += 2) 95 | { 96 | sum += p[i] << 8; 97 | sum += p[i+1]; 98 | } 99 | 100 | u32 l = sum & 0x0000FFFF; 101 | u32 h = sum >> 16; 102 | u16 checksum = l + h; 103 | 104 | return ~checksum; 105 | } 106 | 107 | static inline u16 udp_checksum(u8 *u, __be32 saddr, __be32 daddr, u16 size) 108 | { 109 | u32 sum = 0; 110 | u8 *p; 111 | __be32 l; 112 | 113 | p = (u8 *)&saddr; 114 | 115 | sum += p[0] << 8; 116 | sum += p[1]; 117 | 118 | sum += p[2] << 8; 119 | sum += p[3]; 120 | 121 | p = (u8 *)&daddr; 122 | 123 | sum += p[0] << 8; 124 | sum += p[1]; 125 | 126 | sum += p[2] << 8; 127 | sum += p[3]; 128 | 129 | sum += IPPROTO_UDP; 130 | 131 | 132 | l = htons(size); 133 | 134 | p = (u8 *)&l; 135 | 136 | sum += p[0] << 8; 137 | sum += p[1]; 138 | 139 | return checksum(u, size, sum); 140 | } 141 | 142 | #define sum32(sum, val) { \ 143 | int carry; \ 144 | sum += val; \ 145 | carry = sum < val; \ 146 | sum += carry; \ 147 | } 148 | 149 | static inline u32 udp6_hdr_csum(u32 sum, struct in6_addr *saddr, 150 | struct in6_addr *daddr, u32 size) 151 | { 152 | sum32(sum, saddr->s6_addr32[0]); 153 | sum32(sum, saddr->s6_addr32[1]); 154 | sum32(sum, saddr->s6_addr32[2]); 155 | sum32(sum, saddr->s6_addr32[3]); 156 | 157 | sum32(sum, daddr->s6_addr32[0]); 158 | sum32(sum, daddr->s6_addr32[1]); 159 | sum32(sum, daddr->s6_addr32[2]); 160 | sum32(sum, daddr->s6_addr32[3]); 161 | 162 | sum32(sum, htonl(size)); 163 | sum32(sum, htonl(IPPROTO_UDP)); 164 | 165 | return sum; 166 | } 167 | 168 | static inline u32 do_csum(unsigned char *buf, u32 size) 169 | { 170 | u32 sum = 0; 171 | 172 | while (size >= 4) { 173 | sum32(sum, *(u32 *)buf); 174 | buf += 4; 175 | size -= 4; 176 | } 177 | 178 | sum = (sum & 0xffff) + (sum >> 16); 179 | 180 | if (size & 2) { 181 | sum += *(u16 *)buf; 182 | buf += 2; 183 | } 184 | 185 | if (size & 1) { 186 | #ifdef __LITTLE_ENDIAN 187 | sum += *buf; 188 | #else 189 | sum += (*buf << 8); 190 | #endif 191 | } 192 | 193 | return sum; 194 | } 195 | 196 | static inline u32 do_csum_v1(char *buf, u32 size) 197 | { 198 | u32 sum = 0; 199 | 200 | while (size >= 4) { 201 | sum32(sum, *(u32 *)buf); 202 | buf += 4; 203 | size -= 4; 204 | } 205 | 206 | if (size) { 207 | char t[4] = {0}; 208 | 209 | switch(size) { 210 | case 3: 211 | t[2] = *(buf + 2); 212 | case 2: 213 | t[1] = *(buf + 1); 214 | case 1: 215 | t[0] = *buf; 216 | } 217 | 218 | sum32(sum, *(u32 *)t); 219 | } 220 | 221 | return sum; 222 | } 223 | 224 | static inline u16 csum_fold(u32 sum) 225 | { 226 | sum = (sum & 0xffff) + (sum >> 16); 227 | sum = (sum & 0xffff) + (sum >> 16); 228 | return (u16)~sum; 229 | } 230 | #endif 231 | -------------------------------------------------------------------------------- /xudp/group_api.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include "xudp_types.h" 15 | 16 | int __xudp_group_num(xudp *x) 17 | { 18 | return x->conf.group_num; 19 | } 20 | 21 | int __xudp_group_unaligned(xudp *x) 22 | { 23 | return x->conf.unaligned; 24 | } 25 | 26 | int __xudp_nic_num(xudp *x) 27 | { 28 | return x->nic_n; 29 | } 30 | 31 | int __xudp_dict_active(xudp *x) 32 | { 33 | return x->map_dict_active; 34 | } 35 | 36 | struct kern_info *__xudp_kern_info(xudp *x) 37 | { 38 | return &x->kern_info; 39 | } 40 | 41 | static struct xudp_nic *xudp_get_nic(xudp *x, int ifindex) 42 | { 43 | struct xudp_nic *n; 44 | 45 | for (n = x->nic; n; n = n->next) { 46 | if (n->nic_index == ifindex) 47 | return n; 48 | } 49 | 50 | return NULL; 51 | } 52 | 53 | int __xudp_nic_ifindex(xudp *x, int nicid) 54 | { 55 | struct xudp_nic *n; 56 | int i = 0; 57 | 58 | for (n = x->nic; n; n = n->next) { 59 | if (i == nicid) 60 | return n->nic_index; 61 | ++i; 62 | } 63 | 64 | return 0; 65 | } 66 | 67 | int __xudp_nic_queue_num(xudp *x, int ifindex) 68 | { 69 | struct xudp_nic *n; 70 | 71 | n = xudp_get_nic(x, ifindex); 72 | 73 | if (!n) 74 | return 0; 75 | 76 | return n->queue; 77 | } 78 | 79 | static struct xdp_umem *umem_get(xudp *x, int ifindex, int queue_id) 80 | { 81 | struct xudp_nic *n; 82 | 83 | n = xudp_get_nic(x, ifindex); 84 | if (!n) 85 | return NULL; 86 | 87 | return n->umem[queue_id]; 88 | } 89 | 90 | char *__xudp_umem_get_frames(xudp *x, int ifindex, int queue_id) 91 | { 92 | struct xdp_umem *umem; 93 | 94 | umem = umem_get(x, ifindex, queue_id); 95 | if (!umem) 96 | return NULL; 97 | 98 | return umem->frames; 99 | } 100 | 101 | struct xdp_umem_uqueue *__xudp_umem_get_fq(xudp *x, int ifindex, int queue_id) 102 | { 103 | struct xdp_umem *umem; 104 | 105 | umem = umem_get(x, ifindex, queue_id); 106 | if (!umem) 107 | return NULL; 108 | 109 | return &umem->fq; 110 | } 111 | 112 | int __xudp_umem_get_headroom(xudp *x, int ifindex, int queue_id) 113 | { 114 | struct xdp_umem *umem; 115 | 116 | umem = umem_get(x, ifindex, queue_id); 117 | if (!umem) 118 | return 0; 119 | 120 | return umem->headroom; 121 | } 122 | 123 | int __xudp_get_bpf_map(xudp *x, char *name) 124 | { 125 | return bpf_map_get(&x->bpf, name); 126 | } 127 | 128 | struct log *__xudp_log(xudp *x) 129 | { 130 | return x->log; 131 | } 132 | 133 | struct xdpsock *__xudp_rx_ref_tx(struct xdpsock *tx) 134 | { 135 | return tx->tx_xsk; 136 | } 137 | 138 | -------------------------------------------------------------------------------- /xudp/kern_ops.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include "xudp_types.h" 15 | 16 | #define bin_kern(name) \ 17 | extern uint8_t _binary_kern_ ##name## _o_start[]; \ 18 | extern uint8_t _binary_kern_ ##name## _o_end[]; \ 19 | extern uint8_t _binary_kern_ ##name## _o_size; 20 | 21 | bin_kern(rr); 22 | bin_kern(hash); 23 | bin_kern(dict); 24 | 25 | #define bin_start(name) ((_binary_kern_ ##name## _o_start)) 26 | #define bin_size(name) (_binary_kern_ ##name## _o_end - bin_start(name)) 27 | 28 | int kern_init(struct bpf *b, enum xudp_flow_dispatch_type type, 29 | void *bin, int size, const char *path) 30 | { 31 | switch(type) { 32 | case XUDP_FLOW_DISPATCH_TYPE_HASH: 33 | bin = bin_start(hash); 34 | size = bin_size(hash); 35 | break; 36 | 37 | case XUDP_FLOW_DISPATCH_TYPE_RR: 38 | bin = bin_start(rr); 39 | size = bin_size(rr); 40 | break; 41 | 42 | case XUDP_FLOW_DISPATCH_TYPE_DICT: 43 | bin = bin_start(dict); 44 | size = bin_size(dict); 45 | break; 46 | 47 | case XUDP_FLOW_DISPATCH_TYPE_CUSTOM: 48 | if (path && !bin) { 49 | int fd; 50 | struct stat buf; 51 | 52 | fd = open(path, O_RDONLY); 53 | if (fd < 0) 54 | return -1; 55 | 56 | if (fstat(fd, &buf) < 0) { 57 | close(fd); 58 | return -1; 59 | } 60 | 61 | size = buf.st_size; 62 | bin = malloc(size); 63 | if (!bin) { 64 | close(fd); 65 | return -1; 66 | } 67 | 68 | if (size != read(fd, bin, size)) { 69 | free(bin); 70 | close(fd); 71 | return -1; 72 | } 73 | close(fd); 74 | } 75 | break; 76 | } 77 | 78 | return bpf_load(b, "xdp_sock", BPF_PROG_TYPE_XDP, bin, size); 79 | } 80 | 81 | #if TEST 82 | #include 83 | #include 84 | int bpf_set_link_xdp_fd(int ifindex, int fd, __u32 flags); 85 | int main(void) 86 | { 87 | struct bpf B = {}, *b; 88 | 89 | b = &B; 90 | int ret; 91 | 92 | ret = kern_init(b); 93 | if (ret) { 94 | logerr("bpf load ret: %d\n", ret); 95 | } 96 | ret = bpf_set_link_xdp_fd(13, ret, 0); 97 | logdebug("link set xdp ret: %d\n", ret); 98 | sleep(3600); 99 | 100 | } 101 | #endif 102 | -------------------------------------------------------------------------------- /xudp/kern_ops.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __KERN_OPS_H__ 15 | #define __KERN_OPS_H__ 16 | 17 | #include "xudp_types.h" 18 | 19 | int kern_init(struct bpf *b, enum xudp_flow_dispatch_type type, 20 | void *bin, int size, const char *path); 21 | #endif 22 | 23 | 24 | -------------------------------------------------------------------------------- /xudp/neigh.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | 24 | #include "ifapi.h" 25 | #include "neigh.h" 26 | #include "xudp_types.h" 27 | #include "ip6.h" 28 | 29 | /* this just for test, aliyun ecs not can set config.noarp. */ 30 | 31 | struct ipmac { 32 | union { 33 | __be32 addr; 34 | struct in6_addr addr6; 35 | }; 36 | unsigned char mac[6]; 37 | }; 38 | 39 | static struct ipmac cache[10]; 40 | static int cache_n; 41 | static pthread_spinlock_t lock; 42 | 43 | int ping(const char *ip); 44 | 45 | int neigh_init() 46 | { 47 | return pthread_spin_init(&lock, PTHREAD_PROCESS_SHARED); 48 | } 49 | 50 | static int ng_dst_eq(struct ipmac *p, void *dst, int family) 51 | { 52 | if (family == AF_INET) 53 | return p->addr == *(__be32*)dst; 54 | 55 | return ip6_eq(&p->addr6, (struct in6_addr*)dst); 56 | } 57 | 58 | static unsigned char *__neigh_get(void* dst, int family, struct log *log) 59 | { 60 | int i; 61 | struct ipmac *p; 62 | unsigned char mac[6]; 63 | 64 | 65 | for (i = 0; i < cache_n; ++i) { 66 | p = cache + i; 67 | 68 | if (ng_dst_eq(p, dst, family)) 69 | return p->mac; 70 | } 71 | 72 | if (cache_n >= sizeof(cache)/sizeof(cache[0])) { 73 | logdebug(log, "tmp: arp limit %d\n", cache_n); 74 | return NULL; 75 | } 76 | 77 | pthread_spin_lock(&lock); 78 | 79 | for (i = 0; i < cache_n; ++i) { 80 | p = cache + i; 81 | 82 | if (ng_dst_eq(p, dst, family)) { 83 | pthread_spin_unlock(&lock); 84 | return p->mac; 85 | } 86 | } 87 | 88 | p = &cache[cache_n]; 89 | 90 | if (family == AF_INET) { 91 | char ip[20] = {}; 92 | struct in_addr a; 93 | 94 | a.s_addr = *(__be32*)dst; 95 | 96 | strcpy(ip, inet_ntoa(a)); 97 | 98 | logdebug(log, "neigh: get arp. ping sent. for %s\n", ip); 99 | ping(ip); 100 | 101 | if (nl_neigh(*(__be32*)dst, mac, log)) { 102 | pthread_spin_unlock(&lock); 103 | logdebug(log, "neigh: get arp fail\n"); 104 | return NULL; 105 | } 106 | 107 | p->addr = *(__be32 *)dst; 108 | 109 | } else { 110 | if (nl_neigh6((struct in6_addr *)dst, mac, log)) { 111 | pthread_spin_unlock(&lock); 112 | logdebug(log, "neigh: get arp fail\n"); 113 | return NULL; 114 | } 115 | ip6_cpy(&p->addr6, dst); 116 | } 117 | 118 | memcpy(p->mac, mac, 6); 119 | ++cache_n; 120 | 121 | pthread_spin_unlock(&lock); 122 | return p->mac; 123 | } 124 | 125 | unsigned char *neigh_get(__be32 addr, struct log *log) 126 | { 127 | return __neigh_get(&addr, AF_INET, log); 128 | } 129 | 130 | unsigned char *neigh6_get(struct in6_addr *addr, struct log *log) 131 | { 132 | return __neigh_get(addr, AF_INET6, log); 133 | 134 | } 135 | -------------------------------------------------------------------------------- /xudp/neigh.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __NEIGH_H__ 15 | #define __NEIGH_H__ 16 | #include "log.h" 17 | 18 | int neigh_init(); 19 | unsigned char *neigh_get(__be32 addr, struct log *log); 20 | unsigned char *neigh6_get(struct in6_addr *addr, struct log *log); 21 | #endif 22 | 23 | 24 | -------------------------------------------------------------------------------- /xudp/packet.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include 15 | 16 | #include "packet.h" 17 | #include "checksum.h" 18 | 19 | #define IP_XUDP_TTL 64 20 | #define IP_DF 0x4000 21 | #define IP_VIT ((4 << 12) | (5 << 8) | (0 & 0xff)) 22 | 23 | #define CSUM_MANGLED_0 ((u16)0xffff) 24 | 25 | typedef unsigned char u8; 26 | 27 | //#ifdef __x86_64__ 28 | //static inline void *__movsb(void *d, const void *s, size_t n) 29 | //{ 30 | // asm volatile ("rep movsb" 31 | // : "=D" (d), 32 | // "=S" (s), 33 | // "=c" (n) 34 | // : "0" (d), 35 | // "1" (s), 36 | // "2" (n) 37 | // : "memory"); 38 | // return d; 39 | //} 40 | //#define memcpy __movsb 41 | //#endif 42 | 43 | static void xudp_checksum_half(struct iphdr *iph) 44 | { 45 | #define IP_HALF_SUM (ntohs(IP_VIT) + \ 46 | ntohs((IP_XUDP_TTL << 8) + IPPROTO_UDP) + \ 47 | ntohs(IP_DF)); 48 | u32 sum; 49 | u16 *p; 50 | 51 | sum = IP_HALF_SUM; 52 | 53 | sum += iph->tot_len; 54 | 55 | p = (u16 *)&iph->saddr; 56 | 57 | sum += *p++; 58 | sum += *p++; 59 | sum += *p++; 60 | sum += *p; 61 | 62 | sum = (sum & 0xFFFF) + (sum >> 16); 63 | sum += sum >> 16; 64 | 65 | iph->check = (u16)~sum; 66 | } 67 | 68 | static void iph_build(struct iphdr *iph, u32 size, 69 | struct sockaddr_in *src, struct sockaddr_in *dst) 70 | { 71 | /* ip */ 72 | // version, ihl, tos 73 | *((__be16 *)iph) = htons(IP_VIT); 74 | 75 | iph->tot_len = htons(sizeof(*iph) + size); 76 | iph->id = 0; 77 | iph->frag_off = htons(IP_DF); 78 | iph->ttl = IP_XUDP_TTL; 79 | iph->protocol = IPPROTO_UDP; 80 | iph->saddr = src->sin_addr.s_addr; 81 | iph->daddr = dst->sin_addr.s_addr; 82 | 83 | xudp_checksum_half(iph); 84 | } 85 | 86 | static inline void ip6_flow_hdr(struct ipv6hdr *hdr, unsigned int tclass, 87 | __be32 flowlabel) 88 | { 89 | *(__be32 *)hdr = htonl(0x60000000 | (tclass << 20) | flowlabel); 90 | } 91 | 92 | static void iph_build6(struct ipv6hdr *iph6, u32 size, 93 | struct sockaddr_in6 *src, 94 | struct sockaddr_in6 *dst) 95 | { 96 | ip6_flow_hdr(iph6, 0, (0x3 << 16) + src->sin6_port); 97 | 98 | iph6->payload_len = htons(size); 99 | iph6->nexthdr = IPPROTO_UDP; 100 | iph6->hop_limit = IP_XUDP_TTL; 101 | iph6->saddr = src->sin6_addr; 102 | iph6->daddr = dst->sin6_addr; 103 | } 104 | 105 | static void udp_csum6(struct udphdr *udp, u32 size, 106 | struct in6_addr *saddr, struct in6_addr *daddr) 107 | { 108 | u32 sum; 109 | 110 | sum = do_csum((unsigned char *)udp, size); 111 | sum = udp6_hdr_csum(sum, saddr, daddr, size); 112 | 113 | udp->check = csum_fold(sum); 114 | 115 | if (udp->check == 0) 116 | udp->check = CSUM_MANGLED_0; 117 | } 118 | 119 | static void udp_build(struct udphdr *udp, u32 size, __be16 sport, __be16 dport) 120 | { 121 | udp->source = sport; 122 | udp->dest = dport; 123 | udp->len = htons(size); 124 | 125 | udp->check = 0; // must 126 | } 127 | 128 | static void copy_eth(unsigned char *dst, unsigned char *src) 129 | { 130 | u32 *d1 = (u32 *)dst; 131 | u32 *s1 = (u32 *)src; 132 | u16 *d2 = (uint16_t *)(dst + sizeof(u32)); 133 | u16 *s2 = (uint16_t *)(src + sizeof(u32)); 134 | 135 | *d1 = *s1; 136 | *d2 = *s2; 137 | } 138 | 139 | static void eth_build(struct ethhdr *eth, 140 | unsigned char *smac, unsigned char *dmac, 141 | u16 prot) 142 | { 143 | /* eth */ 144 | copy_eth(eth->h_source, smac); 145 | copy_eth(eth->h_dest, dmac); 146 | 147 | eth->h_proto = htons(prot); 148 | } 149 | 150 | #define IPV4_HEADROOM \ 151 | (sizeof(struct ethhdr) + sizeof(struct iphdr) + sizeof(struct udphdr)) 152 | 153 | #define IPV6_HEADROOM \ 154 | (sizeof(struct ethhdr) + sizeof(struct ipv6hdr) + sizeof(struct udphdr)) 155 | 156 | void xudp_packet_udp(struct packet_info *info) 157 | { 158 | struct ethhdr *eth; 159 | struct iphdr *iph; 160 | struct ipv6hdr *iph6; 161 | struct udphdr *udp; 162 | u32 size; 163 | 164 | size = sizeof(*udp) + info->payload_size; 165 | 166 | if (info->family == AF_INET) { 167 | eth = (void *)(info->data - IPV4_HEADROOM); 168 | iph = (void *)(eth + 1); 169 | udp = (void *)(iph + 1); 170 | 171 | eth_build(eth, info->smac, info->dmac, ETH_P_IP); 172 | iph_build(iph, size, info->from, info->to); 173 | udp_build(udp, size, info->from->sin_port, info->to->sin_port); 174 | 175 | info->len = info->payload_size + IPV4_HEADROOM; 176 | } else { 177 | eth = (void *)(info->data - IPV6_HEADROOM); 178 | iph6 = (void *)(eth + 1); 179 | udp = (void *)(iph6 + 1); 180 | 181 | eth_build(eth, info->smac, info->dmac, ETH_P_IPV6); 182 | iph_build6(iph6, size, info->from6, info->to6); 183 | udp_build(udp, size, info->from6->sin6_port, info->to6->sin6_port); 184 | /* udp over ipv6, checksum is must. 185 | * 186 | * https://datatracker.ietf.org/doc/html/rfc2460#section-8.1 187 | */ 188 | udp_csum6(udp, size, &info->from6->sin6_addr, &info->to6->sin6_addr); 189 | 190 | info->len = info->payload_size + IPV6_HEADROOM; 191 | } 192 | 193 | info->packet = (void *)eth; 194 | } 195 | 196 | void xudp_packet_udp_payload(struct packet_info *info) 197 | { 198 | info->data = info->head + XUDP_TX_HEADROOM; 199 | 200 | memcpy(info->data, info->payload, info->payload_size); 201 | 202 | xudp_packet_udp(info); 203 | } 204 | -------------------------------------------------------------------------------- /xudp/packet.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __PACKET_H__ 15 | #define __PACKET_H__ 16 | #include 17 | #include 18 | #include 19 | #include 20 | 21 | #include 22 | #include "log.h" 23 | typedef __u64 u64; 24 | typedef __u32 u32; 25 | typedef __u16 u16; 26 | typedef __u8 u8; 27 | 28 | struct packet_info { 29 | u8 family; 30 | unsigned char *dmac; 31 | unsigned char *smac; 32 | union { 33 | struct sockaddr_in *to; 34 | struct sockaddr_in6 *to6; 35 | }; 36 | union { 37 | struct sockaddr_in *from; 38 | struct sockaddr_in6 *from6; 39 | }; 40 | union { 41 | struct sockaddr_in _from; 42 | struct sockaddr_in6 _from6; 43 | }; 44 | 45 | char *head; 46 | char *data; 47 | 48 | char *payload; 49 | int payload_size; 50 | 51 | char *packet; 52 | int len; 53 | }; 54 | 55 | void xudp_packet_udp(struct packet_info *info); 56 | void xudp_packet_udp_payload(struct packet_info *info); 57 | 58 | #define XUDP_TX_HEADROOM (sizeof(struct ethhdr) + 2 + \ 59 | sizeof(struct ipv6hdr) + \ 60 | sizeof(struct udphdr)) 61 | #endif 62 | 63 | 64 | -------------------------------------------------------------------------------- /xudp/ping.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | 24 | 25 | static unsigned short checksum(unsigned short *buf, int bufsz){ 26 | unsigned long sum = 0xffff; 27 | 28 | while(bufsz > 1){ 29 | sum += *buf; 30 | buf++; 31 | bufsz -= 2; 32 | } 33 | 34 | if(bufsz == 1) 35 | sum += *(unsigned char*)buf; 36 | 37 | sum = (sum & 0xffff) + (sum >> 16); 38 | sum = (sum & 0xffff) + (sum >> 16); 39 | 40 | return ~sum; 41 | } 42 | 43 | int ping(const char *ip) 44 | { 45 | int sd; 46 | struct icmphdr hdr; 47 | struct sockaddr_in addr; 48 | int num; 49 | char buf[1024]; 50 | 51 | sd = socket(PF_INET, SOCK_RAW, IPPROTO_ICMP); 52 | if(sd < 0){ 53 | return -1; 54 | } 55 | 56 | memset(&hdr, 0, sizeof(hdr)); 57 | 58 | hdr.type = ICMP_ECHO; 59 | hdr.code = 0; 60 | hdr.checksum = 0; 61 | hdr.un.echo.id = 0; 62 | hdr.un.echo.sequence = 0; 63 | 64 | hdr.checksum = checksum((unsigned short*)&hdr, sizeof(hdr)); 65 | 66 | addr.sin_family = PF_INET; // IPv4 67 | addr.sin_addr.s_addr = inet_addr(ip); 68 | 69 | num = sendto(sd, (char*)&hdr, sizeof(hdr), 0, 70 | (struct sockaddr*)&addr, sizeof(addr)); 71 | if(num < 1){ 72 | return -1; 73 | } 74 | 75 | memset(buf, 0, sizeof(buf)); 76 | 77 | num = recv(sd, buf, sizeof(buf), 0); 78 | if(num < 1){ 79 | return -1; 80 | } 81 | 82 | close(sd); 83 | return 0; 84 | } 85 | -------------------------------------------------------------------------------- /xudp/route.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include 15 | #include 16 | #include 17 | 18 | #include "common.h" 19 | #include "log.h" 20 | #include "ifapi.h" 21 | #include "route.h" 22 | #include "list.h" 23 | #include "xudp_types.h" 24 | 25 | 26 | #define rt24_num (1<<24) 27 | #define rt24_rule_num (1<<8) 28 | 29 | #define debug 0 30 | 31 | #if debug 32 | #define IP(a, b, c, d) ntohl((a << 24) + (b << 16) + (c << 8) + d) 33 | #define IPX(a, b, c, d) ((a << 24) + (b << 16) + (c << 8) + d) 34 | #define IPS(i) i>>24,i>>16&0xff,i>>8&0xff,i&0xff 35 | #define IPSH(i) IPS(ntohl(i)) 36 | 37 | static u8 route_get(struct route *r, u32 dst) 38 | { 39 | struct nl_route *nr; 40 | 41 | list_for_each_entry(nr, &r->nr, list) { 42 | if ((dst & nr->mask) == nr->dst_h) 43 | return nr->index + 1; 44 | } 45 | 46 | return 0; // not found goto default 47 | } 48 | #endif 49 | 50 | static void logdebug_route(struct route *r, u32 hash, u8 off, 51 | struct route_rule *rule) 52 | { 53 | char buf_dst[20]; 54 | char buf_src[20]; 55 | char buf_next[20]; 56 | const char *dst, *src, *next; 57 | 58 | dst = inet_ntop(AF_INET, &rule->dst, buf_dst, sizeof(buf_dst)); 59 | src = inet_ntop(AF_INET, &rule->pref_src, buf_src, sizeof(buf_src)); 60 | next = inet_ntop(AF_INET, &rule->next_hop, buf_next, sizeof(buf_next)); 61 | 62 | logdebug(r->log, "route: %s/%d src %s via %s dev %d index: %d " 63 | "data: %p\n", 64 | dst, rule->dst_len, src, next, rule->ifid, 65 | rule->index, rule->data); 66 | } 67 | 68 | static int route_init_rt8(struct route *r, struct route_rule *rule, u8 def) 69 | { 70 | u32 len, off, prefix; 71 | struct route_rt8 *rt8; 72 | 73 | prefix = rule->dst_h >> 8; 74 | 75 | list_for_each_entry(rt8, &r->rt8, list) { 76 | if (rt8->prefix == prefix) 77 | goto found; 78 | } 79 | 80 | rt8 = malloc(sizeof(*rt8) + (1<<8)); 81 | memset(rt8, def, sizeof(*rt8) + (1<<8)); 82 | rt8->prefix = prefix; 83 | list_insert(&r->rt8, &rt8->list); 84 | 85 | found: 86 | rule->rt8 = rt8; 87 | 88 | off = rule->dst_h & 0xff; 89 | len = 1 << (32 - rule->dst_len); 90 | 91 | memset(rt8->rt8 + off, rule->index, len); 92 | return 0; 93 | } 94 | 95 | static int route_get_osroute(struct list_head *osroute, struct log *log) 96 | { 97 | int ret; 98 | 99 | ret = nl_route(osroute, log); 100 | if (ret < 0) { 101 | logerr(log, "get sys route fail\n"); 102 | return -1; 103 | } 104 | 105 | // save one for default 106 | if (ret > rt24_rule_num - 1) { 107 | logerr(log, "too many route rule\n"); 108 | nl_route_free(osroute); 109 | return -1; 110 | } 111 | 112 | ret += 1; 113 | 114 | return ret; 115 | } 116 | 117 | static int route_init_rule(struct route *r) 118 | { 119 | struct nl_route *nr; 120 | struct route_rule *r1, *r2; 121 | int i, j, n, size; 122 | struct list_head osroute; 123 | bool got_default = false; 124 | 125 | INIT_LIST_HEAD(&osroute); 126 | 127 | n = route_get_osroute(&osroute, r->log); 128 | if (n < 0) 129 | return -1; 130 | 131 | size = n * sizeof(*r->rules); 132 | r->rules = malloc(size); 133 | memset(r->rules, 0, size); 134 | 135 | list_for_each_entry(nr, &osroute, list) { 136 | ++r->rules_n; 137 | 138 | if (!nr->dst_len) 139 | got_default = true; 140 | 141 | r1 = r->rules + nr->index; 142 | 143 | r1->index = nr->index; 144 | r1->dst = nr->dst; 145 | r1->dst_h = nr->dst_h; 146 | r1->dst_len = nr->dst_len; 147 | r1->ifid = nr->ifid; 148 | r1->pref_src = nr->pref_src; 149 | r1->next_hop = nr->next_hop; 150 | } 151 | 152 | if (!got_default) 153 | ++r->rules_n; 154 | 155 | nl_route_free(&osroute); 156 | 157 | for (i = 0; i < r->rules_n; ++i) { 158 | r1 = r->rules + i; 159 | 160 | logdebug_route(r, 0, 0, r1); 161 | 162 | if (r1->issub) 163 | continue; 164 | 165 | /* check for the same dst rules */ 166 | for (j = i + 1; j < r->rules_n; ++j) { 167 | r2 = r->rules + j; 168 | 169 | if (r2->dst_len != r1->dst_len) 170 | continue; 171 | 172 | if (r2->dst != r1->dst) 173 | continue; 174 | 175 | r2->issub = true; 176 | r2->next = r1->next; 177 | r1->next = r2; 178 | } 179 | 180 | } 181 | 182 | loginfo(r->log, "route: got %d rules from system.\n\n", r->rules_n); 183 | 184 | return 0; 185 | } 186 | 187 | static int route_init_rt24(struct route *r) 188 | { 189 | struct route_rule *r1; 190 | u32 num, off; 191 | int i; 192 | 193 | for (i = r->rules_n - 1; i >= 0; --i) { 194 | r1 = r->rules + i; 195 | 196 | if (log_enable(r->log, DEBUG)) 197 | logdebug_route(r, 0, 0, r1); 198 | 199 | if (r1->dst_len == 0) 200 | continue; 201 | 202 | if (r1->issub) 203 | continue; 204 | 205 | off = r1->dst_h >> 8; 206 | if (r1->dst_len >= 24) { 207 | num = 1; 208 | route_init_rt8(r, r1, r->rt24[off]); 209 | 210 | } else { 211 | num = (1 << (24 - r1->dst_len)); 212 | } 213 | 214 | logdebug(r->log, "route: init rt24. hash: %u off: %d num: %d\n", 215 | off, r1->index, num); 216 | 217 | memset(r->rt24 + off, r1->index, num); 218 | } 219 | 220 | return 0; 221 | } 222 | 223 | struct route_rule *route_lookup(struct route *r, u32 dst) 224 | { 225 | struct route_rule *rule; 226 | u32 hash; 227 | u8 off; 228 | 229 | hash = dst >> 8; 230 | 231 | off = r->rt24[hash]; 232 | 233 | rule = r->rules + off; 234 | 235 | if (log_enable(r->log, DEBUG)) 236 | logdebug_route(r, hash, off, rule); 237 | 238 | if (rule->rt8) { 239 | hash = dst & 0xff; 240 | off = rule->rt8->rt8[hash]; 241 | rule = r->rules + off; 242 | 243 | logdebug(r->log, "route lookup1. hash: %d off: %d\n", hash, off); 244 | } 245 | 246 | /* lookup done */ 247 | return rule; 248 | } 249 | 250 | struct route *route_init(struct log *log) 251 | { 252 | struct route *r; 253 | int ret; 254 | 255 | zobj(r); 256 | 257 | r->rt24 = anon_map(rt24_num * sizeof(*r->rt24)); 258 | if (MAP_FAILED == r->rt24) { 259 | free(r); 260 | logerr(log, "route init alloc by mmap fail.\n"); 261 | return NULL; 262 | } 263 | 264 | r->log = log; 265 | 266 | INIT_LIST_HEAD(&r->rt8); 267 | 268 | ret = route_init_rule(r); 269 | 270 | if (ret) 271 | return NULL; 272 | 273 | route_init_rt24(r); 274 | 275 | return r; 276 | } 277 | 278 | void route_free(struct route *r) 279 | { 280 | struct route_rt8 *rt8, *safe; 281 | 282 | munmap(r->rt24, rt24_num * sizeof(*r->rt24)); 283 | 284 | list_for_each_entry_safe(rt8, safe, &r->rt8, list) { 285 | free(rt8); 286 | } 287 | 288 | free(r->rules); 289 | free(r); 290 | } 291 | 292 | #if debug 293 | static int show(struct route *r, u8 a, u8 b, u8 c, u8 d) 294 | { 295 | struct route_rule *rule; 296 | 297 | u32 ip; 298 | 299 | rule = route_lookup(r, IPX(a,b,c,d)); 300 | printf("lookup: %d.%d.%d.%d next_hop: %d.%d.%d.%d src: %d.%d.%d.%d dev: %d\n", a, b, c, d, IPSH(rule->next_hop), IPSH(rule->pref_src), rule->ifid); 301 | 302 | rule = r->rules + route_get(r, IPX(a,b,c,d)); 303 | printf(" %d.%d.%d.%d next_hop: %d.%d.%d.%d src: %d.%d.%d.%d dev: %d\n", a, b, c, d, IPSH(rule->next_hop), IPSH(rule->pref_src), rule->ifid); 304 | 305 | } 306 | 307 | int main() 308 | { 309 | struct route *r; 310 | struct log log = {}; 311 | u32 ip; 312 | 313 | log.level = LOG_LEVEL_DEBUG; 314 | 315 | r = route_init(&log); 316 | 317 | show(r, 192, 168, 100, 1); 318 | show(r, 192, 168, 122, 1); 319 | show(r, 1, 1, 1, 1); 320 | show(r, 10, 1, 1, 1); 321 | show(r, 11,160,229,128); 322 | show(r, 11,160,229,28); 323 | } 324 | #endif 325 | -------------------------------------------------------------------------------- /xudp/route.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __ROUTE_H__ 15 | #define __ROUTE_H__ 16 | 17 | #include "list.h" 18 | #include "log.h" 19 | typedef __u32 u32; 20 | 21 | typedef unsigned char u8; 22 | 23 | struct route_rt8 { 24 | struct list_head list; 25 | 26 | u32 prefix; 27 | char rt8[]; 28 | }; 29 | 30 | struct route_rule { 31 | __be32 pref_src; 32 | __be32 next_hop; 33 | int ifid; 34 | int index; 35 | struct route_rt8 *rt8; 36 | void *data; 37 | 38 | 39 | u32 dst_h; 40 | __be32 dst; 41 | char dst_len; 42 | 43 | // for the same next_hop and dst_len 44 | struct route_rule *next; 45 | bool issub; 46 | }; 47 | 48 | struct route { 49 | struct list_head rt8; 50 | struct log *log; 51 | 52 | char *rt24; 53 | struct route_rule *rules; 54 | u8 rules_n; 55 | }; 56 | 57 | struct route_rule *route_lookup(struct route *r, u32 dst); 58 | struct route *route_init(struct log *log); 59 | void route_free(struct route *r); 60 | #endif 61 | 62 | 63 | -------------------------------------------------------------------------------- /xudp/route6.c: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | 5 | #include "common.h" 6 | #include "log.h" 7 | #include "ifapi.h" 8 | #include "route6.h" 9 | #include "list.h" 10 | #include "xudp_types.h" 11 | #include "ip6.h" 12 | 13 | #define RT_MAIN_NUM (1<<24) 14 | #define RT_SUB_NUM (1<<8) 15 | #define RULES_MAX (1<<16) 16 | 17 | static int ru_sub_prefix_len(struct route_rule6 *ru) 18 | { 19 | int l; 20 | 21 | if (ru->dst_len <= 24) 22 | return 0; 23 | 24 | l = ru->dst_len & (~0x7); 25 | if (l == ru->dst_len) 26 | return l - 8; 27 | 28 | return l; 29 | } 30 | 31 | static void log_os_rule(struct nl_route *nr, struct log *log) 32 | { 33 | char buf_dst[INET6_ADDRSTRLEN + 1]; 34 | char buf_src[INET6_ADDRSTRLEN + 1]; 35 | char buf_via[INET6_ADDRSTRLEN + 1]; 36 | 37 | inet_ntop(AF_INET6, &nr->dst6, buf_dst, sizeof(buf_dst)); 38 | inet_ntop(AF_INET6, &nr->next_hop6, buf_via, sizeof(buf_via)); 39 | inet_ntop(AF_INET6, &nr->pref_src6, buf_src, sizeof(buf_src)); 40 | 41 | loginfo(log, "os route: %d %s/%d via %s src %s dev %d\n", 42 | nr->index, buf_dst, nr->dst_len, buf_via, buf_src, 43 | nr->ifid); 44 | } 45 | 46 | static struct route_rule6* route6_get_osrules(struct route6 *r6, struct log *log) 47 | { 48 | struct route_rule6 *rules, *ru; 49 | bool got_default = false; 50 | struct list_head osrules; 51 | struct nl_route *nr; 52 | int ret, n, size; 53 | 54 | INIT_LIST_HEAD(&osrules); 55 | 56 | ret = nl_route6(&osrules, log); 57 | if (ret < 0) { 58 | logerr(log, "get sys route fail\n"); 59 | return NULL; 60 | } 61 | 62 | n = ret; 63 | 64 | // save one for default 65 | if (ret > RULES_MAX - 1) { 66 | nl_route_free(&osrules); 67 | logerr(log, "too many route rule\n"); 68 | return NULL; 69 | } 70 | 71 | n += 1; 72 | 73 | size = n * sizeof(struct route_rule6); 74 | rules = malloc(size); 75 | if (!rules) 76 | return NULL; 77 | 78 | memset(rules, 0, size); 79 | 80 | // copy from os route 81 | list_for_each_entry(nr, &osrules, list) { 82 | ++r6->rules_n; 83 | 84 | if (!nr->dst_len) 85 | got_default = true; 86 | 87 | ru = rules + nr->index; 88 | 89 | log_os_rule(nr, log); 90 | 91 | ru->index = nr->index; 92 | ru->dst_len = nr->dst_len; 93 | ru->ifid = nr->ifid; 94 | 95 | ip6_cpy(&ru->dst, &nr->dst6); 96 | ip6_cpy(&ru->pref_src, &nr->pref_src6); 97 | ip6_cpy(&ru->next_hop, &nr->next_hop6); 98 | 99 | if (ip6_is_zero(&ru->next_hop)) 100 | ru->with_next_hop = false; 101 | else 102 | ru->with_next_hop = true; 103 | } 104 | 105 | loginfo(r6->log, "route: got %d rules from system.\n", r6->rules_n); 106 | 107 | if (!got_default) // rules[0] is empty, no default route 108 | ++r6->rules_n; 109 | 110 | nl_route_free(&osrules); 111 | 112 | return rules; 113 | } 114 | 115 | static struct route_map6 *route6_map_get(struct route6 *r6, 116 | struct route_rule6 *ru) 117 | { 118 | struct route_rule6 *t; 119 | struct route_map6 *m; 120 | int len, i; 121 | 122 | len = ru->dst_len & (~0x7); 123 | 124 | for (; len >= 24; len = len - 8) { 125 | for (i = r6->rules_n - 1; i >= 1; --i) { 126 | t = r6->rules + i; 127 | if (!t->map) 128 | continue; 129 | 130 | m = t->map; 131 | if (m->dst_len != len) 132 | continue; 133 | 134 | if (ip6_cmp(m->dst, &ru->dst, len)) 135 | return m; 136 | } 137 | } 138 | 139 | return r6->map; 140 | } 141 | 142 | static void mapset(struct route_map6 *m, rt_map_item v, u32 off, u32 num) 143 | { 144 | int i; 145 | 146 | for (i = 0; i < num; ++i) 147 | m->map[off + i] = v; 148 | } 149 | 150 | static void log_map_new(struct route_map6 *m, struct log *log, int def) 151 | { 152 | struct in6_addr dst = {}; 153 | char buf[100]; 154 | 155 | ip6_copy_prefix(&dst, m->dst, m->dst_len); 156 | 157 | inet_ntop(AF_INET6, &dst, buf, sizeof(buf)); 158 | 159 | logdebug(log, "create new map(%p) %s/%d def: %d\n", m, buf, m->dst_len, 160 | def); 161 | } 162 | 163 | static void log_map_set(struct log *log, struct route_map6 *m, 164 | struct route_rule6 *ru, u32 off, 165 | u32 num, u32 ref) 166 | { 167 | char buf[100]; 168 | 169 | inet_ntop(AF_INET6, m->dst, buf, sizeof(buf)); 170 | 171 | logdebug(log, "set ru %d to map(%p) %s/%d off: %x num: %d\n", 172 | ru->index, m, buf, m->dst_len, off, num); 173 | } 174 | 175 | static struct route_map6 *route6_map_new(struct route6 *r6, int def, 176 | struct in6_addr *dst, u8 len) 177 | { 178 | struct route_map6 *m; 179 | int size, num; 180 | 181 | m = malloc(sizeof(*m)); 182 | if (!m) 183 | return NULL; 184 | 185 | num = len == 0 ? RT_MAIN_NUM : RT_SUB_NUM; 186 | 187 | size = sizeof(rt_map_item) * num; 188 | 189 | m->map = malloc(size); 190 | if (!m->map) 191 | return NULL; 192 | 193 | memset(m->map, 0, size); 194 | 195 | m->dst = dst; 196 | m->dst_len = len; 197 | m->ref = 0; 198 | m->def = def; 199 | 200 | log_map_new(m, r6->log, def); 201 | 202 | return m; 203 | } 204 | 205 | static int route6_map_set(struct route6 *r6, struct route_map6 *m, 206 | struct route_rule6 *ru) 207 | { 208 | int map_size; 209 | u32 off, num; 210 | 211 | map_size = m->dst_len ? 8 : 24; 212 | 213 | off = ip6_slice(&ru->dst, m->dst_len, map_size); 214 | 215 | if (ru->dst_len > m->dst_len + map_size) { 216 | ru->map = route6_map_new(r6, m->map[off], &ru->dst, 217 | ru_sub_prefix_len(ru)); 218 | if (!ru->map) 219 | return -1; 220 | 221 | if (route6_map_set(r6, ru->map, ru)) 222 | return -1; 223 | 224 | num = 1; 225 | m->map[off] = ru->index; 226 | } else { 227 | num = 1 << (m->dst_len + map_size - ru->dst_len); 228 | mapset(m, ru->index, off, num); 229 | } 230 | 231 | m->ref += 1; 232 | 233 | log_map_set(r6->log, m, ru, off, num, m->ref); 234 | 235 | return 0; 236 | } 237 | 238 | static void route6_map_free(struct route6 *r6, bool noref) 239 | { 240 | struct route_rule6 *ru; 241 | u32 i; 242 | 243 | for (i = r6->rules_n - 1; i >= 1; --i) { 244 | ru = r6->rules + i; 245 | if (ru->issub) 246 | continue; 247 | 248 | if (ru->map) { 249 | if (noref && ru->map->ref) 250 | continue; 251 | 252 | free(ru->map->map); 253 | free(ru->map); 254 | } 255 | 256 | ru->map = NULL; 257 | } 258 | } 259 | 260 | static int route6_map_init(struct route6 *r6) 261 | { 262 | struct route_rule6 *ru; 263 | struct route_map6 *m; 264 | u32 i; 265 | 266 | m = route6_map_new(r6, 0, &r6->rules[0].dst, 0); 267 | if (!m) 268 | return -1; 269 | 270 | r6->map = m; 271 | 272 | for (i = r6->rules_n - 1; i >= 1; --i) { 273 | ru = r6->rules + i; 274 | if (ru->issub) 275 | continue; 276 | 277 | m = route6_map_get(r6, ru); 278 | if (route6_map_set(r6, m, ru)) 279 | return -1; 280 | } 281 | 282 | route6_map_free(r6, true); 283 | 284 | return 0; 285 | } 286 | 287 | static int route6_check_rules(struct route6 *r6) 288 | { 289 | struct route_rule6 *r1, *r2; 290 | int i, j; 291 | 292 | // found the same dst 293 | for (i = 0; i < r6->rules_n; ++i) { 294 | r1 = r6->rules + i; 295 | if (r1->issub) 296 | continue; 297 | 298 | /* check for the same dst rules */ 299 | for (j = i + 1; j < r6->rules_n; ++j) { 300 | r2 = r6->rules + j; 301 | 302 | if (r2->dst_len != r1->dst_len) 303 | continue; 304 | 305 | if (!ip6_eq(&r2->dst, &r1->dst)) 306 | continue; 307 | 308 | r2->issub = true; 309 | r2->next = r1->next; 310 | r1->next = r2; 311 | } 312 | } 313 | 314 | return 0; 315 | } 316 | 317 | struct route6 *route6_init(struct log *log) 318 | { 319 | struct route6 *r6; 320 | int ret; 321 | 322 | zobj(r6); 323 | if (!r6) 324 | return NULL; 325 | 326 | r6->log = log; 327 | 328 | r6->rules = route6_get_osrules(r6, r6->log); 329 | if (!r6->rules) { 330 | free(r6); 331 | return NULL; 332 | } 333 | 334 | route6_check_rules(r6); 335 | 336 | ret = route6_map_init(r6); 337 | if (ret) { 338 | route6_free(r6); 339 | return NULL; 340 | } 341 | 342 | return r6; 343 | } 344 | 345 | void route6_free(struct route6 *r6) 346 | { 347 | route6_map_free(r6, false); 348 | free(r6->rules); 349 | free(r6); 350 | } 351 | 352 | -------------------------------------------------------------------------------- /xudp/route6.h: -------------------------------------------------------------------------------- 1 | #ifndef __ROUTE6_H__ 2 | #define __ROUTE6_H__ 3 | 4 | #include "list.h" 5 | #include "log.h" 6 | #include "ip6.h" 7 | 8 | typedef u16 rt_map_item; 9 | 10 | #define ip6_prefix24(ip) ((ip[0] << 16) + (ip[1] << 8) + ip[2]) 11 | #define ip6_slice8(ip, pl) (ip[pl >> 3]) 12 | 13 | struct route_map6 { 14 | rt_map_item *map; 15 | struct in6_addr *dst; 16 | u8 dst_len; 17 | int ref; 18 | u32 def; 19 | }; 20 | 21 | struct route_rule6 { 22 | struct in6_addr dst; 23 | u8 dst_len; 24 | bool with_next_hop; 25 | 26 | struct in6_addr pref_src; 27 | struct in6_addr next_hop; 28 | 29 | int ifid; 30 | int index; 31 | 32 | void *data; 33 | 34 | // for the same next_hop and dst_len 35 | struct route_rule6 *next; 36 | struct route_map6 *map; 37 | 38 | bool issub; 39 | }; 40 | 41 | struct route6 { 42 | struct log *log; 43 | 44 | struct route_map6 *map; 45 | 46 | struct route_rule6 *rules; 47 | 48 | u32 rules_n; 49 | }; 50 | 51 | static inline struct route_rule6 *route6_lookup(struct route6 *r6, struct in6_addr *dst) 52 | { 53 | struct route_rule6 *rule; 54 | rt_map_item item; 55 | u32 hash; 56 | 57 | hash = ip6_prefix24(dst->s6_addr); 58 | 59 | item = r6->map->map[hash]; 60 | 61 | while (true) { 62 | rule = r6->rules + item; 63 | 64 | if (!rule->map) 65 | return rule; 66 | 67 | hash = ip6_slice8(dst->s6_addr, rule->map->dst_len); 68 | item = rule->map->map[hash]; 69 | 70 | if (item == 0) 71 | return r6->rules + rule->map->def; 72 | 73 | if (item == rule->index) 74 | return rule; 75 | } 76 | } 77 | 78 | struct route6 *route6_init(struct log *log); 79 | void route6_free(struct route6 *r6); 80 | #endif 81 | 82 | 83 | -------------------------------------------------------------------------------- /xudp/types.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __TYPES_H__ 15 | #define __TYPES_H__ 16 | 17 | #define TX_BATCH_SIZE 100 18 | 19 | enum{ 20 | XDP_OFFSET_RX, 21 | XDP_OFFSET_TX, 22 | XDP_OFFSET_FR, 23 | XDP_OFFSET_CR, 24 | }; 25 | 26 | struct xdp_ring_offset_v1 { 27 | __u64 producer; 28 | __u64 consumer; 29 | __u64 desc; 30 | }; 31 | 32 | struct xdp_mmap_offsets_v1 { 33 | struct xdp_ring_offset_v1 rx; 34 | struct xdp_ring_offset_v1 tx; 35 | struct xdp_ring_offset_v1 fr; 36 | struct xdp_ring_offset_v1 cr; 37 | }; 38 | 39 | 40 | struct xdp_ring_offset_v2 { 41 | __u64 producer; 42 | __u64 consumer; 43 | __u64 desc; 44 | __u64 flags; 45 | }; 46 | 47 | struct xdp_mmap_offsets_v2 { 48 | struct xdp_ring_offset_v2 rx; 49 | struct xdp_ring_offset_v2 tx; 50 | struct xdp_ring_offset_v2 fr; /* Fill */ 51 | struct xdp_ring_offset_v2 cr; /* Completion */ 52 | }; 53 | #define XSK_V1 1 54 | #define XSK_V2 2 55 | 56 | 57 | #ifndef XDP_OPTIONS 58 | #define XDP_OPTIONS 8 59 | 60 | struct xdp_options { 61 | __u32 flags; 62 | }; 63 | #endif 64 | 65 | 66 | #ifndef XDP_OPTIONS_ZEROCOPY 67 | /* Flags for the flags field of struct xdp_options */ 68 | #define XDP_OPTIONS_ZEROCOPY (1 << 0) 69 | #endif 70 | 71 | #endif 72 | 73 | 74 | -------------------------------------------------------------------------------- /xudp/xsk.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __XSKMAP_H__ 15 | #define __XSKMAP_H__ 16 | 17 | int xudp_tx_xsk_init(xudp *x); 18 | void xudp_tx_xsk_free(xudp *x); 19 | int xudp_umem_check(xudp *x); 20 | int xudp_group_create_all(xudp *x); 21 | 22 | int xsk_cq_cache_init(xudp *x, struct txch *txch, 23 | struct xdp_umem *umem); 24 | 25 | 26 | #endif 27 | 28 | 29 | -------------------------------------------------------------------------------- /xudp/xudp_map.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include 15 | #include 16 | #include 17 | #include "bpf.h" 18 | #include "kern.h" 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include 24 | #include 25 | 26 | enum { 27 | opt_null, 28 | opt_update, 29 | opt_read, 30 | opt_list, 31 | }; 32 | 33 | static u32 current_ns_map[100]; 34 | static bool show_map_of_other_ns; 35 | 36 | static int show_stats_ts(int fd) 37 | { 38 | struct tm *tm_info; 39 | int key, err; 40 | time_t value; 41 | char buf[512]; 42 | 43 | key = XUDP_MAP_TS; 44 | 45 | err = bpf_map_lookup_elem(fd, &key, &value); 46 | if (err) 47 | return -1; 48 | 49 | tm_info = localtime(&value); 50 | strftime(buf, sizeof(buf), "xudp xdp bind: %Y-%m-%d %H:%M:%S", tm_info); 51 | 52 | printf("# %s\n\n", buf); 53 | return 0; 54 | } 55 | 56 | static int init_ns() 57 | { 58 | u32 nsid, value; 59 | struct bpf_map_info info; 60 | int id = 0, key, i, fd, err, num; 61 | 62 | key = XUDP_MAP_NS; 63 | 64 | nsid = net_namespace_id(); 65 | 66 | while (true) { 67 | id = bpf_next_id(id); 68 | if (id < 0) 69 | break; 70 | 71 | if (bpf_get_info(id, &info)) 72 | return -1; 73 | 74 | if (0 == strcmp(MAP_STATS, info.name)) { 75 | fd = bpf_get_fd(id); 76 | if (fd < 0) 77 | return -1; 78 | 79 | err = bpf_map_lookup_elem(fd, &key, &value); 80 | if (err) 81 | continue; 82 | 83 | if (value != nsid) 84 | continue; 85 | 86 | goto found; 87 | } 88 | } 89 | return -1; 90 | found: 91 | show_stats_ts(fd); 92 | 93 | key = XUDP_MAP_NUM; 94 | err = bpf_map_lookup_elem(fd, &key, &num); 95 | if (err) 96 | return -1; 97 | 98 | key = XUDP_MAP_ID; 99 | 100 | for (i = 0; i < num; ++i, ++key) { 101 | err = bpf_map_lookup_elem(fd, &key, &value); 102 | if (err) 103 | return -1; 104 | 105 | current_ns_map[i] = value; 106 | } 107 | 108 | current_ns_map[i] = info.id; 109 | 110 | return 0; 111 | } 112 | 113 | static bool is_current_ns_map(u32 id) 114 | { 115 | u32 i; 116 | 117 | for (i = 0; current_ns_map[i]; ++i) { 118 | if (current_ns_map[i] == id) 119 | return true; 120 | } 121 | 122 | return false; 123 | } 124 | 125 | static inline int bpf_lookup_map_ns(char *name) 126 | { 127 | struct bpf_map_info info; 128 | int id = 0; 129 | 130 | while (true) { 131 | id = bpf_next_id(id); 132 | if (id < 0) 133 | break; 134 | 135 | if (!is_current_ns_map(id)) 136 | continue; 137 | 138 | if (bpf_get_info(id, &info)) 139 | return -1; 140 | 141 | if (strcmp(name, info.name)) 142 | continue; 143 | 144 | return id; 145 | 146 | } 147 | 148 | return -1; 149 | } 150 | 151 | static inline int map_list(int argc, char *argv[]) 152 | { 153 | struct bpf_map_info info; 154 | int id = 0; 155 | 156 | if (argc >= 3 && 0 == strcmp(argv[2], "-a")) 157 | show_map_of_other_ns = true; 158 | 159 | while (true) { 160 | id = bpf_next_id(id); 161 | if (id < 0) { 162 | if (errno != ENOENT) 163 | printf("find map err. %s\n", strerror(errno)); 164 | 165 | break; 166 | } 167 | 168 | if (!show_map_of_other_ns && !is_current_ns_map(id)) 169 | continue; 170 | 171 | if (bpf_get_info(id, &info)) 172 | return -1; 173 | 174 | printf("map %-15s type: %2u id: %5u vsize: %5u max_entries: %8u flags: %3u ifindex: %3u dev: %3llu ino: %3llu\n", 175 | info.name, info.type, info.id, info.value_size, info.max_entries, 176 | info.map_flags, info.ifindex, info.netns_dev, info.netns_ino); 177 | } 178 | return 0; 179 | } 180 | 181 | static int dump_ipport() 182 | { 183 | struct kern_ipport ipport; 184 | struct in_addr addr; 185 | int id, fd, i, err; 186 | int key = 0; 187 | 188 | id = bpf_lookup_map_ns(MAP_IPPORT); 189 | if (id == -1) { 190 | printf("%s. map not found. %s\n", MAP_IPPORT, strerror(errno)); 191 | return 0; 192 | } 193 | 194 | fd = bpf_get_fd(id); 195 | if (fd < 0) 196 | return 0; 197 | 198 | err = bpf_map_lookup_elem(fd, &key, &ipport); 199 | if (err) { 200 | printf("err lookup\n"); 201 | return 0; 202 | } 203 | 204 | for (i = 0; i < ipport.ipport_n; ++i) { 205 | addr.s_addr = ipport.addr[i]; 206 | printf("%s:%u\n", inet_ntoa(addr), ntohs(ipport.port[i])); 207 | } 208 | 209 | char buf[100]; 210 | for (i = 0; i < ipport.ipport6_n; ++i) { 211 | inet_ntop(AF_INET6, &ipport.addr6[i], buf, sizeof(buf)); 212 | printf("%s:%u\n", buf, ntohs(ipport.port6[i])); 213 | } 214 | 215 | return 0; 216 | } 217 | 218 | static void exit_help(void) 219 | { 220 | printf("xudp-map list [-a]\n"); 221 | printf("xudp-map ipport -- dump ipport\n"); 222 | printf("xudp-map [value]\n"); 223 | exit(0); 224 | } 225 | 226 | int main(int argc, char *argv[]) 227 | { 228 | int id, fd, key, value, opt, err; 229 | char *map_name; 230 | 231 | opt = opt_null; 232 | key = 0; 233 | 234 | if (init_ns()) { 235 | printf("fail init ns. try sudo.\n"); 236 | return -1; 237 | } 238 | 239 | if (argc <= 1 || 0 == strcmp(argv[1], "list")) 240 | return map_list(argc, argv); 241 | 242 | if (0 == strcmp(argv[1], "help")) 243 | exit_help(); 244 | 245 | if (0 == strcmp(argv[1], "-h")) 246 | exit_help(); 247 | 248 | if (0 == strcmp(argv[1], "ipport")) 249 | return dump_ipport(); 250 | 251 | if (0 == strcmp(argv[1], "update")) { 252 | if (argc != 5) 253 | exit_help(); 254 | 255 | opt = opt_update; 256 | value = atoi(argv[4]); 257 | } 258 | 259 | if (0 == strcmp(argv[1], "read")) { 260 | if (argc != 4) 261 | exit_help(); 262 | 263 | opt = opt_read; 264 | } 265 | 266 | if (opt == opt_null) 267 | exit_help(); 268 | 269 | map_name = argv[2]; 270 | key = atoi(argv[3]); 271 | 272 | id = bpf_lookup_map_ns(map_name); 273 | if (id == -1) { 274 | printf("%s. map not found. %s\n", map_name, strerror(errno)); 275 | return -1; 276 | } 277 | 278 | fd = bpf_get_fd(id); 279 | if (fd < 0) 280 | return -1; 281 | 282 | if (opt == opt_update) 283 | return bpf_map_update_elem(fd, &key, &value, 0); 284 | 285 | err = bpf_map_lookup_elem(fd, &key, &value); 286 | if (!err) 287 | printf("%d\n", value); 288 | 289 | return err; 290 | } 291 | -------------------------------------------------------------------------------- /xudp/xudp_route.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #include 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | 23 | #include "route.h" 24 | #include "log.h" 25 | #include "xudp.h" 26 | 27 | #define IP(a, b, c, d) ntohl((a << 24) + (b << 16) + (c << 8) + d) 28 | #define IPX(a, b, c, d) ((a << 24) + (b << 16) + (c << 8) + d) 29 | #define IPS(i) i>>24,i>>16&0xff,i>>8&0xff,i&0xff 30 | #define IPSH(i) IPS(ntohl(i)) 31 | 32 | int main(int argc, char **argv) 33 | { 34 | struct log log = {}; 35 | struct route_rule *rule; 36 | struct route *r; 37 | u32 dst; 38 | 39 | log.level = XUDP_LOG_DEBUG; 40 | 41 | 42 | r = route_init(&log); 43 | 44 | if (!r) { 45 | printf("route init fail\n"); 46 | return -1; 47 | } 48 | 49 | if (argc == 2) { 50 | dst = inet_addr(argv[1]); 51 | rule = route_lookup(r, ntohl(dst)); 52 | 53 | printf("\nlookup: %s id: %d next_hop: %d.%d.%d.%d src: %d.%d.%d.%d dev: %d\n", 54 | argv[1], rule->index, 55 | IPSH(rule->next_hop), IPSH(rule->pref_src), rule->ifid); 56 | } 57 | 58 | return 0; 59 | } 60 | -------------------------------------------------------------------------------- /xudp/xudp_route6.c: -------------------------------------------------------------------------------- 1 | 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | 8 | #include "route6.h" 9 | 10 | int main(int argc, char *argv[]) 11 | { 12 | struct route_rule6 *rt6; 13 | struct route6 *r6; 14 | struct in6_addr a6; 15 | struct log log = {}; 16 | char buf[100]; 17 | int ret; 18 | 19 | log.level = LOG_LEVEL_INFO; 20 | 21 | if (argc == 3) { 22 | if (strcmp("debug", argv[2]) == 0) 23 | log.level = LOG_LEVEL_DEBUG; 24 | } 25 | 26 | r6 = route6_init(&log); 27 | 28 | if (argc == 1) 29 | goto end; 30 | 31 | ret = inet_pton(AF_INET6, argv[1], &a6); 32 | 33 | rt6 = route6_lookup(r6, &a6); 34 | 35 | printf("\n"); 36 | 37 | printf("target: %s\n", argv[1]); 38 | printf("ret: %d\n", ret); 39 | 40 | printf("ifid: %d\n", rt6->ifid); 41 | printf("index: %d\n", rt6->index); 42 | 43 | printf("dstlen: %d\n", rt6->dst_len); 44 | 45 | inet_ntop(AF_INET6, &rt6->dst, buf, sizeof(buf)); 46 | printf("dst: %s\n", buf); 47 | 48 | inet_ntop(AF_INET6, &rt6->next_hop, buf, sizeof(buf)); 49 | printf("via: %s\n", buf); 50 | 51 | inet_ntop(AF_INET6, &rt6->pref_src, buf, sizeof(buf)); 52 | printf("src: %s\n", buf); 53 | 54 | end: 55 | route6_free(r6); 56 | 57 | return 0; 58 | } 59 | -------------------------------------------------------------------------------- /xudp/xudp_types.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2021 Alibaba Group Holding Limited 3 | * Express UDP is licensed under Mulan PSL v2. 4 | * You can use this software according to the terms and conditions of the Mulan PSL v2. 5 | * You may obtain a copy of Mulan PSL v2 at: 6 | * http://license.coscl.org.cn/MulanPSL2 7 | * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, 8 | * EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, 9 | * MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE. 10 | * See the Mulan PSL v2 for more details. 11 | * 12 | */ 13 | 14 | #ifndef __XUDP_TYPES_H__ 15 | #define __XUDP_TYPES_H__ 16 | 17 | #include 18 | #include 19 | 20 | #include 21 | #include 22 | #include 23 | #include 24 | 25 | #include 26 | #include 27 | #include 28 | #include 29 | #include 30 | #include 31 | 32 | #include 33 | #include 34 | #include 35 | 36 | #include 37 | #include 38 | #include 39 | 40 | #include 41 | #include 42 | 43 | #include "packet.h" 44 | #include "bpf.h" 45 | #include "xudp.h" 46 | #include "kern_ops.h" 47 | #include "list.h" 48 | #include "config.h" 49 | #include "types.h" 50 | #include "log.h" 51 | #include "common.h" 52 | #include "queue.h" 53 | #include "ip6.h" 54 | #include "kern.h" 55 | #include "channel.h" 56 | #include "group_api.h" 57 | #include "dump.h" 58 | 59 | struct xdp_umem { 60 | /* list for nic */ 61 | struct list_head list; 62 | 63 | int queue_id; 64 | int sfd; 65 | bool shared; 66 | 67 | char *frames; 68 | u64 frames_size; 69 | struct xdp_umem_uqueue fq; 70 | struct xdp_umem_uqueue cq; 71 | 72 | int version; 73 | 74 | u32 cq_used; 75 | u32 cq_cache_max; 76 | u32 headroom; 77 | u32 tx_proc_n; 78 | }; 79 | 80 | struct txch { 81 | struct xdpsock xsk; 82 | 83 | struct xdp_umem *umem; 84 | 85 | struct tx_frame_info *frame; 86 | /* alloc to user */ 87 | u64 frame_alloc; 88 | /* the frame holded by tx */ 89 | u64 frame_queue; 90 | u64 frame_sent; 91 | 92 | u32 need_commit; 93 | 94 | bool zc; 95 | 96 | /* fast link */ 97 | struct xudp_nic *nic; 98 | }; 99 | 100 | struct xudp_nicxsk{ 101 | /* list for xskmap */ 102 | struct list_head list; 103 | 104 | struct xudp_nic *nic; 105 | struct xudp_xskmap *xskmap; 106 | 107 | struct xsk_group *xsk_group; 108 | u32 xsk_group_size; 109 | }; 110 | 111 | struct xudp_xskmap{ 112 | struct xudp_xskmap *next; 113 | 114 | int id; 115 | u16 arrayid; 116 | int channels_n; 117 | 118 | int mapfd;// bpf xsk map fd 119 | 120 | struct list_head nicxsk; 121 | 122 | struct xudp *x; 123 | 124 | bool freed; 125 | }; 126 | 127 | struct xudp_nic { 128 | struct xudp_nic *next; 129 | 130 | struct xudp *x; 131 | 132 | int nic_index; 133 | int queue; 134 | int txring_n; 135 | 136 | struct nl_info *ni; 137 | 138 | 139 | int link_fd; 140 | 141 | /* this addr include the port and ip. ip may be 0.0.0.0 */ 142 | struct sockaddr_in *addr; 143 | struct sockaddr_in6 *addr6; 144 | 145 | int addr4_n; 146 | int addr6_n; 147 | 148 | int tx_n; 149 | 150 | struct xdp_umem **umem; 151 | }; 152 | 153 | struct xudp { 154 | /* this must first one */ 155 | struct dump dump; 156 | 157 | bool udp_alloc; 158 | int udp_fd; 159 | 160 | struct bpf bpf; 161 | bool bpf_need_free; 162 | 163 | int stats_map; 164 | 165 | int flags; 166 | u32 group_counter; 167 | 168 | struct txch *tx_xsk; 169 | 170 | /* nic */ 171 | struct xudp_nic *nic; 172 | int nic_n; 173 | int queue_n; 174 | 175 | struct log *log; 176 | struct nl_info *nlinfo; 177 | 178 | /* read mostly */ 179 | struct route *route; 180 | struct route6 *route6; 181 | int tx_batch_num; 182 | int frame_size; 183 | int noarp; 184 | 185 | struct xudp_conf conf; 186 | 187 | bool map_xskmaps_active; 188 | bool map_dict_active; 189 | 190 | pthread_spinlock_t xskmap_set_lock; 191 | struct kern_info kern_info; 192 | u32 map_xskmap_set_num; 193 | 194 | struct list_head dummy_xsk; 195 | 196 | /* save all groups without isolate */ 197 | struct xudp_group **groups; 198 | }; 199 | 200 | #endif 201 | 202 | 203 | --------------------------------------------------------------------------------