├── .gitignore ├── COPYING ├── Kbuild ├── Makefile ├── README.md ├── dkms.conf ├── io_peer_mem.c └── test ├── Makefile ├── client.c └── server.c /.gitignore: -------------------------------------------------------------------------------- 1 | *.cmd 2 | *.o 3 | *.ko 4 | modules.order 5 | Module.symvers 6 | .tmp_versions 7 | *.mod.c 8 | test/client 9 | test/server 10 | -------------------------------------------------------------------------------- /COPYING: -------------------------------------------------------------------------------- 1 | 2 | NOTE! This copyright does *not* cover user programs that use kernel 3 | services by normal system calls - this is merely considered normal use 4 | of the kernel, and does *not* fall under the heading of "derived work". 5 | Also note that the GPL below is copyrighted by the Free Software 6 | Foundation, but the instance of code that it refers to (the Linux 7 | kernel) is copyrighted by me and others who actually wrote it. 8 | 9 | Also note that the only valid version of the GPL as far as the kernel 10 | is concerned is _this_ particular version of the license (ie v2, not 11 | v2.2 or v3.x or whatever), unless explicitly otherwise stated. 12 | 13 | Linus Torvalds 14 | 15 | ---------------------------------------- 16 | 17 | GNU GENERAL PUBLIC LICENSE 18 | Version 2, June 1991 19 | 20 | Copyright (C) 1989, 1991 Free Software Foundation, Inc. 21 | 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA 22 | Everyone is permitted to copy and distribute verbatim copies 23 | of this license document, but changing it is not allowed. 24 | 25 | Preamble 26 | 27 | The licenses for most software are designed to take away your 28 | freedom to share and change it. By contrast, the GNU General Public 29 | License is intended to guarantee your freedom to share and change free 30 | software--to make sure the software is free for all its users. This 31 | General Public License applies to most of the Free Software 32 | Foundation's software and to any other program whose authors commit to 33 | using it. (Some other Free Software Foundation software is covered by 34 | the GNU Library General Public License instead.) You can apply it to 35 | your programs, too. 36 | 37 | When we speak of free software, we are referring to freedom, not 38 | price. Our General Public Licenses are designed to make sure that you 39 | have the freedom to distribute copies of free software (and charge for 40 | this service if you wish), that you receive source code or can get it 41 | if you want it, that you can change the software or use pieces of it 42 | in new free programs; and that you know you can do these things. 43 | 44 | To protect your rights, we need to make restrictions that forbid 45 | anyone to deny you these rights or to ask you to surrender the rights. 46 | These restrictions translate to certain responsibilities for you if you 47 | distribute copies of the software, or if you modify it. 48 | 49 | For example, if you distribute copies of such a program, whether 50 | gratis or for a fee, you must give the recipients all the rights that 51 | you have. You must make sure that they, too, receive or can get the 52 | source code. And you must show them these terms so they know their 53 | rights. 54 | 55 | We protect your rights with two steps: (1) copyright the software, and 56 | (2) offer you this license which gives you legal permission to copy, 57 | distribute and/or modify the software. 58 | 59 | Also, for each author's protection and ours, we want to make certain 60 | that everyone understands that there is no warranty for this free 61 | software. If the software is modified by someone else and passed on, we 62 | want its recipients to know that what they have is not the original, so 63 | that any problems introduced by others will not reflect on the original 64 | authors' reputations. 65 | 66 | Finally, any free program is threatened constantly by software 67 | patents. We wish to avoid the danger that redistributors of a free 68 | program will individually obtain patent licenses, in effect making the 69 | program proprietary. To prevent this, we have made it clear that any 70 | patent must be licensed for everyone's free use or not licensed at all. 71 | 72 | The precise terms and conditions for copying, distribution and 73 | modification follow. 74 | 75 | GNU GENERAL PUBLIC LICENSE 76 | TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 77 | 78 | 0. This License applies to any program or other work which contains 79 | a notice placed by the copyright holder saying it may be distributed 80 | under the terms of this General Public License. The "Program", below, 81 | refers to any such program or work, and a "work based on the Program" 82 | means either the Program or any derivative work under copyright law: 83 | that is to say, a work containing the Program or a portion of it, 84 | either verbatim or with modifications and/or translated into another 85 | language. (Hereinafter, translation is included without limitation in 86 | the term "modification".) Each licensee is addressed as "you". 87 | 88 | Activities other than copying, distribution and modification are not 89 | covered by this License; they are outside its scope. The act of 90 | running the Program is not restricted, and the output from the Program 91 | is covered only if its contents constitute a work based on the 92 | Program (independent of having been made by running the Program). 93 | Whether that is true depends on what the Program does. 94 | 95 | 1. You may copy and distribute verbatim copies of the Program's 96 | source code as you receive it, in any medium, provided that you 97 | conspicuously and appropriately publish on each copy an appropriate 98 | copyright notice and disclaimer of warranty; keep intact all the 99 | notices that refer to this License and to the absence of any warranty; 100 | and give any other recipients of the Program a copy of this License 101 | along with the Program. 102 | 103 | You may charge a fee for the physical act of transferring a copy, and 104 | you may at your option offer warranty protection in exchange for a fee. 105 | 106 | 2. You may modify your copy or copies of the Program or any portion 107 | of it, thus forming a work based on the Program, and copy and 108 | distribute such modifications or work under the terms of Section 1 109 | above, provided that you also meet all of these conditions: 110 | 111 | a) You must cause the modified files to carry prominent notices 112 | stating that you changed the files and the date of any change. 113 | 114 | b) You must cause any work that you distribute or publish, that in 115 | whole or in part contains or is derived from the Program or any 116 | part thereof, to be licensed as a whole at no charge to all third 117 | parties under the terms of this License. 118 | 119 | c) If the modified program normally reads commands interactively 120 | when run, you must cause it, when started running for such 121 | interactive use in the most ordinary way, to print or display an 122 | announcement including an appropriate copyright notice and a 123 | notice that there is no warranty (or else, saying that you provide 124 | a warranty) and that users may redistribute the program under 125 | these conditions, and telling the user how to view a copy of this 126 | License. (Exception: if the Program itself is interactive but 127 | does not normally print such an announcement, your work based on 128 | the Program is not required to print an announcement.) 129 | 130 | These requirements apply to the modified work as a whole. If 131 | identifiable sections of that work are not derived from the Program, 132 | and can be reasonably considered independent and separate works in 133 | themselves, then this License, and its terms, do not apply to those 134 | sections when you distribute them as separate works. But when you 135 | distribute the same sections as part of a whole which is a work based 136 | on the Program, the distribution of the whole must be on the terms of 137 | this License, whose permissions for other licensees extend to the 138 | entire whole, and thus to each and every part regardless of who wrote it. 139 | 140 | Thus, it is not the intent of this section to claim rights or contest 141 | your rights to work written entirely by you; rather, the intent is to 142 | exercise the right to control the distribution of derivative or 143 | collective works based on the Program. 144 | 145 | In addition, mere aggregation of another work not based on the Program 146 | with the Program (or with a work based on the Program) on a volume of 147 | a storage or distribution medium does not bring the other work under 148 | the scope of this License. 149 | 150 | 3. You may copy and distribute the Program (or a work based on it, 151 | under Section 2) in object code or executable form under the terms of 152 | Sections 1 and 2 above provided that you also do one of the following: 153 | 154 | a) Accompany it with the complete corresponding machine-readable 155 | source code, which must be distributed under the terms of Sections 156 | 1 and 2 above on a medium customarily used for software interchange; or, 157 | 158 | b) Accompany it with a written offer, valid for at least three 159 | years, to give any third party, for a charge no more than your 160 | cost of physically performing source distribution, a complete 161 | machine-readable copy of the corresponding source code, to be 162 | distributed under the terms of Sections 1 and 2 above on a medium 163 | customarily used for software interchange; or, 164 | 165 | c) Accompany it with the information you received as to the offer 166 | to distribute corresponding source code. (This alternative is 167 | allowed only for noncommercial distribution and only if you 168 | received the program in object code or executable form with such 169 | an offer, in accord with Subsection b above.) 170 | 171 | The source code for a work means the preferred form of the work for 172 | making modifications to it. For an executable work, complete source 173 | code means all the source code for all modules it contains, plus any 174 | associated interface definition files, plus the scripts used to 175 | control compilation and installation of the executable. However, as a 176 | special exception, the source code distributed need not include 177 | anything that is normally distributed (in either source or binary 178 | form) with the major components (compiler, kernel, and so on) of the 179 | operating system on which the executable runs, unless that component 180 | itself accompanies the executable. 181 | 182 | If distribution of executable or object code is made by offering 183 | access to copy from a designated place, then offering equivalent 184 | access to copy the source code from the same place counts as 185 | distribution of the source code, even though third parties are not 186 | compelled to copy the source along with the object code. 187 | 188 | 4. You may not copy, modify, sublicense, or distribute the Program 189 | except as expressly provided under this License. Any attempt 190 | otherwise to copy, modify, sublicense or distribute the Program is 191 | void, and will automatically terminate your rights under this License. 192 | However, parties who have received copies, or rights, from you under 193 | this License will not have their licenses terminated so long as such 194 | parties remain in full compliance. 195 | 196 | 5. You are not required to accept this License, since you have not 197 | signed it. However, nothing else grants you permission to modify or 198 | distribute the Program or its derivative works. These actions are 199 | prohibited by law if you do not accept this License. Therefore, by 200 | modifying or distributing the Program (or any work based on the 201 | Program), you indicate your acceptance of this License to do so, and 202 | all its terms and conditions for copying, distributing or modifying 203 | the Program or works based on it. 204 | 205 | 6. Each time you redistribute the Program (or any work based on the 206 | Program), the recipient automatically receives a license from the 207 | original licensor to copy, distribute or modify the Program subject to 208 | these terms and conditions. You may not impose any further 209 | restrictions on the recipients' exercise of the rights granted herein. 210 | You are not responsible for enforcing compliance by third parties to 211 | this License. 212 | 213 | 7. If, as a consequence of a court judgment or allegation of patent 214 | infringement or for any other reason (not limited to patent issues), 215 | conditions are imposed on you (whether by court order, agreement or 216 | otherwise) that contradict the conditions of this License, they do not 217 | excuse you from the conditions of this License. If you cannot 218 | distribute so as to satisfy simultaneously your obligations under this 219 | License and any other pertinent obligations, then as a consequence you 220 | may not distribute the Program at all. For example, if a patent 221 | license would not permit royalty-free redistribution of the Program by 222 | all those who receive copies directly or indirectly through you, then 223 | the only way you could satisfy both it and this License would be to 224 | refrain entirely from distribution of the Program. 225 | 226 | If any portion of this section is held invalid or unenforceable under 227 | any particular circumstance, the balance of the section is intended to 228 | apply and the section as a whole is intended to apply in other 229 | circumstances. 230 | 231 | It is not the purpose of this section to induce you to infringe any 232 | patents or other property right claims or to contest validity of any 233 | such claims; this section has the sole purpose of protecting the 234 | integrity of the free software distribution system, which is 235 | implemented by public license practices. Many people have made 236 | generous contributions to the wide range of software distributed 237 | through that system in reliance on consistent application of that 238 | system; it is up to the author/donor to decide if he or she is willing 239 | to distribute software through any other system and a licensee cannot 240 | impose that choice. 241 | 242 | This section is intended to make thoroughly clear what is believed to 243 | be a consequence of the rest of this License. 244 | 245 | 8. If the distribution and/or use of the Program is restricted in 246 | certain countries either by patents or by copyrighted interfaces, the 247 | original copyright holder who places the Program under this License 248 | may add an explicit geographical distribution limitation excluding 249 | those countries, so that distribution is permitted only in or among 250 | countries not thus excluded. In such case, this License incorporates 251 | the limitation as if written in the body of this License. 252 | 253 | 9. The Free Software Foundation may publish revised and/or new versions 254 | of the General Public License from time to time. Such new versions will 255 | be similar in spirit to the present version, but may differ in detail to 256 | address new problems or concerns. 257 | 258 | Each version is given a distinguishing version number. If the Program 259 | specifies a version number of this License which applies to it and "any 260 | later version", you have the option of following the terms and conditions 261 | either of that version or of any later version published by the Free 262 | Software Foundation. If the Program does not specify a version number of 263 | this License, you may choose any version ever published by the Free Software 264 | Foundation. 265 | 266 | 10. If you wish to incorporate parts of the Program into other free 267 | programs whose distribution conditions are different, write to the author 268 | to ask for permission. For software which is copyrighted by the Free 269 | Software Foundation, write to the Free Software Foundation; we sometimes 270 | make exceptions for this. Our decision will be guided by the two goals 271 | of preserving the free status of all derivatives of our free software and 272 | of promoting the sharing and reuse of software generally. 273 | 274 | NO WARRANTY 275 | 276 | 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY 277 | FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN 278 | OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES 279 | PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED 280 | OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 281 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS 282 | TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE 283 | PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, 284 | REPAIR OR CORRECTION. 285 | 286 | 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 287 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR 288 | REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, 289 | INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING 290 | OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED 291 | TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY 292 | YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER 293 | PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE 294 | POSSIBILITY OF SUCH DAMAGES. 295 | 296 | END OF TERMS AND CONDITIONS 297 | 298 | How to Apply These Terms to Your New Programs 299 | 300 | If you develop a new program, and you want it to be of the greatest 301 | possible use to the public, the best way to achieve this is to make it 302 | free software which everyone can redistribute and change under these terms. 303 | 304 | To do so, attach the following notices to the program. It is safest 305 | to attach them to the start of each source file to most effectively 306 | convey the exclusion of warranty; and each file should have at least 307 | the "copyright" line and a pointer to where the full notice is found. 308 | 309 | 310 | Copyright (C) 311 | 312 | This program is free software; you can redistribute it and/or modify 313 | it under the terms of the GNU General Public License as published by 314 | the Free Software Foundation; either version 2 of the License, or 315 | (at your option) any later version. 316 | 317 | This program is distributed in the hope that it will be useful, 318 | but WITHOUT ANY WARRANTY; without even the implied warranty of 319 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 320 | GNU General Public License for more details. 321 | 322 | You should have received a copy of the GNU General Public License 323 | along with this program; if not, write to the Free Software 324 | Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA 325 | 326 | 327 | Also add information on how to contact you by electronic and paper mail. 328 | 329 | If the program is interactive, make it output a short notice like this 330 | when it starts in an interactive mode: 331 | 332 | Gnomovision version 69, Copyright (C) year name of author 333 | Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 334 | This is free software, and you are welcome to redistribute it 335 | under certain conditions; type `show c' for details. 336 | 337 | The hypothetical commands `show w' and `show c' should show the appropriate 338 | parts of the General Public License. Of course, the commands you use may 339 | be called something other than `show w' and `show c'; they could even be 340 | mouse-clicks or menu items--whatever suits your program. 341 | 342 | You should also get your employer (if you work as a programmer) or your 343 | school, if any, to sign a "copyright disclaimer" for the program, if 344 | necessary. Here is a sample; alter the names: 345 | 346 | Yoyodyne, Inc., hereby disclaims all copyright interest in the program 347 | `Gnomovision' (which makes passes at compilers) written by James Hacker. 348 | 349 | , 1 April 1989 350 | Ty Coon, President of Vice 351 | 352 | This General Public License does not permit incorporating your program into 353 | proprietary programs. If your program is a subroutine library, you may 354 | consider it more useful to permit linking proprietary applications with the 355 | library. If this is what you want to do, use the GNU Library General 356 | Public License instead of this License. 357 | -------------------------------------------------------------------------------- /Kbuild: -------------------------------------------------------------------------------- 1 | obj-m := io_peer_mem.o 2 | io_peer_mem-y := 3 | 4 | ifdef DEBUG 5 | ccflags-y += -DDEBUG 6 | endif 7 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # 2 | # By default, the build is done against the running linux kernel source. 3 | # To build against a different kernel source tree, set SYSSRC: 4 | # 5 | # make SYSSRC=/path/to/kernel/source 6 | 7 | ifdef SYSSRC 8 | KERNEL_SOURCES = $(SYSSRC) 9 | else 10 | KERNEL_UNAME := $(shell uname -r) 11 | KERNEL_SOURCES = /lib/modules/$(KERNEL_UNAME)/build 12 | endif 13 | 14 | 15 | build: modules 16 | .PHONY: build 17 | 18 | install: modules_install 19 | .PHONY: install 20 | 21 | 22 | %:: 23 | $(MAKE) -C $(KERNEL_SOURCES) M=$$PWD $@ 24 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | io_peer_mem: Peer Memory Client for IO/PFN/MIXEDMAP memory. 2 | =========================================================== 3 | 4 | This kernel module is a client for Yishai Hadas's IB Peer Memory patch 5 | set [1]. The patch allows RDMA transfers with various types of memory 6 | like mmaped devices (that create PFN mappings) and mmaped files from 7 | DAX filesystems, such as those that reside on NVRAM devices (though 8 | these require the kernel patch in [2]). 9 | 10 | This can be tested with either donard_rdma [3] or a patched version of 11 | the IB perftest toolset [4]. 12 | 13 | Some examples include: 14 | 15 | * donard_rdma_server -m /mnt/dax_fs/test.dat 16 | * ib_read_bw -n 20 -R -a --mmap=/mnt/dax_fs/test.dat 17 | * ib_read_bw -n 20 -R -a --mmap=/dev/pfn_mmapable_char_dev 18 | * ib_read_bw -n 20 -R -a --mmap=/sys/bus/pci/devices/0000\:03\:00.0/resource4_wc 19 | 20 | 21 | 22 | --------------------------------------- 23 | 24 | [1] http://comments.gmane.org/gmane.linux.drivers.rdma/21849 25 | 26 | [2] https://github.com/sbates130272/linux-donard/commit/e50a659f074c285968a7404c0da2295093579509 27 | 28 | [3] https://github.com/sbates130272/donard_rdma 29 | 30 | [4] https://github.com/lsgunth/perftest 31 | -------------------------------------------------------------------------------- /dkms.conf: -------------------------------------------------------------------------------- 1 | PACKAGE_NAME="io_peer_mem" 2 | PACKAGE_VERSION="0.1" 3 | 4 | BUILT_MODULE_NAME="io_peer_mem" 5 | DEST_MODULE_LOCATION="/extra" 6 | AUTOINSTALL=yes 7 | REMAKE_INITRD=yes 8 | 9 | MAKE="env CCACHE_DISABLE=1 \ 10 | make -C . SYSSRC=${kernel_source_dir}" 11 | CLEAN="make -C . SYSSRC=${kernel_source_dir} clean" 12 | -------------------------------------------------------------------------------- /io_peer_mem.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2015, PMC-Sierra Inc. 3 | * 4 | * This software is available to you under a choice of one of two 5 | * licenses. You may choose to be licensed under the terms of the GNU 6 | * General Public License (GPL) Version 2, available from the file 7 | * COPYING in the main directory of this source tree, or the 8 | * OpenIB.org BSD license below: 9 | * 10 | * Redistribution and use in source and binary forms, with or 11 | * without modification, are permitted provided that the following 12 | * conditions are met: 13 | * 14 | * - Redistributions of source code must retain the above 15 | * copyright notice, this list of conditions and the following 16 | * disclaimer. 17 | * 18 | * - Redistributions in binary form must reproduce the above 19 | * copyright notice, this list of conditions and the following 20 | * disclaimer in the documentation and/or other materials 21 | * provided with the distribution. 22 | * 23 | * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, 24 | * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF 25 | * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 26 | * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS 27 | * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN 28 | * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN 29 | * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 30 | * SOFTWARE. 31 | */ 32 | 33 | 34 | #include 35 | 36 | #include 37 | #include 38 | #include 39 | #include 40 | #include 41 | 42 | #define NAME "io_peer_mem" 43 | #define VERSION "0.2" 44 | 45 | MODULE_AUTHOR("Logan Gunthorpe"); 46 | MODULE_DESCRIPTION("MMAP'd IO memory plug-in"); 47 | MODULE_LICENSE("Dual BSD/GPL"); 48 | MODULE_VERSION(VERSION); 49 | 50 | #ifdef DEBUG 51 | #define debug_msg(FMT, ARGS...) printk(NAME ": " FMT, ## ARGS) 52 | #else 53 | #define debug_msg(FMT, ARGS...) 54 | #endif 55 | 56 | /** 57 | * This is copied from drivers/media/v4l2-core/videobuf2-memops.c: 58 | * 59 | * This function attempts to acquire an area mapped in the userspace for 60 | * the duration of a hardware mapping. The area is "locked" by performing 61 | * the same set of operation that are done when process calls fork() and 62 | * memory areas are duplicated. 63 | */ 64 | static struct vm_area_struct *get_vma(struct vm_area_struct *vma) 65 | { 66 | struct vm_area_struct *vma_copy; 67 | 68 | vma_copy = kmalloc(sizeof(*vma_copy), GFP_KERNEL); 69 | if (vma_copy == NULL) 70 | return NULL; 71 | 72 | if (vma->vm_ops && vma->vm_ops->open) 73 | vma->vm_ops->open(vma); 74 | 75 | if (vma->vm_file) 76 | get_file(vma->vm_file); 77 | 78 | memcpy(vma_copy, vma, sizeof(*vma)); 79 | 80 | vma_copy->vm_next = NULL; 81 | vma_copy->vm_prev = NULL; 82 | 83 | return vma_copy; 84 | } 85 | 86 | static void put_vma(struct vm_area_struct *vma) 87 | { 88 | if (!vma) 89 | return; 90 | 91 | if (vma->vm_ops && vma->vm_ops->close) 92 | vma->vm_ops->close(vma); 93 | 94 | if (vma->vm_file) 95 | fput(vma->vm_file); 96 | 97 | kfree(vma); 98 | } 99 | 100 | struct context { 101 | unsigned long addr; 102 | size_t size; 103 | struct vm_area_struct *vma; 104 | }; 105 | 106 | static void fault_missing_pages(struct vm_area_struct *vma, unsigned long start, 107 | unsigned long end) 108 | { 109 | unsigned long pfn; 110 | 111 | if (!(vma->vm_flags & VM_MIXEDMAP)) 112 | return; 113 | 114 | for (; start < end; start += PAGE_SIZE) { 115 | if (!follow_pfn(vma, start, &pfn)) 116 | continue; 117 | 118 | handle_mm_fault(current->mm, vma, start, FAULT_FLAG_WRITE); 119 | } 120 | } 121 | 122 | static int is_uverbs_file(struct vm_area_struct *vma) 123 | { 124 | /* There is an ugly corner case leak possible: 125 | * If a malicous userspace process grabs the uverbs file descriptor 126 | * (for /dev/infinband/uverbs*), mmaps it, and tries to register 127 | * that memory via io_peer_mem then this module will successfully 128 | * register the memory, and take a reference to that file. 129 | * 130 | * Seeing the uverbs code only cleans up memory regions only after 131 | * that file is closed, and io_peer_mem will only release that file 132 | * when the cleanup occurs, a deadlock will occur and the registration 133 | * will be permanently leaked. 134 | * 135 | * This code handles this situation in a bit of an hackish way: 136 | * we check if the backing file's i_cdev's kobjs's name starts with 137 | * "uverbs" and refuses to map it if it does. 138 | */ 139 | struct inode *i; 140 | 141 | if (!vma->vm_file || !vma->vm_file->f_inode) 142 | return 0; 143 | 144 | i = vma->vm_file->f_inode; 145 | 146 | if (!S_ISCHR(i->i_mode) || i->i_cdev == NULL) 147 | return 0; 148 | 149 | return strncmp(i->i_cdev->kobj.name, "uverbs", strlen("uverbs")) == 0; 150 | } 151 | 152 | 153 | static int acquire(unsigned long addr, size_t size, void *peer_mem_private_data, 154 | char *peer_mem_name, void **context) 155 | { 156 | struct vm_area_struct *vma = NULL; 157 | struct context *ctx = kzalloc(sizeof(*ctx), GFP_KERNEL); 158 | unsigned long pfn, end; 159 | 160 | if (!ctx) 161 | return 0; 162 | 163 | ctx->addr = addr; 164 | ctx->size = size; 165 | 166 | end = addr + size; 167 | 168 | vma = find_vma(current->mm, addr); 169 | 170 | if (!vma || vma->vm_end < end) 171 | goto err; 172 | 173 | debug_msg("vma: %lx %lx %lx %zx\n", addr, vma->vm_end - vma->vm_start, 174 | vma->vm_flags, size); 175 | 176 | if (is_uverbs_file(vma)) { 177 | pr_err(NAME ": can't register a uverbs device!\n"); 178 | goto err; 179 | } 180 | 181 | if (!(vma->vm_flags & VM_WRITE)) 182 | goto err; 183 | 184 | fault_missing_pages(vma, addr & PAGE_MASK, end); 185 | 186 | if (follow_pfn(vma, addr, &pfn)) 187 | goto err; 188 | 189 | debug_msg("pfn: %lx\n", pfn << PAGE_SHIFT); 190 | debug_msg("acquire %p\n", ctx); 191 | 192 | ctx->vma = get_vma(vma); 193 | 194 | if (ctx->vma == NULL) { 195 | printk(NAME ": could not allocate VMA!\n"); 196 | goto err; 197 | } 198 | 199 | 200 | *context = ctx; 201 | __module_get(THIS_MODULE); 202 | return 1; 203 | 204 | err: 205 | kfree(ctx); 206 | return 0; 207 | } 208 | 209 | static void release(void *context) 210 | { 211 | struct context *ctx = (struct context *) context; 212 | 213 | debug_msg("release %p\n", context); 214 | 215 | put_vma(ctx->vma); 216 | 217 | kfree(context); 218 | module_put(THIS_MODULE); 219 | } 220 | 221 | static int get_pages(unsigned long addr, size_t size, int write, int force, 222 | struct sg_table *sg_head, void *context, 223 | u64 core_context) 224 | { 225 | struct context *ctx = (struct context *) context; 226 | 227 | int ret = sg_alloc_table(sg_head, (ctx->size + PAGE_SIZE-1) / PAGE_SIZE, 228 | GFP_KERNEL); 229 | if (ret) 230 | return ret; 231 | 232 | return 0; 233 | } 234 | 235 | static void put_pages(struct sg_table *sg_head, void *context) 236 | { 237 | sg_free_table(sg_head); 238 | } 239 | 240 | static int dma_map(struct sg_table *sg_head, void *context, 241 | struct device *dma_device, int dmasync, 242 | int *nmap) 243 | { 244 | struct scatterlist *sg; 245 | struct context *ctx = (struct context *) context; 246 | unsigned long pfn; 247 | unsigned long addr = ctx->addr; 248 | unsigned long size = ctx->size; 249 | int i, ret; 250 | 251 | *nmap = ctx->size / PAGE_SIZE; 252 | 253 | for_each_sg(sg_head->sgl, sg, (ctx->size + PAGE_SIZE-1) / PAGE_SIZE, i) { 254 | sg_set_page(sg, NULL, PAGE_SIZE, 0); 255 | 256 | if ((ret = follow_pfn(ctx->vma, addr, &pfn))) 257 | return ret; 258 | 259 | sg->dma_address = (pfn << PAGE_SHIFT);; 260 | sg->dma_length = PAGE_SIZE; 261 | sg->offset = addr & ~PAGE_MASK; 262 | 263 | debug_msg("sg[%d] %lx %x %d\n", i, 264 | (unsigned long) sg->dma_address, 265 | sg->dma_length, sg->offset); 266 | 267 | addr += sg->dma_length - sg->offset; 268 | size -= sg->dma_length - sg->offset; 269 | 270 | if (!size) { 271 | *nmap = i+1; 272 | break; 273 | } 274 | } 275 | 276 | return 0; 277 | } 278 | 279 | static int dma_unmap(struct sg_table *sg_head, void *context, 280 | struct device *dma_device) 281 | { 282 | return 0; 283 | } 284 | 285 | static unsigned long get_page_size(void *context) 286 | { 287 | return PAGE_SIZE; 288 | } 289 | 290 | static struct peer_memory_client io_mem_client = { 291 | .name = NAME, 292 | .version = VERSION, 293 | .acquire = acquire, 294 | .get_pages = get_pages, 295 | .dma_map = dma_map, 296 | .dma_unmap = dma_unmap, 297 | .put_pages = put_pages, 298 | .get_page_size = get_page_size, 299 | .release = release, 300 | }; 301 | 302 | 303 | static void *reg_handle; 304 | static int __init io_mem_init(void) 305 | { 306 | reg_handle = ib_register_peer_memory_client(&io_mem_client, NULL); 307 | 308 | if (!reg_handle) 309 | return -EINVAL; 310 | 311 | printk(NAME ": module loaded\n"); 312 | 313 | return 0; 314 | } 315 | 316 | static void __exit io_mem_cleanup(void) 317 | { 318 | printk(NAME ": module unloaded\n"); 319 | ib_unregister_peer_memory_client(reg_handle); 320 | } 321 | 322 | module_init(io_mem_init); 323 | module_exit(io_mem_cleanup); 324 | -------------------------------------------------------------------------------- /test/Makefile: -------------------------------------------------------------------------------- 1 | 2 | CFLAGS = -Wall -Werror -std=gnu99 -g -O2 3 | LDLIBS = -lrdmacm -libverbs 4 | 5 | 6 | all: client server 7 | 8 | 9 | clean: 10 | rm -f client server 11 | -------------------------------------------------------------------------------- /test/client.c: -------------------------------------------------------------------------------- 1 | //////////////////////////////////////////////////////////////////////// 2 | // 3 | // Copyright 2015 PMC-Sierra, Inc. 4 | // 5 | // Licensed under the Apache License, Version 2.0 (the "License"); you 6 | // may not use this file except in compliance with the License. You may 7 | // obtain a copy of the License at 8 | // http://www.apache.org/licenses/LICENSE-2.0 Unless required by 9 | // applicable law or agreed to in writing, software distributed under the 10 | // License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR 11 | // CONDITIONS OF ANY KIND, either express or implied. See the License for 12 | // the specific language governing permissions and limitations under the 13 | // License. 14 | // 15 | //////////////////////////////////////////////////////////////////////// 16 | 17 | //////////////////////////////////////////////////////////////////////// 18 | // 19 | // Author: Logan Gunthorpe 20 | // 21 | // Description: 22 | // RDMA Test Client 23 | // 24 | //////////////////////////////////////////////////////////////////////// 25 | 26 | #include 27 | #include 28 | 29 | #include 30 | #include 31 | #include 32 | #include 33 | #include 34 | 35 | #include 36 | #include 37 | #include 38 | #include 39 | 40 | const size_t len = 4 * (1 << 20); 41 | const size_t offset = 8; 42 | 43 | static struct rdma_cm_id *do_connect(char *addr, char *port) 44 | { 45 | struct rdma_cm_id *id; 46 | struct rdma_addrinfo hints, *res; 47 | struct ibv_qp_init_attr attr; 48 | 49 | memset(&hints, 0, sizeof hints); 50 | hints.ai_port_space = RDMA_PS_TCP; 51 | if (rdma_getaddrinfo(addr, port, &hints, &res)) { 52 | perror("rdma_getaddrinfo"); 53 | return NULL; 54 | } 55 | 56 | memset(&attr, 0, sizeof attr); 57 | attr.cap.max_send_wr = attr.cap.max_recv_wr = 16; 58 | attr.cap.max_send_sge = attr.cap.max_recv_sge = 16; 59 | attr.cap.max_inline_data = 0; 60 | attr.qp_context = id; 61 | attr.sq_sig_all = 1; 62 | int ret = rdma_create_ep(&id, res, NULL, &attr); 63 | rdma_freeaddrinfo(res); 64 | if (ret) { 65 | perror("rdma_create_ep"); 66 | return NULL; 67 | } 68 | 69 | if (rdma_connect(id, NULL)) { 70 | perror("rdma_connect"); 71 | return NULL; 72 | } 73 | 74 | return id; 75 | } 76 | 77 | static void init_file(int fd) 78 | { 79 | lseek(fd, 0, SEEK_SET); 80 | void *buf = malloc(len); 81 | memset(buf, 0, len); 82 | write(fd, buf, len); 83 | fsync(fd); 84 | } 85 | 86 | static void check_file(const char *fname, int byte_len, int mask) 87 | { 88 | int fd = open(fname, O_RDONLY); 89 | if (fd < 0) { 90 | fprintf(stderr, "Error opening %s: %m\n", fname); 91 | exit(1); 92 | } 93 | 94 | uint32_t *map = mmap(NULL, byte_len + offset, 95 | PROT_READ, MAP_SHARED, fd, 0); 96 | uint32_t *d = map; 97 | 98 | for (int i = 0; i < offset / sizeof(*d); i++) { 99 | if (*d != 0) { 100 | printf("Offset %d not zero'd: %08" PRIx32 "\n", i, *d); 101 | } 102 | d++; 103 | } 104 | 105 | for (int i = 0; i < (byte_len - offset) / sizeof(*d); i++) { 106 | if (*d != (mask | i)) { 107 | printf("Incorrect Value at %d: %08" PRIx32 " %08x\n", i, 108 | *d, mask | i); 109 | exit(5); 110 | } 111 | d++; 112 | } 113 | 114 | munmap(map, byte_len + offset); 115 | close(fd); 116 | 117 | printf("PASSED: File matches expected data!\n"); 118 | } 119 | 120 | int main(int argc, char *argv[]) 121 | { 122 | if (argc != 3) { 123 | printf("USAGE: %s SERVER FILE\n", argv[0]); 124 | return 1; 125 | } 126 | 127 | int fd = open(argv[2], O_RDWR | O_CREAT, 0664); 128 | if (fd < 0) { 129 | fprintf(stderr, "Error opening %s: %m\n", argv[2]); 130 | return 1; 131 | } 132 | 133 | init_file(fd); 134 | 135 | void *m = mmap(NULL, len, PROT_WRITE | PROT_READ, MAP_SHARED, fd, 0); 136 | 137 | printf("Mmap Buffer: %p\n", m); 138 | if (m == MAP_FAILED) { 139 | perror("mmap"); 140 | close(fd); 141 | return 1; 142 | } 143 | 144 | close(fd); 145 | 146 | struct rdma_cm_id *id = do_connect(argv[1], "11935"); 147 | if (!id) 148 | return 3; 149 | 150 | uint32_t mask = *((uint32_t *) id->event->param.conn.private_data); 151 | printf("Mask: %08x\n", mask); 152 | 153 | struct ibv_mr *mr = ibv_reg_mr(id->pd, m+offset, len-offset, 154 | IBV_ACCESS_LOCAL_WRITE | 155 | IBV_ACCESS_REMOTE_WRITE | 156 | IBV_ACCESS_REMOTE_READ); 157 | 158 | if (mr == NULL) { 159 | perror("ibv_reg_mr"); 160 | return 4; 161 | } 162 | 163 | munmap(m, len); 164 | 165 | printf("Posting Recv\n"); 166 | rdma_post_recv(id, NULL, m+offset, len-offset, mr); 167 | 168 | struct ibv_wc wc; 169 | rdma_get_recv_comp(id, &wc); 170 | printf("Recv'd: %d\n", wc.status); 171 | 172 | if (ibv_dereg_mr(mr)) 173 | perror("ibv_dereg_mr"); 174 | 175 | check_file(argv[2], wc.byte_len, mask); 176 | 177 | return 0; 178 | } 179 | -------------------------------------------------------------------------------- /test/server.c: -------------------------------------------------------------------------------- 1 | //////////////////////////////////////////////////////////////////////// 2 | // 3 | // Copyright 2015 PMC-Sierra, Inc. 4 | // 5 | // Licensed under the Apache License, Version 2.0 (the "License"); you 6 | // may not use this file except in compliance with the License. You may 7 | // obtain a copy of the License at 8 | // http://www.apache.org/licenses/LICENSE-2.0 Unless required by 9 | // applicable law or agreed to in writing, software distributed under the 10 | // License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR 11 | // CONDITIONS OF ANY KIND, either express or implied. See the License for 12 | // the specific language governing permissions and limitations under the 13 | // License. 14 | // 15 | //////////////////////////////////////////////////////////////////////// 16 | 17 | //////////////////////////////////////////////////////////////////////// 18 | // 19 | // Author: Logan Gunthorpe 20 | // 21 | // Description: 22 | // RDMA Test Server 23 | // 24 | //////////////////////////////////////////////////////////////////////// 25 | 26 | #include 27 | 28 | #include 29 | #include 30 | #include 31 | #include 32 | #include 33 | #include 34 | #include 35 | #include 36 | 37 | #include 38 | #include 39 | #include 40 | #include 41 | #include 42 | 43 | struct rdma_cm_id *setup_server(void) 44 | { 45 | struct rdma_addrinfo hints, *res; 46 | struct ibv_qp_init_attr attr; 47 | 48 | memset(&hints, 0, sizeof hints); 49 | hints.ai_flags = RAI_PASSIVE; 50 | hints.ai_port_space = RDMA_PS_TCP; 51 | if (rdma_getaddrinfo(NULL, "11935", &hints, &res)) { 52 | perror("rdma_getaddrinfo"); 53 | return NULL; 54 | } 55 | 56 | struct rdma_cm_id *listen_id; 57 | 58 | memset(&attr, 0, sizeof attr); 59 | attr.cap.max_send_wr = attr.cap.max_recv_wr = 8; 60 | attr.cap.max_send_sge = attr.cap.max_recv_sge = 8; 61 | attr.cap.max_inline_data = 16; 62 | attr.sq_sig_all = 1; 63 | if (rdma_create_ep(&listen_id, res, NULL, &attr)) { 64 | perror("rdma_create_ep"); 65 | return NULL; 66 | } 67 | 68 | if (rdma_listen(listen_id, 0)) { 69 | perror("rdma_listen"); 70 | return NULL; 71 | } 72 | 73 | return listen_id; 74 | } 75 | 76 | int main(int argc, char *argv[]) 77 | { 78 | struct rdma_cm_id *listen_id = setup_server(); 79 | if (listen_id == NULL) 80 | return 2; 81 | 82 | printf("Listening on port %s\n", "11935"); 83 | 84 | srand(time(NULL)); 85 | 86 | while(1) { 87 | printf("\n"); 88 | struct rdma_cm_id *id; 89 | 90 | if (rdma_get_request(listen_id, &id)) { 91 | perror("rdma_get_request"); 92 | return 1; 93 | } 94 | 95 | const int buflen = (1 << 19); 96 | uint32_t *buf = malloc(buflen * sizeof(*buf)); 97 | 98 | uint32_t mask = rand() << 16; 99 | printf("Mask: %08x\n", mask); 100 | 101 | for (int i = 0; i < buflen; i++) 102 | buf[i] = mask | i; 103 | 104 | 105 | struct ibv_mr *mr = ibv_reg_mr(id->pd, buf, buflen*sizeof(*buf), 106 | IBV_ACCESS_LOCAL_WRITE | 107 | IBV_ACCESS_REMOTE_WRITE | 108 | IBV_ACCESS_REMOTE_READ); 109 | 110 | 111 | struct rdma_conn_param conn_param; 112 | memset(&conn_param, 0, sizeof(conn_param)); 113 | conn_param.private_data_len = sizeof(mask); 114 | conn_param.private_data = &mask; 115 | conn_param.responder_resources = 2; 116 | conn_param.initiator_depth = 2; 117 | conn_param.retry_count = 5; 118 | conn_param.rnr_retry_count = 5; 119 | 120 | printf("Accepting Client Connection\n"); 121 | if (rdma_accept(id, &conn_param)) { 122 | perror("rdma_accept"); 123 | goto disconnect; 124 | } 125 | 126 | printf("Post Send\n"); 127 | rdma_post_send(id, NULL, buf, buflen*sizeof(*buf), 128 | mr, 0); 129 | 130 | struct ibv_wc wc; 131 | rdma_get_send_comp(id, &wc); 132 | 133 | 134 | printf("Sent: %d\n", wc.status); 135 | 136 | disconnect: 137 | rdma_disconnect(id); 138 | } 139 | 140 | 141 | return 0; 142 | } 143 | --------------------------------------------------------------------------------