├── COPYING ├── Makefile ├── README.adoc ├── perf.c ├── sender.c └── sender.h /COPYING: -------------------------------------------------------------------------------- 1 | GNU GENERAL PUBLIC LICENSE 2 | Version 2, June 1991 3 | 4 | Copyright (C) 1989, 1991 Free Software Foundation, Inc., 5 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA 6 | Everyone is permitted to copy and distribute verbatim copies 7 | of this license document, but changing it is not allowed. 8 | 9 | Preamble 10 | 11 | The licenses for most software are designed to take away your 12 | freedom to share and change it. By contrast, the GNU General Public 13 | License is intended to guarantee your freedom to share and change free 14 | software--to make sure the software is free for all its users. This 15 | General Public License applies to most of the Free Software 16 | Foundation's software and to any other program whose authors commit to 17 | using it. (Some other Free Software Foundation software is covered by 18 | the GNU Lesser General Public License instead.) You can apply it to 19 | your programs, too. 20 | 21 | When we speak of free software, we are referring to freedom, not 22 | price. Our General Public Licenses are designed to make sure that you 23 | have the freedom to distribute copies of free software (and charge for 24 | this service if you wish), that you receive source code or can get it 25 | if you want it, that you can change the software or use pieces of it 26 | in new free programs; and that you know you can do these things. 27 | 28 | To protect your rights, we need to make restrictions that forbid 29 | anyone to deny you these rights or to ask you to surrender the rights. 30 | These restrictions translate to certain responsibilities for you if you 31 | distribute copies of the software, or if you modify it. 32 | 33 | For example, if you distribute copies of such a program, whether 34 | gratis or for a fee, you must give the recipients all the rights that 35 | you have. You must make sure that they, too, receive or can get the 36 | source code. And you must show them these terms so they know their 37 | rights. 38 | 39 | We protect your rights with two steps: (1) copyright the software, and 40 | (2) offer you this license which gives you legal permission to copy, 41 | distribute and/or modify the software. 42 | 43 | Also, for each author's protection and ours, we want to make certain 44 | that everyone understands that there is no warranty for this free 45 | software. If the software is modified by someone else and passed on, we 46 | want its recipients to know that what they have is not the original, so 47 | that any problems introduced by others will not reflect on the original 48 | authors' reputations. 49 | 50 | Finally, any free program is threatened constantly by software 51 | patents. We wish to avoid the danger that redistributors of a free 52 | program will individually obtain patent licenses, in effect making the 53 | program proprietary. To prevent this, we have made it clear that any 54 | patent must be licensed for everyone's free use or not licensed at all. 55 | 56 | The precise terms and conditions for copying, distribution and 57 | modification follow. 58 | 59 | GNU GENERAL PUBLIC LICENSE 60 | TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 61 | 62 | 0. This License applies to any program or other work which contains 63 | a notice placed by the copyright holder saying it may be distributed 64 | under the terms of this General Public License. The "Program", below, 65 | refers to any such program or work, and a "work based on the Program" 66 | means either the Program or any derivative work under copyright law: 67 | that is to say, a work containing the Program or a portion of it, 68 | either verbatim or with modifications and/or translated into another 69 | language. (Hereinafter, translation is included without limitation in 70 | the term "modification".) Each licensee is addressed as "you". 71 | 72 | Activities other than copying, distribution and modification are not 73 | covered by this License; they are outside its scope. The act of 74 | running the Program is not restricted, and the output from the Program 75 | is covered only if its contents constitute a work based on the 76 | Program (independent of having been made by running the Program). 77 | Whether that is true depends on what the Program does. 78 | 79 | 1. You may copy and distribute verbatim copies of the Program's 80 | source code as you receive it, in any medium, provided that you 81 | conspicuously and appropriately publish on each copy an appropriate 82 | copyright notice and disclaimer of warranty; keep intact all the 83 | notices that refer to this License and to the absence of any warranty; 84 | and give any other recipients of the Program a copy of this License 85 | along with the Program. 86 | 87 | You may charge a fee for the physical act of transferring a copy, and 88 | you may at your option offer warranty protection in exchange for a fee. 89 | 90 | 2. You may modify your copy or copies of the Program or any portion 91 | of it, thus forming a work based on the Program, and copy and 92 | distribute such modifications or work under the terms of Section 1 93 | above, provided that you also meet all of these conditions: 94 | 95 | a) You must cause the modified files to carry prominent notices 96 | stating that you changed the files and the date of any change. 97 | 98 | b) You must cause any work that you distribute or publish, that in 99 | whole or in part contains or is derived from the Program or any 100 | part thereof, to be licensed as a whole at no charge to all third 101 | parties under the terms of this License. 102 | 103 | c) If the modified program normally reads commands interactively 104 | when run, you must cause it, when started running for such 105 | interactive use in the most ordinary way, to print or display an 106 | announcement including an appropriate copyright notice and a 107 | notice that there is no warranty (or else, saying that you provide 108 | a warranty) and that users may redistribute the program under 109 | these conditions, and telling the user how to view a copy of this 110 | License. (Exception: if the Program itself is interactive but 111 | does not normally print such an announcement, your work based on 112 | the Program is not required to print an announcement.) 113 | 114 | These requirements apply to the modified work as a whole. If 115 | identifiable sections of that work are not derived from the Program, 116 | and can be reasonably considered independent and separate works in 117 | themselves, then this License, and its terms, do not apply to those 118 | sections when you distribute them as separate works. But when you 119 | distribute the same sections as part of a whole which is a work based 120 | on the Program, the distribution of the whole must be on the terms of 121 | this License, whose permissions for other licensees extend to the 122 | entire whole, and thus to each and every part regardless of who wrote it. 123 | 124 | Thus, it is not the intent of this section to claim rights or contest 125 | your rights to work written entirely by you; rather, the intent is to 126 | exercise the right to control the distribution of derivative or 127 | collective works based on the Program. 128 | 129 | In addition, mere aggregation of another work not based on the Program 130 | with the Program (or with a work based on the Program) on a volume of 131 | a storage or distribution medium does not bring the other work under 132 | the scope of this License. 133 | 134 | 3. You may copy and distribute the Program (or a work based on it, 135 | under Section 2) in object code or executable form under the terms of 136 | Sections 1 and 2 above provided that you also do one of the following: 137 | 138 | a) Accompany it with the complete corresponding machine-readable 139 | source code, which must be distributed under the terms of Sections 140 | 1 and 2 above on a medium customarily used for software interchange; or, 141 | 142 | b) Accompany it with a written offer, valid for at least three 143 | years, to give any third party, for a charge no more than your 144 | cost of physically performing source distribution, a complete 145 | machine-readable copy of the corresponding source code, to be 146 | distributed under the terms of Sections 1 and 2 above on a medium 147 | customarily used for software interchange; or, 148 | 149 | c) Accompany it with the information you received as to the offer 150 | to distribute corresponding source code. (This alternative is 151 | allowed only for noncommercial distribution and only if you 152 | received the program in object code or executable form with such 153 | an offer, in accord with Subsection b above.) 154 | 155 | The source code for a work means the preferred form of the work for 156 | making modifications to it. For an executable work, complete source 157 | code means all the source code for all modules it contains, plus any 158 | associated interface definition files, plus the scripts used to 159 | control compilation and installation of the executable. However, as a 160 | special exception, the source code distributed need not include 161 | anything that is normally distributed (in either source or binary 162 | form) with the major components (compiler, kernel, and so on) of the 163 | operating system on which the executable runs, unless that component 164 | itself accompanies the executable. 165 | 166 | If distribution of executable or object code is made by offering 167 | access to copy from a designated place, then offering equivalent 168 | access to copy the source code from the same place counts as 169 | distribution of the source code, even though third parties are not 170 | compelled to copy the source along with the object code. 171 | 172 | 4. You may not copy, modify, sublicense, or distribute the Program 173 | except as expressly provided under this License. Any attempt 174 | otherwise to copy, modify, sublicense or distribute the Program is 175 | void, and will automatically terminate your rights under this License. 176 | However, parties who have received copies, or rights, from you under 177 | this License will not have their licenses terminated so long as such 178 | parties remain in full compliance. 179 | 180 | 5. You are not required to accept this License, since you have not 181 | signed it. However, nothing else grants you permission to modify or 182 | distribute the Program or its derivative works. These actions are 183 | prohibited by law if you do not accept this License. Therefore, by 184 | modifying or distributing the Program (or any work based on the 185 | Program), you indicate your acceptance of this License to do so, and 186 | all its terms and conditions for copying, distributing or modifying 187 | the Program or works based on it. 188 | 189 | 6. Each time you redistribute the Program (or any work based on the 190 | Program), the recipient automatically receives a license from the 191 | original licensor to copy, distribute or modify the Program subject to 192 | these terms and conditions. You may not impose any further 193 | restrictions on the recipients' exercise of the rights granted herein. 194 | You are not responsible for enforcing compliance by third parties to 195 | this License. 196 | 197 | 7. If, as a consequence of a court judgment or allegation of patent 198 | infringement or for any other reason (not limited to patent issues), 199 | conditions are imposed on you (whether by court order, agreement or 200 | otherwise) that contradict the conditions of this License, they do not 201 | excuse you from the conditions of this License. If you cannot 202 | distribute so as to satisfy simultaneously your obligations under this 203 | License and any other pertinent obligations, then as a consequence you 204 | may not distribute the Program at all. For example, if a patent 205 | license would not permit royalty-free redistribution of the Program by 206 | all those who receive copies directly or indirectly through you, then 207 | the only way you could satisfy both it and this License would be to 208 | refrain entirely from distribution of the Program. 209 | 210 | If any portion of this section is held invalid or unenforceable under 211 | any particular circumstance, the balance of the section is intended to 212 | apply and the section as a whole is intended to apply in other 213 | circumstances. 214 | 215 | It is not the purpose of this section to induce you to infringe any 216 | patents or other property right claims or to contest validity of any 217 | such claims; this section has the sole purpose of protecting the 218 | integrity of the free software distribution system, which is 219 | implemented by public license practices. Many people have made 220 | generous contributions to the wide range of software distributed 221 | through that system in reliance on consistent application of that 222 | system; it is up to the author/donor to decide if he or she is willing 223 | to distribute software through any other system and a licensee cannot 224 | impose that choice. 225 | 226 | This section is intended to make thoroughly clear what is believed to 227 | be a consequence of the rest of this License. 228 | 229 | 8. If the distribution and/or use of the Program is restricted in 230 | certain countries either by patents or by copyrighted interfaces, the 231 | original copyright holder who places the Program under this License 232 | may add an explicit geographical distribution limitation excluding 233 | those countries, so that distribution is permitted only in or among 234 | countries not thus excluded. In such case, this License incorporates 235 | the limitation as if written in the body of this License. 236 | 237 | 9. The Free Software Foundation may publish revised and/or new versions 238 | of the General Public License from time to time. Such new versions will 239 | be similar in spirit to the present version, but may differ in detail to 240 | address new problems or concerns. 241 | 242 | Each version is given a distinguishing version number. If the Program 243 | specifies a version number of this License which applies to it and "any 244 | later version", you have the option of following the terms and conditions 245 | either of that version or of any later version published by the Free 246 | Software Foundation. If the Program does not specify a version number of 247 | this License, you may choose any version ever published by the Free Software 248 | Foundation. 249 | 250 | 10. If you wish to incorporate parts of the Program into other free 251 | programs whose distribution conditions are different, write to the author 252 | to ask for permission. For software which is copyrighted by the Free 253 | Software Foundation, write to the Free Software Foundation; we sometimes 254 | make exceptions for this. Our decision will be guided by the two goals 255 | of preserving the free status of all derivatives of our free software and 256 | of promoting the sharing and reuse of software generally. 257 | 258 | NO WARRANTY 259 | 260 | 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY 261 | FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN 262 | OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES 263 | PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED 264 | OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF 265 | MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS 266 | TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE 267 | PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, 268 | REPAIR OR CORRECTION. 269 | 270 | 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING 271 | WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR 272 | REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, 273 | INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING 274 | OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED 275 | TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY 276 | YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER 277 | PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE 278 | POSSIBILITY OF SUCH DAMAGES. 279 | 280 | END OF TERMS AND CONDITIONS 281 | 282 | How to Apply These Terms to Your New Programs 283 | 284 | If you develop a new program, and you want it to be of the greatest 285 | possible use to the public, the best way to achieve this is to make it 286 | free software which everyone can redistribute and change under these terms. 287 | 288 | To do so, attach the following notices to the program. It is safest 289 | to attach them to the start of each source file to most effectively 290 | convey the exclusion of warranty; and each file should have at least 291 | the "copyright" line and a pointer to where the full notice is found. 292 | 293 | 294 | Copyright (C) 295 | 296 | This program is free software; you can redistribute it and/or modify 297 | it under the terms of the GNU General Public License as published by 298 | the Free Software Foundation; either version 2 of the License, or 299 | (at your option) any later version. 300 | 301 | This program is distributed in the hope that it will be useful, 302 | but WITHOUT ANY WARRANTY; without even the implied warranty of 303 | MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 304 | GNU General Public License for more details. 305 | 306 | You should have received a copy of the GNU General Public License along 307 | with this program; if not, write to the Free Software Foundation, Inc., 308 | 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. 309 | 310 | Also add information on how to contact you by electronic and paper mail. 311 | 312 | If the program is interactive, make it output a short notice like this 313 | when it starts in an interactive mode: 314 | 315 | Gnomovision version 69, Copyright (C) year name of author 316 | Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. 317 | This is free software, and you are welcome to redistribute it 318 | under certain conditions; type `show c' for details. 319 | 320 | The hypothetical commands `show w' and `show c' should show the appropriate 321 | parts of the General Public License. Of course, the commands you use may 322 | be called something other than `show w' and `show c'; they could even be 323 | mouse-clicks or menu items--whatever suits your program. 324 | 325 | You should also get your employer (if you work as a programmer) or your 326 | school, if any, to sign a "copyright disclaimer" for the program, if 327 | necessary. Here is a sample; alter the names: 328 | 329 | Yoyodyne, Inc., hereby disclaims all copyright interest in the program 330 | `Gnomovision' (which makes passes at compilers) written by James Hacker. 331 | 332 | , 1 April 1989 333 | Ty Coon, President of Vice 334 | 335 | This General Public License does not permit incorporating your program into 336 | proprietary programs. If your program is a subroutine library, you may 337 | consider it more useful to permit linking proprietary applications with the 338 | library. If this is what you want to do, use the GNU Lesser General 339 | Public License instead of this License. 340 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | NAME = ntpperf 2 | 3 | CPPFLAGS = -D_GNU_SOURCE 4 | CFLAGS = -O2 -Wall -g 5 | LDFLAGS = -lpcap -lm 6 | 7 | ifdef NTPPERF_NTS 8 | CPPFLAGS += -DNTS 9 | CFLAGS += $(shell pkg-config --cflags gnutls) 10 | LDFLAGS += $(shell pkg-config --libs gnutls) 11 | endif 12 | 13 | all: $(NAME) 14 | 15 | $(NAME): perf.o sender.o 16 | $(CC) $(CFLAGS) -o $@ $^ $(LDFLAGS) 17 | 18 | clean: 19 | rm -f $(NAME) *.o 20 | -------------------------------------------------------------------------------- /README.adoc: -------------------------------------------------------------------------------- 1 | = ntpperf 2 | 3 | `ntpperf` is a tool for measuring performance of NTP servers and PTP masters. 4 | It simulates NTP clients or PTP slaves sending requests at an increasing rate 5 | and prints how many responses were received and lost. If the computer running 6 | `ntpperf` has an accurate clock and the network delay to the server/master is 7 | constant and known (e.g. there is a direct connection), it can also show how 8 | does the accuracy of the transmit timestamp change with increasing rate of 9 | requests. 10 | 11 | `ntpperf` generates and processes raw Ethernet frames using `libpcap`. Requests 12 | from the simulated clients/slaves have source addresses from a specified 13 | private IPv4 network. The server/master and/or network needs to be configured 14 | to route the responses to the computer running `ntpperf`, which should drop 15 | them (ideally in the raw table of iptables to minimize the time wasted in the 16 | networking stack). 17 | 18 | Network Time Security (NTS) support using the `gnutls` library can be enabled 19 | by setting the `NTPPERF_NTS` environment variable for the build. 20 | 21 | == Usage 22 | 23 | ``` 24 | Usage: ./ntpperf MODE NETWORK-OPTIONS [OTHER-OPTIONS] 25 | 26 | Mode: 27 | -B send NTP client requests in basic mode 28 | -I send NTP client requests in interleaved mode 29 | -D DOMAIN send PTP delay requests 30 | -N DOMAIN send PTP NetSync Monitor (NSM) requests 31 | 32 | Network options: 33 | -i INTERFACE specify network interface 34 | -s NETWORK/BITS specify source IPv4 network 35 | -d IP-ADDRESS specify destination IPv4 address 36 | -m MAC specify destination MAC address 37 | 38 | Other options: 39 | -M send multicast PTP delay requests to 224.0.1.129 40 | -r RATE[-RATE] specify minimum and maximum rate (1000-1000000) 41 | -p NUMBER specify number of processes to send requests (1) 42 | -e make transmit interval exponentially distributed 43 | -l allow late transmissions 44 | -x MULT specify rate multiplier (1.5) 45 | -t INTERVAL specify sampling interval (2.0 seconds) 46 | -o CORRECTION print offset between remote TX and local RX timestamp 47 | with specified correction (e.g. network and RX delay) 48 | -O with -o use remote (RX+TX)/2 instead of TX timestamp 49 | -H enable HW timestamping for TX offset statistics 50 | -S C2S,COOKIE authenticate NTP requests with NTS 51 | -h print this help message 52 | ``` 53 | 54 | == Example 55 | 56 | In this example `ntpperf` runs on a computer with address 192.168.123.2. It is 57 | configured to drop all UDP packets sent to the network 172.18.0.0/16: 58 | 59 | ``` 60 | # iptables -t raw -A PREROUTING -p udp -m udp -d 172.18.0.0/16 -j DROP 61 | ``` 62 | 63 | An NTP server and PTP master is running on 192.168.123.1. The MAC address of 64 | its interface is AA:BB:CC:DD:EE:FF. It is configured to route the responses to 65 | 192.168.123.2: 66 | 67 | ``` 68 | # ip route add 172.18.0.0/16 via 192.168.123.2 dev eth0 69 | ``` 70 | 71 | The following test measures the throughput of the NTP server in the basic mode 72 | and accuracy of server's transmit timestamps relative to the local hardware 73 | clock of eth0 assuming a network delay and RX+TX compensation of 1.45 74 | microseconds: 75 | 76 | ``` 77 | # ./ntpperf -i eth0 -m AA:BB:CC:DD:EE:FF -d 192.168.123.1 -s 172.18.0.0/16 -B -o 1.95e-6 -H 78 | | responses | TX timestamp offset (ns) 79 | rate clients | lost invalid basic xleave | min mean max stddev 80 | 1000 100 0.00% 0.00% 100.00% 0.00% +5343 +12047 +23267 1090 81 | 1500 150 0.00% 0.00% 100.00% 0.00% +6085 +12035 +23378 910 82 | 2250 225 0.00% 0.00% 100.00% 0.00% +5382 +11585 +24165 1739 83 | 3375 337 0.00% 0.00% 100.00% 0.00% +4656 +11442 +24104 1822 84 | 5062 506 0.00% 0.00% 100.00% 0.00% +4455 +11066 +23308 2386 85 | 7593 759 0.00% 0.00% 100.00% 0.00% +4408 +9283 +27831 3332 86 | 11389 1138 0.00% 0.00% 100.00% 0.00% +3762 +7585 +31063 4742 87 | 17083 1708 0.00% 0.00% 100.00% 0.00% +3650 +5034 +25045 1203 88 | 25624 2562 0.00% 0.00% 100.00% 0.00% +3527 +4731 +24358 805 89 | 38436 3843 0.00% 0.00% 100.00% 0.00% +3558 +4765 +35514 1041 90 | 57654 5765 0.00% 0.00% 100.00% 0.00% +2675 +3302 +13421 460 91 | 86481 8648 0.00% 0.00% 100.00% 0.00% +2519 +3046 +12824 413 92 | 129721 12972 0.00% 0.00% 100.00% 0.00% +2524 +3062 +151362 610 93 | 194581 16384 0.00% 0.00% 100.00% 0.00% +2511 +3082 +55417 1696 94 | 291871 16384 0.09% 0.00% 99.91% 0.00% +2552 +3105 +47080 1884 95 | 437806 16384 5.58% 0.00% 94.42% 0.00% +2562 +2975 +28727 1480 96 | 656709 16384 49.78% 0.00% 50.22% 0.00% +2557 +3367 +230131 3570 97 | Could not send requests at rate 985063 98 | ``` 99 | 100 | The following test measures the performance of the NTP server in the 101 | interleaved mode: 102 | 103 | ``` 104 | # ./ntpperf -i eth0 -m AA:BB:CC:DD:EE:FF -d 192.168.123.1 -s 172.18.0.0/16 -I -o 1.95e-6 -H 105 | | responses | TX timestamp offset (ns) 106 | rate clients | lost invalid basic xleave | min mean max stddev 107 | 1000 100 0.00% 0.00% 0.00% 100.00% -23 +17 +53 14 108 | 1500 150 0.00% 0.00% 0.00% 100.00% -21 +20 +57 14 109 | 2250 225 0.00% 0.00% 0.00% 100.00% -43 +12 +55 17 110 | 3375 337 0.00% 0.00% 0.00% 100.00% -54 -8 +41 15 111 | 5062 506 0.00% 0.00% 0.00% 100.00% -15 +39 +80 16 112 | 7593 759 0.00% 0.00% 0.00% 100.00% -26 +19 +73 15 113 | 11389 1138 0.00% 0.00% 0.00% 100.00% +19 +70 +119 16 114 | 17083 1708 0.00% 0.00% 0.00% 100.00% +8 +50 +98 15 115 | 25624 2562 0.00% 0.00% 0.00% 100.00% -67 -1 +70 25 116 | 38436 3843 0.00% 0.00% 0.00% 100.00% -70 -32 +9 14 117 | 57654 5765 0.00% 0.00% 0.00% 100.00% -80 -36 +3 14 118 | 86481 8648 21.02% 0.00% 1.49% 77.49% -92 +1173 +52305 3326 119 | 129721 12972 26.84% 0.00% 1.59% 71.57% -43 +1226 +34199 1743 120 | 194581 16384 51.98% 0.00% 2.25% 45.77% -77 +2179 +26984 2176 121 | ``` 122 | 123 | To test NTP performance with NTS authentication it is necessary to obtain a 124 | valid C2S key and cookie using a separate NTS-KE client. The AEAD algorithm is 125 | `AEAD_AES_SIV_CMAC_256` (15). The key and cookie are specified as hexadecimal 126 | numbers with the `-S` option. Without the `-o` option the server response time 127 | (interval between remote RX and TX timestamps) is printed instead of the TX 128 | timestamp offset: 129 | 130 | ``` 131 | # ./ntpperf -i eth0 -m AA:BB:CC:DD:EE:FF -d 192.168.123.1 -s 172.18.0.0/16 -B -S 44508CC523BD3FF2334A3B73E70969BF69BC4753EB745FD8EC054FF083DF788C,2D1FF1538B637C3DDE7A1AE23D0A1B104A8F68C6E1E1BB54A3139955BACE08307C2C3210F6A039A956A72038A42FBF3489EBFB207EDB6CDBFFF2B4F25818787D78CD170637475394A8E597827CC06D78E9CD6CED3D8573D69AA7E1303CB79C0499D7BB21 132 | | responses | response time (ns) 133 | rate clients | lost invalid basic xleave | min mean max stddev 134 | 1000 100 0.00% 0.00% 100.00% 0.00% +19172 +55881 +91692 10729 135 | 1500 150 0.00% 0.00% 100.00% 0.00% +20906 +55601 +92014 10875 136 | 2250 225 0.00% 0.00% 100.00% 0.00% +19151 +54690 +224118 11974 137 | 3375 337 0.00% 0.00% 100.00% 0.00% +16020 +53464 +75994 12282 138 | 5062 506 0.00% 0.00% 100.00% 0.00% +14089 +49891 +168482 15643 139 | 7593 759 0.00% 0.00% 100.00% 0.00% +13429 +38142 +184297 18957 140 | 11389 1138 0.00% 0.00% 100.00% 0.00% +11065 +28241 +156426 16472 141 | 17083 1708 0.00% 0.00% 100.00% 0.00% +9592 +18460 +154719 6312 142 | 25624 2562 0.00% 0.00% 100.00% 0.00% +8077 +19027 +164717 3842 143 | 38436 3843 0.00% 0.00% 100.00% 0.00% +4782 +12554 +237316 3736 144 | 57654 5765 0.00% 0.00% 100.00% 0.00% +3951 +11393 +122890 3255 145 | 86481 8648 0.00% 0.00% 100.00% 0.00% +3762 +44640 +428393 47749 146 | 129721 12972 0.00% 0.00% 100.00% 0.00% +3696 +23200 +280790 15876 147 | 194581 16384 0.00% 0.00% 100.00% 0.00% +3950 +35853 +412163 21372 148 | 291871 16384 31.54% 0.00% 68.46% 0.00% +256319 +417028 +855913 32953 149 | 437806 16384 60.92% 0.00% 39.08% 0.00% +284758 +503148 +933666 88104 150 | ``` 151 | 152 | The following test measures the throughput of the PTP master using delay 153 | requests: 154 | 155 | ``` 156 | # ./ntpperf -i eth0 -m AA:BB:CC:DD:EE:FF -d 192.168.123.1 -s 172.18.0.0/16 -D 0 157 | | responses | 158 | rate clients | lost invalid delay sync/fw | 159 | 1000 100 0.00% 0.00% 100.00% 0.00% 160 | 1500 150 0.00% 0.00% 100.00% 0.00% 161 | 2250 225 0.00% 0.00% 100.00% 0.00% 162 | 3375 337 0.00% 0.00% 100.00% 0.00% 163 | 5062 506 0.00% 0.00% 100.00% 0.00% 164 | 7593 759 0.00% 0.00% 100.00% 0.00% 165 | 11389 1138 0.00% 0.00% 100.00% 0.00% 166 | 17083 1708 0.00% 0.00% 100.00% 0.00% 167 | 25624 2562 0.00% 0.00% 100.00% 0.00% 168 | 38436 3843 0.00% 0.00% 100.00% 0.00% 169 | 57654 5765 0.01% 0.00% 99.99% 0.00% 170 | 86481 8648 0.04% 0.00% 99.96% 0.00% 171 | 129721 12972 0.00% 0.00% 100.00% 0.00% 172 | 194581 16384 0.08% 0.00% 99.92% 0.00% 173 | 291871 16384 0.09% 0.00% 99.91% 0.00% 174 | 437806 16384 80.90% 0.00% 19.10% 0.00% 175 | ``` 176 | 177 | And this test measures the performance of the PTP master using NetSync Monitor 178 | requests: 179 | 180 | ``` 181 | # ./ntpperf -i eth0 -m AA:BB:CC:DD:EE:FF -d 192.168.123.1 -s 172.18.0.0/16 -N 0 -o 1.95e-6 -H 182 | | responses | TX timestamp offset (ns) 183 | rate clients | lost invalid delay sync/fw | min mean max stddev 184 | 1000 100 0.00% 0.00% 100.00% 100.00% +13 +27 +38 7 185 | 1500 150 0.00% 0.00% 100.00% 100.00% -22 +21 +47 18 186 | 2250 225 0.00% 0.00% 100.00% 100.00% -52 -41 -22 7 187 | 3375 337 0.00% 0.00% 100.00% 100.00% -48 -34 -25 7 188 | 5062 506 0.00% 0.00% 100.00% 100.00% -65 -46 -27 9 189 | 7593 759 0.00% 0.00% 100.00% 100.00% -29 +6 +19 12 190 | 11389 1138 0.00% 0.00% 100.00% 100.00% -7 +12 +24 8 191 | 17083 1708 0.00% 0.00% 100.00% 100.00% -3 +7 +26 4 192 | 25624 2562 0.00% 0.00% 100.00% 100.00% -35 -22 +15 12 193 | 38436 3843 0.00% 0.00% 100.00% 100.00% -39 +56 +136 70 194 | 57654 5765 0.00% 0.00% 100.00% 100.00% -56 -18 +61 32 195 | 86481 8648 168.41% 0.00% 15.96% 15.63% -49 +246 +634 235 196 | ``` 197 | 198 | == Author 199 | 200 | Miroslav Lichvar 201 | 202 | == License 203 | 204 | GPLv2+ 205 | -------------------------------------------------------------------------------- /perf.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2018 Miroslav Lichvar 3 | * 4 | * This program is free software; you can redistribute it and/or modify 5 | * it under the terms of the GNU General Public License as published by 6 | * the Free Software Foundation; either version 2 of the License, or 7 | * (at your option) any later version. 8 | * 9 | * This program is distributed in the hope that it will be useful, 10 | * but WITHOUT ANY WARRANTY; without even the implied warranty of 11 | * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 12 | * GNU General Public License for more details. 13 | * 14 | * You should have received a copy of the GNU General Public License 15 | * along with this program. If not, see . 16 | */ 17 | 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include 24 | #include 25 | #include 26 | #include 27 | #include 28 | #include 29 | #include 30 | 31 | #include 32 | 33 | #include "sender.h" 34 | 35 | #define MAX_CLIENTS 16384 36 | 37 | #define PTP_MCAST_ADDR 0xe0000181 /* 224.0.1.129 */ 38 | 39 | struct config { 40 | enum request_mode mode; 41 | char *interface; 42 | char dst_mac[6]; 43 | uint32_t dst_address; 44 | uint32_t src_network; 45 | int src_bits; 46 | int ptp_domain; 47 | int ptp_mcast; 48 | int min_rate; 49 | int max_rate; 50 | unsigned int senders; 51 | bool exp_distribution; 52 | bool allow_late_tx; 53 | double multiplier; 54 | double sampling_interval; 55 | double offset_correction; 56 | bool offset_middle; 57 | bool hw_timestamping; 58 | struct { 59 | const char *c2s; 60 | const char *cookie; 61 | } nts; 62 | }; 63 | 64 | struct client { 65 | uint64_t remote_id; 66 | uint64_t local_id; 67 | struct timespec remote_rx; 68 | struct timespec local_rx; 69 | int warmup; 70 | }; 71 | 72 | struct perf_stats { 73 | int clients; 74 | int requests; 75 | int invalid_responses; 76 | union { 77 | int basic_responses; 78 | int delay_responses; 79 | }; 80 | union { 81 | int interleaved_responses; 82 | int sync_responses; 83 | }; 84 | int offset_updates; 85 | double sum_m_offset; 86 | double sum_s_offset; 87 | double min_offset; 88 | double max_offset; 89 | }; 90 | 91 | static pcap_t *open_pcap(struct config *config) { 92 | char errbuf[PCAP_ERRBUF_SIZE]; 93 | pcap_t *pcap; 94 | int r; 95 | 96 | if (!(pcap = pcap_create(config->interface, errbuf))) { 97 | fprintf(stderr, "pcap: %s\n", errbuf); 98 | goto err; 99 | } 100 | 101 | if ((r = pcap_set_snaplen(pcap, 128)) || 102 | (r = pcap_set_promisc(pcap, config->ptp_mcast)) || 103 | (r = pcap_set_timeout(pcap, 10)) || 104 | (r = pcap_set_immediate_mode(pcap, 1)) || 105 | (r = pcap_set_buffer_size(pcap, 1 << 24)) || 106 | (r = pcap_set_tstamp_type(pcap, config->hw_timestamping ? 107 | PCAP_TSTAMP_ADAPTER_UNSYNCED : PCAP_TSTAMP_HOST)) || 108 | (r = pcap_set_tstamp_precision(pcap, PCAP_TSTAMP_PRECISION_NANO))) { 109 | fprintf(stderr, "pcap: %s\n", pcap_statustostr(r)); 110 | goto err; 111 | } 112 | 113 | if ((r = pcap_activate(pcap))) { 114 | fprintf(stderr, "pcap: %s\n", pcap_statustostr(r)); 115 | if (r < 0) 116 | goto err; 117 | } 118 | 119 | if (pcap_set_datalink(pcap, DLT_EN10MB)) { 120 | fprintf(stderr, "Could not set pcap datalink\n"); 121 | goto err; 122 | } 123 | 124 | if (pcap_setdirection(pcap, PCAP_D_IN)) { 125 | fprintf(stderr, "Could not set pcap direction\n"); 126 | goto err; 127 | } 128 | 129 | if (pcap_setnonblock(pcap, 1, errbuf)) { 130 | fprintf(stderr, "pcap: %s\n", errbuf); 131 | goto err; 132 | } 133 | 134 | return pcap; 135 | err: 136 | if (pcap) 137 | pcap_close(pcap); 138 | return NULL; 139 | } 140 | 141 | static bool get_iface_mac(struct config *config, char mac[6]) { 142 | struct ifreq req; 143 | int sock_fd; 144 | 145 | sock_fd = socket(AF_INET, SOCK_DGRAM, 0); 146 | if (sock_fd < 0) 147 | return false; 148 | 149 | snprintf(req.ifr_name, sizeof (req.ifr_name), "%s", config->interface); 150 | 151 | if (ioctl(sock_fd, SIOCGIFHWADDR, &req)) { 152 | fprintf(stderr, "Could not get MAC address of %s\n", config->interface); 153 | close(sock_fd); 154 | return false; 155 | } 156 | close(sock_fd); 157 | 158 | memcpy(mac, req.ifr_hwaddr.sa_data, 6); 159 | return true; 160 | } 161 | 162 | static void add_nsec_to_ts(struct timespec *ts, uint64_t nsec) { 163 | ts->tv_sec += nsec / 1000000000U; 164 | ts->tv_nsec += nsec % 1000000000U; 165 | while (ts->tv_nsec >= 1000000000U) { 166 | ts->tv_nsec -= 1000000000U; 167 | ts->tv_sec++; 168 | } 169 | } 170 | 171 | static int compare_ts(struct timespec *ts1, struct timespec *ts2) { 172 | if (ts1->tv_sec != ts2->tv_sec) 173 | return ts1->tv_sec - ts2->tv_sec; 174 | if (ts1->tv_nsec != ts2->tv_nsec) 175 | return ts1->tv_nsec - ts2->tv_nsec; 176 | return 0; 177 | } 178 | 179 | static double diff_ts(struct timespec *ts1, struct timespec *ts2) { 180 | return (ts1->tv_sec - ts2->tv_sec) + 1e-9 * (ts1->tv_nsec - ts2->tv_nsec); 181 | } 182 | 183 | static struct timespec convert_ntp_ts(uint64_t ntp_ts) { 184 | struct timespec ts; 185 | 186 | ntp_ts = be64toh(ntp_ts); 187 | ts.tv_sec = (ntp_ts >> 32) - 2208988800; 188 | ts.tv_nsec = (ntp_ts & 0xffffffffU) / 4.294967296; 189 | 190 | return ts; 191 | } 192 | 193 | static struct timespec convert_ptp_ts(uint16_t hi, uint32_t mid, uint32_t lo) { 194 | struct timespec ts; 195 | 196 | ts.tv_sec = (uint64_t)ntohs(hi) << 32 | ntohl(mid); 197 | ts.tv_nsec = ntohl(lo); 198 | 199 | return ts; 200 | } 201 | 202 | static void make_request(struct sender_request *request, struct client *client, int index, 203 | struct config *config, struct timespec *when) { 204 | request->when = *when; 205 | request->src_address = config->src_network ^ (index % (1U << (32 - config->src_bits))); 206 | request->_pad = 0; 207 | request->remote_id = client->remote_id; 208 | request->local_id = (uint64_t)random() << 32 | random(); 209 | 210 | client->local_id = request->local_id; 211 | } 212 | 213 | static bool process_response(struct pcap_pkthdr *header, const u_char *data, struct config *config, 214 | struct perf_stats *stats, struct client *clients, int num_clients) { 215 | struct client *client; 216 | struct timespec local_rx = { .tv_sec = header->ts.tv_sec, .tv_nsec = header->ts.tv_usec }; 217 | struct timespec prev_local_rx, prev_remote_rx, remote_rx = {0}, remote_tx = {0}; 218 | int src_port, dst_port, ptp_type = 0, ptp_maj_ver = 0; 219 | uint32_t dst_address; 220 | bool valid; 221 | double offset, delta; 222 | 223 | if (header->caplen < 86) 224 | return false; 225 | 226 | if (memcmp(config->dst_mac, data + 6, 6)) 227 | return false; 228 | 229 | if (ntohs(*(uint16_t *)(data + 12)) != 0x0800) 230 | return false; 231 | 232 | data += 14; 233 | if (data[0] >> 4 != 4 || (data[0] & 0xf) != 5 || data[9] != 17 || 234 | ntohl(*(uint32_t *)(data + 12)) != config->dst_address) 235 | return false; 236 | 237 | dst_address = ntohl(*(uint32_t *)(data + 16)); 238 | src_port = ntohs(*(uint16_t *)(data + 20)); 239 | dst_port = ntohs(*(uint16_t *)(data + 22)); 240 | data += 28; 241 | 242 | if (config->ptp_mcast) { 243 | if (dst_address != PTP_MCAST_ADDR || (data[0] & 0xf) != 9) 244 | return false; 245 | dst_address = ntohl(*(uint32_t *)(data + 44)); 246 | } 247 | 248 | if ((dst_address ^ config->src_network) >> (32 - config->src_bits)) 249 | return false; 250 | 251 | client = &clients[(dst_address ^ config->src_network) % (uint32_t)num_clients]; 252 | prev_remote_rx = client->remote_rx; 253 | prev_local_rx = client->local_rx; 254 | 255 | switch (config->mode) { 256 | case NTP_BASIC: 257 | case NTP_INTERLEAVED: 258 | if (src_port != 123) 259 | return false; 260 | 261 | valid = header->caplen >= 90 && (data[0] & 0x7) == 0x4 && 262 | (*(uint64_t *)(data + 24) & -2ULL) == (client->local_id & -2ULL) && 263 | (!config->nts.cookie || header->len > 90 + 4 + 32 + 4 + 16 + 16 + 4); 264 | if (valid) { 265 | if (config->mode == NTP_INTERLEAVED) 266 | client->remote_id = *(uint64_t *)(data + 32); 267 | else 268 | client->remote_id = *(uint64_t *)(data + 40); 269 | client->remote_rx = convert_ntp_ts(*(uint64_t *)(data + 32)); 270 | client->local_rx = local_rx; 271 | } 272 | break; 273 | case PTP_DELAY: 274 | case PTP_NSM: 275 | if (dst_port != 319 && dst_port != 320) 276 | return false; 277 | 278 | ptp_type = data[0] & 0xf; 279 | ptp_maj_ver = data[1] & 0xf; 280 | valid = header->caplen >= 86 && ptp_maj_ver == 2 && 281 | *(uint16_t *)(data + 30) == (uint16_t)client->local_id && 282 | ((ptp_type == 9 && dst_port == 320) || 283 | (config->mode == PTP_NSM && 284 | ((ptp_type == 0 && dst_port == 319) || 285 | (ptp_type == 8 && dst_port == 320)))); 286 | if (!valid) 287 | break; 288 | 289 | switch (ptp_type) { 290 | case 0: 291 | client->local_rx = local_rx; 292 | break; 293 | case 9: 294 | client->remote_rx = convert_ptp_ts(*(uint16_t *)(data + 34), 295 | *(uint32_t *)(data + 36), 296 | *(uint32_t *)(data + 40)); 297 | /* Fall through */ 298 | case 8: 299 | memset(&client->local_rx, 0, sizeof client->local_rx); 300 | break; 301 | default: 302 | assert(0); 303 | } 304 | break; 305 | default: 306 | assert(0); 307 | } 308 | 309 | if (!valid) { 310 | if (!client->warmup) 311 | stats->invalid_responses++; 312 | return false; 313 | } 314 | 315 | if (client->warmup) 316 | return true; 317 | 318 | switch (config->mode) { 319 | case NTP_BASIC: 320 | case NTP_INTERLEAVED: 321 | if (*(uint64_t *)(data + 24) == client->local_id) { 322 | stats->basic_responses++; 323 | if (config->mode != NTP_BASIC) 324 | return true; 325 | remote_rx = client->remote_rx; 326 | } else { 327 | stats->interleaved_responses++; 328 | if (config->mode != NTP_INTERLEAVED) 329 | return true; 330 | remote_rx = prev_remote_rx; 331 | local_rx = prev_local_rx; 332 | } 333 | 334 | remote_tx = convert_ntp_ts(*(uint64_t *)(data + 40)); 335 | break; 336 | case PTP_DELAY: 337 | case PTP_NSM: 338 | switch (ptp_type) { 339 | case 8: 340 | local_rx = prev_local_rx; 341 | /* Fall through */ 342 | case 0: 343 | /* TODO: handle reversed order of sync and followup */ 344 | if (ptp_type == 0 && data[6] & 0x2) 345 | return true; 346 | 347 | stats->sync_responses++; 348 | 349 | remote_rx = client->remote_rx; 350 | remote_tx = convert_ptp_ts(*(uint16_t *)(data + 34), 351 | *(uint32_t *)(data + 36), 352 | *(uint32_t *)(data + 40)); 353 | break; 354 | case 9: 355 | stats->delay_responses++; 356 | return true; 357 | default: 358 | assert(0); 359 | } 360 | break; 361 | default: 362 | assert(0); 363 | } 364 | 365 | if (config->offset_correction) { 366 | if (!local_rx.tv_sec || !remote_tx.tv_sec || !remote_rx.tv_sec) 367 | return true; 368 | offset = diff_ts(&local_rx, &remote_tx) - config->offset_correction; 369 | if (config->offset_middle) 370 | offset += diff_ts(&remote_tx, &remote_rx) / 2; 371 | } else { 372 | if (!remote_tx.tv_sec || !remote_rx.tv_sec) 373 | return true; 374 | offset = diff_ts(&remote_tx, &remote_rx); 375 | } 376 | 377 | stats->offset_updates++; 378 | /* Accumulate for mean and variance using Welford's algorithm. */ 379 | delta = offset - stats->sum_m_offset; 380 | stats->sum_m_offset += delta / stats->offset_updates; 381 | stats->sum_s_offset += delta * (offset - stats->sum_m_offset); 382 | if (stats->min_offset > offset) 383 | stats->min_offset = offset; 384 | if (stats->max_offset < offset) 385 | stats->max_offset = offset; 386 | 387 | return true; 388 | } 389 | 390 | static bool measure_perf(struct config *config, pcap_t *pcap, int *senders, int rate, 391 | struct perf_stats *stats) { 392 | struct sender_request requests[config->senders][MAX_SENDER_REQUESTS]; 393 | struct client clients[MAX_CLIENTS]; 394 | int num_requests[config->senders]; 395 | unsigned int i, num_clients, interval, sender_index = 0, client_index = 0; 396 | struct pcap_pkthdr *header; 397 | const u_char *data; 398 | struct timespec tx_next, max_tx_ahead, now, tx_end, rx_end; 399 | 400 | memset(stats, 0, sizeof *stats); 401 | stats->min_offset = DBL_MAX; 402 | stats->max_offset = -DBL_MAX; 403 | 404 | interval = 1e9 / rate; 405 | 406 | num_clients = rate / 10; 407 | if (num_clients < 1) 408 | num_clients = 1; 409 | if (num_clients > MAX_CLIENTS) 410 | num_clients = MAX_CLIENTS; 411 | if (num_clients > 1U << (32 - config->src_bits)) { 412 | fprintf(stderr, "Warning: source network might be too small for rate %d\n", rate); 413 | num_clients = 1U << (32 - config->src_bits); 414 | } 415 | 416 | assert(num_clients > 0 && num_clients <= MAX_CLIENTS); 417 | 418 | stats->clients = num_clients; 419 | 420 | memset(clients, 0, sizeof clients); 421 | for (i = 0; i < num_clients; i++) 422 | clients[i].warmup = 2 + 1; 423 | 424 | clock_gettime(CLOCK_MONOTONIC, &now); 425 | tx_next = tx_end = now; 426 | 427 | add_nsec_to_ts(&tx_end, 1.0e9 * config->sampling_interval + 428 | interval * num_clients * (clients[0].warmup - 1)); 429 | rx_end = tx_end; 430 | add_nsec_to_ts(&rx_end, interval * num_clients); 431 | 432 | while (1) { 433 | max_tx_ahead = now; 434 | add_nsec_to_ts(&max_tx_ahead, interval * num_clients / 4); 435 | 436 | if (compare_ts(&now, &rx_end) > 0) 437 | break; 438 | 439 | for (i = 0; i < config->senders; i++) 440 | num_requests[i] = 0; 441 | 442 | for (i = 0; i < MAX_SENDER_REQUESTS * config->senders; i++) { 443 | if (compare_ts(&tx_next, &tx_end) > 0 || 444 | compare_ts(&tx_next, &max_tx_ahead) > 0) 445 | break; 446 | 447 | if (compare_ts(&now, &tx_next) > 0 && !config->allow_late_tx) { 448 | fprintf(stderr, "Could not send requests at rate %d\n", rate); 449 | return false; 450 | } 451 | 452 | make_request(&requests[sender_index][num_requests[sender_index]++], 453 | &clients[client_index], client_index, config, &tx_next); 454 | 455 | if (clients[client_index].warmup) 456 | clients[client_index].warmup--; 457 | if (!clients[client_index].warmup) 458 | stats->requests++; 459 | 460 | sender_index = (sender_index + 1) % config->senders; 461 | client_index = (client_index + 1) % num_clients; 462 | 463 | add_nsec_to_ts(&tx_next, !config->exp_distribution ? interval : 464 | interval * -log((random() & 0x7fffffff) / 2147483647.0)); 465 | } 466 | 467 | for (i = 0; i < config->senders; i++) { 468 | if (num_requests[i] == 0) 469 | continue; 470 | if (!sender_send_requests(senders[i], requests[i], num_requests[i])) 471 | return false; 472 | } 473 | 474 | while (pcap_next_ex(pcap, &header, &data)) { 475 | process_response(header, data, config, stats, clients, num_clients); 476 | } 477 | 478 | clock_gettime(CLOCK_MONOTONIC, &now); 479 | } 480 | 481 | return true; 482 | } 483 | 484 | static void print_header(struct config *config) { 485 | int offset; 486 | 487 | printf(" | responses |"); 488 | 489 | if (config->mode == PTP_DELAY) 490 | offset = 0, printf("\n"); 491 | else if (config->offset_correction && !config->offset_middle) 492 | offset = 1, printf(" TX timestamp offset (ns)\n"); 493 | else if (config->offset_correction) 494 | offset = 1, printf(" (RX+TX)/2 offset (ns)\n"); 495 | else 496 | offset = 1, printf(" response time (ns)\n"); 497 | 498 | printf("rate clients | lost invalid %15s |%s\n", 499 | config->mode <= NTP_INTERLEAVED ? "basic xleave" : "delay sync/fw", 500 | offset ? " min mean max stddev" : ""); 501 | } 502 | 503 | static int get_lost_packets(struct perf_stats *stats, struct config *config) { 504 | switch (config->mode) { 505 | case NTP_BASIC: 506 | case NTP_INTERLEAVED: 507 | return stats->requests - stats->invalid_responses - 508 | stats->basic_responses - stats->interleaved_responses; 509 | case PTP_DELAY: 510 | return stats->requests - stats->invalid_responses - 511 | stats->delay_responses; 512 | case PTP_NSM: 513 | return 2 * stats->requests - stats->invalid_responses - 514 | stats->delay_responses - stats->sync_responses; 515 | default: 516 | assert(0); 517 | } 518 | } 519 | 520 | static void print_stats(struct perf_stats *stats, struct config *config, int rate) { 521 | printf("%-8d %5d %6.2f%% %6.2f%% %6.2f%% %6.2f%%", 522 | rate, stats->clients, 523 | 100.0 * get_lost_packets(stats, config) / stats->requests, 524 | 100.0 * stats->invalid_responses / stats->requests, 525 | 100.0 * stats->basic_responses / stats->requests, 526 | 100.0 * stats->interleaved_responses / stats->requests); 527 | if (stats->offset_updates > 1) 528 | printf(" %+7.0f %+7.0f %+7.0f %6.0f", 529 | 1e9 * stats->min_offset, 1e9 * stats->sum_m_offset, 530 | 1e9 * stats->max_offset, 531 | 1e9 * sqrt(stats->sum_s_offset / (stats->offset_updates - 1.5))); 532 | if (0) 533 | printf(" | %7d", stats->requests); 534 | 535 | printf("\n"); 536 | } 537 | 538 | static bool run_perf(struct config *config) { 539 | struct sender_config sender_config; 540 | struct perf_stats stats; 541 | pcap_t *pcap; 542 | int i, rate, senders[config->senders]; 543 | bool ret = true; 544 | 545 | sender_config.mode = config->mode; 546 | if (!get_iface_mac(config, sender_config.src_mac)) 547 | return false; 548 | 549 | memcpy(sender_config.dst_mac, config->dst_mac, 6); 550 | sender_config.dst_address = config->ptp_mcast ? 551 | PTP_MCAST_ADDR : config->dst_address; 552 | sender_config.ptp_domain = config->ptp_domain; 553 | sender_config.ptp_mcast = config->ptp_mcast; 554 | sender_config.nts.c2s = config->nts.c2s; 555 | sender_config.nts.cookie = config->nts.cookie; 556 | 557 | pcap = open_pcap(config); 558 | if (!pcap) 559 | return false; 560 | 561 | sender_config.sock_fd = pcap_fileno(pcap); 562 | 563 | for (i = 0; i < config->senders; i++) { 564 | senders[i] = sender_start(&sender_config); 565 | if (!senders[i]) { 566 | for (i--; i >= 0; i--) 567 | sender_stop(senders[i]); 568 | pcap_close(pcap); 569 | return false; 570 | } 571 | } 572 | 573 | print_header(config); 574 | 575 | for (rate = config->min_rate; rate <= config->max_rate; rate *= config->multiplier) { 576 | ret = measure_perf(config, pcap, senders, rate, &stats); 577 | if (!ret) 578 | break; 579 | 580 | print_stats(&stats, config, rate); 581 | 582 | if (get_lost_packets(&stats, config) + stats.invalid_responses > 583 | stats.requests / 2) 584 | break; 585 | } 586 | 587 | for (i = 0; i < config->senders; i++) 588 | sender_stop(senders[i]); 589 | pcap_close(pcap); 590 | 591 | return ret; 592 | } 593 | 594 | static bool is_local_network(uint32_t net, int bits) { 595 | return ((net ^ ntohl(inet_addr("10.0.0.0"))) >> 24 == 0 && bits >= 8) || 596 | ((net ^ ntohl(inet_addr("172.16.0.0"))) >> 20 == 0 && bits >= 12) || 597 | ((net ^ ntohl(inet_addr("192.168.0.0"))) >> 16 == 0 && bits >= 16); 598 | } 599 | 600 | int main(int argc, char **argv) { 601 | struct config config; 602 | char *s; 603 | int opt, dst_mac_set = 0; 604 | 605 | srandom(time(NULL)); 606 | setvbuf(stdout, NULL, _IOLBF, BUFSIZ); 607 | 608 | memset(&config, 0, sizeof config); 609 | config.mode = INVALID_MODE; 610 | config.min_rate = 1000; 611 | config.max_rate = 1000000; 612 | config.senders = 1; 613 | config.multiplier = 1.5; 614 | config.sampling_interval = 2.0; 615 | 616 | while ((opt = getopt(argc, argv, "BID:N:i:s:d:m:Mr:p:elt:x:o:OHS:h")) != -1) { 617 | switch (opt) { 618 | case 'B': 619 | config.mode = NTP_BASIC; 620 | break; 621 | case 'I': 622 | config.mode = NTP_INTERLEAVED; 623 | break; 624 | case 'D': 625 | config.mode = PTP_DELAY; 626 | config.ptp_domain = atoi(optarg); 627 | break; 628 | case 'N': 629 | config.mode = PTP_NSM; 630 | config.ptp_domain = atoi(optarg); 631 | break; 632 | case 'i': 633 | config.interface = optarg; 634 | break; 635 | case 's': 636 | if (!(s = strchr(optarg, '/'))) 637 | goto err; 638 | *s = '\0'; 639 | if (inet_pton(AF_INET, optarg, &config.src_network) <= 0) 640 | goto err; 641 | config.src_network = ntohl(config.src_network); 642 | config.src_bits = atoi(s + 1); 643 | break; 644 | case 'd': 645 | if (inet_pton(AF_INET, optarg, &config.dst_address) <= 0) 646 | goto err; 647 | config.dst_address = ntohl(config.dst_address); 648 | break; 649 | case 'm': 650 | if (sscanf(optarg, "%hhx:%hhx:%hhx:%hhx:%hhx:%hhx", 651 | config.dst_mac + 0, config.dst_mac + 1, 652 | config.dst_mac + 2, config.dst_mac + 3, 653 | config.dst_mac + 4, config.dst_mac + 5) != 6) 654 | goto err; 655 | dst_mac_set = 1; 656 | break; 657 | case 'M': 658 | config.ptp_mcast = 1; 659 | break; 660 | case 'r': 661 | if ((s = strchr(optarg, '-'))) { 662 | *s = '\0'; 663 | config.min_rate = atoi(optarg); 664 | config.max_rate = atoi(s + 1); 665 | } else { 666 | config.min_rate = config.max_rate = atoi(optarg); 667 | } 668 | break; 669 | case 'p': 670 | config.senders = atoi(optarg); 671 | break; 672 | case 'e': 673 | config.exp_distribution = true; 674 | break; 675 | case 'l': 676 | config.allow_late_tx = true; 677 | break; 678 | case 'x': 679 | config.multiplier = atof(optarg); 680 | break; 681 | case 't': 682 | config.sampling_interval = atof(optarg); 683 | break; 684 | case 'o': 685 | config.offset_correction = atof(optarg); 686 | break; 687 | case 'O': 688 | config.offset_middle = true; 689 | break; 690 | case 'H': 691 | config.hw_timestamping = true; 692 | break; 693 | #ifdef NTS 694 | case 'S': 695 | if (!optarg || !(s = strchr(optarg, ','))) 696 | goto err; 697 | *s = '\0'; 698 | config.nts.c2s = optarg; 699 | config.nts.cookie = s + 1; 700 | break; 701 | #endif 702 | default: 703 | goto err; 704 | } 705 | } 706 | 707 | if (config.mode == INVALID_MODE || !config.interface || !dst_mac_set || 708 | (config.ptp_mcast && config.mode != PTP_DELAY) || 709 | !config.dst_address || config.src_bits < 8 || config.src_bits > 32 || 710 | config.min_rate < 1 || config.multiplier < 1.0 || config.sampling_interval < 0.2 || 711 | config.senders < 1 || config.senders > 16) 712 | goto err; 713 | 714 | if (!is_local_network(config.dst_address, 32) || 715 | !is_local_network(config.src_network, config.src_bits)) { 716 | fprintf(stderr, "Non-local source or destination network\n"); 717 | return 1; 718 | } 719 | 720 | return !run_perf(&config); 721 | err: 722 | fprintf(stderr, "Usage: %s MODE NETWORK-OPTIONS [OTHER-OPTIONS]\n", argv[0]); 723 | fprintf(stderr, "\nMode:\n"); 724 | fprintf(stderr, "\t-B send NTP client requests in basic mode\n"); 725 | fprintf(stderr, "\t-I send NTP client requests in interleaved mode\n"); 726 | fprintf(stderr, "\t-D DOMAIN send PTP delay requests\n"); 727 | fprintf(stderr, "\t-N DOMAIN send PTP NetSync Monitor (NSM) requests\n"); 728 | fprintf(stderr, "\nNetwork options:\n"); 729 | fprintf(stderr, "\t-i INTERFACE specify network interface\n"); 730 | fprintf(stderr, "\t-s NETWORK/BITS specify source IPv4 network\n"); 731 | fprintf(stderr, "\t-d IP-ADDRESS specify destination IPv4 address\n"); 732 | fprintf(stderr, "\t-m MAC specify destination MAC address\n"); 733 | fprintf(stderr, "\nOther options:\n"); 734 | fprintf(stderr, "\t-M send multicast PTP delay requests\n"); 735 | fprintf(stderr, "\t-r RATE[-RATE] specify minimum and maximum rate (1000-1000000)\n"); 736 | fprintf(stderr, "\t-p NUMBER specify number of processes to send requests (1)\n"); 737 | fprintf(stderr, "\t-e make transmit interval exponentially distributed\n"); 738 | fprintf(stderr, "\t-l allow late transmissions\n"); 739 | fprintf(stderr, "\t-x MULT specify rate multiplier (1.5)\n"); 740 | fprintf(stderr, "\t-t INTERVAL specify sampling interval (2.0 seconds)\n"); 741 | fprintf(stderr, "\t-o CORRECTION print offset between remote TX and local RX timestamp\n"); 742 | fprintf(stderr, "\t with specified correction (e.g. network and RX delay)\n"); 743 | fprintf(stderr, "\t-O with -o use remote (RX+TX)/2 instead of TX timestamp\n"); 744 | fprintf(stderr, "\t-H enable HW timestamping for TX offset statistics\n"); 745 | #ifdef NTS 746 | fprintf(stderr, "\t-S C2S,COOKIE authenticate NTP requests with NTS\n"); 747 | #endif 748 | fprintf(stderr, "\t-h print this help message\n"); 749 | return 1; 750 | } 751 | -------------------------------------------------------------------------------- /sender.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2018 Miroslav Lichvar 3 | * 4 | * This program is free software; you can redistribute it and/or modify 5 | * it under the terms of the GNU General Public License as published by 6 | * the Free Software Foundation; either version 2 of the License, or 7 | * (at your option) any later version. 8 | * 9 | * This program is distributed in the hope that it will be useful, 10 | * but WITHOUT ANY WARRANTY; without even the implied warranty of 11 | * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 12 | * GNU General Public License for more details. 13 | * 14 | * You should have received a copy of the GNU General Public License 15 | * along with this program. If not, see . 16 | */ 17 | 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include 24 | #include 25 | #include 26 | #include 27 | #include 28 | #include 29 | 30 | #ifdef NTS 31 | #include 32 | #endif 33 | 34 | #include "sender.h" 35 | 36 | #ifdef NTS 37 | #define MAX_PACKET_LENGTH 512 38 | #else 39 | #define MAX_PACKET_LENGTH 128 40 | #endif 41 | 42 | #define MAX_PACKETS 128 43 | 44 | struct nts_context { 45 | int cookie_len; 46 | #ifdef NTS 47 | unsigned char cookie[256]; 48 | gnutls_aead_cipher_hd_t cipher; 49 | #endif 50 | }; 51 | 52 | #ifdef NTS 53 | static unsigned int convert_hex_to_bytes(const char *hex, void *buf, unsigned int len) { 54 | char *p, byte[3]; 55 | unsigned int i; 56 | 57 | for (i = 0; i < len && *hex != '\0'; i++) { 58 | byte[0] = *hex++; 59 | if (*hex == '\0') 60 | return 0; 61 | byte[1] = *hex++; 62 | byte[2] = '\0'; 63 | ((char *)buf)[i] = strtol(byte, &p, 16); 64 | 65 | if (p != byte + 2) 66 | return 0; 67 | } 68 | 69 | return *hex == '\0' ? i : 0; 70 | } 71 | #endif 72 | 73 | static bool initialize_nts(struct sender_config *config, struct nts_context *nts) { 74 | #ifdef NTS 75 | unsigned char key[256]; 76 | gnutls_datum_t datum; 77 | 78 | if (!config->nts.c2s || !config->nts.cookie || 79 | !(config->mode == NTP_BASIC || config->mode == NTP_INTERLEAVED)) { 80 | nts->cookie_len = 0; 81 | nts->cipher = NULL; 82 | return true; 83 | } 84 | 85 | nts->cookie_len = convert_hex_to_bytes(config->nts.cookie, 86 | nts->cookie, sizeof nts->cookie); 87 | if (nts->cookie_len == 0 || nts->cookie_len % 4 != 0) { 88 | fprintf(stderr, "Invalid cookie length %d\n", nts->cookie_len); 89 | return false; 90 | } 91 | 92 | datum.data = key; 93 | datum.size = convert_hex_to_bytes(config->nts.c2s, key, sizeof key); 94 | 95 | if (gnutls_aead_cipher_init(&nts->cipher, GNUTLS_CIPHER_AES_128_SIV, &datum) < 0) { 96 | fprintf(stderr, "Invalid key length: %d\n", datum.size); 97 | return false; 98 | } 99 | 100 | #else 101 | nts->cookie_len = 0; 102 | #endif 103 | return true; 104 | } 105 | 106 | static void destroy_nts(struct nts_context *nts) { 107 | #ifdef NTS 108 | if (nts->cipher) 109 | gnutls_aead_cipher_deinit(nts->cipher); 110 | #endif 111 | } 112 | 113 | static int make_packet(struct sender_request *request, struct sender_config *config, 114 | struct nts_context *nts, unsigned char *buf, int max_len) { 115 | unsigned char *auth = NULL; 116 | uint32_t sum = 0; 117 | uint16_t carry; 118 | int i, len = 0, data_len, src_port, dst_port; 119 | 120 | switch (config->mode) { 121 | case NTP_BASIC: 122 | case NTP_INTERLEAVED: 123 | src_port = 32768 + random() % 28000; 124 | dst_port = 123; 125 | data_len = 48; 126 | if (nts->cookie_len > 0) 127 | data_len += 4 + 32 + 4 + nts->cookie_len + 4 + 4 + 16 + 16; 128 | break; 129 | case PTP_DELAY: 130 | case PTP_NSM: 131 | src_port = dst_port = 319; 132 | data_len = config->mode == PTP_NSM ? 48 : 44; 133 | break; 134 | default: 135 | assert(0); 136 | } 137 | 138 | assert(max_len >= 128); 139 | memset(buf, 0, max_len); 140 | 141 | /* Ethernet header */ 142 | memcpy(buf + 0, config->dst_mac, 6); 143 | memcpy(buf + 6, config->src_mac, 6); 144 | *(uint16_t *)(buf + 12) = htons(0x0800); 145 | buf += 14, len += 14; 146 | 147 | /* IP header */ 148 | memcpy(buf, "\x45\x00\x00\x00\xd7\xe9\x40\x00\x40\x11", 10); 149 | *(uint16_t *)(buf + 2) = htons(20 + 8 + data_len); 150 | *(uint32_t *)(buf + 12) = htonl(request->src_address); 151 | *(uint32_t *)(buf + 16) = htonl(config->dst_address); 152 | 153 | for (i = 0; i < 10; i++) 154 | sum += ((uint16_t *)buf)[i]; 155 | while ((carry = sum >> 16)) 156 | sum = (sum & 0xffff) + carry; 157 | 158 | *(uint16_t *)(buf + 10) = ~sum; 159 | buf += 20, len += 20; 160 | 161 | /* UDP header and data */ 162 | *(uint16_t *)(buf + 0) = htons(src_port); 163 | *(uint16_t *)(buf + 2) = htons(dst_port); 164 | *(uint16_t *)(buf + 4) = htons(8 + data_len); 165 | buf += 8, len += 8; 166 | 167 | assert(max_len >= len + data_len); 168 | 169 | switch (config->mode) { 170 | case NTP_BASIC: 171 | case NTP_INTERLEAVED: 172 | buf[0] = 0xe3; 173 | *(uint64_t *)(buf + 24) = request->remote_id; 174 | *(uint64_t *)(buf + 32) = request->local_id ^ 1; 175 | *(uint64_t *)(buf + 40) = request->local_id; 176 | auth = buf + 48; 177 | break; 178 | case PTP_NSM: 179 | *(uint32_t *)(buf + 44) = htonl(0x21fe0000); 180 | /* Fall through */ 181 | case PTP_DELAY: 182 | *(uint16_t *)(buf + 0) = htons(0x0102); 183 | *(uint16_t *)(buf + 2) = htons(data_len); 184 | *(uint8_t *)(buf + 4) = config->ptp_domain; 185 | buf[6] = config->ptp_mcast ? 0 : 0x4; 186 | *(uint32_t *)(buf + 20) = htonl(request->src_address); 187 | *(uint16_t *)(buf + 30) = request->local_id; 188 | buf[32] = 0x1; 189 | break; 190 | default: 191 | assert(0); 192 | } 193 | 194 | if (auth && nts->cookie_len > 0) { 195 | #ifdef NTS 196 | size_t clen; 197 | 198 | /* Unique Identifier */ 199 | *(uint16_t *)(auth + 0) = htons(0x0104); 200 | *(uint16_t *)(auth + 2) = htons(4 + 32); 201 | *(uint32_t *)(auth + 4) = random(); 202 | *(uint32_t *)(auth + 8) = random(); 203 | auth += 4 + 32; 204 | 205 | /* Cookie */ 206 | *(uint16_t *)(auth + 0) = htons(0x0204); 207 | *(uint16_t *)(auth + 2) = htons(4 + nts->cookie_len); 208 | memcpy(auth + 4, nts->cookie, nts->cookie_len); 209 | auth += 4 + nts->cookie_len; 210 | 211 | /* Authenticator */ 212 | *(uint16_t *)(auth + 0) = htons(0x0404); 213 | *(uint16_t *)(auth + 2) = htons(4 + 4 + 16 + 16); 214 | *(uint16_t *)(auth + 4) = htons(16); 215 | *(uint16_t *)(auth + 6) = htons(16); 216 | *(uint32_t *)(auth + 8) = random(); 217 | *(uint32_t *)(auth + 12) = random(); 218 | *(uint32_t *)(auth + 16) = random(); 219 | *(uint32_t *)(auth + 20) = random(); 220 | clen = 16; 221 | if (gnutls_aead_cipher_encrypt(nts->cipher, 222 | auth + 4 + 4, 16, buf, auth - buf, 0, 223 | "", 0, auth + 4 + 4 + 16, &clen) < 0 || 224 | clen != 16) 225 | assert(0); 226 | auth += 4 + 4 + 16 + 16; 227 | #endif 228 | } 229 | 230 | return len + data_len; 231 | } 232 | 233 | static bool run_sender(int perf_fd, struct sender_config *config) { 234 | struct sender_request requests[MAX_SENDER_REQUESTS]; 235 | struct mmsghdr msg_headers[MAX_PACKETS]; 236 | unsigned char packets[MAX_PACKETS][MAX_PACKET_LENGTH]; 237 | struct iovec msg_iovs[MAX_PACKETS]; 238 | struct nts_context nts; 239 | struct timespec now; 240 | int i, j, r, n, next_tx, sent = 0; 241 | 242 | if (!initialize_nts(config, &nts)) 243 | return false; 244 | 245 | while (1) { 246 | r = read(perf_fd, requests, sizeof requests); 247 | if (r < 0) { 248 | fprintf(stderr, "read() failed: %m\n"); 249 | return false; 250 | } 251 | 252 | assert(r % sizeof requests[0] == 0); 253 | n = r / sizeof requests[0]; 254 | 255 | if (n == 0) 256 | break; 257 | 258 | for (i = 0; i < n; ) { 259 | clock_gettime(CLOCK_MONOTONIC, &now); 260 | 261 | for (j = 0; i < n && j < MAX_PACKETS; i++, j++) { 262 | next_tx = (requests[i].when.tv_sec - now.tv_sec) * 1000000000 + 263 | requests[i].when.tv_nsec - now.tv_nsec; 264 | 265 | if (next_tx > 0) 266 | break; 267 | 268 | memset(&msg_headers[j], 0, sizeof msg_headers[j]); 269 | msg_iovs[j].iov_base = packets[j]; 270 | msg_iovs[j].iov_len = make_packet(&requests[i], config, &nts, 271 | packets[j], sizeof packets[j]); 272 | 273 | msg_headers[j].msg_hdr.msg_iov = &msg_iovs[j]; 274 | msg_headers[j].msg_hdr.msg_iovlen = 1; 275 | } 276 | 277 | if (j > 0) { 278 | for (sent = 0; sent < j; ) { 279 | r = sendmmsg(config->sock_fd, &msg_headers[sent], 280 | j - sent, 0); 281 | if (r < 0) { 282 | if (errno == EAGAIN) 283 | continue; 284 | fprintf(stderr, "send() failed: %m\n"); 285 | return false; 286 | } 287 | sent += r; 288 | } 289 | } 290 | } 291 | } 292 | 293 | destroy_nts(&nts); 294 | 295 | return true; 296 | } 297 | 298 | int sender_start(struct sender_config *config) { 299 | pid_t pid; 300 | int fd, fds[2]; 301 | bool ret; 302 | 303 | if (pipe2(fds, O_DIRECT)) { 304 | fprintf(stderr, "pipe2() failed(): %m\n"); 305 | return 0; 306 | } 307 | 308 | pid = fork(); 309 | 310 | if (pid < 0) { 311 | fprintf(stderr, "fork() failed: %m\n"); 312 | return 0; 313 | } 314 | 315 | if (pid) { 316 | close(fds[0]); 317 | return fds[1]; 318 | } 319 | 320 | for (fd = 3; fd < 100; fd++) { 321 | if (fd != fds[0] && fd != config->sock_fd) 322 | close(fd); 323 | } 324 | 325 | ret = run_sender(fds[0], config); 326 | 327 | close(fds[0]); 328 | close(config->sock_fd); 329 | 330 | exit(!ret); 331 | } 332 | 333 | bool sender_send_requests(int sender_fd, struct sender_request *requests, int num) { 334 | if (write(sender_fd, requests, sizeof (struct sender_request) * num) != 335 | sizeof (struct sender_request) * num) 336 | return false; 337 | return true; 338 | } 339 | 340 | void sender_stop(int sender_fd) { 341 | close(sender_fd); 342 | wait(NULL); 343 | } 344 | -------------------------------------------------------------------------------- /sender.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (C) 2018 Miroslav Lichvar 3 | * 4 | * This program is free software; you can redistribute it and/or modify 5 | * it under the terms of the GNU General Public License as published by 6 | * the Free Software Foundation; either version 2 of the License, or 7 | * (at your option) any later version. 8 | * 9 | * This program is distributed in the hope that it will be useful, 10 | * but WITHOUT ANY WARRANTY; without even the implied warranty of 11 | * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the 12 | * GNU General Public License for more details. 13 | * 14 | * You should have received a copy of the GNU General Public License 15 | * along with this program. If not, see . 16 | */ 17 | 18 | #ifndef SENDER_H 19 | #define SENDER_H 20 | 21 | #include 22 | #include 23 | #include 24 | #include 25 | 26 | #define MAX_SENDER_REQUESTS (PIPE_BUF / sizeof (struct sender_request)) 27 | 28 | enum request_mode { 29 | INVALID_MODE, 30 | NTP_BASIC, 31 | NTP_INTERLEAVED, 32 | PTP_DELAY, 33 | PTP_NSM, 34 | }; 35 | 36 | struct sender_config { 37 | int sock_fd; 38 | enum request_mode mode; 39 | char src_mac[6]; 40 | char dst_mac[6]; 41 | uint32_t dst_address; 42 | int ptp_domain; 43 | int ptp_mcast; 44 | struct { 45 | const char *c2s; 46 | const char *cookie; 47 | } nts; 48 | }; 49 | 50 | struct sender_request { 51 | struct timespec when; 52 | uint32_t src_address; 53 | uint32_t _pad; 54 | uint64_t remote_id; 55 | uint64_t local_id; 56 | }; 57 | 58 | int sender_start(struct sender_config *config); 59 | bool sender_send_requests(int sender_fd, struct sender_request *requests, int num); 60 | void sender_stop(int sender_fd); 61 | 62 | #endif 63 | --------------------------------------------------------------------------------