├── LICENSE ├── README.md ├── sflow ├── CMakeLists.txt ├── FEATURE.yaml ├── node.c ├── sflow.api ├── sflow.c ├── sflow.h ├── sflow.rst ├── sflow_common.h ├── sflow_dlapi.h ├── sflow_dropmon.c ├── sflow_dropmon.h ├── sflow_netlink.c ├── sflow_netlink.h ├── sflow_psample.c ├── sflow_psample.h ├── sflow_psample_fields.h ├── sflow_test.c ├── sflow_usersock.c └── sflow_usersock.h ├── shmtest.c ├── test └── test_sflow.py └── veth_pair_test.sh /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # vpp-sflow 2 | sFlow plugin for VPP 3 | ![vpp-sflow 001](https://github.com/user-attachments/assets/40044f09-4cdc-4e29-9f79-5ba66d2bd124) 4 | 5 | # Getting Started 6 | In a directory $SRC (so that $SRC/vpp has the vpp project) you can clone this project and then use a soft link to make the plugin appear as $SRC/vpp/src/plugins/sflow. After than you can just rebuild VPP. So the steps are... 7 | ``` 8 | cd $SRC && git clone https://github.com/sflow/vpp-sflow 9 | cd $SRC/vpp/src/plugins && ln -s $SRC/vpp-sflow/sflow 10 | cd $SRC/vpp && make rebuild 11 | ``` 12 | 13 | # Load Kernel Module 14 | It is necesary for the "psample" kernel module to be loaded (both now and on reboot): 15 | ``` 16 | sudo modprobe psample 17 | sudo sh -c 'echo "psample" > /etc/modules-load.d/sflow.conf' 18 | ``` 19 | 20 | # Logging 21 | You may want your VPP startup.conf file to have an entry like this: 22 | ``` 23 | logging { 24 | class sflow/all { rate-limit 10000 level debug syslog-level debug } 25 | } 26 | ``` 27 | 28 | # Example CLI config: 29 | ``` 30 | vppctl sflow sampling-rate 10000 31 | vppctl sflow polling-interval 20 32 | vppctl sflow direction both 33 | vppctl sflow drop-monitoring enable 34 | vppctl sflow enable GigabitEthernet0/8/0 35 | vppctl sflow enable GigabitEthernet0/9/0 36 | vppctl sflow enable GigabitEthernet0/a/0 37 | ``` 38 | 39 | # hsflowd required 40 | To export standard sFlow hsflowd must be running, with its mod_vpp module compiled and enabled. The steps are... 41 | ``` 42 | cd $SRC && git clone https://github.com/sflow/host-sflow 43 | cd $SRC/host-sflow 44 | make FEATURES=VPP 45 | sudo make install 46 | # Now edit /etc/hsflowd.conf to enable mod_vpp and mod_psample. See example below. 47 | sudo systemctl enable hsflowd 48 | sudo systemctl start hsflowd 49 | ``` 50 | 51 | Example /etc/hsflowd.conf: 52 | ``` 53 | sflow { 54 | collector { ip=127.0.0.1 udpport=6343 } 55 | psample { group=1 egress=on } 56 | dropmon { start=on limit=50 } 57 | vpp { } 58 | } 59 | 60 | ``` 61 | You can add multiple collectors. If one is only reachable in another namespace you can use: 62 | ``` 63 | collector { ip=172.16.1.1 namespace=mgmt } 64 | ``` 65 | Or in a VRF represented by a Linux netdev: 66 | ``` 67 | collector { ip=192.168.100.2 dev=mgmt0 } 68 | ``` 69 | For more details on hsflowd.conf features and config, see https://sflow.net/host-sflow-linux-config.php 70 | 71 | # Confirm sFlow output 72 | The sflowtool utility can asciify the sFlow feed in various ways. If you have docker you can invoke: 73 | ``` 74 | docker run sflow/sflowtool 75 | ``` 76 | OR, to build and run sflowtool from sources the steps are: 77 | ``` 78 | cd $SRC && git clone https://github.com/sflow/sflowtool 79 | cd $SRC/sflowtool 80 | ./boot.sh 81 | ./configure 82 | make 83 | sudo make install 84 | sflowtool 85 | ``` 86 | 87 | To start with you may only see counter-samples for each interface and for the host as a whole, but when significant traffic enters the VPP interfaces that were configured for sFlow then you should also see packet-samples printed by sflowtool. 88 | 89 | You can adjust the sampling-rate dynamically at any time at the vpp CLI (if you are just running 'ping' then you can set it to 1): 90 | ``` 91 | sflow sampling-rate 100 92 | ``` 93 | 94 | 95 | # Python API 96 | 97 | An example that shows how to manipulate the sFlow plugin programmtically in Python: 98 | 99 | ```python 100 | from vpp_papi import VPPApiClient, VPPApiJSONFiles 101 | import sys 102 | 103 | vpp_api_dir = VPPApiJSONFiles.find_api_dir([]) 104 | vpp_api_files = VPPApiJSONFiles.find_api_files(api_dir=vpp_api_dir) 105 | vpp = VPPApiClient(apifiles=vpp_api_files, server_address="/run/vpp/api.sock") 106 | vpp.connect("sflow-api-client") 107 | print(vpp.api.show_version()) 108 | 109 | print(vpp.api.sflow_sampling_rate_set(sampling_N=10000)) 110 | print(vpp.api.sflow_sampling_rate_get()) 111 | print(vpp.api.sflow_polling_interval_set(polling_S=30)) 112 | print(vpp.api.sflow_polling_interval_get()) 113 | print(vpp.api.sflow_header_bytes_set(header_B=96)) 114 | print(vpp.api.sflow_header_bytes_get()) 115 | 116 | print(vpp.api.sflow_enable_disable(hw_if_index=1, enable_disable=True)) 117 | print(vpp.api.sflow_enable_disable(hw_if_index=2, enable_disable=True)) 118 | print(vpp.api.sflow_interface_dump()) # Both interfaces 119 | print(vpp.api.sflow_interface_dump(hw_if_index=2)) # Single interface 120 | print(vpp.api.sflow_interface_dump(hw_if_index=1234)) # Non-existent 121 | 122 | print(vpp.api.sflow_enable_disable(hw_if_index=1, enable_disable=False)) 123 | print(vpp.api.sflow_interface_dump()) # Only interface 2 124 | print(vpp.api.sflow_enable_disable(hw_if_index=2, enable_disable=False)) 125 | print(vpp.api.sflow_interface_dump()) # No interfaces 126 | ``` 127 | -------------------------------------------------------------------------------- /sflow/CMakeLists.txt: -------------------------------------------------------------------------------- 1 | 2 | # Copyright (c) 2024 InMon Corp. 3 | # Licensed under the Apache License, Version 2.0 (the "License"); 4 | # you may not use this file except in compliance with the License. 5 | # You may obtain a copy of the License at: 6 | # 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | # 9 | # Unless required by applicable law or agreed to in writing, software 10 | # distributed under the License is distributed on an "AS IS" BASIS, 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | # See the License for the specific language governing permissions and 13 | # limitations under the License. 14 | 15 | if ("${CMAKE_SYSTEM_NAME}" STREQUAL "FreeBSD") 16 | message(WARNING "sflow is not supported on FreeBSD - sflow plugin disabled") 17 | return() 18 | endif() 19 | 20 | add_vpp_plugin(sflow 21 | SOURCES 22 | sflow.c 23 | node.c 24 | sflow_common.h 25 | sflow.h 26 | sflow_dlapi.h 27 | sflow_psample.c 28 | sflow_psample.h 29 | sflow_psample_fields.h 30 | sflow_usersock.c 31 | sflow_usersock.h 32 | sflow_dropmon.h 33 | sflow_dropmon.c 34 | sflow_netlink.c 35 | sflow_netlink.h 36 | 37 | MULTIARCH_SOURCES 38 | node.c 39 | 40 | API_FILES 41 | sflow.api 42 | 43 | API_TEST_SOURCES 44 | sflow_test.c 45 | 46 | LINK_LIBRARIES 47 | vppapiclient 48 | vapiclient 49 | ) 50 | -------------------------------------------------------------------------------- /sflow/FEATURE.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | name: sFlow 3 | maintainer: Neil McKee 4 | 5 | description: |- 6 | This plugin implements the random packet-sampling and interface 7 | telemetry streaming required to support standard sFlow export 8 | on Linux platforms. The overhead incurred by this monitoring is 9 | minimal, so that detailed, real-time traffic analysis can be 10 | achieved even under high load conditions, with visibility into 11 | any fields that appear in the packet headers. If the linux-cp 12 | plugin is running then interfaces will be mapped to their 13 | equivalent Linux tap ports. 14 | 15 | state: experimental 16 | properties: [CLI, MULTITHREAD] 17 | -------------------------------------------------------------------------------- /sflow/node.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2024 InMon Corp. 3 | * Licensed under the Apache License, Version 2.0 (the "License"); 4 | * you may not use this file except in compliance with the License. 5 | * You may obtain a copy of the License at: 6 | * 7 | * http://www.apache.org/licenses/LICENSE-2.0 8 | * 9 | * Unless required by applicable law or agreed to in writing, software 10 | * distributed under the License is distributed on an "AS IS" BASIS, 11 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | * See the License for the specific language governing permissions and 13 | * limitations under the License. 14 | */ 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | 22 | typedef struct 23 | { 24 | u32 next_index; 25 | u32 sw_if_index; 26 | u8 new_src_mac[6]; 27 | u8 new_dst_mac[6]; 28 | } sflow_trace_t; 29 | 30 | #ifndef CLIB_MARCH_VARIANT 31 | static u8 * 32 | my_format_mac_address (u8 *s, va_list *args) 33 | { 34 | u8 *a = va_arg (*args, u8 *); 35 | return format (s, "%02x:%02x:%02x:%02x:%02x:%02x", a[0], a[1], a[2], a[3], 36 | a[4], a[5]); 37 | } 38 | 39 | /* packet trace format function */ 40 | static u8 * 41 | format_sflow_trace (u8 *s, va_list *args) 42 | { 43 | CLIB_UNUSED (vlib_main_t * vm) = va_arg (*args, vlib_main_t *); 44 | CLIB_UNUSED (vlib_node_t * node) = va_arg (*args, vlib_node_t *); 45 | sflow_trace_t *t = va_arg (*args, sflow_trace_t *); 46 | 47 | s = format (s, "SFLOW: sw_if_index %d, next index %d\n", t->sw_if_index, 48 | t->next_index); 49 | s = format (s, " src %U -> dst %U", my_format_mac_address, t->new_src_mac, 50 | my_format_mac_address, t->new_dst_mac); 51 | return s; 52 | } 53 | 54 | vlib_node_registration_t sflow_node; 55 | vlib_node_registration_t sflow_egress_node; 56 | vlib_node_registration_t sflow_drop_node; 57 | 58 | #endif /* CLIB_MARCH_VARIANT */ 59 | 60 | #ifndef CLIB_MARCH_VARIANT 61 | static char *sflow_error_strings[] = { 62 | #define _(sym, string) string, 63 | foreach_sflow_error 64 | #undef _ 65 | }; 66 | #endif /* CLIB_MARCH_VARIANT */ 67 | 68 | typedef enum 69 | { 70 | SFLOW_NEXT_ETHERNET_INPUT_OR_INTERFACE_OUTPUT, 71 | SFLOW_N_NEXT, 72 | } sflow_next_t; 73 | 74 | static_always_inline uword 75 | sflow_node_ingress_egress (vlib_main_t *vm, vlib_node_runtime_t *node, 76 | vlib_frame_t *frame, 77 | sflow_enum_sample_t sample_type) 78 | { 79 | u32 n_left_from, *from, *to_next; 80 | sflow_next_t next_index; 81 | 82 | sflow_main_t *smp = &sflow_main; 83 | from = vlib_frame_vector_args (frame); 84 | n_left_from = frame->n_vectors; 85 | 86 | uword thread_index = os_get_thread_index (); 87 | sflow_per_thread_data_t *sfwk = 88 | vec_elt_at_index (smp->per_thread_data, thread_index); 89 | 90 | /* note that sfwk->skip==1 means "take the next packet", 91 | so we never see sfwk->skip==0. */ 92 | 93 | u32 pkts = n_left_from; 94 | if (PREDICT_TRUE (sfwk->skip > pkts)) 95 | { 96 | /* skip the whole frame-vector */ 97 | sfwk->skip -= pkts; 98 | sfwk->pool += pkts; 99 | } 100 | else 101 | { 102 | while (pkts >= sfwk->skip) 103 | { 104 | /* reach in to get the one we want. */ 105 | vlib_buffer_t *bN = vlib_get_buffer (vm, from[sfwk->skip - 1]); 106 | 107 | /* Sample this packet header. */ 108 | u32 hdr = bN->current_length; 109 | if (hdr > smp->headerB) 110 | hdr = smp->headerB; 111 | 112 | ethernet_header_t *en = vlib_buffer_get_current (bN); 113 | u32 if_index = vnet_buffer (bN)->sw_if_index[VLIB_RX]; 114 | u32 if_index_out = 0; 115 | vnet_hw_interface_t *hw = 116 | vnet_get_sup_hw_interface (smp->vnet_main, if_index); 117 | if (hw) 118 | if_index = hw->hw_if_index; 119 | else 120 | { 121 | // TODO: can we get interfaces that have no hw interface? 122 | // If so, should we ignore the sample? 123 | } 124 | 125 | if (sample_type == SFLOW_SAMPLETYPE_EGRESS) 126 | { 127 | if_index_out = vnet_buffer (bN)->sw_if_index[VLIB_TX]; 128 | vnet_hw_interface_t *hw_out = 129 | vnet_get_sup_hw_interface (smp->vnet_main, if_index_out); 130 | if (hw_out) 131 | if_index_out = hw_out->hw_if_index; 132 | } 133 | 134 | sflow_sample_t sample = { 135 | .sample_type = sample_type, 136 | .samplingN = sfwk->smpN, 137 | .input_if_index = if_index, 138 | .output_if_index = if_index_out, 139 | .sampled_packet_size = 140 | bN->current_length + bN->total_length_not_including_first_buffer, 141 | .header_bytes = hdr 142 | }; 143 | 144 | // TODO: what bit in the buffer can we set right here to indicate 145 | // that this packet was sampled (and perhaps another bit to say if it 146 | // was dropped or sucessfully enqueued)? That way we can check it 147 | // below if the packet is traced, and indicate that in the trace 148 | // output. 149 | 150 | // TODO: we end up copying the header twice here. Consider allowing 151 | // the enqueue to be just a little more complex. Like this: 152 | // if(!sflow_fifo_enqueue(&sfwk->fifo, &sample, en, hdr). 153 | // With headerB==128 that would be memcpy(,,24) plus memcpy(,,128) 154 | // instead of the memcpy(,,128) plus memcpy(,,24+256) that we do 155 | // here. (We also know that it could be done as a multiple of 8 156 | // (aligned) bytes because the sflow_sample_t fields are (6xu32) and 157 | // the headerB setting is quantized to the nearest 32 bytes, so there 158 | // may be ways to make it even easier for the compiler.) 159 | sfwk->smpl++; 160 | memcpy (sample.header, en, hdr); 161 | if (PREDICT_FALSE (!sflow_fifo_enqueue (&sfwk->fifo, &sample))) 162 | sfwk->drop++; 163 | 164 | pkts -= sfwk->skip; 165 | sfwk->pool += sfwk->skip; 166 | sfwk->skip = sflow_next_random_skip (sfwk); 167 | } 168 | /* We took a sample (or several) from this frame-vector, but now we are 169 | skipping the rest. */ 170 | sfwk->skip -= pkts; 171 | sfwk->pool += pkts; 172 | } 173 | 174 | /* the rest of this is boilerplate code just to make sure 175 | * that packets are passed on the same way as they would 176 | * have been if this node were not enabled. 177 | * TODO: If there is ever a way to do this in one step 178 | * (i.e. pass on the whole frame-vector unchanged) then it 179 | * might help performance. 180 | */ 181 | 182 | next_index = node->cached_next_index; 183 | 184 | while (n_left_from > 0) 185 | { 186 | u32 n_left_to_next; 187 | 188 | vlib_get_next_frame (vm, node, next_index, to_next, n_left_to_next); 189 | 190 | while (n_left_from >= 8 && n_left_to_next >= 4) 191 | { 192 | u32 next0 = SFLOW_NEXT_ETHERNET_INPUT_OR_INTERFACE_OUTPUT; 193 | u32 next1 = SFLOW_NEXT_ETHERNET_INPUT_OR_INTERFACE_OUTPUT; 194 | u32 next2 = SFLOW_NEXT_ETHERNET_INPUT_OR_INTERFACE_OUTPUT; 195 | u32 next3 = SFLOW_NEXT_ETHERNET_INPUT_OR_INTERFACE_OUTPUT; 196 | ethernet_header_t *en0, *en1, *en2, *en3; 197 | u32 bi0, bi1, bi2, bi3; 198 | vlib_buffer_t *b0, *b1, *b2, *b3; 199 | 200 | /* Prefetch next iteration. */ 201 | { 202 | vlib_buffer_t *p4, *p5, *p6, *p7; 203 | 204 | p4 = vlib_get_buffer (vm, from[4]); 205 | p5 = vlib_get_buffer (vm, from[5]); 206 | p6 = vlib_get_buffer (vm, from[6]); 207 | p7 = vlib_get_buffer (vm, from[7]); 208 | 209 | vlib_prefetch_buffer_header (p4, LOAD); 210 | vlib_prefetch_buffer_header (p5, LOAD); 211 | vlib_prefetch_buffer_header (p6, LOAD); 212 | vlib_prefetch_buffer_header (p7, LOAD); 213 | 214 | CLIB_PREFETCH (p4->data, CLIB_CACHE_LINE_BYTES, STORE); 215 | CLIB_PREFETCH (p5->data, CLIB_CACHE_LINE_BYTES, STORE); 216 | CLIB_PREFETCH (p6->data, CLIB_CACHE_LINE_BYTES, STORE); 217 | CLIB_PREFETCH (p7->data, CLIB_CACHE_LINE_BYTES, STORE); 218 | } 219 | 220 | /* speculatively enqueue b0-b3 to the current next frame */ 221 | to_next[0] = bi0 = from[0]; 222 | to_next[1] = bi1 = from[1]; 223 | to_next[2] = bi2 = from[2]; 224 | to_next[3] = bi3 = from[3]; 225 | from += 4; 226 | to_next += 4; 227 | n_left_from -= 4; 228 | n_left_to_next -= 4; 229 | 230 | b0 = vlib_get_buffer (vm, bi0); 231 | b1 = vlib_get_buffer (vm, bi1); 232 | b2 = vlib_get_buffer (vm, bi2); 233 | b3 = vlib_get_buffer (vm, bi3); 234 | 235 | /* do this to always pass on to the next node on feature arc */ 236 | vnet_feature_next (&next0, b0); 237 | vnet_feature_next (&next1, b1); 238 | vnet_feature_next (&next2, b2); 239 | vnet_feature_next (&next3, b3); 240 | 241 | ASSERT (b0->current_data == 0); 242 | ASSERT (b1->current_data == 0); 243 | ASSERT (b2->current_data == 0); 244 | ASSERT (b3->current_data == 0); 245 | 246 | en0 = vlib_buffer_get_current (b0); 247 | en1 = vlib_buffer_get_current (b1); 248 | en2 = vlib_buffer_get_current (b2); 249 | en3 = vlib_buffer_get_current (b3); 250 | 251 | if (PREDICT_FALSE (b0->flags & VLIB_BUFFER_IS_TRACED)) 252 | { 253 | sflow_trace_t *t = vlib_add_trace (vm, node, b0, sizeof (*t)); 254 | t->sw_if_index = vnet_buffer (b0)->sw_if_index[VLIB_RX]; 255 | t->next_index = next0; 256 | clib_memcpy (t->new_src_mac, en0->src_address, 257 | sizeof (t->new_src_mac)); 258 | clib_memcpy (t->new_dst_mac, en0->dst_address, 259 | sizeof (t->new_dst_mac)); 260 | } 261 | 262 | if (PREDICT_FALSE (b1->flags & VLIB_BUFFER_IS_TRACED)) 263 | { 264 | sflow_trace_t *t = vlib_add_trace (vm, node, b1, sizeof (*t)); 265 | t->sw_if_index = vnet_buffer (b1)->sw_if_index[VLIB_RX]; 266 | t->next_index = next1; 267 | clib_memcpy (t->new_src_mac, en1->src_address, 268 | sizeof (t->new_src_mac)); 269 | clib_memcpy (t->new_dst_mac, en1->dst_address, 270 | sizeof (t->new_dst_mac)); 271 | } 272 | 273 | if (PREDICT_FALSE (b2->flags & VLIB_BUFFER_IS_TRACED)) 274 | { 275 | sflow_trace_t *t = vlib_add_trace (vm, node, b2, sizeof (*t)); 276 | t->sw_if_index = vnet_buffer (b2)->sw_if_index[VLIB_RX]; 277 | t->next_index = next2; 278 | clib_memcpy (t->new_src_mac, en2->src_address, 279 | sizeof (t->new_src_mac)); 280 | clib_memcpy (t->new_dst_mac, en2->dst_address, 281 | sizeof (t->new_dst_mac)); 282 | } 283 | 284 | if (PREDICT_FALSE (b3->flags & VLIB_BUFFER_IS_TRACED)) 285 | { 286 | sflow_trace_t *t = vlib_add_trace (vm, node, b3, sizeof (*t)); 287 | t->sw_if_index = vnet_buffer (b3)->sw_if_index[VLIB_RX]; 288 | t->next_index = next3; 289 | clib_memcpy (t->new_src_mac, en3->src_address, 290 | sizeof (t->new_src_mac)); 291 | clib_memcpy (t->new_dst_mac, en3->dst_address, 292 | sizeof (t->new_dst_mac)); 293 | } 294 | 295 | /* verify speculative enqueues, maybe switch current next frame */ 296 | vlib_validate_buffer_enqueue_x4 (vm, node, next_index, to_next, 297 | n_left_to_next, bi0, bi1, bi2, bi3, 298 | next0, next1, next2, next3); 299 | } 300 | 301 | while (n_left_from > 0 && n_left_to_next > 0) 302 | { 303 | u32 bi0; 304 | vlib_buffer_t *b0; 305 | u32 next0 = SFLOW_NEXT_ETHERNET_INPUT_OR_INTERFACE_OUTPUT; 306 | ethernet_header_t *en0; 307 | 308 | /* speculatively enqueue b0 to the current next frame */ 309 | bi0 = from[0]; 310 | to_next[0] = bi0; 311 | from += 1; 312 | to_next += 1; 313 | n_left_from -= 1; 314 | n_left_to_next -= 1; 315 | 316 | b0 = vlib_get_buffer (vm, bi0); 317 | 318 | /* do this to always pass on to the next node on feature arc */ 319 | vnet_feature_next (&next0, b0); 320 | 321 | /* 322 | * Direct from the driver, we should be at offset 0 323 | * aka at &b0->data[0] 324 | */ 325 | ASSERT (b0->current_data == 0); 326 | 327 | en0 = vlib_buffer_get_current (b0); 328 | 329 | if (PREDICT_FALSE (b0->flags & VLIB_BUFFER_IS_TRACED)) 330 | { 331 | sflow_trace_t *t = vlib_add_trace (vm, node, b0, sizeof (*t)); 332 | t->sw_if_index = vnet_buffer (b0)->sw_if_index[VLIB_RX]; 333 | t->next_index = next0; 334 | clib_memcpy (t->new_src_mac, en0->src_address, 335 | sizeof (t->new_src_mac)); 336 | clib_memcpy (t->new_dst_mac, en0->dst_address, 337 | sizeof (t->new_dst_mac)); 338 | } 339 | 340 | /* verify speculative enqueue, maybe switch current next frame */ 341 | vlib_validate_buffer_enqueue_x1 (vm, node, next_index, to_next, 342 | n_left_to_next, bi0, next0); 343 | } 344 | 345 | vlib_put_next_frame (vm, node, next_index, n_left_to_next); 346 | } 347 | return frame->n_vectors; 348 | } 349 | 350 | VLIB_NODE_FN (sflow_node) 351 | (vlib_main_t *vm, vlib_node_runtime_t *node, vlib_frame_t *frame) 352 | { 353 | return sflow_node_ingress_egress (vm, node, frame, SFLOW_SAMPLETYPE_INGRESS); 354 | } 355 | 356 | VLIB_NODE_FN (sflow_egress_node) 357 | (vlib_main_t *vm, vlib_node_runtime_t *node, vlib_frame_t *frame) 358 | { 359 | return sflow_node_ingress_egress (vm, node, frame, SFLOW_SAMPLETYPE_EGRESS); 360 | } 361 | 362 | typedef enum 363 | { 364 | SFLOW_DROP_NEXT_DROP, 365 | SFLOW_DROP_N_NEXT, 366 | } sflow_drop_next_t; 367 | 368 | static_always_inline void 369 | buffer_rewind_current (vlib_buffer_t *bN) 370 | { 371 | /* 372 | * Typically, we'll need to rewind the buffer 373 | * if l2_hdr_offset is valid, make sure to rewind to the start of 374 | * the L2 header. This may not be the buffer start in case we pop-ed 375 | * vlan tags. 376 | * Otherwise, rewind to buffer start and hope for the best. 377 | */ 378 | /* 379 | * If the packet was rewritten the start may be somewhere 380 | * in buffer->pre_data, which comes before buffer->data. In 381 | * other words, the buffer->current_data index can be negative. 382 | */ 383 | if (bN->flags & VNET_BUFFER_F_L2_HDR_OFFSET_VALID) 384 | { 385 | if (bN->current_data > vnet_buffer (bN)->l2_hdr_offset) 386 | vlib_buffer_advance (bN, vnet_buffer (bN)->l2_hdr_offset - 387 | bN->current_data); 388 | } 389 | else if (bN->current_data > 0) 390 | { 391 | vlib_buffer_advance (bN, (word) -bN->current_data); 392 | } 393 | } 394 | 395 | VLIB_NODE_FN (sflow_drop_node) 396 | (vlib_main_t *vm, vlib_node_runtime_t *node, vlib_frame_t *frame) 397 | { 398 | u32 n_left_from, *from, *to_next, n_left_to_next; 399 | sflow_drop_next_t next_index; 400 | 401 | from = vlib_frame_vector_args (frame); 402 | n_left_from = frame->n_vectors; 403 | 404 | sflow_main_t *smp = &sflow_main; 405 | uword thread_index = os_get_thread_index (); 406 | sflow_per_thread_data_t *sfwk = 407 | vec_elt_at_index (smp->per_thread_data, thread_index); 408 | 409 | for (u32 pkt = n_left_from; pkt > 0; --pkt) 410 | { 411 | vlib_buffer_t *bN = vlib_get_buffer (vm, from[pkt - 1]); 412 | buffer_rewind_current (bN); 413 | // drops are subject to header_bytes limit too 414 | u32 hdr = bN->current_length; 415 | if (hdr > smp->headerB) 416 | hdr = smp->headerB; 417 | ethernet_header_t *en = vlib_buffer_get_current (bN); 418 | // Where did this packet come in originally? 419 | // (Doesn't have to be known) 420 | u32 if_index = vnet_buffer (bN)->sw_if_index[VLIB_RX]; 421 | if (if_index) 422 | { 423 | vnet_hw_interface_t *hw = 424 | vnet_get_sup_hw_interface (smp->vnet_main, if_index); 425 | if (hw) 426 | if_index = hw->hw_if_index; 427 | } 428 | // queue the discard sample for the main thread 429 | sflow_sample_t discard = { .sample_type = SFLOW_SAMPLETYPE_DISCARD, 430 | .input_if_index = if_index, 431 | .sampled_packet_size = 432 | bN->current_length + 433 | bN->total_length_not_including_first_buffer, 434 | .header_bytes = hdr, 435 | // .header_protocol = 0, 436 | .drop_reason = bN->error }; 437 | sfwk->dsmp++; // drop-samples 438 | memcpy (discard.header, en, hdr); 439 | if (PREDICT_FALSE ( 440 | !sflow_drop_fifo_enqueue (&sfwk->drop_fifo, &discard))) 441 | sfwk->ddrp++; // drop-sample drops 442 | } 443 | 444 | /* the rest of this is boilerplate code to pass packets on - typically to 445 | "drop" */ 446 | /* TODO: put back tracing code? */ 447 | /* TODO: process 2 or 4 at a time? */ 448 | /* TODO: by using this variant of the pipeline are we assuming that 449 | we are in a feature arc where frames are not converging or dividing? Just 450 | processing through a linear list of nodes that will each pass the whole 451 | frame of buffers on unchanged ("lighting fools the way to dusty death"). 452 | And if so, how do we make that assumption explicit? 453 | To improve the flexibility would we have to go back and change the way 454 | that interface_output.c (error-drop) launches the frame along the arc 455 | in the first place? 456 | */ 457 | next_index = node->cached_next_index; 458 | while (n_left_from > 0) 459 | { 460 | vlib_get_next_frame (vm, node, next_index, to_next, n_left_to_next); 461 | while (n_left_from > 0 && n_left_to_next > 0) 462 | { 463 | u32 bi0; 464 | vlib_buffer_t *b0; 465 | u32 next0 = SFLOW_DROP_NEXT_DROP; 466 | /* enqueue b0 to the current next frame */ 467 | bi0 = from[0]; 468 | to_next[0] = bi0; 469 | from += 1; 470 | to_next += 1; 471 | n_left_from -= 1; 472 | n_left_to_next -= 1; 473 | b0 = vlib_get_buffer (vm, bi0); 474 | /* do this to always pass on to the next node on feature arc */ 475 | vnet_feature_next (&next0, b0); 476 | } 477 | vlib_put_next_frame (vm, node, next_index, n_left_to_next); 478 | } 479 | return frame->n_vectors; 480 | } 481 | 482 | #ifndef CLIB_MARCH_VARIANT 483 | VLIB_REGISTER_NODE (sflow_node) = 484 | { 485 | .name = "sflow", 486 | .vector_size = sizeof (u32), 487 | .format_trace = format_sflow_trace, 488 | .type = VLIB_NODE_TYPE_INTERNAL, 489 | .n_errors = ARRAY_LEN(sflow_error_strings), 490 | .error_strings = sflow_error_strings, 491 | .n_next_nodes = SFLOW_N_NEXT, 492 | /* edit / add dispositions here */ 493 | .next_nodes = { 494 | [SFLOW_NEXT_ETHERNET_INPUT_OR_INTERFACE_OUTPUT] = "ethernet-input", 495 | }, 496 | }; 497 | 498 | VLIB_REGISTER_NODE (sflow_egress_node) = 499 | { 500 | .name = "sflow-egress", 501 | .vector_size = sizeof (u32), 502 | .format_trace = format_sflow_trace, 503 | .type = VLIB_NODE_TYPE_INTERNAL, 504 | .n_errors = ARRAY_LEN(sflow_error_strings), 505 | .error_strings = sflow_error_strings, 506 | .n_next_nodes = SFLOW_N_NEXT, 507 | /* edit / add dispositions here */ 508 | .next_nodes = { 509 | [SFLOW_NEXT_ETHERNET_INPUT_OR_INTERFACE_OUTPUT] = "interface-output", 510 | }, 511 | }; 512 | 513 | VLIB_REGISTER_NODE (sflow_drop_node) = 514 | { 515 | .name = "sflow-drop", 516 | .vector_size = sizeof (u32), 517 | .format_trace = format_sflow_trace, 518 | .type = VLIB_NODE_TYPE_INTERNAL, 519 | .n_errors = ARRAY_LEN(sflow_error_strings), 520 | .error_strings = sflow_error_strings, 521 | .n_next_nodes = SFLOW_DROP_N_NEXT, 522 | /* edit / add dispositions here */ 523 | .next_nodes = { 524 | //[SFLOW_DROP_NEXT_DROP] = "error-drop", 525 | [SFLOW_DROP_NEXT_DROP] = "drop", 526 | }, 527 | }; 528 | #endif /* CLIB_MARCH_VARIANT */ 529 | /* 530 | * fd.io coding-style-patch-verification: ON 531 | * 532 | * Local Variables: 533 | * eval: (c-set-style "gnu") 534 | * End: 535 | */ 536 | -------------------------------------------------------------------------------- /sflow/sflow.api: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2024 InMon Corp. 3 | * Licensed under the Apache License, Version 2.0 (the "License"); 4 | * you may not use this file except in compliance with the License. 5 | * You may obtain a copy of the License at: 6 | * 7 | * http://www.apache.org/licenses/LICENSE-2.0 8 | * 9 | * Unless required by applicable law or agreed to in writing, software 10 | * distributed under the License is distributed on an "AS IS" BASIS, 11 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | * See the License for the specific language governing permissions and 13 | * limitations under the License. 14 | */ 15 | 16 | /** 17 | * @file sflow.api 18 | * @brief VPP control-plane API messages. 19 | * 20 | * This file defines VPP control-plane binary API messages which are generally 21 | * called through a shared memory interface. 22 | */ 23 | 24 | /* Version and type recitations */ 25 | 26 | option version = "0.1.0"; 27 | import "vnet/interface_types.api"; 28 | 29 | 30 | /** @brief API to enable / disable sflow 31 | @param client_index - opaque cookie to identify the sender 32 | @param context - sender context, to match reply w/ request 33 | @param enable_disable - 1 to enable, 0 to disable the feature 34 | @param hw_if_index - hardware interface handle 35 | */ 36 | 37 | autoreply define sflow_enable_disable { 38 | /* Client identifier, set from api_main.my_client_index */ 39 | u32 client_index; 40 | 41 | /* Arbitrary context, so client can match reply to request */ 42 | u32 context; 43 | 44 | /* Enable / disable the feature */ 45 | bool enable_disable; 46 | 47 | /* Interface handle */ 48 | vl_api_interface_index_t hw_if_index; 49 | }; 50 | 51 | /** @brief API to get sflow sampling-rate 52 | @param client_index - opaque cookie to identify the sender 53 | @param context - sender context, to match reply w/ request 54 | */ 55 | 56 | define sflow_sampling_rate_get { 57 | /* Client identifier, set from api_main.my_client_index */ 58 | u32 client_index; 59 | 60 | /* Arbitrary context, so client can match reply to request */ 61 | u32 context; 62 | }; 63 | 64 | /** \brief reply to get the sflow sampling-rate 65 | @param client_index - opaque cookie to identify the sender 66 | @param context - sender context, to match reply w/ request 67 | @param sampling_N - the current 1-in-N sampling rate 68 | */ 69 | 70 | define sflow_sampling_rate_get_reply 71 | { 72 | u32 context; 73 | u32 sampling_N; 74 | option in_progress; 75 | }; 76 | 77 | /** @brief API to set sflow sampling-rate 78 | @param client_index - opaque cookie to identify the sender 79 | @param context - sender context, to match reply w/ request 80 | @param sampling_N - 1-in-N random sampling rate 81 | */ 82 | 83 | autoreply define sflow_sampling_rate_set { 84 | /* Client identifier, set from api_main.my_client_index */ 85 | u32 client_index; 86 | 87 | /* Arbitrary context, so client can match reply to request */ 88 | u32 context; 89 | 90 | /* Sampling_N */ 91 | u32 sampling_N [default=10000]; 92 | }; 93 | 94 | /** @brief API to set sflow polling-interval 95 | @param client_index - opaque cookie to identify the sender 96 | @param context - sender context, to match reply w/ request 97 | @param polling_S - polling interval in seconds 98 | */ 99 | 100 | autoreply define sflow_polling_interval_set { 101 | /* Client identifier, set from api_main.my_client_index */ 102 | u32 client_index; 103 | 104 | /* Arbitrary context, so client can match reply to request */ 105 | u32 context; 106 | 107 | /* Polling_S */ 108 | u32 polling_S [default=20]; 109 | }; 110 | 111 | /** @brief API to get sflow polling-interval 112 | @param client_index - opaque cookie to identify the sender 113 | @param context - sender context, to match reply w/ request 114 | */ 115 | 116 | define sflow_polling_interval_get { 117 | /* Client identifier, set from api_main.my_client_index */ 118 | u32 client_index; 119 | 120 | /* Arbitrary context, so client can match reply to request */ 121 | u32 context; 122 | }; 123 | 124 | /** \brief reply to get the sflow polling-interval 125 | @param client_index - opaque cookie to identify the sender 126 | @param context - sender context, to match reply w/ request 127 | @param polling_S - current polling interval in seconds 128 | */ 129 | 130 | define sflow_polling_interval_get_reply 131 | { 132 | u32 context; 133 | u32 polling_S; 134 | option in_progress; 135 | }; 136 | 137 | /** @brief API to set sflow header-bytes 138 | @param client_index - opaque cookie to identify the sender 139 | @param context - sender context, to match reply w/ request 140 | @param header_B - max header length in bytes 141 | */ 142 | 143 | autoreply define sflow_header_bytes_set { 144 | /* Client identifier, set from api_main.my_client_index */ 145 | u32 client_index; 146 | 147 | /* Arbitrary context, so client can match reply to request */ 148 | u32 context; 149 | 150 | /* header_B */ 151 | u32 header_B [default=128]; 152 | }; 153 | 154 | /** @brief API to get sflow header-bytes 155 | @param client_index - opaque cookie to identify the sender 156 | @param context - sender context, to match reply w/ request 157 | */ 158 | 159 | define sflow_header_bytes_get { 160 | /* Client identifier, set from api_main.my_client_index */ 161 | u32 client_index; 162 | 163 | /* Arbitrary context, so client can match reply to request */ 164 | u32 context; 165 | }; 166 | 167 | /** \brief reply to get the sflow header-bytes 168 | @param client_index - opaque cookie to identify the sender 169 | @param context - sender context, to match reply w/ request 170 | @param header_B - current maximum header length in bytes 171 | */ 172 | 173 | define sflow_header_bytes_get_reply 174 | { 175 | u32 context; 176 | u32 header_B; 177 | option in_progress; 178 | }; 179 | 180 | /** @brief API to set sflow direction 181 | @param client_index - opaque cookie to identify the sender 182 | @param context - sender context, to match reply w/ request 183 | @param sampling_D - direction 184 | */ 185 | 186 | autoreply define sflow_direction_set { 187 | /* Client identifier, set from api_main.my_client_index */ 188 | u32 client_index; 189 | 190 | /* Arbitrary context, so client can match reply to request */ 191 | u32 context; 192 | 193 | /* sampling_D */ 194 | u32 sampling_D; 195 | }; 196 | 197 | /** @brief API to get sflow direction 198 | @param client_index - opaque cookie to identify the sender 199 | @param context - sender context, to match reply w/ request 200 | */ 201 | 202 | define sflow_direction_get { 203 | /* Client identifier, set from api_main.my_client_index */ 204 | u32 client_index; 205 | 206 | /* Arbitrary context, so client can match reply to request */ 207 | u32 context; 208 | }; 209 | 210 | /** \brief reply to get the sflow direction 211 | @param client_index - opaque cookie to identify the sender 212 | @param context - sender context, to match reply w/ request 213 | @param sampling_D - direction 214 | */ 215 | 216 | define sflow_direction_get_reply 217 | { 218 | u32 context; 219 | u32 sampling_D; 220 | option in_progress; 221 | }; 222 | 223 | /** @brief API to set sflow drop-monitoring 224 | @param client_index - opaque cookie to identify the sender 225 | @param context - sender context, to match reply w/ request 226 | @param drop_M - enable drop monitoring 227 | */ 228 | 229 | autoreply define sflow_drop_monitoring_set { 230 | /* Client identifier, set from api_main.my_client_index */ 231 | u32 client_index; 232 | 233 | /* Arbitrary context, so client can match reply to request */ 234 | u32 context; 235 | 236 | /* drop_M */ 237 | u32 drop_M; 238 | }; 239 | 240 | /** @brief API to get sflow drop-monitoring 241 | @param client_index - opaque cookie to identify the sender 242 | @param context - sender context, to match reply w/ request 243 | */ 244 | 245 | define sflow_drop_monitoring_get { 246 | /* Client identifier, set from api_main.my_client_index */ 247 | u32 client_index; 248 | 249 | /* Arbitrary context, so client can match reply to request */ 250 | u32 context; 251 | }; 252 | 253 | /** \brief reply to get the sflow drop-monitoring 254 | @param client_index - opaque cookie to identify the sender 255 | @param context - sender context, to match reply w/ request 256 | @param drop_M - is drop monitoring enabled 257 | */ 258 | 259 | define sflow_drop_monitoring_get_reply 260 | { 261 | u32 context; 262 | u32 drop_M; 263 | option in_progress; 264 | }; 265 | 266 | /** \brief Dump sflow enabled interface(s) 267 | @param client_index - opaque cookie to identify the sender 268 | @param hw_if_index - hw_if_index of a specific interface, or -1 (default) 269 | to return all sflow enabled interfaces 270 | */ 271 | define sflow_interface_dump 272 | { 273 | u32 client_index; 274 | u32 context; 275 | vl_api_interface_index_t hw_if_index [default=0xffffffff]; 276 | }; 277 | 278 | /** \brief sflow enabled interface details 279 | */ 280 | define sflow_interface_details 281 | { 282 | u32 context; 283 | vl_api_interface_index_t hw_if_index; 284 | }; 285 | -------------------------------------------------------------------------------- /sflow/sflow.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2024 InMon Corp. 3 | * Licensed under the Apache License, Version 2.0 (the "License"); 4 | * you may not use this file except in compliance with the License. 5 | * You may obtain a copy of the License at: 6 | * 7 | * http://www.apache.org/licenses/LICENSE-2.0 8 | * 9 | * Unless required by applicable law or agreed to in writing, software 10 | * distributed under the License is distributed on an "AS IS" BASIS, 11 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | * See the License for the specific language governing permissions and 13 | * limitations under the License. 14 | */ 15 | 16 | #include 17 | #include 18 | #include 19 | 20 | #include 21 | #include 22 | #include 23 | #include 24 | 25 | #include 26 | #include 27 | #include 28 | 29 | #include 30 | #include 31 | 32 | #define REPLY_MSG_ID_BASE smp->msg_id_base 33 | #include 34 | 35 | sflow_main_t sflow_main; 36 | vlib_log_class_t sflow_logger; 37 | 38 | static void 39 | sflow_stat_segment_client_init (void) 40 | { 41 | stat_client_main_t *scm = &stat_client_main; 42 | vlib_stats_segment_t *sm = vlib_stats_get_segment (); 43 | uword size; 44 | 45 | size = sm->memory_size ? sm->memory_size : STAT_SEGMENT_DEFAULT_SIZE; 46 | scm->memory_size = size; 47 | scm->shared_header = sm->shared_header; 48 | scm->directory_vector = 49 | stat_segment_adjust (scm, (void *) scm->shared_header->directory_vector); 50 | } 51 | 52 | static void 53 | update_counter_vector_simple (stat_segment_data_t *res, 54 | sflow_counters_t *ifCtrs, u32 hw_if_index) 55 | { 56 | for (int th = 0; th < vec_len (res->simple_counter_vec); th++) 57 | { 58 | for (int intf = 0; intf < vec_len (res->simple_counter_vec[th]); intf++) 59 | { 60 | if (intf == hw_if_index) 61 | { 62 | u64 count = res->simple_counter_vec[th][intf]; 63 | if (count) 64 | { 65 | if (strcmp (res->name, "/if/rx-error") == 0) 66 | ifCtrs->rx.errs += count; 67 | else if (strcmp (res->name, "/if/tx-error") == 0) 68 | ifCtrs->tx.errs += count; 69 | else if (strcmp (res->name, "/if/drops") == 0) 70 | ifCtrs->tx.drps += count; 71 | else if (strcmp (res->name, "/if/rx-miss") == 0 || 72 | strcmp (res->name, "/if/rx-no-buf") == 0) 73 | ifCtrs->rx.drps += count; 74 | } 75 | } 76 | } 77 | } 78 | } 79 | 80 | static void 81 | update_counter_vector_combined (stat_segment_data_t *res, 82 | sflow_counters_t *ifCtrs, u32 hw_if_index) 83 | { 84 | for (int th = 0; th < vec_len (res->simple_counter_vec); th++) 85 | { 86 | for (int intf = 0; intf < vec_len (res->combined_counter_vec[th]); 87 | intf++) 88 | { 89 | if (intf == hw_if_index) 90 | { 91 | u64 pkts = res->combined_counter_vec[th][intf].packets; 92 | u64 byts = res->combined_counter_vec[th][intf].bytes; 93 | if (pkts || byts) 94 | { 95 | if (strcmp (res->name, "/if/rx") == 0) 96 | { 97 | ifCtrs->rx.pkts += pkts; 98 | ifCtrs->rx.byts += byts; 99 | } 100 | else if (strcmp (res->name, "/if/tx") == 0) 101 | { 102 | ifCtrs->tx.byts += byts; 103 | ifCtrs->tx.pkts += pkts; 104 | } 105 | // TODO: do multicasts include broadcasts, or are they 106 | // counted separately? (test with traffic) 107 | else if (strcmp (res->name, "/if/rx-multicast") == 0) 108 | ifCtrs->rx.m_pkts += pkts; 109 | else if (strcmp (res->name, "/if/tx-multicast") == 0) 110 | ifCtrs->tx.m_pkts += pkts; 111 | else if (strcmp (res->name, "/if/rx-broadcast") == 0) 112 | ifCtrs->rx.b_pkts += pkts; 113 | else if (strcmp (res->name, "/if/tx-broadcast") == 0) 114 | ifCtrs->tx.b_pkts += pkts; 115 | } 116 | } 117 | } 118 | } 119 | } 120 | 121 | static int 122 | startsWith (u8 *str, char *prefix) 123 | { 124 | if (str && prefix) 125 | { 126 | int len1 = vec_len (str); 127 | int len2 = strlen (prefix); 128 | if (len1 >= len2) 129 | { 130 | return (memcmp (str, prefix, len2) == 0); 131 | } 132 | } 133 | return false; 134 | } 135 | 136 | static void 137 | update_counters (sflow_main_t *smp, sflow_per_interface_data_t *sfif) 138 | { 139 | vnet_sw_interface_t *sw = 140 | vnet_get_sw_interface (smp->vnet_main, sfif->sw_if_index); 141 | vnet_hw_interface_t *hw = 142 | vnet_get_hw_interface (smp->vnet_main, sfif->hw_if_index); 143 | // This gives us a list of stat integers 144 | u32 *stats = stat_segment_ls (NULL); 145 | stat_segment_data_t *res = NULL; 146 | // read vector of stat_segment_data_t objects 147 | retry: 148 | res = stat_segment_dump (stats); 149 | if (res == NULL) 150 | { 151 | /* Memory layout has changed */ 152 | if (stats) 153 | vec_free (stats); 154 | stats = stat_segment_ls (NULL); 155 | goto retry; 156 | } 157 | sflow_counters_t ifCtrs = {}; 158 | // and accumulate the (per-thread) entries for this interface 159 | for (int ii = 0; ii < vec_len (res); ii++) 160 | { 161 | switch (res[ii].type) 162 | { 163 | case STAT_DIR_TYPE_COUNTER_VECTOR_SIMPLE: 164 | update_counter_vector_simple (&res[ii], &ifCtrs, sfif->hw_if_index); 165 | break; 166 | case STAT_DIR_TYPE_COUNTER_VECTOR_COMBINED: 167 | update_counter_vector_combined (&res[ii], &ifCtrs, 168 | sfif->hw_if_index); 169 | break; 170 | case STAT_DIR_TYPE_SCALAR_INDEX: 171 | case STAT_DIR_TYPE_NAME_VECTOR: 172 | case STAT_DIR_TYPE_EMPTY: 173 | default: 174 | break; 175 | } 176 | } 177 | stat_segment_data_free (res); 178 | vec_free (stats); 179 | // send the structure via netlink 180 | SFLOWUS *ust = &smp->sflow_usersock; 181 | SFLOWUS_set_msg_type (ust, SFLOW_VPP_MSG_IF_COUNTERS); 182 | SFLOWUS_set_attr (ust, SFLOW_VPP_ATTR_PORTNAME, hw->name, 183 | vec_len (hw->name)); 184 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_IFINDEX, sfif->sw_if_index); 185 | 186 | if (smp->lcp_itf_pair_get_vif_index_by_phy) 187 | { 188 | sfif->linux_if_index = 189 | (*smp->lcp_itf_pair_get_vif_index_by_phy) (sfif->sw_if_index); 190 | } 191 | 192 | if (sfif->linux_if_index != INDEX_INVALID) 193 | { 194 | // We know the corresponding Linux ifIndex for this interface, so include 195 | // that here. 196 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_OSINDEX, sfif->linux_if_index); 197 | } 198 | 199 | // Report consistent with vpp-snmp-agent 200 | u64 ifSpeed = (hw->link_speed == ~0) ? 0 : ((u64) hw->link_speed * 1000); 201 | if (startsWith (hw->name, "loop") || startsWith (hw->name, "tap")) 202 | ifSpeed = 1e9; 203 | 204 | u32 ifType = startsWith (hw->name, "loop") ? 24 // softwareLoopback 205 | : 206 | 6; // ethernetCsmacd 207 | 208 | u32 ifDirection = (hw->flags & VNET_HW_INTERFACE_FLAG_HALF_DUPLEX) ? 209 | 2 // half-duplex 210 | : 211 | 1; // full-duplex 212 | 213 | u32 operUp = (hw->flags & VNET_HW_INTERFACE_FLAG_LINK_UP) ? 1 : 0; 214 | u32 adminUp = (sw->flags & VNET_SW_INTERFACE_FLAG_ADMIN_UP) ? 1 : 0; 215 | 216 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_IFSPEED, ifSpeed); 217 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_IFTYPE, ifType); 218 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_IFDIRECTION, ifDirection); 219 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_OPER_UP, operUp); 220 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_ADMIN_UP, adminUp); 221 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_RX_OCTETS, ifCtrs.rx.byts); 222 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_TX_OCTETS, ifCtrs.tx.byts); 223 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_RX_PKTS, ifCtrs.rx.pkts); 224 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_TX_PKTS, ifCtrs.tx.pkts); 225 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_RX_MCASTS, ifCtrs.rx.m_pkts); 226 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_TX_MCASTS, ifCtrs.tx.m_pkts); 227 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_RX_BCASTS, ifCtrs.rx.b_pkts); 228 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_TX_BCASTS, ifCtrs.tx.b_pkts); 229 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_RX_ERRORS, ifCtrs.rx.errs); 230 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_TX_ERRORS, ifCtrs.tx.errs); 231 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_RX_DISCARDS, ifCtrs.rx.drps); 232 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_TX_DISCARDS, ifCtrs.tx.drps); 233 | SFLOWUS_set_attr (ust, SFLOW_VPP_ATTR_HW_ADDRESS, hw->hw_address, 234 | vec_len (hw->hw_address)); 235 | smp->unixsock_seq++; 236 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_SEQ, smp->unixsock_seq); 237 | if (SFLOWUS_send (ust) < 0) 238 | smp->csample_send_drops++; 239 | smp->csample_send++; 240 | } 241 | 242 | static u32 243 | total_drops (sflow_main_t *smp) 244 | { 245 | // sum sendmsg and worker-fifo drops 246 | u32 all_drops = smp->psample_send_drops; 247 | for (clib_thread_index_t thread_index = 0; thread_index < smp->total_threads; 248 | thread_index++) 249 | { 250 | sflow_per_thread_data_t *sfwk = 251 | vec_elt_at_index (smp->per_thread_data, thread_index); 252 | all_drops += sfwk->drop; 253 | } 254 | return all_drops; 255 | } 256 | 257 | static void 258 | send_sampling_status_info (sflow_main_t *smp) 259 | { 260 | SFLOWUS *ust = &smp->sflow_usersock; 261 | u32 all_pipeline_drops = total_drops (smp); 262 | SFLOWUS_set_msg_type (ust, SFLOW_VPP_MSG_STATUS); 263 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_UPTIME_S, smp->now_mono_S); 264 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_DROPS, all_pipeline_drops); 265 | ++smp->unixsock_seq; 266 | SFLOWUS_set_attr_int (ust, SFLOW_VPP_ATTR_SEQ, smp->unixsock_seq); 267 | SFLOWUS_send (ust); 268 | } 269 | 270 | static int 271 | counter_polling_check (sflow_main_t *smp) 272 | { 273 | // see if we should poll one or more interfaces 274 | int polled = 0; 275 | for (int ii = 0; ii < vec_len (smp->per_interface_data); ii++) 276 | { 277 | sflow_per_interface_data_t *sfif = 278 | vec_elt_at_index (smp->per_interface_data, ii); 279 | if (sfif && sfif->sflow_enabled && 280 | (sfif->polled == 0 // always send the first time 281 | || (smp->now_mono_S % smp->pollingS) == 282 | (sfif->hw_if_index % smp->pollingS))) 283 | { 284 | update_counters (smp, sfif); 285 | sfif->polled++; 286 | polled++; 287 | } 288 | } 289 | return polled; 290 | } 291 | 292 | static void 293 | lowercase_and_replace_white (char *str, int len, char replace) 294 | { 295 | if (str) 296 | for (int ii = 0; ii < len; ii++) 297 | { 298 | if (isspace (str[ii])) 299 | str[ii] = replace; 300 | else 301 | str[ii] = tolower (str[ii]); 302 | } 303 | } 304 | 305 | static int 306 | compose_trap_str (char *buf, int buf_len, char *str, int str_len) 307 | { 308 | int prefix_len = strlen (SFLOW_TRAP_PREFIX); 309 | int max_cont_len = buf_len - prefix_len - 1; 310 | int cont_len = (str_len > max_cont_len) ? max_cont_len : str_len; 311 | clib_memcpy_fast (buf, SFLOW_TRAP_PREFIX, prefix_len); 312 | clib_memcpy_fast (buf + prefix_len, str, cont_len); 313 | lowercase_and_replace_white (buf + prefix_len, cont_len, SFLOW_TRAP_WHITE); 314 | buf[prefix_len + cont_len] = '\0'; 315 | return prefix_len + cont_len; 316 | } 317 | 318 | static int 319 | send_packet_sample (vlib_main_t *vm, sflow_main_t *smp, sflow_sample_t *sample) 320 | { 321 | if (sample->header_bytes > smp->headerB) 322 | { 323 | // We get here if header-bytes setting is reduced dynamically 324 | // and a sample that was in the FIFO appears with a larger 325 | // header. 326 | return 0; 327 | } 328 | SFLOWPS *pst = &smp->sflow_psample; 329 | u32 ps_group, seqNo; 330 | switch (sample->sample_type) 331 | { 332 | case SFLOW_SAMPLETYPE_INGRESS: 333 | ps_group = SFLOW_VPP_PSAMPLE_GROUP_INGRESS; 334 | seqNo = ++smp->psample_seq_ingress; 335 | break; 336 | case SFLOW_SAMPLETYPE_EGRESS: 337 | ps_group = SFLOW_VPP_PSAMPLE_GROUP_EGRESS; 338 | seqNo = ++smp->psample_seq_egress; 339 | break; 340 | default: 341 | return 0; 342 | } 343 | // TODO: is it always ethernet? (affects ifType counter as well) 344 | u16 header_protocol = 1; /* ethernet */ 345 | SFLOWPS_set_attr_int (pst, SFLOWPS_PSAMPLE_ATTR_SAMPLE_GROUP, ps_group); 346 | SFLOWPS_set_attr_int (pst, SFLOWPS_PSAMPLE_ATTR_IIFINDEX, 347 | sample->input_if_index); 348 | SFLOWPS_set_attr_int (pst, SFLOWPS_PSAMPLE_ATTR_OIFINDEX, 349 | sample->output_if_index); 350 | SFLOWPS_set_attr_int (pst, SFLOWPS_PSAMPLE_ATTR_ORIGSIZE, 351 | sample->sampled_packet_size); 352 | SFLOWPS_set_attr_int (pst, SFLOWPS_PSAMPLE_ATTR_GROUP_SEQ, seqNo); 353 | SFLOWPS_set_attr_int (pst, SFLOWPS_PSAMPLE_ATTR_SAMPLE_RATE, 354 | sample->samplingN); 355 | SFLOWPS_set_attr (pst, SFLOWPS_PSAMPLE_ATTR_DATA, sample->header, 356 | sample->header_bytes); 357 | SFLOWPS_set_attr_int (pst, SFLOWPS_PSAMPLE_ATTR_PROTO, header_protocol); 358 | if (SFLOWPS_send (pst) < 0) 359 | return -1; 360 | return 1; 361 | } 362 | 363 | static int 364 | send_discard_sample (vlib_main_t *vm, sflow_main_t *smp, 365 | sflow_sample_t *sample) 366 | { 367 | SFLOWDM *dmt = &smp->sflow_dropmon; 368 | if (sample->header_bytes > smp->headerB) 369 | { 370 | // We get here if header-bytes setting is reduced dynamically 371 | // and a sample that was in the FIFO appears with a larger 372 | // header. 373 | return 0; 374 | } 375 | if (sample->sample_type != SFLOW_SAMPLETYPE_DISCARD) 376 | { 377 | SFLOW_ERR ("send_discard_sample sample-sample_type=%u", 378 | sample->sample_type); 379 | return 0; 380 | } 381 | vlib_error_main_t *em = &vm->error_main; 382 | if (sample->drop_reason >= vec_len (em->counters_heap)) 383 | return 0; 384 | if (sample->drop_reason >= vec_len (vm->node_main.node_by_error)) 385 | return 0; 386 | u32 err_node_idx = vm->node_main.node_by_error[sample->drop_reason]; 387 | // Are all the ones we want classed as errors, or might some be WARN or INFO? 388 | // if (err->severity == VL_COUNTER_SEVERITY_ERROR) 389 | // set TRAP_GROUP_NAME to "vpp_" 390 | char trap_grp[SFLOW_MAX_TRAP_LEN]; 391 | char trap[SFLOW_MAX_TRAP_LEN]; 392 | vlib_node_t *n = vlib_get_node (vm, err_node_idx); 393 | int trap_grp_len = compose_trap_str (trap_grp, SFLOW_MAX_TRAP_LEN, 394 | (char *) n->name, vec_len (n->name)); 395 | // set TRAP_NAME to "vpp_" 396 | vlib_error_desc_t *err = &em->counters_heap[sample->drop_reason]; 397 | int err_name_len = clib_strnlen (err->name, SFLOW_MAX_TRAP_LEN); 398 | int trap_len = 399 | compose_trap_str (trap, SFLOW_MAX_TRAP_LEN, err->name, err_name_len); 400 | // populate the netlink attributes 401 | u16 origin = NET_DM_ORIGIN_SW; 402 | SFLOWDM_set_attr_int (dmt, NET_DM_ATTR_ORIGIN, origin); 403 | // include NUL termination char in netlink strings. 404 | SFLOWDM_set_attr (dmt, NET_DM_ATTR_HW_TRAP_GROUP_NAME, trap_grp, 405 | trap_grp_len + 1); 406 | SFLOWDM_set_attr (dmt, NET_DM_ATTR_HW_TRAP_NAME, trap, trap_len + 1); 407 | SFLOWDM_set_attr_int (dmt, NET_DM_ATTR_ORIG_LEN, 408 | sample->sampled_packet_size); 409 | SFLOWDM_set_attr_int (dmt, NET_DM_ATTR_TRUNC_LEN, sample->header_bytes); 410 | // TODO: read from header? (really just needs to be non-zero for hsflowd) 411 | u16 proto = 0x0800; 412 | SFLOWDM_set_attr_int (dmt, NET_DM_ATTR_PROTO, proto); 413 | SFLOWDM_set_attr_int (dmt, NET_DM_ATTR_IN_PORT, sample->input_if_index); 414 | SFLOWDM_set_attr (dmt, NET_DM_ATTR_PAYLOAD, sample->header, 415 | sample->header_bytes); 416 | if (SFLOWDM_send (dmt) < 0) 417 | return -1; 418 | return 1; 419 | } 420 | 421 | static u32 422 | read_worker_fifos (vlib_main_t *vm, sflow_main_t *smp) 423 | { 424 | // Our maximum samples/sec is approximately: 425 | // (SFLOW_READ_BATCH * smp->total_threads) / SFLOW_POLL_WAIT_S 426 | // but it may also be affected by SFLOW_FIFO_DEPTH 427 | // and whether vlib_process_wait_for_event_or_clock() really waits for 428 | // SFLOW_POLL_WAIT_S every time. 429 | // If there are too many samples then dropping them as early as possible 430 | // (and as randomly as possible) is preferred, so SFLOW_FIFO_DEPTH should not 431 | // be any bigger than it strictly needs to be. If there is a system 432 | // bottleneck it could be in the PSAMPLE netlink channel, the hsflowd 433 | // encoder, the UDP stack, the network path, the collector, or a faraway 434 | // application. Any kind of "clipping" will result in systematic bias so we 435 | // try to make this fair even when it's running hot. For example, we'll 436 | // round-robin the thread FIFO dequeues here to make sure we give them equal 437 | // access to the PSAMPLE channel. Another factor in sizing SFLOW_FIFO_DEPTH 438 | // is to ensure that we can absorb a short-term line-rate burst without 439 | // dropping samples. This implies a deeper FIFO. In fact it looks like this 440 | // requirement ends up being the dominant one. A value of SFLOW_FIFO_DEPTH 441 | // that will absorb an n-second line-rate burst may well result in the max 442 | // sustainable samples/sec being higher than we really need. But it's not a 443 | // serious problem because the samples are packed into UDP datagrams and the 444 | // network or collector can drop those anywhere they need to. The protocol is 445 | // designed to be tolerant to random packet-loss in transit. For example, 1% 446 | // loss should just make it look like the sampling-rate setting was 1:10100 447 | // instead of 1:10000. 448 | u32 batch = 0; 449 | for (; batch < SFLOW_READ_BATCH; batch++) 450 | { 451 | u32 psample_found = 0, dropmon_found = 0; 452 | u32 psample_send = 0, psample_send_fail = 0; 453 | u32 dropmon_send = 0, dropmon_send_fail = 0; 454 | for (clib_thread_index_t thread_index = 0; 455 | thread_index < smp->total_threads; thread_index++) 456 | { 457 | sflow_per_thread_data_t *sfwk = 458 | vec_elt_at_index (smp->per_thread_data, thread_index); 459 | sflow_sample_t sample; 460 | if (sflow_fifo_dequeue (&sfwk->fifo, &sample)) 461 | { 462 | psample_found++; 463 | int sent = send_packet_sample (vm, smp, &sample); 464 | if (sent == 1) 465 | psample_send++; 466 | if (sent == -1) 467 | psample_send_fail++; 468 | } 469 | if (sflow_drop_fifo_dequeue (&sfwk->drop_fifo, &sample)) 470 | { 471 | dropmon_found++; 472 | int sent = send_discard_sample (vm, smp, &sample); 473 | if (sent == 1) 474 | dropmon_send++; 475 | if (sent == -1) 476 | dropmon_send_fail++; 477 | } 478 | } 479 | if (psample_found == 0 && dropmon_found == 0) 480 | { 481 | // nothing found on FIFOs this time through, so terminate batch early 482 | break; 483 | } 484 | else 485 | { 486 | if (psample_send > 0) 487 | { 488 | vlib_node_increment_counter (smp->vlib_main, sflow_node.index, 489 | SFLOW_ERROR_PSAMPLE_SEND, 490 | psample_send); 491 | smp->psample_send += psample_send; 492 | } 493 | if (psample_send_fail > 0) 494 | { 495 | vlib_node_increment_counter (smp->vlib_main, sflow_node.index, 496 | SFLOW_ERROR_PSAMPLE_SEND_FAIL, 497 | psample_send_fail); 498 | smp->psample_send_drops += psample_send_fail; 499 | } 500 | if (dropmon_send > 0) 501 | { 502 | vlib_node_increment_counter (smp->vlib_main, sflow_node.index, 503 | SFLOW_ERROR_DROPMON_SEND, 504 | dropmon_send); 505 | smp->dropmon_send += dropmon_send; 506 | } 507 | if (dropmon_send_fail > 0) 508 | { 509 | vlib_node_increment_counter (smp->vlib_main, sflow_node.index, 510 | SFLOW_ERROR_DROPMON_SEND_FAIL, 511 | dropmon_send_fail); 512 | smp->dropmon_send_drops += dropmon_send_fail; 513 | } 514 | } 515 | } 516 | return batch; 517 | } 518 | 519 | static void 520 | read_node_counters (sflow_main_t *smp, sflow_err_ctrs_t *ctrs) 521 | { 522 | for (u32 ec = 0; ec < SFLOW_N_ERROR; ec++) 523 | ctrs->counters[ec] = 0; 524 | for (clib_thread_index_t thread_index = 0; thread_index < smp->total_threads; 525 | thread_index++) 526 | { 527 | sflow_per_thread_data_t *sfwk = 528 | vec_elt_at_index (smp->per_thread_data, thread_index); 529 | ctrs->counters[SFLOW_ERROR_PROCESSED] += sfwk->pool; 530 | ctrs->counters[SFLOW_ERROR_SAMPLED] += sfwk->smpl; 531 | ctrs->counters[SFLOW_ERROR_DROPPED] += sfwk->drop; 532 | ctrs->counters[SFLOW_ERROR_DIPROCESSED] += sfwk->dsmp; 533 | ctrs->counters[SFLOW_ERROR_DIDROPPED] += sfwk->ddrp; 534 | } 535 | } 536 | 537 | static void 538 | update_node_cntr (sflow_main_t *smp, sflow_err_ctrs_t *prev, 539 | sflow_err_ctrs_t *latest, sflow_error_t ee) 540 | { 541 | u32 delta = latest->counters[ee] - prev->counters[ee]; 542 | vlib_node_increment_counter (smp->vlib_main, sflow_node.index, ee, delta); 543 | } 544 | 545 | static void 546 | update_node_counters (sflow_main_t *smp, sflow_err_ctrs_t *prev, 547 | sflow_err_ctrs_t *latest) 548 | { 549 | // TODO: is it OK to assess all counters against sflow_node or do we 550 | // need to distinguish sflow_drop_node and sflow_egress_node? 551 | update_node_cntr (smp, prev, latest, SFLOW_ERROR_PROCESSED); 552 | update_node_cntr (smp, prev, latest, SFLOW_ERROR_SAMPLED); 553 | update_node_cntr (smp, prev, latest, SFLOW_ERROR_DROPPED); 554 | update_node_cntr (smp, prev, latest, SFLOW_ERROR_DIPROCESSED); 555 | update_node_cntr (smp, prev, latest, SFLOW_ERROR_DIDROPPED); 556 | *prev = *latest; // latch for next time 557 | } 558 | 559 | static uword 560 | sflow_process_samples (vlib_main_t *vm, vlib_node_runtime_t *node, 561 | vlib_frame_t *frame) 562 | { 563 | sflow_main_t *smp = &sflow_main; 564 | clib_time_t ctm; 565 | clib_time_init (&ctm); 566 | 567 | sflow_err_ctrs_t prev = {}; 568 | read_node_counters (smp, &prev); 569 | 570 | while (1) 571 | { 572 | 573 | // We don't have anything for the main loop to edge-trigger on, so 574 | // we are just asking to be called back regularly. More regularly 575 | // if sFlow is actually enabled... 576 | f64 poll_wait_S = smp->running ? SFLOW_POLL_WAIT_S : 1.0; 577 | vlib_process_wait_for_event_or_clock (vm, poll_wait_S); 578 | if (!smp->running) 579 | { 580 | // Nothing to do. Just yield again. 581 | continue; 582 | } 583 | 584 | // PSAMPLE channel may need extra step (e.g. to learn family_id) 585 | // before it is ready to send 586 | EnumSFLOWNLState psState = SFLOWPS_state (&smp->sflow_psample); 587 | if (psState != SFLOWNL_STATE_READY) 588 | { 589 | SFLOWPS_open_step (&smp->sflow_psample); 590 | } 591 | 592 | // DROPMON channel may need extra step (e.g. to learn family_id) 593 | // before it is ready to send 594 | EnumSFLOWNLState dmState = SFLOWDM_state (&smp->sflow_dropmon); 595 | if (dmState != SFLOWNL_STATE_READY) 596 | { 597 | SFLOWDM_open_step (&smp->sflow_dropmon); 598 | } 599 | 600 | // What we want is a monotonic, per-second clock. This seems to do it 601 | // because it is based on the CPU clock. 602 | f64 tnow = clib_time_now (&ctm); 603 | u32 tnow_S = (u32) tnow; 604 | if (tnow_S != smp->now_mono_S) 605 | { 606 | // second rollover 607 | smp->now_mono_S = tnow_S; 608 | // send status info 609 | send_sampling_status_info (smp); 610 | // poll counters for interfaces that are due 611 | counter_polling_check (smp); 612 | } 613 | // process samples from workers 614 | read_worker_fifos (vm, smp); 615 | 616 | // and sync the global counters 617 | sflow_err_ctrs_t latest = {}; 618 | read_node_counters (smp, &latest); 619 | update_node_counters (smp, &prev, &latest); 620 | } 621 | return 0; 622 | } 623 | 624 | VLIB_REGISTER_NODE (sflow_process_samples_node, static) = { 625 | .function = sflow_process_samples, 626 | .name = "sflow-process-samples", 627 | .type = VLIB_NODE_TYPE_PROCESS, 628 | .process_log2_n_stack_bytes = 17, 629 | }; 630 | 631 | static void 632 | sflow_set_worker_sampling_state (sflow_main_t *smp) 633 | { 634 | /* set up (or reset) sampling context for each thread */ 635 | vlib_thread_main_t *tm = &vlib_thread_main; 636 | smp->total_threads = 1 + tm->n_threads; 637 | vec_validate (smp->per_thread_data, smp->total_threads); 638 | for (clib_thread_index_t thread_index = 0; thread_index < smp->total_threads; 639 | thread_index++) 640 | { 641 | sflow_per_thread_data_t *sfwk = 642 | vec_elt_at_index (smp->per_thread_data, thread_index); 643 | if (sfwk->smpN != smp->samplingN) 644 | { 645 | sfwk->smpN = smp->samplingN; 646 | sfwk->seed = thread_index; 647 | sfwk->skip = sflow_next_random_skip (sfwk); 648 | SFLOW_DBG ( 649 | "sflowset_worker_sampling_state: samplingN=%u thread=%u skip=%u", 650 | smp->samplingN, thread_index, sfwk->skip); 651 | } 652 | } 653 | } 654 | 655 | static void 656 | sflow_sampling_start (sflow_main_t *smp) 657 | { 658 | SFLOW_INFO ("sflow_sampling_start"); 659 | 660 | smp->running = true; 661 | // Reset this clock so that the per-second netlink status updates 662 | // will communicate a restart to hsflowd. This helps to distinguish: 663 | // (1) vpp restarted with sFlow off => no status updates (went quiet) 664 | // (2) vpp restarted with default sFlow => status updates (starting again 665 | // from 0) 666 | smp->now_mono_S = 0; 667 | 668 | // reset sequence numbers to indicated discontinuity 669 | smp->psample_seq_ingress = 0; 670 | smp->psample_seq_egress = 0; 671 | smp->psample_send = 0; 672 | smp->psample_send_drops = 0; 673 | smp->csample_send = 0; 674 | smp->csample_send_drops = 0; 675 | smp->dropmon_send = 0; 676 | smp->dropmon_send_drops = 0; 677 | 678 | /* open PSAMPLE netlink channel for writing packet samples */ 679 | SFLOWPS_init (&smp->sflow_psample); 680 | SFLOWPS_open (&smp->sflow_psample); 681 | /* open USERSOCK netlink channel for writing counters */ 682 | SFLOWUS_init (&smp->sflow_usersock); 683 | SFLOWUS_open (&smp->sflow_usersock); 684 | /* open DROPMON netlink channel for writing discard events */ 685 | SFLOWDM_init (&smp->sflow_dropmon); 686 | SFLOWDM_open (&smp->sflow_dropmon); 687 | /* set up (or reset) sampling context for each thread */ 688 | sflow_set_worker_sampling_state (smp); 689 | } 690 | 691 | static void 692 | sflow_sampling_stop (sflow_main_t *smp) 693 | { 694 | SFLOW_INFO ("sflow_sampling_stop"); 695 | smp->running = false; 696 | SFLOWPS_close (&smp->sflow_psample); 697 | SFLOWUS_close (&smp->sflow_usersock); 698 | SFLOWDM_close (&smp->sflow_dropmon); 699 | } 700 | 701 | static void 702 | sflow_sampling_start_stop (sflow_main_t *smp) 703 | { 704 | int run = 705 | ((smp->samplingN != 0 && smp->interfacesEnabled != 0) || smp->dropM); 706 | if (run != smp->running) 707 | { 708 | if (run) 709 | sflow_sampling_start (smp); 710 | else 711 | sflow_sampling_stop (smp); 712 | } 713 | } 714 | 715 | int 716 | sflow_sampling_rate (sflow_main_t *smp, u32 samplingN) 717 | { 718 | // TODO: this might be the right place to enforce the 719 | // "2 significant" figures constraint so that per-interface 720 | // sampling-rate settings can use HCF+sub-sampling efficiently. 721 | 722 | if (smp->running && smp->samplingN && samplingN) 723 | { 724 | // dynamic change of sampling rate 725 | smp->samplingN = samplingN; 726 | sflow_set_worker_sampling_state (smp); 727 | } 728 | else 729 | { 730 | // potential on/off change 731 | smp->samplingN = samplingN; 732 | sflow_sampling_start_stop (smp); 733 | } 734 | return 0; 735 | } 736 | 737 | int 738 | sflow_polling_interval (sflow_main_t *smp, u32 pollingS) 739 | { 740 | smp->pollingS = pollingS; 741 | return 0; 742 | } 743 | 744 | int 745 | sflow_header_bytes (sflow_main_t *smp, u32 headerB) 746 | { 747 | u32 hdrB = headerB; 748 | // first round up to nearest multiple of SFLOW_HEADER_BYTES_STEP 749 | // (which helps to make worker thread memcpy faster) 750 | hdrB = ((hdrB + SFLOW_HEADER_BYTES_STEP - 1) / SFLOW_HEADER_BYTES_STEP) * 751 | SFLOW_HEADER_BYTES_STEP; 752 | // then check max/min 753 | if (hdrB < SFLOW_MIN_HEADER_BYTES) 754 | hdrB = SFLOW_MIN_HEADER_BYTES; 755 | if (hdrB > SFLOW_MAX_HEADER_BYTES) 756 | hdrB = SFLOW_MAX_HEADER_BYTES; 757 | if (hdrB != headerB) 758 | SFLOW_WARN ("header_bytes rounded from %u to %u\n", headerB, hdrB); 759 | smp->headerB = hdrB; 760 | return 0; 761 | } 762 | 763 | void 764 | sflow_enable_disable_interface (sflow_main_t *smp, 765 | sflow_per_interface_data_t *sfif) 766 | { 767 | bool ingress_on = 768 | sfif->sflow_enabled && (smp->samplingD == SFLOW_DIRN_INGRESS || 769 | smp->samplingD == SFLOW_DIRN_BOTH); 770 | bool egress_on = 771 | sfif->sflow_enabled && 772 | (smp->samplingD == SFLOW_DIRN_EGRESS || smp->samplingD == SFLOW_DIRN_BOTH); 773 | bool drop_on = sfif->sflow_enabled && smp->dropM; 774 | bool ingress_enabled = (vnet_feature_is_enabled ("device-input", "sflow", 775 | sfif->sw_if_index) == 1); 776 | bool egress_enabled = 777 | (vnet_feature_is_enabled ("interface-output", "sflow-egress", 778 | sfif->sw_if_index) == 1); 779 | bool drop_enabled = (vnet_feature_is_enabled ("error-drop", "sflow-drop", 780 | sfif->sw_if_index) == 1); 781 | 782 | if (ingress_on != ingress_enabled) 783 | vnet_feature_enable_disable ("device-input", "sflow", sfif->sw_if_index, 784 | ingress_on, 0, 0); 785 | if (egress_on != egress_enabled) 786 | vnet_feature_enable_disable ("interface-output", "sflow-egress", 787 | sfif->sw_if_index, egress_on, 0, 0); 788 | if (drop_on != drop_enabled) 789 | vnet_feature_enable_disable ("error-drop", "sflow-drop", sfif->sw_if_index, 790 | smp->dropM, 0, 0); 791 | } 792 | 793 | void 794 | sflow_enable_disable_all (sflow_main_t *smp) 795 | { 796 | for (int ii = 0; ii < vec_len (smp->per_interface_data); ii++) 797 | { 798 | sflow_per_interface_data_t *sfif = 799 | vec_elt_at_index (smp->per_interface_data, ii); 800 | if (sfif && sfif->sflow_enabled) 801 | sflow_enable_disable_interface (smp, sfif); 802 | } 803 | } 804 | 805 | int 806 | sflow_direction (sflow_main_t *smp, sflow_direction_t samplingD) 807 | { 808 | if (samplingD != smp->samplingD) 809 | { 810 | // direction changed - tell all active interfaces. 811 | smp->samplingD = samplingD; 812 | sflow_enable_disable_all (smp); 813 | } 814 | return 0; 815 | } 816 | 817 | int 818 | sflow_drop_monitoring (sflow_main_t *smp, bool dropM) 819 | { 820 | if (dropM != smp->dropM) 821 | { 822 | // drop-monitoring changed. 823 | smp->dropM = dropM; 824 | // Tell all active interfaces. 825 | sflow_enable_disable_all (smp); 826 | } 827 | return 0; 828 | } 829 | 830 | int 831 | sflow_enable_disable (sflow_main_t *smp, u32 sw_if_index, bool enable_disable) 832 | { 833 | vnet_sw_interface_t *sw; 834 | 835 | /* Utterly wrong? */ 836 | if (pool_is_free_index (smp->vnet_main->interface_main.sw_interfaces, 837 | sw_if_index)) 838 | return VNET_API_ERROR_INVALID_SW_IF_INDEX; 839 | 840 | /* Not a physical port? */ 841 | sw = vnet_get_sw_interface (smp->vnet_main, sw_if_index); 842 | if (sw->type != VNET_SW_INTERFACE_TYPE_HARDWARE) 843 | return VNET_API_ERROR_INVALID_SW_IF_INDEX; 844 | 845 | // note: vnet_interface_main_t has "fast lookup table" called 846 | // he_if_index_by_sw_if_index. 847 | SFLOW_DBG ("sw_if_index=%u, sup_sw_if_index=%u, hw_if_index=%u\n", 848 | sw->sw_if_index, sw->sup_sw_if_index, sw->hw_if_index); 849 | 850 | // note: vnet_hw_interface_t has uword *bond_info 851 | // (where 0=>none, ~0 => slave, other=>ptr to bitmap of slaves) 852 | 853 | vec_validate (smp->per_interface_data, sw->hw_if_index); 854 | sflow_per_interface_data_t *sfif = 855 | vec_elt_at_index (smp->per_interface_data, sw->hw_if_index); 856 | if (enable_disable == sfif->sflow_enabled) 857 | { 858 | // redundant enable or disable 859 | return VNET_API_ERROR_VALUE_EXIST; 860 | } 861 | else 862 | { 863 | // OK, turn it on/off 864 | sfif->sw_if_index = sw_if_index; 865 | sfif->hw_if_index = sw->hw_if_index; 866 | sfif->polled = 0; 867 | sfif->sflow_enabled = enable_disable; 868 | sflow_enable_disable_interface (smp, sfif); 869 | smp->interfacesEnabled += (enable_disable) ? 1 : -1; 870 | } 871 | 872 | sflow_sampling_start_stop (smp); 873 | return 0; 874 | } 875 | 876 | static clib_error_t * 877 | sflow_sampling_rate_command_fn (vlib_main_t *vm, unformat_input_t *input, 878 | vlib_cli_command_t *cmd) 879 | { 880 | sflow_main_t *smp = &sflow_main; 881 | u32 sampling_N = ~0; 882 | 883 | int rv; 884 | 885 | while (unformat_check_input (input) != UNFORMAT_END_OF_INPUT) 886 | { 887 | if (unformat (input, "%u", &sampling_N)) 888 | ; 889 | else 890 | break; 891 | } 892 | 893 | if (sampling_N == ~0) 894 | return clib_error_return (0, "Please specify a sampling rate..."); 895 | 896 | rv = sflow_sampling_rate (smp, sampling_N); 897 | 898 | switch (rv) 899 | { 900 | case 0: 901 | break; 902 | default: 903 | return clib_error_return (0, "sflow_enable_disable returned %d", rv); 904 | } 905 | return 0; 906 | } 907 | 908 | static clib_error_t * 909 | sflow_polling_interval_command_fn (vlib_main_t *vm, unformat_input_t *input, 910 | vlib_cli_command_t *cmd) 911 | { 912 | sflow_main_t *smp = &sflow_main; 913 | u32 polling_S = ~0; 914 | 915 | int rv; 916 | 917 | while (unformat_check_input (input) != UNFORMAT_END_OF_INPUT) 918 | { 919 | if (unformat (input, "%u", &polling_S)) 920 | ; 921 | else 922 | break; 923 | } 924 | 925 | if (polling_S == ~0) 926 | return clib_error_return (0, "Please specify a polling interval..."); 927 | 928 | rv = sflow_polling_interval (smp, polling_S); 929 | 930 | switch (rv) 931 | { 932 | case 0: 933 | break; 934 | default: 935 | return clib_error_return (0, "sflow_polling_interval returned %d", rv); 936 | } 937 | return 0; 938 | } 939 | 940 | static clib_error_t * 941 | sflow_header_bytes_command_fn (vlib_main_t *vm, unformat_input_t *input, 942 | vlib_cli_command_t *cmd) 943 | { 944 | sflow_main_t *smp = &sflow_main; 945 | u32 header_B = ~0; 946 | 947 | int rv; 948 | 949 | while (unformat_check_input (input) != UNFORMAT_END_OF_INPUT) 950 | { 951 | if (unformat (input, "%u", &header_B)) 952 | ; 953 | else 954 | break; 955 | } 956 | 957 | if (header_B == ~0) 958 | return clib_error_return (0, "Please specify a header bytes limit..."); 959 | 960 | rv = sflow_header_bytes (smp, header_B); 961 | 962 | switch (rv) 963 | { 964 | case 0: 965 | break; 966 | default: 967 | return clib_error_return (0, "sflow_header_bytes returned %d", rv); 968 | } 969 | return 0; 970 | } 971 | 972 | static clib_error_t * 973 | sflow_direction_command_fn (vlib_main_t *vm, unformat_input_t *input, 974 | vlib_cli_command_t *cmd) 975 | { 976 | sflow_main_t *smp = &sflow_main; 977 | u32 sampling_D = SFLOW_DIRN_UNDEFINED; 978 | 979 | int rv; 980 | 981 | while (unformat_check_input (input) != UNFORMAT_END_OF_INPUT) 982 | { 983 | if (unformat (input, "rx")) 984 | sampling_D = SFLOW_DIRN_INGRESS; 985 | else if (unformat (input, "tx")) 986 | sampling_D = SFLOW_DIRN_EGRESS; 987 | else if (unformat (input, "both")) 988 | sampling_D = SFLOW_DIRN_BOTH; 989 | else 990 | break; 991 | } 992 | 993 | if (sampling_D == SFLOW_DIRN_UNDEFINED) 994 | return clib_error_return ( 995 | 0, "Please specify a sampling direction (rx|tx|both)..."); 996 | 997 | rv = sflow_direction (smp, sampling_D); 998 | 999 | switch (rv) 1000 | { 1001 | case 0: 1002 | break; 1003 | default: 1004 | return clib_error_return (0, "sflow_direction returned %d", rv); 1005 | } 1006 | return 0; 1007 | } 1008 | 1009 | static clib_error_t * 1010 | sflow_drop_monitoring_command_fn (vlib_main_t *vm, unformat_input_t *input, 1011 | vlib_cli_command_t *cmd) 1012 | { 1013 | sflow_main_t *smp = &sflow_main; 1014 | bool drop_M = true; 1015 | 1016 | int rv; 1017 | 1018 | while (unformat_check_input (input) != UNFORMAT_END_OF_INPUT) 1019 | { 1020 | if (unformat (input, "disable")) 1021 | drop_M = false; 1022 | else if (unformat (input, "enable")) 1023 | drop_M = true; 1024 | else 1025 | break; 1026 | } 1027 | 1028 | rv = sflow_drop_monitoring (smp, drop_M); 1029 | 1030 | switch (rv) 1031 | { 1032 | case 0: 1033 | break; 1034 | default: 1035 | return clib_error_return (0, "sflow_drop_monitoring returned %d", rv); 1036 | } 1037 | return 0; 1038 | } 1039 | 1040 | static clib_error_t * 1041 | sflow_enable_disable_command_fn (vlib_main_t *vm, unformat_input_t *input, 1042 | vlib_cli_command_t *cmd) 1043 | { 1044 | sflow_main_t *smp = &sflow_main; 1045 | u32 sw_if_index = ~0; 1046 | int enable_disable = true; 1047 | 1048 | int rv; 1049 | 1050 | while (unformat_check_input (input) != UNFORMAT_END_OF_INPUT) 1051 | { 1052 | if (unformat (input, "disable")) 1053 | enable_disable = false; 1054 | else if (unformat (input, "enable")) 1055 | enable_disable = true; 1056 | else if (unformat (input, "%U", unformat_vnet_sw_interface, 1057 | smp->vnet_main, &sw_if_index)) 1058 | ; 1059 | else 1060 | break; 1061 | } 1062 | 1063 | if (sw_if_index == ~0) 1064 | return clib_error_return (0, "Please specify an interface..."); 1065 | 1066 | rv = sflow_enable_disable (smp, sw_if_index, enable_disable); 1067 | 1068 | switch (rv) 1069 | { 1070 | case 0: 1071 | break; 1072 | 1073 | case VNET_API_ERROR_INVALID_SW_IF_INDEX: 1074 | return clib_error_return ( 1075 | 0, "Invalid interface, only works on physical ports"); 1076 | break; 1077 | 1078 | case VNET_API_ERROR_UNIMPLEMENTED: 1079 | return clib_error_return (0, 1080 | "Device driver doesn't support redirection"); 1081 | break; 1082 | 1083 | default: 1084 | return clib_error_return (0, "sflow_enable_disable returned %d", rv); 1085 | } 1086 | return 0; 1087 | } 1088 | 1089 | static const char * 1090 | sflow_direction_str (sflow_direction_t direction) 1091 | { 1092 | switch (direction) 1093 | { 1094 | case SFLOW_DIRN_UNDEFINED: 1095 | return "undefined"; 1096 | case SFLOW_DIRN_INGRESS: 1097 | return "rx"; 1098 | case SFLOW_DIRN_EGRESS: 1099 | return "tx"; 1100 | case SFLOW_DIRN_BOTH: 1101 | return "both"; 1102 | } 1103 | return "none"; 1104 | } 1105 | 1106 | static clib_error_t * 1107 | show_sflow_command_fn (vlib_main_t *vm, unformat_input_t *input, 1108 | vlib_cli_command_t *cmd) 1109 | { 1110 | sflow_main_t *smp = &sflow_main; 1111 | clib_error_t *error = NULL; 1112 | vlib_cli_output (vm, "sflow sampling-rate %u\n", smp->samplingN); 1113 | vlib_cli_output (vm, "sflow direction %s\n", 1114 | sflow_direction_str (smp->samplingD)); 1115 | vlib_cli_output (vm, "sflow polling-interval %u\n", smp->pollingS); 1116 | vlib_cli_output (vm, "sflow header-bytes %u\n", smp->headerB); 1117 | vlib_cli_output (vm, "sflow drop-monitoring %s\n", 1118 | smp->dropM ? "enable" : "disable"); 1119 | u32 itfs_enabled = 0; 1120 | for (int ii = 0; ii < vec_len (smp->per_interface_data); ii++) 1121 | { 1122 | sflow_per_interface_data_t *sfif = 1123 | vec_elt_at_index (smp->per_interface_data, ii); 1124 | if (sfif && sfif->sflow_enabled) 1125 | { 1126 | itfs_enabled++; 1127 | vnet_hw_interface_t *hw = 1128 | vnet_get_hw_interface (smp->vnet_main, sfif->hw_if_index); 1129 | vlib_cli_output (vm, "sflow enable %s\n", (char *) hw->name); 1130 | } 1131 | } 1132 | vlib_cli_output (vm, "Status\n"); 1133 | vlib_cli_output (vm, " interfaces enabled: %u\n", itfs_enabled); 1134 | vlib_cli_output (vm, " packet samples sent: %u\n", smp->psample_send); 1135 | vlib_cli_output (vm, " packet samples dropped: %u\n", total_drops (smp)); 1136 | vlib_cli_output (vm, " counter samples sent: %u\n", smp->csample_send); 1137 | vlib_cli_output (vm, " counter samples dropped: %u\n", 1138 | smp->csample_send_drops); 1139 | vlib_cli_output (vm, " drop samples sent: %u\n", smp->dropmon_send); 1140 | vlib_cli_output (vm, " drop samples dropped: %u\n", 1141 | smp->dropmon_send_drops); 1142 | return error; 1143 | } 1144 | 1145 | VLIB_CLI_COMMAND (sflow_enable_disable_command, static) = { 1146 | .path = "sflow enable-disable", 1147 | .short_help = "sflow enable-disable [disable]", 1148 | .function = sflow_enable_disable_command_fn, 1149 | }; 1150 | 1151 | VLIB_CLI_COMMAND (sflow_sampling_rate_command, static) = { 1152 | .path = "sflow sampling-rate", 1153 | .short_help = "sflow sampling-rate ", 1154 | .function = sflow_sampling_rate_command_fn, 1155 | }; 1156 | 1157 | VLIB_CLI_COMMAND (sflow_polling_interval_command, static) = { 1158 | .path = "sflow polling-interval", 1159 | .short_help = "sflow polling-interval ", 1160 | .function = sflow_polling_interval_command_fn, 1161 | }; 1162 | 1163 | VLIB_CLI_COMMAND (sflow_header_bytes_command, static) = { 1164 | .path = "sflow header-bytes", 1165 | .short_help = "sflow header-bytes ", 1166 | .function = sflow_header_bytes_command_fn, 1167 | }; 1168 | 1169 | VLIB_CLI_COMMAND (sflow_direction_command, static) = { 1170 | .path = "sflow direction", 1171 | .short_help = "sflow direction ", 1172 | .function = sflow_direction_command_fn, 1173 | }; 1174 | 1175 | VLIB_CLI_COMMAND (sflow_drop_monitoring_command, static) = { 1176 | .path = "sflow drop-monitoring", 1177 | .short_help = "sflow drop-monitoring ", 1178 | .function = sflow_drop_monitoring_command_fn, 1179 | }; 1180 | 1181 | VLIB_CLI_COMMAND (show_sflow_command, static) = { 1182 | .path = "show sflow", 1183 | .short_help = "show sflow", 1184 | .function = show_sflow_command_fn, 1185 | }; 1186 | 1187 | /* API message handler */ 1188 | static void 1189 | vl_api_sflow_enable_disable_t_handler (vl_api_sflow_enable_disable_t *mp) 1190 | { 1191 | vl_api_sflow_enable_disable_reply_t *rmp; 1192 | sflow_main_t *smp = &sflow_main; 1193 | int rv; 1194 | 1195 | rv = sflow_enable_disable (smp, ntohl (mp->hw_if_index), mp->enable_disable); 1196 | 1197 | REPLY_MACRO (VL_API_SFLOW_ENABLE_DISABLE_REPLY); 1198 | } 1199 | 1200 | static void 1201 | vl_api_sflow_sampling_rate_set_t_handler (vl_api_sflow_sampling_rate_set_t *mp) 1202 | { 1203 | vl_api_sflow_sampling_rate_set_reply_t *rmp; 1204 | sflow_main_t *smp = &sflow_main; 1205 | int rv; 1206 | 1207 | rv = sflow_sampling_rate (smp, ntohl (mp->sampling_N)); 1208 | 1209 | REPLY_MACRO (VL_API_SFLOW_SAMPLING_RATE_SET_REPLY); 1210 | } 1211 | 1212 | static void 1213 | vl_api_sflow_sampling_rate_get_t_handler (vl_api_sflow_sampling_rate_get_t *mp) 1214 | { 1215 | vl_api_sflow_sampling_rate_get_reply_t *rmp; 1216 | sflow_main_t *smp = &sflow_main; 1217 | 1218 | REPLY_MACRO_DETAILS2 (VL_API_SFLOW_SAMPLING_RATE_GET_REPLY, 1219 | ({ rmp->sampling_N = ntohl (smp->samplingN); })); 1220 | } 1221 | 1222 | static void 1223 | vl_api_sflow_polling_interval_set_t_handler ( 1224 | vl_api_sflow_polling_interval_set_t *mp) 1225 | { 1226 | vl_api_sflow_polling_interval_set_reply_t *rmp; 1227 | sflow_main_t *smp = &sflow_main; 1228 | int rv; 1229 | 1230 | rv = sflow_polling_interval (smp, ntohl (mp->polling_S)); 1231 | 1232 | REPLY_MACRO (VL_API_SFLOW_POLLING_INTERVAL_SET_REPLY); 1233 | } 1234 | 1235 | static void 1236 | vl_api_sflow_polling_interval_get_t_handler ( 1237 | vl_api_sflow_polling_interval_get_t *mp) 1238 | { 1239 | vl_api_sflow_polling_interval_get_reply_t *rmp; 1240 | sflow_main_t *smp = &sflow_main; 1241 | 1242 | REPLY_MACRO_DETAILS2 (VL_API_SFLOW_POLLING_INTERVAL_GET_REPLY, 1243 | ({ rmp->polling_S = ntohl (smp->pollingS); })); 1244 | } 1245 | 1246 | static void 1247 | vl_api_sflow_header_bytes_set_t_handler (vl_api_sflow_header_bytes_set_t *mp) 1248 | { 1249 | vl_api_sflow_header_bytes_set_reply_t *rmp; 1250 | sflow_main_t *smp = &sflow_main; 1251 | int rv; 1252 | 1253 | rv = sflow_header_bytes (smp, ntohl (mp->header_B)); 1254 | 1255 | REPLY_MACRO (VL_API_SFLOW_HEADER_BYTES_SET_REPLY); 1256 | } 1257 | 1258 | static void 1259 | vl_api_sflow_header_bytes_get_t_handler (vl_api_sflow_header_bytes_get_t *mp) 1260 | { 1261 | vl_api_sflow_header_bytes_get_reply_t *rmp; 1262 | sflow_main_t *smp = &sflow_main; 1263 | 1264 | REPLY_MACRO_DETAILS2 (VL_API_SFLOW_HEADER_BYTES_GET_REPLY, 1265 | ({ rmp->header_B = ntohl (smp->headerB); })); 1266 | } 1267 | 1268 | static void 1269 | vl_api_sflow_direction_set_t_handler (vl_api_sflow_direction_set_t *mp) 1270 | { 1271 | vl_api_sflow_direction_set_reply_t *rmp; 1272 | sflow_main_t *smp = &sflow_main; 1273 | int rv; 1274 | 1275 | rv = sflow_direction (smp, ntohl (mp->sampling_D)); 1276 | 1277 | REPLY_MACRO (VL_API_SFLOW_DIRECTION_SET_REPLY); 1278 | } 1279 | 1280 | static void 1281 | vl_api_sflow_direction_get_t_handler (vl_api_sflow_direction_get_t *mp) 1282 | { 1283 | vl_api_sflow_direction_get_reply_t *rmp; 1284 | sflow_main_t *smp = &sflow_main; 1285 | 1286 | REPLY_MACRO_DETAILS2 (VL_API_SFLOW_DIRECTION_GET_REPLY, 1287 | ({ rmp->sampling_D = ntohl (smp->samplingD); })); 1288 | } 1289 | 1290 | static void 1291 | vl_api_sflow_drop_monitoring_set_t_handler ( 1292 | vl_api_sflow_drop_monitoring_set_t *mp) 1293 | { 1294 | vl_api_sflow_drop_monitoring_set_reply_t *rmp; 1295 | sflow_main_t *smp = &sflow_main; 1296 | int rv; 1297 | rv = sflow_drop_monitoring (smp, ntohl (mp->drop_M)); 1298 | 1299 | REPLY_MACRO (VL_API_SFLOW_DROP_MONITORING_SET_REPLY); 1300 | } 1301 | 1302 | static void 1303 | vl_api_sflow_drop_monitoring_get_t_handler ( 1304 | vl_api_sflow_drop_monitoring_get_t *mp) 1305 | { 1306 | vl_api_sflow_drop_monitoring_get_reply_t *rmp; 1307 | sflow_main_t *smp = &sflow_main; 1308 | 1309 | REPLY_MACRO_DETAILS2 (VL_API_SFLOW_DROP_MONITORING_GET_REPLY, 1310 | ({ rmp->drop_M = ntohl (smp->dropM); })); 1311 | } 1312 | 1313 | static void 1314 | send_sflow_interface_details (vpe_api_main_t *am, vl_api_registration_t *reg, 1315 | u32 context, const u32 hw_if_index) 1316 | { 1317 | vl_api_sflow_interface_details_t *mp; 1318 | sflow_main_t *smp = &sflow_main; 1319 | 1320 | mp = vl_msg_api_alloc_zero (sizeof (*mp)); 1321 | mp->_vl_msg_id = ntohs (REPLY_MSG_ID_BASE + VL_API_SFLOW_INTERFACE_DETAILS); 1322 | mp->context = context; 1323 | 1324 | mp->hw_if_index = htonl (hw_if_index); 1325 | vl_api_send_msg (reg, (u8 *) mp); 1326 | } 1327 | 1328 | static void 1329 | vl_api_sflow_interface_dump_t_handler (vl_api_sflow_interface_dump_t *mp) 1330 | { 1331 | vpe_api_main_t *am = &vpe_api_main; 1332 | sflow_main_t *smp = &sflow_main; 1333 | vl_api_registration_t *reg; 1334 | u32 hw_if_index = ~0; 1335 | 1336 | reg = vl_api_client_index_to_registration (mp->client_index); 1337 | if (!reg) 1338 | return; 1339 | hw_if_index = ntohl (mp->hw_if_index); 1340 | 1341 | for (int ii = 0; ii < vec_len (smp->per_interface_data); ii++) 1342 | { 1343 | sflow_per_interface_data_t *sfif = 1344 | vec_elt_at_index (smp->per_interface_data, ii); 1345 | if (sfif && sfif->sflow_enabled) 1346 | { 1347 | if (hw_if_index == ~0 || hw_if_index == sfif->hw_if_index) 1348 | { 1349 | send_sflow_interface_details (am, reg, mp->context, 1350 | sfif->hw_if_index); 1351 | } 1352 | } 1353 | } 1354 | } 1355 | 1356 | /* API definitions */ 1357 | #include 1358 | 1359 | static clib_error_t * 1360 | sflow_init (vlib_main_t *vm) 1361 | { 1362 | sflow_logger = vlib_log_register_class ("sflow", "all"); 1363 | 1364 | sflow_main_t *smp = &sflow_main; 1365 | clib_error_t *error = 0; 1366 | 1367 | smp->vlib_main = vm; 1368 | smp->vnet_main = vnet_get_main (); 1369 | 1370 | /* set default sampling-rate and polling-interval so that "enable" is all 1371 | * that is necessary */ 1372 | smp->samplingN = SFLOW_DEFAULT_SAMPLING_N; 1373 | smp->pollingS = SFLOW_DEFAULT_POLLING_S; 1374 | smp->headerB = SFLOW_DEFAULT_HEADER_BYTES; 1375 | smp->samplingD = SFLOW_DIRN_INGRESS; 1376 | smp->dropM = false; 1377 | 1378 | /* Add our API messages to the global name_crc hash table */ 1379 | smp->msg_id_base = setup_message_id_table (); 1380 | 1381 | /* access to counters - TODO: should this only happen on sflow enable? */ 1382 | sflow_stat_segment_client_init (); 1383 | 1384 | smp->lcp_itf_pair_get_vif_index_by_phy = 1385 | vlib_get_plugin_symbol (SFLOW_LCP_LIB, SFLOW_LCP_SYM_GET_VIF_BY_PHY); 1386 | if (smp->lcp_itf_pair_get_vif_index_by_phy) 1387 | { 1388 | SFLOW_NOTICE ("linux-cp found - using LIP vif_index, where available"); 1389 | } 1390 | else 1391 | { 1392 | SFLOW_NOTICE ("linux-cp not found - using VPP sw_if_index"); 1393 | } 1394 | 1395 | return error; 1396 | } 1397 | 1398 | VLIB_INIT_FUNCTION (sflow_init); 1399 | 1400 | VNET_FEATURE_INIT (sflow, static) = { 1401 | .arc_name = "device-input", 1402 | .node_name = "sflow", 1403 | .runs_before = VNET_FEATURES ("ethernet-input"), 1404 | }; 1405 | 1406 | VNET_FEATURE_INIT (sflow_egress, static) = { 1407 | .arc_name = "interface-output", 1408 | .node_name = "sflow-egress", 1409 | .runs_before = VNET_FEATURES ("interface-output-arc-end"), 1410 | }; 1411 | 1412 | /* Add myself to the feature arc */ 1413 | VNET_FEATURE_INIT (sflow_drop, static) = { 1414 | .arc_name = "error-drop", 1415 | .node_name = "sflow-drop", 1416 | .runs_before = VNET_FEATURES ("drop"), 1417 | }; 1418 | 1419 | VLIB_PLUGIN_REGISTER () = { 1420 | .version = VPP_BUILD_VER, 1421 | .description = "sFlow random packet sampling", 1422 | }; 1423 | 1424 | /* 1425 | * fd.io coding-style-patch-verification: ON 1426 | * 1427 | * Local Variables: 1428 | * eval: (c-set-style "gnu") 1429 | * End: 1430 | */ 1431 | -------------------------------------------------------------------------------- /sflow/sflow.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2024 InMon Corp. 3 | * Licensed under the Apache License, Version 2.0 (the "License"); 4 | * you may not use this file except in compliance with the License. 5 | * You may obtain a copy of the License at: 6 | * 7 | * http://www.apache.org/licenses/LICENSE-2.0 8 | * 9 | * Unless required by applicable law or agreed to in writing, software 10 | * distributed under the License is distributed on an "AS IS" BASIS, 11 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | * See the License for the specific language governing permissions and 13 | * limitations under the License. 14 | */ 15 | #ifndef __included_sflow_h__ 16 | #define __included_sflow_h__ 17 | 18 | #include 19 | #include 20 | #include 21 | 22 | #include 23 | #include 24 | #include 25 | #include 26 | #include 27 | #include 28 | #include 29 | 30 | #define SFLOW_DEFAULT_SAMPLING_N 10000 31 | #define SFLOW_DEFAULT_POLLING_S 20 32 | #define SFLOW_DEFAULT_HEADER_BYTES 128 33 | #define SFLOW_MAX_HEADER_BYTES 256 34 | #define SFLOW_MIN_HEADER_BYTES 64 35 | #define SFLOW_HEADER_BYTES_STEP 32 36 | 37 | #define SFLOW_FIFO_DEPTH 2048 // must be power of 2 38 | #define SFLOW_DROP_FIFO_DEPTH 4 // must be power of 2 39 | #define SFLOW_POLL_WAIT_S 0.001 40 | #define SFLOW_READ_BATCH 100 41 | 42 | // use PSAMPLE group number to distinguish VPP samples from others 43 | // (so that hsflowd will know to remap the ifIndex numbers if necessary) 44 | #define SFLOW_VPP_PSAMPLE_GROUP_INGRESS 3 45 | #define SFLOW_VPP_PSAMPLE_GROUP_EGRESS 4 46 | 47 | #define foreach_sflow_error \ 48 | _ (PROCESSED, "sflow packets processed") \ 49 | _ (SAMPLED, "sflow packets sampled") \ 50 | _ (DROPPED, "sflow packets dropped") \ 51 | _ (DIPROCESSED, "sflow discards processed") \ 52 | _ (DIDROPPED, "sflow discards dropped") \ 53 | _ (PSAMPLE_SEND, "sflow PSAMPLE sent") \ 54 | _ (PSAMPLE_SEND_FAIL, "sflow PSAMPLE send failed") \ 55 | _ (DROPMON_SEND, "sflow DROPMON sent") \ 56 | _ (DROPMON_SEND_FAIL, "sflow DROPMON send failed") 57 | 58 | typedef enum 59 | { 60 | #define _(sym, str) SFLOW_ERROR_##sym, 61 | foreach_sflow_error 62 | #undef _ 63 | SFLOW_N_ERROR, 64 | } sflow_error_t; 65 | 66 | typedef struct 67 | { 68 | u32 counters[SFLOW_N_ERROR]; 69 | } sflow_err_ctrs_t; 70 | 71 | /* packet sample */ 72 | typedef struct 73 | { 74 | u32 sample_type; 75 | u32 samplingN; 76 | u32 input_if_index; 77 | u32 output_if_index; 78 | u32 header_protocol; 79 | u32 sampled_packet_size; 80 | u32 header_bytes; 81 | u32 drop_reason; 82 | u8 header[SFLOW_MAX_HEADER_BYTES]; 83 | } sflow_sample_t; 84 | 85 | typedef enum 86 | { 87 | SFLOW_SAMPLETYPE_UNDEFINED = 0, 88 | SFLOW_SAMPLETYPE_INGRESS, 89 | SFLOW_SAMPLETYPE_EGRESS, 90 | SFLOW_SAMPLETYPE_DISCARD 91 | } sflow_enum_sample_t; 92 | 93 | #define SFLOW_MAX_TRAP_LEN 64 94 | #define SFLOW_TRAP_WHITE '_' 95 | #define SFLOW_TRAP_PREFIX "vpp_" 96 | 97 | // Define SPSC FIFO for sending samples worker-to-main. 98 | // (I did try to use VPP svm FIFO, but couldn't 99 | // understand why it was sometimes going wrong). 100 | typedef struct 101 | { 102 | volatile u32 tx; // can change under consumer's feet 103 | volatile u32 rx; // can change under producer's feet 104 | sflow_sample_t samples[SFLOW_FIFO_DEPTH]; 105 | } sflow_fifo_t; 106 | 107 | #define SFLOW_FIFO_NEXT(slot) ((slot + 1) & (SFLOW_FIFO_DEPTH - 1)) 108 | static inline int 109 | sflow_fifo_enqueue (sflow_fifo_t *fifo, sflow_sample_t *sample) 110 | { 111 | u32 curr_rx = clib_atomic_load_acq_n (&fifo->rx); 112 | u32 curr_tx = fifo->tx; // clib_atomic_load_acq_n(&fifo->tx); 113 | u32 next_tx = SFLOW_FIFO_NEXT (curr_tx); 114 | if (next_tx == curr_rx) 115 | return false; // full 116 | memcpy (&fifo->samples[next_tx], sample, sizeof (*sample)); 117 | clib_atomic_store_rel_n (&fifo->tx, next_tx); 118 | return true; 119 | } 120 | 121 | static inline int 122 | sflow_fifo_dequeue (sflow_fifo_t *fifo, sflow_sample_t *sample) 123 | { 124 | u32 curr_rx = fifo->rx; // clib_atomic_load_acq_n(&fifo->rx); 125 | u32 curr_tx = clib_atomic_load_acq_n (&fifo->tx); 126 | if (curr_rx == curr_tx) 127 | return false; // empty 128 | u32 next_rx = SFLOW_FIFO_NEXT (curr_rx); 129 | memcpy (sample, &fifo->samples[next_rx], sizeof (*sample)); 130 | clib_atomic_store_rel_n (&fifo->rx, next_rx); 131 | return true; 132 | } 133 | 134 | // Define SPSC DROP_FIFO for sending discard events worker-to-main. 135 | // For now the only difference from the FIFO above is the max depth, 136 | // but it proved awkward to make depth a variable and this way gives 137 | // us more freedom to experiment, e.g. with rate-limiting. 138 | // We also might decide to separate sflow_sample_t into 139 | // sflow_sample_t and sflow_drop_t if their fields diverge, 140 | // and doing this keeps that option open. 141 | typedef struct 142 | { 143 | volatile u32 tx; // can change under consumer's feet 144 | volatile u32 rx; // can change under producer's feet 145 | sflow_sample_t samples[SFLOW_DROP_FIFO_DEPTH]; 146 | } sflow_drop_fifo_t; 147 | 148 | #define SFLOW_DROP_FIFO_NEXT(slot) ((slot + 1) & (SFLOW_DROP_FIFO_DEPTH - 1)) 149 | static inline int 150 | sflow_drop_fifo_enqueue (sflow_drop_fifo_t *fifo, sflow_sample_t *sample) 151 | { 152 | u32 curr_rx = clib_atomic_load_acq_n (&fifo->rx); 153 | u32 curr_tx = fifo->tx; // clib_atomic_load_acq_n(&fifo->tx); 154 | u32 next_tx = SFLOW_DROP_FIFO_NEXT (curr_tx); 155 | if (next_tx == curr_rx) 156 | return false; // full 157 | memcpy (&fifo->samples[next_tx], sample, sizeof (*sample)); 158 | clib_atomic_store_rel_n (&fifo->tx, next_tx); 159 | return true; 160 | } 161 | 162 | static inline int 163 | sflow_drop_fifo_dequeue (sflow_drop_fifo_t *fifo, sflow_sample_t *sample) 164 | { 165 | u32 curr_rx = fifo->rx; // clib_atomic_load_acq_n(&fifo->rx); 166 | u32 curr_tx = clib_atomic_load_acq_n (&fifo->tx); 167 | if (curr_rx == curr_tx) 168 | return false; // empty 169 | u32 next_rx = SFLOW_DROP_FIFO_NEXT (curr_rx); 170 | memcpy (sample, &fifo->samples[next_rx], sizeof (*sample)); 171 | clib_atomic_store_rel_n (&fifo->rx, next_rx); 172 | return true; 173 | } 174 | 175 | /* private to worker */ 176 | typedef struct 177 | { 178 | u32 smpN; 179 | u32 skip; 180 | u32 pool; 181 | u32 seed; 182 | u32 smpl; 183 | u32 drop; 184 | u32 dsmp; 185 | u32 ddrp; 186 | CLIB_CACHE_LINE_ALIGN_MARK (_fifo); 187 | sflow_fifo_t fifo; 188 | CLIB_CACHE_LINE_ALIGN_MARK (_drop_fifo); 189 | sflow_drop_fifo_t drop_fifo; 190 | } sflow_per_thread_data_t; 191 | 192 | typedef u32 (*IfIndexLookupFn) (u32); 193 | 194 | typedef struct 195 | { 196 | /* API message ID base */ 197 | u16 msg_id_base; 198 | 199 | /* convenience */ 200 | vlib_main_t *vlib_main; 201 | vnet_main_t *vnet_main; 202 | ethernet_main_t *ethernet_main; 203 | 204 | /* sampling state */ 205 | u32 samplingN; 206 | u32 pollingS; 207 | u32 headerB; 208 | sflow_direction_t samplingD; 209 | bool dropM; 210 | u32 total_threads; 211 | sflow_per_interface_data_t *per_interface_data; 212 | sflow_per_thread_data_t *per_thread_data; 213 | 214 | /* psample channel (packet samples) */ 215 | SFLOWPS sflow_psample; 216 | /* usersock channel (periodic counters) */ 217 | SFLOWUS sflow_usersock; 218 | /* dropmon channel (rate-limited discards) */ 219 | SFLOWDM sflow_dropmon; 220 | #define SFLOW_NETLINK_USERSOCK_MULTICAST 29 221 | /* dropmon channel (packet drops) */ 222 | // SFLOWDM sflow_dropmon; 223 | 224 | /* sample-processing */ 225 | u32 now_mono_S; 226 | 227 | /* running control */ 228 | bool running; 229 | u32 interfacesEnabled; 230 | 231 | /* main-thread counters */ 232 | u32 psample_seq_ingress; 233 | u32 psample_seq_egress; 234 | u32 psample_send; 235 | u32 psample_send_drops; 236 | u32 dropmon_send; 237 | u32 dropmon_send_drops; 238 | u32 csample_send; 239 | u32 csample_send_drops; 240 | u32 unixsock_seq; 241 | IfIndexLookupFn lcp_itf_pair_get_vif_index_by_phy; 242 | } sflow_main_t; 243 | 244 | extern sflow_main_t sflow_main; 245 | 246 | extern vlib_node_registration_t sflow_node; 247 | 248 | static inline u32 249 | sflow_next_random_skip (sflow_per_thread_data_t *sfwk) 250 | { 251 | /* skip==1 means "take the next packet" so this 252 | fn must never return 0 */ 253 | if (sfwk->smpN <= 1) 254 | return 1; 255 | u32 lim = (2 * sfwk->smpN) - 1; 256 | return (random_u32 (&sfwk->seed) % lim) + 1; 257 | } 258 | 259 | #endif /* __included_sflow_h__ */ 260 | 261 | /* 262 | * fd.io coding-style-patch-verification: ON 263 | * 264 | * Local Variables: 265 | * eval: (c-set-style "gnu") 266 | * End: 267 | */ 268 | -------------------------------------------------------------------------------- /sflow/sflow.rst: -------------------------------------------------------------------------------- 1 | .. _Sflow_agent: 2 | 3 | .. toctree:: 4 | 5 | SFlow Monitoring Agent 6 | ====================== 7 | 8 | Overview 9 | ________ 10 | 11 | This plugin implements the random packet-sampling and interface 12 | telemetry streaming required to support standard sFlow export 13 | on Linux platforms. The overhead incurred by this monitoring is 14 | minimal, so that detailed, real-time traffic analysis can be 15 | achieved even under high load conditions, with visibility into 16 | any fields that appear in the packet headers. If the VPP linux-cp 17 | plugin is running then interfaces will be mapped to their 18 | equivalent Linux tap ports. 19 | 20 | Example Configuration 21 | _____________________ 22 | 23 | :: 24 | sflow sampling-rate 10000 25 | sflow polling-interval 20 26 | sflow header-bytes 128 27 | sflow enable GigabitEthernet0/8/0 28 | sflow enable GigabitEthernet0/9/0 29 | sflow enable GigabitEthernet0/a/0 30 | ... 31 | sflow enable GigabitEthernet0/a/0 disable 32 | 33 | Detailed notes 34 | ______________ 35 | 36 | Each VPP worker that has at least one interface, will create a FIFO 37 | and enqueues samples to it from the interfaces it is servicing that 38 | are enabled. There is a process running in the main thread that will 39 | dequeue the FIFOs periodically. If the FIFO is full, the worker will 40 | drop samples, which helps ensure that (a) the main thread is not 41 | overloaded with samples and (b) that individual workers and interfaces, 42 | even when under high load, can't crowd out other interfaces and workers. 43 | 44 | You can change the sampling-rate at runtime, but keep in mind that 45 | it is a global variable that applies to workers, not interfaces. 46 | This means that (1) all workers will sample at the same rate, and (2) 47 | if there are multiple interfaces assigned to a worker, they'll share 48 | the sampling rate which will undershoot, and similarly (3) if there 49 | are multiple RX queues assigned to more than one worker, the effective 50 | sampling rate will overshoot. 51 | 52 | External Dependencies 53 | _____________________ 54 | 55 | This plugin writes packet samples to the standard Linux netlink PSAMPLE 56 | channel, so the kernel psample module must be loaded with modprobe or 57 | insmod. As such, this plugin only works for Linux environments. 58 | 59 | It also shares periodic interface counter samples vi netlink USERSOCK. 60 | The host-sflow daemon, hsflowd, at https://sflow.net is one example of 61 | a tool that will consume this feed and emit standard sFlow v5. 62 | -------------------------------------------------------------------------------- /sflow/sflow_common.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2024 InMon Corp. 3 | * Licensed under the Apache License, Version 2.0 (the "License"); 4 | * you may not use this file except in compliance with the License. 5 | * You may obtain a copy of the License at: 6 | * 7 | * http://www.apache.org/licenses/LICENSE-2.0 8 | * 9 | * Unless required by applicable law or agreed to in writing, software 10 | * distributed under the License is distributed on an "AS IS" BASIS, 11 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | * See the License for the specific language governing permissions and 13 | * limitations under the License. 14 | */ 15 | #ifndef __included_sflow_common_h__ 16 | #define __included_sflow_common_h__ 17 | 18 | extern vlib_log_class_t sflow_logger; 19 | #define SFLOW_DBG(...) vlib_log_debug (sflow_logger, __VA_ARGS__); 20 | #define SFLOW_INFO(...) vlib_log_info (sflow_logger, __VA_ARGS__); 21 | #define SFLOW_NOTICE(...) vlib_log_notice (sflow_logger, __VA_ARGS__); 22 | #define SFLOW_WARN(...) vlib_log_warn (sflow_logger, __VA_ARGS__); 23 | #define SFLOW_ERR(...) vlib_log_err (sflow_logger, __VA_ARGS__); 24 | 25 | typedef struct 26 | { 27 | u32 sw_if_index; 28 | u32 hw_if_index; 29 | u32 linux_if_index; 30 | u32 polled; 31 | int sflow_enabled; 32 | } sflow_per_interface_data_t; 33 | 34 | /* mirror sflow_direction enum in sflow.api */ 35 | typedef enum 36 | { 37 | SFLOW_DIRN_UNDEFINED = 0, 38 | SFLOW_DIRN_INGRESS, 39 | SFLOW_DIRN_EGRESS, 40 | SFLOW_DIRN_BOTH 41 | } sflow_direction_t; 42 | 43 | #endif /* __included_sflow_common_h__ */ 44 | 45 | /* 46 | * fd.io coding-style-patch-verification: ON 47 | * 48 | * Local Variables: 49 | * eval: (c-set-style "gnu") 50 | * End: 51 | */ 52 | -------------------------------------------------------------------------------- /sflow/sflow_dlapi.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2025 InMon Corp. 3 | * Licensed under the Apache License, Version 2.0 (the "License"); 4 | * you may not use this file except in compliance with the License. 5 | * You may obtain a copy of the License at: 6 | * 7 | * http://www.apache.org/licenses/LICENSE-2.0 8 | * 9 | * Unless required by applicable law or agreed to in writing, software 10 | * distributed under the License is distributed on an "AS IS" BASIS, 11 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | * See the License for the specific language governing permissions and 13 | * limitations under the License. 14 | */ 15 | #ifndef __included_sflow_dlapi_h__ 16 | #define __included_sflow_dlapi_h__ 17 | /* Dynamic-link API 18 | * If present, linux-cp plugin will be queried to learn the 19 | * Linux if_index for each VPP if_index. If that plugin is not 20 | * compiled and loaded, or if the function symbol is not found, 21 | * then the interfaces will be reported to NETLINK_USERSOCK 22 | * without this extra mapping. 23 | */ 24 | #define SFLOW_LCP_LIB "linux_cp_plugin.so" 25 | #define SFLOW_LCP_SYM_GET_VIF_BY_PHY "lcp_itf_pair_get_vif_index_by_phy" 26 | #endif /* __included_sflow_dyn_api_h__ */ 27 | /* 28 | * fd.io coding-style-patch-verification: ON 29 | * 30 | * Local Variables: 31 | * eval: (c-set-style "gnu") 32 | * End: 33 | */ 34 | -------------------------------------------------------------------------------- /sflow/sflow_dropmon.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2025 InMon Corp. 3 | * Licensed under the Apache License, Version 2.0 (the "License"); 4 | * you may not use this file except in compliance with the License. 5 | * You may obtain a copy of the License at: 6 | * 7 | * http://www.apache.org/licenses/LICENSE-2.0 8 | * 9 | * Unless required by applicable law or agreed to in writing, software 10 | * distributed under the License is distributed on an "AS IS" BASIS, 11 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | * See the License for the specific language governing permissions and 13 | * limitations under the License. 14 | */ 15 | 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | 22 | #include 23 | #include 24 | #include 25 | #include 26 | #include 27 | #include 28 | #include 29 | #include 30 | #include 31 | #include 32 | 33 | #include 34 | #include 35 | 36 | /*_________________---------------------------__________________ 37 | _________________ SFLOWDM_init __________________ 38 | -----------------___________________________------------------ 39 | */ 40 | 41 | EnumSFLOWNLState 42 | SFLOWDM_init (SFLOWDM *dmt) 43 | { 44 | dmt->nl.id = SFLOWNL_DROPMON; 45 | memset (dmt->fam_name, 0, SFLOWDM_FAM_FOOTPRINT); 46 | memcpy (dmt->fam_name, SFLOWDM_FAM, SFLOWDM_FAM_LEN); 47 | dmt->nl.family_name = dmt->fam_name; 48 | dmt->nl.family_len = SFLOWDM_FAM_LEN; 49 | dmt->nl.join_group_id = NET_DM_GRP_ALERT; 50 | dmt->nl.attr = dmt->attr; 51 | dmt->nl.attr_max = SFLOWDM_ATTRS - 1; 52 | dmt->nl.iov = dmt->iov; 53 | dmt->nl.iov_max = SFLOWDM_IOV_FRAGS - 1; 54 | dmt->nl.state = SFLOWNL_STATE_INIT; 55 | return dmt->nl.state; 56 | } 57 | 58 | /*_________________---------------------------__________________ 59 | _________________ SFLOWDM_open __________________ 60 | -----------------___________________________------------------ 61 | */ 62 | 63 | bool 64 | SFLOWDM_open (SFLOWDM *dmt) 65 | { 66 | if (dmt->nl.state == SFLOWNL_STATE_UNDEFINED) 67 | SFLOWDM_init (dmt); 68 | if (dmt->nl.nl_sock == 0) 69 | { 70 | dmt->nl.nl_sock = sflow_netlink_generic_open (&dmt->nl); 71 | if (dmt->nl.nl_sock > 0) 72 | sflow_netlink_generic_get_family (&dmt->nl); 73 | } 74 | return (dmt->nl.nl_sock > 0); 75 | } 76 | 77 | /*_________________---------------------------__________________ 78 | _________________ SFLOWDM_close __________________ 79 | -----------------___________________________------------------ 80 | */ 81 | 82 | bool 83 | SFLOWDM_close (SFLOWDM *dmt) 84 | { 85 | return (sflow_netlink_close (&dmt->nl) == 0); 86 | } 87 | 88 | /*_________________---------------------------__________________ 89 | _________________ SFLOWDM_state __________________ 90 | -----------------___________________________------------------ 91 | */ 92 | 93 | EnumSFLOWNLState 94 | SFLOWDM_state (SFLOWDM *dmt) 95 | { 96 | return dmt->nl.state; 97 | } 98 | 99 | /*_________________---------------------------__________________ 100 | _________________ SFLOWDM_open_step __________________ 101 | -----------------___________________________------------------ 102 | */ 103 | 104 | EnumSFLOWNLState 105 | SFLOWDM_open_step (SFLOWDM *dmt) 106 | { 107 | switch (dmt->nl.state) 108 | { 109 | case SFLOWNL_STATE_UNDEFINED: 110 | SFLOWDM_init (dmt); 111 | break; 112 | case SFLOWNL_STATE_INIT: 113 | SFLOWDM_open (dmt); 114 | break; 115 | case SFLOWNL_STATE_OPEN: 116 | sflow_netlink_generic_get_family (&dmt->nl); 117 | break; 118 | case SFLOWNL_STATE_WAIT_FAMILY: 119 | sflow_netlink_read (&dmt->nl); 120 | break; 121 | case SFLOWNL_STATE_READY: 122 | break; 123 | } 124 | return dmt->nl.state; 125 | } 126 | 127 | /*_________________---------------------------__________________ 128 | _________________ SFLOWDMSpec_setAttr __________________ 129 | -----------------___________________________------------------ 130 | */ 131 | 132 | bool 133 | SFLOWDM_set_attr (SFLOWDM *dmt, int field, void *val, int len) 134 | { 135 | return sflow_netlink_set_attr (&dmt->nl, field, val, len); 136 | } 137 | 138 | /*_________________---------------------------__________________ 139 | _________________ SFLOWDMSpec_send __________________ 140 | -----------------___________________________------------------ 141 | */ 142 | 143 | int 144 | SFLOWDM_send (SFLOWDM *dmt) 145 | { 146 | dmt->nl.ge.cmd = NET_DM_CMD_PACKET_ALERT; 147 | dmt->nl.ge.version = 0; // NET_DM_CFG_VERSION==0 but no NET_DM_CMD_VERSION 148 | int status = sflow_netlink_send_attrs (&dmt->nl, true); 149 | sflow_netlink_reset_attrs (&dmt->nl); 150 | if (status <= 0) 151 | { 152 | SFLOW_ERR ("DROPMON strerror(errno) = %s; errno = %d\n", 153 | strerror (errno), errno); 154 | } 155 | return status; 156 | } 157 | 158 | /* 159 | * fd.io coding-style-patch-verification: ON 160 | * 161 | * Local Variables: 162 | * eval: (c-set-style "gnu") 163 | * End: 164 | */ 165 | -------------------------------------------------------------------------------- /sflow/sflow_dropmon.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2025 InMon Corp. 3 | * Licensed under the Apache License, Version 2.0 (the "License"); 4 | * you may not use this file except in compliance with the License. 5 | * You may obtain a copy of the License at: 6 | * 7 | * http://www.apache.org/licenses/LICENSE-2.0 8 | * 9 | * Unless required by applicable law or agreed to in writing, software 10 | * distributed under the License is distributed on an "AS IS" BASIS, 11 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | * See the License for the specific language governing permissions and 13 | * limitations under the License. 14 | */ 15 | 16 | #ifndef __included_sflow_dropmon_h__ 17 | #define __included_sflow_dropmon_h__ 18 | 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include 24 | 25 | #include 26 | #include 27 | #include 28 | #include 29 | #include 30 | #include 31 | #include 32 | #include 33 | #include 34 | 35 | #include 36 | 37 | #define SFLOWDM_DROPMON_READNL_RCV_BUF 8192 38 | #define SFLOWDM_DROPMON_READNL_SND_BUF 1000000 39 | 40 | #ifndef NET_DM_GENL_NAME 41 | #define NET_DM_GENL_NAME "NET_DM" 42 | #endif 43 | 44 | #define SFLOWDM_FAM NET_DM_GENL_NAME 45 | #define SFLOWDM_FAM_LEN sizeof (SFLOWDM_FAM) 46 | #define SFLOWDM_FAM_FOOTPRINT NLMSG_ALIGN (SFLOWDM_FAM_LEN) 47 | #define SFLOWDM_ATTRS NET_DM_ATTR_MAX + 1 48 | #define SFLOWDM_IOV_FRAGS ((2 * SFLOWDM_ATTRS) + 2) 49 | 50 | typedef struct _SFLOWDM 51 | { 52 | SFLOWNL nl; 53 | char fam_name[SFLOWDM_FAM_FOOTPRINT]; 54 | SFLOWNLAttr attr[SFLOWDM_ATTRS]; 55 | struct iovec iov[SFLOWDM_IOV_FRAGS]; 56 | } SFLOWDM; 57 | 58 | EnumSFLOWNLState SFLOWDM_init (SFLOWDM *dmt); 59 | bool SFLOWDM_open (SFLOWDM *dmt); 60 | bool SFLOWDM_close (SFLOWDM *dmt); 61 | EnumSFLOWNLState SFLOWDM_state (SFLOWDM *dmt); 62 | EnumSFLOWNLState SFLOWDM_open_step (SFLOWDM *dmt); 63 | 64 | bool SFLOWDM_set_attr (SFLOWDM *dmt, int field, void *buf, int len); 65 | #define SFLOWDM_set_attr_int(dmt, field, val) \ 66 | SFLOWDM_set_attr ((dmt), (field), &(val), sizeof (val)) 67 | 68 | int SFLOWDM_send (SFLOWDM *dmt); 69 | 70 | #endif /* __included_sflow_dropmon_h__ */ 71 | 72 | /* 73 | * fd.io coding-style-patch-verification: ON 74 | * 75 | * Local Variables: 76 | * eval: (c-set-style "gnu") 77 | * End: 78 | */ 79 | -------------------------------------------------------------------------------- /sflow/sflow_netlink.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2025 InMon Corp. 3 | * Licensed under the Apache License, Version 2.0 (the "License"); 4 | * you may not use this file except in compliance with the License. 5 | * You may obtain a copy of the License at: 6 | * 7 | * http://www.apache.org/licenses/LICENSE-2.0 8 | * 9 | * Unless required by applicable law or agreed to in writing, software 10 | * distributed under the License is distributed on an "AS IS" BASIS, 11 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | * See the License for the specific language governing permissions and 13 | * limitations under the License. 14 | */ 15 | 16 | #include 17 | #include 18 | #include 19 | #include 20 | 21 | #include 22 | #include 23 | #include 24 | #include 25 | #include 26 | #include 27 | #include 28 | #include 29 | 30 | #include 31 | 32 | /*_________________---------------------------__________________ 33 | _________________ fcntl utils __________________ 34 | -----------------___________________________------------------ 35 | */ 36 | 37 | void 38 | sflow_netlink_set_nonblocking (int fd) 39 | { 40 | // set the socket to non-blocking 41 | int fdFlags = fcntl (fd, F_GETFL); 42 | fdFlags |= O_NONBLOCK; 43 | if (fcntl (fd, F_SETFL, fdFlags) < 0) 44 | { 45 | SFLOW_ERR ("fcntl(O_NONBLOCK) failed: %s\n", strerror (errno)); 46 | } 47 | } 48 | 49 | void 50 | sflow_netlink_set_close_on_exec (int fd) 51 | { 52 | // make sure it doesn't get inherited, e.g. when we fork a script 53 | int fdFlags = fcntl (fd, F_GETFD); 54 | fdFlags |= FD_CLOEXEC; 55 | if (fcntl (fd, F_SETFD, fdFlags) < 0) 56 | { 57 | SFLOW_ERR ("fcntl(F_SETFD=FD_CLOEXEC) failed: %s\n", strerror (errno)); 58 | } 59 | } 60 | 61 | int 62 | sflow_netlink_set_send_buffer (int fd, int requested) 63 | { 64 | int txbuf = 0; 65 | socklen_t txbufsiz = sizeof (txbuf); 66 | if (getsockopt (fd, SOL_SOCKET, SO_SNDBUF, &txbuf, &txbufsiz) < 0) 67 | { 68 | SFLOW_ERR ("getsockopt(SO_SNDBUF) failed: %s", strerror (errno)); 69 | } 70 | if (txbuf < requested) 71 | { 72 | txbuf = requested; 73 | if (setsockopt (fd, SOL_SOCKET, SO_SNDBUF, &txbuf, sizeof (txbuf)) < 0) 74 | { 75 | SFLOW_WARN ("setsockopt(SO_TXBUF=%d) failed: %s", requested, 76 | strerror (errno)); 77 | } 78 | // see what we actually got 79 | txbufsiz = sizeof (txbuf); 80 | if (getsockopt (fd, SOL_SOCKET, SO_SNDBUF, &txbuf, &txbufsiz) < 0) 81 | { 82 | SFLOW_ERR ("getsockopt(SO_SNDBUF) failed: %s", strerror (errno)); 83 | } 84 | } 85 | return txbuf; 86 | } 87 | 88 | /*_________________---------------------------__________________ 89 | _________________ generic_pid __________________ 90 | -----------------___________________________------------------ 91 | choose a 32-bit id that is likely to be unique even if more 92 | than one module in this process wants to bind a netlink socket 93 | */ 94 | 95 | u32 96 | sflow_netlink_generic_pid (u32 mod_id) 97 | { 98 | return ((mod_id << 16) + getpid ()); 99 | } 100 | 101 | /*_________________---------------------------__________________ 102 | _________________ generic_open __________________ 103 | -----------------___________________________------------------ 104 | */ 105 | 106 | int 107 | sflow_netlink_generic_open (SFLOWNL *nl) 108 | { 109 | nl->nl_sock = socket (AF_NETLINK, SOCK_RAW, NETLINK_GENERIC); 110 | if (nl->nl_sock < 0) 111 | { 112 | SFLOW_ERR ("nl_sock open failed: %s\n", strerror (errno)); 113 | return -1; 114 | } 115 | // bind to a suitable id 116 | struct sockaddr_nl sa = { .nl_family = AF_NETLINK, 117 | .nl_pid = sflow_netlink_generic_pid (nl->id) }; 118 | if (bind (nl->nl_sock, (struct sockaddr *) &sa, sizeof (sa)) < 0) 119 | { 120 | SFLOW_ERR ("sflow_netlink_generic_open: bind failed: sa.nl_pid=%u " 121 | "sock=%d id=%d: %s\n", 122 | sa.nl_pid, nl->nl_sock, nl->id, strerror (errno)); 123 | } 124 | sflow_netlink_set_nonblocking (nl->nl_sock); 125 | sflow_netlink_set_close_on_exec (nl->nl_sock); 126 | sflow_netlink_set_send_buffer (nl->nl_sock, SFLOWNL_SND_BUF); 127 | nl->state = SFLOWNL_STATE_OPEN; 128 | return nl->nl_sock; 129 | } 130 | 131 | /*_________________---------------------------__________________ 132 | _________________ usersock_open __________________ 133 | -----------------___________________________------------------ 134 | */ 135 | 136 | int 137 | sflow_netlink_usersock_open (SFLOWNL *nl) 138 | { 139 | nl->nl_sock = socket (AF_NETLINK, SOCK_RAW, NETLINK_USERSOCK); 140 | if (nl->nl_sock < 0) 141 | { 142 | SFLOW_ERR ("nl_sock open failed: %s\n", strerror (errno)); 143 | return -1; 144 | } 145 | sflow_netlink_set_nonblocking (nl->nl_sock); 146 | sflow_netlink_set_close_on_exec (nl->nl_sock); 147 | nl->state = SFLOWNL_STATE_OPEN; 148 | return nl->nl_sock; 149 | } 150 | 151 | /*_________________---------------------------__________________ 152 | _________________ close __________________ 153 | -----------------___________________________------------------ 154 | */ 155 | 156 | int 157 | sflow_netlink_close (SFLOWNL *nl) 158 | { 159 | int err = 0; 160 | if (nl->nl_sock > 0) 161 | { 162 | err = close (nl->nl_sock); 163 | if (err == 0) 164 | { 165 | nl->nl_sock = 0; 166 | } 167 | else 168 | { 169 | SFLOW_ERR ("sflow_netlink_close: returned %d : %s\n", err, 170 | strerror (errno)); 171 | } 172 | } 173 | nl->state = SFLOWNL_STATE_INIT; 174 | return err; 175 | } 176 | 177 | /*_________________---------------------------__________________ 178 | _________________ set_attr __________________ 179 | -----------------___________________________------------------ 180 | */ 181 | 182 | bool 183 | sflow_netlink_set_attr (SFLOWNL *nl, int field, void *val, int len) 184 | { 185 | SFLOWNLAttr *psa = &nl->attr[field]; 186 | if (psa->included) 187 | return false; 188 | psa->included = true; 189 | psa->attr.nla_type = field; 190 | psa->attr.nla_len = sizeof (psa->attr) + len; 191 | int len_w_pad = NLMSG_ALIGN (len); 192 | psa->val.iov_len = len_w_pad; 193 | psa->val.iov_base = val; 194 | nl->n_attrs++; 195 | nl->attrs_len += sizeof (psa->attr); 196 | nl->attrs_len += len_w_pad; 197 | return true; 198 | } 199 | 200 | /*_________________---------------------------__________________ 201 | _________________ generic_send_cmd __________________ 202 | -----------------___________________________------------------ 203 | */ 204 | 205 | int 206 | sflow_netlink_generic_send_cmd (int sockfd, u32 mod_id, int type, int cmd, 207 | int req_type, void *req, int req_len, 208 | int req_footprint, u32 seqNo) 209 | { 210 | struct nlmsghdr nlh = {}; 211 | struct genlmsghdr ge = {}; 212 | struct nlattr attr = {}; 213 | 214 | attr.nla_len = sizeof (attr) + req_len; 215 | attr.nla_type = req_type; 216 | 217 | ge.cmd = cmd; 218 | ge.version = 1; 219 | 220 | nlh.nlmsg_len = NLMSG_LENGTH (req_footprint + sizeof (attr) + sizeof (ge)); 221 | nlh.nlmsg_flags = NLM_F_REQUEST | NLM_F_ACK; 222 | nlh.nlmsg_type = type; 223 | nlh.nlmsg_seq = seqNo; 224 | nlh.nlmsg_pid = sflow_netlink_generic_pid (mod_id); 225 | 226 | struct iovec iov[4] = { { .iov_base = &nlh, .iov_len = sizeof (nlh) }, 227 | { .iov_base = &ge, .iov_len = sizeof (ge) }, 228 | { .iov_base = &attr, .iov_len = sizeof (attr) }, 229 | { .iov_base = req, .iov_len = req_footprint } }; 230 | 231 | struct sockaddr_nl sa = { .nl_family = AF_NETLINK }; 232 | struct msghdr msg = { .msg_name = &sa, 233 | .msg_namelen = sizeof (sa), 234 | .msg_iov = iov, 235 | .msg_iovlen = 4 }; 236 | return sendmsg (sockfd, &msg, 0); 237 | } 238 | 239 | /*_________________---------------------------__________________ 240 | _________________ send_attrs __________________ 241 | -----------------___________________________------------------ 242 | */ 243 | 244 | int 245 | sflow_netlink_send_attrs (SFLOWNL *nl, bool ge) 246 | { 247 | if (ge) 248 | { 249 | nl->nlh.nlmsg_len = NLMSG_LENGTH (sizeof (nl->ge) + nl->attrs_len); 250 | nl->nlh.nlmsg_type = nl->family_id; 251 | nl->nlh.nlmsg_pid = sflow_netlink_generic_pid (nl->id); 252 | } 253 | else 254 | { 255 | nl->nlh.nlmsg_len = NLMSG_LENGTH (nl->attrs_len); 256 | nl->nlh.nlmsg_pid = getpid (); 257 | } 258 | 259 | nl->nlh.nlmsg_flags = 0; 260 | nl->nlh.nlmsg_seq = ++nl->nl_seq; 261 | 262 | struct iovec *iov = nl->iov; 263 | u32 frag = 0; 264 | iov[frag].iov_base = &nl->nlh; 265 | iov[frag].iov_len = sizeof (nl->nlh); 266 | frag++; 267 | if (ge) 268 | { 269 | iov[frag].iov_base = &nl->ge; 270 | iov[frag].iov_len = sizeof (nl->ge); 271 | frag++; 272 | } 273 | int nn = 0; 274 | for (u32 ii = 0; ii <= nl->attr_max; ii++) 275 | { 276 | SFLOWNLAttr *psa = &nl->attr[ii]; 277 | if (psa->included) 278 | { 279 | nn++; 280 | iov[frag].iov_base = &psa->attr; 281 | iov[frag].iov_len = sizeof (psa->attr); 282 | frag++; 283 | iov[frag] = psa->val; // struct copy 284 | frag++; 285 | } 286 | } 287 | ASSERT (nn == nl->n_attrs); 288 | 289 | struct sockaddr_nl da = { .nl_family = AF_NETLINK, 290 | .nl_groups = (1 << (nl->group_id - 1)) }; 291 | 292 | struct msghdr msg = { .msg_name = &da, 293 | .msg_namelen = sizeof (da), 294 | .msg_iov = iov, 295 | .msg_iovlen = frag }; 296 | 297 | return sendmsg (nl->nl_sock, &msg, 0); 298 | } 299 | 300 | /*_________________---------------------------__________________ 301 | _________________ reset_attrs __________________ 302 | -----------------___________________________------------------ 303 | */ 304 | 305 | void 306 | sflow_netlink_reset_attrs (SFLOWNL *nl) 307 | { 308 | for (u32 ii = 0; ii <= nl->attr_max; ii++) 309 | nl->attr[ii].included = false; 310 | nl->n_attrs = 0; 311 | nl->attrs_len = 0; 312 | } 313 | 314 | /*_________________---------------------------__________________ 315 | _________________ generic_get_family __________________ 316 | -----------------___________________________------------------ 317 | */ 318 | 319 | void 320 | sflow_netlink_generic_get_family (SFLOWNL *nl) 321 | { 322 | int status = sflow_netlink_generic_send_cmd ( 323 | nl->nl_sock, nl->id, GENL_ID_CTRL, CTRL_CMD_GETFAMILY, 324 | CTRL_ATTR_FAMILY_NAME, nl->family_name, nl->family_len, 325 | NLMSG_ALIGN (nl->family_len), ++nl->nl_seq); 326 | if (status >= 0) 327 | nl->state = SFLOWNL_STATE_WAIT_FAMILY; 328 | } 329 | 330 | /*_________________---------------------------__________________ 331 | _________________ generic_read __________________ 332 | -----------------___________________________------------------ 333 | */ 334 | 335 | void 336 | sflow_netlink_generic_read (SFLOWNL *nl, struct nlmsghdr *nlh, int numbytes) 337 | { 338 | int msglen = nlh->nlmsg_len; 339 | if (msglen > numbytes) 340 | { 341 | SFLOW_ERR ("generic read msglen too long\n"); 342 | return; 343 | } 344 | if (msglen < (NLMSG_HDRLEN + GENL_HDRLEN + NLA_HDRLEN)) 345 | { 346 | SFLOW_ERR ("generic read msglen too short\n"); 347 | return; 348 | } 349 | char *msg = (char *) NLMSG_DATA (nlh); 350 | msglen -= NLMSG_HDRLEN; 351 | struct genlmsghdr *genl = (struct genlmsghdr *) msg; 352 | SFLOW_DBG ("generic netlink CMD = %u\n", genl->cmd); 353 | msglen -= GENL_HDRLEN; 354 | 355 | struct nlattr *attr0 = (struct nlattr *) (msg + GENL_HDRLEN); 356 | for (int attrs_len = msglen; SFNLA_OK (attr0, attrs_len); 357 | attr0 = SFNLA_NEXT (attr0, attrs_len)) 358 | { 359 | switch (attr0->nla_type) 360 | { 361 | case CTRL_ATTR_VERSION: 362 | nl->genetlink_version = *(u32 *) SFNLA_DATA (attr0); 363 | break; 364 | case CTRL_ATTR_FAMILY_ID: 365 | nl->family_id = *(u16 *) SFNLA_DATA (attr0); 366 | SFLOW_DBG ("generic family id: %u\n", nl->family_id); 367 | break; 368 | case CTRL_ATTR_FAMILY_NAME: 369 | SFLOW_DBG ("generic family name: %s\n", (char *) SFNLA_DATA (attr0)); 370 | break; 371 | case CTRL_ATTR_MCAST_GROUPS: 372 | { 373 | struct nlattr *attr1 = (struct nlattr *) SFNLA_DATA (attr0); 374 | for (int attr0_len = SFNLA_PAYLOAD (attr0); 375 | SFNLA_OK (attr1, attr0_len); 376 | attr1 = SFNLA_NEXT (attr1, attr0_len)) 377 | { 378 | char *grp_name = NULL; 379 | u32 grp_id = 0; 380 | struct nlattr *attr2 = SFNLA_DATA (attr1); 381 | for (int attr1_len = SFNLA_PAYLOAD (attr1); 382 | SFNLA_OK (attr2, attr1_len); 383 | attr2 = SFNLA_NEXT (attr2, attr1_len)) 384 | { 385 | 386 | switch (attr2->nla_type) 387 | { 388 | case CTRL_ATTR_MCAST_GRP_NAME: 389 | grp_name = SFNLA_DATA (attr2); 390 | SFLOW_DBG ("netlink multicast group: %s\n", grp_name); 391 | break; 392 | case CTRL_ATTR_MCAST_GRP_ID: 393 | grp_id = *(u32 *) SFNLA_DATA (attr2); 394 | SFLOW_DBG ("netlink multicast group id: %u\n", grp_id); 395 | break; 396 | } 397 | } 398 | if (nl->group_id == 0 && grp_name && 399 | (((nl->join_group_id != 0) && 400 | grp_id == nl->join_group_id) || 401 | ((nl->join_group_name != NULL) && 402 | !strcmp (grp_name, nl->join_group_name)))) 403 | { 404 | SFLOW_DBG ("netlink found group %s=%u\n", grp_name, 405 | grp_id); 406 | nl->group_id = grp_id; 407 | // We don't need to actually join the group if we 408 | // are only sending to it. 409 | } 410 | } 411 | } 412 | break; 413 | default: 414 | SFLOW_DBG ("netlink attr type: %u (nested=%u) len: %u\n", 415 | attr0->nla_type, attr0->nla_type & NLA_F_NESTED, 416 | attr0->nla_len); 417 | break; 418 | } 419 | } 420 | if (nl->family_id && nl->group_id) 421 | { 422 | SFLOW_DBG ("netlink state->READY\n"); 423 | nl->state = SFLOWNL_STATE_READY; 424 | } 425 | } 426 | 427 | /*_________________---------------------------__________________ 428 | _________________ sflow_netlink_read __________________ 429 | -----------------___________________________------------------ 430 | */ 431 | 432 | void 433 | sflow_netlink_read (SFLOWNL *nl) 434 | { 435 | uint8_t recv_buf[SFLOWNL_RCV_BUF]; 436 | memset (recv_buf, 0, SFLOWNL_RCV_BUF); // for coverity 437 | int numbytes = recv (nl->nl_sock, recv_buf, sizeof (recv_buf), 0); 438 | if (numbytes <= sizeof (struct nlmsghdr)) 439 | { 440 | SFLOW_ERR ("sflow_netlink_read returned %d : %s\n", numbytes, 441 | strerror (errno)); 442 | return; 443 | } 444 | for (struct nlmsghdr *nlh = (struct nlmsghdr *) recv_buf; 445 | NLMSG_OK (nlh, numbytes); nlh = NLMSG_NEXT (nlh, numbytes)) 446 | { 447 | if (nlh->nlmsg_type == NLMSG_DONE) 448 | break; 449 | if (nlh->nlmsg_type == NLMSG_ERROR) 450 | { 451 | struct nlmsgerr *err_msg = (struct nlmsgerr *) NLMSG_DATA (nlh); 452 | if (err_msg->error == 0) 453 | { 454 | SFLOW_DBG ("received Netlink ACK\n"); 455 | } 456 | else 457 | { 458 | SFLOW_ERR ("error in netlink message: %d : %s\n", err_msg->error, 459 | strerror (-err_msg->error)); 460 | } 461 | return; 462 | } 463 | if (nlh->nlmsg_type == NETLINK_GENERIC) 464 | { 465 | sflow_netlink_generic_read (nl, nlh, numbytes); 466 | } 467 | else if (nlh->nlmsg_type == nl->family_id) 468 | { 469 | // We are write-only, don't need to read these. 470 | } 471 | } 472 | } 473 | 474 | /* 475 | * fd.io coding-style-patch-verification: ON 476 | * 477 | * Local Variables: 478 | * eval: (c-set-style "gnu") 479 | * End: 480 | */ 481 | -------------------------------------------------------------------------------- /sflow/sflow_netlink.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2025 InMon Corp. 3 | * Licensed under the Apache License, Version 2.0 (the "License"); 4 | * you may not use this file except in compliance with the License. 5 | * You may obtain a copy of the License at: 6 | * 7 | * http://www.apache.org/licenses/LICENSE-2.0 8 | * 9 | * Unless required by applicable law or agreed to in writing, software 10 | * distributed under the License is distributed on an "AS IS" BASIS, 11 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | * See the License for the specific language governing permissions and 13 | * limitations under the License. 14 | */ 15 | 16 | #ifndef __included_sflow_netlink_h__ 17 | #define __included_sflow_netlink_h__ 18 | 19 | #include 20 | #include 21 | #include 22 | #include 23 | 24 | #include 25 | #include 26 | #include 27 | #include 28 | #include 29 | #include 30 | #include 31 | #include 32 | 33 | #define SFLOWNL_RCV_BUF 8192 34 | #define SFLOWNL_SND_BUF 1000000 35 | 36 | typedef enum 37 | { 38 | SFLOWNL_USERSOCK = 1, 39 | SFLOWNL_PSAMPLE, 40 | SFLOWNL_DROPMON, 41 | } EnumSFLOWNLMod; 42 | 43 | typedef enum 44 | { 45 | SFLOWNL_STATE_UNDEFINED = 0, 46 | SFLOWNL_STATE_INIT, 47 | SFLOWNL_STATE_OPEN, 48 | SFLOWNL_STATE_WAIT_FAMILY, 49 | SFLOWNL_STATE_READY 50 | } EnumSFLOWNLState; 51 | 52 | typedef struct _SFLOWNLAttr 53 | { 54 | bool included : 1; 55 | struct nlattr attr; 56 | struct iovec val; 57 | } SFLOWNLAttr; 58 | 59 | typedef struct _SFLOWNL 60 | { 61 | // connect 62 | EnumSFLOWNLState state; 63 | EnumSFLOWNLMod id; 64 | int nl_sock; 65 | u32 nl_seq; 66 | u32 genetlink_version; 67 | u16 family_id; 68 | u32 group_id; 69 | // setup 70 | char *family_name; 71 | u32 family_len; 72 | u32 join_group_id; 73 | char *join_group_name; 74 | // msg 75 | struct nlmsghdr nlh; 76 | struct genlmsghdr ge; 77 | SFLOWNLAttr *attr; 78 | u32 attr_max; 79 | u32 n_attrs; 80 | u32 attrs_len; 81 | u32 iov_max; 82 | struct iovec *iov; 83 | } SFLOWNL; 84 | 85 | void sflow_netlink_set_nonblocking (int fd); 86 | void sflow_netlink_set_close_on_exec (int fd); 87 | int sflow_netlink_set_send_buffer (int fd, int requested); 88 | u32 sflow_netlink_generic_pid (u32 mod_id); 89 | int sflow_netlink_generic_open (SFLOWNL *nl); 90 | int sflow_netlink_usersock_open (SFLOWNL *nl); 91 | int sflow_netlink_close (SFLOWNL *nl); 92 | bool sflow_netlink_set_attr (SFLOWNL *nl, int field, void *val, int len); 93 | 94 | #define sflow_netlink_set_attr_int(nl, field, val) \ 95 | sflow_netlink_set_attr ((nl), (field), &(val), sizeof (val)) 96 | 97 | int sflow_netlink_generic_send_cmd (int sockfd, u32 mod_id, int type, int cmd, 98 | int req_type, void *req, int req_len, 99 | int req_footprint, u32 seqNo); 100 | int sflow_netlink_send_attrs (SFLOWNL *nl, bool ge); 101 | void sflow_netlink_reset_attrs (SFLOWNL *nl); 102 | void sflow_netlink_generic_get_family (SFLOWNL *nl); 103 | void sflow_netlink_generic_read (SFLOWNL *nl, struct nlmsghdr *nlh, 104 | int numbytes); 105 | void sflow_netlink_read (SFLOWNL *nl); 106 | 107 | /* Provide the netlink attribute-walking macros that are strangely 108 | * missing from netlink.h, so we can walk attributes the same way 109 | * as we walk messages (and satisfy static-analysis algorithms that 110 | * are wary of looping over "tainted" input). 111 | */ 112 | #define SFNLA_OK(nla, len) \ 113 | ((len) > 0 && (nla)->nla_len >= sizeof (struct nlattr) && \ 114 | (nla)->nla_len <= (len)) 115 | #define SFNLA_NEXT(nla, attrlen) \ 116 | ((attrlen) -= NLA_ALIGN ((nla)->nla_len), \ 117 | (struct nlattr *) (((char *) (nla)) + NLA_ALIGN ((nla)->nla_len))) 118 | #define SFNLA_LENGTH(len) (NLA_ALIGN (sizeof (struct nlattr)) + (len)) 119 | #define SFNLA_SPACE(len) (NLA_ALIGN (SFNLA_LENGTH (len))) 120 | #define SFNLA_DATA(nla) ((void *) (((char *) (nla)) + SFNLA_LENGTH (0))) 121 | #define SFNLA_PAYLOAD(nla) ((int) ((nla)->nla_len) - SFNLA_LENGTH (0)) 122 | #endif /* __included_sflow_netlink_h__ */ 123 | 124 | /* 125 | * fd.io coding-style-patch-verification: ON 126 | * 127 | * Local Variables: 128 | * eval: (c-set-style "gnu") 129 | * End: 130 | */ 131 | -------------------------------------------------------------------------------- /sflow/sflow_psample.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2024 InMon Corp. 3 | * Licensed under the Apache License, Version 2.0 (the "License"); 4 | * you may not use this file except in compliance with the License. 5 | * You may obtain a copy of the License at: 6 | * 7 | * http://www.apache.org/licenses/LICENSE-2.0 8 | * 9 | * Unless required by applicable law or agreed to in writing, software 10 | * distributed under the License is distributed on an "AS IS" BASIS, 11 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | * See the License for the specific language governing permissions and 13 | * limitations under the License. 14 | */ 15 | 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | 22 | #include 23 | #include 24 | #include 25 | #include 26 | #include 27 | #include 28 | #include 29 | #include 30 | #include 31 | 32 | #include 33 | #include 34 | 35 | /*_________________---------------------------__________________ 36 | _________________ SFLOWPS_init __________________ 37 | -----------------___________________________------------------ 38 | */ 39 | 40 | EnumSFLOWNLState 41 | SFLOWPS_init (SFLOWPS *pst) 42 | { 43 | pst->nl.id = SFLOWNL_PSAMPLE; 44 | memset (pst->fam_name, 0, SFLOWPS_FAM_FOOTPRINT); 45 | memcpy (pst->fam_name, SFLOWPS_FAM, SFLOWPS_FAM_LEN); 46 | pst->nl.family_name = pst->fam_name; 47 | pst->nl.family_len = SFLOWPS_FAM_LEN; 48 | pst->nl.join_group_name = PSAMPLE_NL_MCGRP_SAMPLE_NAME; 49 | pst->nl.attr = pst->attr; 50 | pst->nl.attr_max = __SFLOWPS_PSAMPLE_ATTRS - 1; 51 | pst->nl.iov = pst->iov; 52 | pst->nl.iov_max = SFLOWPS_IOV_FRAGS - 1; 53 | pst->nl.state = SFLOWNL_STATE_INIT; 54 | return pst->nl.state; 55 | } 56 | 57 | /*_________________---------------------------__________________ 58 | _________________ SFLOWPS_open __________________ 59 | -----------------___________________________------------------ 60 | */ 61 | 62 | bool 63 | SFLOWPS_open (SFLOWPS *pst) 64 | { 65 | if (pst->nl.state == SFLOWNL_STATE_UNDEFINED) 66 | SFLOWPS_init (pst); 67 | if (pst->nl.nl_sock == 0) 68 | { 69 | pst->nl.nl_sock = sflow_netlink_generic_open (&pst->nl); 70 | if (pst->nl.nl_sock > 0) 71 | sflow_netlink_generic_get_family (&pst->nl); 72 | } 73 | return (pst->nl.nl_sock > 0); 74 | } 75 | 76 | /*_________________---------------------------__________________ 77 | _________________ SFLOWPS_close __________________ 78 | -----------------___________________________------------------ 79 | */ 80 | 81 | bool 82 | SFLOWPS_close (SFLOWPS *pst) 83 | { 84 | return (sflow_netlink_close (&pst->nl) == 0); 85 | } 86 | 87 | /*_________________---------------------------__________________ 88 | _________________ SFLOWPS_state __________________ 89 | -----------------___________________________------------------ 90 | */ 91 | 92 | EnumSFLOWNLState 93 | SFLOWPS_state (SFLOWPS *pst) 94 | { 95 | return pst->nl.state; 96 | } 97 | 98 | /*_________________---------------------------__________________ 99 | _________________ SFLOWPS_open_step __________________ 100 | -----------------___________________________------------------ 101 | */ 102 | 103 | EnumSFLOWNLState 104 | SFLOWPS_open_step (SFLOWPS *pst) 105 | { 106 | switch (pst->nl.state) 107 | { 108 | case SFLOWNL_STATE_UNDEFINED: 109 | SFLOWPS_init (pst); 110 | break; 111 | case SFLOWNL_STATE_INIT: 112 | SFLOWPS_open (pst); 113 | break; 114 | case SFLOWNL_STATE_OPEN: 115 | sflow_netlink_generic_get_family (&pst->nl); 116 | break; 117 | case SFLOWNL_STATE_WAIT_FAMILY: 118 | sflow_netlink_read (&pst->nl); 119 | break; 120 | case SFLOWNL_STATE_READY: 121 | break; 122 | } 123 | return pst->nl.state; 124 | } 125 | 126 | /*_________________---------------------------__________________ 127 | _________________ SFLOWPS_set_attr __________________ 128 | -----------------___________________________------------------ 129 | */ 130 | 131 | bool 132 | SFLOWPS_set_attr (SFLOWPS *pst, EnumSFLOWPSAttributes field, void *val, 133 | int len) 134 | { 135 | int expected_len = SFLOWPS_Fields[field].len; 136 | if (expected_len && expected_len != len) 137 | { 138 | SFLOW_ERR ("SFLOWPS_set_attr(%s) length=%u != expected: %u\n", 139 | SFLOWPS_Fields[field].descr, len, expected_len); 140 | return false; 141 | } 142 | return sflow_netlink_set_attr (&pst->nl, field, val, len); 143 | } 144 | 145 | /*_________________---------------------------__________________ 146 | _________________ SFLOWPS_send __________________ 147 | -----------------___________________________------------------ 148 | */ 149 | 150 | int 151 | SFLOWPS_send (SFLOWPS *pst) 152 | { 153 | pst->nl.ge.cmd = PSAMPLE_CMD_SAMPLE; 154 | pst->nl.ge.version = PSAMPLE_GENL_VERSION; 155 | int status = sflow_netlink_send_attrs (&pst->nl, true); 156 | sflow_netlink_reset_attrs (&pst->nl); 157 | if (status <= 0) 158 | { 159 | SFLOW_ERR ("PSAMPLE strerror(errno) = %s; errno = %d\n", 160 | strerror (errno), errno); 161 | } 162 | return status; 163 | } 164 | 165 | /* 166 | * fd.io coding-style-patch-verification: ON 167 | * 168 | * Local Variables: 169 | * eval: (c-set-style "gnu") 170 | * End: 171 | */ 172 | -------------------------------------------------------------------------------- /sflow/sflow_psample.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2024 InMon Corp. 3 | * Licensed under the Apache License, Version 2.0 (the "License"); 4 | * you may not use this file except in compliance with the License. 5 | * You may obtain a copy of the License at: 6 | * 7 | * http://www.apache.org/licenses/LICENSE-2.0 8 | * 9 | * Unless required by applicable law or agreed to in writing, software 10 | * distributed under the License is distributed on an "AS IS" BASIS, 11 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | * See the License for the specific language governing permissions and 13 | * limitations under the License. 14 | */ 15 | 16 | #ifndef __included_sflow_psample_h__ 17 | #define __included_sflow_psample_h__ 18 | 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include 24 | 25 | #include 26 | #include 27 | #include 28 | #include 29 | #include 30 | #include 31 | #include 32 | #include 33 | 34 | #include 35 | 36 | // #define SFLOWPS_DEBUG 37 | 38 | #define SFLOWPS_PSAMPLE_READNL_RCV_BUF 8192 39 | #define SFLOWPS_PSAMPLE_READNL_SND_BUF 1000000 40 | 41 | /* Shadow the attributes in linux/psample.h so 42 | * we can easily compile/test fields that are not 43 | * defined on the kernel we are compiling on. 44 | */ 45 | typedef enum 46 | { 47 | #define SFLOWPS_FIELDDATA(field, len, descr) field, 48 | #include "sflow/sflow_psample_fields.h" 49 | #undef SFLOWPS_FIELDDATA 50 | __SFLOWPS_PSAMPLE_ATTRS 51 | } EnumSFLOWPSAttributes; 52 | 53 | typedef struct _SFLOWPS_field_t 54 | { 55 | EnumSFLOWPSAttributes field; 56 | int len; 57 | char *descr; 58 | } SFLOWPS_field_t; 59 | 60 | static const SFLOWPS_field_t SFLOWPS_Fields[] = { 61 | #define SFLOWPS_FIELDDATA(field, len, descr) { field, len, descr }, 62 | #include "sflow/sflow_psample_fields.h" 63 | #undef SFLOWPS_FIELDDATA 64 | }; 65 | 66 | #define SFLOWPS_FAM PSAMPLE_GENL_NAME 67 | #define SFLOWPS_FAM_LEN sizeof (SFLOWPS_FAM) 68 | #define SFLOWPS_FAM_FOOTPRINT NLMSG_ALIGN (SFLOWPS_FAM_LEN) 69 | #define SFLOWPS_IOV_FRAGS ((2 * __SFLOWPS_PSAMPLE_ATTRS) + 2) 70 | 71 | typedef struct _SFLOWPS 72 | { 73 | SFLOWNL nl; 74 | char fam_name[SFLOWPS_FAM_FOOTPRINT]; 75 | SFLOWNLAttr attr[__SFLOWPS_PSAMPLE_ATTRS]; 76 | struct iovec iov[SFLOWPS_IOV_FRAGS]; 77 | } SFLOWPS; 78 | 79 | EnumSFLOWNLState SFLOWPS_init (SFLOWPS *pst); 80 | bool SFLOWPS_open (SFLOWPS *pst); 81 | bool SFLOWPS_close (SFLOWPS *pst); 82 | EnumSFLOWNLState SFLOWPS_state (SFLOWPS *pst); 83 | EnumSFLOWNLState SFLOWPS_open_step (SFLOWPS *pst); 84 | 85 | bool SFLOWPS_set_attr (SFLOWPS *pst, EnumSFLOWPSAttributes field, void *buf, 86 | int len); 87 | #define SFLOWPS_set_attr_int(pst, field, val) \ 88 | SFLOWPS_set_attr ((pst), (field), &(val), sizeof (val)) 89 | 90 | int SFLOWPS_send (SFLOWPS *pst); 91 | 92 | #endif /* __included_sflow_psample_h__ */ 93 | 94 | /* 95 | * fd.io coding-style-patch-verification: ON 96 | * 97 | * Local Variables: 98 | * eval: (c-set-style "gnu") 99 | * End: 100 | */ 101 | -------------------------------------------------------------------------------- /sflow/sflow_psample_fields.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2024 InMon Corp. 3 | * Licensed under the Apache License, Version 2.0 (the "License"); 4 | * you may not use this file except in compliance with the License. 5 | * You may obtain a copy of the License at: 6 | * 7 | * http://www.apache.org/licenses/LICENSE-2.0 8 | * 9 | * Unless required by applicable law or agreed to in writing, software 10 | * distributed under the License is distributed on an "AS IS" BASIS, 11 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | * See the License for the specific language governing permissions and 13 | * limitations under the License. 14 | */ 15 | 16 | SFLOWPS_FIELDDATA (SFLOWPS_PSAMPLE_ATTR_IIFINDEX, 4, "input if_index") 17 | SFLOWPS_FIELDDATA (SFLOWPS_PSAMPLE_ATTR_OIFINDEX, 4, "output if_index") 18 | SFLOWPS_FIELDDATA (SFLOWPS_PSAMPLE_ATTR_ORIGSIZE, 4, "original packet size") 19 | SFLOWPS_FIELDDATA (SFLOWPS_PSAMPLE_ATTR_SAMPLE_GROUP, 4, "group number") 20 | SFLOWPS_FIELDDATA (SFLOWPS_PSAMPLE_ATTR_GROUP_SEQ, 4, "group sequence number") 21 | SFLOWPS_FIELDDATA (SFLOWPS_PSAMPLE_ATTR_SAMPLE_RATE, 4, "sampling N") 22 | SFLOWPS_FIELDDATA (SFLOWPS_PSAMPLE_ATTR_DATA, 0, "sampled header") 23 | SFLOWPS_FIELDDATA (SFLOWPS_PSAMPLE_ATTR_TUNNEL, 0, "tunnel header") 24 | 25 | /* commands attributes */ 26 | SFLOWPS_FIELDDATA (SFLOWPS_PSAMPLE_ATTR_GROUP_REFCOUNT, 0, 27 | "group reference count") 28 | 29 | SFLOWPS_FIELDDATA (SFLOWPS_PSAMPLE_ATTR_PAD, 0, "pad bytes") 30 | SFLOWPS_FIELDDATA (SFLOWPS_PSAMPLE_ATTR_OUT_TC, 2, "egress queue number") 31 | SFLOWPS_FIELDDATA (SFLOWPS_PSAMPLE_ATTR_OUT_TC_OCC, 8, 32 | "egress queue depth in bytes") 33 | SFLOWPS_FIELDDATA (SFLOWPS_PSAMPLE_ATTR_LATENCY, 8, 34 | "transit latency in nanoseconds") 35 | SFLOWPS_FIELDDATA (SFLOWPS_PSAMPLE_ATTR_TIMESTAMP, 8, "timestamp") 36 | SFLOWPS_FIELDDATA (SFLOWPS_PSAMPLE_ATTR_PROTO, 2, "header protocol") 37 | -------------------------------------------------------------------------------- /sflow/sflow_test.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2024 InMon Corp. 3 | * Licensed under the Apache License, Version 2.0 (the "License"); 4 | * you may not use this file except in compliance with the License. 5 | * You may obtain a copy of the License at: 6 | * 7 | * http://www.apache.org/licenses/LICENSE-2.0 8 | * 9 | * Unless required by applicable law or agreed to in writing, software 10 | * distributed under the License is distributed on an "AS IS" BASIS, 11 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | * See the License for the specific language governing permissions and 13 | * limitations under the License. 14 | */ 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | 21 | #define __plugin_msg_base sflow_test_main.msg_id_base 22 | #include 23 | 24 | uword unformat_sw_if_index (unformat_input_t *input, va_list *args); 25 | 26 | /* Declare message IDs */ 27 | #include 28 | #include 29 | 30 | /* for token names */ 31 | #include 32 | 33 | typedef struct 34 | { 35 | /* API message ID base */ 36 | u16 msg_id_base; 37 | vat_main_t *vat_main; 38 | } sflow_test_main_t; 39 | 40 | sflow_test_main_t sflow_test_main; 41 | 42 | static int 43 | api_sflow_enable_disable (vat_main_t *vam) 44 | { 45 | unformat_input_t *i = vam->input; 46 | int enable_disable = true; 47 | u32 hw_if_index = ~0; 48 | vl_api_sflow_enable_disable_t *mp; 49 | int ret; 50 | 51 | /* Parse args required to build the message */ 52 | while (unformat_check_input (i) != UNFORMAT_END_OF_INPUT) 53 | { 54 | if (unformat (i, "%U", unformat_sw_if_index, vam, &hw_if_index)) 55 | ; 56 | else if (unformat (i, "disable")) 57 | enable_disable = false; 58 | else if (unformat (i, "enable")) 59 | enable_disable = true; 60 | else 61 | break; 62 | } 63 | 64 | if (hw_if_index == ~0) 65 | { 66 | errmsg ("missing interface name / explicit hw_if_index number \n"); 67 | return -99; 68 | } 69 | 70 | /* Construct the API message */ 71 | M (SFLOW_ENABLE_DISABLE, mp); 72 | mp->hw_if_index = ntohl (hw_if_index); 73 | mp->enable_disable = enable_disable; 74 | 75 | /* send it... */ 76 | S (mp); 77 | 78 | /* Wait for a reply... */ 79 | W (ret); 80 | return ret; 81 | } 82 | 83 | static void 84 | vl_api_sflow_sampling_rate_get_reply_t_handler ( 85 | vl_api_sflow_sampling_rate_get_reply_t *mp) 86 | { 87 | vat_main_t *vam = sflow_test_main.vat_main; 88 | clib_warning ("sflow sampling_N: %d", ntohl (mp->sampling_N)); 89 | vam->result_ready = 1; 90 | } 91 | 92 | static int 93 | api_sflow_sampling_rate_get (vat_main_t *vam) 94 | { 95 | vl_api_sflow_sampling_rate_get_t *mp; 96 | int ret; 97 | 98 | /* Construct the API message */ 99 | M (SFLOW_SAMPLING_RATE_GET, mp); 100 | 101 | /* send it... */ 102 | S (mp); 103 | 104 | /* Wait for a reply... */ 105 | W (ret); 106 | return ret; 107 | } 108 | 109 | static int 110 | api_sflow_sampling_rate_set (vat_main_t *vam) 111 | { 112 | unformat_input_t *i = vam->input; 113 | u32 sampling_N = ~0; 114 | vl_api_sflow_sampling_rate_set_t *mp; 115 | int ret; 116 | 117 | /* Parse args required to build the message */ 118 | while (unformat_check_input (i) != UNFORMAT_END_OF_INPUT) 119 | { 120 | if (unformat (i, "sampling_N %d", &sampling_N)) 121 | ; 122 | else 123 | break; 124 | } 125 | 126 | if (sampling_N == ~0) 127 | { 128 | errmsg ("missing sampling_N number \n"); 129 | return -99; 130 | } 131 | 132 | /* Construct the API message */ 133 | M (SFLOW_SAMPLING_RATE_SET, mp); 134 | mp->sampling_N = ntohl (sampling_N); 135 | 136 | /* send it... */ 137 | S (mp); 138 | 139 | /* Wait for a reply... */ 140 | W (ret); 141 | return ret; 142 | } 143 | 144 | static void 145 | vl_api_sflow_polling_interval_get_reply_t_handler ( 146 | vl_api_sflow_polling_interval_get_reply_t *mp) 147 | { 148 | vat_main_t *vam = sflow_test_main.vat_main; 149 | clib_warning ("sflow polling-interval: %d", ntohl (mp->polling_S)); 150 | vam->result_ready = 1; 151 | } 152 | 153 | static int 154 | api_sflow_polling_interval_get (vat_main_t *vam) 155 | { 156 | vl_api_sflow_polling_interval_get_t *mp; 157 | int ret; 158 | 159 | /* Construct the API message */ 160 | M (SFLOW_POLLING_INTERVAL_GET, mp); 161 | 162 | /* send it... */ 163 | S (mp); 164 | 165 | /* Wait for a reply... */ 166 | W (ret); 167 | return ret; 168 | } 169 | 170 | static int 171 | api_sflow_polling_interval_set (vat_main_t *vam) 172 | { 173 | unformat_input_t *i = vam->input; 174 | u32 polling_S = ~0; 175 | vl_api_sflow_polling_interval_set_t *mp; 176 | int ret; 177 | 178 | /* Parse args required to build the message */ 179 | while (unformat_check_input (i) != UNFORMAT_END_OF_INPUT) 180 | { 181 | if (unformat (i, "polling_S %d", &polling_S)) 182 | ; 183 | else 184 | break; 185 | } 186 | 187 | if (polling_S == ~0) 188 | { 189 | errmsg ("missing polling_S number \n"); 190 | return -99; 191 | } 192 | 193 | /* Construct the API message */ 194 | M (SFLOW_POLLING_INTERVAL_SET, mp); 195 | mp->polling_S = ntohl (polling_S); 196 | 197 | /* send it... */ 198 | S (mp); 199 | 200 | /* Wait for a reply... */ 201 | W (ret); 202 | return ret; 203 | } 204 | 205 | static void 206 | vl_api_sflow_header_bytes_get_reply_t_handler ( 207 | vl_api_sflow_header_bytes_get_reply_t *mp) 208 | { 209 | vat_main_t *vam = sflow_test_main.vat_main; 210 | clib_warning ("sflow header-bytes: %d", ntohl (mp->header_B)); 211 | vam->result_ready = 1; 212 | } 213 | 214 | static int 215 | api_sflow_header_bytes_get (vat_main_t *vam) 216 | { 217 | vl_api_sflow_header_bytes_get_t *mp; 218 | int ret; 219 | 220 | /* Construct the API message */ 221 | M (SFLOW_HEADER_BYTES_GET, mp); 222 | 223 | /* send it... */ 224 | S (mp); 225 | 226 | /* Wait for a reply... */ 227 | W (ret); 228 | return ret; 229 | } 230 | 231 | static int 232 | api_sflow_header_bytes_set (vat_main_t *vam) 233 | { 234 | unformat_input_t *i = vam->input; 235 | u32 header_B = ~0; 236 | vl_api_sflow_header_bytes_set_t *mp; 237 | int ret; 238 | 239 | /* Parse args required to build the message */ 240 | while (unformat_check_input (i) != UNFORMAT_END_OF_INPUT) 241 | { 242 | if (unformat (i, "header_B %d", &header_B)) 243 | ; 244 | else 245 | break; 246 | } 247 | 248 | if (header_B == ~0) 249 | { 250 | errmsg ("missing header_B number \n"); 251 | return -99; 252 | } 253 | 254 | /* Construct the API message */ 255 | M (SFLOW_HEADER_BYTES_SET, mp); 256 | mp->header_B = ntohl (header_B); 257 | 258 | /* send it... */ 259 | S (mp); 260 | 261 | /* Wait for a reply... */ 262 | W (ret); 263 | return ret; 264 | } 265 | 266 | static void 267 | vl_api_sflow_direction_get_reply_t_handler ( 268 | vl_api_sflow_direction_get_reply_t *mp) 269 | { 270 | vat_main_t *vam = sflow_test_main.vat_main; 271 | clib_warning ("sflow direction: %d", ntohl (mp->sampling_D)); 272 | vam->result_ready = 1; 273 | } 274 | 275 | static int 276 | api_sflow_direction_get (vat_main_t *vam) 277 | { 278 | vl_api_sflow_direction_get_t *mp; 279 | int ret; 280 | 281 | /* Construct the API message */ 282 | M (SFLOW_DIRECTION_GET, mp); 283 | 284 | /* send it... */ 285 | S (mp); 286 | 287 | /* Wait for a reply... */ 288 | W (ret); 289 | return ret; 290 | } 291 | 292 | static int 293 | api_sflow_direction_set (vat_main_t *vam) 294 | { 295 | unformat_input_t *i = vam->input; 296 | sflow_direction_t sampling_D = SFLOW_DIRN_UNDEFINED; 297 | vl_api_sflow_direction_set_t *mp; 298 | int ret; 299 | 300 | /* Parse args required to build the message */ 301 | while (unformat_check_input (i) != UNFORMAT_END_OF_INPUT) 302 | { 303 | if (unformat (i, "sampling_D rx")) 304 | sampling_D = SFLOW_DIRN_INGRESS; 305 | else if (unformat (i, "sampling_D tx")) 306 | sampling_D = SFLOW_DIRN_INGRESS; 307 | else if (unformat (i, "sampling_D both")) 308 | sampling_D = SFLOW_DIRN_BOTH; 309 | else 310 | break; 311 | } 312 | 313 | if (sampling_D == SFLOW_DIRN_UNDEFINED) 314 | { 315 | errmsg ("missing sampling_D direction\n"); 316 | return -99; 317 | } 318 | 319 | /* Construct the API message */ 320 | M (SFLOW_DIRECTION_SET, mp); 321 | mp->sampling_D = ntohl (sampling_D); 322 | 323 | /* send it... */ 324 | S (mp); 325 | 326 | /* Wait for a reply... */ 327 | W (ret); 328 | return ret; 329 | } 330 | 331 | static void 332 | vl_api_sflow_drop_monitoring_get_reply_t_handler ( 333 | vl_api_sflow_drop_monitoring_get_reply_t *mp) 334 | { 335 | vat_main_t *vam = sflow_test_main.vat_main; 336 | clib_warning ("sflow drop_M: %d", mp->drop_M); 337 | vam->result_ready = 1; 338 | } 339 | 340 | static int 341 | api_sflow_drop_monitoring_get (vat_main_t *vam) 342 | { 343 | vl_api_sflow_drop_monitoring_get_t *mp; 344 | int ret; 345 | 346 | /* Construct the API message */ 347 | M (SFLOW_DROP_MONITORING_GET, mp); 348 | 349 | /* send it... */ 350 | S (mp); 351 | 352 | /* Wait for a reply... */ 353 | W (ret); 354 | return ret; 355 | } 356 | 357 | static int 358 | api_sflow_drop_monitoring_set (vat_main_t *vam) 359 | { 360 | unformat_input_t *i = vam->input; 361 | u32 drop_M = 1; 362 | vl_api_sflow_drop_monitoring_set_t *mp; 363 | int ret; 364 | 365 | /* Parse args required to build the message */ 366 | while (unformat_check_input (i) != UNFORMAT_END_OF_INPUT) 367 | { 368 | if (unformat (i, "disable")) 369 | drop_M = 0; 370 | else if (unformat (i, "enable")) 371 | drop_M = 1; 372 | else 373 | break; 374 | } 375 | 376 | /* Construct the API message */ 377 | M (SFLOW_DROP_MONITORING_SET, mp); 378 | mp->drop_M = ntohl (drop_M); 379 | 380 | /* send it... */ 381 | S (mp); 382 | 383 | /* Wait for a reply... */ 384 | W (ret); 385 | return ret; 386 | } 387 | 388 | static void 389 | vl_api_sflow_interface_details_t_handler (vl_api_sflow_interface_details_t *mp) 390 | { 391 | vat_main_t *vam = sflow_test_main.vat_main; 392 | clib_warning ("sflow enable: %d", ntohl (mp->hw_if_index)); 393 | vam->result_ready = 1; 394 | } 395 | 396 | static int 397 | api_sflow_interface_dump (vat_main_t *vam) 398 | { 399 | vl_api_sflow_interface_dump_t *mp; 400 | int ret; 401 | 402 | /* Construct the API message */ 403 | M (SFLOW_INTERFACE_DUMP, mp); 404 | 405 | /* send it... */ 406 | S (mp); 407 | 408 | /* Wait for a reply... */ 409 | W (ret); 410 | return ret; 411 | } 412 | 413 | /* 414 | * List of messages that the sflow test plugin sends, 415 | * and that the data plane plugin processes 416 | */ 417 | #include 418 | 419 | /* 420 | * fd.io coding-style-patch-verification: ON 421 | * 422 | * Local Variables: 423 | * eval: (c-set-style "gnu") 424 | * End: 425 | */ 426 | -------------------------------------------------------------------------------- /sflow/sflow_usersock.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2024 InMon Corp. 3 | * Licensed under the Apache License, Version 2.0 (the "License"); 4 | * you may not use this file except in compliance with the License. 5 | * You may obtain a copy of the License at: 6 | * 7 | * http://www.apache.org/licenses/LICENSE-2.0 8 | * 9 | * Unless required by applicable law or agreed to in writing, software 10 | * distributed under the License is distributed on an "AS IS" BASIS, 11 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | * See the License for the specific language governing permissions and 13 | * limitations under the License. 14 | */ 15 | 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | 22 | #include 23 | #include 24 | #include 25 | #include 26 | #include 27 | #include 28 | #include 29 | 30 | #include 31 | #include 32 | 33 | /*_________________---------------------------__________________ 34 | _________________ SFLOWUS_init __________________ 35 | -----------------___________________________------------------ 36 | */ 37 | 38 | EnumSFLOWNLState 39 | SFLOWUS_init (SFLOWUS *ust) 40 | { 41 | ust->nl.id = SFLOWNL_USERSOCK; 42 | ust->nl.group_id = SFLOW_NETLINK_USERSOCK_MULTICAST; 43 | ust->nl.attr = ust->attr; 44 | ust->nl.attr_max = SFLOWUS_ATTRS - 1; 45 | ust->nl.iov = ust->iov; 46 | ust->nl.iov_max = SFLOWUS_IOV_FRAGS - 1; 47 | ust->nl.state = SFLOWNL_STATE_INIT; 48 | return ust->nl.state; 49 | } 50 | 51 | /*_________________---------------------------__________________ 52 | _________________ SFLOWUS_open __________________ 53 | -----------------___________________________------------------ 54 | */ 55 | 56 | bool 57 | SFLOWUS_open (SFLOWUS *ust) 58 | { 59 | if (ust->nl.state == SFLOWNL_STATE_UNDEFINED) 60 | SFLOWUS_init (ust); 61 | if (ust->nl.nl_sock == 0) 62 | sflow_netlink_usersock_open (&ust->nl); 63 | if (ust->nl.nl_sock > 0) 64 | { 65 | ust->nl.state = SFLOWNL_STATE_READY; 66 | return true; 67 | } 68 | return false; 69 | } 70 | 71 | /*_________________---------------------------__________________ 72 | _________________ SFLOWUS_close __________________ 73 | -----------------___________________________------------------ 74 | */ 75 | 76 | bool 77 | SFLOWUS_close (SFLOWUS *ust) 78 | { 79 | return (sflow_netlink_close (&ust->nl) == 0); 80 | } 81 | 82 | /*_________________---------------------------__________________ 83 | _________________ SFLOWUS_set_msg_type __________________ 84 | -----------------___________________________------------------ 85 | */ 86 | 87 | bool 88 | SFLOWUS_set_msg_type (SFLOWUS *ust, EnumSFlowVppMsgType msgType) 89 | { 90 | ust->nl.nlh.nlmsg_type = msgType; 91 | return true; 92 | } 93 | 94 | /*_________________---------------------------__________________ 95 | _________________ SFLOWUS_open_step __________________ 96 | -----------------___________________________------------------ 97 | */ 98 | 99 | EnumSFLOWNLState 100 | SFLOWUS_open_step (SFLOWUS *ust) 101 | { 102 | switch (ust->nl.state) 103 | { 104 | case SFLOWNL_STATE_UNDEFINED: 105 | SFLOWUS_init (ust); 106 | break; 107 | case SFLOWNL_STATE_INIT: 108 | SFLOWUS_open (ust); 109 | break; 110 | case SFLOWNL_STATE_OPEN: 111 | case SFLOWNL_STATE_WAIT_FAMILY: 112 | case SFLOWNL_STATE_READY: 113 | break; 114 | } 115 | return ust->nl.state; 116 | } 117 | 118 | /*_________________---------------------------__________________ 119 | _________________ SFLOWUS_set_attr __________________ 120 | -----------------___________________________------------------ 121 | */ 122 | 123 | bool 124 | SFLOWUS_set_attr (SFLOWUS *ust, EnumSFlowVppAttributes field, void *val, 125 | int len) 126 | { 127 | return sflow_netlink_set_attr (&ust->nl, field, val, len); 128 | } 129 | 130 | /*_________________---------------------------__________________ 131 | _________________ SFLOWUS_send __________________ 132 | -----------------___________________________------------------ 133 | */ 134 | 135 | int 136 | SFLOWUS_send (SFLOWUS *ust) 137 | { 138 | int status = sflow_netlink_send_attrs (&ust->nl, false); 139 | sflow_netlink_reset_attrs (&ust->nl); 140 | if (status <= 0) 141 | { 142 | // Linux replies with ECONNREFUSED when 143 | // a multicast is sent via NETLINK_USERSOCK, but 144 | // it's not an error so we can just ignore it here. 145 | if (errno != ECONNREFUSED) 146 | { 147 | SFLOW_DBG ("USERSOCK strerror(errno) = %s\n", strerror (errno)); 148 | return -1; 149 | } 150 | } 151 | return 0; 152 | } 153 | 154 | /* 155 | * fd.io coding-style-patch-verification: ON 156 | * 157 | * Local Variables: 158 | * eval: (c-set-style "gnu") 159 | * End: 160 | */ 161 | -------------------------------------------------------------------------------- /sflow/sflow_usersock.h: -------------------------------------------------------------------------------- 1 | /* 2 | * Copyright (c) 2024 InMon Corp. 3 | * Licensed under the Apache License, Version 2.0 (the "License"); 4 | * you may not use this file except in compliance with the License. 5 | * You may obtain a copy of the License at: 6 | * 7 | * http://www.apache.org/licenses/LICENSE-2.0 8 | * 9 | * Unless required by applicable law or agreed to in writing, software 10 | * distributed under the License is distributed on an "AS IS" BASIS, 11 | * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | * See the License for the specific language governing permissions and 13 | * limitations under the License. 14 | */ 15 | 16 | #ifndef __included_sflow_usersock_h__ 17 | #define __included_sflow_usersock_h__ 18 | 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include 24 | 25 | #include 26 | #include 27 | #include 28 | #include 29 | #include 30 | #include 31 | 32 | #include 33 | 34 | // ==================== shared with hsflowd mod_vpp ========================= 35 | // See https://github.com/sflow/host-sflow 36 | 37 | #define SFLOW_VPP_NETLINK_USERSOCK_MULTICAST 29 38 | 39 | typedef enum 40 | { 41 | SFLOW_VPP_MSG_STATUS = 1, 42 | SFLOW_VPP_MSG_IF_COUNTERS 43 | } EnumSFlowVppMsgType; 44 | 45 | typedef enum 46 | { 47 | SFLOW_VPP_ATTR_PORTNAME, /* string */ 48 | SFLOW_VPP_ATTR_IFINDEX, /* u32 */ 49 | SFLOW_VPP_ATTR_IFTYPE, /* u32 */ 50 | SFLOW_VPP_ATTR_IFSPEED, /* u64 */ 51 | SFLOW_VPP_ATTR_IFDIRECTION, /* u32 */ 52 | SFLOW_VPP_ATTR_OPER_UP, /* u32 */ 53 | SFLOW_VPP_ATTR_ADMIN_UP, /* u32 */ 54 | SFLOW_VPP_ATTR_RX_OCTETS, /* u64 */ 55 | SFLOW_VPP_ATTR_TX_OCTETS, /* u64 */ 56 | SFLOW_VPP_ATTR_RX_PKTS, /* u64 */ 57 | SFLOW_VPP_ATTR_TX_PKTS, /* u64 */ 58 | SFLOW_VPP_ATTR_RX_BCASTS, /* u64 */ 59 | SFLOW_VPP_ATTR_TX_BCASTS, /* u64 */ 60 | SFLOW_VPP_ATTR_RX_MCASTS, /* u64 */ 61 | SFLOW_VPP_ATTR_TX_MCASTS, /* u64 */ 62 | SFLOW_VPP_ATTR_RX_DISCARDS, /* u64 */ 63 | SFLOW_VPP_ATTR_TX_DISCARDS, /* u64 */ 64 | SFLOW_VPP_ATTR_RX_ERRORS, /* u64 */ 65 | SFLOW_VPP_ATTR_TX_ERRORS, /* u64 */ 66 | SFLOW_VPP_ATTR_HW_ADDRESS, /* binary */ 67 | SFLOW_VPP_ATTR_UPTIME_S, /* u32 */ 68 | SFLOW_VPP_ATTR_OSINDEX, /* u32 Linux ifIndex number, where applicable */ 69 | SFLOW_VPP_ATTR_DROPS, /* u32 all FIFO and netlink sendmsg drops */ 70 | SFLOW_VPP_ATTR_SEQ, /* u32 send seq no */ 71 | /* enum shared with hsflowd, so only add here */ 72 | __SFLOW_VPP_ATTRS 73 | } EnumSFlowVppAttributes; 74 | 75 | #define SFLOW_VPP_PSAMPLE_GROUP_INGRESS 3 76 | #define SFLOW_VPP_PSAMPLE_GROUP_EGRESS 4 77 | 78 | // ========================================================================= 79 | typedef struct 80 | { 81 | u64 byts; 82 | u64 pkts; 83 | u64 m_pkts; 84 | u64 b_pkts; 85 | u64 errs; 86 | u64 drps; 87 | } sflow_ctrs_t; 88 | 89 | typedef struct 90 | { 91 | sflow_ctrs_t tx; 92 | sflow_ctrs_t rx; 93 | } sflow_counters_t; 94 | 95 | typedef struct _SFLOWUS_field_t 96 | { 97 | EnumSFlowVppAttributes field; 98 | int len; 99 | } SFLOWUS_field_t; 100 | 101 | #define SFLOWUS_ATTRS __SFLOW_VPP_ATTRS 102 | #define SFLOWUS_IOV_FRAGS \ 103 | ((2 * SFLOWUS_ATTRS) + 2) // TODO: may only be +1 -- no ge header? 104 | 105 | typedef struct _SFLOWUS 106 | { 107 | SFLOWNL nl; 108 | SFLOWNLAttr attr[__SFLOW_VPP_ATTRS]; 109 | struct iovec iov[SFLOWUS_IOV_FRAGS]; 110 | } SFLOWUS; 111 | 112 | EnumSFLOWNLState SFLOWUS_init (SFLOWUS *ust); 113 | bool SFLOWUS_open (SFLOWUS *ust); 114 | bool SFLOWUS_close (SFLOWUS *ust); 115 | 116 | bool SFLOWUS_set_msg_type (SFLOWUS *ust, EnumSFlowVppMsgType type); 117 | bool SFLOWUS_set_attr (SFLOWUS *ust, EnumSFlowVppAttributes field, void *buf, 118 | int len); 119 | #define SFLOWUS_set_attr_int(ust, field, val) \ 120 | SFLOWUS_set_attr ((ust), (field), &(val), sizeof (val)) 121 | 122 | int SFLOWUS_send (SFLOWUS *ust); 123 | 124 | #endif /* __included_sflow_usersock_h__ */ 125 | 126 | /* 127 | * fd.io coding-style-patch-verification: ON 128 | * 129 | * Local Variables: 130 | * eval: (c-set-style "gnu") 131 | * End: 132 | */ 133 | -------------------------------------------------------------------------------- /shmtest.c: -------------------------------------------------------------------------------- 1 | #include 2 | #include 3 | #include 4 | #include 5 | #include 6 | #include 7 | #include 8 | #include 9 | #include 10 | 11 | #define errExit(msg) do { perror(msg); exit(EXIT_FAILURE); } while (0) 12 | 13 | #define SHM_SIZE 2000000 14 | #define SFLOW_SHM_PATH "/vpp-sflow-counters-1" 15 | 16 | int 17 | main(int argc, char *argv[]) 18 | { 19 | int fd = shm_open(SFLOW_SHM_PATH, O_RDWR, 0); 20 | if (fd == -1) 21 | errExit("shm_open"); 22 | uint64_t *datap = mmap(NULL, SHM_SIZE, 23 | PROT_READ | PROT_WRITE, 24 | MAP_SHARED, fd, 0); 25 | if (datap == MAP_FAILED) 26 | errExit("mmap"); 27 | for(int ii = 0; ii < 32; ii++) 28 | printf("%u: =%"PRIu64"\n", ii, datap[ii]); 29 | exit(EXIT_SUCCESS); 30 | } 31 | -------------------------------------------------------------------------------- /test/test_sflow.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import unittest 4 | from framework import VppTestCase 5 | from asfframework import VppTestRunner 6 | from scapy.layers.l2 import Ether 7 | from scapy.packet import Raw 8 | from scapy.layers.inet import IP, UDP 9 | from random import randint 10 | import re # for finding counters in "sh errors" output 11 | 12 | 13 | class SFlowTestCase(VppTestCase): 14 | """sFlow test case""" 15 | 16 | @classmethod 17 | def setUpClass(self): 18 | super(SFlowTestCase, self).setUpClass() 19 | 20 | @classmethod 21 | def teadDownClass(cls): 22 | super(SFlowTestCase, cls).tearDownClass() 23 | 24 | def setUp(self): 25 | self.create_pg_interfaces(range(2)) # create pg0 and pg1 26 | for i in self.pg_interfaces: 27 | i.admin_up() # put the interface up 28 | i.config_ip4() # configure IPv4 address on the interface 29 | i.resolve_arp() # resolve ARP, so that we know VPP MAC 30 | 31 | def tearDown(self): 32 | for i in self.pg_interfaces: 33 | i.admin_down() 34 | i.unconfig() 35 | i.set_table_ip4(0) 36 | i.set_table_ip6(0) 37 | 38 | def is_hw_interface_in_dump(self, dump, hw_if_index): 39 | for i in dump: 40 | if i.hw_if_index == hw_if_index: 41 | return True 42 | else: 43 | return False 44 | 45 | def enable_sflow_via_api(self): 46 | ## TEST: Enable one interface 47 | ret = self.vapi.sflow_enable_disable(hw_if_index=1, enable_disable=True) 48 | self.assertEqual(ret.retval, 0) 49 | 50 | ## TEST: interface dump all 51 | ret = self.vapi.sflow_interface_dump() 52 | self.assertTrue(self.is_hw_interface_in_dump(ret, 1)) 53 | 54 | ## TEST: Disable one interface 55 | ret = self.vapi.sflow_enable_disable(hw_if_index=1, enable_disable=False) 56 | self.assertEqual(ret.retval, 0) 57 | 58 | ## TEST: interface dump all after enable + disable 59 | ret = self.vapi.sflow_interface_dump() 60 | self.assertEqual(len(ret), 0) 61 | 62 | ## TEST: Enable both interfaces 63 | ret = self.vapi.sflow_enable_disable(hw_if_index=1, enable_disable=True) 64 | self.assertEqual(ret.retval, 0) 65 | ret = self.vapi.sflow_enable_disable(hw_if_index=2, enable_disable=True) 66 | self.assertEqual(ret.retval, 0) 67 | 68 | ## TEST: interface dump all 69 | ret = self.vapi.sflow_interface_dump() 70 | self.assertTrue(self.is_hw_interface_in_dump(ret, 1)) 71 | self.assertTrue(self.is_hw_interface_in_dump(ret, 2)) 72 | 73 | ## TEST: the default sampling rate 74 | ret = self.vapi.sflow_sampling_rate_get() 75 | self.assert_equal(ret.sampling_N, 10000) 76 | 77 | ## TEST: sflow_sampling_rate_set() 78 | self.vapi.sflow_sampling_rate_set(sampling_N=1) 79 | ret = self.vapi.sflow_sampling_rate_get() 80 | self.assert_equal(ret.sampling_N, 1) 81 | 82 | ## TEST: the default polling interval 83 | ret = self.vapi.sflow_polling_interval_get() 84 | self.assert_equal(ret.polling_S, 20) 85 | 86 | ## TEST: sflow_polling_interval_set() 87 | self.vapi.sflow_polling_interval_set(polling_S=10) 88 | ret = self.vapi.sflow_polling_interval_get() 89 | self.assert_equal(ret.polling_S, 10) 90 | 91 | ## TEST: the default header bytes 92 | ret = self.vapi.sflow_header_bytes_get() 93 | self.assert_equal(ret.header_B, 128) 94 | 95 | ## TEST: sflow_header_bytes_set() 96 | self.vapi.sflow_header_bytes_set(header_B=96) 97 | ret = self.vapi.sflow_header_bytes_get() 98 | self.assert_equal(ret.header_B, 96) 99 | 100 | def create_stream(self, src_if, dst_if, count): 101 | packets = [] 102 | for i in range(count): 103 | # create packet info stored in the test case instance 104 | info = self.create_packet_info(src_if, dst_if) 105 | # convert the info into packet payload 106 | payload = self.info_to_payload(info) 107 | # create the packet itself 108 | p = ( 109 | Ether(dst=src_if.local_mac, src=src_if.remote_mac) 110 | / IP(src=src_if.remote_ip4, dst=dst_if.remote_ip4) 111 | / UDP(sport=randint(49152, 65535), dport=5678) 112 | / Raw(payload) 113 | ) 114 | # store a copy of the packet in the packet info 115 | info.data = p.copy() 116 | # append the packet to the list 117 | packets.append(p) 118 | # return the created packet list 119 | return packets 120 | 121 | def verify_capture(self, src_if, dst_if, capture): 122 | packet_info = None 123 | for packet in capture: 124 | try: 125 | ip = packet[IP] 126 | udp = packet[UDP] 127 | # convert the payload to packet info object 128 | payload_info = self.payload_to_info(packet[Raw]) 129 | # make sure the indexes match 130 | self.assert_equal( 131 | payload_info.src, src_if.sw_if_index, "source sw_if_index" 132 | ) 133 | self.assert_equal( 134 | payload_info.dst, dst_if.sw_if_index, "destination sw_if_index" 135 | ) 136 | packet_info = self.get_next_packet_info_for_interface2( 137 | src_if.sw_if_index, dst_if.sw_if_index, packet_info 138 | ) 139 | # make sure we didn't run out of saved packets 140 | self.assertIsNotNone(packet_info) 141 | self.assert_equal( 142 | payload_info.index, packet_info.index, "packet info index" 143 | ) 144 | saved_packet = packet_info.data # fetch the saved packet 145 | # assert the values match 146 | self.assert_equal(ip.src, saved_packet[IP].src, "IP source address") 147 | self.assert_equal(udp.sport, saved_packet[UDP].sport, "UDP source port") 148 | except: 149 | self.logger.error("Unexpected or invalid packet:", packet) 150 | raise 151 | remaining_packet = self.get_next_packet_info_for_interface2( 152 | src_if.sw_if_index, dst_if.sw_if_index, packet_info 153 | ) 154 | self.assertIsNone( 155 | remaining_packet, 156 | "Interface %s: Packet expected from interface " 157 | "%s didn't arrive" % (dst_if.name, src_if.name), 158 | ) 159 | 160 | def get_sflow_counter(self, counter): 161 | counters = self.vapi.cli("sh errors").split("\n") 162 | for i in range(1, len(counters) - 1): 163 | results = counters[i].split() 164 | if results[1] == "sflow": 165 | if re.search(counter, counters[i]) is not None: 166 | return int(results[0]) 167 | return None 168 | 169 | def verify_sflow(self, count): 170 | ctr_processed = "sflow packets processed" 171 | ctr_sampled = "sflow packets sampled" 172 | ctr_dropped = "sflow packets dropped" 173 | ctr_ps_sent = "sflow PSAMPLE sent" 174 | ctr_ps_fail = "sflow PSAMPLE send failed" 175 | processed = self.get_sflow_counter(ctr_processed) 176 | sampled = self.get_sflow_counter(ctr_sampled) 177 | dropped = self.get_sflow_counter(ctr_dropped) 178 | ps_sent = self.get_sflow_counter(ctr_ps_sent) 179 | ps_fail = self.get_sflow_counter(ctr_ps_fail) 180 | self.assert_equal(processed, count, ctr_processed) 181 | self.assert_equal(sampled, count, ctr_sampled) 182 | self.assert_equal(dropped, None, ctr_dropped) 183 | # TODO decide how to warn if PSAMPLE is not working 184 | # It requires a prior "sudo modprobe psample", but 185 | # that should probably be done at system boot time 186 | # or maybe in a systemctl startup script, so we 187 | # should only warn here. 188 | self.logger.info(ctr_ps_sent + "=" + str(ps_sent)) 189 | self.logger.info(ctr_ps_fail + "=" + str(ps_fail)) 190 | 191 | def test_basic(self): 192 | self.enable_sflow_via_api() 193 | count = 7 194 | # create the packet stream 195 | packets = self.create_stream(self.pg0, self.pg1, count) 196 | # add the stream to the source interface 197 | self.pg0.add_stream(packets) 198 | # enable capture on both interfaces 199 | self.pg0.enable_capture() 200 | self.pg1.enable_capture() 201 | # start the packet generator 202 | self.pg_start() 203 | # get capture - the proper count of packets was saved by 204 | # create_packet_info() based on dst_if parameter 205 | capture = self.pg1.get_capture() 206 | # assert nothing captured on pg0 (always do this last, so that 207 | # some time has already passed since pg_start()) 208 | self.pg0.assert_nothing_captured() 209 | # verify capture 210 | self.verify_capture(self.pg0, self.pg1, capture) 211 | # verify sflow counters 212 | self.verify_sflow(count) 213 | -------------------------------------------------------------------------------- /veth_pair_test.sh: -------------------------------------------------------------------------------- 1 | # The following creates two veth pairs p1-p2 and p3-p4, 2 | # then bridges p1 and p3 together in vpp, 3 | # then assigns IP addresses to p2 and p4 and puts 4 | # one of them in a separate namespace so that when 5 | # they ping each other the ICMP packets go through VPP. 6 | 7 | ip link add dev p1 type veth peer name p2 8 | ip link add dev p3 type veth peer name p4 9 | ip link set dev p1 up 10 | ip link set dev p3 up 11 | ip link set dev p4 up 12 | ip netns add ns2 13 | ip link set p2 netns ns2 14 | ip -n ns2 link set dev p2 up 15 | ip -n ns2 addr add 172.16.111.2/24 dev p2 16 | ip addr add 172.16.111.4/24 dev p4 17 | 18 | vppctl create host-interface name p1 19 | vppctl create host-interface name p3 20 | vppctl set interface state host-p1 up 21 | vppctl set interface state host-p3 up 22 | vppctl set interface l2 bridge host-p1 1 23 | vppctl set interface l2 bridge host-p3 1 24 | vppctl sflow enable host-p1 25 | vppctl sflow enable host-p3 26 | vppctl sflow sampling 10 27 | 28 | # then test with "sudo ip netns exec ns2 ping -i 0.01 172.16.111.4" 29 | --------------------------------------------------------------------------------