├── .gitignore ├── CONTRIBUTING.md ├── LICENSE ├── Makefile ├── README.md ├── af_smc.7 ├── dev.c ├── dev.h ├── info.c ├── info.h ├── libnetlink.c ├── libnetlink.h ├── linkgroup.c ├── linkgroup.h ├── seid.c ├── seid.h ├── smc-device.8 ├── smc-linkgroup.8 ├── smc-preload.c ├── smc-tools.autocomplete ├── smc.8 ├── smc.c ├── smc_chk ├── smc_chk.8 ├── smc_dbg ├── smc_pnet.8 ├── smc_pnet.c ├── smc_rnics ├── smc_rnics.8 ├── smc_run ├── smc_run.8 ├── smcd-device.8 ├── smcd-info.8 ├── smcd-linkgroup.8 ├── smcd-seid.8 ├── smcd-stats.8 ├── smcd-ueid.8 ├── smcd.8 ├── smcr.8 ├── smcss.8 ├── smcss.c ├── smctools_common.h ├── stats.c ├── stats.h ├── ueid.c ├── ueid.h ├── util.c └── util.h /.gitignore: -------------------------------------------------------------------------------- 1 | *.o 2 | *.so 3 | smc 4 | smc_pnet 5 | smcss 6 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | Contributing to smc-tools 2 | ========================= 3 | 4 | 5 | License 6 | ------- 7 | All contributions have to be submitted under the Eclipse Public License v1.0. 8 | See also the [LICENSE](LICENSE) file. 9 | 10 | 11 | Developer's Certificate of Origin and Signed-off-by 12 | --------------------------------------------------- 13 | The sign-off is a simple line at the end of the explanation for the patch, 14 | which certifies that you wrote it or otherwise have the right to pass it on as 15 | an open-source patch. 16 | 17 | With the Signed-off-by line you certify the below: 18 | 19 | ``` 20 | Developer's Certificate of Origin 1.1 21 | 22 | By making a contribution to this project, I certify that: 23 | 24 | (a) The contribution was created in whole or in part by me and I 25 | have the right to submit it under the open source license 26 | indicated in the file; or 27 | 28 | (b) The contribution is based upon previous work that, to the best 29 | of my knowledge, is covered under an appropriate open source 30 | license and I have the right under that license to submit that 31 | work with modifications, whether created in whole or in part 32 | by me, under the same open source license (unless I am 33 | permitted to submit under a different license), as indicated 34 | in the file; or 35 | 36 | (c) The contribution was provided directly to me by some other 37 | person who certified (a), (b) or (c) and I have not modified 38 | it. 39 | 40 | (d) I understand and agree that this project and the contribution 41 | are public and that a record of the contribution (including all 42 | personal information I submit with it, including my sign-off) is 43 | maintained indefinitely and may be redistributed consistent with 44 | this project or the open source license(s) involved. 45 | ``` 46 | 47 | If you can certify the above, just add a line stating the following at the 48 | bottom of each of your commit messages: 49 | 50 | ``` 51 | Signed-off-by: Random Developer 52 | ``` 53 | 54 | Please use your real name and a valid e-mail address (no pseudonyms or anonymous 55 | contributions). 56 | 57 | 58 | Submitting code 59 | --------------- 60 | The preferred way is to create GitHub pull requests for your code contributions. 61 | Please create separate pull requests for each logical enhancement, new feature, 62 | or fix. 63 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | 2 | Eclipse Public License - v 1.0 3 | 4 | THE ACCOMPANYING PROGRAM IS PROVIDED UNDER THE TERMS OF THIS ECLIPSE 5 | PUBLIC LICENSE ("AGREEMENT"). ANY USE, REPRODUCTION OR DISTRIBUTION OF 6 | THE PROGRAM CONSTITUTES RECIPIENT'S ACCEPTANCE OF THIS AGREEMENT. 7 | 8 | 1. DEFINITIONS 9 | 10 | "Contribution" means: 11 | a) in the case of the initial Contributor, the initial code and 12 | documentation distributed under this Agreement, and 13 | b) in the case of each subsequent Contributor: 14 | 15 | i) changes to the Program, and 16 | 17 | ii) additions to the Program; 18 | 19 | where such changes and/or additions to the Program originate from and 20 | are distributed by that particular Contributor. A Contribution 21 | 'originates' from a Contributor if it was added to the Program by 22 | such Contributor itself or anyone acting on such Contributor's 23 | behalf. Contributions do not include additions to the Program which: 24 | (i) are separate modules of software distributed in conjunction with 25 | the Program under their own license agreement, and (ii) are not 26 | derivative works of the Program. 27 | 28 | "Contributor" means any person or entity that distributes the Program. 29 | 30 | "Licensed Patents" mean patent claims licensable by a Contributor which 31 | are necessarily infringed by the use or sale of its Contribution alone 32 | or when combined with the Program. 33 | 34 | "Program" means the Contributions distributed in accordance with this 35 | Agreement. 36 | 37 | "Recipient" means anyone who receives the Program under this Agreement, 38 | including all Contributors. 39 | 40 | 2. GRANT OF RIGHTS 41 | a) Subject to the terms of this Agreement, each Contributor hereby 42 | grants Recipient a non-exclusive, worldwide, royalty-free copyright 43 | license to reproduce, prepare derivative works of, publicly display, 44 | publicly perform, distribute and sublicense the Contribution of such 45 | Contributor, if any, and such derivative works, in source code and 46 | object code form. 47 | 48 | b) Subject to the terms of this Agreement, each Contributor hereby 49 | grants Recipient a non-exclusive, worldwide, royalty-free patent 50 | license under Licensed Patents to make, use, sell, offer to sell, 51 | import and otherwise transfer the Contribution of such Contributor, 52 | if any, in source code and object code form. This patent license 53 | shall apply to the combination of the Contribution and the Program 54 | if, at the time the Contribution is added by the Contributor, such 55 | addition of the Contribution causes such combination to be covered 56 | by the Licensed Patents. The patent license shall not apply to any 57 | other combinations which include the Contribution. No hardware per 58 | se is licensed hereunder. 59 | 60 | c) Recipient understands that although each Contributor grants the 61 | licenses to its Contributions set forth herein, no assurances are 62 | provided by any Contributor that the Program does not infringe the 63 | patent or other intellectual property rights of any other entity. 64 | Each Contributor disclaims any liability to Recipient for claims 65 | brought by any other entity based on infringement of intellectual 66 | property rights or otherwise. As a condition to exercising the 67 | rights and licenses granted hereunder, each Recipient hereby assumes 68 | sole responsibility to secure any other intellectual property rights 69 | needed, if any. For example, if a third party patent license is 70 | required to allow Recipient to distribute the Program, it is 71 | Recipient's responsibility to acquire that license before 72 | distributing the Program. 73 | 74 | d) Each Contributor represents that to its knowledge it has sufficient 75 | copyright rights in its Contribution, if any, to grant the copyright 76 | license set forth in this Agreement. 77 | 78 | 3. REQUIREMENTS 79 | 80 | A Contributor may choose to distribute the Program in object code form 81 | under its own license agreement, provided that: 82 | a) it complies with the terms and conditions of this Agreement; and 83 | 84 | b) its license agreement: 85 | 86 | i) effectively disclaims on behalf of all Contributors all warranties 87 | and conditions, express and implied, including warranties or 88 | conditions of title and non-infringement, and implied warranties or 89 | conditions of merchantability and fitness for a particular purpose; 90 | 91 | ii) effectively excludes on behalf of all Contributors all liability 92 | for damages, including direct, indirect, special, incidental and 93 | consequential damages, such as lost profits; 94 | 95 | iii) states that any provisions which differ from this Agreement are 96 | offered by that Contributor alone and not by any other party; and 97 | 98 | iv) states that source code for the Program is available from such 99 | Contributor, and informs licensees how to obtain it in a 100 | reasonable manner on or through a medium customarily used for 101 | software exchange. 102 | 103 | When the Program is made available in source code form: 104 | a) it must be made available under this Agreement; and 105 | 106 | b) a copy of this Agreement must be included with each copy of the 107 | Program. 108 | 109 | Contributors may not remove or alter any copyright notices contained 110 | within the Program. 111 | 112 | Each Contributor must identify itself as the originator of its 113 | Contribution, if any, in a manner that reasonably allows subsequent 114 | Recipients to identify the originator of the Contribution. 115 | 116 | 4. COMMERCIAL DISTRIBUTION 117 | 118 | Commercial distributors of software may accept certain responsibilities 119 | with respect to end users, business partners and the like. While this 120 | license is intended to facilitate the commercial use of the Program, 121 | the Contributor who includes the Program in a commercial product 122 | offering should do so in a manner which does not create potential 123 | liability for other Contributors. Therefore, if a Contributor includes 124 | the Program in a commercial product offering, such Contributor 125 | ("Commercial Contributor") hereby agrees to defend and indemnify every 126 | other Contributor ("Indemnified Contributor") against any losses, 127 | damages and costs (collectively "Losses") arising from claims, lawsuits 128 | and other legal actions brought by a third party against the 129 | Indemnified Contributor to the extent caused by the acts or omissions 130 | of such Commercial Contributor in connection with its distribution of 131 | the Program in a commercial product offering. The obligations in this 132 | section do not apply to any claims or Losses relating to any actual or 133 | alleged intellectual property infringement. In order to qualify, an 134 | Indemnified Contributor must: a) promptly notify the Commercial 135 | Contributor in writing of such claim, and b) allow the Commercial 136 | Contributor to control, and cooperate with the Commercial Contributor 137 | in, the defense and any related settlement negotiations. The Indemnified 138 | Contributor may participate in any such claim at its own expense. 139 | 140 | For example, a Contributor might include the Program in a commercial 141 | product offering, Product X. That Contributor is then a Commercial 142 | Contributor. If that Commercial Contributor then makes performance 143 | claims, or offers warranties related to Product X, those performance 144 | claims and warranties are such Commercial Contributor's responsibility 145 | alone. Under this section, the Commercial Contributor would have to 146 | defend claims against the other Contributors related to those 147 | performance claims and warranties, and if a court requires any other 148 | Contributor to pay any damages as a result, the Commercial Contributor 149 | must pay those damages. 150 | 151 | 5. NO WARRANTY 152 | 153 | EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, THE PROGRAM IS PROVIDED 154 | ON AN "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, EITHER 155 | EXPRESS OR IMPLIED INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OR 156 | CONDITIONS OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY OR FITNESS FOR A 157 | PARTICULAR PURPOSE. Each Recipient is solely responsible for determining 158 | the appropriateness of using and distributing the Program and assumes all 159 | risks associated with its exercise of rights under this Agreement , 160 | including but not limited to the risks and costs of program errors, 161 | compliance with applicable laws, damage to or loss of data, programs or 162 | equipment, and unavailability or interruption of operations. 163 | 164 | 6. DISCLAIMER OF LIABILITY 165 | 166 | EXCEPT AS EXPRESSLY SET FORTH IN THIS AGREEMENT, NEITHER RECIPIENT 167 | NOR ANY CONTRIBUTORS SHALL HAVE ANY LIABILITY FOR ANY DIRECT, INDIRECT, 168 | INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING 169 | WITHOUT LIMITATION LOST PROFITS), HOWEVER CAUSED AND ON ANY THEORY OF 170 | LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING 171 | NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OR 172 | DISTRIBUTION OF THE PROGRAM OR THE EXERCISE OF ANY RIGHTS GRANTED 173 | HEREUNDER, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. 174 | 175 | 7. GENERAL 176 | 177 | If any provision of this Agreement is invalid or unenforceable under 178 | applicable law, it shall not affect the validity or enforceability of 179 | the remainder of the terms of this Agreement, and without further 180 | action by the parties hereto, such provision shall be reformed to the 181 | minimum extent necessary to make such provision valid and enforceable. 182 | 183 | If Recipient institutes patent litigation against any entity (including 184 | a cross-claim or counterclaim in a lawsuit) alleging that the Program 185 | itself (excluding combinations of the Program with other software or 186 | hardware) infringes such Recipient's patent(s), then such Recipient's 187 | rights granted under Section 2(b) shall terminate as of the date such 188 | litigation is filed. 189 | 190 | All Recipient's rights under this Agreement shall terminate if it fails 191 | to comply with any of the material terms or conditions of this Agreement 192 | and does not cure such failure in a reasonable period of time after 193 | becoming aware of such noncompliance. If all Recipient's rights under 194 | this Agreement terminate, Recipient agrees to cease use and distribution 195 | of the Program as soon as reasonably practicable. However, Recipient's 196 | obligations under this Agreement and any licenses granted by Recipient 197 | relating to the Program shall continue and survive. 198 | 199 | Everyone is permitted to copy and distribute copies of this Agreement, 200 | but in order to avoid inconsistency the Agreement is copyrighted and 201 | may only be modified in the following manner. The Agreement Steward 202 | reserves the right to publish new versions (including revisions) of 203 | this Agreement from time to time. No one other than the Agreement 204 | Steward has the right to modify this Agreement. The Eclipse Foundation 205 | is the initial Agreement Steward. The Eclipse Foundation may assign the 206 | responsibility to serve as the Agreement Steward to a suitable separate 207 | entity. Each new version of the Agreement will be given a distinguishing 208 | version number. The Program (including Contributions) may always be 209 | distributed subject to the version of the Agreement under which it was 210 | received. In addition, after a new version of the Agreement is published, 211 | Contributor may elect to distribute the Program (including its 212 | Contributions) under the new version. Except as expressly stated in 213 | Sections 2(a) and 2(b) above, Recipient receives no rights or licenses 214 | to the intellectual property of any Contributor under this Agreement, 215 | whether expressly, by implication, estoppel or otherwise. All rights in 216 | the Program not expressly granted under this Agreement are reserved. 217 | 218 | This Agreement is governed by the laws of the State of New York and the 219 | intellectual property laws of the United States of America. No party to 220 | this Agreement will bring a legal action under this Agreement more than 221 | one year after the cause of action arose. Each party waives its rights 222 | to a jury trial in any resulting litigation. 223 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # 2 | # SMC Tools - Shared Memory Communication Tools 3 | # 4 | # Copyright IBM Corp. 2016, 2018 5 | # 6 | # All rights reserved. This program and the accompanying materials 7 | # are made available under the terms of the Eclipse Public License v1.0 8 | # which accompanies this distribution, and is available at 9 | # http://www.eclipse.org/legal/epl-v10.html 10 | # 11 | 12 | SMC_TOOLS_RELEASE = 1.8.4 13 | VER_MAJOR = $(shell echo $(SMC_TOOLS_RELEASE) | cut -d '.' -f 1) 14 | 15 | ARCHTYPE = $(shell uname -m) 16 | ARCH := $(shell getconf LONG_BIT) 17 | DISTRO := $(shell lsb_release -si 2>/dev/null) 18 | 19 | ifneq ("${V}","1") 20 | MAKEFLAGS += --quiet 21 | cmd = echo $1$2; 22 | else 23 | cmd = 24 | endif 25 | CCC = $(call cmd," CC ",$@)${CC} 26 | LINK = $(call cmd," LINK ",$@)${CC} 27 | GEN = $(call cmd," GEN ",$@)sed 28 | DESTDIR ?= 29 | PREFIX = /usr 30 | BINDIR = ${PREFIX}/bin 31 | MANDIR = ${PREFIX}/share/man 32 | BASH_AUTODIR = $(shell pkg-config --variable=completionsdir bash-completion 2>/dev/null) 33 | OWNER = $(shell id -un) 34 | GROUP = $(shell id -gn) 35 | INSTALL_FLAGS_BIN = -g $(GROUP) -o $(OWNER) -m755 36 | INSTALL_FLAGS_MAN = -g $(GROUP) -o $(OWNER) -m644 37 | INSTALL_FLAGS_LIB = -g $(GROUP) -o $(OWNER) -m4755 38 | 39 | STUFF_32BIT = 0 40 | # 41 | # Check that 31/32-bit build tools are available. 42 | # 43 | ifeq ($(ARCH),64) 44 | ifeq ($(DISTRO),Ubuntu) 45 | LIBDIR = ${PREFIX}/usr/lib 46 | else 47 | LIBDIR = ${PREFIX}/lib64 48 | endif 49 | ifneq ("$(wildcard ${PREFIX}/include/gnu/stubs-32.h)","") 50 | STUFF_32BIT = 1 51 | LIBDIR32 = ${PREFIX}/lib 52 | endif 53 | else 54 | ifeq ($(DISTRO),Ubuntu) 55 | LIBDIR = ${PREFIX}/lib/s390-linux-gnu 56 | else 57 | LIBDIR = ${PREFIX}/lib 58 | endif 59 | endif 60 | 61 | all: libsmc-preload.so libsmc-preload32.so smcd smcr smcss smc_pnet 62 | 63 | CFLAGS ?= -Wall -O3 -g 64 | ifneq ($(shell sh -c 'command -v pkg-config'),) 65 | LIBNL_CFLAGS = $(shell pkg-config --silence-errors --cflags libnl-genl-3.0) 66 | LIBNL_LFLAGS = $(shell pkg-config --silence-errors --libs libnl-genl-3.0) 67 | else 68 | LIBNL_CFLAGS = -I /usr/include/libnl3 69 | LIBNL_LFLAGS = -lnl-genl-3 -lnl-3 70 | endif 71 | ALL_CFLAGS += ${CFLAGS} -DSMC_TOOLS_RELEASE=$(SMC_TOOLS_RELEASE) \ 72 | ${LIBNL_CFLAGS} ${OPTFLAGS} 73 | ALL_LDFLAGS += ${LDFLAGS} ${LIBNL_LFLAGS} -lm 74 | 75 | ifeq ($(ARCHTYPE),s390x) 76 | MACHINE_OPT32="-m31" 77 | else 78 | MACHINE_OPT32="-m32" 79 | endif 80 | 81 | util.o: util.c util.h 82 | ${CCC} ${ALL_CFLAGS} -c util.c 83 | 84 | libnetlink.o: libnetlink.c libnetlink.h 85 | ${CCC} ${ALL_CFLAGS} ${ALL_LDFLAGS} -c libnetlink.c 86 | 87 | smc-preload.o: smc-preload.c 88 | ${CCC} ${ALL_CFLAGS} -fPIC -c smc-preload.c 89 | 90 | libsmc-preload.so: smc-preload.o 91 | ${LINK} ${ALL_LDFLAGS} -shared smc-preload.o -ldl -Wl,-z,defs,-soname,$@.$(VER_MAJOR) -o $@ 92 | chmod u+s $@ 93 | 94 | libsmc-preload32.so: smc-preload.c 95 | ifeq ($(ARCH),64) 96 | ifeq ($(STUFF_32BIT),1) 97 | ${CCC} ${ALL_CFLAGS} -fPIC -c ${MACHINE_OPT32} $< -o smc-preload32.o 98 | ${LINK} ${ALL_LDFLAGS} -shared smc-preload32.o ${MACHINE_OPT32} -ldl -Wl,-soname,$@.$(VER_MAJOR) -o $@ 99 | chmod u+s $@ 100 | else 101 | $(warning "Warning: Skipping 31/32-bit library build because 31/32-bit build tools") 102 | $(warning " are unavailable. SMC will not support 31/32 bit applications") 103 | $(warning " unless the glibc devel package for the appropriate addressing") 104 | $(warning " mode is installed and the preload libraries are rebuilt.") 105 | endif 106 | endif 107 | 108 | 109 | %d.o: %.c smctools_common.h 110 | ${CCC} ${ALL_CFLAGS} -DSMCD -c $< -o $@ 111 | 112 | %r.o: %.c smctools_common.h 113 | ${CCC} ${ALL_CFLAGS} -DSMCR -c $< -o $@ 114 | 115 | %.o: %.c smctools_common.h 116 | ${CCC} ${ALL_CFLAGS} -c $< -o $@ 117 | 118 | smc: smc.o info.o ueid.o seid.o dev.o linkgroup.o libnetlink.o util.o 119 | ${CCC} ${ALL_CFLAGS} ${ALL_LDFLAGS} $^ -o $@ 120 | 121 | smcd: smcd.o infod.o ueidd.o seidd.o devd.o linkgroupd.o statsd.o libnetlink.o util.o 122 | ${CCC} ${ALL_CFLAGS} $^ ${ALL_LDFLAGS} -o $@ 123 | 124 | smcr: smcr.o infor.o ueidr.o seidr.o devr.o linkgroupr.o statsr.o libnetlink.o util.o 125 | ${CCC} ${ALL_CFLAGS} $^ ${ALL_LDFLAGS} -o $@ 126 | 127 | smc_pnet: smc_pnet.c smctools_common.h 128 | @if [ ! -e /usr/include/libnl3/netlink/netlink.h ]; then \ 129 | printf "**************************************************************\n" >&2; \ 130 | printf "* Missing build requirement for: %-45s\n" $@ >&2; \ 131 | printf "* Install package..............: %-45s\n" "devel package for libnl3" >&2; \ 132 | printf "* Install package..............: %-45s\n" "devel package for libnl3-genl" >&2; \ 133 | printf "* NOTE: Package names might differ by platform\n" >&2; \ 134 | printf "* On Ubuntu try libnl-3-dev and libnl-genl-3-dev\n" >&2; \ 135 | printf "**************************************************************\n" >&2; \ 136 | exit 1; \ 137 | fi 138 | ${CCC} ${ALL_CFLAGS} $< ${ALL_LDFLAGS} -o $@ 139 | 140 | smcss: smcss.o libnetlink.o 141 | ${CCC} ${ALL_CFLAGS} $^ ${ALL_LDFLAGS} -o $@ 142 | 143 | install: all 144 | echo " INSTALL" 145 | install -d -m755 $(DESTDIR)$(LIBDIR) $(DESTDIR)$(BINDIR) $(DESTDIR)$(MANDIR)/man7 \ 146 | $(DESTDIR)$(BASH_AUTODIR) $(DESTDIR)$(MANDIR)/man8 147 | install $(INSTALL_FLAGS_LIB) libsmc-preload.so $(DESTDIR)$(LIBDIR) 148 | #ifeq ($(STUFF_32BIT),1) 149 | # install -d -m755 $(DESTDIR)$(LIBDIR32) 150 | # install $(INSTALL_FLAGS_LIB) libsmc-preload32.so $(DESTDIR)$(LIBDIR32)/libsmc-preload.so 151 | #endif 152 | install $(INSTALL_FLAGS_BIN) smc_run $(DESTDIR)$(BINDIR) 153 | install $(INSTALL_FLAGS_BIN) smcd $(DESTDIR)$(BINDIR) 154 | install $(INSTALL_FLAGS_BIN) smcr $(DESTDIR)$(BINDIR) 155 | install $(INSTALL_FLAGS_BIN) smcss $(DESTDIR)$(BINDIR) 156 | install $(INSTALL_FLAGS_BIN) smc_pnet $(DESTDIR)$(BINDIR) 157 | install $(INSTALL_FLAGS_BIN) smc_dbg $(DESTDIR)$(BINDIR) 158 | ifeq ($(shell uname -m | cut -c1-4),s390) 159 | install $(INSTALL_FLAGS_BIN) smc_rnics $(DESTDIR)$(BINDIR) 160 | install $(INSTALL_FLAGS_MAN) smc_rnics.8 $(DESTDIR)$(MANDIR)/man8 161 | install $(INSTALL_FLAGS_BIN) smc_chk $(DESTDIR)$(BINDIR) 162 | install $(INSTALL_FLAGS_MAN) smc_chk.8 $(DESTDIR)$(MANDIR)/man8 163 | endif 164 | install $(INSTALL_FLAGS_MAN) af_smc.7 $(DESTDIR)$(MANDIR)/man7 165 | install $(INSTALL_FLAGS_MAN) smc_run.8 $(DESTDIR)$(MANDIR)/man8 166 | install $(INSTALL_FLAGS_MAN) smc_pnet.8 $(DESTDIR)$(MANDIR)/man8 167 | install $(INSTALL_FLAGS_MAN) smcss.8 $(DESTDIR)$(MANDIR)/man8 168 | install $(INSTALL_FLAGS_MAN) smcd.8 $(DESTDIR)$(MANDIR)/man8 169 | install $(INSTALL_FLAGS_MAN) smcr.8 $(DESTDIR)$(MANDIR)/man8 170 | install $(INSTALL_FLAGS_MAN) smcd-linkgroup.8 $(DESTDIR)$(MANDIR)/man8 171 | install $(INSTALL_FLAGS_MAN) smcd-device.8 $(DESTDIR)$(MANDIR)/man8 172 | install $(INSTALL_FLAGS_MAN) smcd-info.8 $(DESTDIR)$(MANDIR)/man8 173 | install $(INSTALL_FLAGS_MAN) smcd-stats.8 $(DESTDIR)$(MANDIR)/man8 174 | install $(INSTALL_FLAGS_MAN) smcd-ueid.8 $(DESTDIR)$(MANDIR)/man8 175 | install $(INSTALL_FLAGS_MAN) smcd-seid.8 $(DESTDIR)$(MANDIR)/man8 176 | ln -sfr $(DESTDIR)$(MANDIR)/man8/smcd-linkgroup.8 $(DESTDIR)$(MANDIR)/man8/smcr-linkgroup.8 177 | ln -sfr $(DESTDIR)$(MANDIR)/man8/smcd-device.8 $(DESTDIR)$(MANDIR)/man8/smcr-device.8 178 | ln -sfr $(DESTDIR)$(MANDIR)/man8/smcd-info.8 $(DESTDIR)$(MANDIR)/man8/smcr-info.8 179 | ln -sfr $(DESTDIR)$(MANDIR)/man8/smcd-stats.8 $(DESTDIR)$(MANDIR)/man8/smcr-stats.8 180 | ln -sfr $(DESTDIR)$(MANDIR)/man8/smcd-ueid.8 $(DESTDIR)$(MANDIR)/man8/smcr-ueid.8 181 | ifneq ($(BASH_AUTODIR),) 182 | install $(INSTALL_FLAGS_MAN) smc-tools.autocomplete $(DESTDIR)$(BASH_AUTODIR)/smc-tools 183 | ln -sfr $(DESTDIR)$(BASH_AUTODIR)/smc-tools $(DESTDIR)$(BASH_AUTODIR)/smc_rnics 184 | ln -sfr $(DESTDIR)$(BASH_AUTODIR)/smc-tools $(DESTDIR)$(BASH_AUTODIR)/smc_chk 185 | ln -sfr $(DESTDIR)$(BASH_AUTODIR)/smc-tools $(DESTDIR)$(BASH_AUTODIR)/smc_dbg 186 | ln -sfr $(DESTDIR)$(BASH_AUTODIR)/smc-tools $(DESTDIR)$(BASH_AUTODIR)/smcss 187 | ln -sfr $(DESTDIR)$(BASH_AUTODIR)/smc-tools $(DESTDIR)$(BASH_AUTODIR)/smc_pnet 188 | ln -sfr $(DESTDIR)$(BASH_AUTODIR)/smc-tools $(DESTDIR)$(BASH_AUTODIR)/smc 189 | endif 190 | 191 | check: 192 | if ( command -v cppcheck >/dev/null ); then \ 193 | echo "Running cppcheck"; \ 194 | cppcheck . 2>&1; \ 195 | else \ 196 | echo "cppcheck not available"; \ 197 | fi 198 | @echo; 199 | #if type -p valgrind >/dev/null; then \ 200 | if ( command -v valgrind >/dev/null ); then \ 201 | echo "Running valgrind"; \ 202 | valgrind --leak-check=full --show-leak-kinds=all ./smcss 2>&1; \ 203 | valgrind --leak-check=full --show-leak-kinds=all ./smc_pnet 2>&1; \ 204 | valgrind --leak-check=full --show-leak-kinds=all ./smcd info 2>&1; \ 205 | valgrind --leak-check=full --show-leak-kinds=all ./smcd stats 2>&1; \ 206 | else \ 207 | echo "valgrind not available"; \ 208 | fi 209 | @echo; 210 | clean: 211 | echo " CLEAN" 212 | rm -f *.o *.so *.a smc smcd smcr smcss smc_pnet 213 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | SMC Tools 2 | ========= 3 | 4 | Utilities for use with `AF_SMC` sockets. 5 | 6 | This package consists of the following tools: 7 | 8 | - `libsmc-preload.so` : preload library. 9 | - `smc` : List linkgroups, links, devices, and more 10 | - `smc_chk` : SMC support diagnostics 11 | - `smc_pnet` : C program for PNET Table handling 12 | - `smc_rnics` : List available RDMA NICs 13 | - `smc_run` : preload library environment setup script. 14 | - `smcss` : C program for displaying the information about active 15 | SMC sockets. 16 | 17 | The preload shared library `libsmc-preload.so` provides mapping of TCP socket 18 | operations to SMC sockets. 19 | The environment setup script `smc_run` sets up the preload environment 20 | for the `libsmc-preload.so` shared library before starting application. 21 | The `smcss` program is used to gather and display information about the 22 | SMC sockets. 23 | The `smc_pnet` program is used to create, destroy, and change the SMC-R PNET 24 | table. 25 | 26 | In addition the package contains the `AF_SMC` manpage (`man af_smc`). 27 | 28 | 29 | License 30 | ------- 31 | See [LICENSE](LICENSE). 32 | 33 | 34 | Code Contributions 35 | ------------------ 36 | See [CONTRIBUTING.md](CONTRIBUTING.md). 37 | 38 | 39 | Release History: 40 | ================ 41 | 42 | * __v1.8.4 (2024-12-17)__ 43 | 44 | Changes: 45 | - `smcd`/`smcr` statistics: add statistics on sndbufs/RMBs usage 46 | 47 | Bug fixes: 48 | - `smc_rnics`: Fix smc_rnics showing the wrong physical port 49 | - `util.h`: Fix build with CFLAGS="-g" 50 | - `smc_run`: Fix segfault issue during creating concurrent sockets 51 | - `Makefile`: Make sure to show the right release number 52 | 53 | * __v1.8.3 (2023-07-28)__ 54 | 55 | Bug fixes: 56 | - `Makefile`: Adjust default library path for Ubuntu 57 | - `smc_stats`: Fix man page name 58 | 59 | * __v1.8.2 (2022-09-26)__ 60 | 61 | Bug fixes: 62 | - `smc_run`: Fix for single quotes in parameters 63 | - `Makefile`: Fix target `check` 64 | - `smcss`/`smc_pnet`: Fix option `--version` 65 | - `smcr`/`smc_dbg`: Fix for showing the correct hardware capabailities 66 | for RoCE Express3 cards 67 | 68 | * __v1.8.1 (2022-04-14)__ 69 | 70 | Changes: 71 | - `smc_rnics`: Recognize RoCE Express3 cards 72 | 73 | * __v1.8.0 (2022-04-11)__ 74 | 75 | Changes: 76 | - `smc_dbg`: Add stats and `smc info` output 77 | - `smc_rnics`: 78 | - List unknown devices with option `-a` 79 | - Include software-set PNET IDs 80 | - `smc_chk`: Indicate PNET IDs set by `smc_pnet`. 81 | 82 | Bug fixes: 83 | - `smc_rnics`: Display correct PNET ID for unknown Mellanox cards 84 | - `smc_run`: Fix output of version info 85 | 86 | * __v1.7.0 (2021-10-29)__ 87 | 88 | Changes: 89 | - Add support for SMC-Rv2 90 | - `smcd`/`smcr`: Add support for new commands `seid` and `ueid` to 91 | manage system and user EIDs 92 | 93 | * __v1.6.1 (2021-10-01)__ 94 | 95 | Bug fixes: 96 | - `smcd`/`smcr` statistics: 97 | - Fix memory overread in is_data_consistent() 98 | - Fix memory and file handle leaks 99 | - Use correct fallback counter values after reset 100 | 101 | * __v1.6.0 (2021-07-01)__ 102 | 103 | Changes: 104 | - `smcd`/`smcr`: Add new command `stats` 105 | - `smc_rnics`: Recognize unknown Mellanox cards 106 | - `smc_run`: Add various command-line switches 107 | 108 | Bug fixes: 109 | - `smc_chk`: Remove 'EXPERIMENTAL' flag 110 | - `smc_chk`: Improve cleanup 111 | - `smc_chk`: Start server with intended port 112 | - `Makefile`: Install `smc_chk.8` on s390 only 113 | - `Makefile`: Fix extra compile flags handling 114 | - `smc_rnics`: Handle malformed FID arguments 115 | 116 | * __v1.5.0 (2021-01-29)__ 117 | 118 | Changes: 119 | - `smcd`/`smcr`: Add new command `info` 120 | - `smc_rnics`: Use `n/a` to indicate missing PNET ID 121 | - `smc_chk`: New tool to perform SMC eligilibilty checks, requires `man` and 122 | `python3` to be installed 123 | - `man` pages: Consistency improvements 124 | 125 | Bug fixes: 126 | - `smc_pnet.8`: Use correct spelling for 'PNET ID' 127 | - `smc_rnics`: Suppress output of port attribute for offline devices 128 | 129 | * __v1.4.0 (2020-11-03)__ 130 | 131 | Changes: 132 | - Add SMC-Dv2 support 133 | - `smc`: Add new tools `smcd` and `smcr` to list linkgroups, links and 134 | devices. Requires Linux kernel 5.11 or higher. 135 | - `smc_rnics`: Display enabled devices per default, add new option `--all` 136 | - `smc_rnics`: Sort output by FID 137 | 138 | Bug fixes: 139 | - `smc_rnics`/`smc_dbg`: Fix PNETID for multiport devices 140 | - `smcss`/`smc_pnet`: Consistent use of option `-v` 141 | 142 | * __v1.3.1 (2020-09-14)__ 143 | 144 | Changes: 145 | - `smcss`: Add further error codes to man page 146 | 147 | Bug fixes: 148 | - `smcss`: Display more than 321 connections 149 | - `smc_rnics`: Suppress any unknown non-networking device unless 150 | option `-r` is specified 151 | 152 | * __v1.3.0 (2020-06-16)__ 153 | 154 | Changes: 155 | - `smcss`: Add description of Linux error codes to man page 156 | - `smc_rnics`: 157 | * Sort output by PCHID 158 | * Replace spaces in output by underscores for easier parsing 159 | * Add new option `--IB-dev` to display IB-specific attributes 160 | 161 | Bug fixes: 162 | - smc_rnics: 163 | * FIDs can have up to 4 digits and are planned to be extended 164 | to a total of 8 digits - adjusting output format accordingly 165 | * Do not display port attribute for RoCE Express2 devices 166 | unless we have an accurate value 167 | 168 | * __v1.2.2 (2019-10-24)__ 169 | Changes: 170 | - Add bash autocompletion support 171 | - `Makefile`: Drop 31 Bit install due to rpmbuild conflict 172 | 173 | Bug fixes: 174 | - `smcss`: Do not show connection mode for already closed sockets 175 | - `smc_rnics`: Set interface to "n/a" for ISM devices 176 | 177 | * __v1.2.1 (2019-04-15)__ 178 | Bug fixes: 179 | - `smc_rnics`: Install man page on s390 only 180 | - `libsmc..`: Handle behavior flags in type argument to `socket()` call 181 | - `Makefile`: Fixed install target on Ubuntu for platforms other than s390 182 | - `smc_pnet`: Changes in support of kernel 5.1 183 | 184 | * __v1.2.0 (2019-02-08)__ 185 | Changes: 186 | - `smc_rnics`: Initial version added 187 | - `smc_dbg`: Initial version added 188 | 189 | Bug fixes: 190 | - `smcss`: Parse address family of ip address 191 | 192 | * __v1.1.0 (2018-06-29)__ 193 | Changes: 194 | - `smcss`: Add IPv6 support 195 | - `libsmc..`: Add IPv6 support 196 | - `smcss`: Output format changed 197 | - `libsmc..`: Rename preload library to `libsmc-preload.so` 198 | - `Makefile`: Improve distro compatibility 199 | - `Makefile`: Add `SONAME` to shared libraries 200 | - `Makefile`: Do not strip binaries on install 201 | - `Makefile`: Use `LDFLAGS` to allow addition of externally set link flags 202 | - `libsmc..`: Remove hardcoded reference to libc 203 | - Manpages: Formatting changes 204 | 205 | Bug fixes: 206 | - `Makefile`: Fix target `install` dependencies 207 | - `smcss`: Fix `--version` output 208 | - `smc_pnet`: Fix `--version` output 209 | - `smc_run`: Append preload library to `LD_PRELOAD` instead of potentially 210 | overwriting pre-set values 211 | - `libsmc..`: Set suid flag to work with suid executables 212 | 213 | * __v1.0.0 (2017-02-13)__ 214 | The initial version 215 | 216 | 217 | Copyright IBM Corp. 2016, 2020 218 | -------------------------------------------------------------------------------- /af_smc.7: -------------------------------------------------------------------------------- 1 | .\" 2 | .\" Copyright IBM Corp. 2016, 2018 3 | .\" Author(s): Ursula Braun 4 | .\" Thomas Richter 5 | .\" ---------------------------------------------------------------------- 6 | .TH AF_SMC 7 "January 2017" "smc-tools" "Linux Programmer's Manual" 7 | .SH NAME 8 | AF_SMC - Sockets for SMC communication 9 | .SH SYNOPSIS 10 | .B #include 11 | .sp 12 | .B "#define AF_SMC 43" 13 | .sp 14 | .B "#define SMCPROTO_SMC 0" 15 | .sp 16 | .B "#define SMCPROTO_SMC6 1" 17 | .PP 18 | .IB tcp_sockfd " = socket(" AF_SMC ", " SOCK_STREAM ", " SMCPROTO_SMC ); 19 | .sp 20 | .IB tcp_sockfd " = socket(" AF_SMC ", " SOCK_STREAM ", " SMCPROTO_SMC6 ); 21 | .SH DESCRIPTION 22 | .I Shared Memory Communication via RDMA 23 | (SMC) is a socket over the RDMA 24 | communication protocol that allows existing TCP socket applications 25 | to transparently benefit from RDMA when exchanging data 26 | over an 27 | .I RDMA over Converged Ethernet 28 | (RoCE) network. Those networks are not routable. 29 | SMC provides 30 | host-to-host direct memory access without traditional TCP/IP 31 | processing overhead. 32 | SMC offers preservation of existing IP topology and IP security, and 33 | introduces minimal administrative and operational changes. 34 | The exploitation of SMC is transparent to TCP socket applications. 35 | .PP 36 | The new address family 37 | .B AF_SMC 38 | supports the SMC protocol on Linux. 39 | It keeps the address format of 40 | .B AF_INET 41 | and 42 | .B AF_INET6 43 | sockets and supports streaming socket types only. 44 | .SS Usage modes 45 | Two usage modes are possible: 46 | .IP "AF_SMC native usage" 47 | uses the socket domain 48 | .B AF_SMC 49 | instead of 50 | .B AF_INET 51 | and 52 | .BR AF_INET6 . 53 | Specify 54 | .B SMCPROTO_SMC 55 | for 56 | .B AF_INET 57 | compatible socket semantics, and 58 | .B SMC_PROTO_SMC6 59 | for 60 | .B AF_INET6 61 | respectively. 62 | .IP "Usage of AF_INET socket applications with SMC preload library" 63 | converts 64 | .B AF_INET 65 | and 66 | .B AF_INET6 67 | sockets to 68 | .B AF_SMC 69 | sockets. 70 | The SMC preload library is part of the SMC tools package. 71 | .PP 72 | SMC socket capabilities are negotiated at connection setup. If 73 | one peer is not SMC capable, further socket processing falls 74 | back to TCP usage automatically. 75 | .SS Implementation details: Links and Link Groups 76 | To run RDMA traffic to a peer, a so-called 77 | .I link 78 | is established between 79 | a local RoCE card and a remote RoCE card. To enhance availability, you can 80 | configure alternate links with automatic fail over. 81 | Primary and backup links to a certain peer are combined in a so-called 82 | .I link 83 | .IR group . 84 | .SS RoCE adapter mapping: Creation of a pnet table 85 | The SMC protocol requires grouping of multiple physical networks - 86 | standard Ethernet and RoCE networks. Such groups are called 87 | .I Physical Networks 88 | (PNets). For SMC, RoCE adapter mapping is configured within a table called 89 | .BR "pnet table" . 90 | Any available Ethernet interface can be combined with available 91 | .I RDMA-capable network interface cards 92 | (RNICs), 93 | if they belong to the same Converged Ethernet fabric. 94 | To configure RoCE Adapter mapping, you must create a pnet table. 95 | Modify the table with the smc-tools command 96 | .IR smc_pnet . 97 | .PP 98 | For details see 99 | .BR smc_pnet (8). 100 | .SS Displaying SMC socket state information 101 | SMC socket state information can be obtained with the smc-tools command 102 | .IR smcss . 103 | For details see 104 | .BR smcss (8). 105 | .SS Starting a TCP application to work with SMC 106 | To use an existing TCP application to work with SMC, use the SMC 107 | preload library. 108 | The SMC Tools package provides the command 109 | .I smc_run 110 | to convert 111 | .B AF_INET 112 | and 113 | .B AF_INET6 114 | socket calls to 115 | .B AF_SMC 116 | socket calls by means of the preload technique. 117 | For more information about the preload mechanism, see also 118 | .BR ld.so (8). 119 | .PP 120 | Example: 121 | .IP 122 | .B smc_run ftp 123 | .PP 124 | This command-line example starts an FTP client over SMC. 125 | .PP 126 | .SS MTU and Infiniband data transfer 127 | Infiniband traffic may use MTU values 256, 512, 1024, 2048, or 4096. 128 | SMC determines the configured MTU size of the RoCE Ethernet port, 129 | announces this MTU size to the peer during connection start, and chooses 130 | the minimum MTU size of both peers. 131 | 132 | .SH "SEE ALSO" 133 | .BR socket (2), 134 | .BR ip (7), 135 | .BR tcp (7), 136 | .BR socket (7), 137 | .BR smc_chk (8) 138 | .BR smc_run (8), 139 | .BR smcss (8), 140 | .BR smc_pnet (8) 141 | .SH "HISTORY" 142 | .TP 143 | .B AF_SMC, version 1.0.0 144 | .RS 4 145 | .IP "\bu" 2 146 | Initial version. 147 | .RE 148 | -------------------------------------------------------------------------------- /dev.h: -------------------------------------------------------------------------------- 1 | /* 2 | * SMC Tools - Shared Memory Communication Tools 3 | * 4 | * Copyright IBM Corp. 2020 5 | * 6 | * Author(s): Guvenc Gulce 7 | * 8 | * Userspace program for SMC Information display 9 | * 10 | * All rights reserved. This program and the accompanying materials 11 | * are made available under the terms of the Eclipse Public License v1.0 12 | * which accompanies this distribution, and is available at 13 | * http://www.eclipse.org/legal/epl-v10.html 14 | */ 15 | 16 | #ifndef DEV_H_ 17 | #define DEV_H_ 18 | 19 | extern struct rtnl_handle rth; 20 | 21 | int invoke_devs(int argc, char **argv, int detail_level); 22 | int dev_count_ism_devices(int *ism_count); 23 | int dev_count_roce_devices(int *rocev1_count, int *rocev2_count, int *rocev3_count); 24 | 25 | #endif /* DEV_H_ */ 26 | -------------------------------------------------------------------------------- /info.c: -------------------------------------------------------------------------------- 1 | /* 2 | * SMC Tools - Shared Memory Communication Tools 3 | * 4 | * Copyright IBM Corp. 2021 5 | * 6 | * Userspace program for SMC Information display 7 | * 8 | * All rights reserved. This program and the accompanying materials 9 | * are made available under the terms of the Eclipse Public License v1.0 10 | * which accompanies this distribution, and is available at 11 | * http://www.eclipse.org/legal/epl-v10.html 12 | */ 13 | #include 14 | #include 15 | #include 16 | #include 17 | #include 18 | #include 19 | 20 | #include "smctools_common.h" 21 | #include "util.h" 22 | #include "libnetlink.h" 23 | #include "info.h" 24 | #include "dev.h" 25 | 26 | static int show_cmd = 0; 27 | static int ism_count, rocev1_count, rocev2_count, rocev3_count; 28 | 29 | static struct nla_policy 30 | smc_gen_info_policy[SMC_NLA_SYS_MAX + 1] = { 31 | [SMC_NLA_SYS_UNSPEC] = { .type = NLA_UNSPEC }, 32 | [SMC_NLA_SYS_VER] = { .type = NLA_U8 }, 33 | [SMC_NLA_SYS_REL] = { .type = NLA_U8 }, 34 | [SMC_NLA_SYS_IS_ISM_V2] = { .type = NLA_U8 }, 35 | [SMC_NLA_SYS_LOCAL_HOST]= { .type = NLA_NUL_STRING }, 36 | [SMC_NLA_SYS_SEID] = { .type = NLA_NUL_STRING }, 37 | [SMC_NLA_SYS_IS_SMCR_V2]= { .type = NLA_U8 }, 38 | }; 39 | 40 | static void usage(void) 41 | { 42 | fprintf(stderr, 43 | #if defined(SMCD) 44 | "Usage: smcd info [show]\n" 45 | #elif defined(SMCR) 46 | "Usage: smcr info [show]\n" 47 | #else 48 | "Usage: smc info [show]\n" 49 | #endif 50 | ); 51 | exit(-1); 52 | } 53 | 54 | static int handle_gen_info_reply(struct nl_msg *msg, void *arg) 55 | { 56 | struct nlattr *info_attrs[SMC_NLA_SYS_MAX + 1]; 57 | struct nlattr *attrs[SMC_GEN_MAX + 1]; 58 | struct nlmsghdr *hdr = nlmsg_hdr(msg); 59 | int rc = NL_OK; 60 | char tmp[80]; 61 | int smc_version = 1; 62 | 63 | if (!show_cmd) 64 | return rc; 65 | 66 | if (genlmsg_parse(hdr, 0, attrs, SMC_GEN_MAX, 67 | (struct nla_policy *)smc_gen_net_policy) < 0) { 68 | fprintf(stderr, "Error: Invalid data returned: smc_gen_net_policy\n"); 69 | nl_msg_dump(msg, stderr); 70 | return NL_STOP; 71 | } 72 | 73 | if (!attrs[SMC_GEN_SYS_INFO]) 74 | return rc; 75 | 76 | if (nla_parse_nested(info_attrs, SMC_NLA_DEV_MAX, 77 | attrs[SMC_GEN_SYS_INFO], 78 | smc_gen_info_policy)) { 79 | fprintf(stderr, "Error: Failed to parse nested attributes: smc_gen_info_policy\n"); 80 | return NL_STOP; 81 | } 82 | 83 | printf("Kernel Capabilities\n"); 84 | 85 | /* Version */ 86 | tmp[0] = '\0'; 87 | if (info_attrs[SMC_NLA_SYS_VER] && info_attrs[SMC_NLA_SYS_REL]) { 88 | sprintf(tmp, "%d.%d", nla_get_u8(info_attrs[SMC_NLA_SYS_VER]), nla_get_u8(info_attrs[SMC_NLA_SYS_REL])); 89 | smc_version = nla_get_u8(info_attrs[SMC_NLA_SYS_VER]); 90 | } 91 | printf("SMC Version: %s\n", (tmp[0] != '\0' ? tmp : "n/a")); 92 | 93 | /* Hostname */ 94 | tmp[0] = '\0'; 95 | if (info_attrs[SMC_NLA_SYS_LOCAL_HOST]) { 96 | sprintf(tmp, "%s", nla_get_string(info_attrs[SMC_NLA_SYS_LOCAL_HOST])); 97 | } 98 | printf("SMC Hostname: %s\n", (tmp[0] != '\0' ? tmp : "n/a")); 99 | 100 | /* SMC-D */ 101 | sprintf(tmp, "%s", "v1"); 102 | if (smc_version >= 2) { 103 | strcat(tmp, " v2"); 104 | } 105 | printf("SMC-D Features: %s\n", tmp); 106 | 107 | /* SMC-R */ 108 | sprintf(tmp, "%s", "v1"); 109 | if (info_attrs[SMC_NLA_SYS_IS_SMCR_V2] && nla_get_u8(info_attrs[SMC_NLA_SYS_IS_SMCR_V2])) { 110 | strcat(tmp, " v2"); 111 | } 112 | printf("SMC-R Features: %s\n", tmp); 113 | 114 | printf("\n"); 115 | printf("Hardware Capabilities\n"); 116 | 117 | /* SEID */ 118 | tmp[0] = '\0'; 119 | if (info_attrs[SMC_NLA_SYS_SEID]) { 120 | sprintf(tmp, "%s", nla_get_string(info_attrs[SMC_NLA_SYS_SEID])); 121 | } 122 | printf("SEID: %s\n", (tmp[0] != '\0' ? tmp : "n/a")); 123 | 124 | /* ISM hardware */ 125 | tmp[0] = '\0'; 126 | if (ism_count) { 127 | /* Kernel found any ISM device */ 128 | sprintf(tmp, "%s", "v1"); /* dev found, v1 is possible */ 129 | if (info_attrs[SMC_NLA_SYS_IS_ISM_V2]) { 130 | if (nla_get_u8(info_attrs[SMC_NLA_SYS_IS_ISM_V2])) 131 | strcat(tmp, " v2"); 132 | } 133 | } 134 | printf("ISM: %s\n", (tmp[0] != '\0' ? tmp : "n/a")); 135 | 136 | /* RoCE hardware */ 137 | tmp[0] = '\0'; 138 | if (rocev1_count || rocev2_count || rocev3_count) { 139 | /* Kernel found any RoCE device */ 140 | strcpy(tmp, ""); 141 | if (rocev1_count || rocev2_count || rocev3_count) 142 | strcat(tmp, "v1 "); 143 | if (rocev2_count || rocev3_count) 144 | strcat(tmp, "v2"); 145 | } 146 | printf("RoCE: %s\n", (tmp[0] != '\0' ? tmp : "n/a")); 147 | 148 | return rc; 149 | } 150 | 151 | static void handle_cmd_params(int argc, char **argv) 152 | { 153 | if (argc == 0) { 154 | show_cmd = 1; /* no object given, so use the default "show" */ 155 | return; 156 | } 157 | 158 | while (1) { 159 | if (contains(argv[0], "help") == 0) { 160 | usage(); 161 | } else if (contains(argv[0], "show") == 0) { 162 | show_cmd = 1; 163 | break; 164 | } else { 165 | usage(); 166 | } 167 | if (!NEXT_ARG_OK()) 168 | break; 169 | NEXT_ARG(); 170 | } 171 | /* Too many parameters or wrong sequence of parameters */ 172 | if (NEXT_ARG_OK()) 173 | usage(); 174 | } 175 | 176 | int invoke_info(int argc, char **argv, int detail_level) 177 | { 178 | int rc = EXIT_SUCCESS; 179 | 180 | handle_cmd_params(argc, argv); 181 | 182 | if (show_cmd) { 183 | if (dev_count_ism_devices(&ism_count)) { 184 | fprintf(stderr, "Error: Failed to retrieve ISM device count\n"); 185 | return EXIT_FAILURE; 186 | } 187 | if (dev_count_roce_devices(&rocev1_count, &rocev2_count, &rocev3_count)) { 188 | fprintf(stderr, "Error: Failed to retrieve RoCE device count\n"); 189 | return EXIT_FAILURE; 190 | } 191 | 192 | rc = gen_nl_handle_dump(SMC_NETLINK_GET_SYS_INFO, handle_gen_info_reply, NULL); 193 | } else { 194 | printf("Error: Unknown command\n"); /* we should never come here ... */ 195 | return EXIT_FAILURE; 196 | } 197 | 198 | return rc; 199 | } 200 | -------------------------------------------------------------------------------- /info.h: -------------------------------------------------------------------------------- 1 | /* 2 | * SMC Tools - Shared Memory Communication Tools 3 | * 4 | * Copyright IBM Corp. 2021 5 | * 6 | * Userspace program for SMC Information display 7 | * 8 | * All rights reserved. This program and the accompanying materials 9 | * are made available under the terms of the Eclipse Public License v1.0 10 | * which accompanies this distribution, and is available at 11 | * http://www.eclipse.org/legal/epl-v10.html 12 | */ 13 | 14 | #ifndef INFO_H_ 15 | #define INFO_H_ 16 | 17 | extern struct rtnl_handle rth; 18 | 19 | int invoke_info(int argc, char **argv, int detail_level); 20 | 21 | #endif /* INFO_H_ */ 22 | -------------------------------------------------------------------------------- /libnetlink.c: -------------------------------------------------------------------------------- 1 | /* 2 | * SMC Tools - Shared Memory Communication Tools 3 | * 4 | * Copyright IBM Corp. 2020 5 | * 6 | * Author(s): Ursula Braun 7 | * Guvenc Gulce 8 | * 9 | * Userspace program for SMC Information display 10 | * 11 | * All rights reserved. This program and the accompanying materials 12 | * are made available under the terms of the Eclipse Public License v1.0 13 | * which accompanies this distribution, and is available at 14 | * http://www.eclipse.org/legal/epl-v10.html 15 | */ 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include 24 | #include 25 | #include 26 | #include 27 | #include 28 | #include 29 | #include 30 | 31 | #include "smctools_common.h" 32 | #include "libnetlink.h" 33 | 34 | #define MAGIC_SEQ 123456 35 | 36 | int smc_id = 0; 37 | struct nl_sock *sk; 38 | 39 | /* Operations on sock_diag netlink socket */ 40 | 41 | int rtnl_open(struct rtnl_handle *rth) 42 | { 43 | socklen_t addr_len; 44 | int rcvbuf = 1024 * 1024; 45 | int sndbuf = 32768; 46 | 47 | rth->fd = socket(AF_NETLINK, SOCK_RAW | SOCK_CLOEXEC, 48 | NETLINK_SOCK_DIAG); 49 | if (rth->fd < 0) { 50 | perror("Error: Cannot open netlink socket"); 51 | return EXIT_FAILURE; 52 | } 53 | if (setsockopt(rth->fd, SOL_SOCKET, SO_SNDBUF, &sndbuf, 54 | sizeof(sndbuf)) < 0) { 55 | perror("Error: SO_SNDBUF"); 56 | return EXIT_FAILURE; 57 | } 58 | if (setsockopt(rth->fd, SOL_SOCKET, SO_RCVBUF, &rcvbuf, 59 | sizeof(rcvbuf)) < 0) { 60 | perror("Error: SO_RCVBUF"); 61 | return EXIT_FAILURE; 62 | } 63 | memset(&rth->local, 0, sizeof(rth->local)); 64 | rth->local.nl_family = AF_NETLINK; 65 | rth->local.nl_groups = 0; 66 | if (bind(rth->fd, (struct sockaddr*)&rth->local, 67 | sizeof(rth->local)) < 0) { 68 | perror("Error: Cannot bind netlink socket"); 69 | return EXIT_FAILURE; 70 | } 71 | addr_len = sizeof(rth->local); 72 | if (getsockname(rth->fd, (struct sockaddr*)&rth->local, 73 | &addr_len) < 0) { 74 | perror("Error: getsockname"); 75 | return EXIT_FAILURE; 76 | } 77 | if (addr_len != sizeof(rth->local)) { 78 | fprintf(stderr, "Error: Wrong address length %d\n", addr_len); 79 | return EXIT_FAILURE; 80 | } 81 | if (rth->local.nl_family != AF_NETLINK) { 82 | fprintf(stderr, "Error: Wrong address family %d\n", 83 | rth->local.nl_family); 84 | return EXIT_FAILURE; 85 | } 86 | 87 | rth->seq = time(NULL); 88 | return 0; 89 | } 90 | 91 | void rtnl_close(struct rtnl_handle *rth) 92 | { 93 | if (rth->fd >= 0) { 94 | close(rth->fd); 95 | rth->fd = -1; 96 | } 97 | } 98 | 99 | int rtnl_dump(struct rtnl_handle *rth, void (*handler)(struct nlmsghdr *nlh)) 100 | { 101 | int msglen, found_done = 0; 102 | struct sockaddr_nl nladdr; 103 | struct iovec iov; 104 | struct msghdr msg = { 105 | .msg_name = &nladdr, 106 | .msg_namelen = sizeof(nladdr), 107 | .msg_iov = &iov, 108 | .msg_iovlen = 1, 109 | }; 110 | char buf[32768]; 111 | struct nlmsghdr *h = (struct nlmsghdr *)buf; 112 | 113 | memset(buf, 0, sizeof(buf)); 114 | iov.iov_base = buf; 115 | iov.iov_len = sizeof(buf); 116 | again: 117 | msglen = recvmsg(rth->fd, &msg, 0); 118 | if (msglen < 0) { 119 | if (errno == EINTR || errno == EAGAIN) 120 | goto again; 121 | fprintf(stderr, "Error: Netlink receive error %s (%d)\n", 122 | strerror(errno), errno); 123 | return EXIT_FAILURE; 124 | } 125 | if (msglen == 0) { 126 | fprintf(stderr, "Error: Unexpected EOF on netlink\n"); 127 | return EXIT_FAILURE; 128 | } 129 | 130 | while(NLMSG_OK(h, msglen)) { 131 | if (h->nlmsg_flags & NLM_F_DUMP_INTR) 132 | fprintf(stderr, "Error: Dump interrupted\n"); 133 | if (h->nlmsg_type == NLMSG_DONE) { 134 | found_done = 1; 135 | break; 136 | } 137 | if (h->nlmsg_type == NLMSG_ERROR) { 138 | if (h->nlmsg_len < NLMSG_LENGTH(sizeof(struct nlmsgerr))) { 139 | fprintf(stderr, "Error: Incomplete message\n"); 140 | } else { 141 | perror("RTNETLINK answers"); 142 | } 143 | return EXIT_FAILURE; 144 | } 145 | (*handler)(h); 146 | h = NLMSG_NEXT(h, msglen); 147 | } 148 | if (msg.msg_flags & MSG_TRUNC) { 149 | fprintf(stderr, "Error: Message truncated\n"); 150 | goto again; 151 | } 152 | if (!found_done) { 153 | h = (struct nlmsghdr *)buf; 154 | goto again; 155 | } 156 | return EXIT_SUCCESS; 157 | } 158 | 159 | void parse_rtattr(struct rtattr *tb[], int max, struct rtattr *rta, 160 | int len) 161 | { 162 | unsigned short type; 163 | 164 | memset(tb, 0, sizeof(struct rtattr *) * (max + 1)); 165 | while (RTA_OK(rta, len)) { 166 | type = rta->rta_type; 167 | if ((type <= max) && (!tb[type])) 168 | tb[type] = rta; 169 | rta = RTA_NEXT(rta,len); 170 | } 171 | if (len) 172 | fprintf(stderr, "Error: Deficit %d, rta_len=%d\n", len, rta->rta_len); 173 | } 174 | 175 | int sockdiag_send(int fd, unsigned char cmd) 176 | { 177 | struct sockaddr_nl nladdr = { .nl_family = AF_NETLINK }; 178 | DIAG_REQUEST(req, struct smc_diag_req r, MAGIC_SEQ); 179 | struct msghdr msg; 180 | struct iovec iov[1]; 181 | int iovlen = 1; 182 | 183 | memset(&req.r, 0, sizeof(req.r)); 184 | req.r.diag_family = PF_SMC; 185 | 186 | iov[0] = (struct iovec) { 187 | .iov_base = &req, 188 | .iov_len = sizeof(req) 189 | }; 190 | 191 | msg = (struct msghdr) { 192 | .msg_name = (void *)&nladdr, 193 | .msg_namelen = sizeof(nladdr), 194 | .msg_iov = iov, 195 | .msg_iovlen = iovlen, 196 | }; 197 | 198 | req.r.diag_ext = cmd; 199 | 200 | if (sendmsg(fd, &msg, 0) < 0) { 201 | close(fd); 202 | return EXIT_FAILURE; 203 | } 204 | 205 | return 0; 206 | } 207 | 208 | /* Operations on generic netlink sockets */ 209 | 210 | int gen_nl_open(char *pname) 211 | { 212 | int rc = EXIT_FAILURE; 213 | 214 | /* Allocate a netlink socket and connect to it */ 215 | sk = nl_socket_alloc(); 216 | if (!sk) { 217 | nl_perror(NLE_NOMEM, "Error"); 218 | return rc; 219 | } 220 | rc = genl_connect(sk); 221 | if (rc) { 222 | nl_perror(rc, "Error"); 223 | rc = EXIT_FAILURE; 224 | goto err1; 225 | } 226 | smc_id = genl_ctrl_resolve(sk, SMC_GENL_FAMILY_NAME); 227 | if (smc_id < 0) { 228 | rc = EXIT_FAILURE; 229 | if (smc_id == -NLE_OBJ_NOTFOUND) 230 | fprintf(stderr, "Error: SMC module not loaded\n"); 231 | else 232 | nl_perror(smc_id, "Error"); 233 | goto err2; 234 | } 235 | 236 | return EXIT_SUCCESS; 237 | err2: 238 | nl_close(sk); 239 | err1: 240 | nl_socket_free(sk); 241 | return rc; 242 | 243 | } 244 | 245 | int gen_nl_handle(int cmd, int nlmsg_flags, 246 | int (*cb_handler)(struct nl_msg *msg, void *arg), void *arg) 247 | { 248 | struct nl_msg *msg; 249 | int rc; 250 | 251 | nl_socket_modify_cb(sk, NL_CB_VALID, NL_CB_CUSTOM, cb_handler, arg); 252 | 253 | /* Allocate a netlink message and set header information. */ 254 | msg = nlmsg_alloc(); 255 | if (!msg) { 256 | nl_perror(NLE_NOMEM, "Error"); 257 | goto errout; 258 | } 259 | 260 | if (!genlmsg_put(msg, NL_AUTO_PORT, NL_AUTO_SEQ, smc_id, 0, nlmsg_flags, 261 | cmd, SMC_GENL_FAMILY_VERSION)) { 262 | nl_perror(NLE_NOMEM, "Error"); 263 | goto errout; 264 | } 265 | 266 | /* Send message */ 267 | rc = nl_send_auto(sk, msg); 268 | if (rc < 0) { 269 | nl_perror(rc, "Error"); 270 | goto errout; 271 | } 272 | 273 | /* Receive reply message, returns number of cb invocations. */ 274 | rc = nl_recvmsgs_default(sk); 275 | if (rc < 0) { 276 | if (rc == -NLE_OPNOTSUPP) { 277 | fprintf(stderr, "Error: Operation not supported by kernel\n"); 278 | } else { 279 | nl_perror(rc, "Error"); 280 | } 281 | goto errout; 282 | } 283 | 284 | nlmsg_free(msg); 285 | return EXIT_SUCCESS; 286 | 287 | errout: 288 | nlmsg_free(msg); 289 | return EXIT_FAILURE; 290 | } 291 | 292 | int gen_nl_handle_dump(int cmd, int (*cb_handler)(struct nl_msg *msg, void *arg), void *arg) 293 | { 294 | return gen_nl_handle(cmd, NLM_F_DUMP, cb_handler, arg); 295 | } 296 | 297 | void gen_nl_close() 298 | { 299 | if (sk) { 300 | nl_close(sk); 301 | nl_socket_free(sk); 302 | sk = NULL; 303 | } 304 | } 305 | 306 | uint64_t nl_attr_get_uint(const struct nlattr *nla) 307 | { 308 | if (nla && nla_len(nla) == sizeof(uint32_t)) 309 | return nla_get_u32(nla); 310 | return nla_get_u64(nla); 311 | } 312 | -------------------------------------------------------------------------------- /libnetlink.h: -------------------------------------------------------------------------------- 1 | /* 2 | * SMC Tools - Shared Memory Communication Tools 3 | * 4 | * Copyright IBM Corp. 2020 5 | * 6 | * Author(s): Ursula Braun 7 | * Guvenc Gulce 8 | * 9 | * 10 | * Userspace program for SMC Information display 11 | * 12 | * All rights reserved. This program and the accompanying materials 13 | * are made available under the terms of the Eclipse Public License v1.0 14 | * which accompanies this distribution, and is available at 15 | * http://www.eclipse.org/legal/epl-v10.html 16 | */ 17 | #ifndef SMC_LIBNETLINK_H_ 18 | #define SMC_LIBNETLINK_H_ 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include 24 | #include 25 | #include 26 | 27 | static const struct nla_policy smc_gen_net_policy[SMC_GEN_MAX + 1] = { 28 | [SMC_GEN_UNSPEC] = { .type = NLA_UNSPEC, }, 29 | [SMC_GEN_STATS] = { .type = NLA_NESTED, }, 30 | [SMC_GEN_FBACK_STATS] = { .type = NLA_NESTED, }, 31 | }; 32 | 33 | struct rtnl_handle { 34 | int fd; 35 | struct sockaddr_nl local; 36 | struct sockaddr_nl peer; 37 | __u32 seq; 38 | __u32 dump; 39 | int proto; 40 | FILE *dump_fp; 41 | int flags; 42 | }; 43 | 44 | #define DIAG_REQUEST(_req, _r, _seq) \ 45 | struct { \ 46 | struct nlmsghdr nlh; \ 47 | _r; \ 48 | } _req = { \ 49 | .nlh = { \ 50 | .nlmsg_type = SOCK_DIAG_BY_FAMILY, \ 51 | .nlmsg_flags = NLM_F_ROOT|NLM_F_REQUEST, \ 52 | .nlmsg_seq = _seq, \ 53 | .nlmsg_len = sizeof(_req), \ 54 | }, \ 55 | } 56 | 57 | int rtnl_open(struct rtnl_handle *rth); 58 | void rtnl_close(struct rtnl_handle *rth); 59 | int rtnl_dump(struct rtnl_handle *rth, void (*handler)(struct nlmsghdr *nlh)); 60 | void parse_rtattr(struct rtattr *tb[], int max, struct rtattr *rta, int len); 61 | int sockdiag_send(int fd, unsigned char cmd); 62 | void set_extension(int ext); 63 | int gen_nl_open(); 64 | void gen_nl_close(); 65 | int gen_nl_handle(int cmd, int nlmsg_flags, 66 | int (*cb_handler)(struct nl_msg *msg, void *arg), void *arg); 67 | int gen_nl_handle_dump(int cmd, int (*cb_handler)(struct nl_msg *msg, void *arg), void *arg); 68 | uint64_t nl_attr_get_uint(const struct nlattr *nla); 69 | #endif /* SMC_LIBNETLINK_H_ */ 70 | -------------------------------------------------------------------------------- /linkgroup.h: -------------------------------------------------------------------------------- 1 | /* 2 | * SMC Tools - Shared Memory Communication Tools 3 | * 4 | * Copyright IBM Corp. 2020 5 | * 6 | * Author(s): Guvenc Gulce 7 | * 8 | * User space program for SMC Information display 9 | * 10 | * All rights reserved. This program and the accompanying materials 11 | * are made available under the terms of the Eclipse Public License v1.0 12 | * which accompanies this distribution, and is available at 13 | * http://www.eclipse.org/legal/epl-v10.html 14 | */ 15 | 16 | #ifndef LINKGROUP_H_ 17 | #define LINKGROUP_H_ 18 | extern struct rtnl_handle rth; 19 | 20 | int invoke_lgs(int argc, char **argv, int detail_level); 21 | 22 | #endif /* LINKGROUP_H_ */ 23 | -------------------------------------------------------------------------------- /seid.c: -------------------------------------------------------------------------------- 1 | /* 2 | * SMC Tools - Shared Memory Communication Tools 3 | * 4 | * Copyright IBM Corp. 2020 5 | * 6 | * Userspace program for SMC Information display 7 | * 8 | * All rights reserved. This program and the accompanying materials 9 | * are made available under the terms of the Eclipse Public License v1.0 10 | * which accompanies this distribution, and is available at 11 | * http://www.eclipse.org/legal/epl-v10.html 12 | */ 13 | #include 14 | #include 15 | #include 16 | #include 17 | #include 18 | #include 19 | 20 | #include "smctools_common.h" 21 | #include "util.h" 22 | #include "libnetlink.h" 23 | #include "seid.h" 24 | 25 | static int enable_cmd = 0; 26 | static int disable_cmd = 0; 27 | static int show_cmd = 0; 28 | 29 | extern int smc_id; 30 | extern struct nl_sock *sk; 31 | 32 | const struct nla_policy 33 | smc_gen_seid_policy[SMC_NLA_SEID_TABLE_MAX + 1] = { 34 | [SMC_NLA_SEID_UNSPEC] = { .type = NLA_UNSPEC }, 35 | [SMC_NLA_SEID_ENTRY] = { .type = NLA_NUL_STRING }, 36 | [SMC_NLA_SEID_ENABLED] = { .type = NLA_U8 }, 37 | }; 38 | 39 | static void usage(void) 40 | { 41 | fprintf(stderr, 42 | "Usage: smcd seid [show]\n" 43 | " smcd seid enable\n" 44 | " smcd seid disable\n" 45 | ); 46 | exit(-1); 47 | } 48 | 49 | /* arg is an (int *) */ 50 | static int is_seid_defined_reply(struct nl_msg *msg, void *arg) 51 | { 52 | struct nlattr *attrs[SMC_GEN_MAX + 1]; 53 | struct nlmsghdr *hdr = nlmsg_hdr(msg); 54 | int *is_seid = (int *)arg; 55 | 56 | if (genlmsg_parse(hdr, 0, attrs, SMC_NLA_SEID_TABLE_MAX, 57 | (struct nla_policy *)smc_gen_seid_policy) < 0) { 58 | fprintf(stderr, "Error: Invalid data returned: smc_gen_seid_policy\n"); 59 | nl_msg_dump(msg, stderr); 60 | return NL_STOP; 61 | } 62 | 63 | if (!attrs[SMC_NLA_SEID_ENTRY]) 64 | *is_seid = 0; 65 | else 66 | *is_seid = 1; 67 | 68 | return NL_OK; 69 | } 70 | 71 | static int is_seid_defined(int *is_seid) 72 | { 73 | *is_seid = 0; 74 | return gen_nl_handle_dump(SMC_NETLINK_DUMP_SEID, is_seid_defined_reply, is_seid); 75 | } 76 | 77 | static int handle_gen_seid_reply(struct nl_msg *msg, void *arg) 78 | { 79 | struct nlattr *attrs[SMC_NLA_SEID_TABLE_MAX + 1]; 80 | struct nlmsghdr *hdr = nlmsg_hdr(msg); 81 | int rc = NL_OK; 82 | char *state; 83 | 84 | if (!show_cmd) 85 | return rc; 86 | 87 | if (genlmsg_parse(hdr, 0, attrs, SMC_NLA_SEID_TABLE_MAX, 88 | (struct nla_policy *)smc_gen_seid_policy) < 0) { 89 | fprintf(stderr, "Error: invalid data returned: smc_gen_seid_policy\n"); 90 | nl_msg_dump(msg, stderr); 91 | return NL_STOP; 92 | } 93 | 94 | if (!attrs[SMC_NLA_SEID_ENTRY] || !attrs[SMC_NLA_SEID_ENABLED]) { 95 | printf("n/a\n"); 96 | return NL_STOP; 97 | } 98 | 99 | if (nla_get_u8(attrs[SMC_NLA_SEID_ENABLED])) 100 | state = "[enabled]"; 101 | else 102 | state = "[disabled]"; 103 | 104 | printf("%s %s\n", nla_get_string(attrs[SMC_NLA_SEID_ENTRY]), state); 105 | return rc; 106 | } 107 | 108 | int gen_nl_seid_handle(int cmd, char dump, int (*cb_handler)(struct nl_msg *msg, void *arg)) 109 | { 110 | int rc = EXIT_FAILURE, nlmsg_flags = 0; 111 | struct nl_msg *msg; 112 | 113 | nl_socket_modify_cb(sk, NL_CB_VALID, NL_CB_CUSTOM, cb_handler, NULL); 114 | 115 | /* Allocate a netlink message and set header information. */ 116 | msg = nlmsg_alloc(); 117 | if (!msg) { 118 | nl_perror(NLE_NOMEM, "Error"); 119 | rc = EXIT_FAILURE; 120 | goto err; 121 | } 122 | 123 | if (dump) 124 | nlmsg_flags = NLM_F_DUMP; 125 | 126 | if (!genlmsg_put(msg, NL_AUTO_PORT, NL_AUTO_SEQ, smc_id, 0, nlmsg_flags, 127 | cmd, SMC_GENL_FAMILY_VERSION)) { 128 | nl_perror(rc, "Error"); 129 | rc = EXIT_FAILURE; 130 | goto err; 131 | } 132 | 133 | /* Send message */ 134 | rc = nl_send_auto(sk, msg); 135 | if (rc < 0) { 136 | nl_perror(rc, "Error"); 137 | rc = EXIT_FAILURE; 138 | goto err; 139 | } 140 | 141 | /* Receive reply message, returns number of cb invocations. */ 142 | rc = nl_recvmsgs_default(sk); 143 | 144 | if (rc < 0) { 145 | /* For cmd "SEID disable" the kernel might return ENOENT when 146 | * no UEID is defined and the SEID cannot be disabled. 147 | * This is mapped to NLE_OBJ_NOTFOUND in libnl, lib/error.c. 148 | */ 149 | if (rc == -NLE_OPNOTSUPP) { 150 | fprintf(stderr, "Error: operation not supported by kernel\n"); 151 | } else if (cmd == SMC_NETLINK_DISABLE_SEID && rc == -NLE_OBJ_NOTFOUND) { 152 | fprintf(stderr, "Error: System EID cannot be disabled because no User EID is defined\n"); 153 | } else { 154 | nl_perror(rc, "Error"); 155 | } 156 | rc = EXIT_FAILURE; 157 | goto err; 158 | } 159 | 160 | nlmsg_free(msg); 161 | return EXIT_SUCCESS; 162 | err: 163 | nlmsg_free(msg); 164 | return rc; 165 | } 166 | 167 | static void handle_cmd_params(int argc, char **argv) 168 | { 169 | if (argc == 0) { 170 | show_cmd = 1; /* no object given, so use the default "show" */ 171 | return; 172 | } 173 | 174 | while (1) { 175 | if (contains(argv[0], "help") == 0) { 176 | usage(); 177 | } else if (contains(argv[0], "enable") == 0) { 178 | enable_cmd = 1; 179 | } else if (contains(argv[0], "disable") == 0) { 180 | disable_cmd = 1; 181 | } else if (contains(argv[0], "show") == 0) { 182 | show_cmd = 1; 183 | break; 184 | } else { 185 | usage(); 186 | } 187 | if (!NEXT_ARG_OK()) 188 | break; 189 | NEXT_ARG(); 190 | } 191 | /* Too many parameters or wrong sequence of parameters */ 192 | if (NEXT_ARG_OK()) 193 | usage(); 194 | 195 | /* Only single cmd expected */ 196 | if ((enable_cmd + disable_cmd + show_cmd) != 1) 197 | usage(); 198 | } 199 | 200 | int invoke_seid(int argc, char **argv, int detail_level) 201 | { 202 | int rc = EXIT_SUCCESS; 203 | 204 | handle_cmd_params(argc, argv); 205 | 206 | if (enable_cmd || disable_cmd) { 207 | int is_seid = 0; 208 | 209 | is_seid_defined(&is_seid); 210 | if (!is_seid) { 211 | printf("Error: System EID not available\n"); 212 | return EXIT_FAILURE; 213 | } 214 | } 215 | 216 | if (enable_cmd) { 217 | rc = gen_nl_seid_handle(SMC_NETLINK_ENABLE_SEID, 0, handle_gen_seid_reply); 218 | } else if (disable_cmd) { 219 | rc = gen_nl_seid_handle(SMC_NETLINK_DISABLE_SEID, 0, handle_gen_seid_reply); 220 | } else if (show_cmd) { 221 | rc = gen_nl_seid_handle(SMC_NETLINK_DUMP_SEID, 1, handle_gen_seid_reply); 222 | } else { 223 | printf("Error: unknown command\n"); /* we should never come here ... */ 224 | } 225 | 226 | return rc; 227 | } 228 | -------------------------------------------------------------------------------- /seid.h: -------------------------------------------------------------------------------- 1 | /* 2 | * SMC Tools - Shared Memory Communication Tools 3 | * 4 | * Copyright IBM Corp. 2020 5 | * 6 | * Userspace program for SMC Information display 7 | * 8 | * All rights reserved. This program and the accompanying materials 9 | * are made available under the terms of the Eclipse Public License v1.0 10 | * which accompanies this distribution, and is available at 11 | * http://www.eclipse.org/legal/epl-v10.html 12 | */ 13 | 14 | #ifndef SEID_H_ 15 | #define SEID_H_ 16 | 17 | extern struct rtnl_handle rth; 18 | 19 | int invoke_seid(int argc, char **argv, int detail_level); 20 | 21 | #endif /* SEID_H_ */ 22 | -------------------------------------------------------------------------------- /smc-device.8: -------------------------------------------------------------------------------- 1 | .\" smc-device.8 2 | .\" 3 | .\" 4 | .\" Copyright IBM Corp. 2020 5 | .\" Author(s): Guvenc Gulce 6 | .\" ---------------------------------------------------------------------- 7 | .\" 8 | .TH SMC-DEVICE 8 "June 2020" "smc-tools" "Linux Programmer's Manual" 9 | 10 | .SH NAME 11 | smc-device \- Print information about SMC devices 12 | .SH "SYNOPSIS" 13 | .sp 14 | .ad l 15 | .in +8 16 | .ti -8 17 | .B smc 18 | .RI "[ " OPTIONS " ]" 19 | .B device 20 | .RI " { " COMMAND " | " 21 | .BR help " }" 22 | .sp 23 | 24 | .ti -8 25 | .BR "smc device" 26 | .RI "[" 27 | .B "show" 28 | .RI "] [" 29 | .B "all" 30 | .RI "] [ " 31 | .B type 32 | .IR TYPE " ] [ " 33 | .B netdev 34 | .IR NETDEV " ] [ " 35 | .B ibdev 36 | .IR IBDEV " ] 37 | 38 | .ti -8 39 | .IR TYPE " := [ " 40 | .BR smcr " | " 41 | .BR smcd " ]" 42 | 43 | .SH "DESCRIPTION" 44 | The 45 | .B smc device 46 | command displays SMC devices and their properties. Devices can be 47 | listed for SMC-R and SMC-D. The command operates with SMC-R type 48 | per default on all command levels. 49 | 50 | .SS smc device show - look at the device properties 51 | 52 | .TP 53 | .B all (default) 54 | Show all the devices. 55 | 56 | .TP 57 | .BI type " TYPE" 58 | List only the devices of the given type. 59 | 60 | .TP 61 | .BI netdev " NETDEV" 62 | List only the device with the given network device name. 63 | 64 | .TP 65 | .BI ibdev " IBDEV" 66 | List only the device ports with the given RoCE (InfiniBand) device name. 67 | 68 | .SH OUTPUT 69 | 70 | .SS "Net-Dev" 71 | Network device name. 72 | .SS "IB-Dev" 73 | RoCE (InfiniBand) device name. 74 | .SS "IB-P" 75 | InfiniBand port of the RoCE device. 76 | .SS "IB-State" 77 | State of the RoCE device port. 78 | .TP 79 | .I 80 | INACTIVE 81 | The RoCE device port is inactive. 82 | .TP 83 | .I 84 | ACTIVE 85 | The RoCE device port is active. 86 | .SS "Type" 87 | Type of the underlying PCI device. 88 | .TP 89 | .I 90 | RoCE_Express 91 | Underlying used device is RoCE Express. 92 | .TP 93 | .I 94 | RoCE_Express2 95 | Underlying used device is RoCE Express 2. 96 | .TP 97 | .I 98 | RoCE_Express3 99 | Underlying used device is RoCE Express 3. 100 | .TP 101 | .I 102 | ISM 103 | Underlying used device is ISM. 104 | .SS "Crit" 105 | Show whether the device is critical i.e. without failover possibility. 106 | .TP 107 | .I 108 | Yes 109 | In case of SMC-R, there is at least one linkgroup running on the 110 | device with state "SINGLE" or locally "ASYMMETRIC" which 111 | means the linkgroups do not have any fail-over device in case of 112 | a failure. 113 | In case of SMC-D, there is at least one linkgroup running on the 114 | ISM device. 115 | .TP 116 | .I 117 | No 118 | In case of SMC-R, there is no linkgroup running on the device with 119 | state "SINGLE" or locally "ASYMMETRIC" which means the linkgroup(s) 120 | have a fallback device in case of a failure. 121 | In case of SMC-D, there is no linkgroup running on the ISM device. 122 | .SS "FID" 123 | Functional ID of the PCI device. 124 | .SS "PCI-ID" 125 | ID of the PCI device. 126 | .SS "PCHID" 127 | Physical channel ID of the PCI device. 128 | .SS "#Links" 129 | Number of links(SMC-R)/linkgroups(SMC-D) on the device. 130 | .SS "PNET-ID" 131 | PNET-ID of the device. "*" means PNET-ID is set by the user. 132 | 133 | .SH "EXAMPLES" 134 | .br 135 | 1. Show all devices of the type smcd: 136 | .br 137 | 138 | \fB# smc device show all type smcd\fP 139 | .br 140 | 141 | 2. Show all devices of the type smcr: 142 | .br 143 | 144 | \fB# smc devices show all type smcr\fP 145 | .br 146 | 147 | 3. Shows all devices with RoCE (InfiniBand) device name "mlx4_0": 148 | .br 149 | 150 | \fB# smc device show ibdev mlx4_0\fP 151 | .br 152 | 153 | 4. Shows all devices with network device name "eth0": 154 | .br 155 | 156 | \fB# smc device show netdev eth0\fP 157 | .br 158 | 159 | .SH SEE ALSO 160 | .br 161 | .BR smcd (8), 162 | .BR smcr (8) 163 | -------------------------------------------------------------------------------- /smc-linkgroup.8: -------------------------------------------------------------------------------- 1 | .\" smc-linkgroup.8 2 | .\" 3 | .\" 4 | .\" Copyright IBM Corp. 2020 5 | .\" Author(s): Guvenc Gulce 6 | .\" ---------------------------------------------------------------------- 7 | .\" 8 | .TH SMC-LINKGROUP 8 "June 2020" "smc-tools" "Linux Programmer's Manual" 9 | 10 | .SH NAME 11 | smc-linkgroup \- Print information about SMC linkgroups and links 12 | .SH "SYNOPSIS" 13 | .sp 14 | .ad l 15 | .in +8 16 | .ti -8 17 | .B smc 18 | .RI "[ " OPTIONS " ]" 19 | .B linkgroup 20 | .RI " { " COMMAND " | " 21 | .BR help " }" 22 | .sp 23 | 24 | .ti -8 25 | .BR "smc linkgroup" " { " show " | " link-show " } [" 26 | .B "all " 27 | .RI "| " LG-ID " ] [ " 28 | .B type 29 | .IR TYPE " ] [ " 30 | .B netdev 31 | .IR NETDEV " ] [ " 32 | .B ibdev 33 | .IR IBDEV " ] 34 | 35 | .ti -8 36 | .IR TYPE " := [ " 37 | .BR smcr " | " 38 | .BR smcd " ]" 39 | 40 | .SH "DESCRIPTION" 41 | The 42 | .B smc linkgroup 43 | command displays linkgroups, links and their properties. Links can be 44 | listed only for SMC-R linkgroups. The command operates with type smcr 45 | per default on all command levels. 46 | 47 | .SS smc linkgroup show - look at the linkgroup properties 48 | 49 | .TP 50 | .B all (default) 51 | Show all the linkgroups. 52 | 53 | .TP 54 | .I LG-ID 55 | Show the linkgroup with the id 56 | .I LG-ID 57 | 58 | .TP 59 | .BI type " TYPE" 60 | List only linkgroups of the given type. 61 | 62 | .TP 63 | .BI netdev " NETDEV" 64 | List linkgroups making use of the given network device only. 65 | 66 | .TP 67 | .BI ibdev " IBDEV" 68 | List only linkgroups of the given RoCE (infiniband) device. 69 | 70 | .SS smc linkgroup link-show - look at the link properties (SMC-R only) 71 | 72 | .TP 73 | .B all (default) 74 | Show all the links of the linkgroups. 75 | 76 | .TP 77 | .I LG-ID 78 | Show links of the linkgroup with the id 79 | .I LG-ID 80 | 81 | .TP 82 | .BI type " TYPE" 83 | List only links of the linkgroups of the given type. 84 | 85 | .TP 86 | .BI netdev " NETDEV" 87 | List only links of the linkgroups making use of the given network device. 88 | 89 | .TP 90 | .BI ibdev " IBDEV" 91 | List only links of the linkgroups of the given RoCE (InfiniBand) device. 92 | 93 | .SH OUTPUT 94 | 95 | .SS "LG-ID" 96 | ID of the linkgroup. 97 | .SS "LG-Role" 98 | Role of the linkgroup. 99 | .TP 100 | .I 101 | SERV 102 | The linkgroup has a SERVER role. 103 | .TP 104 | .I 105 | CLNT 106 | The linkgroup has a CLIENT role. 107 | .SS "LG-Type" 108 | Linkgroup type of the linkgroup. 109 | .TP 110 | .I 111 | NONE 112 | The linkgroup has the initial type. 113 | .TP 114 | .I 115 | SINGLE 116 | The linkgroup has only a single link, i.e. 117 | the local and the peer system can offer one device port only for this linkgroup, 118 | which means a link outage on any side cannot be covered. 119 | .TP 120 | .I 121 | SYM 122 | The linkgroup has two symmetric links, i.e. 123 | the local and the peer system can offer two device ports for this linkgroup, 124 | which means a link outage on any side can be covered. 125 | .TP 126 | .I 127 | ASYMP 128 | The linkgroup has asymmetric links, i.e. 129 | the peer system can offer one device port only for this linkgroup, 130 | which means a link outage on the peer side cannot be covered. 131 | .TP 132 | .I 133 | ASYML 134 | The linkgroup has asymmetric links, i.e. 135 | the local system can offer one device port only for this linkgroup, 136 | which means a link outage on the local side cannot be covered. 137 | .SS "VLAN" 138 | VLAN the linkgroup belongs to. 139 | .SS "#Conns" 140 | Number of connections(sockets) running on the link/linkgroup. 141 | .SS "PNET-ID" 142 | PNET-ID of the linkgroup. "*" means PNET-ID is set by the user. 143 | .SS "Net-Dev" 144 | Network device name corresponding to the link. 145 | .SS "Link-State" 146 | The state of the link. 147 | .TP 148 | .I 149 | LINK_UNUSED 150 | The link is not in use and in initial state. 151 | .TP 152 | .I 153 | LINK_INACTIVE 154 | The link is inactive and will go away. 155 | .TP 156 | .I 157 | LINK_ACTIVATING 158 | The link is being activated with the peer. 159 | .TP 160 | .I 161 | LINK_ACTIVE 162 | The link is active and operates on an established link with the peer. 163 | Data is being exchanged via RDMA. 164 | .SS "Link-UID" 165 | Unique identifier of the link. This identifier consists of linkgroup id and 166 | link id. 167 | .SS "Peer-UID" 168 | Unique identifier of the link on peer side. This identifier consists of 169 | linkgroup id and link id. 170 | .SS "IB-Dev" 171 | Name of the RoCE device used by the link. 172 | .SS "IB-P" 173 | Port of the RoCE device used by the link. 174 | .SS "Local-GID" 175 | GID of the RoCE port used by the link. 176 | .SS "Peer-GID" 177 | GID of the peer RoCE port used by the link. 178 | 179 | .SH "EXAMPLES" 180 | 181 | .br 182 | 183 | 1. Show all linkgroups of the type smcd: 184 | .br 185 | 186 | \fB# smc linkgroup show all type smcd\fP 187 | .br 188 | 189 | 2. Show all links of the linkgroups with type smcr: 190 | .br 191 | 192 | \fB# smc linkgroup link-show all type smcr\fP 193 | .br 194 | 195 | 3. Show all links with linkgroup id 40: 196 | .br 197 | 198 | \fB# smc linkgroup link-show 40\fP 199 | .br 200 | 201 | 4. Show all links on RoCE device "mlx4_0": 202 | .br 203 | 204 | \fB# smc linkgroup link-show ibdev mlx4_0\fP 205 | .br 206 | 207 | 5. Shows all links on network device "eth0": 208 | .br 209 | 210 | \fB# smc linkgroup link-show netdev eth0\fP 211 | .br 212 | 213 | .SH SEE ALSO 214 | .br 215 | .BR smcd (8), 216 | .BR smcr (8) 217 | -------------------------------------------------------------------------------- /smc-preload.c: -------------------------------------------------------------------------------- 1 | /* 2 | * SMC Tools - Shared Memory Communication Tools 3 | * 4 | * Copyright IBM Corp. 2016, 2018 5 | * 6 | * All rights reserved. This program and the accompanying materials 7 | * are made available under the terms of the Eclipse Public License v1.0 8 | * which accompanies this distribution, and is available at 9 | * http://www.eclipse.org/legal/epl-v10.html 10 | */ 11 | #include 12 | #include 13 | #include 14 | #include 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include 24 | #include 25 | #include 26 | 27 | #define DLOPEN_FLAG RTLD_LAZY 28 | 29 | #ifndef AF_SMC 30 | #define AF_SMC 43 31 | #endif 32 | 33 | #ifndef SMCPROTO_SMC 34 | #define SMCPROTO_SMC 0 /* SMC protocol, IPv4 */ 35 | #define SMCPROTO_SMC6 1 /* SMC protocol, IPv6 */ 36 | #endif 37 | 38 | int (*orig_socket)(int domain, int type, int protocol) = NULL; 39 | static void *dl_handle = NULL; 40 | static pthread_mutex_t mutex = PTHREAD_MUTEX_INITIALIZER; 41 | 42 | static void initialize(void); 43 | 44 | static int debug_mode = 0; 45 | 46 | #define GET_FUNC(x) \ 47 | if (dl_handle) { \ 48 | char *err; \ 49 | dlerror(); \ 50 | orig_ ## x=dlsym(dl_handle,#x); \ 51 | if ((!orig_ ## x)&&(err=dlerror())) { \ 52 | fprintf(stderr, "dlsym failed on " #x ": %s\n",err); \ 53 | orig_ ## x=&emergency_ ## x; \ 54 | } \ 55 | } else { \ 56 | orig_ ## x=&emergency_ ## x; \ 57 | } 58 | 59 | static void dbg_msg(FILE *f, const char *format, ...) 60 | { 61 | va_list vl; 62 | 63 | if (debug_mode) { 64 | va_start(vl, format); 65 | vfprintf(f, format, vl); 66 | va_end(vl); 67 | } 68 | } 69 | 70 | static int emergency_socket(int domain, int type, int protocol) 71 | { 72 | errno = EINVAL; 73 | return -1; 74 | } 75 | 76 | static void set_bufsize(int socket, int opt, const char *envname) { 77 | char *val, *end; 78 | int size; 79 | int rc; 80 | 81 | val = getenv(envname); 82 | if (!val) 83 | return; 84 | size = strtol(val, &end, 10); 85 | if (end != NULL) { 86 | switch (toupper(*end)) { 87 | case 'K': size *= 1024; 88 | break; 89 | case 'M': size *= 1048576; 90 | break; 91 | default: break; 92 | } 93 | } 94 | rc = setsockopt(socket, SOL_SOCKET, opt, &size, sizeof(size)); 95 | dbg_msg(stderr, "sockopt %d set to %d\n", opt, size, rc); 96 | } 97 | 98 | int socket(int domain, int type, int protocol) 99 | { 100 | int rc; 101 | 102 | if (!orig_socket) 103 | initialize(); 104 | 105 | /* check if socket is eligible for AF_SMC */ 106 | if ((domain == AF_INET || domain == AF_INET6) && 107 | // see kernel code, include/linux/net.h, SOCK_TYPE_MASK 108 | (type & 0xf) == SOCK_STREAM && 109 | (protocol == IPPROTO_IP || protocol == IPPROTO_TCP)) { 110 | dbg_msg(stderr, "libsmc-preload: map sock to AF_SMC\n"); 111 | if (domain == AF_INET) 112 | protocol = SMCPROTO_SMC; 113 | else /* AF_INET6 */ 114 | protocol = SMCPROTO_SMC6; 115 | 116 | domain = AF_SMC; 117 | } 118 | 119 | rc = (*orig_socket)(domain, type, protocol); 120 | if (rc != -1) { 121 | set_bufsize(rc, SO_SNDBUF, "SMC_SNDBUF"); 122 | set_bufsize(rc, SO_RCVBUF, "SMC_RCVBUF"); 123 | } 124 | 125 | return rc; 126 | } 127 | 128 | static void set_debug_mode(const char *var_name) 129 | { 130 | char *var_value; 131 | 132 | var_value = getenv(var_name); 133 | debug_mode = 0; 134 | if (var_value != NULL) 135 | debug_mode = (var_value[0] != '0'); 136 | } 137 | 138 | static void initialize(void) 139 | { 140 | pthread_mutex_lock(&mutex); 141 | if (orig_socket) { 142 | pthread_mutex_unlock(&mutex); 143 | return; 144 | } 145 | 146 | set_debug_mode("SMC_DEBUG"); 147 | 148 | dl_handle = dlopen(LIBC_SO, DLOPEN_FLAG); 149 | if (!dl_handle) 150 | dbg_msg(stderr, "dlopen failed: %s\n", dlerror()); 151 | GET_FUNC(socket); 152 | pthread_mutex_unlock(&mutex); 153 | } 154 | -------------------------------------------------------------------------------- /smc-tools.autocomplete: -------------------------------------------------------------------------------- 1 | _smc() 2 | { 3 | local cur prev opts 4 | COMPREPLY=() 5 | cur="${COMP_WORDS[COMP_CWORD]}" 6 | prev="${COMP_WORDS[COMP_CWORD-1]}" 7 | opts="device linkgroup info stats ueid -a -d -dd -v" 8 | opts_smcd="device linkgroup info stats ueid seid -a -d -v" 9 | opts_short="device linkgroup" 10 | opts_show="show link-show" 11 | opts_show_smcd="show" 12 | opts_stats="show reset" 13 | opts_ueid="show add del flush" 14 | opts_seid="show enable disable" 15 | opts_type="smcd smcr" 16 | opts_final="all netdev ibdev" 17 | opts_final_smcd="all" 18 | case "${prev}" in 19 | -v) 20 | if [ $1 = "smcr" ]; then 21 | COMPREPLY=( $(compgen -W "${opts_short}" -- ${cur})) 22 | fi 23 | return 0 24 | ;; 25 | -vv) 26 | if [ $1 = "smcr" ]; then 27 | COMPREPLY=( $(compgen -W "${opts_short}" -- ${cur})) 28 | fi 29 | return 0 30 | ;; 31 | device) 32 | if [ $1 = "smcr" ]; then 33 | COMPREPLY=( $(compgen -W "${opts_show}" -- ${cur})) 34 | else 35 | COMPREPLY=( $(compgen -W "${opts_show_smcd}" -- ${cur})) 36 | fi 37 | return 0 38 | ;; 39 | linkgroup) 40 | if [ $1 = "smcr" ]; then 41 | COMPREPLY=( $(compgen -W "${opts_show}" -- ${cur}) ) 42 | else 43 | COMPREPLY=( $(compgen -W "${opts_show_smcd}" -- ${cur}) ) 44 | fi 45 | return 0 46 | ;; 47 | info) 48 | COMPREPLY=( $(compgen -W "${opts_show_smcd}" -- ${cur}) ) 49 | return 0 50 | ;; 51 | show) 52 | if [ $1 = "smcr" ]; then 53 | COMPREPLY=( $(compgen -W "${opts_final}" -- ${cur}) ) 54 | else 55 | COMPREPLY=( $(compgen -W "${opts_final_smcd}" -- ${cur}) ) 56 | fi 57 | return 0 58 | ;; 59 | link-show) 60 | COMPREPLY=( $(compgen -W "${opts_final}" -- ${cur}) ) 61 | return 0 62 | ;; 63 | smcd) 64 | COMPREPLY=( $(compgen -W "${opts_smcd}" -- ${cur}) ) 65 | return 0 66 | ;; 67 | smcr) 68 | COMPREPLY=( $(compgen -W "${opts}" -- ${cur}) ) 69 | return 0 70 | ;; 71 | stats) 72 | COMPREPLY=( $(compgen -W "${opts_stats}" -- ${cur}) ) 73 | return 0 74 | ;; 75 | ueid) 76 | COMPREPLY=( $(compgen -W "${opts_ueid}" -- ${cur}) ) 77 | return 0 78 | ;; 79 | seid) 80 | COMPREPLY=( $(compgen -W "${opts_seid}" -- ${cur}) ) 81 | return 0 82 | ;; 83 | *) 84 | ;; 85 | esac 86 | } 87 | 88 | function _smc_pnet_complete_() { 89 | case "${COMP_WORDS[COMP_CWORD-1]}" in 90 | --interface*|-I*) 91 | COMPREPLY=($(compgen -W "$(ls -1 /sys/class/net/)" -- "${COMP_WORDS[COMP_CWORD]}")) 92 | return;; 93 | --ibdevice*|-D*) 94 | which smc_rnics >/dev/null 95 | if [ $? -eq 0 ]; then 96 | COMPREPLY=($(compgen -W "$(smc_rnics | tail -n +3 | awk '{print($3)}' | uniq)" -- "${COMP_WORDS[COMP_CWORD]}")) 97 | else 98 | COMPREPLY=($(compgen -W "$(ls -1 /sys/bus/pci/devices)" -- "${COMP_WORDS[COMP_CWORD]}")) 99 | fi 100 | return;; 101 | --ibport*|-P*) 102 | ;; 103 | esac 104 | 105 | COMPREPLY=($(compgen -W "--help --version --add --delete --show --flush --interface --ibdevice --ibport" -- "${COMP_WORDS[COMP_CWORD]}")) 106 | } 107 | 108 | function _smc_rnics_complete_() { 109 | case "${COMP_WORDS[COMP_CWORD-1]}" in 110 | --enable|-e) 111 | COMPREPLY=($(compgen -W "$(smc_rnics | grep -e "^ [[:space:]0-9a-f]\{2\} 0" | awk '{print($1)}')" -- "${COMP_WORDS[COMP_CWORD]}")) 112 | return;; 113 | --disable|-d) 114 | COMPREPLY=($(compgen -W "$(smc_rnics | grep -e "^ [[:space:]0-9a-f]\{2\} 1" | awk '{print($1)}')" -- "${COMP_WORDS[COMP_CWORD]}")) 115 | return;; 116 | esac 117 | 118 | COMPREPLY=($(compgen -W "--help --version --disable --enable --rawids" -- "${COMP_WORDS[COMP_CWORD]}")) 119 | } 120 | 121 | function _smc_chk_complete_() { 122 | case "${COMP_WORDS[COMP_CWORD-1]}" in 123 | --pnetid|-i) 124 | COMPREPLY=($(compgen -W "$(ip link show | grep -e "^[0-9]\+:" | awk '{print($2)}' | sed s'/:$//') $(ip link show up | grep -e "^\s*altname" | awk '{print($2)}')" -- "${COMP_WORDS[COMP_CWORD]}")) 125 | return;; 126 | --connect|-C|--port|-p) 127 | COMPREPLY=() 128 | return;; 129 | esac 130 | 131 | COMPREPLY=($(compgen -W "$(ip link show up | grep -e "^[0-9]\+:" | awk '{print($2)}' | sed s'/:$//') $(ip link show up | grep -e "^\s*altname" | awk '{print($2)}') --connect --help --version --debug --pnetid --port --server --static-analysis --live-test --ipv6" -- "${COMP_WORDS[COMP_CWORD]}")) 132 | } 133 | 134 | complete -W "--help --tgz --version" smc_dbg 135 | complete -W "--help --version --all --listening --debug --wide --smcd --smcr" smcss 136 | complete -F _smc smcd 137 | complete -F _smc smcr 138 | complete -F _smc_pnet_complete_ smc_pnet 139 | complete -F _smc_rnics_complete_ smc_rnics 140 | complete -F _smc_chk_complete_ smc_chk 141 | -------------------------------------------------------------------------------- /smc.8: -------------------------------------------------------------------------------- 1 | .\" smc.8 2 | .\" 3 | .\" 4 | .\" Copyright IBM Corp. 2020 5 | .\" Author(s): Guvenc Gulce 6 | .\" ---------------------------------------------------------------------- 7 | .\" 8 | .TH SMC 8 "June 2020" "smc-tools" "Linux Programmer's Manual" 9 | 10 | .SH NAME 11 | smc \- Print information about SMC linkgroups, links, devices 12 | 13 | .SH SYNOPSIS 14 | .B smc 15 | .RI "[ " OPTIONS " ] " OBJECT " { " COMMAND " | " 16 | .BR help " }" 17 | .sp 18 | 19 | .IR OBJECT " := { " 20 | .BR device " | " info " | " linkgroup " | " stats " }" 21 | .sp 22 | 23 | .IR OPTIONS " := { " 24 | \fB\-V\fR[\fIersion\fR] | 25 | \fB\-a\fR[\fIbsolute\fR] | 26 | \fB\-v\fR[\fIerbose\fR] | 27 | \fB\-vv\fR[\fIerbose\fR]} 28 | 29 | .SH OPTIONS 30 | 31 | .TP 32 | .BR "\-V" , " -Version" 33 | Print the version of the 34 | .B smc 35 | utility and exit. 36 | 37 | .TP 38 | .BR "\-a", " \-absolute" 39 | Print absolute statistic value (valid only for stats). 40 | 41 | .TP 42 | .BR "\-v", " \-verbose" 43 | Print detailed information. 44 | 45 | .TP 46 | .BR "\-vv", " \-vverbose" 47 | Print more detailed information. 48 | 49 | .SH SMC - COMMAND SYNTAX 50 | 51 | .SS 52 | .I OBJECT 53 | 54 | .B device 55 | Device(s) as used by SMC 56 | 57 | .TP 58 | .B info 59 | Generic SMC information 60 | 61 | .TP 62 | .B linkgroup 63 | Linkgroup(s) or link(s) as used by SMC 64 | 65 | .TP 66 | .B stats 67 | SMC statistics 68 | 69 | .PP 70 | The names of all objects can be abbreviated down to 71 | a unique stem. For example, 72 | .B device 73 | can be abbreviated to 74 | .B dev 75 | or just 76 | .B d. 77 | Man pages in 78 | .B SEE ALSO 79 | section contain more 80 | information for individual objects. 81 | 82 | .SS 83 | .I COMMAND 84 | 85 | Specifies the action to perform on the object. 86 | The set of possible actions depends on the object type. 87 | As a rule, it is possible to 88 | .BR " show " or " link-show" 89 | objects, but some objects do not allow all of these operations. The 90 | .B help 91 | command is available for all objects. It prints 92 | out a list of available commands and argument syntax conventions. 93 | .sp 94 | If no command is given, a default command 95 | is assumed. 96 | 97 | .SH RETURN CODES 98 | Successful 99 | .IR smc 100 | commands return 0 and display the requested information. 101 | If an error occurs, 102 | .IR smc 103 | writes a message to stderr and completes with a return code other than 0. 104 | .P 105 | .SH SEE ALSO 106 | .BR af_smc (7), 107 | .BR smc-device (8), 108 | .BR smc-info (8), 109 | .BR smc-linkgroup (8) 110 | .BR smc-stats (8) 111 | -------------------------------------------------------------------------------- /smc.c: -------------------------------------------------------------------------------- 1 | /* 2 | * SMC Tools - Shared Memory Communication Tools 3 | * 4 | * Copyright IBM Corp. 2020 5 | * 6 | * Author(s): Guvenc Gulce 7 | * 8 | * Userspace program for SMC Information display 9 | * 10 | * All rights reserved. This program and the accompanying materials 11 | * are made available under the terms of the Eclipse Public License v1.0 12 | * which accompanies this distribution, and is available at 13 | * http://www.eclipse.org/legal/epl-v10.html 14 | */ 15 | 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include 24 | #include 25 | 26 | #include "smctools_common.h" 27 | #include "libnetlink.h" 28 | #include "util.h" 29 | #include "linkgroup.h" 30 | #include "dev.h" 31 | #include "ueid.h" 32 | #include "seid.h" 33 | #include "info.h" 34 | #include "stats.h" 35 | 36 | static int option_detail = 0; 37 | #if defined(SMCD) 38 | char *myname = "smcd"; 39 | #elif defined(SMCR) 40 | char *myname = "smcr"; 41 | #else 42 | char *myname = "smc"; 43 | #endif 44 | 45 | static void version(void) 46 | { 47 | fprintf(stderr, 48 | "%s utility, smc-tools-%s\n", myname, RELEASE_STRING); 49 | exit(-1); 50 | } 51 | static void usage(void) 52 | { 53 | fprintf(stderr, 54 | "Usage: %s [ OPTIONS ] OBJECT {COMMAND | help}\n" 55 | #if defined(SMCD) 56 | "where OBJECT := {info | linkgroup | device | stats | ueid | seid}\n" 57 | " OPTIONS := {-v[ersion] | -d[etails] | -a[bsolute]}\n", myname); 58 | #else 59 | "where OBJECT := {info | linkgroup | device | stats | ueid}\n" 60 | " OPTIONS := {-v[ersion] | -d[etails] | -dd[etails] | -a[bsolute]}\n", myname); 61 | #endif 62 | } 63 | 64 | static int invoke_help(int argc, char **argv, int k) 65 | { 66 | usage(); 67 | return 0; 68 | } 69 | 70 | static const struct cmd { 71 | const char *cmd; 72 | int (*func)(int argc, char **argv, int option_detail); 73 | } cmds[] = { 74 | { "device", invoke_devs }, 75 | { "linkgroup", invoke_lgs }, 76 | { "info", invoke_info }, 77 | { "stats", invoke_stats }, 78 | { "ueid", invoke_ueid }, 79 | #if defined(SMCD) 80 | { "seid", invoke_seid }, 81 | #endif 82 | { "help", invoke_help }, 83 | { 0 } 84 | }; 85 | 86 | static int run_cmd(const char *argv0, int argc, char **argv) 87 | { 88 | const struct cmd *c; 89 | 90 | for (c = cmds; c->cmd; ++c) { 91 | if (contains(argv0, c->cmd) == 0) 92 | return -(c->func(argc-1, argv+1, option_detail)); 93 | } 94 | 95 | #if defined(SMCR) 96 | /* Special warning for those who mixed up smcd and smcr */ 97 | if (contains(argv0, "seid") == 0) { 98 | fprintf(stderr, 99 | "Error: Object \"%s\" is valid for SMC-D only, try \"%s help\".\n", 100 | argv0, myname); 101 | return EXIT_FAILURE; 102 | } 103 | #endif 104 | 105 | fprintf(stderr, "Error: Object \"%s\" is unknown, try \"%s help\".\n", argv0, myname); 106 | return EXIT_FAILURE; 107 | } 108 | 109 | int main(int argc, char **argv) 110 | { 111 | int rc = 0; 112 | 113 | while (argc > 1) { 114 | char *opt = argv[1]; 115 | 116 | if (strcmp(opt, "--") == 0) { 117 | argc--; argv++; 118 | break; 119 | } 120 | if (opt[0] != '-') 121 | break; 122 | if (opt[1] == '-') 123 | opt++; 124 | 125 | if ((strncmp(opt, "-ad", 3) == 0) || (strncmp(opt, "-da", 3) == 0)) { 126 | option_detail = SMC_OPTION_DETAIL_ABS; 127 | } else if (contains(opt, "-absolute") == 0) { 128 | option_detail = SMC_OPTION_ABS; 129 | } else if (contains(opt, "-version") == 0) { 130 | version(); 131 | } else if (contains(opt, "-details") == 0) { 132 | option_detail = SMC_DETAIL_LEVEL_V; 133 | } else if (contains(opt, "-ddetails") == 0) { 134 | option_detail = SMC_DETAIL_LEVEL_VV; 135 | } else if (contains(opt, "-help") == 0) { 136 | usage(); 137 | goto out; 138 | } else { 139 | fprintf(stderr, 140 | "Error: Option \"%s\" is unknown, try \"%s help\".\n", 141 | opt, myname); 142 | exit(-1); 143 | } 144 | argc--; argv++; 145 | } 146 | 147 | if (gen_nl_open(myname)) 148 | exit(1); 149 | if (argc > 1) { 150 | rc = run_cmd(argv[1], argc-1, argv+1); 151 | goto out; 152 | } 153 | usage(); 154 | out: 155 | gen_nl_close(); 156 | return rc; 157 | } 158 | -------------------------------------------------------------------------------- /smc_chk: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Copyright IBM Corp. 2021 4 | 5 | VERSION="1.8.4"; 6 | 7 | 8 | function usage() { 9 | echo; 10 | echo "Usage: smc_chk [OPTIONS] -C "; 11 | echo " smc_chk [OPTIONS] -S"; 12 | echo " smc_chk -i "; 13 | echo; 14 | echo "Check SMC setup"; 15 | echo; 16 | echo " -C, --connect connect to specified IP"; 17 | echo " -d, --debug show debug messages"; 18 | echo " -h, --help display this message"; 19 | echo " -i, --pnetid print PNET ID and exit"; 20 | echo " -p, --port use the next free port starting"; 21 | echo " with PORT (default: $PRT_DFT)"; 22 | echo " -S, --server start server only"; 23 | echo " -v, --version display version info"; 24 | echo " -6, --ipv6 IP address is IPv6"; 25 | echo; 26 | } 27 | 28 | function debug() { 29 | if [ $dbg -gt 0 ]; then 30 | echo "[DEBUG] $@"; 31 | fi 32 | } 33 | 34 | function get_free_port() { 35 | local i; 36 | 37 | for ((i=$1;; i=i+1)); do 38 | ss -tan | awk '{print($4)}' | sed 's/.*://' | sort | uniq | grep -w $i >/dev/null 39 | [ $? -ne 0 ] && break 40 | done 41 | echo $i; 42 | } 43 | 44 | # Params: 45 | # $1 Port 46 | # $2 Set to '-6' for IPv6 47 | function run_server() { 48 | local i; 49 | 50 | cmd="smc_run $srv -p $1 $2"; 51 | debug "Starting server: $cmd"; 52 | $cmd >/dev/null 2>&1 & 53 | pidsrv=$!; 54 | for (( i=0; i<100; i++ )); do # wait 10s max 55 | ss -tln | awk '{print($4)}' | sed 's/.*://' | grep -w $1 >/dev/null 56 | [ $? -eq 0 ] && break; 57 | sleep 0.1; 58 | done 59 | } 60 | 61 | # Implicit params: 62 | # $1 IP 63 | # $2 Port 64 | # $3 Set to '-6' for IPv6 65 | function run_client() { 66 | local mode; 67 | local i; 68 | 69 | cmd="smc_run $clt $1 -p $2 $3"; 70 | debug "Running client: $cmd"; 71 | $cmd >/dev/null 2>&1 & 72 | pidclt=$!; 73 | for (( i=0; i<100; i++ )); do # wait 10s max 74 | res="`smcss | awk -v id="$1:$2" '$5 == id {print($7" "$8)}'`"; 75 | [ "$res" != "" ] && break; 76 | sleep 0.1; 77 | done 78 | kill -INT $pidclt >/dev/null 2>&1; 79 | debug "Client result: $res"; 80 | if [ "$res" == "" ]; then 81 | echo " Failed, no connection" 82 | else 83 | mode=`echo $res | awk '{print($1)}'`; 84 | err_clt="`echo $res | awk '{print($2)}' | sed 's#/.*##'`"; 85 | if [ `echo $res | awk '{print($2)}' | grep -c /` -eq 1 ]; then 86 | err_srv="`echo $res | awk '{print($2)}' | sed 's#.*/##'`"; 87 | else 88 | err_srv=""; 89 | fi 90 | if [ "$mode" == "TCP" ]; then 91 | echo " Failed (TCP fallback), reasons:" 92 | res="`man smcss | grep $err_clt`"; 93 | if [ "$res" == "" ]; then 94 | res="$err_srv (Unkown error code)"; 95 | fi 96 | echo " Client: $res"; 97 | if [ "$err_srv" != "" ]; then 98 | res="`man smcss | grep $err_srv`"; 99 | if [ "$res" == "" ]; then 100 | res=" $err_srv (Unknown error code or non-Linux OS)"; 101 | fi 102 | echo " Server: $res"; 103 | fi 104 | else 105 | echo " Success, using ${mode:0:3}-${mode:3:1}"; 106 | fi 107 | fi 108 | } 109 | 110 | function is_python3_available() { 111 | if ! which python3 >/dev/null; then 112 | echo "Error: python3 is not available"; 113 | signal_handler; 114 | fi 115 | } 116 | 117 | function init_server() { 118 | if [ $init_srv -ne 0 ]; then 119 | return; 120 | fi 121 | init_srv=1; 122 | is_python3_available; 123 | port=`get_free_port $port`; 124 | port6=`get_free_port $(expr $port + 1)`; 125 | srv=`mktemp /tmp/echo-srv.XXXXXX`; 126 | cat <<-EOF > $srv 127 | #!/usr/bin/env python3 128 | 129 | import argparse 130 | import signal 131 | import socket 132 | import sys 133 | 134 | def receiveSignal(signalNumber, frame): 135 | if conn: 136 | conn.close() 137 | s.close() 138 | sys.exit(0) 139 | 140 | signal.signal(signal.SIGINT, receiveSignal) 141 | conn = None 142 | parser = argparse.ArgumentParser(description='Echo server implemented in python3') 143 | parser.add_argument('-p', '--port', required=True, dest='port', action='store', help='listen port') 144 | parser.add_argument('-6', '--ipv6', dest='ipv6', action='store_true', help='IPv6 mode') 145 | args = parser.parse_args() 146 | 147 | host = '' # Symbolic name meaning all available interfaces 148 | if args.ipv6: 149 | s = socket.socket(socket.AF_INET6, socket.SOCK_STREAM) 150 | else: 151 | s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 152 | s.bind((host, int(args.port))) 153 | s.listen(1) 154 | while True: 155 | conn, addr = s.accept() 156 | print('Connected from', addr) 157 | while True: 158 | data = conn.recv(1024) 159 | if not data: 160 | break 161 | conn.close() 162 | EOF 163 | chmod +x $srv; 164 | } 165 | 166 | function init_client() { 167 | if [ $init_clt -ne 0 ]; then 168 | return; 169 | fi 170 | init_clt=1; 171 | is_python3_available; 172 | clt=`mktemp /tmp/echo-clt.XXXXXX`; 173 | cat <<-EOF > $clt 174 | #!/usr/bin/env python3 175 | 176 | import argparse 177 | import socket 178 | import signal 179 | import sys 180 | import time 181 | 182 | def receiveSignal(signalNumber, frame): 183 | s.close() 184 | sys.exit(0) 185 | 186 | signal.signal(signal.SIGINT, receiveSignal) 187 | parser = argparse.ArgumentParser(description='Echo client implemented in python3') 188 | parser.add_argument('-p', '--port', required=True, dest='port', action='store', help='target port') 189 | parser.add_argument(dest='dest', action='store', help='destination address') 190 | parser.add_argument('-6', '--ipv6', dest='ipv6', action='store_true', help='IPv6 mode') 191 | args = parser.parse_args() 192 | 193 | if args.ipv6: 194 | s = socket.socket(socket.AF_INET6, socket.SOCK_STREAM) 195 | else: 196 | s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 197 | s.connect((args.dest, int(args.port))) 198 | time.sleep(10) 199 | EOF 200 | chmod +x $clt; 201 | } 202 | 203 | function test_init() { 204 | init_client; 205 | init_server; 206 | } 207 | 208 | function test_deinit() { 209 | debug "Cleaning up PIDs: $pidsrv $pidsrv6 $pidclt"; 210 | kill -INT $pidsrv 2>/dev/null 211 | [ "$pidsrv6" != "" ] && kill -INT $pidsrv6 2>/dev/null 212 | kill -INT $pidclt 2>/dev/null 213 | [ "$clt" != "" ] && [ -e $clt ] && rm $clt; 214 | [ "$srv" != "" ] && [ -e $srv ] && rm $srv; 215 | } 216 | 217 | function signal_handler() { 218 | test_deinit; 219 | exit 1; 220 | } 221 | 222 | function test_iface() { 223 | local i; 224 | 225 | if [ $mode -eq $MODE_CONNECT ]; then 226 | echo " Live test (SMC-D and SMC-R)"; 227 | else 228 | echo " Live test (SMC-D and SMC-R, EXPERIMENTAL)"; 229 | fi 230 | if [ "$1" != "" ]; then 231 | debug "Determine IP for interface $1"; 232 | ip="`get_netmasks $1 | head -1 | sed 's#/.*##'`"; 233 | if [ "$ip" == "" ]; then 234 | echo " No usable IP address configured, skipping"; 235 | echo; 236 | return; 237 | fi 238 | fi 239 | if [[ $ip == *:* ]]; then 240 | run_client $ip $port6 "-6"; 241 | else 242 | run_client $ip $port; 243 | fi 244 | echo; 245 | } 246 | 247 | function get_netmasks() { 248 | # filter out link-locals 249 | ip addr show $1 | grep -we "inet[6]\?" | grep -v "scope link" | awk '{print($2)}'; 250 | } 251 | 252 | function set_pnetid() { 253 | debug "Determine PNET ID for $1"; 254 | smc_rnics | grep -e "$1\$" >/dev/null; 255 | if [ $? -eq 0 ]; then 256 | # PCI device - use smc_rnics for easy PNET_ID access 257 | debug "PCI device, retrieve PNET ID via smc_rnics"; 258 | pnetid="`smc_rnics | grep -e "$1\$" | awk '{print($7)}'`"; 259 | [ "$pnetid" != "" ] && return; 260 | fi 261 | if [ -e /sys/class/net/$1/device/portno ] && [ -e /sys/class/net/$1/device/chpid ]; then 262 | # CCW device 263 | debug "CCW device, retrieve PNET ID via sysfs"; 264 | portno=`cat /sys/class/net/$1/device/portno`; 265 | chpid=`cat /sys/class/net/$1/device/chpid`; 266 | chpid=${chpid,,}; 267 | pnetids="`cat /sys/devices/css0/chp0.$chpid/util_string | sed 's/\x0/\x40/g' | iconv -f IBM-1047 -t ASCII 2>/dev/null`"; 268 | (( idx=16*$portno+1 )) 269 | (( end=$idx+15 )) 270 | pnetid="`echo "$pnetids" | cut -c $idx-$end | sed 's/ //g'`"; 271 | [ "$pnetid" != "" ] && return; 272 | fi 273 | # Check for a software-defined PNET ID 274 | debug "No luck so far - try the SW PNET table"; 275 | pnetid="`smc_pnet | awk -v id="$1" '$2 == id {print($1)}'`"; 276 | if [ "$pnetid" != "" ]; then 277 | pnetid="$pnetid*"; 278 | fi 279 | debug "PNET ID is '$pnetid'"; 280 | } 281 | 282 | function is_smcd_available() { 283 | # Verify version availability via new 'smcd info' command 284 | smcd info | grep -e "^SMC-D Features:" | grep -w "$1" >/dev/null 285 | 286 | return $?; 287 | } 288 | 289 | # Returns ISM device with PNET ID == $1 290 | function set_ism() { 291 | is_smcd_available "v1"; 292 | [ $? -ne 0 ] && return; 293 | ism="`smc_rnics | awk -v pn="$1" '$5 == "ISM" && $7 == pn {print($1)}' | sed 's/ $//'`"; 294 | } 295 | 296 | # Returns all ISMv2-eligible devices (PNET ID not set or same as *some* NIC) 297 | function set_ismv2() { 298 | local pnet; 299 | local i; 300 | 301 | is_smcd_available "v2"; 302 | [ $? -ne 0 ] && return; 303 | debug "Determine all PNET IDs in use"; 304 | all_pnets="`smc_pnet | awk '{print($1)}'`"; 305 | for i in `get_iface_realname`; do 306 | set_pnetid $i; 307 | all_pnets="$all_pnets"$'\n'"$pnetid"; 308 | done 309 | all_pnets="`echo "$all_pnets" | sort | uniq`"; 310 | debug "All pnets found: `echo $all_pnets | tr '\n' ' '`"; 311 | ismv2=""; 312 | while read line; do 313 | [ "$line" == "" ] && return; 314 | pnet=`echo $line | awk '{print($2)}'`; 315 | if [ "$pnet" != "n/a" ]; then 316 | echo "$all_pnets" | grep -w $pnet >/dev/null; 317 | [ $? -ne 0 ] && continue; 318 | fi 319 | ismv2="$ismv2 `echo $line | awk '{print($1)}'`"; 320 | done <<< $(smc_rnics | awk '$5 == "ISM" {print($1" "$7)}') 321 | ismv2="`echo $ismv2`"; # strip leading blank 322 | } 323 | 324 | function get_mode_param() { 325 | case $1 in 326 | $MODE_STATIC) echo "'-s/--static-analysis'";; 327 | $MODE_LIVE) echo "'-l/--live-test'";; 328 | $MODE_PPNETID) echo "'-i/--pnetid'";; 329 | $MODE_CONNECT) echo "'-C/--connect'";; 330 | $MODE_SERVER) echo "'-S/--server'";; 331 | esac 332 | } 333 | 334 | function set_mode() { 335 | if [ $mode -eq $1 ]; then 336 | return; 337 | fi 338 | if [ $mode -eq $MODE_LIVE ] && [ $1 -eq $MODE_STATIC ]; then 339 | mode=$MODE_ALL; 340 | return; 341 | fi 342 | if [ $mode -lt 0 ]; then 343 | mode=$1; 344 | else 345 | echo "Error: Cannot combine options `get_mode_param $mode` and `get_mode_param $1`"; 346 | exit 1; 347 | fi 348 | } 349 | 350 | function analyze_iface() { 351 | local smcd_configured=0; 352 | local tab=0; 353 | local i; 354 | local TAB1=25; 355 | local TAB2=25; 356 | 357 | echo " Static Analysis (SMC-D only, EXPERIMENTAL)"; 358 | set_pnetid $1; 359 | set_ism $pnetid; 360 | 361 | # SMC version 362 | echo -n " Configuration: "; 363 | if [ "$ism" != "" ]; then 364 | smcd_configured=1; 365 | tab=$TAB1; 366 | echo "SMC-Dv1 (ISMv1 FID(s): $ism)"; 367 | fi 368 | if [ "$ismv2" != "" ]; then 369 | smcd_configured=2; 370 | printf "%*sSMC-Dv2 (ISMv2 FID(s): %s\n" $tab "" "$ismv2)"; 371 | fi 372 | if [ $smcd_configured -eq 0 ]; then 373 | echo "SMC-D not configured"; 374 | echo; 375 | return; 376 | fi 377 | 378 | # PNET ID 379 | echo " PNET ID: $pnetid"; 380 | 381 | # (S) EID in use 382 | tab=0; 383 | smcd seid show >/dev/null 2>&1; 384 | if [ $? -eq 0 ]; then 385 | seid="`smcd seid show | awk '$2 == "[enabled]" {print($1)}'`"; 386 | if [ "$seid" != "" ]; then 387 | echo " Advertising SEID: $seid"; 388 | fi 389 | for i in `smcd ueid show`; do 390 | [ $tab -eq 0 ] && echo -n " Advertising UEIDs: "; 391 | printf "%*s%s\n" $tab " " $i; 392 | tab=$TAB2; 393 | done 394 | fi 395 | 396 | # Reachable IP subnets 397 | echo -n " Reachable subnets: "; 398 | tab=0; 399 | case $smcd_configured in 400 | 0) echo "None";; # Shouldn't happen, but... 401 | 1) for i in `get_netmasks $1`; do 402 | printf "%*s%s\n" $tab "" $i; 403 | tab=$TAB1; 404 | done;; 405 | 2) echo "Any";; 406 | esac 407 | 408 | echo; 409 | } 410 | 411 | # Returns the real name of interface $1 (in case $1 is an altname) 412 | # Call with $1 == "" for a list of all interfaces 413 | # Call with $1 == "up" for a list of all active interfaces 414 | function get_iface_realname() { 415 | ip link show $1 | grep -e "^[0-9]\+:" | awk '{print($2)}' | sed s'/:$//'; 416 | } 417 | 418 | 419 | trap signal_handler SIGINT SIGTERM; 420 | pid=""; 421 | pidclt=""; 422 | pidsrv=""; 423 | pidsrv6=""; 424 | MODE_ALL=0; 425 | MODE_STATIC=1; 426 | MODE_LIVE=2; 427 | MODE_PPNETID=3; 428 | MODE_CONNECT=4; 429 | MODE_SERVER=5; 430 | MODE_DFT=$MODE_LIVE; 431 | PRT_DFT=37373 432 | port=$PRT_DFT; 433 | ipv6=""; 434 | fb=$(tput bold 2>/dev/null) # bold font 435 | fn=$(tput sgr0 2>/dev/null) # normal font 436 | init_clt=0; 437 | init_srv=0; 438 | args=`getopt -u -o C:dhi:lp:sSv6 -l connect:,debug,help,port:,pnetid:,server,static-analysis,live-test,version,ipv6 -- $*`; 439 | [ $? -ne 0 ] && exit 2; 440 | set -- $args; 441 | tgt=""; 442 | mode=-1; 443 | dbg=0; 444 | rc=0; 445 | while [ $# -gt 0 ]; do 446 | case $1 in 447 | "-C" | "--connect" ) 448 | set_mode $MODE_CONNECT; 449 | ip="`getent ahosts $2 | awk '{print($1)}' | head -1`"; 450 | if [ "$ip" == "" ]; then 451 | echo "Error: Unknown destination '$2'"; 452 | exit 1; 453 | fi 454 | ifaces="`ip route get $ip 2>/dev/null | grep -oP '(?<=dev )\w+'`"; 455 | if [ "$ifaces" == "" ]; then 456 | echo "Error: No route to host: $2"; 457 | exit 1; 458 | fi 459 | shift;; 460 | "-d" | "--debug" ) 461 | let dbg++;; 462 | "-h" | "--help" ) 463 | usage; 464 | exit 0;; 465 | "-i" | "--pnetid" ) 466 | set_mode $MODE_PPNETID; 467 | tgt="$2"; 468 | shift;; 469 | "-l" | "--live-test" ) 470 | set_mode $MODE_LIVE;; 471 | "-p" | "--port" ) 472 | port="$2"; 473 | shift;; 474 | "-s" | "--static-analysis" ) 475 | set_mode $MODE_STATIC;; 476 | "-S" | "--server" ) 477 | set_mode $MODE_SERVER; 478 | ifaces="`ip link show up | head -1 | awk '{print($2)}' | sed s'/:$//'`";; 479 | "-v" | "--version" ) 480 | echo "smc_chk utility, smc-tools-$VERSION"; 481 | exit 0;; 482 | "-6" | "--ipv6" ) 483 | ipv6="-6";; 484 | "--" ) ;; 485 | * ) 486 | if [ $mode == $MODE_PPNETID ]; then 487 | echo "Error: Option -P/--print-PNETID takes no extra targets"; 488 | exit 1; 489 | fi 490 | if [ "$tgt" == "" ]; then 491 | tgt="$1"; 492 | else 493 | tgt="$tgt $1"; 494 | fi 495 | esac 496 | shift 497 | done 498 | 499 | if [ $mode -lt 0 ]; then 500 | usage; 501 | exit 0; 502 | fi 503 | 504 | if [ $mode -le $MODE_PPNETID ]; then 505 | if [ "$tgt" != "" ]; then 506 | ipv6=""; # we got an interface - if '-6' was specified, it is moot 507 | up="up"; 508 | for i in $tgt; do 509 | ip link show $i >/dev/null 2>&1; 510 | if [ $? -ne 0 ]; then 511 | echo "Error: Interface $i does not exist"; 512 | exit 1; 513 | fi 514 | if [ $mode -ne $MODE_PPNETID ]; then 515 | if [ `ip link show up $i | wc -l` -eq 0 ]; then 516 | echo "Error: $i is not an active interface"; 517 | exit 2; 518 | fi 519 | fi 520 | done 521 | ifaces="$tgt"; 522 | else 523 | ifaces="`get_iface_realname "up"`"; 524 | fi 525 | fi 526 | debug "Interfaces to check: $ifaces"; 527 | 528 | if [ $mode -eq $MODE_ALL ] || [ $mode -eq $MODE_LIVE ]; then 529 | test_init; 530 | run_server $port6 "-6"; # We cannot know whether we need to check an interface with IPv6, 531 | pidsrv6=$pidsrv; # so we start servers for both to be on the safe side 532 | run_server $port; 533 | fi 534 | 535 | for i in $ifaces; do 536 | i=`get_iface_realname $i`; 537 | case $mode in 538 | $MODE_ALL ) 539 | echo "Checking active link: ${fb}$i${fn}"; 540 | analyze_iface $i; 541 | test_iface $i;; 542 | $MODE_STATIC ) 543 | echo "Checking active link: ${fb}$i${fn}"; 544 | set_ismv2; 545 | analyze_iface $i;; 546 | $MODE_LIVE ) 547 | echo "Checking active link: ${fb}$i${fn}"; 548 | test_iface $i;; 549 | $MODE_PPNETID ) 550 | set_pnetid $i; 551 | [ "$pnetid" != "" ] && echo $pnetid;; 552 | $MODE_CONNECT ) 553 | echo "Test with target IP $ip and port $port"; 554 | port6=$port; 555 | init_client; 556 | test_iface;; 557 | $MODE_SERVER ) 558 | init_server $port; 559 | if [ "$ipv6" == "" ]; then 560 | run_server $port; 561 | p=$port; 562 | else 563 | run_server $port6 "-6"; 564 | p=$port6; 565 | fi 566 | echo "Server started on port $p"; 567 | wait $pidsrv;; 568 | esac 569 | done 570 | test_deinit; 571 | 572 | exit 0; 573 | -------------------------------------------------------------------------------- /smc_chk.8: -------------------------------------------------------------------------------- 1 | .\" Copyright IBM Corp. 2021 2 | 3 | .TH SMC_CHK 8 "January 2021" "smc-tools" "Linux Programmer's Manual" 4 | 5 | 6 | .SH NAME 7 | smc_chk \- SMC support diagnostics 8 | 9 | 10 | .SH SYNOPSIS 11 | .nf 12 | .BI "smc_chk [OPTIONS] -C " IP 13 | .BI "smc_chk [OPTIONS] -S" 14 | .BI "smc_chk -i "INTERFACE 15 | 16 | .SH DESCRIPTION 17 | Use 18 | .B -C/--connect 19 | to obtain diagnostic information about the SMC support of a service that runs at a 20 | given 21 | .IR IP 22 | address. 23 | Otherwise, you can use 24 | .B -S/--server 25 | to start a server before connecting with 26 | .IR -C . 27 | Use 28 | .B -i/--pnetid 29 | to print the PNET ID of a specified 30 | .IR INTERFACE . 31 | 32 | 33 | .SH OPTIONS 34 | .TP 35 | .BI "\-C, \-\-connect " IP 36 | Test SMC-D and SMC-R connectivity to 37 | .IR IP . 38 | Use option 39 | .B -p/--port 40 | to specify a 41 | .IR PORT . 42 | .I IP 43 | can specify any service, including remote services. 44 | By default, 45 | .I IP 46 | is assumed to be in IPv4 format. 47 | Specify 48 | .B -6/--IPv6 49 | if 50 | .I IP 51 | is an address in IPv6 format. 52 | .TP 53 | .BR "\-d, \-\-debug " 54 | Show debug messages. 55 | .TP 56 | .BR "\-h, \-\-help" 57 | Display a brief 58 | .B smc_chk 59 | usage information. 60 | .TP 61 | .BI "\-i, \-\-pnetid " INTERFACE 62 | Print the PNET ID of interface 63 | .I INTERFACE 64 | and exit. An appended asterisk * indicates that the PNET ID was defined via 65 | .BR smc_pnet . 66 | .TP 67 | .BI "\-p, \-\-port " PORT 68 | Use port 69 | .I PORT 70 | for any live tests. When starting a server via 71 | .BR "\-S, \-\-server" , 72 | and if 73 | .I PORT 74 | is already in use, the next free port is used. 75 | .TP 76 | .BR "\-S, \-\-server" 77 | Start a server for manual tests. Use option 78 | .B -p/--port 79 | to specify a port. 80 | .TP 81 | .BR "\-v, \-\-version" 82 | Display version information. 83 | .TP 84 | .BR "\-6, \-\-ipv6" 85 | .I IP 86 | address provided is in IPv6 format. 87 | 88 | 89 | .SH Examples 90 | .SS "Check whether a z/OS instance running at 192.168.37.1 is enabled for \ 91 | SMC using the 3270 console service running on port 23" 92 | smc_chk -C 192.168.37.1 -p 23 93 | 94 | .SS "Print PNET ID of interface eth0" 95 | smc_chk -i eth0 96 | 97 | .SS "Start server on port 45901 (or the next successive free port) for smc_chk -C to connect to" 98 | smc_chk -S -p 45901 99 | 100 | 101 | .SH RETURN CODES 102 | On success, 103 | .B smc_chk 104 | returns 0. 105 | If an error occurs, a return code other than 0 is returned. 106 | .P 107 | 108 | 109 | .SH SEE ALSO 110 | .BR af_smc (7), 111 | .BR smc_pnet (8), 112 | .BR smc_run (8), 113 | .BR smcd (8), 114 | .BR smcr (8), 115 | .BR smcss (8) 116 | -------------------------------------------------------------------------------- /smc_dbg: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Copyright IBM Corp. 2019 4 | 5 | VERSION="1.8.4"; 6 | 7 | 8 | function usage() { 9 | echo; 10 | echo "Usage: smc_dbg [ OPTIONS ]"; 11 | echo; 12 | echo "Collect debug information"; 13 | echo; 14 | echo " -h, --help display this message"; 15 | echo " -t, --tgz generate .tgz file"; 16 | echo " -v, --version display version info"; 17 | echo; 18 | } 19 | 20 | function redirect() { 21 | if [ "$tgz" == "on" ]; then 22 | exec &>$tmpdir/$1; 23 | else 24 | echo; 25 | fi 26 | } 27 | 28 | tgz="off"; 29 | ARCH=`uname -m | cut -c1-4`; 30 | args=`getopt -u -o hvt -l help,version,tgz -- $*`; 31 | [ $? -ne 0 ] && exit 1; 32 | set -- $args; 33 | while [ $# -gt 0 ]; do 34 | case $1 in 35 | "-h" | "--help" ) 36 | usage; 37 | exit 0;; 38 | "-t" | "--tgz" ) 39 | tgz="on";; 40 | "-v" | "--version" ) 41 | echo "smc_dbg utility, smc-tools-$VERSION"; 42 | exit 0;; 43 | * ) 44 | esac 45 | shift; 46 | done 47 | if [ "$tgz" == "on" ]; then 48 | exec 3>&1 4>&2 49 | tmpdir=`mktemp -d /tmp/smc_dbg-XXXXXX`; 50 | fi 51 | 52 | redirect version.txt; 53 | smcss -v 54 | smc_dbg -v 55 | smc_pnet -v 56 | smc_rnics -v 57 | smc_chk -v 58 | smcd -v 59 | smcr -v 60 | 61 | if [ "$ARCH" == "s390" ]; then 62 | redirect devices.txt; 63 | echo "CCW Devices:" 64 | printf " Device CHPID Port PNET ID\n"; 65 | echo " -------------------------------------------"; 66 | for device in `ls -1 /sys/bus/ccwgroup/devices`; do 67 | chpid=`cat /sys/bus/ccwgroup/devices/$device/chpid | tr [A-F] [a-f]`; 68 | osaport=`cat /sys/bus/ccwgroup/devices/$device/portno`; 69 | iface=`cat /sys/bus/ccwgroup/devices/$device/if_name`; 70 | printf " %8s %4s %-4s %s\n" $device 0x$chpid $osaport `smc_chk -i $iface`; 71 | done 72 | echo; 73 | 74 | echo "PCI Devices:" 75 | smc_rnics | sed 's/^/ /'; 76 | 77 | redirect smcss_smcd; 78 | smcss --smcd; 79 | fi 80 | 81 | redirect smcss_all.txt; 82 | smcss --all --debug; 83 | 84 | redirect smcss_smcr; 85 | smcss --smcr; 86 | 87 | redirect pnet_table.txt; 88 | smc_pnet --show; 89 | 90 | redirect smcr_links.txt; 91 | smcr -d linkgroup link-show; 92 | 93 | redirect smcd_lgs.txt; 94 | smcd -d linkgroup show; 95 | 96 | redirect smcd_info.txt; 97 | smcd info; 98 | 99 | redirect smcd_stats.txt; 100 | smcd -d stats; 101 | 102 | redirect smcr_stats.txt; 103 | smcr -d stats; 104 | 105 | 106 | if [ "$tgz" == "on" ]; then 107 | exec >&3 2>&4 108 | cd /tmp; 109 | tar cvfz $tmpdir.tgz `basename $tmpdir` >/dev/null 2>&1; 110 | rm -rf $tmpdir; 111 | echo "Debug output written to $tmpdir.tgz"; 112 | fi 113 | exit 0; 114 | -------------------------------------------------------------------------------- /smc_pnet.8: -------------------------------------------------------------------------------- 1 | .\" smc_pnet.8 2 | .\" 3 | .\" 4 | .\" Copyright IBM Corp. 2017, 2019 5 | .\" Author(s): Thomas Richter 6 | .\" Ursula Braun 7 | .\" ---------------------------------------------------------------------- 8 | .\" 9 | 10 | .TH SMC_PNET 8 "January 2017" "smc-tools" "Linux Programmer's Manual" 11 | 12 | .SH NAME 13 | smc_pnet \- create, destroy, and change the SMC PNET table 14 | 15 | .SH SYNOPSIS 16 | 17 | .B smc_pnet 18 | { \fB\-a\fR | \fB\-\-add\fR \fI\fR } 19 | { \fB\-I\fR | \fB\-\-interface\fR } 20 | .P 21 | .B smc_pnet 22 | { \fB\-a\fR | \fB\-\-add\fR \fI\fR } 23 | { \fB\-D\fR | \fB\-\-ibdevice\fR } 24 | [ \fB\-P\fR | \fB\-\-ibport\fR ] 25 | .P 26 | .B smc_pnet 27 | { \fB\-a\fR | \fB\-\-add\fR \fI\fR } 28 | { \fB\-I\fR | \fB\-\-interface\fR } 29 | { \fB\-D\fR | \fB\-\-ibdevice\fR } 30 | [ \fB\-P\fR | \fB\-\-ibport\fR ] 31 | .P 32 | .B smc_pnet 33 | { \fB\-s\fR | \fB\-\-show\fR \fI\fR } 34 | .P 35 | .B smc_pnet 36 | { \fB\-d\fR | \fB\-\-delete\fR \fI\fR } 37 | .P 38 | .B smc_pnet 39 | { \fB\-f\fR | \fB\-\-flush\fR } 40 | .P 41 | .B smc_pnet 42 | { \fB\-v\fR | \fB\-\-version\fR } 43 | .P 44 | .B smc_pnet 45 | { \fB\-h\fR | \fB\-\-help\fR } 46 | 47 | .SH DESCRIPTION 48 | The SMC protocol requires grouping of standard Ethernet and RoCE networks or ISM 49 | devices. 50 | Such groups are called \fIPhysical Networks\fR (PNETs). The mapping is configured 51 | within a table called \fIpnet table\fR. Any available Ethernet interface can be 52 | combined with an available RDMA-capable network interface card (RNIC) or a 53 | DMA-capable ISM device, if they 54 | belong to the same Converged Ethernet fabric. To configure mapping of a RoCE Adapter 55 | port or an ISM device to a standard Ethernet interface, both devices need to have 56 | the same PNET ID; either hardware-defined or user-defined using the pnet table. 57 | Hardware-defined PNET IDs cannot be overwritten. 58 | .P 59 | The 60 | .B smc_pnet 61 | command configures the pnet table. 62 | 63 | .SH OPTIONS 64 | By default, 65 | .B smc_pnet 66 | shows all entries of the pnet table. 67 | .TP 68 | .IR 69 | defines a name for a grouping of Ethernet interface and RNICs or ISM devices. 70 | A PNET ID consists of up to 16 alphanumeric uppercase characters without blanks. 71 | .TP 72 | .BR "\-a, \-\-add" 73 | creates a new PNET ID definition to the pnet table (if it does not already exist). 74 | Only one PNET ID can be defined for a certain 75 | Ethernet interface, a certain InfiniBand device port or a certain ISM device. 76 | Adding more than one PNET ID fails. Hardware defined PNET IDs cannot be overwritten. 77 | .TP 78 | .BR "\-s, \-\-show" 79 | shows a certain PNET ID definition in the pnet table. 80 | .TP 81 | .BR "\-d, \-\-delete" 82 | deletes an existing PNET ID definition from the pnet table. 83 | .TP 84 | .BR "\-f, \-\-flush" 85 | removes all PNET ID definitions from the pnet table. 86 | .TP 87 | .BR "\-I, \-\-interface " 88 | specifies the name of the Ethernet interface to be added for a certain PNET 89 | ID 90 | definition. 91 | .TP 92 | .BR "\-D, \-\-ibdevice " 93 | specifies the ID of the InfiniBand device or ISM device. 94 | .TP 95 | .BR "\-P, \-\-ibport " 96 | specifies the port number of the InfiniBand device port. Valid numbers are 97 | 1 or 2. 98 | The default value is 1. 99 | .TP 100 | .BR "\-v, \-\-version" 101 | displays smc_pnet program version. 102 | .TP 103 | .BR "\-h, \-\-help" 104 | displays a brief smc_pnet usage information. 105 | 106 | .SH EXAMPLES 107 | .B Define PNET ID ABC for the ethernet device names encf500 and bond0, and define 108 | .B PNET ID ABC for the InfiniBand device ID 0001:00:00.0 (port 2) and the ISM 109 | .B device ID 0004:00:00.0: 110 | .RS 4 111 | .PP 112 | .nf 113 | $ smc_pnet \-a ABC \-I encf500 114 | $ smc_pnet \-a ABC \-I bond0 115 | $ smc_pnet \-a ABC \-D 0001:00:00:00.0 \-P 2 116 | $ smc_pnet \-a ABC \-D 0004:00:00:00.0 117 | .RE 118 | .PP 119 | . 120 | .B Show all pnet table entries: 121 | .RS 4 122 | .PP 123 | .nf 124 | $ smc_pnet 125 | ABC encf500 n/a 255 126 | ABC bond0 n/a 255 127 | ABC n/a 0001:00:00.0 2 128 | ABC n/a 0004:00:00.0 1 129 | .RE 130 | .PP 131 | . 132 | .B Define PNET ID XYZ for the ethernet interface name vlan0201 and the InfiniBand 133 | .B device ID 0001:00:00.0 (port 1): 134 | .RS 4 135 | .PP 136 | $ smc_pnet \-a XYZ \-I vlan0201 \-D 0001:00:00.0 \-P 1 137 | .RE 138 | .PP 139 | . 140 | .B Show all entries for PNET ID XYZ: 141 | .RS 4 142 | .PP 143 | .nf 144 | $ smc_pnet \-s XYZ 145 | XYZ vlan0201 n/a 255 146 | XYZ n/a 0001:00:00.0 1 147 | .RE 148 | .PP 149 | . 150 | .B Delete all pnet table entries with PNET ID named ABC: 151 | .RS 4 152 | .PP 153 | $ smc_pnet \-d ABC 154 | .RE 155 | .PP 156 | . 157 | .B Delete all entries in the pnet table: 158 | .RS 4 159 | .PP 160 | $ smc_pnet \-f 161 | .RE 162 | .PP 163 | . 164 | . 165 | .SH RETURN CODES 166 | Successful \fBsmc_pnet\fR commands return 0. 167 | If an error occurs, \fBsmc_pnet\fR writes a message to stderr and 168 | completes with a return code other than 0. 169 | .P 170 | .SH SEE ALSO 171 | .BR af_smc (7), 172 | .BR smc_chk (8) 173 | .BR smc_rnics (8), 174 | .BR smc_run (8), 175 | .BR smcd (8), 176 | .BR smcr (8), 177 | .BR smcss (8) 178 | -------------------------------------------------------------------------------- /smc_pnet.c: -------------------------------------------------------------------------------- 1 | /* 2 | * Shared Memory Communications over RDMA (SMC-R) and RoCE 3 | * 4 | * Copyright IBM Corp. 2017 5 | * 6 | * Author(s): Thomas Richter 7 | * 8 | * User space program for SMC-R PNET Table manipulation with generic netlink. 9 | * 10 | * All rights reserved. This program and the accompanying materials 11 | * are made available under the terms of the Eclipse Public License v1.0 12 | * which accompanies this distribution, and is available at 13 | * http://www.eclipse.org/legal/epl-v10.html 14 | */ 15 | 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include 24 | #include 25 | #include 26 | #include 27 | #include 28 | 29 | #include 30 | #include 31 | #include 32 | #include 33 | #include 34 | 35 | #include "smctools_common.h" 36 | 37 | static char *progname; 38 | 39 | static struct pnetentry { 40 | char *pnetid; /* Pnetid */ 41 | char *ethname; /* Ethernet device name */ 42 | char *ibname; /* Infiniband/ISM device name */ 43 | int ibport; /* Infiniband device port number */ 44 | unsigned char cmd; /* Command to execute */ 45 | } pnetcmd; 46 | 47 | static void _usage(FILE *dest) 48 | { 49 | fprintf(dest, 50 | "Usage: %s [ OPTIONS ] [pnetid]\n" 51 | "\t-h, --help this message\n" 52 | "\t-v, --version show version information\n" 53 | "\t-a, --add add a pnetid entry, requires interface or ib/ism device\n" 54 | "\t-d, --delete delete a pnetid entry\n" 55 | "\t-s, --show show a pnetid entry\n" 56 | "\t-f, --flush flush the complete pnet table\n" 57 | "\t-I, --interface Ethernet interface name of a pnetid entry\n" 58 | "\t-D, --ibdevice Infiniband/ISM device name of a pnetid entry\n" 59 | "\t-P, --ibport Infiniband device port (default: 1)\n" 60 | "\t\n" 61 | "\tno OPTIONS show complete pnet table\n", 62 | progname); 63 | } 64 | 65 | static void help(void) __attribute__((noreturn)); 66 | static void help(void) 67 | { 68 | _usage(stdout); 69 | exit(EXIT_SUCCESS); 70 | } 71 | 72 | static void usage(void) __attribute__((noreturn)); 73 | static void usage(void) 74 | { 75 | _usage(stderr); 76 | exit(EXIT_FAILURE); 77 | } 78 | 79 | static int convert(char *string) 80 | { 81 | unsigned long no; 82 | char *endp; 83 | 84 | no = strtoul(string, &endp, 0); 85 | if (*endp != '\0' || no > 2) { 86 | fprintf(stderr, "%s invalid ib port:%s\n", progname, string); 87 | usage(); 88 | } 89 | return no; 90 | } 91 | 92 | static const struct option long_opts[] = { 93 | { "interface", 1, 0, 'I' }, 94 | { "ibdevice", 1, 0, 'D' }, 95 | { "ibport", 1, 0, 'P' }, 96 | { "flush", 0, 0, 'f' }, 97 | { "add", 0, 0, 'a'}, 98 | { "show", 0, 0, 's'}, 99 | { "delete", 0, 0, 'd'}, 100 | { "version", 0, 0, 'v' }, 101 | { "help", 0, 0, 'h' }, 102 | { NULL, 0, NULL, 0} 103 | }; 104 | 105 | static struct nla_policy smc_pnet_policy[SMC_PNETID_MAX + 1] = { 106 | [SMC_PNETID_NAME] = { 107 | .type = NLA_STRING, 108 | .maxlen = 17 109 | }, 110 | [SMC_PNETID_ETHNAME] = { 111 | .type = NLA_STRING, 112 | .maxlen = 16 113 | }, 114 | [SMC_PNETID_IBNAME] = { 115 | .type = NLA_STRING, 116 | .maxlen = 64 117 | }, 118 | [SMC_PNETID_IBPORT] = { 119 | .type = NLA_U8, 120 | .maxlen = 1 121 | } 122 | }; 123 | 124 | 125 | /* Netlink library call back handler to be called on data reception. */ 126 | static int cb_handler(struct nl_msg *msg, void *arg) 127 | { 128 | struct nlattr *attrs[SMC_PNETID_MAX + 1]; 129 | struct nlmsghdr *hdr = nlmsg_hdr(msg); 130 | 131 | if (genlmsg_validate(hdr, 0, SMC_PNETID_MAX, smc_pnet_policy) || 132 | genlmsg_parse(hdr, 0, attrs, SMC_PNETID_MAX, smc_pnet_policy) < 0) { 133 | fprintf(stderr, "%s: invalid data returned\n", progname); 134 | nl_msg_dump(msg, stderr); 135 | return NL_STOP; 136 | } 137 | printf("%s %s %s %d\n", nla_get_string(attrs[SMC_PNETID_NAME]), 138 | nla_get_string(attrs[SMC_PNETID_ETHNAME]), 139 | nla_get_string(attrs[SMC_PNETID_IBNAME]), 140 | nla_get_u8(attrs[SMC_PNETID_IBPORT])); 141 | return NL_OK; 142 | } 143 | 144 | static int genl_command(void) 145 | { 146 | int rc = EXIT_FAILURE, id, nlmsg_flags = 0; 147 | struct nl_sock *sk; 148 | struct nl_msg *msg; 149 | 150 | /* Allocate a netlink socket and connect to it */ 151 | sk = nl_socket_alloc(); 152 | if (!sk) { 153 | nl_perror(NLE_NOMEM, progname); 154 | return rc; 155 | } 156 | rc = genl_connect(sk); 157 | if (rc) { 158 | nl_perror(rc, progname); 159 | rc = EXIT_FAILURE; 160 | goto out1; 161 | } 162 | id = genl_ctrl_resolve(sk, SMCR_GENL_FAMILY_NAME); 163 | if (id < 0) { 164 | rc = EXIT_FAILURE; 165 | if (id == -NLE_OBJ_NOTFOUND) 166 | fprintf(stderr, "%s: SMC module not loaded\n", 167 | progname); 168 | else 169 | nl_perror(id, progname); 170 | goto out2; 171 | } 172 | nl_socket_modify_cb(sk, NL_CB_VALID, NL_CB_CUSTOM, cb_handler, NULL); 173 | 174 | /* Allocate a netlink message and set header information. */ 175 | msg = nlmsg_alloc(); 176 | if (!msg) { 177 | nl_perror(NLE_NOMEM, progname); 178 | rc = EXIT_FAILURE; 179 | goto out2; 180 | } 181 | 182 | if ((pnetcmd.cmd == SMC_PNETID_DEL || pnetcmd.cmd == SMC_PNETID_GET) && 183 | !pnetcmd.pnetid) /* List all */ 184 | nlmsg_flags = NLM_F_DUMP; 185 | 186 | if (!genlmsg_put(msg, NL_AUTO_PORT, NL_AUTO_SEQ, id, 0, nlmsg_flags, 187 | pnetcmd.cmd, SMCR_GENL_FAMILY_VERSION)) { 188 | nl_perror(rc, progname); 189 | rc = EXIT_FAILURE; 190 | goto out3; 191 | } 192 | 193 | switch (pnetcmd.cmd) { /* Start message construction */ 194 | case SMC_PNETID_ADD: 195 | if (pnetcmd.ethname) 196 | rc = nla_put_string(msg, SMC_PNETID_ETHNAME, 197 | pnetcmd.ethname); 198 | if (rc < 0) { 199 | nl_perror(rc, progname); 200 | rc = EXIT_FAILURE; 201 | goto out3; 202 | } 203 | 204 | if (pnetcmd.ibname) 205 | rc = nla_put_string(msg, SMC_PNETID_IBNAME, 206 | pnetcmd.ibname); 207 | if (rc < 0) { 208 | nl_perror(rc, progname); 209 | rc = EXIT_FAILURE; 210 | goto out3; 211 | } 212 | 213 | if (pnetcmd.ibname) 214 | rc = nla_put_u8(msg, SMC_PNETID_IBPORT, pnetcmd.ibport); 215 | if (rc < 0) { 216 | nl_perror(rc, progname); 217 | rc = EXIT_FAILURE; 218 | goto out3; 219 | } 220 | /* Fall through intended */ 221 | case SMC_PNETID_DEL: 222 | case SMC_PNETID_GET: 223 | if (!pnetcmd.pnetid) /* List all */ 224 | break; 225 | rc = nla_put_string(msg, SMC_PNETID_NAME, pnetcmd.pnetid); 226 | if (rc < 0) { 227 | nl_perror(rc, progname); 228 | rc = EXIT_FAILURE; 229 | goto out3; 230 | } 231 | } 232 | 233 | /* Send message */ 234 | rc = nl_send_auto(sk, msg); 235 | if (rc < 0) { 236 | nl_perror(rc, progname); 237 | rc = EXIT_FAILURE; 238 | goto out3; 239 | } 240 | 241 | /* Receive reply message, returns number of cb invocations. */ 242 | rc = nl_recvmsgs_default(sk); 243 | /* Kernel commit a9d8b0b1e3d689346b016316bd91980d60c6885d 244 | * introduced a misbehavior that a FLUSH of an empty table 245 | * returned -ENOENT. Fix it in smc-tools as long as kernel patch did'nt 246 | * land in the distros. 247 | */ 248 | if (pnetcmd.cmd == SMC_PNETID_FLUSH && rc != -NLE_OBJ_NOTFOUND) 249 | rc = 0; 250 | if (rc < 0) { 251 | nl_perror(rc, progname); 252 | rc = EXIT_FAILURE; 253 | goto out3; 254 | } 255 | rc = EXIT_SUCCESS; 256 | out3: 257 | nlmsg_free(msg); 258 | out2: 259 | nl_close(sk); 260 | out1: 261 | nl_socket_free(sk); 262 | return rc; 263 | } 264 | 265 | int main(int argc, char **argv) 266 | { 267 | char *slash; 268 | int rc, ch; 269 | 270 | progname = (slash = strrchr(argv[0], '/')) ? slash + 1 : argv[0]; 271 | while ((ch = getopt_long(argc, argv, "I:D:P:fasdhv", long_opts, 272 | NULL )) != EOF) { 273 | switch (ch) { 274 | case 'f': 275 | if (pnetcmd.cmd) 276 | usage(); 277 | pnetcmd.cmd = SMC_PNETID_FLUSH; 278 | break; 279 | case 's': 280 | if (pnetcmd.cmd) 281 | usage(); 282 | pnetcmd.cmd = SMC_PNETID_GET; 283 | pnetcmd.pnetid = optarg; 284 | break; 285 | case 'd': 286 | if (pnetcmd.cmd) 287 | usage(); 288 | pnetcmd.cmd = SMC_PNETID_DEL; 289 | pnetcmd.pnetid = optarg; 290 | break; 291 | case 'a': 292 | if (pnetcmd.cmd) 293 | usage(); 294 | pnetcmd.cmd = SMC_PNETID_ADD; 295 | pnetcmd.pnetid = optarg; 296 | break; 297 | case 'I': 298 | pnetcmd.ethname = optarg; 299 | break; 300 | case 'D': 301 | pnetcmd.ibname = optarg; 302 | break; 303 | case 'P': 304 | pnetcmd.ibport = convert(optarg); 305 | break; 306 | case 'v': 307 | printf("smc_pnet utility, smc-tools-%s\n", 308 | RELEASE_STRING); 309 | exit(0); 310 | case 'h': 311 | help(); 312 | case '?': 313 | default: 314 | usage(); 315 | } 316 | } 317 | 318 | if (optind + 1 < argc) { 319 | fprintf(stderr, "%s too many parameters\n", progname); 320 | usage(); 321 | } 322 | if (optind + 1 == argc) 323 | pnetcmd.pnetid = argv[optind]; 324 | if (!pnetcmd.cmd) { 325 | if (optind < argc) { 326 | fprintf(stderr, "%s: parameters without option\n", 327 | progname); 328 | usage(); 329 | } 330 | pnetcmd.cmd = SMC_PNETID_GET; 331 | } 332 | if (pnetcmd.cmd == SMC_PNETID_FLUSH) { 333 | if (optind < argc) { 334 | fprintf(stderr, "%s: -f takes no parameters\n", 335 | progname); 336 | usage(); 337 | } 338 | } 339 | 340 | if (pnetcmd.cmd == SMC_PNETID_ADD) { 341 | if (!pnetcmd.ethname && !pnetcmd.ibname) { 342 | fprintf(stderr, "%s: interface or device missing\n", 343 | progname); 344 | usage(); 345 | } 346 | if (!pnetcmd.ibport) 347 | pnetcmd.ibport = 1; 348 | } 349 | 350 | if (pnetcmd.cmd == SMC_PNETID_GET || pnetcmd.cmd == SMC_PNETID_DEL) { 351 | if (pnetcmd.ethname) { 352 | fprintf(stderr, "%s: interface %s ignored\n", progname, 353 | pnetcmd.ethname); 354 | pnetcmd.ethname = NULL; 355 | } 356 | if (pnetcmd.ibname) { 357 | fprintf(stderr, "%s: device %s ignored\n", progname, 358 | pnetcmd.ibname); 359 | pnetcmd.ibname = NULL; 360 | } 361 | if (pnetcmd.ibport) { 362 | fprintf(stderr, "%s: ibport %d ignored\n", progname, 363 | pnetcmd.ibport); 364 | pnetcmd.ibport = 0; 365 | } 366 | } 367 | rc = genl_command(); 368 | return rc; 369 | } 370 | -------------------------------------------------------------------------------- /smc_rnics: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Copyright IBM Corp. 2018, 2022 4 | 5 | VERSION="1.8.4"; 6 | 7 | 8 | function usage() { 9 | echo; 10 | echo "Usage: smc_rnics [ OPTIONS ] [ FID ]"; 11 | echo; 12 | echo "List RNICs"; 13 | echo; 14 | echo " -a, --all include disabled devices in output"; 15 | echo " -d, --disable disable the specified FID"; 16 | echo " -e, --enable enable the specified FID"; 17 | echo " -h, --help display this message"; 18 | echo " -I, --IB-dev display IB-dev instead of netdev attributes"; 19 | echo " -r, --rawids display 'type' as raw vendor/device IDs"; 20 | echo " -v, --version display version info"; 21 | echo; 22 | } 23 | 24 | function print_header() { 25 | if [ $IBdev -eq 0 ]; then 26 | printf " FID Power PCI_ID PCHID Type PPrt PNET_ID Net-Dev\n"; 27 | else 28 | printf " FID Power PCI_ID PCHID Type IPrt PNET_ID IB-Dev\n"; 29 | fi 30 | echo '------------------------------------------------------------------------------------------'; 31 | } 32 | 33 | function get_softset_pnet_id() { 34 | local res="n/a"; 35 | local line; 36 | local id; 37 | local iface; 38 | local dev; 39 | local prt; 40 | 41 | while read -r line; do 42 | read id iface dev prt <<< $line; 43 | if [[ ("$iface" != "n/a" && "$iface" == "$int") || ("$dev" != "n/a" && "$dev" == "$addr") ]]; then 44 | if [ "$prt" != "255" ] && [ "$prt" != "$iport" ]; then 45 | continue; 46 | fi 47 | res="$id*"; 48 | fi 49 | done <<< "$(smc_pnet)" 50 | 51 | echo "$res"; 52 | } 53 | 54 | function get_pnet_from_port() { 55 | local idx; 56 | local end; 57 | local lport=$port; 58 | local iport; 59 | local res; 60 | 61 | if [ "$lport" == "" ]; then 62 | echo ""; 63 | return; 64 | fi 65 | if [ "$lport" == "n/a" ] || [ "$dev_type" != "RoCE_Express" ]; then 66 | lport=0; 67 | else 68 | [ $IBdev -ne 0 ] && let lport=$lport-1; 69 | fi 70 | (( iport=$lport+1 )) 71 | (( idx=16*$lport+1 )) 72 | (( end=$idx+15 )) 73 | res="`echo "$pnetids" | cut -c $idx-$end | sed 's/ //g'`"; 74 | if [ "$res" == "" ]; then 75 | res="`get_softset_pnet_id`"; 76 | fi 77 | echo $res; 78 | } 79 | 80 | function print_rnic() { 81 | printf "%8x %-5s %-12s %-4s %-14s %-4s %-17s %s\n" "$((16#$fid))" "$power" "$addr" "$pchid" "$dev_type" "$port" "`get_pnet_from_port`" "$int"; 82 | (( printed++ )); 83 | } 84 | 85 | function set_RoCE_dev_and_port() { 86 | dev_type="$1"; 87 | if [ -e port ]; then 88 | port=`cat port`; 89 | if [ $port -eq 0 ]; then 90 | port="n/a"; 91 | else 92 | if [ $IBdev -eq 0 ]; then 93 | let port=$port-1; 94 | else 95 | port=1; 96 | fi 97 | fi 98 | fi; 99 | } 100 | 101 | function set_by_firmware_lvl() { 102 | local iface; 103 | local name; 104 | local lvl; 105 | 106 | name="Mlx_$id"; 107 | which ethtool >/dev/null 2>&1; 108 | if [ $? -eq 0 ] && [ "$int" != "n/a" ] && [ -d "net" ]; then 109 | iface="`ls -1 net | head -1`"; 110 | lvl="`ethtool -i $iface | grep -e "^firmware-version:" | awk '{print($2)}'`"; 111 | if [ "${lvl%%.*}" == "22" ]; then 112 | name="RoCE_Express3"; 113 | fi 114 | fi 115 | set_RoCE_dev_and_port $name; 116 | } 117 | 118 | function print_rnics() { 119 | # iterate over slots, as powered-off devices won't show elsewhere 120 | for fid in `ls -1 /sys/bus/pci/slots`; do 121 | cd /sys/bus/pci/slots/$fid; 122 | fid="$fid"; 123 | if [ "$target" != "" ] && [ "$fid" != "$target" ]; then 124 | continue; 125 | fi 126 | power=`cat power`; 127 | interfaces=""; 128 | port="n/a"; 129 | addr=""; 130 | int=""; 131 | if [ $power -eq 0 ]; then 132 | # device not yet hotplugged 133 | if [ $all -ne 0 ]; then 134 | dev_type=""; 135 | pchid=""; 136 | pnet=""; 137 | port=""; 138 | print_rnic; 139 | fi 140 | continue; 141 | fi 142 | # device is hotplugged - locate it 143 | for dev in `ls -1 /sys/bus/pci/devices`; do 144 | cd /sys/bus/pci/devices/$dev; 145 | if [ "`cat function_id`" == "0x$fid" ]; then 146 | addr=$dev; 147 | break; 148 | fi 149 | done 150 | if [ "$addr" == "" ]; then 151 | echo "Error: No matching device found for FID $fid" >&2; 152 | continue; 153 | fi 154 | cd /sys/bus/pci/devices/$addr; 155 | id=`cat device`; 156 | vend=`cat vendor`; 157 | dev_type="${vend#0x}:${id#0x}"; 158 | if [ $rawIDs -eq 0 ]; then 159 | case "$vend" in 160 | "0x1014" ) # IBM 161 | case "$id" in 162 | "0x04ed") dev_type="ISM"; 163 | int="n/a";; 164 | *) 165 | continue; 166 | esac;; 167 | "0x15b3" ) # Mellanox 168 | case "$id" in 169 | "0x1003" | \ 170 | "0x1004") dev_type="RoCE_Express";; 171 | "0x1016") set_RoCE_dev_and_port "RoCE_Express2";; 172 | "0x101e") set_by_firmware_lvl;; 173 | *) set_RoCE_dev_and_port "Mlx_$id";; 174 | esac;; 175 | *) [ $all -eq 0 ] && continue 176 | esac 177 | fi 178 | pchid="`cat pchid | sed 's/^0x//'`"; 179 | pnetids="`cat util_string | sed 's/\x0/\x40/g' | iconv -f IBM-1047 -t ASCII`"; 180 | if [ $IBdev -eq 0 ]; then 181 | if [ -d "net" ]; then 182 | interfaces="`ls -1 net`"; 183 | else 184 | int="n/a"; 185 | print_rnic; 186 | continue; 187 | fi 188 | # one device can have multiple interfaces (one per port) 189 | for int in $interfaces; do 190 | cd /sys/bus/pci/devices/$addr/net/$int; 191 | if [ "$dev_type" == "RoCE_Express" ] && [ -e dev_port ]; then 192 | port=`cat dev_port`; 193 | fi 194 | print_rnic; 195 | done 196 | else 197 | if [ -d "infiniband" ]; then 198 | int="`ls -1 infiniband`"; 199 | else 200 | int="n/a"; 201 | print_rnic; 202 | continue; 203 | fi 204 | # only one IB interface per card 205 | cd /sys/bus/pci/devices/$addr/infiniband/$int 206 | for port in `ls -1 ports`; do 207 | print_rnic; 208 | done 209 | fi 210 | done 211 | } 212 | 213 | function format_fid() { 214 | res="${1#0x}"; 215 | 216 | if [[ ! "$res" =~ ^[[:xdigit:]]+$ ]]; then 217 | printf "Error: '%s' is not a valid FID\n" "$res" >&2; 218 | exit 3; 219 | fi 220 | 221 | res="`printf "%08x" $((16#$res))`"; 222 | } 223 | 224 | args=`getopt -u -o hIrvae:d: -l all,enable:,disable:,help,IB-dev,rawids,version -- $*`; 225 | [ $? -ne 0 ] && exit 2; 226 | set -- $args; 227 | action="print"; 228 | rawIDs=0; 229 | all=0; 230 | target=""; 231 | IBdev=0; 232 | printed=0; 233 | while [ $# -gt 0 ]; do 234 | case $1 in 235 | "-a" | "--all" ) 236 | all=1;; 237 | "-e" | "--enable" ) 238 | action="enable"; 239 | fid=$2; 240 | shift;; 241 | "-d" | "--disable" ) 242 | action="disable"; 243 | fid=$2; 244 | shift;; 245 | "-h" | "--help" ) 246 | usage; 247 | exit 0;; 248 | "-I" | "--IB-dev" ) 249 | IBdev=1;; 250 | "-r" | "--rawids" ) 251 | rawIDs=1;; 252 | "-v" | "--version" ) 253 | echo "smc_rnics utility, smc-tools-$VERSION"; 254 | exit 0;; 255 | "--" ) ;; 256 | * ) format_fid "$1"; 257 | target="$res"; 258 | esac 259 | shift; 260 | done 261 | 262 | if [ "`uname -m`" != "s390x" ] && [ "`uname -m`" != "s390" ]; then 263 | printf "Error: s390/s390x supported only\n" >&2; 264 | exit 1; 265 | fi 266 | 267 | if [ "$action" != "print" ]; then 268 | if [ "$target" != "" ]; then 269 | usage; 270 | exit 4; 271 | fi 272 | format_fid "$fid"; 273 | fid="$res"; 274 | ufid=`printf "%x" $((16#$fid))`; # representation without leading zeros 275 | if [ ! -d /sys/bus/pci/slots/$fid ]; then 276 | echo "Error: FID $ufid does not exist" >&2; 277 | exit 5; 278 | fi 279 | power=`cat /sys/bus/pci/slots/$fid/power 2>/dev/null`; 280 | val=0; 281 | [ "$action" == "enable" ] && val=1; 282 | if [ $power -eq $val ]; then 283 | echo "Error: FID $ufid is already ${action}d" >&2; 284 | exit 6; 285 | fi 286 | echo $val > /sys/bus/pci/slots/$fid/power 2>/dev/null; 287 | if [ $? -ne 0 ]; then 288 | echo "Error: Failed to $action FID $ufid" >&2; 289 | exit 7; 290 | fi 291 | exit 0; 292 | fi 293 | 294 | print_header; 295 | print_rnics | sort -k 1; 296 | 297 | if [ "$target" != "" ] && [ $printed -eq 0 ]; then 298 | exit 8; 299 | fi 300 | 301 | exit 0; 302 | -------------------------------------------------------------------------------- /smc_rnics.8: -------------------------------------------------------------------------------- 1 | .\" Copyright IBM Corp. 2018 2 | 3 | .TH SMC_RNICS 8 "October 2018" "smc-tools" "Linux Programmer's Manual" 4 | 5 | 6 | .SH NAME 7 | smc_rnics \- list, enable and disable (R)NICS as used by SMC-R and SMC-D 8 | 9 | 10 | .SH SYNOPSIS 11 | .B smc_rnics 12 | .RB [ \-ahrv ] 13 | .RB [ \-d 14 | .IR FID ] 15 | .RB [ \-e 16 | .IR FID ] 17 | .RI [ FID ] 18 | 19 | 20 | .SH DESCRIPTION 21 | The SMC protocol requires an (R)NIC for the (R)DMA traffic. 22 | Use 23 | .B smc_rnics 24 | to handle, and to identify hotplugged (R)NICs. 25 | 26 | 27 | .SH OPTIONS 28 | By default, 29 | .B smc_rnics 30 | shows all enabled (R)NICs in the system. 31 | .TP 32 | .I FID 33 | Limit output to the specified function ID 34 | .RI ( FID ). 35 | .TP 36 | .BR "\-a, \-\-all" 37 | Include disabled and unknown devices in output 38 | .TP 39 | .BR "\-d, \-\-disable " \fIFID 40 | Set (R)NIC 41 | .I FID 42 | offline. 43 | .TP 44 | .BR "\-e, \-\-enable " \fIFID 45 | Set (R)NIC 46 | .I FID 47 | online. Note that devices other than (R)NICs are not displayed by default and 48 | therefore, once enabled, will not appear in further output unless option 49 | .BR -a 50 | or 51 | .BR -r 52 | is specified. 53 | .TP 54 | .BR "\-h, \-\-help" 55 | Display a brief 56 | .B smc_rnics 57 | usage information. 58 | .TP 59 | .BR "\-I, \-\-IB-dev" 60 | Display IB device information. 61 | .TP 62 | .BR "\-r, \-\-rawids" 63 | Display raw PCI vendor and device codes in column. Note that this will 64 | also include unknown devices. 65 | .TP 66 | .BR "\-v, \-\-version" 67 | Display version information. 68 | 69 | .SH OUTPUT 70 | .SS "FID" 71 | Function ID. 72 | 73 | .SS "Power" 74 | Indicates whether the PCI slot is on (1) or off (0). See option 75 | .B -e 76 | on how to enable a device that is offline. 77 | 78 | .SS "PCI_ID" 79 | PCI ID in BDF (Bus:Device.Function) notation. 80 | 81 | .SS "PCHID" 82 | Physical channel identifier, or virtual channel identifier (VCHID) for 83 | ISM devices. 84 | 85 | .SS "Type" 86 | Device type in human readable form. See option 87 | .B -r 88 | to switch to display of PCI vendor and device instead. 89 | 90 | .SS "PPrt" 91 | Corresponding physical port of an RNIC, if applicable. Starts counting at 0. 92 | 93 | .SS "IPrt" 94 | Corresponding Infiniband port of an RNIC, if applicable. Starts counting at 1. 95 | 96 | .SS "PNET_ID" 97 | Physical network ID. Has an asterisk 98 | .B * 99 | appended if defined via 100 | .BR smc_pnet . 101 | 102 | .SS "Net-Dev" 103 | Network interface in Linux, if applicable. 104 | 105 | .SS "IB-Dev" 106 | Infiniband interface in Linux, if applicable. 107 | 108 | 109 | .SH RETURN CODES 110 | On success, 111 | .B smc_rnics 112 | returns 0. 113 | If an error occurs, a return code other than 0 is returned. 114 | .P 115 | 116 | 117 | .SH SEE ALSO 118 | .BR af_smc (7), 119 | .BR smc_pnet (8), 120 | .BR smc_run (8), 121 | .BR smcd (8), 122 | .BR smcr (8), 123 | .BR smcss (8) 124 | -------------------------------------------------------------------------------- /smc_run: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # SMC Tools - Shared Memory Communication Tools 4 | # 5 | # Copyright IBM Corp. 2017, 2018 6 | # 7 | # All rights reserved. This program and the accompanying materials 8 | # are made available under the terms of the Eclipse Public License v1.0 9 | # which accompanies this distribution, and is available at 10 | # http://www.eclipse.org/legal/epl-v10.html 11 | # 12 | LIB_NAME="libsmc-preload.so" 13 | VERSION="1.8.4"; 14 | 15 | 16 | function usage() { 17 | echo; 18 | echo "Usage: smc_run [ OPTIONS ] COMMAND"; 19 | echo; 20 | echo "Run COMMAND using SMC for TCP sockets"; 21 | echo; 22 | echo " -d enable debug mode"; 23 | echo " -h display this message"; 24 | echo " -r request receive buffer size in Bytes"; 25 | echo " -t request transmit buffer size in Bytes"; 26 | echo " -v display version info"; 27 | } 28 | 29 | function check_size() { 30 | if [[ ! "$1" =~ ^[0-9]+[k|K|m|M]?$ ]]; then 31 | echo "Error: Invalid buffer size specified: '$1'"; 32 | exit 1; 33 | fi 34 | return 35 | } 36 | 37 | function adjust_core_net_max() { 38 | case ${2: -1} in 39 | k | K) (( OPTARG=${2%?}*1024 ));; 40 | m | M) (( OPTARG=${2%?}*1048576 ));; 41 | *) 42 | esac 43 | if [ `sysctl -n net.core.$1_max` -lt $OPTARG ]; then 44 | sysctl -w net.core.$1_max=$OPTARG >/dev/null; 45 | fi 46 | } 47 | 48 | # 49 | # Verify command line arguments and specify the preload library debug mode 50 | # if necessary. 51 | # 52 | SMC_DEBUG=0; 53 | while getopts "dhr:t:v" opt; do 54 | case $opt in 55 | d) 56 | SMC_DEBUG=1;; 57 | h) usage; 58 | exit 0;; 59 | r) check_size $OPTARG; 60 | adjust_core_net_max rmem $OPTARG; 61 | export SMC_RCVBUF=$OPTARG;; 62 | t) check_size $OPTARG; 63 | adjust_core_net_max wmem $OPTARG; 64 | export SMC_SNDBUF=$OPTARG;; 65 | v) echo "smc_run utility, smc-tools-$VERSION"; 66 | exit;; 67 | \?) echo "`basename "$0"`: Error: Invalid option: -$OPTARG"; 68 | exit 1;; 69 | esac 70 | done 71 | 72 | shift $(expr $OPTIND - 1); 73 | if [ $# -eq 0 ]; then 74 | echo "`basename "$0"`: Error: Missing command parameter"; 75 | exit 1; 76 | fi 77 | 78 | export SMC_DEBUG; 79 | # 80 | # Execute the specified command. 81 | # 82 | export LD_PRELOAD=$LD_PRELOAD:$LIB_NAME; 83 | 84 | exec "$@" 85 | exit $?; 86 | -------------------------------------------------------------------------------- /smc_run.8: -------------------------------------------------------------------------------- 1 | .\" smc_run.8 2 | .\" 3 | .\" 4 | .\" Copyright IBM Corp. 2017 5 | .\" Author(s): Ursula Braun 6 | .\" ---------------------------------------------------------------------- 7 | .\" 8 | .TH SMC_RUN 8 "January 2017" "smc-tools" "Linux Programmer's Manual " 9 | .SH NAME 10 | smc_run \- start a TCP socket program with the capability to use SMC as 11 | networking protocol 12 | 13 | .SH SYNOPSIS 14 | 15 | .B smc_run 16 | .RB [ \-dhrtv ] [ \-r 17 | .IR SIZE ] 18 | .RB [-t 19 | .IR SIZE ] 20 | .I program 21 | .I parameters 22 | 23 | .SH DESCRIPTION 24 | .B smc_run 25 | starts a 26 | .IR program 27 | specified as argument with its 28 | .IR parameters , 29 | allowing to use the SMC protocol for program-used TCP socket connections. 30 | .br 31 | The script specifies libsmc-preload.so as a preload shared library for the 32 | Linux program loader, and may adjust transmit and receive buffer sizes to 33 | .I SIZE 34 | by setting socket options SO_RCVBUF and SO_SNDBUF respectively. 35 | .I SIZE 36 | can be specified in Bytes or using metric prefixes k and m, e.g. 37 | 8k...1024k/1m. 38 | .br 39 | The preload library libsmc-preload.so intercepts a few TCP socket calls and 40 | triggers the equivalent execution through SMC. 41 | .br 42 | Note: If it is not possibile to use 43 | .IR smc_run , 44 | the libsmc-preload.so may be installed as apreload library via environment 45 | variable LD_PRELOAD. Use environment varibles SMC_SNDBUF and SMC_RCVBUF to 46 | request specific transmit and receive buffer sizes respectively. Supports 47 | metric prefixes k and m. 48 | 49 | The following options can be specified: 50 | .TP 51 | .BR "\-d" 52 | Display additional diagnostic messages during the program execution. 53 | .TP 54 | .BR "\-h" 55 | Display a brief usage information. 56 | .TP 57 | .BR "\-r " \fISIZE 58 | Request receive buffer size 59 | .IR SIZE . 60 | .br 61 | .BR Notes : 62 | .RS 63 | .IP \[bu] 2 64 | May be overridden by the application. 65 | .IP \[bu] 66 | Increases net.core.rmem_max if necessary. 67 | .RE 68 | .TP 69 | .BR "\-t " \fISIZE 70 | Request transmit buffer size 71 | .IR SIZE . 72 | .br 73 | .BR Notes : 74 | .RS 75 | .IP \[bu] 2 76 | May be overridden by the application. 77 | .IP \[bu] 78 | Increases net.core.wmem_max if necessary. 79 | .RE 80 | .TP 81 | .BR "\-v" 82 | Display version information. 83 | .SH RETURN CODES 84 | On success, the 85 | .IR smc_run 86 | command returns 0. 87 | If an error occurs 88 | .IR smc_run 89 | writes a message to stdout and completes with a return code other 90 | than 0. 91 | 92 | .TP 93 | .B 1 94 | An invalid option was specified. 95 | .P 96 | 97 | .SH EXAMPLES 98 | .B Run program foo using SMC, requesting a receive buffer size of 512KB 99 | .RS 4 100 | .PP 101 | $ smc_run -r 512k ./foo 102 | .P 103 | 104 | .SH SEE ALSO 105 | .BR af_smc (7), 106 | .BR smc_chk (8) 107 | .BR smc_pnet (8), 108 | .BR smc_rnics (8), 109 | .BR smcd (8), 110 | .BR smcr (8), 111 | .BR smcss (8), 112 | .BR tcp (7) 113 | -------------------------------------------------------------------------------- /smcd-device.8: -------------------------------------------------------------------------------- 1 | .\" smcd-device.8 2 | .\" 3 | .\" 4 | .\" Copyright IBM Corp. 2020 5 | .\" Author(s): Guvenc Gulce 6 | .\" ---------------------------------------------------------------------- 7 | .\" 8 | .TH SMCD-DEVICE 8 "June 2020" "smc-tools" "Linux Programmer's Manual" 9 | 10 | .SH NAME 11 | smcd-device \- Print information about SMC-D devices 12 | 13 | smcr-device \- Print information about SMC-R devices 14 | 15 | .SH "SYNOPSIS" 16 | .sp 17 | .ad l 18 | .in +8 19 | .ti -8 20 | .B smcd 21 | .RI "[ " OPTIONS " ]" 22 | .B device 23 | .RI " { " COMMAND " | " 24 | .BR help " }" 25 | .sp 26 | 27 | .ti -8 28 | .BR "smcd device" 29 | .RI "[" 30 | .B "show" 31 | .RI "] [" 32 | .B "all" 33 | .RI "] " 34 | 35 | .ti -8 36 | .B smcr 37 | .RI "[ " OPTIONS " ]" 38 | .B device 39 | .RI " { " COMMAND " | " 40 | .BR help " }" 41 | .sp 42 | 43 | .ti -8 44 | .BR "smcr device" 45 | .RI "[" 46 | .B "show" 47 | .RI "] [" 48 | .B "all" 49 | .RI "] [ " 50 | .B netdev 51 | .IR NETDEV " ] [ " 52 | .B ibdev 53 | .IR IBDEV " ] 54 | 55 | .SH "DESCRIPTION" 56 | The 57 | .B smcd device 58 | command displays SMC-D devices and their properties. 59 | The 60 | .B smcr device 61 | command displays SMC-R devices and their properties. 62 | 63 | .SS smcd,smcr device show 64 | inspect the device properties 65 | 66 | .TP 67 | .B all (default) 68 | List all devices. 69 | 70 | .TP 71 | .BI netdev " NETDEV" 72 | .B SMC-R 73 | only: limit the command output to the device with the specified network device name. 74 | 75 | .TP 76 | .BI ibdev " IBDEV" 77 | .B SMC-R 78 | only: limit the command output to the device port with the specified RoCE device name. 79 | 80 | .SH OUTPUT 81 | 82 | .SS "Net-Dev" 83 | Network device name. 84 | .SS "IB-Dev" 85 | RoCE (InfiniBand) device name. 86 | .SS "IB-P" 87 | InfiniBand port of the RoCE device. According to the InfiniBand conventions, the port count starts at 1. Consequently, devices where each port is represented as a separate device will indicate the port as the first port for all ports. 88 | .SS "IB-State" 89 | State of the RoCE device port. 90 | .TP 91 | .I 92 | INACTIVE 93 | The RoCE device port is inactive. 94 | .TP 95 | .I 96 | ACTIVE 97 | The RoCE device port is active. 98 | .SS "Type" 99 | Type of the underlying PCI device. 100 | .TP 101 | .I 102 | RoCE_Express 103 | Underlying used device is RoCE Express. 104 | .TP 105 | .I 106 | RoCE_Express2 107 | Underlying used device is RoCE Express 2. 108 | .TP 109 | .I 110 | ISM 111 | Underlying used device is ISM. 112 | .SS "Crit / InUse" 113 | Show whether the device is critical i.e. without failover possibility(SMC-R) or 114 | whether the device is in use.(SMC-D) 115 | .TP 116 | .I 117 | Yes 118 | For SMC-R, there is at least one link group running on the 119 | device with state "SINGLE" or locally "ASYMMETRIC" which 120 | means one or more link groups without a failover device in case of a link failure. 121 | In case of SMC-D, there is at least one link group running on the 122 | ISM device. 123 | .TP 124 | .I 125 | No 126 | For SMC-R, there is no link group running on the device with 127 | state "SINGLE" or locally "ASYMMETRIC" which means the link group(s) 128 | have a fallback device in case of a failure. 129 | In case of SMC-D, there is no link group running on the ISM device. 130 | .SS "FID" 131 | Function ID of the PCI device. 132 | .SS "PCI-ID" 133 | ID of the PCI device. 134 | .SS "PCHID" 135 | Physical channel ID of the PCI device. 136 | .SS "#Links" 137 | Number of links (SMC-R)/link groups (SMC-D) on the device. 138 | .SS "PNET-ID" 139 | PNET ID of the device. Leading asterisk "*" means PNET ID is set by the user. (e.g "*PNET-ID") 140 | 141 | .SH "EXAMPLES" 142 | .br 143 | .HP 2 144 | 1. Show all SMC-D devices: 145 | .br 146 | \fB# smcd device show all\fP 147 | .br 148 | .HP 2 149 | 2. Show all SMC-R devices: 150 | .br 151 | \fB# smcr devices show all\fP 152 | .br 153 | .HP 2 154 | 3. Show all SMC-R devices with RoCE (InfiniBand) device name "mlx4_0": 155 | .br 156 | \fB# smcr device show ibdev mlx4_0\fP 157 | .br 158 | .HP 2 159 | 4. Show all SMC-R devices with network device name "eth0": 160 | .br 161 | \fB# smcr device show netdev eth0\fP 162 | .br 163 | .SH SEE ALSO 164 | .br 165 | .BR smcd (8), 166 | .BR smcr (8) 167 | -------------------------------------------------------------------------------- /smcd-info.8: -------------------------------------------------------------------------------- 1 | .\" smcd-info.8 2 | .\" 3 | .\" 4 | .\" Copyright IBM Corp. 2021 5 | .\" ---------------------------------------------------------------------- 6 | .\" 7 | .TH SMCD-INFO 8 "January 2021" "smc-tools" "Linux Programmer's Manual" 8 | 9 | 10 | .SH NAME 11 | smcd-info \- Print generic information about SMC 12 | 13 | smcr-info \- Print generic information about SMC 14 | 15 | 16 | .SH "SYNOPSIS" 17 | .sp 18 | .ad l 19 | .in +8 20 | .ti -8 21 | .B smcd 22 | .RI "[ " OPTIONS " ]" 23 | .B info 24 | .RI " { " 25 | .BR show " | " 26 | .BR help " }" 27 | .sp 28 | 29 | .ti -8 30 | .B smcr 31 | .RI "[ " OPTIONS " ]" 32 | .B info 33 | .RI " { " 34 | .BR show " | " 35 | .BR help " } " 36 | .sp 37 | 38 | 39 | .SH "DESCRIPTION" 40 | The 41 | .B smcd info 42 | and 43 | .B smcr info 44 | commands display generic SMC information from the kernel and the hardware. 45 | 46 | .SS smcd,smcr info show 47 | show the information output 48 | .TP 49 | .SS Kernel Capabilities 50 | Shows the Linux kernel's SMC capabilities independent of any hardware 51 | prerequisites 52 | .TP 53 | .SS Hardware Capabilities 54 | Shows the hardware's capabilities independent of support in Linux 55 | 56 | 57 | .SH "EXAMPLES" 58 | 59 | .HP 2 60 | 1. Show the SMC information 61 | .br 62 | \fB# smcd info\fP 63 | .br 64 | \fB# smcr info\fP 65 | .br 66 | 67 | 68 | .SH SEE ALSO 69 | .br 70 | .BR smcd (8), 71 | .BR smcr (8) 72 | -------------------------------------------------------------------------------- /smcd-linkgroup.8: -------------------------------------------------------------------------------- 1 | .\" smcd-linkgroup.8 2 | .\" 3 | .\" 4 | .\" Copyright IBM Corp. 2020 5 | .\" Author(s): Guvenc Gulce 6 | .\" ---------------------------------------------------------------------- 7 | .\" 8 | .TH SMCD-LINKGROUP 8 "June 2020" "smc-tools" "Linux Programmer's Manual" 9 | 10 | .SH NAME 11 | smcd-linkgroup \- Print information about SMC-D link groups 12 | 13 | smcr-linkgroup \- Print information about SMC-R link groups and links 14 | 15 | .SH "SYNOPSIS" 16 | .sp 17 | .ad l 18 | .in +8 19 | .ti -8 20 | .B smcd 21 | .RI "[ " OPTIONS " ]" 22 | .B linkgroup 23 | .RI " { " COMMAND " | " 24 | .BR help " }" 25 | .sp 26 | 27 | .ti -8 28 | .BR "smcd linkgroup" " [ " show " ] [" 29 | .B "all " 30 | .RI "| " LG-ID " 31 | .RI ] 32 | 33 | .ti -8 34 | .B smcr 35 | .RI "[ " OPTIONS " ]" 36 | .B linkgroup 37 | .RI " { " COMMAND " | " 38 | .BR help " }" 39 | .sp 40 | 41 | .ti -8 42 | .BR "smcr linkgroup" " { " show " | " link-show " } [" 43 | .B "all " 44 | .RI "| " LG-ID " ] [ " 45 | .B netdev 46 | .IR NETDEV " ] [ " 47 | .B ibdev 48 | .IR IBDEV " ] 49 | 50 | .SH "DESCRIPTION" 51 | The 52 | .B smcd linkgroup 53 | command displays SMC-D link groups and their properties. 54 | The 55 | .B smcr linkgroup 56 | command displays SMC-R link groups and links with their properties. 57 | 58 | .SS smcd,smcr linkgroup show 59 | inspect the link group properties 60 | 61 | .TP 62 | .B all (default) 63 | Show all link groups. 64 | 65 | .TP 66 | .I LG-ID 67 | Show the link group with the ID 68 | .I LG-ID 69 | 70 | .TP 71 | .BI netdev " NETDEV" 72 | .B SMC-R 73 | only: list the link groups for the specified network device. 74 | 75 | .TP 76 | .BI ibdev " IBDEV" 77 | .B SMC-R 78 | only: list the link groups for the specified RoCE device. 79 | 80 | .SS smcr linkgroup link-show 81 | SMC-R only: Inspect the link properties 82 | 83 | .TP 84 | .B all (default) 85 | List all links of all link groups. 86 | 87 | .TP 88 | .I LG-ID 89 | Show links of the link group with the id. 90 | .I LG-ID 91 | 92 | .TP 93 | .BI netdev " NETDEV" 94 | List the links of the link groups for the specified network device. 95 | 96 | .TP 97 | .BI ibdev " IBDEV" 98 | List the links of the link groups for the specified RoCE device. 99 | 100 | .SH OUTPUT 101 | 102 | .SS "LG-ID" 103 | ID of the link group. 104 | .SS "LG-Role" 105 | Role of the link group. 106 | .TP 107 | .I 108 | SERV 109 | The link group has a SERVER role. 110 | .TP 111 | .I 112 | CLNT 113 | The link group has a CLIENT role. 114 | .SS "LG-Type" 115 | Type of the link group. 116 | .TP 117 | .I 118 | NONE 119 | The link group has the initial type. 120 | .TP 121 | .I 122 | SINGLE 123 | The link group has only a single link, i.e. 124 | the local and the peer system can offer one device port only for this link group, 125 | which means a link outage on either side cannot be recovered. 126 | .TP 127 | .I 128 | SYM 129 | The link group has two symmetric links, i.e. 130 | the local and the peer system can offer two device ports for this link group, 131 | which means a link outage on either side can be recovered. 132 | .TP 133 | .I 134 | ASYMP 135 | The link group has asymmetric links, i.e. 136 | the peer system can offer one device port only for this link group, 137 | which means a link outage on the peer side cannot be recovered. 138 | .TP 139 | .I 140 | ASYML 141 | The link group has asymmetric links, i.e. 142 | the local system can offer one device port only for this link group, 143 | which means a link outage on the local side cannot be recovered. 144 | .SS "VLAN" 145 | The VLAN to which the link group belongs. 146 | .SS "#Conns" 147 | Number of connections(sockets) running on the link or link group. 148 | .SS "PNET-ID" 149 | PNET ID of the link group. Asterisk "*" means PNET ID is set by the user. 150 | .SS "Net-Dev" 151 | Network device name corresponding to the link. 152 | .SS "Link-State" 153 | The state of the link. 154 | .TP 155 | .I 156 | LINK_UNUSED 157 | The link is not in use and in initial state. 158 | .TP 159 | .I 160 | LINK_INACTIVE 161 | The link is inactive and will go away. 162 | .TP 163 | .I 164 | LINK_ACTIVATING 165 | The link is being activated with the peer. 166 | .TP 167 | .I 168 | LINK_ACTIVE 169 | The link is active and operates on an established link with the peer. 170 | Data is being exchanged via RDMA. 171 | .SS "Link-UID" 172 | Unique identifier of the link. This identifier consists of link group id and 173 | link id. 174 | .SS "Peer-UID" 175 | Unique identifier of the link on the peer side. This identifier consists of 176 | link group id and link id. 177 | .SS "IB-Dev" 178 | Name of the RoCE device used by the link. 179 | .SS "IB-P" 180 | Port of the RoCE device used by the link. 181 | .SS "Local-GID" 182 | GID of the RoCE port used by the link. 183 | .SS "Peer-GID" 184 | GID of the peer RoCE port used by the link. 185 | 186 | .SH "EXAMPLES" 187 | 188 | .HP 2 189 | 1. Show all SMC-D link groups: 190 | .br 191 | \fB# smcd linkgroup show all\fP 192 | .br 193 | .HP 2 194 | 2. Show all SMC-D link groups with link group id 50: 195 | .br 196 | \fB# smcd linkgroup show 50\fP 197 | .br 198 | .HP 2 199 | 3. Show all SMC-R links: 200 | .br 201 | \fB# smcr linkgroup link-show all\fP 202 | .br 203 | .HP 2 204 | 4. Show all SMC-R links with link group ID 40: 205 | .br 206 | \fB# smcr linkgroup link-show 40\fP 207 | .br 208 | .HP 2 209 | 5. Show all SMC-R links on RoCE device "mlx4_0": 210 | .br 211 | \fB# smcr linkgroup link-show ibdev mlx4_0\fP 212 | .br 213 | .HP 2 214 | 6. Show all SMC-R links on network device "eth0": 215 | .br 216 | \fB# smcr linkgroup link-show netdev eth0\fP 217 | .br 218 | 219 | .SH SEE ALSO 220 | .br 221 | .BR smcd (8), 222 | .BR smcr (8) 223 | -------------------------------------------------------------------------------- /smcd-seid.8: -------------------------------------------------------------------------------- 1 | .\" smcd-seid.8 2 | .\" 3 | .\" 4 | .\" Copyright IBM Corp. 2021 5 | .\" ---------------------------------------------------------------------- 6 | .\" 7 | .TH SMCD-SEID 8 "January 2021" "smc-tools" "Linux Programmer's Manual" 8 | 9 | 10 | .SH NAME 11 | smcd-seid \- Control the system EID 12 | 13 | 14 | .SH "SYNOPSIS" 15 | .sp 16 | .ad l 17 | .in +8 18 | .ti -8 19 | .B smcd 20 | .RI "[ " OPTIONS " ]" 21 | .B seid 22 | .RI " { " 23 | .BR show " | " 24 | .BR enable " | " 25 | .BR disable " | " 26 | .BR help " }" 27 | .sp 28 | 29 | 30 | .SH "DESCRIPTION" 31 | Use the 32 | .B smcd seid 33 | command to control the system enterprise ID (EID). 34 | You can disable the system EID while at least one user defined EID entry exists. 35 | A disabled system EID is automatically enabled when the last user defined EID entry is deleted. 36 | 37 | .SS smcd seid show 38 | show the system EID 39 | 40 | .SS smcd seid enable 41 | enable the system EID 42 | 43 | .SS smcd seid disable 44 | disable the system EID 45 | 46 | 47 | .SH "EXAMPLES" 48 | 49 | .HP 2 50 | 1. Disable the system EID 51 | .br 52 | \fB# smcd seid disable\fP 53 | .br 54 | 55 | 56 | .SH SEE ALSO 57 | .br 58 | .BR smcd (8), 59 | .BR smcr (8) 60 | -------------------------------------------------------------------------------- /smcd-stats.8: -------------------------------------------------------------------------------- 1 | .TH SMCD-STATS 8 "June 2021" "smc-tools" "Linux Programmer's Manual" 2 | 3 | .SH NAME 4 | smcd-stats \- Print statistics about SMC-D usage 5 | 6 | smcr-stats \- Print statistics about SMC-R usage 7 | 8 | .SH "SYNOPSIS" 9 | .sp 10 | .ad l 11 | .in +8 12 | .ti -8 13 | .B smcd 14 | .RI "[ " OPTIONS " ]" 15 | .B stats 16 | .RI " { " COMMAND " | " 17 | .BR help " }" 18 | .sp 19 | 20 | .ti -8 21 | .B smcr 22 | .RI "[ " OPTIONS " ]" 23 | .B stats 24 | .RI " { " COMMAND " | " 25 | .BR help " }" 26 | .sp 27 | 28 | .SH DESCRIPTION 29 | The smcd stats and smcr stats commands display statistics about SMC-D and 30 | SMC-R, respectively. Data includes number of connections, number of failed 31 | connections, buffer information, and special calls. 32 | Statistics are collected at various data points in the smc kernel module. 33 | Therefore, minor inconsistencies in the data can occur and are to be 34 | expected. 35 | 36 | .SH COMMANDS 37 | 38 | .TP 39 | .BR "show " (default) 40 | Display statistics on respective SMC mode usage. Values displayed are 41 | cumulative since last reset, unless option 42 | .B -a/--absolute 43 | is specified. 44 | 45 | .TP 46 | .BR reset 47 | Display current statistics and reset all counters to zero. 48 | 49 | .TP 50 | .BR json 51 | Display current statistics in JSON format. 52 | 53 | .SH OPTIONS 54 | 55 | .TP 56 | .B \-d, \-\-details 57 | Display additional fields and information in the output. 58 | 59 | .TP 60 | .B \-a, \-\-absolute 61 | Display values since smc module load, ignoring any resets. 62 | 63 | .SH OUTPUT 64 | The 65 | .B -d/--details 66 | option displays a number of additional fields as indicated in the following. 67 | It also breaks up a number of counters in section 68 | .I Connections Summary 69 | into separate counts for client and server. 70 | 71 | .SS "Total connections handled" 72 | Total number of connections handled by the SMC module. Includes 73 | .I TCP fallback 74 | connections as well as 75 | .IR "handshake errors" . 76 | 77 | .SS "SMC connections" 78 | Number of connections that successfully entered the respective SMC mode. 79 | .B -d/--details 80 | breaks up connections by SMC versions. 81 | 82 | .SS Handshake errors 83 | Number of connections that failed due to errors during the handshaking 84 | phase, e.g. peer no longer responding. 85 | 86 | .SS Avg requests per SMC conn 87 | Average number of requests sent and received (including 88 | .IR "special socket calls" ) 89 | per SMC connection. 90 | 91 | .SS TCP fallback 92 | Number of connections that fell back to TCP/IP. 93 | 94 | .SS Data transmitted 95 | Amount of data sent (TX) or received (RX) in Bytes. 96 | 97 | .SS Total requests 98 | Total number of individual send (TX) or receive (RX) requests handled. 99 | Includes requests that ended with errors or did not transfer any data. 100 | 101 | .SS Buffer full 102 | Number of occurrences where the respective send buffer (TX) could not contain 103 | all data to be sent, or did not contain as much data as requested in a 104 | receive call (RX). 105 | 106 | .SS Buffer full (remote) (\-\-details only) 107 | Number of occurrences where the peer's receive buffer was exceeded by 108 | writing data. That is, requests that fill the buffer up to the last bit are not 109 | included in the count. 110 | 111 | .SS Buffer too small (\-\-details only) 112 | Number of occurrences where a send request was larger than the local send 113 | buffer's total capacity. 114 | 115 | .SS Buffer too small (remote) (\-\-details only) 116 | Number of occurrences where a send request exceeded the total capacity of the 117 | peer's receive buffer. 118 | 119 | .SS Buffer downgrades (\-\-details only) 120 | Number of occurrences where a buffer of the requested size could not be 121 | allocated for a new connection, and a smaller buffer was used. 122 | 123 | .SS Buffer reuses (\-\-details only) 124 | Number of occurrences where a buffer was provided as requested for a new 125 | connection by reusing a buffer from a previous connection. 126 | 127 | .SS Bufs 128 | Histogram of buffer sizes for all connections, including 129 | .I buffer downgrades 130 | and 131 | .IR "buffer reuses" . 132 | The histogram scale presents exact buffer sizes. 133 | 134 | .SS Reqs 135 | Histogram of request sizes. The histogram scale includes upper boundaries of 136 | request sizes. Counts reflect requested send sizes for TX, and actual receive 137 | sizes for RX. Difference to 138 | .I "Total requests" 139 | is due to requests not transferring any data and/or erroneous requests. 140 | 141 | .SS Special socket calls 142 | Summarizes the total number of sockets calls that require special handling 143 | in SMC. 144 | To categorize these calls into individual counters as follows, use the 145 | .I -d/\-\-details 146 | option. 147 | 148 | .TP 149 | .I cork 150 | Counts occurrences of sockopt TCP_CORK enablements. That is, does not reflect 151 | the number of send requests with TCP_CORK enabled. 152 | .TP 153 | .I nodelay 154 | Counts occurrences of sockopt TCP_NODELAY enablements. That is, does not 155 | reflect the number of send requests with TCP_CORK enabled. 156 | .TP 157 | .I sendpage 158 | Counts occurrences of the AF_SMC implementation of the sendpage call. 159 | .TP 160 | .I splice 161 | Counts number of calls of the splice() system call. 162 | .TP 163 | .I urgent data 164 | Counts number of send and receive calls with MSG_OOB set. 165 | 166 | .SH "EXAMPLES" 167 | 168 | .HP 2 169 | 1. Show SMC-D statistics: 170 | .br 171 | \fB# smcd stats\fP 172 | .br 173 | .HP 2 174 | 2. Show detailed SMC-R statistics: 175 | .br 176 | \fB# smcr -d stats show\fP 177 | .br 178 | .HP 2 179 | 3. Show detailed SMC-R statistics and reset SMC-R statistics counters: 180 | .br 181 | \fB# smcr -d stats reset\fP 182 | .br 183 | .HP 2 184 | 4. Show detailed SMC-D statistics since module load in JSON format: 185 | .br 186 | \fB# smcd -da stats json\fP 187 | .br 188 | .HP 2 189 | .HP 2 190 | 5. Show SMC-R statistics since module load: 191 | .br 192 | \fB# smcr -a stats\fP 193 | .br 194 | .HP 2 195 | 196 | 197 | .P 198 | .SH SEE ALSO 199 | .BR smcd (8), 200 | .BR smcr (8) 201 | 202 | -------------------------------------------------------------------------------- /smcd-ueid.8: -------------------------------------------------------------------------------- 1 | .\" smcd-ueid.8 2 | .\" 3 | .\" 4 | .\" Copyright IBM Corp. 2021 5 | .\" ---------------------------------------------------------------------- 6 | .\" 7 | .TH SMCD-UEID 8 "January 2021" "smc-tools" "Linux Programmer's Manual" 8 | 9 | 10 | .SH NAME 11 | smcd-ueid \- Manage user defined EIDs 12 | 13 | smcr-ueid \- Manage user defined EIDs 14 | 15 | 16 | .SH "SYNOPSIS" 17 | .sp 18 | .ad l 19 | .in +8 20 | .ti -8 21 | .B smcd 22 | .RI "[ " OPTIONS " ]" 23 | .B ueid 24 | .RI " { " 25 | .BR show " | " 26 | .BR add " | " 27 | .BR del " | " 28 | .BR flush " | " 29 | .BR help " }" 30 | .sp 31 | 32 | .ti -8 33 | .B smcr 34 | .RI "[ " OPTIONS " ]" 35 | .B ueid 36 | .RI " { " 37 | .BR show " | " 38 | .BR add " | " 39 | .BR del " | " 40 | .BR flush " | " 41 | .BR help " }" 42 | .sp 43 | 44 | 45 | .SH "DESCRIPTION" 46 | Use the 47 | .B smcd ueid 48 | and 49 | .B smcr ueid 50 | commands to manage user defined enterprise IDs (EIDs). 51 | 52 | .SS smcd,smcr ueid show 53 | show all user defined EID entries 54 | 55 | .SS smcd,smcr ueid add [ueid] 56 | add an entry with the specified EID 57 | 58 | For [ueid], specify up to 32 uppercase alphabetic characters (A-Z), numerals (0-9), hyphens (-), and dots (.). 59 | The first character must be alphanumeric, and dots must not be consecutive. An invalid [ueid] is rejected. 60 | 61 | .SS smcd,smcr ueid del [ueid] 62 | delete the specified user defined EID 63 | 64 | .SS smcd,smcr ueid flush 65 | delete all user defined EIDs 66 | 67 | 68 | .SH "EXAMPLES" 69 | 70 | .HP 2 71 | 1. Add a user defined EID 72 | .br 73 | \fB# smcd ueid add LOCATION-WEST\fP 74 | .br 75 | 76 | .HP 2 77 | 2. Show the user defined EIDs 78 | .br 79 | \fB# smcd ueid show\fP 80 | .br 81 | \fBLOCATION-WEST\fP 82 | .br 83 | \fBLOCAL-WEST\fP 84 | .br 85 | 86 | 87 | .SH SEE ALSO 88 | .br 89 | .BR smcd (8), 90 | .BR smcr (8) 91 | -------------------------------------------------------------------------------- /smcd.8: -------------------------------------------------------------------------------- 1 | .\" smcd.8 2 | .\" 3 | .\" 4 | .\" Copyright IBM Corp. 2020 5 | .\" Author(s): Guvenc Gulce 6 | .\" ---------------------------------------------------------------------- 7 | .\" 8 | .TH SMCD 8 "June 2020" "smc-tools" "Linux Programmer's Manual" 9 | 10 | .SH NAME 11 | smcd \- Print information about SMC-D link groups and devices 12 | 13 | .SH SYNOPSIS 14 | .B smcd 15 | .RI "[ " OPTIONS " ] " OBJECT " { " COMMAND " | " 16 | .BR help " }" 17 | .sp 18 | 19 | .IR OBJECT " := { " 20 | .BR info " | " linkgroup " | " device " | " stats " | " ueid " | " seid " }" 21 | .sp 22 | 23 | .IR OPTIONS " := { " 24 | \fB\-v\fR[\fIersion\fR] | 25 | \fB\-d\fR[\fIetails\fR] | 26 | \fB\-a\fR[\fIbsolute\fR]} 27 | 28 | 29 | .SH OPTIONS 30 | 31 | .TP 32 | .BR "\-v" , " -version" 33 | Print the version of the 34 | .B smcd 35 | utility and exit. 36 | 37 | .TP 38 | .BR "\-d", " \-details" 39 | Print detailed information (valid only for stats). 40 | 41 | .TP 42 | .BR "\-a", " \-absolute" 43 | Print absolute statistic value (valid only for stats). 44 | 45 | .SH SMCD - COMMAND SYNTAX 46 | 47 | .SS 48 | .I OBJECT 49 | 50 | .TP 51 | .B device 52 | One or more SMC-D devices. 53 | 54 | .TP 55 | .B info 56 | Generic SMC information. 57 | 58 | .TP 59 | .B linkgroup 60 | One or more SMC-D link groups or links. 61 | 62 | .TP 63 | .B stats 64 | SMC-D statistics. 65 | 66 | .TP 67 | .B ueid 68 | Work with User defined Enterprise IDs (UEID). 69 | 70 | .TP 71 | .B seid 72 | Maintain the System defined Enterprise ID (SEID). 73 | 74 | .PP 75 | The names of all objects can be abbreviated down to 76 | a unique stem. For example 77 | .B device 78 | can be abbreviated to 79 | .B dev 80 | or just 81 | .B d. 82 | For more information about individual objects see the man pages in 83 | .B SEE ALSO 84 | section. 85 | 86 | .SS 87 | .I COMMAND 88 | 89 | Specifies the action to perform on the object. 90 | The set of possible actions depends on the object type. 91 | For most objects you can specify the 92 | .BR " show " or " link-show" 93 | command. Use the 94 | .B help 95 | command for an object to print information about the available actions and the specific syntax for that object. 96 | .sp 97 | If no command is given, a default command 98 | is assumed. 99 | 100 | .SH RETURN CODES 101 | Successful 102 | .IR smcd 103 | commands return 0 and display the 104 | requested link group or device information. 105 | If an error occurs, 106 | .IR smcd 107 | writes a message to stderr and completes with a return code other than 0. 108 | Possible error messages to stderr in case of non-zero return code: 109 | .TP 110 | .BR "SMC module not loaded" 111 | Either kernel is not supporting the 112 | .IR smcd 113 | tool or the smc kernel module is not loaded. 114 | .P 115 | .SH SEE ALSO 116 | .BR af_smc (7), 117 | .BR smcr (8), 118 | .BR smcd-device (8), 119 | .BR smcd-info (8), 120 | .BR smcd-linkgroup (8), 121 | .BR smcd-stats (8), 122 | .BR smcd-ueid (8), 123 | .BR smcd-seid (8) 124 | -------------------------------------------------------------------------------- /smcr.8: -------------------------------------------------------------------------------- 1 | .\" smcr.8 2 | .\" 3 | .\" 4 | .\" Copyright IBM Corp. 2020 5 | .\" Author(s): Guvenc Gulce 6 | .\" ---------------------------------------------------------------------- 7 | .\" 8 | .TH SMCR 8 "June 2020" "smc-tools" "Linux Programmer's Manual" 9 | 10 | .SH NAME 11 | 12 | smcr \- Print information about SMC-R link groups, links and devices 13 | 14 | .SH SYNOPSIS 15 | .B smcr 16 | .RI "[ " OPTIONS " ] " OBJECT " { " COMMAND " | " 17 | .BR help " }" 18 | .sp 19 | 20 | .IR OBJECT " := { " 21 | .BR info " | " linkgroup " | " device " | " stats " | " ueid " }" 22 | .sp 23 | 24 | .IR OPTIONS " := { " 25 | \fB\-v\fR[\fIersion\fR] | 26 | \fB\-a\fR[\fIbsolute\fR] | 27 | \fB\-d\fR[\fIetails\fR] | 28 | \fB\-dd\fR[\fIetails\fR] } 29 | 30 | .SH OPTIONS 31 | 32 | .TP 33 | .BR "\-v" , " -version" 34 | Print the version of the 35 | .B smcr 36 | utility and exit. 37 | 38 | .TP 39 | .BR "\-a", " \-absolute" 40 | Print absolute statistic value (valid only for stats). 41 | 42 | .TP 43 | .BR "\-d", " \-details" 44 | Print detailed information. 45 | 46 | .TP 47 | .BR "\-dd", " \-ddetails" 48 | Print more detailed information. 49 | 50 | .SH SMCR - COMMAND SYNTAX 51 | 52 | .SS 53 | .I OBJECT 54 | 55 | .TP 56 | .B device 57 | One or more SMC-R devices. 58 | 59 | .TP 60 | .B info 61 | Generic SMC information. 62 | 63 | .TP 64 | .B linkgroup 65 | One or more SMC-R link groups or links. 66 | 67 | .TP 68 | .B stats 69 | SMC-R statistics. 70 | 71 | .TP 72 | .B ueid 73 | Work with User defined Enterprise IDs (UEID). 74 | 75 | .PP 76 | The names of all objects can be abbreviated down to 77 | a unique stem. For example 78 | .B device 79 | can be abbreviated to 80 | .B dev 81 | or just 82 | .B d. 83 | For more information about individual objects see the man pages in 84 | .B SEE ALSO 85 | section. 86 | 87 | .SS 88 | .I COMMAND 89 | 90 | Specifies the action to perform on the object. 91 | The set of possible actions depends on the object type. 92 | For most objects you can specify the 93 | .BR " show " or " link-show" 94 | command. Use the 95 | .B help 96 | command for an object to print information about the available actions and the specific syntax for that object. 97 | .sp 98 | If no command is given, a default command 99 | is assumed. 100 | 101 | .SH RETURN CODES 102 | Successful 103 | .IR smcr 104 | commands return 0 and display the 105 | requested link group, link or device information. 106 | If an error occurs, 107 | .IR smcr 108 | writes a message to stderr and completes with a return code other than 0. Possible 109 | error messages to stderr in case of non-zero return code: 110 | .TP 111 | .BR "SMC module not loaded" 112 | Either kernel is not supporting the 113 | .IR smcr 114 | tool or the smc kernel module is not loaded. 115 | .P 116 | .SH SEE ALSO 117 | .BR af_smc (7), 118 | .BR smcd (8), 119 | .BR smcr (8), 120 | .BR smcr-device (8), 121 | .BR smcr-info (8), 122 | .BR smcr-linkgroup (8) 123 | .BR smcr-stats (8), 124 | .BR smcr-ueid (8) 125 | -------------------------------------------------------------------------------- /smcss.8: -------------------------------------------------------------------------------- 1 | .\" smcss.8 2 | .\" 3 | .\" 4 | .\" Copyright IBM Corp. 2017, 2018 5 | .\" Author(s): Ursula Braun 6 | .\" ---------------------------------------------------------------------- 7 | .\" 8 | 9 | .TH SMCSS 8 "June 2018" "smc-tools" "Linux Programmer's Manual" 10 | 11 | .SH NAME 12 | smcss \- print information about the AF_SMC sockets and link groups 13 | 14 | .SH SYNOPSIS 15 | 16 | .B smcss 17 | .RB [ \-\-debug | \-d ] 18 | .RB [ \-\-smcd | \-D ] 19 | .RB [ \-\-wide | \-W ] 20 | .P 21 | .B smcss 22 | .RB { \-\-all | -a } 23 | .RB [ \-\-debug | \-d ] 24 | .RB [ \-\-smcd | \-D ] 25 | .RB [ \-\-wide | \-W ] 26 | .P 27 | .B smcss 28 | .RB [ \-\-debug | \-d ] 29 | .RB [ \-\-smcr | \-R ] 30 | .RB [ \-\-wide | \-W ] 31 | .P 32 | .B smcss 33 | .RB { \-\-all | -a } 34 | .RB [ \-\-debug | \-d ] 35 | .RB [ \-\-smcr | \-R ] 36 | .RB [ \-\-wide | \-W ] 37 | .P 38 | .B smcss 39 | .RB { \-\-listening | \-l } 40 | .RB [ \-\-wide | \-W ] 41 | .P 42 | .B smcss 43 | .RB { \-\-version | \-v } 44 | .P 45 | .B smcss 46 | .RB { \-\-help | \-h } 47 | 48 | .SH DESCRIPTION 49 | 50 | .B smcss 51 | prints information about the Linux AF_SMC sockets and link groups. 52 | The type of information printed is controlled by the first argument, as follows: 53 | .TP 54 | .BR (none) 55 | .br 56 | displays a list of connecting, closing, or connected SMC sockets with basic 57 | information. 58 | 59 | .TP 60 | .BR "\-a, \-\-all" 61 | displays all types of SMC sockets: listening, opening, closing, and connected. 62 | 63 | .TP 64 | .BR "\-l, \-\-listening" 65 | shows listening sockets only. These are omitted by default. 66 | 67 | .SH OPTIONS 68 | 69 | .TP 70 | .BR "\-d, \-\-debug" 71 | displays additional debug information, such as shutdown state. 72 | 73 | .TP 74 | .BR "\-D, \-\-smcd 75 | displays additional SMC-D specific information. Shows SMC-D sockets only. 76 | 77 | .TP 78 | .BR "\-h, \-\-help" 79 | displays usage information. 80 | 81 | .TP 82 | .BR "\-R, \-\-smcr 83 | displays additional SMC-R specific information. Shows SMC-R sockets only. 84 | 85 | .TP 86 | .BR "\-v, \-\-version" 87 | displays program version. 88 | 89 | .TP 90 | .BR "\-W, \-\-wide" 91 | do not truncate IP addresses. 92 | 93 | .SH OUTPUT 94 | 95 | .SS "State" 96 | The state of the socket. 97 | The state can be one of these values: 98 | .TP 99 | .I 100 | INIT 101 | The SMC socket is being initialized. It is not connected nor 102 | listening yet. 103 | .TP 104 | .I 105 | CLOSED 106 | The SMC socket is closed. It is not connected nor listening 107 | anymore. 108 | .TP 109 | .I 110 | LISTEN 111 | The SMC socket is a listening socket, waiting for incoming connection requests. 112 | .TP 113 | .I 114 | ACTIVE 115 | The SMC socket has an established connection. 116 | In this state, the TCP connection is fully established, 117 | rendezvous processing has been completed, and SMC peers 118 | can exchange data via RDMA. 119 | .TP 120 | .I 121 | PEERCLW1 122 | No further data will be sent to the peer. 123 | .TP 124 | .I 125 | PEERCLW2 126 | No further data will be sent to or received from the peer. 127 | .TP 128 | .I 129 | APPLCLW1 130 | No further data will be received from the peer. 131 | .TP 132 | .I 133 | APPLCLW2 134 | No further data will be received from or sent to the peer. 135 | .TP 136 | .I 137 | APPLFINCLW 138 | The peer has closed the socket. 139 | .TP 140 | .I 141 | PEERFINCLW 142 | The socket is closed locally. 143 | .TP 144 | .I 145 | PEERABORTW 146 | The socket was abnormally closed locally. 147 | .TP 148 | .I 149 | PROCESSABORT 150 | The peer has closed the socket abnormally. 151 | .SS "UID" 152 | User ID of the SMC socket. 153 | .SS "Inode" 154 | Inode attribute of the socket. 155 | .SS "Local Address" 156 | Address and port number of the local end of the SMC socket. 157 | The displayed address is truncated when it ends with '..'. 158 | .BR "-W, --wide" 159 | can be used to display addresses untruncated. 160 | .SS "Peer Address" 161 | Address and port number of the remote end of the socket. 162 | Analogous to "Local Address". 163 | .SS "Intf" 164 | When the socket is explicitly bound with setsockopt option SO_BINDTODEVICE then 165 | Intf shows the interface number of the Ethernet device to which the socket is bound. 166 | .SS "Mode" 167 | .TP 168 | .I 169 | SMCD 170 | The SMC socket uses SMC-D for data exchange. 171 | .TP 172 | .I 173 | SMCR 174 | The SMC socket uses SMC-R for data exchange. 175 | .TP 176 | .I 177 | TCP 178 | The SMC socket uses the TCP protocol for data exchange, because an SMC connection 179 | could not be established. 180 | .TP 181 | In case of an error during connection setup, respective error codes will be displayed in the format /. 182 | .HP 183 | Linux error codes: 184 | .TP 13 185 | .I 186 | 0x01010000 187 | Out of memory 188 | .TP 13 189 | .I 190 | 0x02010000 191 | Timeout while waiting for confirm link message over RDMA device 192 | .TP 193 | .I 194 | 0x02020000 195 | Timeout while waiting for RDMA device to be added 196 | .TP 197 | .I 198 | 0x03000000 199 | Configuration error 200 | .TP 201 | .I 202 | 0x03010000 203 | Peer does not support SMC 204 | .TP 205 | .I 206 | 0x03020000 207 | Connection uses IPsec 208 | .TP 209 | .I 210 | 0x03030000 211 | No SMC devices found (R and D) 212 | .TP 213 | .I 214 | 0x03030001 215 | No ISM device for SMC-D found 216 | .TP 217 | .I 218 | 0x03030002 219 | No RDMA device for SMC-R found 220 | .TP 221 | .I 222 | 0x03030003 223 | Hardware has no ISMv2 support 224 | .TP 225 | .I 226 | 0x03030004 227 | Peer sent no SMCv2 extension 228 | .TP 229 | .I 230 | 0x03030005 231 | Peer sent no SMC-Dv2 extension 232 | .TP 233 | .I 234 | 0x03030006 235 | Peer sent no ISMv2 SEID 236 | .TP 237 | .I 238 | 0x03030007 239 | No SMC-Dv2 device found, but required 240 | .TP 241 | .I 242 | 0x03030008 243 | Peer sent no UEID 244 | .TP 245 | .I 246 | 0x03040000 247 | SMC modes mismatch (R or D) 248 | .TP 249 | .I 250 | 0x03050000 251 | Peer has eyecatcher in RMBE 252 | .TP 253 | .I 254 | 0x03060000 255 | Fastopen sockopt not supported 256 | .TP 257 | .I 258 | 0x03070000 259 | IP prefix / subnet mismatch 260 | .TP 261 | .I 262 | 0x03080000 263 | Error retrieving VLAN ID of IP device 264 | .TP 265 | .I 266 | 0x03090000 267 | Error while registering VLAN ID on ISM device 268 | .TP 269 | .I 270 | 0x030a0000 271 | No active SMC-R link in link group 272 | .TP 273 | .I 274 | 0x030b0000 275 | SMC-R link from server not found 276 | .TP 277 | .I 278 | 0x030c0000 279 | SMC version mismatch 280 | .TP 281 | .I 282 | .I 283 | 0x030d0000 284 | SMC-D connection limit reached 285 | .TP 286 | .I 287 | .TP 288 | .I 289 | 0x030e0000 290 | SMC-Rv2 connection found no route to peer 291 | .TP 292 | .I 293 | 0x030f0000 294 | SMC-Rv2 connection mismatch direct/indirect with peer 295 | 0x04000000 296 | Synchronization error 297 | .TP 298 | .I 299 | 0x05000000 300 | Peer declined during handshake 301 | .TP 302 | .I 303 | 0x09990000 304 | Internal error 305 | .TP 306 | .I 307 | 0x09990001 308 | rtoken handling failed 309 | .TP 310 | .I 311 | 0x09990002 312 | RDMA link failed 313 | .TP 314 | .I 315 | 0x09990003 316 | RMB registration failed 317 | 318 | .SS "ShutD" 319 | .TP 320 | .I 321 | <-> 322 | The SMC socket has not been shut down. 323 | .TP 324 | .I 325 | R-> 326 | The SMC socket is shut down one-way and cannot receive data. 327 | .TP 328 | .I 329 | <-W 330 | The SMC socket is shut down one-way and cannot send data. 331 | .TP 332 | .I 333 | R-W 334 | The SMC socket is shut down in both ways and cannot receive or send data. 335 | .SS "Token" 336 | Unique ID of the SMC socket connection. 337 | .SS "Sndbuf" 338 | Size of the to-be-sent window of the SMC socket connection. 339 | .SS "Rcvbuf" 340 | Size of the receiving window of the SMC socket connection (filled by peer). 341 | .SS "Peerbuf" 342 | Size of the peer receiving window of the SMC socket connection (to fill 343 | during RDMA-transfer). 344 | .SS "rxprod-Cursor" 345 | Describes the current cursor location of the "Rcvbuf" for data to be received 346 | from the peer. 347 | .SS "rxcons-Cursor" 348 | Describes the current cursor location of the "Peerbuf" for data sent to peer 349 | and confirmed by the peer. 350 | .SS "rxFlags" 351 | SMC socket connection flags set by and received from the peer. 352 | .SS "txprod-Cursor" 353 | Describes the current cursor location of the "Peerbuf" for data sent to peer. 354 | .SS "txcons-Cursor" 355 | Describes the current cursor location of the "Rcvbuf" for data received from 356 | the peer and confirmed to the peer. 357 | .SS "txFlags" 358 | SMC socket connection flags set locally and sent to the peer. 359 | .SS "txprep-Cursor" 360 | Describes the current cursor location of the "Sndbuf" for data to be sent. 361 | The data is to be moved to the "Peerbuf". 362 | .SS "txsent-Cursor" 363 | Describes the current cursor location of the "Sndbuf" for data sent. 364 | The data was moved to the "Peerbuf". 365 | .SS "txfin-Cursor" 366 | Describes the current cursor location of the "Sndbuf" for data sent and 367 | send completion confirmed. 368 | The data was moved to the "Peerbuf" and completion was confirmed. 369 | .SS "Role" 370 | .TP 371 | .I 372 | CLNT 373 | The link group of the SMC socket is used for client connections. 374 | .TP 375 | .I 376 | SERV 377 | The link group of the SMC socket is used for server connections. 378 | .SS "IB-Device" 379 | Name of the RoCE device used by the link group to which the SMC socket belongs. 380 | .SS "Port" 381 | Port of the RoCE device used by the link group to which the SMC socket belongs. 382 | .SS "Linkid" 383 | Unique link ID of the link within the link group to which the SMC socket belongs. 384 | .SS "GID" 385 | Gid of the RoCE port used by the link group to which the SMC socket belongs. 386 | .SS "Peer-GID" 387 | Gid of the Foreign RoCE port used by the link group to which the SMC socket belongs. 388 | .SS "VLAN" 389 | tbd. 390 | .SH RETURN CODES 391 | Successful 392 | .IR smcss 393 | commands return 0 and display the 394 | requested socket state table or link group information. 395 | If an error occurs, 396 | .IR smcss 397 | writes a message to stderr and completes with a return code other than 0. 398 | .P 399 | .SH SEE ALSO 400 | .BR af_smc (7), 401 | .BR smc_chk (8) 402 | .BR smc_rnics (8), 403 | .BR smc_run (8), 404 | .BR smc_pnet (8), 405 | .BR smcd (8), 406 | .BR smcr (8) 407 | 408 | -------------------------------------------------------------------------------- /smcss.c: -------------------------------------------------------------------------------- 1 | /* 2 | * SMC Tools - Shared Memory Communication Tools 3 | * 4 | * Copyright IBM Corp. 2017, 2018 5 | * 6 | * Author(s): Ursula Braun 7 | * 8 | * User space program for SMC Socket display 9 | * 10 | * All rights reserved. This program and the accompanying materials 11 | * are made available under the terms of the Eclipse Public License v1.0 12 | * which accompanies this distribution, and is available at 13 | * http://www.eclipse.org/legal/epl-v10.html 14 | */ 15 | #include 16 | #include 17 | #include 18 | #include 19 | #include 20 | #include 21 | #include 22 | #include 23 | #include 24 | #include 25 | 26 | #include "smctools_common.h" 27 | #include "libnetlink.h" 28 | 29 | #define ADDR_LEN_SHORT 23 30 | 31 | static char *progname; 32 | int show_debug; 33 | int show_smcr; 34 | int show_smcd; 35 | int show_wide; 36 | int listening = 0; 37 | int all = 0; 38 | 39 | static void print_header(void) 40 | { 41 | printf("State "); 42 | printf("UID "); 43 | printf("Inode "); 44 | printf("Local Address "); 45 | printf("Peer Address "); 46 | printf("Intf "); 47 | printf("Mode "); 48 | 49 | if (show_debug) { 50 | printf("Shutd "); 51 | printf("Token "); 52 | printf("Sndbuf "); 53 | printf("Rcvbuf "); 54 | printf("Peerbuf "); 55 | printf("rxprod-Cursor "); 56 | printf("rxcons-Cursor "); 57 | printf("rxFlags "); 58 | printf("txprod-Cursor "); 59 | printf("txcons-Cursor "); 60 | printf("txFlags "); 61 | printf("txprep-Cursor "); 62 | printf("txsent-Cursor "); 63 | printf("txfin-Cursor "); 64 | } 65 | 66 | if (show_smcr) { 67 | printf("Role "); 68 | printf("IB-device "); 69 | printf("Port "); 70 | printf("Linkid "); 71 | printf("GID "); 72 | printf("Peer-GID"); 73 | } 74 | 75 | if (show_smcd) { 76 | printf("GID "); 77 | printf("Token "); 78 | printf("Peer-GID "); 79 | printf("Peer-Token "); 80 | printf("Linkid"); 81 | } 82 | 83 | printf("\n"); 84 | } 85 | 86 | static const char *smc_state(unsigned char x) 87 | { 88 | static char buf[16]; 89 | 90 | switch (x) { 91 | case 1: return "ACTIVE"; 92 | case 2: return "INIT"; 93 | case 7: return "CLOSED"; 94 | case 10: return "LISTEN"; 95 | case 20: return "PEERCLOSEWAIT1"; 96 | case 21: return "PEERCLOSEWAIT2"; 97 | case 22: return "APPCLOSEWAIT1"; 98 | case 23: return "APPCLOSEWAIT2"; 99 | case 24: return "APPFINCLOSEWAIT1"; 100 | case 25: return "PEERFINCLOSEWAIT"; 101 | case 26: return "PEERABORTWAIT"; 102 | case 27: return "PROCESSABORT"; 103 | default: sprintf(buf, "%#x?", x); return buf; 104 | } 105 | } 106 | 107 | /* format one sockaddr / port */ 108 | static void addr_format(char *buf, size_t buf_len, size_t short_len, 109 | __be32 addr[4], int port) 110 | { 111 | char addr_buf[INET6_ADDRSTRLEN + 1], port_buf[16]; 112 | int addr_len, port_len; 113 | int af; 114 | 115 | /* There was an upstream discussion about the content of the 116 | * diag_family field. Originally it was AF_SMC, but was changed with 117 | * IPv6 support to indicate AF_INET or AF_INET6. Upstream complained 118 | * later that there is no way to separate AF_INET from AF_SMC diag msgs. 119 | * We now change back the value of the diag_family field to be always 120 | * AF_SMC. We now 'parse' the IP address type. 121 | * Note that smc_diag.c in kernel always clears the whole addr field 122 | * before the ip address is copied into and we can rely on that here. 123 | */ 124 | if (addr[1] == 0 && addr[2] == 0 && addr[3] == 0) 125 | af = AF_INET; 126 | else 127 | af = AF_INET6; 128 | 129 | if (buf_len < 20) 130 | return; /* no space for errmsg */ 131 | 132 | if (!inet_ntop(af, addr, addr_buf, sizeof(addr_buf))) { 133 | strcpy(buf, "(inet_ntop error)"); 134 | return; 135 | } 136 | sprintf(port_buf, "%d", port); 137 | addr_len = strlen(addr_buf); 138 | port_len = strlen(port_buf); 139 | if (!show_wide && (addr_len + 1 + port_len > short_len)) { 140 | if (buf_len < short_len + 1) { 141 | strcpy(buf, "(buf to small)"); 142 | return; 143 | } 144 | /* truncate addr string */ 145 | addr_len = short_len - 1 - port_len - 2; 146 | strncpy(buf, addr_buf, addr_len); 147 | buf[addr_len] = '\0'; 148 | strcat(buf, ".."); /* indicate truncation */ 149 | strcat(buf, ":"); 150 | strcat(buf, port_buf); 151 | } else { 152 | if (buf_len < addr_len + 1 + port_len + 1) { 153 | strcpy(buf, "(buf to small)"); 154 | return; 155 | } 156 | snprintf(buf, buf_len, "%s:%s", addr_buf, port_buf); 157 | } 158 | } 159 | 160 | static void show_one_smc_sock(struct nlmsghdr *nlh) 161 | { 162 | struct smc_diag_msg *r = NLMSG_DATA(nlh); 163 | struct rtattr *tb[SMC_DIAG_MAX + 1]; 164 | unsigned long long inode; 165 | char txtbuf[128]; 166 | 167 | parse_rtattr(tb, SMC_DIAG_MAX, (struct rtattr *)(r+1), 168 | nlh->nlmsg_len - NLMSG_LENGTH(sizeof(*r))); 169 | 170 | if (listening) { 171 | if ( r->diag_state != 10) 172 | return; 173 | } else { 174 | if (!all && (r->diag_state == 10 || r->diag_state == 2)) 175 | return; 176 | } 177 | if (show_smcr && r->diag_mode != SMC_DIAG_MODE_SMCR) 178 | return; /* show only SMC-R sockets */ 179 | if (show_smcd && r->diag_mode != SMC_DIAG_MODE_SMCD) 180 | return; /* show only SMC-D sockets */ 181 | 182 | printf("%-14s ", smc_state(r->diag_state)); 183 | printf("%05d ", r->diag_uid); 184 | inode = r->diag_inode; 185 | printf("%07llu ", inode); 186 | if (r->diag_state == 2) /* INIT state */ 187 | goto newline; 188 | 189 | addr_format(txtbuf, sizeof(txtbuf), ADDR_LEN_SHORT, 190 | r->id.idiag_src, ntohs(r->id.idiag_sport)); 191 | printf("%-*s ", (int)MAX(ADDR_LEN_SHORT, strlen(txtbuf)), txtbuf); 192 | if (r->diag_state == 10) /* LISTEN state */ 193 | goto newline; 194 | 195 | addr_format(txtbuf, sizeof(txtbuf), ADDR_LEN_SHORT, 196 | r->id.idiag_dst, ntohs(r->id.idiag_dport)); 197 | printf("%-*s ", (int)MAX(ADDR_LEN_SHORT, strlen(txtbuf)), txtbuf); 198 | printf("%04x ", r->id.idiag_if); 199 | if (r->diag_state == 7) /* CLOSED state */ 200 | goto newline; 201 | 202 | if (r->diag_mode == SMC_DIAG_MODE_FALLBACK_TCP) { 203 | printf("TCP "); 204 | /* when available print local and peer fallback reason code */ 205 | if (tb[SMC_DIAG_FALLBACK] && 206 | tb[SMC_DIAG_FALLBACK]->rta_len >= sizeof(struct smc_diag_fallback)) 207 | { 208 | struct smc_diag_fallback fallback; 209 | 210 | fallback = *(struct smc_diag_fallback *)RTA_DATA(tb[SMC_DIAG_FALLBACK]); 211 | printf("0x%08x", fallback.reason); 212 | if (fallback.peer_diagnosis) 213 | printf("/0x%08x", fallback.peer_diagnosis); 214 | } 215 | goto newline; 216 | 217 | } else if (r->diag_mode == SMC_DIAG_MODE_SMCD) 218 | printf("%4s ", "SMCD"); 219 | else 220 | printf("%4s ", "SMCR"); 221 | 222 | if (show_debug) { 223 | if (tb[SMC_DIAG_SHUTDOWN] && 224 | tb[SMC_DIAG_SHUTDOWN]->rta_len >= sizeof(__u8)) 225 | { 226 | unsigned char mask; 227 | 228 | mask = *(__u8 *)RTA_DATA(tb[SMC_DIAG_SHUTDOWN]); 229 | printf(" %c-%c ", mask & 1 ? 'R' : '<', mask & 2 ? 'W' : '>'); 230 | } 231 | 232 | if (tb[SMC_DIAG_CONNINFO] && 233 | tb[SMC_DIAG_CONNINFO]->rta_len >= sizeof(struct smc_diag_conninfo)) 234 | { 235 | struct smc_diag_conninfo cinfo; 236 | 237 | cinfo = *(struct smc_diag_conninfo *)RTA_DATA(tb[SMC_DIAG_CONNINFO]); 238 | printf("%08x ", cinfo.token); 239 | printf("%08x ", cinfo.sndbuf_size); 240 | printf("%08x ", cinfo.rmbe_size); 241 | printf("%08x ", cinfo.peer_rmbe_size); 242 | 243 | printf("%04x:%08x ", cinfo.rx_prod.wrap, cinfo.rx_prod.count); 244 | printf("%04x:%08x ", cinfo.rx_cons.wrap, cinfo.rx_cons.count); 245 | printf("%02x:%02x ", cinfo.rx_prod_flags, cinfo.rx_conn_state_flags); 246 | printf("%04x:%08x ", cinfo.tx_prod.wrap, cinfo.tx_prod.count); 247 | printf("%04x:%08x ", cinfo.tx_cons.wrap, cinfo.tx_cons.count); 248 | printf("%02x:%02x ", cinfo.tx_prod_flags, cinfo.tx_conn_state_flags); 249 | printf("%04x:%08x ", cinfo.tx_prep.wrap, cinfo.tx_prep.count); 250 | printf("%04x:%08x ", cinfo.tx_sent.wrap, cinfo.tx_sent.count); 251 | printf("%04x:%08x ", cinfo.tx_fin.wrap, cinfo.tx_fin.count); 252 | } 253 | } 254 | 255 | if (show_smcr) { 256 | if (tb[SMC_DIAG_LGRINFO] && 257 | tb[SMC_DIAG_LGRINFO]->rta_len >= sizeof(struct smc_diag_lgrinfo)) 258 | { 259 | struct smc_diag_lgrinfo linfo; 260 | 261 | linfo = *(struct smc_diag_lgrinfo *)RTA_DATA(tb[SMC_DIAG_LGRINFO]); 262 | printf("%4s ", linfo.role ? "SERV" : "CLNT"); 263 | printf("%-15s ", linfo.lnk[0].ibname); 264 | printf("%02x ", linfo.lnk[0].ibport); 265 | printf("%02x ", linfo.lnk[0].link_id); 266 | printf("%-40s ", linfo.lnk[0].gid); 267 | printf("%s", linfo.lnk[0].peer_gid); 268 | } 269 | } 270 | 271 | if (show_smcd) { 272 | if (tb[SMC_DIAG_DMBINFO] && 273 | tb[SMC_DIAG_DMBINFO]->rta_len >= sizeof(struct smcd_diag_dmbinfo)) 274 | { 275 | struct smcd_diag_dmbinfo dinfo; 276 | 277 | dinfo = *(struct smcd_diag_dmbinfo *)RTA_DATA(tb[SMC_DIAG_DMBINFO]); 278 | printf("%016llx ", dinfo.my_gid); 279 | printf("%016llx ", dinfo.token); 280 | printf("%016llx ", dinfo.peer_gid); 281 | printf("%016llx ", dinfo.peer_token); 282 | printf("%08x ", dinfo.linkid); 283 | } 284 | } 285 | 286 | newline: 287 | printf("\n"); 288 | } 289 | 290 | static int smc_show_netlink() 291 | { 292 | struct rtnl_handle rth; 293 | unsigned char cmd = 0; 294 | int rc = 0; 295 | 296 | if ((rc = rtnl_open(&rth))) 297 | return EXIT_FAILURE; 298 | 299 | rth.dump = MAGIC_SEQ; 300 | 301 | if (show_debug) 302 | cmd |= (1<<(SMC_DIAG_CONNINFO-1)); 303 | 304 | if (show_smcr) 305 | cmd |= (1<<(SMC_DIAG_LGRINFO-1)); 306 | 307 | if (show_smcd) 308 | cmd |= (1<<(SMC_DIAG_DMBINFO-1)); 309 | 310 | if ((rc = sockdiag_send(rth.fd, cmd))) 311 | goto exit; 312 | 313 | print_header(); 314 | 315 | rc = rtnl_dump(&rth, show_one_smc_sock); 316 | 317 | exit: 318 | rtnl_close(&rth); 319 | return rc; 320 | } 321 | 322 | static const struct option long_opts[] = { 323 | { "all", 0, 0, 'a' }, 324 | { "debug", 0, 0, 'd' }, 325 | { "listening", 0, 0, 'l' }, 326 | { "smcd", 0, 0, 'D' }, 327 | { "smcr", 0, 0, 'R' }, 328 | { "version", 0, 0, 'v' }, 329 | { "wide", 0, 0, 'W' }, 330 | { "help", 0, 0, 'h' }, 331 | { NULL, 0, NULL, 0} 332 | }; 333 | 334 | static void _usage(FILE *dest) 335 | { 336 | fprintf(dest, 337 | "Usage: %s [ OPTIONS ]\n" 338 | "\t-h, --help this message\n" 339 | "\t-v, --version show version information\n" 340 | "\t-a, --all show all sockets\n" 341 | "\t-l, --listening show listening sockets\n" 342 | "\t-d, --debug show debug socket information\n" 343 | "\t-W, --wide do not truncate IP addresses\n" 344 | "\t-D, --smcd show detailed SMC-D information (shows only SMC-D sockets)\n" 345 | "\t-R, --smcr show detailed SMC-R information (shows only SMC-R sockets)\n" 346 | "\tno OPTIONS show all connected sockets\n", 347 | progname); 348 | } 349 | 350 | static void help(void) __attribute__((noreturn)); 351 | static void help(void) 352 | { 353 | _usage(stdout); 354 | exit(EXIT_SUCCESS); 355 | } 356 | 357 | static void usage(void) __attribute__((noreturn)); 358 | static void usage(void) 359 | { 360 | _usage(stderr); 361 | exit(EXIT_FAILURE); 362 | } 363 | 364 | int main(int argc, char *argv[]) 365 | { 366 | char *slash; 367 | int ch; 368 | 369 | progname = (slash = strrchr(argv[0], '/')) ? slash + 1 : argv[0]; 370 | 371 | while ((ch = getopt_long(argc, argv, "aldDRhvW", long_opts, NULL)) != EOF) { 372 | switch (ch) { 373 | case 'a': 374 | all++; 375 | break; 376 | case 'l': 377 | listening++; 378 | break; 379 | case 'd': 380 | show_debug++; 381 | break; 382 | case 'D': 383 | show_smcd++; 384 | break; 385 | case 'R': 386 | show_smcr++; 387 | break; 388 | case 'v': 389 | printf("smcss utility, smc-tools-%s\n", RELEASE_STRING); 390 | exit(0); 391 | case 'W': 392 | show_wide++; 393 | break; 394 | case 'h': 395 | help(); 396 | case '?': 397 | default: 398 | usage(); 399 | } 400 | } 401 | 402 | if (show_smcr && show_smcd) { 403 | fprintf(stderr, "--smcd together with --smcr is not supported\n"); 404 | usage(); 405 | } 406 | if (listening && show_debug) { 407 | fprintf(stderr, "--listening together with --debug is not supported\n"); 408 | usage(); 409 | } 410 | if (listening && all) { 411 | fprintf(stderr, "--listening together with --all is not supported\n"); 412 | usage(); 413 | } 414 | if (listening && show_smcr) { 415 | fprintf(stderr, "--listening together with --smcr is not supported\n"); 416 | usage(); 417 | } 418 | if (listening && show_smcd) { 419 | fprintf(stderr, "--listening together with --smcd is not supported\n"); 420 | usage(); 421 | } 422 | return smc_show_netlink(); 423 | } 424 | -------------------------------------------------------------------------------- /smctools_common.h: -------------------------------------------------------------------------------- 1 | /* 2 | * smc-tools/smctools_common.h 3 | * 4 | * Copyright IBM Corp. 2017 5 | * 6 | * Author(s): Ursula Braun (ubraun@linux.ibm.com) 7 | * 8 | * Copyright IBM Corp. 2017 9 | * 10 | * All rights reserved. This program and the accompanying materials 11 | * are made available under the terms of the Eclipse Public License v1.0 12 | * which accompanies this distribution, and is available at 13 | * http://www.eclipse.org/legal/epl-v10.html 14 | */ 15 | 16 | #ifndef SMCTOOLS_COMMON_H 17 | #define SMCTOOLS_COMMON_H 18 | 19 | #define STRINGIFY_1(x) #x 20 | #define STRINGIFY(x) STRINGIFY_1(x) 21 | 22 | #define RELEASE_STRING "1.8.4" 23 | 24 | #define PF_SMC 43 25 | 26 | #include 27 | 28 | /*********************************************************** 29 | * Mimic definitions in kernel/include/uapi/linux/smc.h 30 | ***********************************************************/ 31 | 32 | #define SMC_MAX_PNETID_LEN 16 /* Max. length of PNET id */ 33 | #define SMC_LGR_ID_SIZE 4 34 | #define SMC_MAX_IBNAME 8 35 | #define SMC_MAX_HOSTNAME_LEN 32 /* Max length of hostname */ 36 | #define SMC_MAX_EID_LEN 32 /* Max length of eid */ 37 | #define SMC_MAX_UEID 8 /* Max number of eids */ 38 | #define SMC_MAX_PORTS 2 /* Max # of ports per ib device */ 39 | #define SMC_PCI_ID_STR_LEN 16 /* Max length of pci id string */ 40 | 41 | /* Netlink SMC_PNETID attributes */ 42 | enum { 43 | SMC_PNETID_UNSPEC, 44 | SMC_PNETID_NAME, 45 | SMC_PNETID_ETHNAME, 46 | SMC_PNETID_IBNAME, 47 | SMC_PNETID_IBPORT, 48 | __SMC_PNETID_MAX, 49 | SMC_PNETID_MAX = __SMC_PNETID_MAX - 1 50 | }; 51 | 52 | enum { /* SMC PNET Table commands */ 53 | SMC_PNETID_GET = 1, 54 | SMC_PNETID_ADD, 55 | SMC_PNETID_DEL, 56 | SMC_PNETID_FLUSH 57 | }; 58 | 59 | #define SMCR_GENL_FAMILY_NAME "SMC_PNETID" 60 | #define SMCR_GENL_FAMILY_VERSION 1 61 | /* Use for accessing non-socket information like */ 62 | /* SMC links, linkgroups and devices */ 63 | #define SMC_GENL_FAMILY_NAME "SMC_GEN_NETLINK" 64 | #define SMC_GENL_FAMILY_VERSION 1 65 | 66 | /* SMC_GENL_FAMILY commands */ 67 | enum { 68 | SMC_NETLINK_GET_SYS_INFO = 1, 69 | SMC_NETLINK_GET_LGR_SMCR, 70 | SMC_NETLINK_GET_LINK_SMCR, 71 | SMC_NETLINK_GET_LGR_SMCD, 72 | SMC_NETLINK_GET_DEV_SMCD, 73 | SMC_NETLINK_GET_DEV_SMCR, 74 | SMC_NETLINK_GET_STATS, 75 | SMC_NETLINK_GET_FBACK_STATS, 76 | SMC_NETLINK_DUMP_UEID, 77 | SMC_NETLINK_ADD_UEID, 78 | SMC_NETLINK_REMOVE_UEID, 79 | SMC_NETLINK_FLUSH_UEID, 80 | SMC_NETLINK_DUMP_SEID, 81 | SMC_NETLINK_ENABLE_SEID, 82 | SMC_NETLINK_DISABLE_SEID, 83 | }; 84 | 85 | /* SMC_GENL_FAMILY top level attributes */ 86 | enum { 87 | SMC_GEN_UNSPEC, 88 | SMC_GEN_SYS_INFO, /* nest */ 89 | SMC_GEN_LGR_SMCR, /* nest */ 90 | SMC_GEN_LINK_SMCR, /* nest */ 91 | SMC_GEN_LGR_SMCD, /* nest */ 92 | SMC_GEN_DEV_SMCD, /* nest */ 93 | SMC_GEN_DEV_SMCR, /* nest */ 94 | SMC_GEN_STATS, /* nest */ 95 | SMC_GEN_FBACK_STATS, /* nest */ 96 | __SMC_GEN_MAX, 97 | SMC_GEN_MAX = __SMC_GEN_MAX - 1 98 | }; 99 | 100 | /* SMC_GEN_SYS_INFO attributes */ 101 | enum { 102 | SMC_NLA_SYS_UNSPEC, 103 | SMC_NLA_SYS_VER, /* u8 */ 104 | SMC_NLA_SYS_REL, /* u8 */ 105 | SMC_NLA_SYS_IS_ISM_V2, /* u8 */ 106 | SMC_NLA_SYS_LOCAL_HOST, /* string */ 107 | SMC_NLA_SYS_SEID, /* string */ 108 | SMC_NLA_SYS_IS_SMCR_V2, /* u8 */ 109 | __SMC_NLA_SYS_MAX, 110 | SMC_NLA_SYS_MAX = __SMC_NLA_SYS_MAX - 1 111 | }; 112 | 113 | /* SMC_NLA_LGR_D_V2_COMMON and SMC_NLA_LGR_R_V2_COMMON nested attributes */ 114 | enum { 115 | SMC_NLA_LGR_V2_VER, /* u8 */ 116 | SMC_NLA_LGR_V2_REL, /* u8 */ 117 | SMC_NLA_LGR_V2_OS, /* u8 */ 118 | SMC_NLA_LGR_V2_NEG_EID, /* string */ 119 | SMC_NLA_LGR_V2_PEER_HOST, /* string */ 120 | __SMC_NLA_LGR_V2_MAX, 121 | SMC_NLA_LGR_V2_MAX = __SMC_NLA_LGR_V2_MAX - 1 122 | }; 123 | 124 | /* SMC_NLA_LGR_R_V2 nested attributes */ 125 | enum { 126 | SMC_NLA_LGR_R_V2_UNSPEC, 127 | SMC_NLA_LGR_R_V2_DIRECT, /* u8 */ 128 | __SMC_NLA_LGR_R_V2_MAX, 129 | SMC_NLA_LGR_R_V2_MAX = __SMC_NLA_LGR_R_V2_MAX - 1 130 | }; 131 | 132 | /* SMC_GEN_LGR_SMCR attributes */ 133 | enum { 134 | SMC_NLA_LGR_R_UNSPEC, 135 | SMC_NLA_LGR_R_ID, /* u32 */ 136 | SMC_NLA_LGR_R_ROLE, /* u8 */ 137 | SMC_NLA_LGR_R_TYPE, /* u8 */ 138 | SMC_NLA_LGR_R_PNETID, /* string */ 139 | SMC_NLA_LGR_R_VLAN_ID, /* u8 */ 140 | SMC_NLA_LGR_R_CONNS_NUM, /* u32 */ 141 | SMC_NLA_LGR_R_V2_COMMON, /* nest */ 142 | SMC_NLA_LGR_R_V2, /* nest */ 143 | SMC_NLA_LGR_R_NET_COOKIE, /* u64 */ 144 | SMC_NLA_LGR_R_PAD, /* flag */ 145 | SMC_NLA_LGR_R_BUF_TYPE, /* u8 */ 146 | SMC_NLA_LGR_R_SNDBUF_ALLOC, /* uint */ 147 | SMC_NLA_LGR_R_RMB_ALLOC, /* uint */ 148 | __SMC_NLA_LGR_R_MAX, 149 | SMC_NLA_LGR_R_MAX = __SMC_NLA_LGR_R_MAX - 1 150 | }; 151 | 152 | /* SMC_GEN_LINK_SMCR attributes */ 153 | enum { 154 | SMC_NLA_LINK_UNSPEC, 155 | SMC_NLA_LINK_ID, /* u8 */ 156 | SMC_NLA_LINK_IB_DEV, /* string */ 157 | SMC_NLA_LINK_IB_PORT, /* u8 */ 158 | SMC_NLA_LINK_GID, /* string */ 159 | SMC_NLA_LINK_PEER_GID, /* string */ 160 | SMC_NLA_LINK_CONN_CNT, /* u32 */ 161 | SMC_NLA_LINK_NET_DEV, /* string */ 162 | SMC_NLA_LINK_UID, /* u32 */ 163 | SMC_NLA_LINK_PEER_UID, /* u32 */ 164 | SMC_NLA_LINK_STATE, /* u32 */ 165 | __SMC_NLA_LINK_MAX, 166 | SMC_NLA_LINK_MAX = __SMC_NLA_LINK_MAX - 1 167 | }; 168 | 169 | /* SMC_GEN_LGR_SMCD attributes */ 170 | enum { 171 | SMC_NLA_LGR_D_UNSPEC, 172 | SMC_NLA_LGR_D_ID, /* u32 */ 173 | SMC_NLA_LGR_D_GID, /* u64 */ 174 | SMC_NLA_LGR_D_PEER_GID, /* u64 */ 175 | SMC_NLA_LGR_D_VLAN_ID, /* u8 */ 176 | SMC_NLA_LGR_D_CONNS_NUM, /* u32 */ 177 | SMC_NLA_LGR_D_PNETID, /* string */ 178 | SMC_NLA_LGR_D_CHID, /* u16 */ 179 | SMC_NLA_LGR_D_PAD, /* flag */ 180 | SMC_NLA_LGR_D_V2_COMMON, /* nest */ 181 | SMC_NLA_LGR_D_EXT_GID, /* u64 */ 182 | SMC_NLA_LGR_D_PEER_EXT_GID, /* u64 */ 183 | SMC_NLA_LGR_D_SNDBUF_ALLOC, /* uint */ 184 | SMC_NLA_LGR_D_DMB_ALLOC, /* uint */ 185 | __SMC_NLA_LGR_D_MAX, 186 | SMC_NLA_LGR_D_MAX = __SMC_NLA_LGR_D_MAX - 1 187 | }; 188 | 189 | /* SMC_NLA_DEV_PORT nested attributes */ 190 | enum { 191 | SMC_NLA_DEV_PORT_UNSPEC, 192 | SMC_NLA_DEV_PORT_PNET_USR, /* u8 */ 193 | SMC_NLA_DEV_PORT_PNETID, /* string */ 194 | SMC_NLA_DEV_PORT_NETDEV, /* string */ 195 | SMC_NLA_DEV_PORT_STATE, /* u8 */ 196 | SMC_NLA_DEV_PORT_VALID, /* u8 */ 197 | SMC_NLA_DEV_PORT_LNK_CNT, /* u32 */ 198 | __SMC_NLA_DEV_PORT_MAX, 199 | SMC_NLA_DEV_PORT_MAX = __SMC_NLA_DEV_PORT_MAX - 1 200 | }; 201 | 202 | /* SMC_GEN_DEV_SMCD and SMC_GEN_DEV_SMCR attributes */ 203 | enum { 204 | SMC_NLA_DEV_UNSPEC, 205 | SMC_NLA_DEV_USE_CNT, /* u32 */ 206 | SMC_NLA_DEV_IS_CRIT, /* u8 */ 207 | SMC_NLA_DEV_PCI_FID, /* u32 */ 208 | SMC_NLA_DEV_PCI_CHID, /* u16 */ 209 | SMC_NLA_DEV_PCI_VENDOR, /* u16 */ 210 | SMC_NLA_DEV_PCI_DEVICE, /* u16 */ 211 | SMC_NLA_DEV_PCI_ID, /* string */ 212 | SMC_NLA_DEV_PORT, /* nest */ 213 | SMC_NLA_DEV_PORT2, /* nest */ 214 | SMC_NLA_DEV_IB_NAME, /* string */ 215 | __SMC_NLA_DEV_MAX, 216 | SMC_NLA_DEV_MAX = __SMC_NLA_DEV_MAX - 1 217 | }; 218 | 219 | /* SMC_NLA_STATS_T_TX(RX)_RMB_SIZE nested attributes */ 220 | /* SMC_NLA_STATS_TX(RX)PLOAD_SIZE nested attributes */ 221 | enum { 222 | SMC_NLA_STATS_PLOAD_PAD, 223 | SMC_NLA_STATS_PLOAD_8K, /* u64 */ 224 | SMC_NLA_STATS_PLOAD_16K, /* u64 */ 225 | SMC_NLA_STATS_PLOAD_32K, /* u64 */ 226 | SMC_NLA_STATS_PLOAD_64K, /* u64 */ 227 | SMC_NLA_STATS_PLOAD_128K, /* u64 */ 228 | SMC_NLA_STATS_PLOAD_256K, /* u64 */ 229 | SMC_NLA_STATS_PLOAD_512K, /* u64 */ 230 | SMC_NLA_STATS_PLOAD_1024K, /* u64 */ 231 | SMC_NLA_STATS_PLOAD_G_1024K, /* u64 */ 232 | __SMC_NLA_STATS_PLOAD_MAX, 233 | SMC_NLA_STATS_PLOAD_MAX = __SMC_NLA_STATS_PLOAD_MAX - 1 234 | }; 235 | 236 | /* SMC_NLA_STATS_T_TX(RX)_RMB_STATS nested attributes */ 237 | enum { 238 | SMC_NLA_STATS_RMB_PAD, 239 | SMC_NLA_STATS_RMB_SIZE_SM_PEER_CNT, /* u64 */ 240 | SMC_NLA_STATS_RMB_SIZE_SM_CNT, /* u64 */ 241 | SMC_NLA_STATS_RMB_FULL_PEER_CNT, /* u64 */ 242 | SMC_NLA_STATS_RMB_FULL_CNT, /* u64 */ 243 | SMC_NLA_STATS_RMB_REUSE_CNT, /* u64 */ 244 | SMC_NLA_STATS_RMB_ALLOC_CNT, /* u64 */ 245 | SMC_NLA_STATS_RMB_DGRADE_CNT, /* u64 */ 246 | __SMC_NLA_STATS_RMB_MAX, 247 | SMC_NLA_STATS_RMB_MAX = __SMC_NLA_STATS_RMB_MAX - 1 248 | }; 249 | 250 | /* SMC_NLA_STATS_SMCD_TECH and _SMCR_TECH nested attributes */ 251 | enum { 252 | SMC_NLA_STATS_T_PAD, 253 | SMC_NLA_STATS_T_TX_RMB_SIZE, /* nest */ 254 | SMC_NLA_STATS_T_RX_RMB_SIZE, /* nest */ 255 | SMC_NLA_STATS_T_TXPLOAD_SIZE, /* nest */ 256 | SMC_NLA_STATS_T_RXPLOAD_SIZE, /* nest */ 257 | SMC_NLA_STATS_T_TX_RMB_STATS, /* nest */ 258 | SMC_NLA_STATS_T_RX_RMB_STATS, /* nest */ 259 | SMC_NLA_STATS_T_CLNT_V1_SUCC, /* u64 */ 260 | SMC_NLA_STATS_T_CLNT_V2_SUCC, /* u64 */ 261 | SMC_NLA_STATS_T_SRV_V1_SUCC, /* u64 */ 262 | SMC_NLA_STATS_T_SRV_V2_SUCC, /* u64 */ 263 | SMC_NLA_STATS_T_SENDPAGE_CNT, /* u64 */ 264 | SMC_NLA_STATS_T_SPLICE_CNT, /* u64 */ 265 | SMC_NLA_STATS_T_CORK_CNT, /* u64 */ 266 | SMC_NLA_STATS_T_NDLY_CNT, /* u64 */ 267 | SMC_NLA_STATS_T_URG_DATA_CNT, /* u64 */ 268 | SMC_NLA_STATS_T_RX_BYTES, /* u64 */ 269 | SMC_NLA_STATS_T_TX_BYTES, /* u64 */ 270 | SMC_NLA_STATS_T_RX_CNT, /* u64 */ 271 | SMC_NLA_STATS_T_TX_CNT, /* u64 */ 272 | SMC_NLA_STATS_T_RX_RMB_USAGE, /* uint */ 273 | SMC_NLA_STATS_T_TX_RMB_USAGE, /* uint */ 274 | __SMC_NLA_STATS_T_MAX, 275 | SMC_NLA_STATS_T_MAX = __SMC_NLA_STATS_T_MAX - 1 276 | }; 277 | 278 | /* SMC_GEN_STATS attributes */ 279 | enum { 280 | SMC_NLA_STATS_PAD, 281 | SMC_NLA_STATS_SMCD_TECH, /* nest */ 282 | SMC_NLA_STATS_SMCR_TECH, /* nest */ 283 | SMC_NLA_STATS_CLNT_HS_ERR_CNT, /* u64 */ 284 | SMC_NLA_STATS_SRV_HS_ERR_CNT, /* u64 */ 285 | __SMC_NLA_STATS_MAX, 286 | SMC_NLA_STATS_MAX = __SMC_NLA_STATS_MAX - 1 287 | }; 288 | 289 | /* SMC_GEN_FBACK_STATS attributes */ 290 | enum { 291 | SMC_NLA_FBACK_STATS_PAD, 292 | SMC_NLA_FBACK_STATS_TYPE, /* u8 */ 293 | SMC_NLA_FBACK_STATS_SRV_CNT, /* u64 */ 294 | SMC_NLA_FBACK_STATS_CLNT_CNT, /* u64 */ 295 | SMC_NLA_FBACK_STATS_RSN_CODE, /* u32 */ 296 | SMC_NLA_FBACK_STATS_RSN_CNT, /* u16 */ 297 | __SMC_NLA_FBACK_STATS_MAX, 298 | SMC_NLA_FBACK_STATS_MAX = __SMC_NLA_FBACK_STATS_MAX - 1 299 | }; 300 | 301 | /* SMC_NETLINK_UEID attributes */ 302 | enum { 303 | SMC_NLA_EID_TABLE_UNSPEC, 304 | SMC_NLA_EID_TABLE_ENTRY, /* string */ 305 | __SMC_NLA_EID_TABLE_MAX, 306 | SMC_NLA_EID_TABLE_MAX = __SMC_NLA_EID_TABLE_MAX - 1 307 | }; 308 | 309 | /* SMC_NETLINK_SEID attributes */ 310 | enum { 311 | SMC_NLA_SEID_UNSPEC, 312 | SMC_NLA_SEID_ENTRY, /* string */ 313 | SMC_NLA_SEID_ENABLED, /* u8 */ 314 | __SMC_NLA_SEID_TABLE_MAX, 315 | SMC_NLA_SEID_TABLE_MAX = __SMC_NLA_SEID_TABLE_MAX - 1 316 | }; 317 | 318 | /*********************************************************** 319 | * Mimic definitions in kernel/include/uapi/linux/smc_diag.h 320 | ***********************************************************/ 321 | #include 322 | #include 323 | #include 324 | 325 | #define SMC_DIAG_EXTS_PER_CMD 16 326 | /* Sequence numbers */ 327 | enum { 328 | MAGIC_SEQ = 123456, 329 | MAGIC_SEQ_V2, 330 | MAGIC_SEQ_V2_ACK, 331 | }; 332 | 333 | /* Request structure */ 334 | struct smc_diag_req { 335 | __u8 diag_family; 336 | __u8 pad[2]; 337 | __u8 diag_ext; /* Query extended information */ 338 | struct inet_diag_sockid id; 339 | }; 340 | 341 | /* Request structure v2 */ 342 | struct smc_diag_req_v2 { 343 | __u8 diag_family; 344 | __u8 pad[2]; 345 | __u8 diag_ext; /* Query extended information */ 346 | struct inet_diag_sockid id; 347 | __u32 cmd; 348 | __u32 cmd_ext; 349 | __u8 cmd_val[8]; 350 | }; 351 | 352 | /* Base info structure. It contains socket identity (addrs/ports/cookie) based 353 | * on the internal clcsock, and more SMC-related socket data 354 | */ 355 | struct smc_diag_msg { 356 | __u8 diag_family; 357 | __u8 diag_state; 358 | __u8 diag_mode; 359 | __u8 diag_shutdown; 360 | struct inet_diag_sockid id; 361 | 362 | __u32 diag_uid; 363 | __u64 diag_inode; 364 | }; 365 | 366 | /* Mode of a connection */ 367 | enum { 368 | SMC_DIAG_MODE_SMCR, 369 | SMC_DIAG_MODE_FALLBACK_TCP, 370 | SMC_DIAG_MODE_SMCD, 371 | }; 372 | 373 | /* GET_SOCK_DIAG command extensions */ 374 | 375 | enum { 376 | SMC_DIAG_NONE, 377 | SMC_DIAG_CONNINFO, 378 | SMC_DIAG_LGRINFO, 379 | SMC_DIAG_SHUTDOWN, 380 | SMC_DIAG_DMBINFO, 381 | SMC_DIAG_FALLBACK, 382 | __SMC_DIAG_MAX, 383 | }; 384 | 385 | /* V2 Commands */ 386 | enum { 387 | SMC_DIAG_GET_LGR_INFO = SMC_DIAG_EXTS_PER_CMD, 388 | SMC_DIAG_GET_DEV_INFO, 389 | SMC_DIAG_GET_SYS_INFO, 390 | __SMC_DIAG_EXT_MAX, 391 | }; 392 | 393 | /* SMC_DIAG_GET_LGR_INFO command extensions */ 394 | enum { 395 | SMC_DIAG_LGR_INFO_SMCR = 1, 396 | SMC_DIAG_LGR_INFO_SMCR_LINK, 397 | SMC_DIAG_LGR_INFO_SMCD, 398 | }; 399 | 400 | /* SMC_DIAG_GET_DEV_INFO command extensions */ 401 | enum { 402 | SMC_DIAG_DEV_INFO_SMCD = 1, 403 | SMC_DIAG_DEV_INFO_SMCR, 404 | }; 405 | 406 | /* SMC_DIAG_GET_SYS_INFO command extensions */ 407 | enum { 408 | SMC_DIAG_SYS_INFO = 1, 409 | }; 410 | 411 | #define SMC_DIAG_MAX (__SMC_DIAG_MAX - 1) 412 | #define SMC_DIAG_EXT_MAX (__SMC_DIAG_EXT_MAX - 1) 413 | 414 | /* SMC_DIAG_CONNINFO */ 415 | #define IB_DEVICE_NAME_MAX 64 416 | 417 | struct smc_diag_cursor { 418 | __u16 reserved; 419 | __u16 wrap; 420 | __u32 count; 421 | }; 422 | 423 | struct smc_diag_conninfo { 424 | __u32 token; /* unique connection id */ 425 | __u32 sndbuf_size; /* size of send buffer */ 426 | __u32 rmbe_size; /* size of RMB element */ 427 | __u32 peer_rmbe_size; /* size of peer RMB element */ 428 | /* local RMB element cursors */ 429 | struct smc_diag_cursor rx_prod; /* received producer cursor */ 430 | struct smc_diag_cursor rx_cons; /* received consumer cursor */ 431 | /* peer RMB element cursors */ 432 | struct smc_diag_cursor tx_prod; /* sent producer cursor */ 433 | struct smc_diag_cursor tx_cons; /* sent consumer cursor */ 434 | __u8 rx_prod_flags; /* received producer flags */ 435 | __u8 rx_conn_state_flags; /* recvd connection flags*/ 436 | __u8 tx_prod_flags; /* sent producer flags */ 437 | __u8 tx_conn_state_flags; /* sent connection flags*/ 438 | /* send buffer cursors */ 439 | struct smc_diag_cursor tx_prep; /* prepared to be sent cursor */ 440 | struct smc_diag_cursor tx_sent; /* sent cursor */ 441 | struct smc_diag_cursor tx_fin; /* confirmed sent cursor */ 442 | }; 443 | 444 | struct smc_v2_lgr_info { 445 | __u8 v2_lgr_info_received; 446 | __u8 smc_version; 447 | __u8 peer_smc_release; 448 | __u8 peer_os; /* peer operating system */ 449 | __u8 negotiated_eid[SMC_MAX_EID_LEN + 1]; 450 | __u8 peer_hostname[SMC_MAX_HOSTNAME_LEN + 1]; 451 | /* SMC-R v2 specific */ 452 | __u8 smcr_direct; 453 | }; 454 | 455 | /* unused 456 | struct smc_system_info { 457 | __u8 smc_version; 458 | __u8 smc_release; 459 | __u8 ueid_count; 460 | __u8 smc_ism_is_v2; 461 | __u32 reserved; 462 | __u8 local_hostname[SMC_MAX_HOSTNAME_LEN]; 463 | __u8 seid[SMC_MAX_EID_LEN]; 464 | __u8 ueid[SMC_MAX_UEID][SMC_MAX_EID_LEN]; 465 | }; 466 | */ 467 | 468 | /* SMC_DIAG_LINKINFO */ 469 | 470 | struct smc_diag_linkinfo { 471 | __u8 link_id; /* link identifier */ 472 | __u8 ibname[IB_DEVICE_NAME_MAX]; /* name of the RDMA device */ 473 | __u8 ibport; /* RDMA device port number */ 474 | __u8 gid[40]; /* local GID */ 475 | __u8 peer_gid[40]; /* peer GID */ 476 | }; 477 | 478 | struct smc_diag_linkinfo_v2 { 479 | struct smc_diag_linkinfo v1; 480 | __u32 conn_cnt; 481 | __u8 netdev[IFNAMSIZ]; 482 | __u8 link_uid[4]; 483 | __u8 peer_link_uid[4]; 484 | __u32 link_state; 485 | }; 486 | 487 | struct smc_diag_lgrinfo { 488 | struct smc_diag_linkinfo lnk[1]; 489 | __u8 role; 490 | }; 491 | 492 | struct smc_diag_fallback { 493 | __u32 reason; 494 | __u32 peer_diagnosis; 495 | }; 496 | 497 | struct smcd_diag_dmbinfo { /* SMC-D Socket internals */ 498 | __u32 linkid; /* Link identifier */ 499 | __u64 peer_gid; /* Peer GID */ 500 | __u64 my_gid; /* My GID */ 501 | __u64 token; /* Token of DMB */ 502 | __u64 peer_token; /* Token of remote DMBE */ 503 | }; 504 | 505 | struct smcd_diag_dmbinfo_v2 { 506 | struct smcd_diag_dmbinfo v1; 507 | __u8 pnet_id[SMC_MAX_PNETID_LEN]; 508 | __u32 conns_num; 509 | __u16 chid; 510 | __u8 vlan_id; 511 | __u64 sndbuf_alloc; 512 | __u64 dmb_alloc; 513 | struct smc_v2_lgr_info v2_lgr_info; 514 | }; 515 | 516 | struct smc_diag_dev_info { 517 | __u8 pnet_id[SMC_MAX_PORTS][SMC_MAX_PNETID_LEN]; 518 | __u8 pnetid_by_user[SMC_MAX_PORTS]; 519 | __u32 use_cnt; 520 | __u8 is_critical; 521 | __u32 pci_fid; 522 | __u16 pci_pchid; 523 | __u16 pci_vendor; 524 | __u16 pci_device; 525 | __u8 pci_id[SMC_PCI_ID_STR_LEN]; 526 | __u8 dev_name[IB_DEVICE_NAME_MAX]; 527 | __u8 netdev[SMC_MAX_PORTS][IFNAMSIZ]; 528 | __u8 port_state[SMC_MAX_PORTS]; 529 | __u8 port_valid[SMC_MAX_PORTS]; 530 | __u32 lnk_cnt_by_port[SMC_MAX_PORTS]; /* # lnks per port */ 531 | }; 532 | 533 | 534 | struct smc_diag_lgr { 535 | __u8 lgr_id[SMC_LGR_ID_SIZE]; 536 | __u8 lgr_role; 537 | __u8 lgr_type; 538 | __u8 pnet_id[SMC_MAX_PNETID_LEN]; 539 | __u8 vlan_id; 540 | __u32 conns_num; 541 | __u64 sndbuf_alloc; 542 | __u64 rmb_alloc; 543 | struct smc_v2_lgr_info v2_lgr_info; 544 | }; 545 | #endif /* SMCTOOLS_COMMON_H */ 546 | -------------------------------------------------------------------------------- /stats.h: -------------------------------------------------------------------------------- 1 | /* 2 | * SMC Tools - Shared Memory Communication Tools 3 | * 4 | * Copyright IBM Corp. 2021 5 | * 6 | * Author(s): Guvenc Gulce 7 | * 8 | * Userspace program for SMC Information display 9 | * 10 | * All rights reserved. This program and the accompanying materials 11 | * are made available under the terms of the Eclipse Public License v1.0 12 | * which accompanies this distribution, and is available at 13 | * http://www.eclipse.org/legal/epl-v10.html 14 | */ 15 | 16 | #ifndef SMC_SYSTEM_H_ 17 | #define SMC_SYSTEM_H_ 18 | 19 | #define SMC_CLC_DECL_MEM 0x01010000 /* insufficient memory resources */ 20 | #define SMC_CLC_DECL_TIMEOUT_CL 0x02010000 /* timeout w4 QP confirm link */ 21 | #define SMC_CLC_DECL_TIMEOUT_AL 0x02020000 /* timeout w4 QP add link */ 22 | #define SMC_CLC_DECL_CNFERR 0x03000000 /* configuration error */ 23 | #define SMC_CLC_DECL_PEERNOSMC 0x03010000 /* peer did not indicate SMC */ 24 | #define SMC_CLC_DECL_IPSEC 0x03020000 /* IPsec usage */ 25 | #define SMC_CLC_DECL_NOSMCDEV 0x03030000 /* no SMC device found (R or D) */ 26 | #define SMC_CLC_DECL_NOSMCDDEV 0x03030001 /* no SMC-D device found */ 27 | #define SMC_CLC_DECL_NOSMCRDEV 0x03030002 /* no SMC-R device found */ 28 | #define SMC_CLC_DECL_NOISM2SUPP 0x03030003 /* hardware has no ISMv2 support */ 29 | #define SMC_CLC_DECL_NOV2EXT 0x03030004 /* peer sent no clc v2 extension */ 30 | #define SMC_CLC_DECL_NOV2DEXT 0x03030005 /* peer sent no clc SMC-Dv2 ext. */ 31 | #define SMC_CLC_DECL_NOSEID 0x03030006 /* peer sent no SEID */ 32 | #define SMC_CLC_DECL_NOSMCD2DEV 0x03030007 /* no SMC-Dv2 device found */ 33 | #define SMC_CLC_DECL_MODEUNSUPP 0x03040000 /* smc modes do not match (R or D)*/ 34 | #define SMC_CLC_DECL_RMBE_EC 0x03050000 /* peer has eyecatcher in RMBE */ 35 | #define SMC_CLC_DECL_OPTUNSUPP 0x03060000 /* fastopen sockopt not supported */ 36 | #define SMC_CLC_DECL_DIFFPREFIX 0x03070000 /* IP prefix / subnet mismatch */ 37 | #define SMC_CLC_DECL_GETVLANERR 0x03080000 /* err to get vlan id of ip device*/ 38 | #define SMC_CLC_DECL_ISMVLANERR 0x03090000 /* err to reg vlan id on ism dev */ 39 | #define SMC_CLC_DECL_NOACTLINK 0x030a0000 /* no active smc-r link in lgr */ 40 | #define SMC_CLC_DECL_NOSRVLINK 0x030b0000 /* SMC-R link from srv not found */ 41 | #define SMC_CLC_DECL_VERSMISMAT 0x030c0000 /* SMC version mismatch */ 42 | #define SMC_CLC_DECL_MAX_DMB 0x030d0000 /* SMC-D DMB limit exceeded */ 43 | #define SMC_CLC_DECL_SYNCERR 0x04000000 /* synchronization error */ 44 | #define SMC_CLC_DECL_PEERDECL 0x05000000 /* peer declined during handshake */ 45 | #define SMC_CLC_DECL_INTERR 0x09990000 /* internal error */ 46 | #define SMC_CLC_DECL_ERR_RTOK 0x09990001 /* rtoken handling failed */ 47 | #define SMC_CLC_DECL_ERR_RDYLNK 0x09990002 /* ib ready link failed */ 48 | #define SMC_CLC_DECL_ERR_REGRMB 0x09990003 /* reg rmb failed */ 49 | 50 | #define SMC_TYPE_R 0 51 | #define SMC_TYPE_D 1 52 | #define SMC_SERVER 1 53 | #define SMC_CLIENT 0 54 | 55 | #define SMC_MAX_FBACK_RSN_CNT 30 56 | 57 | enum { 58 | SMC_BUF_8K, 59 | SMC_BUF_16K, 60 | SMC_BUF_32K, 61 | SMC_BUF_64K, 62 | SMC_BUF_128K, 63 | SMC_BUF_256K, 64 | SMC_BUF_512K, 65 | SMC_BUF_1024K, 66 | SMC_BUF_G_1024K, 67 | SMC_BUF_MAX, 68 | }; 69 | 70 | struct smc_stats_fback { 71 | int fback_code; 72 | int count; 73 | }; 74 | 75 | struct smc_stats_rsn { 76 | struct smc_stats_fback srv[SMC_MAX_FBACK_RSN_CNT]; 77 | struct smc_stats_fback clnt[SMC_MAX_FBACK_RSN_CNT]; 78 | __u64 srv_fback_cnt; 79 | __u64 clnt_fback_cnt; 80 | }; 81 | 82 | struct smc_stats_rmbcnt { 83 | __u64 buf_size_small_peer_cnt; 84 | __u64 buf_size_small_cnt; 85 | __u64 buf_full_peer_cnt; 86 | __u64 buf_full_cnt; 87 | __u64 reuse_cnt; 88 | __u64 alloc_cnt; 89 | __u64 dgrade_cnt; 90 | }; 91 | 92 | struct smc_stats_memsize { 93 | __u64 buf[SMC_BUF_MAX]; 94 | }; 95 | 96 | struct smc_stats_tech { 97 | struct smc_stats_memsize tx_rmbsize; 98 | struct smc_stats_memsize rx_rmbsize; 99 | struct smc_stats_memsize tx_pd; 100 | struct smc_stats_memsize rx_pd; 101 | struct smc_stats_rmbcnt rmb_tx; 102 | struct smc_stats_rmbcnt rmb_rx; 103 | __u64 clnt_v1_succ_cnt; 104 | __u64 clnt_v2_succ_cnt; 105 | __u64 srv_v1_succ_cnt; 106 | __u64 srv_v2_succ_cnt; 107 | __u64 sendpage_cnt; 108 | __u64 urg_data_cnt; 109 | __u64 splice_cnt; 110 | __u64 cork_cnt; 111 | __u64 ndly_cnt; 112 | __u64 rx_bytes; 113 | __u64 tx_bytes; 114 | __u64 rx_cnt; 115 | __u64 tx_cnt; 116 | __u64 rx_rmbuse; 117 | __u64 tx_rmbuse; 118 | }; 119 | 120 | struct smc_stats { 121 | struct smc_stats_tech smc[2]; 122 | __u64 clnt_hshake_err_cnt; 123 | __u64 srv_hshake_err_cnt; 124 | }; 125 | 126 | int invoke_stats(int argc, char **argv, int detail_level); 127 | 128 | #endif /* SMC_SYSTEM_H_ */ 129 | -------------------------------------------------------------------------------- /ueid.c: -------------------------------------------------------------------------------- 1 | /* 2 | * SMC Tools - Shared Memory Communication Tools 3 | * 4 | * Copyright IBM Corp. 2020 5 | * 6 | * Userspace program for SMC Information display 7 | * 8 | * All rights reserved. This program and the accompanying materials 9 | * are made available under the terms of the Eclipse Public License v1.0 10 | * which accompanies this distribution, and is available at 11 | * http://www.eclipse.org/legal/epl-v10.html 12 | */ 13 | #include 14 | #include 15 | #include 16 | #include 17 | #include 18 | #include 19 | 20 | #include "smctools_common.h" 21 | #include "util.h" 22 | #include "libnetlink.h" 23 | #include "ueid.h" 24 | 25 | static int add_cmd = 0; 26 | static int del_cmd = 0; 27 | static int flush_cmd = 0; 28 | static int show_cmd = 0; 29 | 30 | static char target_eid[SMC_MAX_EID_LEN + 1] = {0}; 31 | 32 | extern int smc_id; 33 | extern struct nl_sock *sk; 34 | 35 | const struct nla_policy 36 | smc_gen_ueid_policy[SMC_NLA_EID_TABLE_MAX + 1] = { 37 | [SMC_NLA_EID_TABLE_UNSPEC] = { .type = NLA_UNSPEC }, 38 | [SMC_NLA_EID_TABLE_ENTRY] = { .type = NLA_NUL_STRING }, 39 | }; 40 | 41 | static void usage(void) 42 | { 43 | fprintf(stderr, 44 | "Usage: smcd ueid [show]\n" 45 | " smcd ueid add \n" 46 | " smcd ueid del \n" 47 | " smcd ueid flush\n" 48 | ); 49 | exit(-1); 50 | } 51 | 52 | static int gen_nl_ueid_handle(int cmd, char *ueid, int (*cb_handler)(struct nl_msg *msg, void *arg)) 53 | { 54 | int rc = EXIT_FAILURE, nlmsg_flags = 0; 55 | struct nl_msg *msg; 56 | 57 | nl_socket_modify_cb(sk, NL_CB_VALID, NL_CB_CUSTOM, cb_handler, NULL); 58 | 59 | /* Allocate a netlink message and set header information. */ 60 | msg = nlmsg_alloc(); 61 | if (!msg) { 62 | nl_perror(NLE_NOMEM, "Error"); 63 | rc = EXIT_FAILURE; 64 | goto err; 65 | } 66 | 67 | if (cmd == SMC_NETLINK_DUMP_UEID) 68 | nlmsg_flags = NLM_F_DUMP; 69 | 70 | if (!genlmsg_put(msg, NL_AUTO_PORT, NL_AUTO_SEQ, smc_id, 0, nlmsg_flags, 71 | cmd, SMC_GENL_FAMILY_VERSION)) { 72 | nl_perror(rc, "Error"); 73 | rc = EXIT_FAILURE; 74 | goto err; 75 | } 76 | 77 | if (ueid && ueid[0]) { 78 | rc = nla_put_string(msg, SMC_NLA_EID_TABLE_ENTRY, ueid); 79 | if (rc < 0) { 80 | nl_perror(rc, "Error"); 81 | rc = EXIT_FAILURE; 82 | goto err; 83 | } 84 | 85 | } 86 | 87 | /* Send message */ 88 | rc = nl_send_auto(sk, msg); 89 | if (rc < 0) { 90 | nl_perror(rc, "Error"); 91 | rc = EXIT_FAILURE; 92 | goto err; 93 | } 94 | 95 | /* Receive reply message, returns number of cb invocations. */ 96 | rc = nl_recvmsgs_default(sk); 97 | 98 | if (rc < 0) { 99 | /* For cmd "UEID remove" the kernel might return ENOENT when 100 | * the specified UEID is not in the list. 101 | * This is mapped to NLE_OBJ_NOTFOUND in libnl, lib/error.c. 102 | */ 103 | if (rc == -NLE_OPNOTSUPP) { 104 | fprintf(stderr, "Error: operation not supported by kernel\n"); 105 | } else if (cmd == SMC_NETLINK_REMOVE_UEID) { 106 | if (rc == -NLE_OBJ_NOTFOUND) { 107 | fprintf(stderr, "Error: specified User EID is not defined\n"); 108 | } else if (rc == -NLE_AGAIN) { 109 | fprintf(stderr, "Info: the System EID was activated because the last User EID was removed\n"); 110 | } else { 111 | fprintf(stderr, "Error: specified User EID is not defined\n"); 112 | } 113 | } else if (cmd == SMC_NETLINK_ADD_UEID) { 114 | if (rc == -NLE_INVAL) { 115 | fprintf(stderr, "Error: specified User EID was rejected by the kernel\n"); 116 | } else if (rc == -NLE_NOMEM) { 117 | fprintf(stderr, "Error: kernel reported an out of memory condition\n"); 118 | } else if (rc == -NLE_RANGE) { 119 | fprintf(stderr, "Error: specified User EID was rejected because the maximum number of User EIDs is reached\n"); 120 | } else if (rc == -NLE_EXIST) { 121 | fprintf(stderr, "Error: specified User EID is already defined\n"); 122 | } else { 123 | nl_perror(rc, "Error"); 124 | } 125 | } else { 126 | nl_perror(rc, "Error"); 127 | } 128 | rc = EXIT_FAILURE; 129 | goto err; 130 | } 131 | 132 | nlmsg_free(msg); 133 | return EXIT_SUCCESS; 134 | err: 135 | nlmsg_free(msg); 136 | return rc; 137 | } 138 | 139 | static int handle_gen_ueid_reply(struct nl_msg *msg, void *arg) 140 | { 141 | struct nlattr *attrs[SMC_NLA_EID_TABLE_ENTRY + 1]; 142 | struct nlmsghdr *hdr = nlmsg_hdr(msg); 143 | int rc = NL_OK; 144 | 145 | if (genlmsg_parse(hdr, 0, attrs, SMC_NLA_EID_TABLE_ENTRY, 146 | (struct nla_policy *)smc_gen_ueid_policy) < 0) { 147 | fprintf(stderr, "Error: invalid data returned: smc_gen_ueid_policy\n"); 148 | nl_msg_dump(msg, stderr); 149 | return NL_STOP; 150 | } 151 | 152 | if (!attrs[SMC_NLA_EID_TABLE_ENTRY]) 153 | return NL_STOP; 154 | 155 | printf("%s\n", nla_get_string(attrs[SMC_NLA_EID_TABLE_ENTRY])); 156 | return rc; 157 | } 158 | 159 | static char ueid_valid(char *ueid) 160 | { 161 | char *end = ueid + SMC_MAX_EID_LEN; 162 | 163 | while (--end >= ueid && isspace(*end)) 164 | ; 165 | if (end < ueid) { 166 | fprintf(stderr, "Error: Invalid User EID specified: EID is empty\n"); 167 | return 0; 168 | } 169 | if (!isalnum(*ueid)) { 170 | fprintf(stderr, "Error: Invalid User EID specified: first character must be alphanumeric\n"); 171 | return 0; 172 | } 173 | if (strstr(ueid, "..")) { 174 | fprintf(stderr, "Error: Invalid User EID specified: consecutive dots not allowed\n"); 175 | return 0; 176 | } 177 | while (ueid <= end) { 178 | if ((!isalnum(*ueid) || islower(*ueid)) && *ueid != '.' && *ueid != '-') { 179 | fprintf(stderr, "Error: Invalid User EID specified: unsupported character: '%c'\n", *ueid); 180 | fprintf(stderr, " Supported characters are: A-Z, 0-9, '.' and '-'\n"); 181 | return 0; 182 | } 183 | ueid++; 184 | } 185 | return 1; 186 | } 187 | 188 | static void set_eid(char *eid) 189 | { 190 | if (strlen(eid) > SMC_MAX_EID_LEN) { 191 | fprintf(stderr, "Error: Invalid User EID specified: EID is longer than 32 characters\n"); 192 | exit(-1); 193 | } 194 | /* pad to 32 byte using blanks */ 195 | sprintf(target_eid, "%-32s", eid); 196 | 197 | if (!ueid_valid(target_eid)) 198 | exit(-1); 199 | } 200 | 201 | static void handle_cmd_params(int argc, char **argv) 202 | { 203 | if (argc == 0) { 204 | show_cmd = 1; /* no object given, so use the default "show" */ 205 | return; 206 | } 207 | 208 | while (1) { 209 | if (add_cmd) { 210 | set_eid(argv[0]); 211 | break; 212 | } else if (del_cmd) { 213 | set_eid(argv[0]); 214 | break; 215 | } else if (contains(argv[0], "help") == 0) { 216 | usage(); 217 | } else if (contains(argv[0], "add") == 0) { 218 | add_cmd = 1; 219 | } else if (contains(argv[0], "del") == 0) { 220 | del_cmd = 1; 221 | } else if (contains(argv[0], "flush") == 0) { 222 | flush_cmd = 1; 223 | break; 224 | } else if (contains(argv[0], "show") == 0) { 225 | show_cmd = 1; 226 | break; 227 | } else { 228 | usage(); 229 | } 230 | if (!NEXT_ARG_OK()) 231 | break; 232 | NEXT_ARG(); 233 | } 234 | /* Too many parameters or wrong sequence of parameters */ 235 | if (NEXT_ARG_OK()) 236 | usage(); 237 | 238 | /* Only single cmd expected */ 239 | if ((add_cmd + del_cmd + flush_cmd + show_cmd) != 1) 240 | usage(); 241 | 242 | /* eid required for command */ 243 | if (!target_eid[0] && (add_cmd || del_cmd)) 244 | usage(); 245 | } 246 | 247 | int invoke_ueid(int argc, char **argv, int detail_level) 248 | { 249 | int rc = EXIT_SUCCESS; 250 | 251 | handle_cmd_params(argc, argv); 252 | 253 | if (add_cmd) { 254 | rc = gen_nl_ueid_handle(SMC_NETLINK_ADD_UEID, target_eid, handle_gen_ueid_reply); 255 | } else if (del_cmd) { 256 | rc = gen_nl_ueid_handle(SMC_NETLINK_REMOVE_UEID, target_eid, handle_gen_ueid_reply); 257 | } else if (flush_cmd) { 258 | rc = gen_nl_ueid_handle(SMC_NETLINK_FLUSH_UEID, NULL, handle_gen_ueid_reply); 259 | } else if (show_cmd) { 260 | rc = gen_nl_ueid_handle(SMC_NETLINK_DUMP_UEID, NULL, handle_gen_ueid_reply); 261 | } else { 262 | printf("Error: Unknown command\n"); /* we should never come here ... */ 263 | } 264 | 265 | return rc; 266 | } 267 | -------------------------------------------------------------------------------- /ueid.h: -------------------------------------------------------------------------------- 1 | /* 2 | * SMC Tools - Shared Memory Communication Tools 3 | * 4 | * Copyright IBM Corp. 2020 5 | * 6 | * Userspace program for SMC Information display 7 | * 8 | * All rights reserved. This program and the accompanying materials 9 | * are made available under the terms of the Eclipse Public License v1.0 10 | * which accompanies this distribution, and is available at 11 | * http://www.eclipse.org/legal/epl-v10.html 12 | */ 13 | 14 | #ifndef UEID_H_ 15 | #define UEID_H_ 16 | 17 | extern struct rtnl_handle rth; 18 | 19 | int invoke_ueid(int argc, char **argv, int detail_level); 20 | 21 | #endif /* UEID_H_ */ 22 | -------------------------------------------------------------------------------- /util.c: -------------------------------------------------------------------------------- 1 | /* 2 | * SMC Tools - Shared Memory Communication Tools 3 | * 4 | * Copyright IBM Corp. 2020 5 | * 6 | * Author(s): Guvenc Gulce 7 | * 8 | * Userspace program for SMC Information display 9 | * 10 | * All rights reserved. This program and the accompanying materials 11 | * are made available under the terms of the Eclipse Public License v1.0 12 | * which accompanies this distribution, and is available at 13 | * http://www.eclipse.org/legal/epl-v10.html 14 | */ 15 | #include 16 | #include 17 | #include 18 | #include 19 | 20 | #include "util.h" 21 | 22 | void print_unsup_msg(void) 23 | { 24 | fprintf(stderr, "Error: Kernel does not support this parameter !\n"); 25 | exit(-1); 26 | } 27 | 28 | void print_type_error(void) { 29 | fprintf(stderr, "Error: You entered an invalid type. Possible values are smcd and smcr !\n"); 30 | exit(-1); 31 | } 32 | 33 | char* trim_space(char *str) 34 | { 35 | char *end; 36 | 37 | while (isspace(*str)) { 38 | str = str + 1; 39 | } 40 | /* remove trailing whitespace */ 41 | end = str + strlen((const char*)str) - 1; 42 | while (end > str && isspace(*end)) { 43 | end = end - 1; 44 | } 45 | *(end+1) = '\0'; 46 | return str; 47 | } 48 | 49 | int contains(const char *prfx, const char *str) 50 | { 51 | if (!*prfx) 52 | return 1; 53 | while (*str && *prfx == *str) { 54 | prfx++; 55 | str++; 56 | } 57 | 58 | return !!*prfx; 59 | } 60 | 61 | static void determine_mag_factor(int leading_places, char *magnitude, 62 | double *factor) 63 | { 64 | if (leading_places < 7) { 65 | *magnitude = 'K'; 66 | *factor = 1000; 67 | } 68 | else if (leading_places < 10) { 69 | *magnitude = 'M'; 70 | *factor = 1000000; 71 | } 72 | else if (leading_places < 13) { 73 | *magnitude = 'G'; 74 | *factor = 1000000000; 75 | } 76 | else { 77 | // this is quite expensive, hence we avoid if possible 78 | *factor = pow(1000, leading_places/3); 79 | if (leading_places < 16) 80 | *magnitude = 'T'; 81 | else if (leading_places < 19) 82 | *magnitude = 'P'; 83 | else 84 | *magnitude = '?'; 85 | } 86 | } 87 | 88 | 89 | static void determine_digs(int leading_places, int max_digs, 90 | int *num_full_digs, int *num_places) 91 | { 92 | *num_full_digs = leading_places % 3; 93 | if (*num_full_digs == 0) 94 | *num_full_digs = 3; 95 | *num_places = max_digs - *num_full_digs - 2; 96 | if (*num_places <= 0) { 97 | *num_places = 0; 98 | *num_full_digs = max_digs - 1; 99 | } 100 | } 101 | 102 | 103 | int get_abbreviated(uint64_t num, int max_digs, char *res) 104 | { 105 | int num_full_digs, leading_places; 106 | char magnitude; 107 | int num_places; 108 | double factor; 109 | char tmp[128]; 110 | 111 | if (num == 0) { 112 | snprintf(res, max_digs + 1, "0"); 113 | return 1; 114 | } 115 | 116 | leading_places = sprintf(tmp, "%lld", (long long int)num); 117 | if (leading_places < 4) { 118 | snprintf(res, max_digs + 1, "%lu", num); 119 | return leading_places; 120 | } 121 | 122 | determine_digs(leading_places, max_digs, &num_full_digs, &num_places); 123 | determine_mag_factor(leading_places, &magnitude, &factor); 124 | 125 | double tmpnum = num / factor; 126 | if (tmpnum + 5 * pow(10, -1 - num_places) >= pow(10, num_full_digs)) { 127 | if (num_places > 0) 128 | // just strip down one decimal place, 129 | // e.g. 9.96... with 1.1 format would result in 130 | // 10.0 otherwise 131 | num_places--; 132 | else { 133 | // indicate that we need one more leading place 134 | // e.g. 999.872 with 3.0 format digits would result 135 | // in 1.000K otherwise 136 | leading_places++; 137 | determine_digs(leading_places, max_digs, &num_full_digs, &num_places); 138 | determine_mag_factor(leading_places, &magnitude, &factor); 139 | tmpnum = num / factor; 140 | } 141 | } 142 | 143 | snprintf(res, max_digs + 1, "%*.*lf%c", max_digs - 1, num_places, num / factor, magnitude); 144 | return 0; 145 | } 146 | -------------------------------------------------------------------------------- /util.h: -------------------------------------------------------------------------------- 1 | /* 2 | * SMC Tools - Shared Memory Communication Tools 3 | * 4 | * Copyright IBM Corp. 2020 5 | * 6 | * Author(s): Guvenc Gulce 7 | * 8 | * Userspace program for SMC Information display 9 | * 10 | * All rights reserved. This program and the accompanying materials 11 | * are made available under the terms of the Eclipse Public License v1.0 12 | * which accompanies this distribution, and is available at 13 | * http://www.eclipse.org/legal/epl-v10.html 14 | */ 15 | #ifndef UTIL_H_ 16 | #define UTIL_H_ 17 | #include 18 | #include 19 | #define SMC_DETAIL_LEVEL_V 1 20 | #define SMC_DETAIL_LEVEL_VV 2 21 | #define SMC_OPTION_ABS -1 22 | #define SMC_OPTION_DETAIL_ABS -2 23 | #define SMC_TYPE_STR_MAX 5 24 | 25 | 26 | #define NEXT_ARG() do { argv++; argc--; } while(0) 27 | #define NEXT_ARG_OK() (argc - 1 > 0) 28 | #define PREV_ARG() do { argv--; argc++; } while(0) 29 | 30 | void print_unsup_msg(void); 31 | void print_type_error(void); 32 | char* trim_space(char *str); 33 | int get_abbreviated(uint64_t num, int max_digs, char *res); 34 | int contains(const char *prfx, const char *str); 35 | 36 | static inline int is_str_empty(char *str) 37 | { 38 | if (str && str[0] == '\0') 39 | return 1; 40 | else 41 | return 0; 42 | } 43 | 44 | #endif /* UTIL_H_ */ 45 | --------------------------------------------------------------------------------