├── .clang-format ├── .vscode └── settings.json ├── README.md ├── benchmarks ├── bench_allgather.py └── bench_p2p.py ├── csrc ├── buffer.cu ├── buffer.cuh ├── internode.cu ├── intranode.cu ├── kernels │ ├── copy.cu │ └── copy.cuh ├── launch.cuh ├── nvshmem.hpp ├── ptx_wrapper.cuh ├── put.cu ├── put.cuh ├── pybind.cu ├── sym.cuh ├── sync.cuh └── utils.hpp ├── docs ├── 00-introduction.md └── 01-initialization.md ├── nvshmem_tutorial ├── __init__.py └── buffer.py ├── scripts ├── install.sh ├── run_bench_internode_allgather.sh ├── run_bench_internode_p2p.sh ├── run_bench_intranode_allgather.sh ├── run_bench_intranode_p2p.sh ├── run_internode_allgather.sh ├── run_internode_nvshmem.sh ├── run_intranode_allgather.sh └── run_intranode_nvshmem.sh ├── setup.py └── tests ├── test_internode_allgather.py ├── test_internode_nvshmem.py ├── test_intranode_allgather.py ├── test_intranode_nvshmem.py └── test_tma_copy.py /.clang-format: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/.clang-format -------------------------------------------------------------------------------- /.vscode/settings.json: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/.vscode/settings.json -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/README.md -------------------------------------------------------------------------------- /benchmarks/bench_allgather.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/benchmarks/bench_allgather.py -------------------------------------------------------------------------------- /benchmarks/bench_p2p.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/benchmarks/bench_p2p.py -------------------------------------------------------------------------------- /csrc/buffer.cu: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/csrc/buffer.cu -------------------------------------------------------------------------------- /csrc/buffer.cuh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/csrc/buffer.cuh -------------------------------------------------------------------------------- /csrc/internode.cu: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/csrc/internode.cu -------------------------------------------------------------------------------- /csrc/intranode.cu: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/csrc/intranode.cu -------------------------------------------------------------------------------- /csrc/kernels/copy.cu: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/csrc/kernels/copy.cu -------------------------------------------------------------------------------- /csrc/kernels/copy.cuh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/csrc/kernels/copy.cuh -------------------------------------------------------------------------------- /csrc/launch.cuh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/csrc/launch.cuh -------------------------------------------------------------------------------- /csrc/nvshmem.hpp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/csrc/nvshmem.hpp -------------------------------------------------------------------------------- /csrc/ptx_wrapper.cuh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/csrc/ptx_wrapper.cuh -------------------------------------------------------------------------------- /csrc/put.cu: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/csrc/put.cu -------------------------------------------------------------------------------- /csrc/put.cuh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/csrc/put.cuh -------------------------------------------------------------------------------- /csrc/pybind.cu: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/csrc/pybind.cu -------------------------------------------------------------------------------- /csrc/sym.cuh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/csrc/sym.cuh -------------------------------------------------------------------------------- /csrc/sync.cuh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/csrc/sync.cuh -------------------------------------------------------------------------------- /csrc/utils.hpp: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/csrc/utils.hpp -------------------------------------------------------------------------------- /docs/00-introduction.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/docs/00-introduction.md -------------------------------------------------------------------------------- /docs/01-initialization.md: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/docs/01-initialization.md -------------------------------------------------------------------------------- /nvshmem_tutorial/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/nvshmem_tutorial/__init__.py -------------------------------------------------------------------------------- /nvshmem_tutorial/buffer.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/nvshmem_tutorial/buffer.py -------------------------------------------------------------------------------- /scripts/install.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | set -e 4 | 5 | pip install -e . -------------------------------------------------------------------------------- /scripts/run_bench_internode_allgather.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/scripts/run_bench_internode_allgather.sh -------------------------------------------------------------------------------- /scripts/run_bench_internode_p2p.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/scripts/run_bench_internode_p2p.sh -------------------------------------------------------------------------------- /scripts/run_bench_intranode_allgather.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/scripts/run_bench_intranode_allgather.sh -------------------------------------------------------------------------------- /scripts/run_bench_intranode_p2p.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/scripts/run_bench_intranode_p2p.sh -------------------------------------------------------------------------------- /scripts/run_internode_allgather.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/scripts/run_internode_allgather.sh -------------------------------------------------------------------------------- /scripts/run_internode_nvshmem.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/scripts/run_internode_nvshmem.sh -------------------------------------------------------------------------------- /scripts/run_intranode_allgather.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/scripts/run_intranode_allgather.sh -------------------------------------------------------------------------------- /scripts/run_intranode_nvshmem.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/scripts/run_intranode_nvshmem.sh -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/setup.py -------------------------------------------------------------------------------- /tests/test_internode_allgather.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/tests/test_internode_allgather.py -------------------------------------------------------------------------------- /tests/test_internode_nvshmem.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/tests/test_internode_nvshmem.py -------------------------------------------------------------------------------- /tests/test_intranode_allgather.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/tests/test_intranode_allgather.py -------------------------------------------------------------------------------- /tests/test_intranode_nvshmem.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/tests/test_intranode_nvshmem.py -------------------------------------------------------------------------------- /tests/test_tma_copy.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/KuangjuX/NVSHMEM-Tutorial/HEAD/tests/test_tma_copy.py --------------------------------------------------------------------------------