├── .github ├── .wordlist.txt ├── ISSUE_TEMPLATE │ └── best_practice.yaml ├── PULL_REQUEST_TEMPLATE │ └── pull_request_template.md ├── dependabot.yml ├── labeler.yml └── workflows │ ├── linkcheck.yml │ ├── linter.yml │ ├── spell.yml │ └── triage.yml ├── .gitignore ├── .markdown-lint.yml ├── .spellcheck.yml ├── .textlintrc ├── CONTRIBUTING.md ├── CONTRIBUTORS.md ├── LICENSE.md ├── LICENSES ├── APACHE-2.txt └── CC-BY-4.0.txt ├── Makefile ├── README.md ├── doc ├── README.md ├── best_cnf_dev.md ├── cbpps │ ├── 0001-cnf-best-practice-proposal-process.md │ ├── 0002-no-root-in-containers.md │ ├── 0003-exceptions.md │ ├── 0004-do-not-run-containers-with-privilege-flag.md │ ├── 0005-single-concern-per-container.md │ ├── NNNN-cbpp-template.md │ ├── README.md │ ├── api-server-configuration-best-practices.md │ └── cnti_best_practice_process.md ├── figures │ ├── cnf-wg-relation.png │ ├── cnf-wg-relation.pptx │ ├── cnf-wg-scope.png │ └── cnf-wg-scope.pptx ├── glossary.md ├── use-case │ ├── 0001-UC-lifecycle-of-infrastructure-where-CNF-is-running.md │ ├── 0002-UC-bgp-enterprise │ │ ├── README.md │ │ └── bgp-customer-overview.svg │ ├── 0003-UC-stateful-cnf │ │ ├── 5GC_CHF_CCS.svg │ │ ├── CCS_Cluster.svg │ │ └── README.md │ ├── 0004-UC-onboarding-of-CNF-to-the-CSP-platform.md │ ├── NNNN-UC-template.md │ └── README.md ├── user-stories │ ├── air-gapped-environments.md │ ├── stateful-cnf-use-cases-and-user-stories.md │ └── supply-chain-attacks.md └── whitepaper │ ├── Accelerating Cloud Native in Telco.pdf │ ├── Accelerating_Cloud_Native_in_Telco.md │ └── README.md ├── events ├── .gitignore ├── 2023-cloud-native-telco-day-chicago │ ├── README.md │ └── images │ │ ├── .gitignore │ │ ├── Topic 1 - Whiteboard.jpg │ │ ├── Topic 1.png │ │ ├── Topic 2 - Whiteboard.jpg │ │ ├── Topic 2.png │ │ ├── Topic 3 - Whiteboard.jpg │ │ ├── Topic 3.png │ │ ├── Topic 4 - Whiteboard.jpg │ │ └── Topic 4.png ├── LF-Cloud-Native-Networking-Program-Alignment-Workshop.md ├── README.md └── telco-community-gathering-kubecon-eu-20230418.md ├── mlc_config.json ├── tox.ini └── use-case └── 5G_RAN ├── 0001-UC-sync-timing-ran.md └── 0002-UC-multi-interface.md /.github/.wordlist.txt: -------------------------------------------------------------------------------- 1 | ABAC 2 | Abdel 3 | ABMF 4 | Aiden 5 | AKS 6 | AlwaysAllow 7 | alwnPIaVT 8 | Analytics 9 | AMF 10 | Analytics 11 | Anuket 12 | Aopen 13 | AOM 14 | APIs 15 | apiserver 16 | approvers 17 | architected 18 | ArgoCD 19 | ARMO 20 | Ashan 21 | ASIC 22 | aspirational 23 | atis 24 | Atomicity 25 | auth 26 | Autoscaling 27 | aYAG 28 | Automatable 29 | Autoscaling 30 | Badging 31 | balancers 32 | Bartolini 33 | baseband 34 | Bernier 35 | bestpractices 36 | BGP 37 | Billingsley 38 | Bitnami 39 | blockquote 40 | BNR 41 | BoF 42 | booleans 43 | BSO 44 | Calin 45 | CAMARA 46 | canPlayFromShare 47 | carte 48 | Caywood 49 | CBPP 50 | CBPPs 51 | CCS 52 | CEST 53 | cfd 54 | cfymedlz 55 | CGF 56 | cgroup 57 | cgroups 58 | CHF 59 | Chitrabasu 60 | Chunghwa 61 | CIS 62 | Clément 63 | CLIS 64 | cloudified 65 | CloudNative 66 | ClusterRoles 67 | CN 68 | CNCF 69 | CNF 70 | CNF's 71 | CNFs 72 | CNI 73 | CNIs 74 | Cnitch 75 | CNPD 76 | CNTI 77 | CNTi's 78 | CNTT 79 | CNzv 80 | componentName 81 | composability 82 | composable 83 | Configmaps 84 | conformant 85 | constrasted 86 | continueMode 87 | Coronalab 88 | CRDs 89 | CRI 90 | Cristian 91 | Csatari 92 | CSP 93 | CSP's 94 | CSPs 95 | CSV 96 | customizable 97 | CVE 98 | cyberattacks 99 | Datacenter 100 | dataplane 101 | declaratively 102 | deliverables 103 | deployable 104 | Deutsche 105 | dev 106 | DevOps 107 | dimensioned 108 | disaggregated 109 | discoverability 110 | divestments 111 | DNS 112 | Dockerfile 113 | Dockerfiles 114 | Docker's 115 | DT 116 | DTAG 117 | eBPF 118 | eg 119 | Ensarguet 120 | Equinix 121 | ETSI 122 | Europaboulevard 123 | Europaplein 124 | Exc 125 | executables 126 | facto 127 | failover 128 | Falco 129 | FAPI 130 | filesystem 131 | filesystems 132 | FluxCD 133 | forklifted 134 | frontlines 135 | Ganbar 136 | gapped 137 | gapping 138 | Gasparetto 139 | GC 140 | Gelrestraat 141 | Gergely 142 | getters 143 | GH 144 | Gigamon 145 | GitOps 146 | GL 147 | gNB 148 | gNodeB 149 | Gojnic 150 | Goygal 151 | GPP 152 | GSMA 153 | Guillaume 154 | GZ 155 | hackmd 156 | Haiby 157 | Heinonen 158 | Hiller 159 | Hirschberg 160 | HItnWMH 161 | hMCcIuBUON 162 | Hogelandplein 163 | hostPath 164 | hostPaths 165 | hostPorts 166 | HPA 167 | href 168 | Huawei 169 | Hyperscaler 170 | hypervised 171 | IaaS 172 | Ildiko 173 | img 174 | impactful 175 | incentivize 176 | infoq 177 | innoq 178 | INNOQ 179 | InfraCloud 180 | Infrastucture 181 | init 182 | INNOQ 183 | Installability 184 | installable 185 | instantiation 186 | integrations 187 | interweaved 188 | IPC 189 | IoT 190 | IOV 191 | IPC 192 | ISSU 193 | Ivanov 194 | Jambi 195 | jauLcz 196 | Jayakumar 197 | Joshi 198 | Josua 199 | jpg 200 | Kata 201 | Kautz 202 | KEP 203 | Khare 204 | Kk 205 | KhlzmhEJK 206 | Kivlin 207 | KPIs 208 | kube 209 | KubeCon 210 | Kubelets 211 | kubenative 212 | kubernetes 213 | Kundral 214 | LF 215 | LFN 216 | Lifecycle 217 | Liles 218 | linter 219 | linters 220 | linuxfoundation 221 | Liron 222 | Loggable 223 | LTS 224 | Lucina 225 | Lupien 226 | Maaslandstraat 227 | Maciej 228 | Makefile 229 | MANO 230 | markdownlint 231 | MATRIXX 232 | Merz 233 | Michal 234 | microservice 235 | microservices 236 | Miklus 237 | Miroslav 238 | mis 239 | mitigations 240 | mkr 241 | mTLS 242 | Multus 243 | Muthurajan 244 | MWC 245 | namespace 246 | namespaces 247 | nativeness 248 | Nephio 249 | NETCONF 250 | NetScout 251 | Nevicato 252 | nFAPI 253 | NFs 254 | NFV 255 | NFVi 256 | NFVO 257 | NG 258 | NGMN 259 | ngmn 260 | NIC 261 | NICs 262 | Nikolaev 263 | Nikolay 264 | Nirmata 265 | NNNN 266 | notesconstraintscaveats 267 | Nussbaumer 268 | O'Reily 269 | observability 270 | OCI 271 | ol 272 | OMUVGG 273 | onboarding 274 | OpenShift 275 | opentracing 276 | OpenGitOps 277 | OpenID 278 | OpenShift 279 | Opex 280 | orchestrator 281 | O'Reily 282 | OSS 283 | overrepresented 284 | Oyj 285 | Pankaj 286 | PCF 287 | PDCP 288 | Pedersen 289 | Petar 290 | PHY 291 | PID 292 | plugfests 293 | PNF 294 | PNFs 295 | PNG 296 | PodDisruptionBudgets 297 | Podman 298 | Polystar 299 | Postconditions 300 | pre 301 | prem 302 | privs 303 | programmability 304 | PRs 305 | PTP 306 | PV 307 | QoS 308 | Rabi 309 | Rader 310 | RAI 311 | Ranny 312 | RBAC 313 | RCE 314 | reachability 315 | README 316 | readthedocs 317 | RedHat 318 | replicable 319 | RFP 320 | RFPs 321 | RFQ 322 | RFx 323 | RHEL 324 | Riccardo 325 | RLC 326 | roadmap 327 | RoleBinding 328 | Ronan 329 | Roundtable 330 | RRC 331 | RSVPd 332 | runc 333 | runnable 334 | runtime 335 | runtimes 336 | RZ 337 | SaaS 338 | Saelens 339 | Sagar 340 | scalability 341 | scalable 342 | SCF 343 | sched 344 | schemas 345 | SCTP 346 | SDAP 347 | seclists 348 | SELinux 349 | Senevirathne 350 | setsid 351 | Sewera 352 | Sheetal 353 | SIG 354 | signoff 355 | SIGs 356 | Skrocki 357 | SLA 358 | SLAs 359 | SMF 360 | Smitholi 361 | src 362 | SREs 363 | stateful 364 | Stori 365 | Stricko 366 | svg 367 | Swisscom 368 | Sylva 369 | Synk 370 | sysctl 371 | Sysdig 372 | Taki 373 | Tal 374 | Tanzu 375 | Tariq 376 | tbd 377 | TCP 378 | Technik 379 | telco 380 | telcos 381 | telecom 382 | Telecom's 383 | Telekom 384 | TELUS 385 | testbed 386 | testcatalog 387 | testsuite 388 | timeframe 389 | TOC 390 | Toktonaliev 391 | tolerations 392 | Torre 393 | txt 394 | tZEwAF 395 | TzWMrZCS 396 | UC 397 | uCZTgnbglWXPnrlw 398 | UE 399 | UID 400 | uid 401 | uncloud 402 | uncomment 403 | unopinionated 404 | untrusted 405 | UPF 406 | Upgradability 407 | upgradeable 408 | uptime 409 | usernetes 410 | Vancsa 411 | VCS 412 | vCU 413 | virtualize 414 | Virtualized 415 | VMware 416 | VNF 417 | VNFs 418 | Vodafone 419 | VPN 420 | VPNs 421 | VRF 422 | VRFs 423 | Vuk 424 | Vulk 425 | VV 426 | Wandelweg 427 | Webhook 428 | wg 429 | WG's 430 | WGs 431 | whitepaper 432 | Whitepapers 433 | WIP 434 | workgroup 435 | writeup 436 | WSh 437 | wsts 438 | wuI 439 | XBnKpmRI 440 | XDP 441 | XGVela 442 | yaml 443 | Zebetian 444 | Zuid 445 | Zuidelijke 446 | zvXTNtpiGtbA 447 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/best_practice.yaml: -------------------------------------------------------------------------------- 1 | --- 2 | name: Best Practice Proposal 3 | description: Submit a proposal for all or part of a best practice 4 | title: "[Best Practice Proposal]: " 5 | labels: ["best practice proposal"] 6 | body: 7 | - type: markdown 8 | attributes: 9 | value: | 10 | Thanks for taking the time to contribute to the CNF-WG! 11 | - type: textarea 12 | id: summary 13 | attributes: 14 | label: Summary 15 | placeholder: A summary of the best practice - couple of sentences. 16 | validations: 17 | required: false 18 | - type: textarea 19 | id: motivation 20 | attributes: 21 | label: Motivation 22 | placeholder: How does this best practice benefit developers, operators, etc.? 23 | validations: 24 | required: false 25 | - type: textarea 26 | id: goals 27 | attributes: 28 | label: Goals 29 | placeholder: What are the goals of this best practice? 30 | validations: 31 | required: false 32 | - type: textarea 33 | id: non-goals 34 | attributes: 35 | label: Non-Goals 36 | placeholder: What are the non-goals of this best practice? 37 | validations: 38 | required: false 39 | - type: textarea 40 | id: proposal 41 | attributes: 42 | label: Proposal 43 | placeholder: Main proposal detail 44 | validations: 45 | required: false 46 | - type: textarea 47 | id: workload-context 48 | attributes: 49 | label: Workload Context 50 | placeholder: What is the scope of this best practice? All workloads, just a subset? 51 | validations: 52 | required: false 53 | - type: textarea 54 | id: user-stories 55 | attributes: 56 | label: User Stories 57 | placeholder: Optionally, are there any user stories that help justify or explain the best practice? 58 | validations: 59 | required: false 60 | - type: textarea 61 | id: notes 62 | attributes: 63 | label: Notes, Caveats, Constraints 64 | placeholder: Anything of note? 65 | validations: 66 | required: false 67 | - type: textarea 68 | id: references 69 | attributes: 70 | label: References 71 | placeholder: Please provide any references to existing documentation, blogs, etc. relevant to the best practice. 72 | validations: 73 | required: false 74 | -------------------------------------------------------------------------------- /.github/PULL_REQUEST_TEMPLATE/pull_request_template.md: -------------------------------------------------------------------------------- 1 | # Title 2 | 3 | 4 | 5 | ## Description 6 | 7 | 8 | ## Motivation and Context 9 | 10 | 11 | 12 | ## Types of changes 13 | 14 | - [ ] Use Case 15 | - [ ] Best Practice 16 | - [ ] Documentation Update 17 | - [ ] Documentation Fix 18 | -------------------------------------------------------------------------------- /.github/dependabot.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # SPDX-license-identifier: Apache-2.0 3 | ############################################################################## 4 | # Copyright (c) 2021 5 | # All rights reserved. This program and the accompanying materials 6 | # are made available under the terms of the Apache License, Version 2.0 7 | # which accompanies this distribution, and is available at 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | ############################################################################## 10 | 11 | version: 2 12 | updates: 13 | - package-ecosystem: "github-actions" 14 | directory: "/" 15 | schedule: 16 | interval: "daily" 17 | -------------------------------------------------------------------------------- /.github/labeler.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # SPDX-license-identifier: Apache-2.0 3 | ############################################################################## 4 | # Copyright (c) 2021 5 | # All rights reserved. This program and the accompanying materials 6 | # are made available under the terms of the Apache License, Version 2.0 7 | # which accompanies this distribution, and is available at 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | ############################################################################## 10 | 11 | documentation: 12 | - doc/* 13 | - "*.md" 14 | 15 | use-case: 16 | - doc/use-case/* 17 | 18 | best-practice: 19 | - doc/cbpps/* 20 | 21 | tools: 22 | - .github/* 23 | -------------------------------------------------------------------------------- /.github/workflows/linkcheck.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # SPDX-license-identifier: Apache-2.0 3 | ############################################################################## 4 | # Copyright (c) 2021 5 | # All rights reserved. This program and the accompanying materials 6 | # are made available under the terms of the Apache License, Version 2.0 7 | # which accompanies this distribution, and is available at 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | ############################################################################## 10 | 11 | name: Check Markdown links 12 | # yamllint disable-line rule:truthy 13 | on: [push, pull_request] 14 | jobs: 15 | markdown-link-check: 16 | runs-on: ubuntu-latest 17 | steps: 18 | - uses: actions/checkout@v4.1.7 19 | - uses: gaurav-nelson/github-action-markdown-link-check@1.0.15 20 | -------------------------------------------------------------------------------- /.github/workflows/linter.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # SPDX-license-identifier: Apache-2.0 3 | ############################################################################## 4 | # Copyright (c) 2021 5 | # All rights reserved. This program and the accompanying materials 6 | # are made available under the terms of the Apache License, Version 2.0 7 | # which accompanies this distribution, and is available at 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | ############################################################################## 10 | 11 | name: Lint Code Base 12 | # yamllint disable-line rule:truthy 13 | on: [push, pull_request] 14 | jobs: 15 | build: 16 | runs-on: ubuntu-latest 17 | steps: 18 | - uses: actions/checkout@v4.1.7 19 | - uses: github/super-linter@v7 20 | env: 21 | GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} 22 | LINTER_RULES_PATH: / 23 | -------------------------------------------------------------------------------- /.github/workflows/spell.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # SPDX-license-identifier: Apache-2.0 3 | ############################################################################## 4 | # Copyright (c) 2021 5 | # All rights reserved. This program and the accompanying materials 6 | # are made available under the terms of the Apache License, Version 2.0 7 | # which accompanies this distribution, and is available at 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | ############################################################################## 10 | 11 | name: Run misspell 12 | # yamllint disable-line rule:truthy 13 | on: [push, pull_request] 14 | 15 | jobs: 16 | check-reviewdog: 17 | runs-on: ubuntu-latest 18 | steps: 19 | - uses: actions/checkout@v4.1.7 20 | - uses: reviewdog/action-misspell@v1.23.0 21 | with: 22 | github_token: ${{ secrets.github_token }} 23 | check-spellcheck: 24 | runs-on: ubuntu-latest 25 | steps: 26 | - uses: actions/checkout@v4.1.7 27 | - uses: igsekor/pyspelling-any@v1.0.4 28 | name: Spellcheck 29 | -------------------------------------------------------------------------------- /.github/workflows/triage.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # SPDX-license-identifier: Apache-2.0 3 | ############################################################################## 4 | # Copyright (c) 2021 5 | # All rights reserved. This program and the accompanying materials 6 | # are made available under the terms of the Apache License, Version 2.0 7 | # which accompanies this distribution, and is available at 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | ############################################################################## 10 | 11 | name: Triage 12 | # yamllint disable-line rule:truthy 13 | on: 14 | - pull_request_target 15 | 16 | jobs: 17 | assign-label: 18 | runs-on: ubuntu-latest 19 | steps: 20 | - uses: actions/labeler@v5.0.0 21 | with: 22 | repo-token: "${{ secrets.GITHUB_TOKEN }}" 23 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Mac annoyances 2 | .DS_Store 3 | # Emacs backup files and in-edit files 4 | *~ 5 | \#*\# 6 | # Vi(m) in-edit files 7 | .\#* 8 | super-linter.log 9 | # IDEA files 10 | .idea/ 11 | dictionary.dic 12 | # VS Code files 13 | .vscode/* 14 | -------------------------------------------------------------------------------- /.markdown-lint.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # SPDX-license-identifier: Apache-2.0 3 | ############################################################################## 4 | # Copyright (c) 2021 5 | # All rights reserved. This program and the accompanying materials 6 | # are made available under the terms of the Apache License, Version 2.0 7 | # which accompanies this distribution, and is available at 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | ############################################################################## 10 | 11 | line-length: false 12 | -------------------------------------------------------------------------------- /.spellcheck.yml: -------------------------------------------------------------------------------- 1 | --- 2 | # SPDX-license-identifier: Apache-2.0 3 | ############################################################################## 4 | # Copyright (c) 2021 5 | # All rights reserved. This program and the accompanying materials 6 | # are made available under the terms of the Apache License, Version 2.0 7 | # which accompanies this distribution, and is available at 8 | # http://www.apache.org/licenses/LICENSE-2.0 9 | ############################################################################## 10 | 11 | matrix: 12 | - name: markdown 13 | dictionary: 14 | wordlists: 15 | - .github/.wordlist.txt 16 | pipeline: 17 | - pyspelling.filters.markdown: 18 | - pyspelling.filters.url: 19 | sources: 20 | - '**/*.md' 21 | aspell: 22 | lang: en 23 | ignore-case: true 24 | -------------------------------------------------------------------------------- /.textlintrc: -------------------------------------------------------------------------------- 1 | 2 | { 3 | "rules": { 4 | "terminology": { 5 | "exclude": [ 6 | "Node(?:js)?", 7 | "OK" 8 | ] 9 | } 10 | } 11 | } 12 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing to the CNTI Best Practices Focus Area 2 | 3 | Welcome to the Cloud Native Telecom Initiative (CNTi) Best Practices Focus Area. This is an open, public group welcoming anyone who would like to help identify cloud native best practices for networking applications. We're glad you're here! 4 | 5 | To learn about this focus area and our different collaboration tools, [read the wiki](https://wiki.lfnetworking.org/display/LN/Best+Practices). 6 | 7 | ## What to Contribute 8 | 9 | We welcome many different types of contributions including: 10 | 11 | - General improvements to documentation 12 | - [Use cases](doc/use-case/) and user stories (these are used to provide context for and select best practices) 13 | - [See the template for more details](doc/use-case/NNNN-UC-template.md) 14 | - Definitions 15 | - [Actors and Roles](https://github.com/lfn-cnti/bestpractices/discussions/30) 16 | - [Glossary](doc/glossary.md) 17 | - [CNTi Best Practices](doc/best_cnf_dev.md) 18 | - [Process to publish a CNTi Best Practice](doc/cbpps/cnti_best_practice_process.md) 19 | - [See the template for more details](doc/cbpps/NNNN-cbpp-template.md) 20 | - Gap Analysis 21 | 22 | ### Code Contributions 23 | 24 | The CNTi Best Practices Focus Area doesn't develop any code, but its work feeds into many groups that do develop code both upstream and downstream. If you want to take the work of the CNTi Best Practices and put it into code, check out: 25 | 26 | - [CNTi Test Catalog](https://github.com/lfn-cnti/testsuite) is the place for test cases used in best practices 27 | - [Kubernetes Network Plumbing](https://github.com/k8snetworkplumbingwg) implements a common standard addressing how one may attach multiple networks to pods in Kubernetes 28 | - [Kubernetes SIG Network](https://github.com/kubernetes/community/tree/master/sig-network) is responsible for the components, interfaces, and APIs which expose networking capabilities to Kubernetes users and workloads 29 | 30 | ## How to Contribute 31 | 32 | ### Come to Meetings 33 | 34 | Absolutely everyone is welcome to come to any of our meetings. You never need an invite to join us. In fact, we want you to join us, even if you don’t have anything you feel like you want to contribute. Just being there is a good start! 35 | 36 | Over time, we hope that you feel comfortable voicing your opinions, ideas and providing feedback with the community. Sharing your own ideas, and experiences can help in ways you might not initially realize. 37 | 38 | ### Join a GitHub Discussion 39 | 40 | - Go to the [GitHub Discussion board](https://github.com/lfn-cnti/bestpractices/discussions/) 41 | - Participate in existing discussions 42 | - eg. [Defining actors and audiences](https://github.com/lfn-cnti/bestpractices/discussions/30) 43 | - Add new discussion topics 44 | - Reference Issues, PRs, and existing content from the [Best Practices repository](https://github.com/lfn-cnti/bestpractices) 45 | 46 | ### Find an Issue 47 | 48 | Review, contribute to, create new [GitHub issues](https://github.com/lfn-cnti/bestpractices/issues). 49 | 50 | We have good first issues for new contributors and help wanted issues suitable for any contributor. [good first issue](https://github.com/lfn-cnti/bestpractices/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) has extra information to 51 | help you make your first contribution. [help wanted](https://github.com/lfn-cnti/bestpractices/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22) are issues suitable for someone who isn't a core maintainer and is good to move onto after your first pull request. 52 | 53 | Sometimes there won’t be any issues with these labels. That’s ok! There is likely still something for you to work on. If you want to contribute but you don’t know where to start or can't find a suitable issue, you can reach out to one of the [TSC members](https://wiki.lfnetworking.org/display/LN/CNTi+Governance). 54 | 55 | Once you see an issue that you'd like to work on, please post a comment saying that you want to work on it. Something like "I want to work on this" is fine. 56 | 57 | ### Submit Pull Requests 58 | 59 | Pull Requests are always welcome, even for small fixes like typos. Check [What to Contribute](#what-to-contribute) above for more information about what kind of PRs we are looking for. 60 | 61 | #### Pull Request Checklist 62 | 63 | When you submit your pull request, or you push new commits to it, our automated systems will run some checks on your new contribution. We require that your pull request passes these checks, but we also have more criteria than just that before we can accept and merge it. We recommend that you check the following things locally before you submit your change: 64 | 65 | - We use GitHub Actions to test all pull requests. We prefer that all tests succeed on a pull request before it is merged. This repository also contains a *Makefile* for running linting tasks locally. Executing `make lint` is another way to check your work before GitHub actions. 66 | 67 | ### Review/comment on Pull Requests 68 | 69 | Reviews and comments on open [Pull Requests](https://github.com/lfn-cnti/bestpractices/pulls) are always appreciated. 70 | A minimum of 3 community approvals, from at least 2 different companies, are needed to merge PRs so your review is greatly appreciated. 71 | 72 | 73 | One of [our values](charter.md#values) is "Commit first, improve later". Once a commit has been made you can [make suggestions for changes directly in the PR](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/commenting-on-a-pull-request#adding-line-comments-to-a-pull-request) to help us move forward faster together. 74 | 75 | 76 | Where the comment is better expressed as a proposed change, the change can be made directly either by editing the file (right hand "..." -> "edit file" for larger changes, or the "document-with-+-" icon above the comment window for smaller ones). The author can accept these changes directly, which is much easier for them than writing a change themselves. 77 | 78 | #### Steps for creating a new PR 79 | 80 | - Once you have made your change or added new content 81 | - Ensure you are up to date with the `main` branch of the cnf-wg repository 82 | - Open a new [Pull Request](https://github.com/lfn-cnti/bestpractices/pulls) 83 | - Choose at least 3 reviewers OR choose a [TSC member](https://wiki.lfnetworking.org/display/LN/CNTi+Governance) who will find reviewers for you 84 | 85 | #### Steps to accept a PR 86 | 87 | 1. Reviewers will review the PR and give their feedback: approve, request change, comment w/o approval 88 | 1. After required number of approvals from reviewers is reached the PR may be merged 89 | 90 | Anyone with merge access can merge the PR after the item has been approved. 91 | 92 | ### Acceptance criteria for Contributions 93 | 94 | Reviewers - Any community member 95 | 96 | - Everyone in the community is able to and encouraged to do PR reviews. 97 | - A minimum of 3 community approvals, from at least 2 different companies, are required before a PR can be merged. 98 | 99 | ## Thank you 100 | 101 | Thank you for your participation. We appreciate your help and look forward to collaborating with you! 102 | -------------------------------------------------------------------------------- /CONTRIBUTORS.md: -------------------------------------------------------------------------------- 1 | # CNTi Best Practices Contributors 2 | 3 | 4 | ## Maintainer 5 | | Name | GitHub ID | Company Name | 6 | | --------------- | --------- | ----------- | 7 | | LJ Illuzzi | [lilluzzi](https://github.com/lilluzzi) | [LF Networking](https://lfnetworking.org/) | 8 | | Lucina Stricko | [lixuna](https://github.com/lixuna) | [Vulk Coop](vulk.coop) | 9 | | Taylor Carpenter | [taylor](https://github.com/taylor) | [Vulk Coop](vulk.coop) | 10 | 11 | 12 | ## Contributor 13 | | Name | GitHub ID | Affiliation | 14 | | --------------- | --------- | ----------- | 15 | | Martin Matyáš | [martin-mat](https://github.com/martin-mat) | [Tietoevry](https://www.tietoevry.com/) | 16 | | Olivier Smith | [smitholi67](https://github.com/smitholi67) | [MATRIXX Software](https://www.matrixx.com/) | 17 | 18 | ### Emeritus Maintainer 19 | | Name | GitHub ID | Affiliation | 20 | | --------------- | --------- | ----------- | 21 | -------------------------------------------------------------------------------- /LICENSE.md: -------------------------------------------------------------------------------- 1 | # License 2 | 3 | Except as otherwise noted, the content of this repository is licensed under the [Creative Commons Attribution 4.0 License](https://creativecommons.org/licenses/by/4.0/) ([local copy](LICENSES/CC-BY-4.0.txt)), and any code is licensed under the [Apache 2.0 License](http://www.apache.org/licenses/LICENSE-2.0.html) ([local copy](LICENSES/APACHE-2.txt)). 4 | -------------------------------------------------------------------------------- /LICENSES/APACHE-2.txt: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /LICENSES/CC-BY-4.0.txt: -------------------------------------------------------------------------------- 1 | Creative Commons Attribution 4.0 International Creative Commons Corporation 2 | ("Creative Commons") is not a law firm and does not provide legal services 3 | or legal advice. Distribution of Creative Commons public licenses does not 4 | create a lawyer-client or other relationship. Creative Commons makes its licenses 5 | and related information available on an "as-is" basis. Creative Commons gives 6 | no warranties regarding its licenses, any material licensed under their terms 7 | and conditions, or any related information. Creative Commons disclaims all 8 | liability for damages resulting from their use to the fullest extent possible. 9 | 10 | Using Creative Commons Public Licenses 11 | 12 | Creative Commons public licenses provide a standard set of terms and conditions 13 | that creators and other rights holders may use to share original works of 14 | authorship and other material subject to copyright and certain other rights 15 | specified in the public license below. The following considerations are for 16 | informational purposes only, are not exhaustive, and do not form part of our 17 | licenses. 18 | 19 | Considerations for licensors: Our public licenses are intended for use by 20 | those authorized to give the public permission to use material in ways otherwise 21 | restricted by copyright and certain other rights. Our licenses are irrevocable. 22 | Licensors should read and understand the terms and conditions of the license 23 | they choose before applying it. Licensors should also secure all rights necessary 24 | before applying our licenses so that the public can reuse the material as 25 | expected. Licensors should clearly mark any material not subject to the license. 26 | This includes other CC-licensed material, or material used under an exception 27 | or limitation to copyright. More considerations for licensors : wiki.creativecommons.org/Considerations_for_licensors 28 | 29 | Considerations for the public: By using one of our public licenses, a licensor 30 | grants the public permission to use the licensed material under specified 31 | terms and conditions. If the licensor's permission is not necessary for any 32 | reason–for example, because of any applicable exception or limitation to copyright–then 33 | that use is not regulated by the license. Our licenses grant only permissions 34 | under copyright and certain other rights that a licensor has authority to 35 | grant. Use of the licensed material may still be restricted for other reasons, 36 | including because others have copyright or other rights in the material. A 37 | licensor may make special requests, such as asking that all changes be marked 38 | or described. Although not required by our licenses, you are encouraged to 39 | respect those requests where reasonable. More considerations for the public 40 | : wiki.creativecommons.org/Considerations_for_licensees Creative Commons Attribution 41 | 4.0 International Public License 42 | 43 | By exercising the Licensed Rights (defined below), You accept and agree to 44 | be bound by the terms and conditions of this Creative Commons Attribution 45 | 4.0 International Public License ("Public License"). To the extent this Public 46 | License may be interpreted as a contract, You are granted the Licensed Rights 47 | in consideration of Your acceptance of these terms and conditions, and the 48 | Licensor grants You such rights in consideration of benefits the Licensor 49 | receives from making the Licensed Material available under these terms and 50 | conditions. 51 | 52 | Section 1 – Definitions. 53 | 54 | a. Adapted Material means material subject to Copyright and Similar Rights 55 | that is derived from or based upon the Licensed Material and in which the 56 | Licensed Material is translated, altered, arranged, transformed, or otherwise 57 | modified in a manner requiring permission under the Copyright and Similar 58 | Rights held by the Licensor. For purposes of this Public License, where the 59 | Licensed Material is a musical work, performance, or sound recording, Adapted 60 | Material is always produced where the Licensed Material is synched in timed 61 | relation with a moving image. 62 | 63 | b. Adapter's License means the license You apply to Your Copyright and Similar 64 | Rights in Your contributions to Adapted Material in accordance with the terms 65 | and conditions of this Public License. 66 | 67 | c. Copyright and Similar Rights means copyright and/or similar rights closely 68 | related to copyright including, without limitation, performance, broadcast, 69 | sound recording, and Sui Generis Database Rights, without regard to how the 70 | rights are labeled or categorized. For purposes of this Public License, the 71 | rights specified in Section 2(b)(1)-(2) are not Copyright and Similar Rights. 72 | 73 | d. Effective Technological Measures means those measures that, in the absence 74 | of proper authority, may not be circumvented under laws fulfilling obligations 75 | under Article 11 of the WIPO Copyright Treaty adopted on December 20, 1996, 76 | and/or similar international agreements. 77 | 78 | e. Exceptions and Limitations means fair use, fair dealing, and/or any other 79 | exception or limitation to Copyright and Similar Rights that applies to Your 80 | use of the Licensed Material. 81 | 82 | f. Licensed Material means the artistic or literary work, database, or other 83 | material to which the Licensor applied this Public License. 84 | 85 | g. Licensed Rights means the rights granted to You subject to the terms and 86 | conditions of this Public License, which are limited to all Copyright and 87 | Similar Rights that apply to Your use of the Licensed Material and that the 88 | Licensor has authority to license. 89 | 90 | h. Licensor means the individual(s) or entity(ies) granting rights under this 91 | Public License. 92 | 93 | i. Share means to provide material to the public by any means or process that 94 | requires permission under the Licensed Rights, such as reproduction, public 95 | display, public performance, distribution, dissemination, communication, or 96 | importation, and to make material available to the public including in ways 97 | that members of the public may access the material from a place and at a time 98 | individually chosen by them. 99 | 100 | j. Sui Generis Database Rights means rights other than copyright resulting 101 | from Directive 96/9/EC of the European Parliament and of the Council of 11 102 | March 1996 on the legal protection of databases, as amended and/or succeeded, 103 | as well as other essentially equivalent rights anywhere in the world. 104 | 105 | k. You means the individual or entity exercising the Licensed Rights under 106 | this Public License. Your has a corresponding meaning. 107 | 108 | Section 2 – Scope. 109 | 110 | a. License grant. 111 | 112 | 1. Subject to the terms and conditions of this Public License, the Licensor 113 | hereby grants You a worldwide, royalty-free, non-sublicensable, non-exclusive, 114 | irrevocable license to exercise the Licensed Rights in the Licensed Material 115 | to: 116 | 117 | A. reproduce and Share the Licensed Material, in whole or in part; and 118 | 119 | B. produce, reproduce, and Share Adapted Material. 120 | 121 | 2. Exceptions and Limitations. For the avoidance of doubt, where Exceptions 122 | and Limitations apply to Your use, this Public License does not apply, and 123 | You do not need to comply with its terms and conditions. 124 | 125 | 3. Term. The term of this Public License is specified in Section 6(a). 126 | 127 | 4. Media and formats; technical modifications allowed. The Licensor authorizes 128 | You to exercise the Licensed Rights in all media and formats whether now known 129 | or hereafter created, and to make technical modifications necessary to do 130 | so. The Licensor waives and/or agrees not to assert any right or authority 131 | to forbid You from making technical modifications necessary to exercise the 132 | Licensed Rights, including technical modifications necessary to circumvent 133 | Effective Technological Measures. For purposes of this Public License, simply 134 | making modifications authorized by this Section 2(a)(4) never produces Adapted 135 | Material. 136 | 137 | 5. Downstream recipients. 138 | 139 | A. Offer from the Licensor – Licensed Material. Every recipient of the Licensed 140 | Material automatically receives an offer from the Licensor to exercise the 141 | Licensed Rights under the terms and conditions of this Public License. 142 | 143 | B. No downstream restrictions. You may not offer or impose any additional 144 | or different terms or conditions on, or apply any Effective Technological 145 | Measures to, the Licensed Material if doing so restricts exercise of the Licensed 146 | Rights by any recipient of the Licensed Material. 147 | 148 | 6. No endorsement. Nothing in this Public License constitutes or may be construed 149 | as permission to assert or imply that You are, or that Your use of the Licensed 150 | Material is, connected with, or sponsored, endorsed, or granted official status 151 | by, the Licensor or others designated to receive attribution as provided in 152 | Section 3(a)(1)(A)(i). 153 | 154 | b. Other rights. 155 | 156 | 1. Moral rights, such as the right of integrity, are not licensed under this 157 | Public License, nor are publicity, privacy, and/or other similar personality 158 | rights; however, to the extent possible, the Licensor waives and/or agrees 159 | not to assert any such rights held by the Licensor to the limited extent necessary 160 | to allow You to exercise the Licensed Rights, but not otherwise. 161 | 162 | 2. Patent and trademark rights are not licensed under this Public License. 163 | 164 | 3. To the extent possible, the Licensor waives any right to collect royalties 165 | from You for the exercise of the Licensed Rights, whether directly or through 166 | a collecting society under any voluntary or waivable statutory or compulsory 167 | licensing scheme. In all other cases the Licensor expressly reserves any right 168 | to collect such royalties. 169 | 170 | Section 3 – License Conditions. 171 | 172 | Your exercise of the Licensed Rights is expressly made subject to the following 173 | conditions. 174 | 175 | a. Attribution. 176 | 177 | 1. If You Share the Licensed Material (including in modified form), You must: 178 | 179 | A. retain the following if it is supplied by the Licensor with the Licensed 180 | Material: 181 | 182 | i. identification of the creator(s) of the Licensed Material and any others 183 | designated to receive attribution, in any reasonable manner requested by the 184 | Licensor (including by pseudonym if designated); 185 | 186 | ii. a copyright notice; 187 | 188 | iii. a notice that refers to this Public License; 189 | 190 | iv. a notice that refers to the disclaimer of warranties; 191 | 192 | v. a URI or hyperlink to the Licensed Material to the extent reasonably practicable; 193 | 194 | B. indicate if You modified the Licensed Material and retain an indication 195 | of any previous modifications; and 196 | 197 | C. indicate the Licensed Material is licensed under this Public License, and 198 | include the text of, or the URI or hyperlink to, this Public License. 199 | 200 | 2. You may satisfy the conditions in Section 3(a)(1) in any reasonable manner 201 | based on the medium, means, and context in which You Share the Licensed Material. 202 | For example, it may be reasonable to satisfy the conditions by providing a 203 | URI or hyperlink to a resource that includes the required information. 204 | 205 | 3. If requested by the Licensor, You must remove any of the information required 206 | by Section 3(a)(1)(A) to the extent reasonably practicable. 207 | 208 | 4. If You Share Adapted Material You produce, the Adapter's License You apply 209 | must not prevent recipients of the Adapted Material from complying with this 210 | Public License. 211 | 212 | Section 4 – Sui Generis Database Rights. 213 | 214 | Where the Licensed Rights include Sui Generis Database Rights that apply to 215 | Your use of the Licensed Material: 216 | 217 | a. for the avoidance of doubt, Section 2(a)(1) grants You the right to extract, 218 | reuse, reproduce, and Share all or a substantial portion of the contents of 219 | the database; 220 | 221 | b. if You include all or a substantial portion of the database contents in 222 | a database in which You have Sui Generis Database Rights, then the database 223 | in which You have Sui Generis Database Rights (but not its individual contents) 224 | is Adapted Material; and 225 | 226 | c. You must comply with the conditions in Section 3(a) if You Share all or 227 | a substantial portion of the contents of the database. 228 | 229 | For the avoidance of doubt, this Section 4 supplements and does not replace 230 | Your obligations under this Public License where the Licensed Rights include 231 | other Copyright and Similar Rights. 232 | 233 | Section 5 – Disclaimer of Warranties and Limitation of Liability. 234 | 235 | a. Unless otherwise separately undertaken by the Licensor, to the extent possible, 236 | the Licensor offers the Licensed Material as-is and as-available, and makes 237 | no representations or warranties of any kind concerning the Licensed Material, 238 | whether express, implied, statutory, or other. This includes, without limitation, 239 | warranties of title, merchantability, fitness for a particular purpose, non-infringement, 240 | absence of latent or other defects, accuracy, or the presence or absence of 241 | errors, whether or not known or discoverable. Where disclaimers of warranties 242 | are not allowed in full or in part, this disclaimer may not apply to You. 243 | 244 | b. To the extent possible, in no event will the Licensor be liable to You 245 | on any legal theory (including, without limitation, negligence) or otherwise 246 | for any direct, special, indirect, incidental, consequential, punitive, exemplary, 247 | or other losses, costs, expenses, or damages arising out of this Public License 248 | or use of the Licensed Material, even if the Licensor has been advised of 249 | the possibility of such losses, costs, expenses, or damages. Where a limitation 250 | of liability is not allowed in full or in part, this limitation may not apply 251 | to You. 252 | 253 | c. The disclaimer of warranties and limitation of liability provided above 254 | shall be interpreted in a manner that, to the extent possible, most closely 255 | approximates an absolute disclaimer and waiver of all liability. 256 | 257 | Section 6 – Term and Termination. 258 | 259 | a. This Public License applies for the term of the Copyright and Similar Rights 260 | licensed here. However, if You fail to comply with this Public License, then 261 | Your rights under this Public License terminate automatically. 262 | 263 | b. Where Your right to use the Licensed Material has terminated under Section 264 | 6(a), it reinstates: 265 | 266 | 1. automatically as of the date the violation is cured, provided it is cured 267 | within 30 days of Your discovery of the violation; or 268 | 269 | 2. upon express reinstatement by the Licensor. 270 | 271 | c. For the avoidance of doubt, this Section 6(b) does not affect any right 272 | the Licensor may have to seek remedies for Your violations of this Public 273 | License. 274 | 275 | d. For the avoidance of doubt, the Licensor may also offer the Licensed Material 276 | under separate terms or conditions or stop distributing the Licensed Material 277 | at any time; however, doing so will not terminate this Public License. 278 | 279 | e. Sections 1, 5, 6, 7, and 8 survive termination of this Public License. 280 | 281 | Section 7 – Other Terms and Conditions. 282 | 283 | a. The Licensor shall not be bound by any additional or different terms or 284 | conditions communicated by You unless expressly agreed. 285 | 286 | b. Any arrangements, understandings, or agreements regarding the Licensed 287 | Material not stated herein are separate from and independent of the terms 288 | and conditions of this Public License. 289 | 290 | Section 8 – Interpretation. 291 | 292 | a. For the avoidance of doubt, this Public License does not, and shall not 293 | be interpreted to, reduce, limit, restrict, or impose conditions on any use 294 | of the Licensed Material that could lawfully be made without permission under 295 | this Public License. 296 | 297 | b. To the extent possible, if any provision of this Public License is deemed 298 | unenforceable, it shall be automatically reformed to the minimum extent necessary 299 | to make it enforceable. If the provision cannot be reformed, it shall be severed 300 | from this Public License without affecting the enforceability of the remaining 301 | terms and conditions. 302 | 303 | c. No term or condition of this Public License will be waived and no failure 304 | to comply consented to unless expressly agreed to by the Licensor. 305 | 306 | d. Nothing in this Public License constitutes or may be interpreted as a limitation 307 | upon, or waiver of, any privileges and immunities that apply to the Licensor 308 | or You, including from the legal processes of any jurisdiction or authority. 309 | 310 | Creative Commons is not a party to its public licenses. Notwithstanding, Creative 311 | Commons may elect to apply one of its public licenses to material it publishes 312 | and in those instances will be considered the "Licensor." The text of the 313 | Creative Commons public licenses is dedicated to the public domain under the 314 | CC0 Public Domain Dedication. Except for the limited purpose of indicating 315 | that material is shared under a Creative Commons public license or as otherwise 316 | permitted by the Creative Commons policies published at creativecommons.org/policies, 317 | Creative Commons does not authorize the use of the trademark "Creative Commons" 318 | or any other trademark or logo of Creative Commons without its prior written 319 | consent including, without limitation, in connection with any unauthorized 320 | modifications to any of its public licenses or any other arrangements, understandings, 321 | or agreements concerning use of licensed material. For the avoidance of doubt, 322 | this paragraph does not form part of the public licenses. 323 | 324 | Creative Commons may be contacted at creativecommons.org. 325 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | # SPDX-license-identifier: Apache-2.0 2 | ############################################################################## 3 | # Copyright (c) 2021 4 | # All rights reserved. This program and the accompanying materials 5 | # are made available under the terms of the Apache License, Version 2.0 6 | # which accompanies this distribution, and is available at 7 | # http://www.apache.org/licenses/LICENSE-2.0 8 | ############################################################################## 9 | 10 | DOCKER_CMD ?= $(shell which docker 2> /dev/null || which podman 2> /dev/null || echo docker) 11 | 12 | .PHONY: lint 13 | lint: 14 | sudo -E $(DOCKER_CMD) run --rm -v $$(pwd):/tmp/lint \ 15 | -e RUN_LOCAL=true \ 16 | -e LINTER_RULES_PATH=/ \ 17 | github/super-linter 18 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Cloud Native Telecom Initiative (CNTi) Best Practices 2 | 3 | The CNTi Best Practices focus area operates under the aegis of LFN. The scope of this focus area is to define cloud native networking best practices. We collaborate with the CNTi [Test Suite](https://lf-networking.atlassian.net/wiki/x/bgDGBw) and CNTi [Certification](https://lf-networking.atlassian.net/wiki/x/yQDGBw) focus areas who work on the implementation and mechanics of the test catalog and the definition of cloud native certifications. 4 | 5 | The [CNTi Test Suite](https://github.com/lfn-cnti/testsuite) will support testing a set of these best practices to allow developers and network operators to evaluate how well a network application follows cloud native principles and best practices. Proposals which have been adopted by the CNTi Best Practices focus area are listed in the [Best Practice Proposal](doc/cbpps/) folder. 6 | 7 | - Key areas of focus 8 | - Works with community members, projects, infra providers, CNF vendors, and end users to identify pressing cloud-native networking challenges. 9 | - Develops vendor-neutral [Best Practices for CNF Developers](doc/best_cnf_dev.md) for cloud-native workloads and other networking-related applications that want to leverage cloud-native principles. 10 | - Proactively engages with LF Networking projects to identify challenges or pain points where best practices (community-agreed approaches) will add value. 11 | - Publishes community-agreed best practices. 12 | 13 | 14 | - Not in scope for this focus area 15 | - Identifying or implementing tests for the testing catalog. 16 | - Identifying or defining cloud-native certifications. 17 | 18 | ## Governance 19 | 20 | The [CNTi Charter](https://github.com/lfn-cnti/cnti/blob/c93c0172376ac5ccb1e50f3f82132d1478ee2ef2/CNTi%20Technical%20Charter%205-31-2024.pdf) further outlines the scope of our group activities as well as intended deliverables. 21 | 22 | ## Get Involved 23 | 24 | Learn more on how to get involved by visiting the [CNTi Best Practices Wiki](https://lf-networking.atlassian.net/wiki/x/PgDGBw) and reading about ways to [contribute](https://github.com/lfn-cnti/cnti/blob/main/CONTRIBUTING.md) to CNTi. 25 | 26 | ## CNTi and Industry Events 27 | 28 | Learn more about [events relevant for the CNTi Community](https://lf-networking.atlassian.net/wiki/x/yQHGBw) 29 | -------------------------------------------------------------------------------- /doc/README.md: -------------------------------------------------------------------------------- 1 | # Cloud Native Telecom Initiative (CNTI) Best Practices 2 | 3 | ## Table of Contents 4 | 5 | * [Introduction](#introduction) 6 | * [Scope](#scope) 7 | * [Best Practices for CNF Developers](#best-practices-for-cnf-developers) 8 | 9 | ## Introduction 10 | 11 | The goal of the Cloud Native Telecom Initiative (CNTI) Best Practices focus area is to define a set of practices to evaluate the extent to which a CNF is [cloud native](https://github.com/cloud-native-principles/cloud-native-principles). CNFs can be evaluated against those best practices through test cases that are developed by other open source projects like the [CNTI Test Catalog](https://github.com/lfn-cnti/testsuite). 12 | 13 | The CNTI Best Practices focus area will rely on other industry initiatives (such as [NGMN](https://www.ngmn.org/highlight/ngmn-publishes-cloud-native-manifesto.html) to come up with a definition of "cloud native" in relation to the telecom industry. 14 | 15 | As the telco industry transitions to cloud native ways of thinking, working, and operating, these best practices will serve as the industry gold standard of what cloud native means in practice. They can be used for a variety of purposes including guiding CNF design and development, in RFPs, to guide daily operations, and to do internal testing. 16 | 17 | Cloud native is not a set point. The industry, technology, and best practices will continually evolve with each other. Thus, being cloud native should not be seen as pass/fail, but rather a spectrum. While it may be impossible for anything to be completely cloud native, implementing even one best practice can help improve business outcomes for everyone involved. Implementing more of them can have an even greater impact. However, depending upon the type of CNF, different best practices may have a greater or smaller impact on the ultimate business outcome. 18 | 19 | These best practices and recommendations shall serve as generic contract between CNFs and cloud native infrastructure / platforms. If both, a CNF and a platform follow these recommendations it should be assured that they will run nicely together out-of-box. In a sense fulfilling this contract represents implicit (pre) certification that the combination can be used in the production. 20 | 21 | Best practices are introduced, modified, and removed to keep up with industry trends and technological advances via a process defined in [here](cbpps). 22 | 23 | **TO BE REPLACED: Figure 1-1** below illustrate CNF WG relation to other communities and how best practices defined by CNF WG are consumed. 24 | 25 | ![CNF WG Relation with other industry communities](./figures/cnf-wg-relation.png) 26 | 27 | ## Scope 28 | 29 | As demonstrated by **Figure 1-2** below, When it comes to Cloud Native Network Functions (CNFs), there are many areas that can be focused on in order to determine different best practices for different Actors. They are: 30 | 31 | * **Consumption of CNFs**: Set of best practices that helps Customers/Consumers to consume and use CNFs in a cloud native way. 32 | * **Development of CNFs**: Set of best practices that helps CNF Developers to build their CNFs in a cloud native way. 33 | * **Operation of CNFs**: Set of best practices that helps CNF Operators with the lifecycle management of CNFs in a cloud native way (for example, deployment, configuration management, upgrade, etc). 34 | * **Building of CNF Infrastructure**: Set of best practices that helps CNF Infrastructure developers to build and develop CNF Infrastucture in a cloud native way. 35 | * **Operation of CNF Infrastructure**: Set of best practices that helps CNF Infra Operators to deploy and manage CNF infrastructure in cloud native way. 36 | 37 | ![Current Scope of CNF WG](./figures/cnf-wg-scope.png) 38 | 39 | The current goal of CNTI Best Practices Focus Area is to define the best practices for the development and deployment of CNFs. Other areas are left for future focus. 40 | 41 | ## Best Practices for CNF Developers 42 | 43 | * [Configuration and Lifecycle](./best_cnf_dev.md#1.0-configuration-and-lifecycle) 44 | * [Installable and Upgradeable](./best_cnf_dev.md#2.0-installable-and-upgradeable) 45 | * [Hardware support](./best_cnf_dev.md#3.0-hardware-support) 46 | * [Microservice](./best_cnf_dev.md#4.0-microservice) 47 | * [Compatibility](./best_cnf_dev.md#5.0-compatibility) 48 | * [State](./best_cnf_dev.md#6.0-state) 49 | * [Security](./best_cnf_dev.md#7.0-security) 50 | * [Scaling](./best_cnf_dev.md#8.0-scaling) 51 | * [Observability](./best_cnf_dev.md#9.0-observability) 52 | -------------------------------------------------------------------------------- /doc/best_cnf_dev.md: -------------------------------------------------------------------------------- 1 | # Best Practices for CNF Developers 2 | 3 | ## Table of Contents 4 | 5 | * [1.0 Compatibility, Installability & Upgradability](#10-compatibility-installability--upgradability) 6 | * [2.0 Configuration](#20-configuration) 7 | * [3.0 Microservices](#30-microservices) 8 | * [4.0 State](#40-state) 9 | * [5.0 Security](#50-security) 10 | * [6.0 Observability and Diagnostics](#60-observability-and-diagnostics) 11 | * [7.0 Reliability, Resilience & Availability](#70-reliability-resilience--availability) 12 | 13 | ### 1.0 Compatibility, Installability & Upgradability 14 | 15 | Best practices affecting the basic management of CNF software - the acceptance of CNFs from a development team, the installation onto the network, the creation of instances in clouds within the network, version management of running software and the compatibility of CNFs on many cloud platforms and with other CNFs. 16 | 17 | ### 2.0 Configuration 18 | 19 | Working with running CNFs: common patterns for setting and changing the behaviour of network functions. 20 | 21 | ### 3.0 Microservices 22 | 23 | The best way to use microservice-based design patterns to deliver CNF functionality. 24 | 25 | #### CBPP-0005: A CNF's containers should handle a single concern and service (normally mapping to one process type) per container 26 | 27 | **Description:** 28 | A CNF with multiple concerns should split services (or process types) for each of its concerns into separate containers. Service dependencies should be handled between containers through well-defined interfaces. 29 | 30 | Pod specs for the CNF should provide scaling and monitoring information for each service running in different containers. 31 | 32 | **Reference:** [CBPP-0005](cbpps/0005-single-concern-per-container.md) 33 | 34 | ### 4.0 State 35 | 36 | Management of short-lived and long lived state, including state associated with network flows, configuration data, subscriber activity data and other data according to the varying requirements of resiliency, rate of change, access rate and persistence. 37 | 38 | ### 5.0 Security 39 | 40 | How to ensure that components are protected against security issues, including security advisories on software components, defence against attacks, and defence in depth. 41 | 42 | #### CBPP-0002: Container should execute process(es) as non-root user 43 | 44 | **Description:** 45 | Containers have a list of their own users independent of the host system, one of which is UID 0, the root user. Containers should run processes as a user other than root which makes it easier to run the container images securely. 46 | 47 | **Reference:** [CBPP-0002](cbpps/0002-no-root-in-containers.md) 48 | 49 | #### CBPP-0004: Do not run containers with the privilege flag 50 | 51 | **Description:** 52 | 53 | Privileged containers can potentially get unrestricted access to host's resources. Therefore, if the privileged container is compromised, an attacker would have full access to the server. Containers should be run with the privilege flag set to false to securely restrict the access to the host resources. 54 | 55 | **Reference:** [CBPP-0004](cbpps/0004-do-not-run-containers-with-privilege-flag.md) 56 | 57 | ### 6.0 Observability and Diagnostics 58 | 59 | How to get critical data about when things are going wrong and how to determine what must be done to put them right. The detection and correction may be through the actions of an operator or via an automated system. Using logs and metrics from all components in the system to narrow down the area where a problem exists, and to drill down into that area to determine a root cause and a fix. 60 | 61 | ### 7.0 Reliability, Resilience & Availability 62 | 63 | How to ensure CNFs are resilient to failures that are inevitable in cloud environments. 64 | -------------------------------------------------------------------------------- /doc/cbpps/0001-cnf-best-practice-proposal-process.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | # **CBPP-0001: CBPP Process** 8 | 9 | - [Release Signoff Checklist](#release-signoff-checklist) 10 | - [Summary](#summary) 11 | - [Motivation](#motivation) 12 | - [Goals](#goals) 13 | - [Non-Goals](#non-goals) 14 | - [Proposal](#proposal) 15 | - [Workload Context](#workload-context) 16 | - [User Stories (Optional)](#user-stories-optional) 17 | - [Story 1](#story-1) 18 | - [Story 2](#story-2) 19 | - [Notes/Constraints/Caveats (Optional)](#notesconstraintscaveats-optional) 20 | - [References](#references) 21 | - [Alternatives (Optional)](#alternatives-optional) 22 | - [Test Plan](#testing-objectives) 23 | - [Scoring](#scoring) 24 | - [Implementation History](#implementation-history) 25 | 26 | ## **Release Signoff Checklist** 27 | 28 | Items marked with (R) are required for the proposed best practice to be included in a release. 29 | 30 | - [ ] (R) CBPP approvers have approved the CBPP status as `implementable` 31 | - [ ] (R) CBPP summary, motivation and best practice details are appropriately documented 32 | - [ ] (R) Test plan is in place, giving consideration to CNTI Test Catalog input 33 | - [ ] (R) Scoring has been determined 34 | - [ ] "Implementation History" section is up-to-date 35 | - [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes 36 | 37 | ## **Summary** 38 | 39 | A standardized development process for CNTI Best Practices is proposed, in order to: 40 | 41 | - provide a common structure for proposing Best Practices 42 | - ensure that the motivation for a Definition is clear 43 | - allow for the enumeration of testing milestones and graduation criteria 44 | - persist project information in a Version Control System (VCS) for future generations 45 | - support the creation of _high-value, user-facing_ information such as: 46 | - an overall roadmap 47 | - motivation for impactful changes 48 | - reserve GitHub issues for tracking work in flight, instead of creating "umbrella" issues 49 | - ensure community participants can successfully drive changes to completion while stakeholders are adequately represented throughout the process 50 | 51 | This process is supported by a unit of work called a CNTI Best Practice Proposal, or CBPP. 52 | 53 | ## **Motivation** 54 | 55 | **Note: Motivation is something that stems from the past, it's the culmination of experiences that drive you to wanting to achieve a specific goal.** 56 | 57 | In a blog post describing the [road to Go 2](https://blog.golang.org/toward-go2), Russ Cox explains: 58 | 59 | “that it is difficult but essential to describe the significance of a problem in a way that someone working in a different environment can understand” 60 | 61 | Without a standardized mechanism for describing important Cloud Native best practices and principles, the wider telco community could struggle to weave a coherent narrative explaining why a particular CNF Definition is important. Additionally, users running critical infrastructure on Kubernetes need a forward-looking roadmap in order to plan their adoption strategies. 62 | 63 | The purpose of the CBPP process is to reduce the amount of "tribal knowledge" in our community. By moving decisions from a smattering of mailing lists, video calls, and hallway conversations into a well tracked artifact, this process aims to enhance communication and discoverability. 64 | 65 | An important goal of the CBPP process is ensuring that the process for submitting the content is both clear and efficient. The CBPP process is intended to create high-quality, uniform design documents for the CNTI Best Practices focus area to deliberate. 66 | 67 | ### **Goals** 68 | 69 | **Note: A goal is something directed at the future, it's something you wish to happen and you work on achieving that goal.** 70 | 71 | Providing a standardized way for the community to propose and discuss Cloud Native Definitions 72 | 73 | ### **Non-Goals** 74 | 75 | Being an unchangeable template that does not meet the current needs of the community 76 | 77 | ## **Proposal** 78 | 79 | The definition of what constitutes a Cloud Native best practice or principle is a foundational concern for the CNTI Best Practice focus area. If a definition would be described in either written or verbal communication to anyone besides the author or developer, then consider creating a CBPP. The exact size and scale of an average CBPP will be determined as the group begins its work. 80 | 81 | ## **CBPP Template** 82 | 83 | The template for a CBPP is defined [here](./NNNN-cbpp-template.md). 84 | 85 | ## **Important Metrics** 86 | 87 | It is proposed that the primary metrics that would signal the success or failure of the CBPP process are: 88 | 89 | - how many "cloud native best practices and principles" are tracked with a CBPP 90 | - CBPP rejection rate 91 | - PRs referencing a CBPP merged per month 92 | - number of issues open which reference a CBPP 93 | - number of contributors who authored a CBPP 94 | - number of contributors who authored a CBPP for the first time 95 | 96 | ## **Workload Context** 97 | 98 | Context should be provided which includes the type of CNF, workload, and CNF component (eg. Pod, container, Operator) this best practice applies to. The context, of an individual best practice, should illustrate the specific type of workload most likely to take advantage of the benefits listed. 99 | 100 | ### **User Stories (Optional)** 101 | 102 | #### **Story 1** 103 | 104 | I want to propose a new Cloud Native best practice definition. I will fill out the create a PR in the CBPP folder to start the discussion around the Cloud Native Definition. With the template, I will be able to articulate my ideas to the community. 105 | 106 | #### **Story 2** 107 | 108 | I agree/disagree with a proposed Cloud Native best practice definition. After reading through the CBPP, I will be able to comment on the PR and contribute to the discussion around it. 109 | 110 | ### **Notes/Constraints/Caveats (Optional)** 111 | 112 | This first structure is still WIP and should not be considered final. As the CNF WG begins to dive into their work, this format and process should be modified to meet the WG's current needs. 113 | 114 | When applicable, it should also provide a warning if the practice would be a determent to a divergent type of workload. 115 | 116 | ### **References** 117 | 118 | The CBPP process, as proposed, was essentially copied from Kubernetes KEP which are similar to the [Rust RFC process](https://github.com/rust-lang/rfcs) which itself resembles the [Python PEP process](https://www.python.org/dev/peps/pep-0001/). 119 | 120 | ### **Alternatives (Optional)** 121 | 122 | Any process has the potential to engender resentment within the community or not fits its needs. There is a risk that the CBPP process as designed will need to be changed or retired completely. 123 | 124 | The centrality of Git and GitHub within the CBPP process also may place too high a barrier to potential contributors. However, given that both Git and GitHub are required to contribute code changes to many projects today, perhaps it would be reasonable to invest in providing support to those unfamiliar with this tooling. 125 | 126 | ## **Testing Objectives** 127 | 128 | This CBPP will not be tested by the CNTI Test Catalog. 129 | 130 | ## **Scoring** 131 | 132 | CBPPs may have different priorities and importance leading to different scores. Passing and failing scores may be different for similar reasons. Some may be mandatory. 133 | 134 | ## **GitHub Issues vs. CBPPs** 135 | 136 | Using GitHub issues to propose changes lacks effective mechanisms for indicating approval or rejection of a proposed change within the CNTI Best Practices focus area. This is because anyone can open a GitHub issue at any given time. 137 | 138 | In addition to the challenge of managing issues over time, searching for text within an issue can be challenging. The flat hierarchy of issues can also make navigation and categorization tricky. Not all community members will be uncomfortable using Git directly, but it is imperative for our community to educate people on a standard set of tools so they can take their experience to other projects they may decide to work on in the future. While git is a fantastic version control system (VCS), it is neither a project management tool nor a cogent way of managing a backlog. This proposal is limited to motivating the creation of a standardized definition of work in order to facilitate project management. This primitive for describing a unit of work may also allow contributors to create their own personalized view of the state of the project while relying on Git and GitHub for consistency and durable storage. 139 | 140 | ## **Implementation History** 141 | 142 | - The first version of this template was created before KubeCon NA 2020. 143 | - Updated on March 4, 2024 to reflect the [migration of these efforts from CNCF to LF Networking](https://lfnetworking.org/cloud-native-telecom-initiative/). 144 | -------------------------------------------------------------------------------- /doc/cbpps/0002-no-root-in-containers.md: -------------------------------------------------------------------------------- 1 | 2 | 7 | 8 | # CBPP-0002: Container should execute process(es) as non-root user 9 | 10 | - [Release Signoff Checklist](#release-signoff-checklist) 11 | - [Summary](#summary) 12 | - [Motivation](#motivation) 13 | - [Goals](#goals) 14 | - [Non-Goals](#non-goals) 15 | - [Proposal](#proposal) 16 | - [Workload Context](#workload-context) 17 | - [User Stories (Optional)](#user-stories-optional) 18 | - [Notes/Constraints/Caveats (Optional)](#notesconstraintscaveats-optional) 19 | - [References](#references) 20 | - [Alternatives (Optional)](#alternatives-optional) 21 | - [Test Plan](#testing-objectives) 22 | - [Scoring](#scoring) 23 | - [Implementation History](#implementation-history) 24 | 25 | ## **Release Signoff Checklist** 26 | 27 | Items marked with (R) are required for the proposed best practice to be included in a release. 28 | 29 | - [x] (R) CBPP approvers have approved the CBPP status as `implementable` 30 | - [x] (R) CBPP summary, motivation and best practice details are appropriately documented 31 | - [x] (R) Test plan is in place, giving consideration to CNF Test Suite input 32 | - [x] (R) Scoring has been determined 33 | - [x] "Implementation History" section is up-to-date 34 | - [x] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes 35 | 36 | ## **Summary** 37 | 38 | Containers have a list of their own users independent of the host system, one of which is UID 0, the root user. Containers should run processes as a user other than root which makes it easier to run the container images securely. 39 | 40 | ## **Motivation** 41 | 42 | This best practice benefits the CNF developer, improving the quality of the CNF and reducing the developer's likelihood of having to diagnose problems with the CNF. Validation of the best practice is the responsibility of the CNF developer. 43 | 44 | Indirectly, improved CNF quality benefits CNF operators. The proposed tests are runnable by CNF operators as acceptance tests. 45 | 46 | SE-Linux based environments will require dropping root privileges. Example: OpenShift 47 | 48 | ### **Goals** 49 | 50 | Avoiding root in containers can help to: 51 | 52 | - Improve the security and behaviour of applications. 53 | - Add to the defense in depth strategy against external compromises. 54 | - Avoid compromised apps from causing more damage. 55 | 56 | ### **Non-Goals** 57 | 58 | This BP recommends that the application not use the UID that can override all protections. This means that file read and write protections can be established. 59 | 60 | It does not consider what filesystem write permissions should be in order to benefit from those protections. CNF developers will additionally want to ensure filesystem permissions are tightened up appropriately so that non-root users are prevented from doing damage. This is outside the BP scope. 61 | 62 | ## **Proposal** 63 | 64 | When building a container, the container should be built to run its processes as a non-root user. setsid processes should not be required to do the work inside a container. 65 | 66 | A container's root user has fewer Linux Kernel capabilities and may be distinct from the platform's root (if the container runtime enables user namespaces remap feature). 67 | 68 | However, the container's root user does have full read/write access to the container's filesystem. It can read or modify any file. No secrets can be kept from it; it cannot be prevented from changing the content of all executable files on the system. 69 | 70 | On a basic level, avoiding the root user means that the container filesystem permissions are enforceable against all processes running in the system. Those processes can be prevented from doing critical things like: 71 | 72 | - viewing secrets they should not be viewing 73 | - modifying binaries within the filesystem that will later be executed 74 | 75 | Obviously, a well-written CNF would not be attempting to do things it should not do. But all software has bugs. Also, executing processes can be compromised by outside forces, and if this happens filesystem protection is a part of a "Defense in depth" strategy to ensure the compromise does not escalate. 76 | 77 | User/group access enforcement will be respected. As an added advantage, fine-grained access enforcement, such as in SELinux, will also hold 78 | 79 | ## **Workload Context** 80 | 81 | All pod types should implement this best practice. 82 | 83 | ### **User Stories (Optional)** 84 | 85 | #### Supply chain attack user stories 86 | 87 | [Supply chain attacks](../user-stories/supply-chain-attacks.md) are a risk at any point in the supply chain. ‘Defence in depth’ says that we should (a) defend against supply chain attacks but also (b) add mitigations in the case that supply chain attacks happen. 88 | 89 | Examples include 90 | 91 | - [A CNF downloads compromised updates](../user-stories/supply-chain-attacks.md#a-cnf-downloads-compromised-updates) 92 | - [A CNF succumbs to code injection](../user-stories/supply-chain-attacks.md#a-cnf-succumbs-to-code-injection) 93 | - [A CNF succumbs to malicious instructions](../user-stories/supply-chain-attacks.md#a-cnf-succumbs-to-malicious-instructions) 94 | - [A CNF has a security-compromising bug](../user-stories/supply-chain-attacks.md#a-cnf-has-a-security-compromising-bug) 95 | 96 | In all of these examples, the CNFs using a non-root user for their container processes, have limited the scope of damage a compromised process may cause. 97 | 98 | See main [defense in depth for supply chain attacks](../user-stories/supply-chain-attacks.md) document for more information. 99 | 100 | ### **Notes/Constraints/Caveats (Optional)** 101 | 102 | Container images are frequently built from upstream image versions made from OS deployments. These will include things like setsid binaries as a part of their base configuration. If CNF developers follow this best practice they will have to audit and clean up any upstream images to respect this rule (removing files, removing packages or changing permissions as appropriate). 103 | 104 | By default, the first process starting in a container runs as root - you have to actively take steps to shed the permissions (Dockerfile USER line, plus installing files with appropriate ownership within the Dockerfile). 105 | 106 | We specifically want the process to run as a non-root user so that its access is limited. Of course, if access is limited then the CNF developer must ensure that access is still available to the things the CNF is going to need to access. This will involve changing permissions on data files and on working directories when the container image is constructed (they cannot be changed on startup because the application does not have the right to do that). This may also affect the use of shared volume mounts or host mounts - ownership of and rights on the root directory must permit access to the container users. 107 | 108 | ### **References** 109 | 110 | - [CNF WG discussion](https://github.com/cncf/cnf-wg/discussions/20) 111 | - TAG Security: [Cloud Native Security Whitepaper - Least Privilege](https://github.com/cncf/tag-security/blob/main/security-whitepaper/v2/cloud-native-security-whitepaper.md#least-privilege) 112 | - Sysdig: [Top 20 Dockerfile best practices](https://sysdig.com/blog/dockerfile-best-practices/) - 2021/03/09 113 | - [Best practices for pod security in Azure Kubernetes Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/developer-best-practices-pod-security) - 2020/07/28 114 | - [RedHat OpenShift Runtime Security Best Practices](https://www.openshift.com/blog/openshift-runtime-security-best-practices) 115 | - K8s Documentation - Securing a Cluster: [Controlling what privileges containers run with](https://kubernetes.io/docs/tasks/administer-cluster/securing-a-cluster/#controlling-what-privileges-containers-run-with) 116 | - [Security Best Practices for Kubernetes Deployment](https://kubernetes.io/blog/2016/08/security-best-practices-kubernetes-deployment/) - 2016/08/31 117 | - [Docker and Kubernetes — root vs. privileged](https://itnext.io/docker-and-kubernetes-root-vs-privileged-9d2a37453dec) - 2020/06/25 118 | - Bitnami on [Use Non-Root Containers](https://docs.bitnami.com/kubernetes/faq/configuration/use-non-root/) - 2020/05/13 119 | - [Running non-root containers on Openshift](https://engineering.bitnami.com/articles/running-non-root-containers-on-openshift.html) 120 | - K8s 11 Ways Not to Get Hacked: [8. Run Containers as a Non-Root User](https://kubernetes.io/blog/2018/07/18/11-ways-not-to-get-hacked/#8-run-containers-as-a-non-root-user) 121 | - Red Hat PDF [A PRACTICAL INTRODUCTION TO CONTAINER SECURITY](https://www.redhat.com/files/summit/session-assets/2017/L99901-apracticalintroductiontocontainersecurity_labguide.pdf) - 2017/05 122 | - [CVE-2019-5736: runc container breakout](https://seclists.org/oss-sec/2019/q1/119) 123 | - [CVE-2022-0492: Linux kernel cgroups v1 missing capabilities check when setting release_agent](https://seclists.org/oss-sec/2022/q1/123) 124 | - [Using the rootless containers in RHEL](https://www.redhat.com/en/blog/using-rootless-containers-tech-preview-rhel-80) 2019/08 125 | - [Understanding root inside and outside a container](https://www.redhat.com/en/blog/understanding-root-inside-and-outside-container) 2019/12 126 | 127 | ### **Alternatives (Optional)** 128 | 129 | These are not strictly alternatives as they can be used with non-root, but can be applied to a container running as root. 130 | 131 | - Disable all capabilities or limit them 132 | - Do not run the container in privileged mode 133 | 134 | Related items include 135 | 136 | - [Rootless](https://medium.com/@tonistiigi/experimenting-with-rootless-docker-416c9ad8c0d6) containers as seen with [usernetes](https://github.com/rootless-containers/usernetes) 137 | - Alternative runtimes like [Kata Containers](https://katacontainers.io/) for a different approach to security 138 | 139 | ## **Testing Objectives** 140 | 141 | An application which follows this best practice will not have any containers with processes running as root 142 | 143 | This CBPP will be tested by the CNF Test Suite. 144 | 145 | ### Static analysis 146 | 147 | A container image can be tested for compliance: 148 | 149 | - The container metadata should indicate that the first process started is started as a non-root user 150 | - The container filesystem will not have setsid-root binaries. 151 | 152 | If available, the Dockerfile can be checked to see if a non-root user is used. (Dockerfiles are not the only way to build containers, and the Dockerfile may not be a part of the CNF deliverable.) 153 | 154 | - See USER and RUN commands, both of which allow the Dockerfile author to express which user is to be used when launching the container. 155 | 156 | The above static analysis definitively confirms that a container cannot elevate privileges to local root as it removes all avenues for doing so. 157 | 158 | ### Runtime analysis 159 | 160 | One can check for processes launched as, or running as container root. 161 | 162 | We offer the following applications as examples that operators might wish to evaluate for this purpose: 163 | 164 | - [Cnitch](https://github.com/nicholasjackson/cnitch) periodically checks the list of running containers in a Docker environment to see if any are running as container root. 165 | - [Falco](https://falco.org/) checks for processes running as container root if the non-root container policy is set. 166 | 167 | Scanning systems that periodically check running processes or may not identify all root-owned processes, as it must conduct a scan at the moment a process is running. Similarly, process monitoring will not identify a problem if behaviour requiring a root process is not triggered. This cannot be used as a definitive guarantee of safety but is useful as a secondary check. 168 | 169 | ## **Scoring** 170 | 171 | This best practice results in a pass/fail on two counts, depending on role. 172 | 173 | Static analysis (all items checked for a pass) - CNF developers (testing before delivery) or CNF operators (testing what is delivered): 174 | 175 | - Container images indicate their processes should be started as a user other than 0 176 | - Container images should not contain setsid-root binaries (user 0, u+s) 177 | 178 | Runtime analysis - CNF operators: 179 | 180 | - Operators may use runtime verification, from outside the application, to confirm that containers in processes are not owned by container root 181 | 182 | ## **Implementation History** 183 | 184 | First version: July 2021 185 | -------------------------------------------------------------------------------- /doc/cbpps/0003-exceptions.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | # CBPP-0003: Document exceptions to following best practices and community standards 8 | 9 | - [Release Signoff Checklist](#release-signoff-checklist) 10 | - [Summary](#summary) 11 | - [Motivation](#motivation) 12 | - [Goals](#goals) 13 | - [Non-Goals](#non-goals) 14 | - [Proposal](#proposal) 15 | - [Workload Context](#workload-context) 16 | - [User Stories (Optional)](#user-stories-optional) 17 | - [Story 1](#story-1) 18 | - [Story 2](#story-2) 19 | - [Notes/Constraints/Caveats (Optional)](#notesconstraintscaveats-optional) 20 | - [References](#references) 21 | - [Alternatives (Optional)](#alternatives-optional) 22 | - [Test Plan](#testing-objectives) 23 | - [Scoring](#scoring) 24 | - [Implementation History](#implementation-history) 25 | 26 | ## **Release Signoff Checklist** 27 | 28 | Items marked with (R) are required for the proposed best practice to be included in a release. 29 | 30 | - [ ] (R) CBPP approvers have approved the CBPP status as `implementable` 31 | - [ ] (R) CBPP summary, motivation and best practice details are appropriately documented 32 | - [ ] (R) Test plan is in place, giving consideration to CNF Test Suite input 33 | - [ ] (R) Scoring has been determined 34 | - [ ] "Implementation History" section is up-to-date 35 | - [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes 36 | 37 | ## **Summary** 38 | 39 | CNFs should be compliant with applicable best practices, and ops processes 40 | should be validated for compliance with them. However, compliance is a 41 | journey, and we expect end users to start with little compliance and 42 | improve over time. 43 | 44 | Compliance is most useful when users, auditors, and other interested parties can identify 45 | exactly _how_ and _why_ you are compliant. If they test your compliance, 46 | and their tests flag problems, they can see that this is intentional, and 47 | they can find out why compliance is not possible. They can see why you 48 | chose not to be compliant and whether this non-compliance is acceptable to 49 | them. They can also review any mitigating steps you might propose for this 50 | non-compliance or work out steps or their own. 51 | 52 | If your team builds a component used in NFV, or operates a component 53 | of NFV, this document explains how you document your compliance 54 | status. We recommend you make this best practice a part of your 55 | delivery process. This should be the very first best practice you attempt 56 | to comply with. 57 | 58 | ## **Motivation** 59 | 60 | Best practices do not always coincide with pragmatic design. For whatever 61 | reason, a best practice may not be applicable to your circumstance; it 62 | may be too expensive to implement; it may even be technologically 63 | impossible based on other constraints. 64 | 65 | Our task as a working group is to help CNF users do _better_, by painting a 66 | picture of a perfect world and explaining how teams can progressively 67 | take steps towards it. We want to help people on the journey. And during 68 | that journey, their components and processes will be imperfect. 69 | 70 | By understanding that exceptions are going to exist, we make it possible 71 | for teams to understand their shortcomings and document them. Where 72 | exceptions to specific best practices are frequent, this will also 73 | feed back into improvements to our best practice recommendations. 74 | 75 | ### **Goals** 76 | 77 | - Let users explain which best practices they are compliant with 78 | - Communicate the level of compliance of individual parts of the system, 79 | both software and process level 80 | - Let users use CNF-WG best practices even if they cannot be compliant 81 | with every single recommendation 82 | - Help users reduce the number of exceptions over time 83 | 84 | ### **Non-Goals** 85 | 86 | - Providing a specific file format for the documentation 87 | - Requiring extensive documentation of each element - we will keep 88 | required information to a minimum 89 | - Preventing delivery of components where non-compliance exists - we want 90 | teams to be able to deliver components that are good enough, even if they 91 | cannot be perfect 92 | - Documenting processes for improving compliance - we recommend teams 93 | regularly review their compliance list, but we do not mandate any specific 94 | method and this will depend on the team and their working structure 95 | 96 | ## **Proposal** 97 | 98 | We document a means of listing compliance and what elements should be 99 | included in any record of non-compliance. 100 | 101 | We describe when and how those lists should be shared and updated. 102 | 103 | ### Format examples 104 | 105 | A simple machine-readable format might be a table in CSV format. This 106 | can contain the records described, and is conveniently machine-readable. 107 | 108 | ... example here ... 109 | 110 | Making records conveniently machine-readable, such as records of compliance 111 | and non-compliance as simple booleans, helps consumers in filtering these 112 | records and even building compliance checks into their acceptance tests. 113 | Similarly, using a standard encoded format for the CNF-WG best practice 114 | names helps to relate the compliance record back to a specific release 115 | and version, and can ensure that all best practices in a release have been 116 | considered and documented. 117 | 118 | ### Compliance of a CNF 119 | 120 | ## **Workload Context** 121 | 122 | ### **User Stories (Optional)** 123 | 124 | #### **Story 1** 125 | 126 | #### **Story 2** 127 | 128 | ### **Notes/Constraints/Caveats (Optional)** 129 | 130 | ### **References** 131 | 132 | ### **Alternatives (Optional)** 133 | 134 | ## **Testing Objectives** 135 | 136 | ## **Scoring** 137 | 138 | - Mandatory (yes/no): 139 | - Passing score (eg. +5): 140 | - Failing score (eg. -1): 141 | 142 | > Note: CBPP may have different priorities and importance leading to different scores 143 | 144 | ## **Implementation History** 145 | -------------------------------------------------------------------------------- /doc/cbpps/0004-do-not-run-containers-with-privilege-flag.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | # CBPP-0004: Do not run containers with the privilege flag 8 | 9 | - [Release Signoff Checklist](#release-signoff-checklist) 10 | - [Summary](#summary) 11 | - [Motivation](#motivation) 12 | - [Goals](#goals) 13 | - [Non-Goals](#non-goals) 14 | - [Proposal](#proposal) 15 | - [Workload Context](#workload-context) 16 | - [User Stories](#user-stories) 17 | - [Notes/Constraints/Caveats](#notesconstraintscaveats) 18 | - [References](#references) 19 | - [Alternatives](#alternatives) 20 | - [Test Plan](#testing-objectives) 21 | - [Scoring](#scoring) 22 | - [Implementation History](#implementation-history) 23 | 24 | ## **Release Signoff Checklist** 25 | 26 | Items marked with (R) are required for the proposed best practice to be included in a release. 27 | 28 | - [ ] (R) CBPP approvers have approved the CBPP status as `implementable` 29 | - [ ] (R) CBPP summary, motivation and best practice details are appropriately documented 30 | - [ ] (R) Test plan is in place, giving consideration to CNF Test Suite input 31 | - [ ] (R) Scoring has been determined 32 | - [ ] "Implementation History" section is up-to-date 33 | - [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes 34 | 35 | ## **Summary** 36 | 37 | Privileged containers can potentially get unrestricted access to host's resources. Therefore, if the privileged container is compromised, an attacker would have full access to the server. Containers should be run with the privilege flag set to false to securely restrict the access to the host resources. 38 | 39 | ## **Motivation** 40 | 41 | This best practice benefits the CNF consumer / operator (e.g. a Communication Service Provider [CSP]) by limiting the exposure to the host system and other workloads available to a compromised or misbehaving CNF. 42 | 43 | Validating that a CNF is following this best practice can be verified by CNF operators as an acceptance test. 44 | 45 | SE-Linux based environments will require dropping root privileges. Example: OpenShift 46 | 47 | ### **Goals** 48 | 49 | - Improve the security and behaviour of applications. 50 | - Add to the defense in depth strategy against external compromises. 51 | - Avoid compromised applications from causing more damage. 52 | - Restricting the Pod access to the host’s resources (e.g. memory, CPU, network) 53 | - Limiting access to other workloads running on the host 54 | 55 | ### **Non-Goals** 56 | 57 | - Denying access to the host entirely 58 | - Fine-grained access control to the host 59 | 60 | ## **Proposal** 61 | 62 | When creating the Pod definition, the privilege flag policy should be set to false. This tells the container runtime to disable privilege mode for the containers on that Pod when it runs. 63 | 64 | The privileged policy is defined by an absence of restrictions. This type of policy is typically aimed at system- and infrastructure- level workloads managed by privileged, trusted users. 65 | 66 | Privileged containers are defined as any container where the container uid 0 is mapped to the host's uid 0. A process within a privileged container can get unrestricted host access. Without proper settings, a process can also gain privileges from its parent. Application containers should not be allowed to execute in privileged mode and privilege escalation should not be allowed. 67 | 68 | Running a pod in a privileged mode means that the pod can access the host's resources and kernel capabilities. 69 | 70 | 71 | 72 | For example, a potential issue can be an [RCE](https://www.netsparker.com/blog/web-security/remote-code-evaluation-execution/) (Remote Code Execution) Vulnerability in one of your third-party dependencies or even in your own code. Using this vulnerability, an attacker might gain access to the running container, and assuming the pod is privileged, the attacker can continue directly to the host. 73 | 74 | In a virtual machine or a bare-metal server, you avoid running your applications with the root user for a simple reason: if the application is compromised, an attacker would have full access to the server. For the same reason, avoid using privileged containers. A privileged container is a container that has access to all the devices of the host machine, bypassing almost all the security features of containers. 75 | 76 | ## **Workload Context** 77 | 78 | All non-system pod types should implement this best practice. 79 | 80 | ### **User Stories** 81 | 82 | #### Supply chain attack user stories 83 | 84 | [Supply chain attacks](../user-stories/supply-chain-attacks.md) are a risk at any point in the supply chain. ‘Defence in depth’ says that we should (a) defend against supply chain attacks but also (b) add mitigations in the case that supply chain attacks happen. 85 | 86 | Examples include 87 | 88 | - [A CNF downloads compromised updates](../user-stories/supply-chain-attacks.md#a-cnf-downloads-compromised-updates) 89 | - [A CNF succumbs to code injection](../user-stories/supply-chain-attacks.md#a-cnf-succumbs-to-code-injection) 90 | - [A CNF succumbs to malicious instructions](../user-stories/supply-chain-attacks.md#a-cnf-succumbs-to-malicious-instructions) 91 | - [A CNF has a security-compromising bug](../user-stories/supply-chain-attacks.md#a-cnf-has-a-security-compromising-bug) 92 | 93 | In all of these examples, the CNFs using a non-root user for their container processes, have limited the scope of damage a compromised process may cause. 94 | 95 | See main [defense in depth for supply chain attacks](../user-stories/supply-chain-attacks.md) document for more information. 96 | 97 | ### **Notes/Constraints/Caveats** 98 | 99 | If your deployment workload resource definition (eg. helm) pulls in external upstream container images during runtime, those images may have Pod definitions with the privilege flag policy set to true for one or more their Pods. It is recommended that the CNF developer audits and cleans up any upstream images to respect this rule ensuring the privilege flag is set to false when following this best practice. 100 | 101 | Some Pods may need privileges to provide required functionality. For example kube-proxy or the Envoy side-car. 102 | 103 | It is the responsibility of the CNF developer to communicate the privileges needed by their CNF. 104 | 105 | The CNF developer needs to be aware of how privileges of a container can be escalated, see . 106 | 107 | ### **References** 108 | 109 | - ["Do not run containers with the privilege flag" as a best practice · Issue #67 · cncf/cnf-wg](https://github.com/cncf/cnf-wg/issues/67) 110 | - [Applying principle of least privilege to CNFs · Discussion #28 · cncf/cnf-wg](https://github.com/cncf/cnf-wg/discussions/28) 111 | - [Configure a Security Context for a Pod or Container | Kubernetes](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) 112 | - [Kubernetes No New Privileges design proposal: design-proposals-archive/no-new-privs.md](https://github.com/kubernetes/design-proposals-archive/blob/main/auth/no-new-privs.md) 113 | - There are 2 [CIS Docker Benchmark](https://www.cisecurity.org/benchmark/docker/) (a reference document that used to establish a secure configuration baseline for Docker containers) guidelines that cover privileged pods: 114 | - Guideline 5.4: Ensure that privileged containers are not used (recommends not to use the privileged mode when running containers) 115 | - Guideline 5.22: Ensure that docker exec commands are not used with the privileged option (recommends not to use the privileged mode when using `docker exec`). 116 | - Kubernetes Security: [Chapter 1. Approaching Kubernetes Security](https://www.oreilly.com/library/view/kubernetes-security/9781492039075/ch01.html) 117 | - X-Factor CNFs: [XIII. Do not require privileges - X-Factor CNFs](https://x.cnf.dev/process-containers/) 118 | - [Hack my mis-configured Kubernetes - privileged pods](https://www.cncf.io/blog/2020/10/16/hack-my-mis-configured-kubernetes-privileged-pods/) Synk CNCF blog post (2010) 119 | - [10 Kubernetes Best Practices You Can Easily Apply to Your Clusters](https://nirmata.com/2020/02/19/10-kubernetes-best-practices-you-can-easily-apply-to-your-clusters/) (2020) 120 | - [11 Ways (Not) to Get Hacked | Kubernetes](https://kubernetes.io/blog/2018/07/18/11-ways-not-to-get-hacked/#2-enable-rbac-with-least-privilege-disable-abac-and-monitor-logs) (2018) 121 | - Google Cloud: [Best practices for operating containers](https://cloud.google.com/architecture/best-practices-for-operating-containers#avoid_privileged_containers) | Cloud Architecture Center 122 | - tag-security/cloud-native-security-lexicon.md at main · cncf/tag-security · GitHub 123 | - [Pod Security Standards | Kubernetes](https://kubernetes.io/docs/concepts/security/pod-security-standards/#privileged) 124 | - [Apply Pod Security Standards at the Cluster Level | Kubernetes](https://kubernetes.io/docs/tutorials/security/cluster-level-pss/) 125 | - Red Hat: [Podman in containers; specifically with regard to Kubernetes.](https://www.redhat.com/sysadmin/podman-inside-kubernetes) (2021) 126 | 127 | ### **Alternatives** 128 | 129 | Role-based access control provides fine-grained policy management for user access to resources, such as access to namespaces. 130 | 131 | ## **Testing Objectives** 132 | 133 | An application which follows this best practice will not have any privileged containers running. 134 | This CBPP will be tested by the CNF Test Suite. 135 | 136 | ## **Scoring** 137 | 138 | This best practice results in a pass/fail. 139 | 140 | Static or runtime analysis (all items checked for a pass) - CNF developers (testing before delivery) or CNF operators (testing what is delivered): 141 | 142 | The Pod definition or any running container manifest has the privilege flag policy set to false. 143 | 144 | ## **Implementation History** 145 | 146 | First version: Feb 2023 147 | -------------------------------------------------------------------------------- /doc/cbpps/0005-single-concern-per-container.md: -------------------------------------------------------------------------------- 1 | # **CBPP-0005: A CNF's containers should handle a single concern and service (normally mapping to one process type) per container** 2 | 3 | A CNF with multiple concerns should split services (or process types) for each of its concerns into separate containers. 4 | 5 | - [Release Signoff Checklist](#release-signoff-checklist) 6 | - [Summary](#summary) 7 | - [Motivation](#motivation) 8 | - [Goals](#goals) 9 | - [Non-Goals](#non-goals) 10 | - [Proposal](#proposal) 11 | - [Workload Context](#workload-context) 12 | - [User Stories (Optional)](#user-stories) 13 | - [Notes/Constraints/Caveats (Optional)](#notesconstraintscaveats) 14 | - [References](#references) 15 | - [Test Plan](#testing-objectives) 16 | - [Implementation History](#implementation-history) 17 | 18 | ## **Release Signoff Checklist** 19 | 20 | Items marked with (R) are required for the proposed best practice to be included in a release. 21 | 22 | - [ ] (R) CBPP approvers have approved the CBPP status as `implementable` 23 | - [ ] (R) CBPP summary, motivation and best practice details are appropriately documented 24 | - [ ] (R) Test plan is in place, giving consideration to CNF Test Suite input 25 | - [ ] (R) Scoring has been determined 26 | - [ ] "Implementation History" section is up-to-date 27 | - [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes 28 | 29 | ## **Summary** 30 | 31 | A CNF's microservice(s) should follow the single concern principle. To help achieve this goal, a CNF's microservice(s) should have only one process type (or set of parent/child processes) per container. The process(es) in the container should not spawn other process types (e.g. executables) as a way to contribute to the workload but rather should interact with other processes through a microservice API. 32 | 33 | In Docker's Advanced concepts documentation regarding running multiple services in a container the Docker community says: _“It’s best practice to separate areas of concern by using one service per container. That service may fork into multiple processes (for example, the Apache web server starts multiple worker processes). It’s ok to have multiple processes, but to get the most benefit out of Docker, avoid one container being responsible for multiple aspects of your overall application.”_ [#3](#references) 34 | 35 | This will help the programmability and flexibility of CNFs and networks through: 36 | 37 | - Improvements in scalability and responsiveness to service needs 38 | - Offloading container orchestration and leveraging the platform's capabilities 39 | - Reducing downtime risk in deployments and upgrades, increasing the speed of delivery of services 40 | - Managing service concerns in independent units 41 | 42 | ## **Motivation** 43 | 44 | Challenges we think this best practice can help solve: 45 | 46 | - Lifecycle management 47 | - If you have more than one process type in a container, you will have to manage the lifecycle of the secondary concerns within the container, which effectively means creating an orchestrator within the parent process type, reducing the value of using the Kubernetes orchestration capabilities. 48 | - Resource utilization is likely to be less efficient in multi-concerned containers, as the container runtime will look to allocate infrastructure resources for all components rather than individual microservices requirements. 49 | - Response time of multi-concerned containers is increased as scaling is required for the combined services rather than the individual services needing scaling. 50 | - Upgrades causing interruption in many services and all their integration at once because none of the components in the system are independent 51 | - Security 52 | - CNFs with multi-concerned containers have a larger surface area for cyberattacks and production bugs 53 | - Security vulnerabilities in one process type affect all other processes in the same container 54 | - Observability 55 | - Reduced visibility of communication and activity of services in the multi-concerned container, as the container runtime will only be monitoring the init process within the container for signals 56 | - Log messages from the multi-concerned container are more complex because they are from many different sources instead of a single process type 57 | - Difficulty in identifying which process a bug resides in increases as a result of multiple process types in a single container 58 | - Software Development Cycle 59 | - A CNF with multiple maintainers (which can be different groups in an organization) can block each other’s development process when a CNF is tightly coupled 60 | - A high degree of test coverage with a complex and tightly coupled CNF is difficult to achieve 61 | - Enable "polyglot" software designs, where the decoupled entities are developed in entirely different frameworks and even computer languages 62 | 63 | ### **Goals** 64 | 65 | - Lifecycle management 66 | - Simplify and consolidate the use of orchestration capabilities instead of adding additional container orchestration solutions 67 | - Align with microservice architectural practices for operations and development 68 | - Make it easier to scale multiple containers of a CNF to produce an efficient resource utilization and faster response: 69 | - Make it easier to reason the best way to scale a CNF based on anticipated load by reducing the complexity of service concerns and allowing focus on each container's anticipated needs 70 | - Support scaling individual processes of the same type for efficient resource utilization and faster response time for changes in service load (increasing/decreasing) 71 | - Simplify deployment, reduce risk in upgrades and support easier rollbacks by managing the process types (service concerns) independently and providing coarse-grained dependencies at the container level 72 | - Security 73 | - Increase confidentiality by creating clearly defined boundaries between software components 74 | - Reduces the attack surface presented in a CNF's containers 75 | - Limits the number of process types and their dependencies such as additional binaries/libraries 76 | - Prevents unnecessary access to data via shared filesystems 77 | - Allows for finer grained attribute-based controls to be implemented between processes and systems 78 | - Increase integrity by restricting a processes ability to perform CRUD operations in a shared container environment 79 | - Protects containers from interfering with one another by leveraging the container namespace system and cgroup implementations 80 | - Increase availability by reducing the size of failure domain 81 | - Allows for granular control of separation of concerns 82 | - Removes the risk of multiple critical processes terminating simultaneously 83 | - Defines access methods via established interfaces 84 | - Observability 85 | - Simplifying troubleshooting and readability of log output. It is easier to consume log messages and reason about their output when they come from one concern or process type as opposed to when they are interweaved with other concerns. This is even more true in a container that prints all log messages to standard out, such as described in the 12-factor cloud native app. 86 | - Improve ability to monitor CNF activity by exposing the inter-process communication 87 | - Software Development Cycle 88 | - Enable better support for multiple development teams to maintain their own independent lifecycle(s) 89 | - Move towards loosely coupled components: 90 | - Reduces the complexity of each individual component 91 | - Simplifies determining where code resides for a given piece of functionality 92 | - Increases the confidence in test coverage of individual components 93 | - Facilitates locating the root cause of problems by eliminating extra variables 94 | - Minimizes the risk of changes in one component causing problems with another component. 95 | - Increase composability of the CNF and it’s component services 96 | - One concern, and therefore process type, per container is much easier to reason about because it is easier to share and verbally communicate about its contents digitally. This makes it easier to reuse in other projects 97 | - Loosely coupled containers with well defined service APIs increase interoperability with other CNFs as well as between a CNFs component services 98 | 99 | ### **Non-Goals** 100 | 101 | - Best practices for CNFs communication with other processes through microservice APIs 102 | - Best practices on process supervision and management 103 | - Reduce risk issues from of home-grown process management systems 104 | - Security, Zombie processes, unknown failures 105 | - Specifying implementation details of each component 106 | - Specifying how a CNF should be split into individual microservices 107 | 108 | ## **Proposal** 109 | 110 | A CNF with multiple concerns should split services (or process types) for each of its concerns into separate containers. Service dependencies should be handled between containers through well-defined interfaces. 111 | 112 | Pod specs for the CNF should provide scaling and monitoring information for each service running in different containers. 113 | 114 | ## **Workload Context** 115 | 116 | All Kubernetes Pod types should implement this best practice. This best practice applies to control and user plane applications. 117 | 118 | ### **User Stories** 119 | 120 | #### 5G applications needing high-degree of flexibility and automation 121 | 122 | _“5G requirements range from enhanced mobile broadband to ultra-reliable, low-latency communications to massive machine-type communications. Various 5G applications will mix and match these characteristics, requiring a high degree of programmability and the ability to easily combine different functionalities.”_ from “Why Use Containers and CloudNative Functions Anyway?” by Muthurajan Jayakumar (M Jay) @ Intel 123 | 124 | Separating service concerns, represented by process types, into different containers supports the goal of flexibility and programmability allowing the developer to provide definitions for the services needs, including hardware requirements, and the CNF consumer to efficiently scale these 5G services while supporting the accommodation of any container specific needs. 125 | 126 | Separating service concerns also helps to support the automation goals of the 5G operators by providing a more composable CNF with well defined interfaces and coarse grained dependencies at the container level increasing confidence in the testing phases of their pipelines and lowering risk during upgrades. 127 | 128 | #### SMF with multiple services for communicating with the AMF, UPF, and PCF needing more responsive scaling and requiring more resources for its AMF and UPF communication 129 | 130 | An SMF with different services providing the communication between AMF, UPF, PCF and other services in an environment where many new sessions are initiated and end during peak times. This may require scaling of the service responsible for communication with the AMF and UPF faster than the service communicating with the PCF. If the services are split into their own containers they can be scaled independently. 131 | 132 | ![Session Management Function within a 5G Service-based Architecture](https://hackmd.io/_uploads/B1qwravAh.png) 133 | _Source: _ 134 | 135 | ### **Notes/Constraints/Caveats** 136 | 137 | “That service may fork into multiple processes (for example, the Apache web server starts multiple worker processes). It’s ok to have multiple processes, but to get the most benefit out of Docker, avoid one container being responsible for multiple aspects of your overall application.” 138 | 139 | Ideally in a container the process 1 (init process) should be the single application running in the container, as the container runtime monitors the PID 1 application process and uses its signals to report events, knowing when a container has stopped. When multiple processes are started it is recommended to use a non-home-grown supervisor such as . See discussion here 140 | 141 | Depending on the specifics of the implemented functionality, the resulting split may have requirements towards the locality of the resulting multiple containers. It may require certain functions to be deployed on the same worker node or allow for distributing them amongst different worker nodes. The cloud-native principles, and the need for a horizontal scaling does imply the latter, i.e. no locality restrictions. However, due to the specifics of the Telco workloads, such limitations might be necessary for the operation of the particular functionality. 142 | In some cases, the workload distribution can span multiple data-centers and geographic locations. The edge deployments are exemplifying such application designs. 143 | 144 | ### Definitions 145 | 146 | - Monolithic application - applications which do not separate concerns into microservices are considered monolithic 147 | - Monolithic CNF - a monolithic application focused on networking concerns 148 | - Multi-concerned containers - a container having more than a single process type providing services for different concerns 149 | 150 | ### **References** 151 | 152 | 1. 153 | 1. 154 | - _“SoC (separation of concerns) - Single concern principle”_ 155 | 1. 156 | 1. 157 | 1. 158 | 1. 159 | _“As a step towards a fully autonomous network and achieving an intent-based management of a network, its architecture must be prepared by raising the level of abstraction in management with e.g., strong separation of concerns.”_ 160 | 1. 161 | - RATIONALE - 162 | 1. 163 | 1. 164 | 1. 165 | 1. 166 | 167 | ## **Testing Objectives** 168 | 169 | Validate if there are more than one process type running inside of a container 170 | 171 | ## **Implementation History** 172 | 173 | Documentation: 174 | 175 | - First version: September 2023 176 | Test: CNF Test Suite 177 | - First version: [July 2021](https://github.com/cncf/cnf-testsuite/commit/7dc8934e1f9bf643f503ca695f6685cb22b18975) 178 | -------------------------------------------------------------------------------- /doc/cbpps/NNNN-cbpp-template.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | # **CBPP-NNNN: Your short, descriptive title (TEMPLATE)** 8 | 9 | - [Release Signoff Checklist](#release-signoff-checklist) 10 | - [Summary](#summary) 11 | - [Motivation](#motivation) 12 | - [Goals](#goals) 13 | - [Non-Goals](#non-goals) 14 | - [Proposal](#proposal) 15 | - [Workload Context](#workload-context) 16 | - [User Stories (Optional)](#user-stories-optional) 17 | - [Story 1](#story-1) 18 | - [Story 2](#story-2) 19 | - [Notes/Constraints/Caveats (Optional)](#notesconstraintscaveats-optional) 20 | - [References](#references) 21 | - [Alternatives (Optional)](#alternatives-optional) 22 | - [Test Plan](#testing-objectives) 23 | - [Scoring](#scoring) 24 | - [Implementation History](#implementation-history) 25 | 26 | ## **Release Signoff Checklist** 27 | 28 | Items marked with (R) are required for the proposed best practice to be included in a release. 29 | 30 | - [ ] (R) CBPP approvers have approved the CBPP status as `implementable` 31 | - [ ] (R) CBPP summary, motivation and best practice details are appropriately documented 32 | - [ ] (R) Test plan is in place, giving consideration to CNTI Test Catalog input 33 | - [ ] (R) Scoring has been determined 34 | - [ ] "Implementation History" section is up-to-date 35 | - [ ] Supporting documentation—e.g., additional design documents, links to mailing list discussions/SIG meetings, relevant PRs/issues, release notes 36 | 37 | ## **Summary** 38 | 39 | ## **Motivation** 40 | 41 | ### **Goals** 42 | 43 | ### **Non-Goals** 44 | 45 | ## **Proposal** 46 | 47 | ## **Workload Context** 48 | 49 | ### **User Stories (Optional)** 50 | 51 | #### **Story 1** 52 | 53 | #### **Story 2** 54 | 55 | ### **Notes/Constraints/Caveats (Optional)** 56 | 57 | ### **References** 58 | 59 | ### **Alternatives (Optional)** 60 | 61 | ## **Testing Objectives** 62 | 63 | ## **Scoring** 64 | 65 | - Mandatory (yes/no): 66 | - Passing score (eg. +5): 67 | - Failing score (eg. -1): 68 | 69 | > Note: CBPP may have different priorities and importance leading to different scores 70 | 71 | ## **Implementation History** 72 | -------------------------------------------------------------------------------- /doc/cbpps/README.md: -------------------------------------------------------------------------------- 1 | # CNTI Best Practice Proposal (CBPP) 2 | 3 | A CNTI Best Practice Proposal (CBPP) is a way to propose, communicate and coordinate on new best practices for the Cloud Native Telecom Initiative (CNTI). You can read the full details of the project in [CBPP-1](0001-cnf-best-practice-proposal-process.md) 4 | 5 | This process is still in an alpha state and feedback is welcome. 6 | 7 | ## Quick start for the CBPP process 8 | 9 | Follow the process outlined in the [CBPP process document](cnf_best_practice_process.md). 10 | 11 | ## CNF Use Cases 12 | 13 | CBPP shall always refer to at least one CNF use case. These use cases serve as the practical and concrete basis for deriving, discussing and evaluating the best practices for CNFs. Their value is in providing context and a reality check for CBPPs. With help of use cases we make sure that CBPPs are addressing a specific problem or need from the real world. Please use the [Use Case Template](../use-case/NNNN-UC-template.md) to contribute a use case. 14 | 15 | ## Using best practices 16 | 17 | CNFs should be compliant with applicable best practices, and ops processes 18 | should be validated for compliance with them. However, compliance is a 19 | journey, and we expect end users to start with little compliance and 20 | improve over time. The [CBPP-3](0003-exceptions.md) discusses how you should 21 | document your compliance and any places where you require an exception for 22 | being non-compliant. This allows any auditor (e.g. a consumer of your CNF, 23 | your security group, an external auditing entity) to determine where you 24 | are not following best practices, _why_ you are not following best practices, 25 | whether you have mitigations in place for any risks this may cause, and 26 | whether this is acceptable to them. 27 | 28 | If your team builds a component used in NFV, or operates a component of 29 | NFV, CBPP-3 explains how you document your compliance status. We recommend 30 | you make this best practice a part of your delivery process. 31 | -------------------------------------------------------------------------------- /doc/cbpps/api-server-configuration-best-practices.md: -------------------------------------------------------------------------------- 1 | # Kubernetes API server security best practices 2 | 3 | ## Disabling anonymous requests 4 | 5 | The Kubernetes API server must be configured to reject requests from unauthenticated/anonymous sources; the default configuration allows such requests. Only authenticated users should be able to make requests to the API server. Set `--anonymous-auth` to `false`. 6 | 7 | ## Enabling audit logging 8 | 9 | API server audit logging is the functionality that enables operators to keep a record of requests to the cluster. It records the requests together with the initiator identity, therefore, enabling security teams to track events happening in the system. 10 | API server configuration enables versatile configuration of audit logs which can be delivered to a file or a webhook. See `--audit-log-path` or `--audit-policy-file` configuration. 11 | 12 | ## API authorization configuration 13 | 14 | Kubernetes API server supports multiple authorization plug-ins: 15 | 16 | * Attribute-Based Access Control (ABAC) allows you to configure fine-grained access control policies per user, environment or resource attributes. The ABAC policies are specified in files; can't be managed using APIs. This means that in order to change these policies, the operator needs to be able to access these files. 17 | * Role-based access control (RBAC) mode allows you to configure `Role` objects and grant them authorizations while enabling them to be associated with users and groups via `RoleBinding` objects. These objects are Kubernetes API objects as opposed to ABAC policies. 18 | * Webhook is an HTTP callback mode that allows you to manage authorization using a remote REST endpoint and practically defer access control decisions to a non-K8s entity 19 | * Node authorization is a special-purpose authorization mode that specifically authorizes API requests made by Kubelets. 20 | * AlwaysAllow allows all requests to be accepted (practically bypass access control, therefore this is not recommended). 21 | 22 | The recommended way is to use "Role-based access control" since it gives a rich and manageable way to define access control in Kubernetes. Beyond the security perspective, this also makes the cluster compatible with applications requiring special authorization setup. 23 | 24 | ## Extending API access control limits with admission controllers 25 | 26 | Kubernetes role based access control has its limitations. For example, if an entity has rights to create PODs it eventually gets full control over nodes running the cluster. This happens due to the fact that RBAC does not differentiate between a common POD and a privileged POD which has direct access to the host resources. 27 | Admission controllers are optional components in a cluster enabling a more complex limitation strategy over the cluster. They are important security tool, especially in multi user environments. 28 | 29 | ## API server client authentication 30 | 31 | API server supports multiple client authentication technologies. These technologies answer different authentication needs. There is not a single choice that can be considered a “best practice”. However, there are best practices that can be applied to multiple authentication technologies. These settings are paramount to the security of the API server and to the cluster in general. 32 | 33 | ### Authentication credentials confidentiality 34 | 35 | All authentication methods require some kind of client credentials to authenticate against. Known credential are: 36 | 37 | * Private key (client certificate authentication) 38 | * Tokens 39 | * Passwords 40 | These credentials need to be kept private in every case since they are the single factor for an attacker to access Kubernetes API resources. 41 | 42 | ### Revocation and rotation of authentication credentials 43 | 44 | An operator must be able to manage authentication credentials and be able to revoke and rotate them. This is a must best security practice. Rotation reduces the risk of misuse of long unchanged credentials (become widely available), while revocation of credentials needs to be performed for users who no longer need them. 45 | 46 | Different authentication methods support this functionality at different levels. Static token authentication requires restarts to the API server every time a token is revoked. Client certificate authentication revocation is not supported, but client certificates can have limited expiration periods therefore they can be “implicitly revoked” after some time and can be rotated regularly. The re-generation of newly signed client certificates makes operations more complex therefore operators tend to give longer expiration dates, which is not a good practice. 47 | 48 | OpenID Connect tokens can be easily configured to have short expiration time and due to the fact of the interactive nature of OpenID protocol it is simple to tackle the key rotation problem. For these reasons, using OpenID Connect it is a good choice for handling authentication credentials. 49 | 50 | ## References 51 | 52 | * [kube-apiserver configuration reference](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-apiserver/) 53 | * [securing Kubernetes system components](https://www.cncf.io/blog/2021/08/20/how-to-secure-your-kubernetes-control-plane-and-node-components/) 54 | * [kube-apiserver access security blog 1](https://goteleport.com/blog/kubernetes-api-access-security/) 55 | * [kube-apiserver access security blog 2](https://developer.okta.com/blog/2021/12/02/k8s-security-best-practices) 56 | -------------------------------------------------------------------------------- /doc/cbpps/cnti_best_practice_process.md: -------------------------------------------------------------------------------- 1 | # Process to publish a CNTI Best Practice 2 | 3 | ## Table of Contents 4 | 5 | * [Understand the Mission Statement](#understand-the-mission-statement) 6 | * [(Optional) Socialize the idea](#optional-socialize-the-idea) 7 | * [Contribute the CNTI Best Practice Proposal](#contribute-the-cnti-best-practice-proposal) 8 | * [Read the Contributing Guide](#read-the-contributing-guide) 9 | * [Create a GitHub Issue for a specific best practice idea](#create-a-github-issue-for-a-specific-best-practice-idea) 10 | * [Create a Pull Request with the suggested CNTI Best Practice](#create-a-pull-request-with-the-suggested-cnti-best-practice) 11 | * [Communicate the PR and work to get it accepted](#communicate-the-pr-and-work-to-get-it-accepted) 12 | * [Add to the list of CNTI Best Practices](#add-to-the-list-of-cnti-best-practices) 13 | 14 | ## Understand the Mission Statement 15 | 16 | The goal of the Cloud Native Telecom Initiative (CNTI) Best Practices focus area is to aid companies such as telecom vendors, communications service providers and large scale enterprises, running internal telecommunications-like infrastructure, to better understand what cloud native means for telecommunications workloads and help build consensus around industry adoption of cloud native technologies. 17 | 18 | It is important that the best practices that we produce work towards that goal. 19 | 20 | Please read the [Charter](../../charter.md) and in particular the [Mission Statement](../../charter.md#mission-statement). 21 | 22 | ## (Optional) Socialize the idea 23 | 24 | Once you're sure the best practice aligns with the mission statement, it's sometimes a good idea to socialise the idea with the working group to get input and different perspectives, or to help focus the best practice on a specific topic. 25 | 26 | The Best Practices focus area has the following communication channels: 27 | 28 | * [GitHub repository discussion board](https://github.com/lfn-cnti/bestpractices/discussions) 29 | * Add an agenda item in the [bi-weekly meeting notes](https://docs.google.com/document/d/1YFimQftjkTUsxNGTsKdakvP7cJtJgCTqViH2kwJOrsc/edit#) 30 | * Start a conversation / thread in the [LFN Tech Slack](https://join.slack.com/t/lfntech/shared_invite/zt-2cfymedlz-358~927JZBYfVJRMA7P9jg) channel: #cnti-bestpractices 31 | * Send a message to the [mailing list](https://lists.lfnetworking.org/g/lfn-cnti) 32 | 33 | ## Contribute the CNTI Best Practice Proposal 34 | 35 | ### Read the Contributing Guide 36 | 37 | Once you're ready to contribute the best practice, it's a good idea to read the [contributing guide](../../CONTRIBUTING.md#how-to-contribute). 38 | 39 | ### Create a GitHub Issue for a specific best practice idea 40 | 41 | Ideally a set of best practices will have their own tickets. 42 | 43 | * First, check that there is not an existing issue ([open or closed](https://github.com/cncf/cnf-wg/issues?q=is%3Aissue)) covering the best practice suggestion. 44 | * If one is closed, then create a new issue and reference the old issue. 45 | * If an existing issue is currently open, add to the current issue unless the idea significantly changes it, in which case it should be in a new issue. 46 | * If an old suggestion was rejected, reference old issue and provide additional information on why the best practice should be reconsidered. 47 | 48 | * Each best practice will be published and recommended (if accepted) individually. 49 | Example: "use least privileges for containers" is a high level set of best practices. "Use non-root users in containers" is a single best practice. 50 | * It is important as the submitter that you respond to comments in the issue to ensure the proposal doesn't become "stale". 51 | 52 | ### Create a Pull Request with the suggested CNTI Best Practice 53 | 54 | * Check that there is not an existing PR ([open or closed](https://github.com/cncf/cnf-wg/pulls?q=is%3Apr)) covering the best practice suggestion. 55 | * Create a draft following the [template](NNNN-cbpp-template.md) and existing best practices. 56 | * Tag current CNF WG members to review. 57 | * Note: PRs for related use stories and use cases can be created independently or combined with a suggested best practice as seems appropriate. 58 | 59 | ### Communicate the PR and work to get it accepted 60 | 61 | 62 | 63 | * Add the PR to upcoming [bi-weekly meeting agenda](https://docs.google.com/document/d/1YFimQftjkTUsxNGTsKdakvP7cJtJgCTqViH2kwJOrsc/edit#) to discuss and attend the meeting. 64 | 65 | 66 | * Respond to comments in the PR, and merge suggestions when agreed. 67 | * It is important to keep the PR active. 68 | * Because we're all volunteers, we try and keep the number of open PRs to a manageable level. 69 | * Therefore, co-chairs will look to close out PRs which have stalled with no progress and submitter is absent for more than 45/60 days. 70 | 71 | * If the PR is rejected, the co-chair(s) will communicate with the contributor, document the reason for rejection, and follow up in the related issue. 72 | * If the PR is accepted, it will be merged by one of the co-chairs. 73 | * The number of reviewers accepting is documented in the [Contributing file](https://github.com/cncf/cnf-wg/blob/main/CONTRIBUTING.md#steps-to-accept-a-pr). 74 | 75 | ### Add to the list of CNTI Best Practices 76 | 77 | Once the best practice PR has been accepted and merged, you can raise another PR to get the best practice added to the [CNTI Best Practice List](../best_cnf_dev.md). 78 | -------------------------------------------------------------------------------- /doc/figures/cnf-wg-relation.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lfn-cnti/bestpractices/4910069b22b6e3c9f4ab9bd3ccaf13df8d2a2d91/doc/figures/cnf-wg-relation.png -------------------------------------------------------------------------------- /doc/figures/cnf-wg-relation.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lfn-cnti/bestpractices/4910069b22b6e3c9f4ab9bd3ccaf13df8d2a2d91/doc/figures/cnf-wg-relation.pptx -------------------------------------------------------------------------------- /doc/figures/cnf-wg-scope.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lfn-cnti/bestpractices/4910069b22b6e3c9f4ab9bd3ccaf13df8d2a2d91/doc/figures/cnf-wg-scope.png -------------------------------------------------------------------------------- /doc/figures/cnf-wg-scope.pptx: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lfn-cnti/bestpractices/4910069b22b6e3c9f4ab9bd3ccaf13df8d2a2d91/doc/figures/cnf-wg-scope.pptx -------------------------------------------------------------------------------- /doc/glossary.md: -------------------------------------------------------------------------------- 1 | # Glossary 2 | 3 | ## Specification 4 | 5 | ### Community Definitions 6 | 7 | **Cloud Native**: TBD 8 | 9 | **Cloud Native Network Function (CNF)**: A network function that is a [cloud native application](https://glossary.cncf.io/cloud-native-apps/), i.e. developed using cloud native principles. 10 | 11 | **Kubernetes**: [Kubernetes.io](https://kubernetes.io/) is a portable, extensible, open-source platform for managing workloads and services, that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available. 12 | 13 | **Network function**: An application that provides networking functionality. Network functions may be components of network services (a "functional block" according to ETSI) or themselves network services. 14 | 15 | **Network service**: TBD 16 | 17 | **Physical Network Function (PNF)**: A network function distinguished by its encapsulation and isolation in discrete hardware. Internally it may use various technologies, including virtualization, containerization, cloud platforms, etc. 18 | 19 | **Virtualized Network Function (VNF)**: A network function deployed entirely or primarily using virtualization technologies, which comprise both hardware and software features and include CPU, RAM, and GPU support, as well as bus features such as SR-IOV. Virtualization technologies are available in hypervised virtual machines as well as containers (see: Containerized network function). Note that a VNF is not necessarily a cloudified network function. Indeed, the history of NFV—the effort to virtualize network functions—predates the move to cloud platforms. 20 | 21 | ### CNTI Specific Definitions 22 | 23 | **Containerized network function**: A network function deployed entirely or primarily as one or more containers. Should not be referred to by the "CNF" acronym in order to avoid confusion with Cloud Native Network Function (CNF). Note that this definition specifies the runtime technology only, not the platform. Thus a containerized network function might not target Kubernetes nor indeed any cloud platform. Also note that some containerization technologies include virtualization features (see also: VNF). 24 | 25 | **Kube-Native**: TBD 26 | 27 | **Additional Definitions?** 28 | 29 | ## References 30 | 31 | * CNCF Git [CNCF Cloud Native Definition](https://github.com/cncf/toc/blob/main/DEFINITION.md) 32 | * Cloud Native Principles [Cloud Native Principles Preamble](https://github.com/cloud-native-principles/cloud-native-principles/blob/master/cloud-native-networking-preamble.md) 33 | * Kubernetes.io [What is Kubernetes?](https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/) 34 | -------------------------------------------------------------------------------- /doc/use-case/0001-UC-lifecycle-of-infrastructure-where-CNF-is-running.md: -------------------------------------------------------------------------------- 1 | # UC-0001: Lifecycle management of infrastructure where CNF runs 2 | 3 | ## Glossary 4 | 5 | - CNF - Cloud Native Network Function 6 | - Cloud Native Platform - A platform that exhibits cloud like properties on which a CNF runs. For this use case, Cloud Native Platform = Kubernetes cluster 7 | 8 | ## Involved processes 9 | 10 | - [ ] Development 11 | - [ ] Deployment 12 | - [ ] System integration 13 | - [ ] Network integration 14 | - [x] Lifecycle management (infrastructure) 15 | - [x] Operations 16 | 17 | ## Involved actors / personas 18 | 19 | - Cloud Native Platform DevOps (CNPD) 20 | - CNF DevOps 21 | 22 | ## Involved system entities 23 | 24 | - Cloud Native Platform 25 | - CNF 26 | - CI/CD/Testing 27 | - Monitoring 28 | 29 | ## Situation 30 | 31 | Cloud Native Platform DevOps (CNPD) teams regularly perform lifecycle operations on Cloud Native Platforms. These operations include both updates (no breaking changes are expected) and upgrades (there is possibility of breaking changes). For agile teams, these changes happen frequently. An average of 2 times per month is not unheard of. For this use case, we will assume that a Kubernetes cluster is the Cloud Native Platform which is hosting the CNF. In reality there are number of additional components involved. 32 | 33 | The Kubernetes cluster in question is defined according to cloud native principles: declarative deployments and immutable infrastructure. As such, this means that changes are performed via rolling upgrades in following sequence: 34 | 35 | - one fresh node gets created and joined to the cluster 36 | - one of existing nodes gets tainted, then drained and finally eliminated from the cluster 37 | - this process repeats until all old nodes are eliminated from the cluster and replaced with fresh nodes 38 | 39 | This process is first executed in a test environment, followed by reference or staging environment and finally a production environment, with a predetermined time distance (typically 7-10 days). 40 | 41 | The CNPD team expect the ability to perform changes on infrastructure "in place", while a CNF is operating, and during regular working hours (no maintenance windows). They rely on the CNF DevOps team to detect anomalies within their CNFs, after changes to the infrastructure, and report them early. Ideally, in the test environment. Finally, the CNPD expects that if any anomalies are detected, both teams will jointly troubleshoot and eliminate them together. The end goal being that teams are able to proceed with infrastructure changes rapidly. 42 | 43 | The CNF DevOps team expects that changes on infrastructure will cause neither downtime of CNF in production nor significant and lasting degradation of its function or performance. They are capable, however; of tolerating some amount of temporary degradation to performance during a change management process (e.g. requested retries etc...) 44 | 45 | ## Expected behavior 46 | 47 | It is expected that a CNF copes well with situations when one or more nodes in the cluster are replaced with fresh nodes with same or better capabilities. Assuming that remaining capacity of the cluster is allowing rescheduling CNF pods to other, equivalent nodes, the CNF should be capable of bearing, without visible impact on its own function or performance, the situations in which its workloads are shifted to those nodes. If, within the course of infrastructure lifecycle operations, the capacity of the cluster gets reduced but remains above minimally allowed level, CNF shall still function with graceful degradation of its performance in the worst case. It is an expected functionality, that a proper CNF is not tightly coupled to any single node in a cluster, or group of nodes in larger clusters. CNF can still demand that nodes with certain capabilities are present (e.g. SR-IOV, XDP, GPU etc). However it shall not have issues nor impact for the customers if its workloads are dynamically shifted by kube-scheduler from one to another node of such type. 48 | 49 | A CNF DevOps team shall be in position to quickly determine if CNF is behaving and functioning properly after lifecycle operation (i.e. change) on infrastructure is done. If all is OK it is confirmation that it is OK to promote that change to next environment and ultimately to the production. With the normal lifecycle pace of Cloud Native Infrastructure CNF DevOps team needs to count that this has to be done 4-6 times a month for all transitions between the environments. Ideally, it shall be happen automatically with combination of application testing and monitoring where the results can be available next day to CNF DevOps team. If that is not feasible it would still be acceptable if CNF DevOps team does that within 7 days per lifecycle operation. 50 | 51 | ## Challenges and limitations with Kube-native approach 52 | 53 | Most of today's (so called) CNFs are retrofitted VNFs that have been forklifted into the containers. The forklifted CNF is frequently tightly coupled to the underlying infrastructure, including going as far as assigning the CNF to a specific node. This means CNF workloads are blocking these nodes from being drained in reasonably short timeframe. These type of CNFs expect that the individual cluster nodes are long living, even highly available. In Kube-native approach cluster nodes are actually ephemeral and relatively short living. They exist typically for couple of days to couple of weeks. In such situation CNFs are implicitly requiring that infrastructure is mutable and that Cloud Native Platform DevOps team keeps uptime of the cluster nodes high. This is neither practical at scale of CSPs nor it is conformant to cloud native practices. 54 | 55 | Also, most of CNFs today are following systems integrated approach. This means that integration of CNF version X with Platform version Y has been tested separately, possibly as part of periodic "plug tests" or the certification procedures. It is typically CNF vendor to Platform vendor activity. Upstream is missed out in these certifications. Unfortunately, systems integrated approach and periodic integration testing with frozen versions, do not work in cloud native scenario where change is only constant. Versions of components are changing frequently and not upgrading means only accumulating the risks and technical debt. This is very expensive from operations perspective. If changes are done less frequently they are typically bigger and bring higher probability of failures. 56 | 57 | ### Established practices to overcome challenges and limitations 58 | 59 | N/A (tbd) 60 | 61 | ### What needs to be done differently in order to overcome challenges and limitations 62 | 63 | CNFs shall be written for cloud. This means that as boundary condition they shall assume that every node is ephemeral and short lived. They need to use native Kubernetes mechanisms to assure their own high availability in spite of dynamic nature of Kubernetes cluster in which they are running. They shall be able to recover from node failures and from node draining with marginal loss of traffic in the worst case and without visible impact on end customers or systems they serve. 64 | 65 | Each CNF shall come with a suite of tests, testing scenarios and monitoring rules that CNF DevOps team can leverage. A team can deploy the test suite tooling via their continuous testing pipeline and integrate the results with their monitoring systems. The end goal being to ensure that a CNF is verified and certified (the best case being daily) on that particular state of infrastructure. 66 | -------------------------------------------------------------------------------- /doc/use-case/0002-UC-bgp-enterprise/README.md: -------------------------------------------------------------------------------- 1 | # UC-0002: BGP on a customer network 2 | 3 | ## Context 4 | 5 | ### Acronyms 6 | 7 | BGP: Border Gateway Protocol. A TCP-based protocol that shares network routes between devices. 8 | 9 | VRF: Virtual Routing and Forwarding. A context in a network device with its own routing table. Multiple contexts may be set up in a device. For instance, in a home network your gateway has two VRFs - the private addresses inside your home and the public internet outside. These have independent route tables - the internal routes to devices are not shared with the world. An interface with an address is homed in a particular VRF. 10 | 11 | TCP: Transmission Control Protocol. A common network protocol that endpoints use to talk to each other. 12 | Endpoint: A device that generates and consumes traffic (as constrasted with a device that forwards traffic). 13 | 14 | BSD socket API: The historic set of API calls typically offered by an operating system that uses networking. Includes the function that creates a socket for use (```socket()```), a function to dial out to a remote server (```connect()```), and functions to enable listening for connections (```bind()```, ```listen()```) among others. 15 | 16 | CNI: Container Network Interface. A standard API that provides network functions to Kubernetes. On deployment, a CNI plugin is installed that provides the CNI to applications on the Kubernetes API endpoint and commits to setting up networking between containers and to the outside world according to defined API semantics. These semantics are largely focused on reachability ('packets will reach their destination'), as opposed to network implementation (it does not specify if routing, switching or other techniques will be used) and providing networking to containers via the OS kernel's BSO socket APIs. 17 | 18 | CRI: Container Runtime Interface. A standard API for providing runtime features to Kubernetes. Any container provider (of which there are a number, one common one being Docker) implements the CRI, and the CRI is consumed by Kubernetes when launching containers. 19 | 20 | ### 'Default' Kubernetes deployment 21 | 22 | For the purpose of identifying requirements of this use case that might be shortcomings of a standard Kubernetes deployment, we measure the needs of this use case against a common core of Kubernetes functionality that is likely to be found in any Kubernetes deployment from any provider. For the purposes of this, we assume the following are available 23 | 24 | - Core APIs, as found in Kubernetes version 1.20 25 | - A Kubernetes CNI - we do not dictate which CNI is required, only that it provides expected CNI functionality, so this document does not expect extensions that a particular CNI provides, only the generalised functionality 26 | 27 | There are many ways in which deployment choices add functionality, both explicit - such as using a CNI with extensions, beta APIs, controllers and other plugins that add CRDs - and implicit - kernel tuning that might add performance guarantees. For the purposes of comparison we do not expect any of this functionality. Designs that satisfy this use case may propose that some extended functionality is available. 28 | 29 | ## Overview 30 | 31 | A service provider wishes to run a BGP server that is attached to a customer network. This is a common use case when using VPNs. 32 | 33 | The customer network is a separate network from the internet, having its own addressing domain (i.e. it is a VRF on the service provider network). In order to provide BGP service to this network. the BGP application run by the BGP server must peer with other BGP speakers on the customer network, and therefore needs to have an interface within that VRF. Simultaneously, the BGP speaker will require a management interface (for configuration and telemetry) in the SP network. The Kubernetes API endpoint will also be in the SP network, unavailable to the customer - the customer's network can _only_ access the BGP speaker. 34 | 35 | ![BGP network overview](bgp-customer-overview.svg) 36 | 37 | ## Network consumption 38 | 39 | In NFV we often consider interfaces that provide per-packet networking for the purposes of creating a fast dataplane. This, however, is not one of those use cases. Here, the BGP protocol involves a simple TCP connection, and the kernel network stack provides to the BGP process. So, note that the BGP peering sessions will be socket-based connections and the BGP speaker will want to consume the network via standard BGP socket APIs (```socket()```, ```listen()``` and ```connect()```). 40 | 41 | BGP speakers are true peers, without a client and a server. Any BGP speaker is expected to listen on a well known port for its neighbours trying to connect. At the same time, it will be attempting connections to those peers. 42 | 43 | The BGP connection is typically described at both ends with the addresses of the opposite end. The Kubernetes-based BGP speaker process will want to know, and reach, the address of its neighbour. Similarly the neighbour will want to know, and reach, the address of the BGP speaker. There is no DNS involved in this lookup. 44 | 45 | ## Problems vs standard Kubernetes 46 | 47 | There are three sorts of networking occurring here. 48 | 49 | 1. We have the connection used by the OSS systems to the BGP speaker using the management VRF, which is isolated from the SP customer. 50 | 1. We have the connection used by the BGP peering sessions to the BGP speaker, using the customer VRF, isolated from other networks. 51 | 1. We have whatever internal networking is required between microservices within the BGP speaker (perhaps, for instance, speaker code versus a DB instance storing the RIB). 52 | 53 | ### CNI-mediated networking 54 | 55 | Kubernetes' CNI interface provides for the internal connectivity - this has no fixed protocol and is likely amenable to the semantics of the CNI. Similarly, the telemetry interface (logging, monitoring and configuration using e.g. a REST interface) will work using the CNI an Kubernetes ingress. 56 | 57 | ### Networking outside the scope of the CNI 58 | 59 | However, the BGP interface will not work using the CNI, for a number of reasons. 60 | 61 | #### Multiple VRFs 62 | 63 | Kubernetes' CNI is designed to attach to the network outside of the platform using a single point of connectivity - basically, into a single VRF - and the defined APIs do not allow a different connection to be specified for other connections. All CNI endpoints should be in a single addressing domain that can reach all other endpoints. 64 | 65 | ### Multiple VRFs attached to one process 66 | 67 | Aside from this, if the BGP process is in two VRFs, it, and at least one of its pods, must attach to two routing tables. This is conventionally done, in Linux, by using two namespaces. This is problematic for stock Kubernetes. It creates a namespace for each pod and gives control of it to the CNI, for the purposes of Kubernetes networking. Kubernetes does not provide a means to create and attach a second namespace. 68 | 69 | It is also not possible for an unprivileged container to change namespaces. A process requires CAP_NET_ADMIN or higher privileges to switch namespaces. However, CAP_NET_ADMIN is indiscriminate - a process with CAP_NET_ADMIN can do anything to any namespace on the server, regardless of whether it is assigned to the container in which the pod resides. 70 | 71 | ### BGP is not designed to be cloud-native 72 | 73 | BGP far predates Kubernetes and it is not designed with cloud-style failure tolerance in mind. Its connection is designed to be direct from one BGP process to another, with the assumption that the network provides reachability and nothing else. 74 | 75 | BGP peers need to know their own address and the address of their peer. Address rewrites (e.g. NAT) in the flow are likely to cause problems rather than help the solution. 76 | 77 | BGP sessions last as long as the TCP connection is held up. If the speaker process dies, the interruption in the connection causes a significant event on the network, and having a failover process to accept the connection, while still useful, is not going to avoid that network event and the major part of the disruption caused by the failure. 78 | 79 | So: current BGP speakers are not going to be able to make use of standard forms of address rewriting (e.g. NAT) or load balancing from the platform, and in fact these features will cause problems if they cannot be avoided. If BGP requires special network functionality it is likely to be specific to the application and therefore built into the application, not general purpose enough to be worth building into the platform. 80 | -------------------------------------------------------------------------------- /doc/use-case/0002-UC-bgp-enterprise/bgp-customer-overview.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 |
SP management network
SP management network
Customer VRF
Customer VRF
BGP speaker
BGP speaker
Customer BGP speaker
Customer BGP speaker
Customer BGP speaker
Customer BGP speaker
...
...
OSS systems
OSS systemsViewer does not support full SVG 1.1 4 | -------------------------------------------------------------------------------- /doc/use-case/0003-UC-stateful-cnf/README.md: -------------------------------------------------------------------------------- 1 | # UC-0003: Network Functions that maintain State 2 | 3 | **NOTE:** 4 | In a 5G network the Convergent Charging System (CCS) comprises several autonomous business functions, including the Account Balance Management Function (ABMF) to manage state. This use case covers Stateful CNFs such as the ABMF. 5 | 6 | ## Glossary 7 | 8 | - State: data about the condition of an "object" or "process" or "system" (or a "thing of concern") at some given instance of time 9 | - Types of State: ephemeral, long-lived or short-lived 10 | - examples of long-lived state objects are: Balance, Subscriber, Device, Quota, Price-plan 11 | - examples of short-lived state objects are: Session 12 | - examples of ephemeral state objects are: temporary files local a Kubernetes Pod, various in-memory structures not mapped to a Session 13 | - CCS: 5G Convergent Charging System (contains the CHF, ABMF, CGF, RF) 14 | - CHF: Charging Function, handles the charging requests on behalf of the CCS 15 | - PCF: The Policy Control Function provides policy rules to other CNFs (such as AMF, CHF) 16 | - ABMF: Account Balance Management Function, part of the CCS. The Stateful CNF. 17 | - CGF: Charging Gateway Function, part of the CCS 18 | - RF: Rating Function, part of the CCS 19 | - Session: a short-lived record of a specific series of requests by a device to use the network (SMF, Session Management) 20 | - Price Plan: the collection of rules controlling how the charging sessions should be processed 21 | - Quota: a long-lived record of how much time/data the device can use / has used on the network 22 | - Balance: a long-lived record of available funds (allowances) grouped into balance types with units such as time, data, currency 23 | - Subscriber: a subscription to use the network services which may be associated with an individual customer or a company, a subscriber has one or more devices 24 | - Device: the device (UE) that the subscriber uses to access the network (AMF, Access Management), may have its own balances and allowances 25 | - Executor: the microservice within the CCS that processes charging session requests (CGF, Charging Gateway), querying balances (ABMF, Account Balance) and recording quotas (RF, Rating) 26 | - CCS Cluster: a collection of microservice executors that work together to handle charging session requests (CHF, CGF), balance queries (ABMF) and quota records (RF) 27 | 28 | ## Involved processes 29 | 30 | - [X] System integration 31 | - The collaboration of the Stateful CNF with other CNFs, examples: 32 | - The 5G network elements that precede the Stateful CNF 33 | - The external systems that consume the event stream 34 | - [X] Network integration 35 | - The Stateful CNF communicates with the wider network in real-time 36 | 37 | ## Involved actors / personas 38 | 39 | - The Subscriber whose state is maintained by the Stateful CNF 40 | - The Business Owner that is providing service using the CNF 41 | - The Business Function (Data warehouse, GL) that is receiving the event stream from the Stateful CNF 42 | 43 | ## Involved system entities 44 | 45 | - The fast persistent shared volumes that will maintain the Stateful CNF state at rest (required if checkpoint synchronization is used) 46 | - Example: database checkpoints and snapshots 47 | - The system memory in which the live state will reside for the running Stateful CNF (if live state is held in memory) 48 | - The network infrastructure used to connect to the Stateful CNF 49 | 50 | ## Situation 51 | 52 | ![5G Core CHF](5GC_CHF_CCS.svg) 53 | 54 | ### Use Case Scenario: A subscription has a device that requests quota to use the network 55 | 56 | - Precondition: A subscription and an associated device have been provisioned in the Stateful CNF 57 | - Primary steps: 58 | 1. A device subscription makes a real-time request to the network for quota by starting a charging session, the network sends the request through various CNFs until it reaches the Stateful CNF 59 | 2. The Stateful CNF verifies that the subscription is valid, if it has sufficient balance the Stateful CNF allocates some quota, the successful response is returned through the network of CNFs to the device 60 | 3. When the quota has expired (either through usage or time), a real-time request for additional quota is sent for the same charging session, either sent from the Stateful CNF through the network of CNFs to the device or from the device through the network of CNFs to the Stateful CNF 61 | 4. At the end of the device's usage scenario, the device sends a final real-time request to the network completing the charging session, the Stateful CNF can calculate the total charge associated with the charging session 62 | - Postconditions: 63 | 1. A charge event for the charging session is generated 64 | 2. The event is streamed to external systems so that the event can be included in the subscriber's monthly bill 65 | - Alternate steps: 66 | 1. After the charging session for the initial quota request has expired a subsequent quota request is sent through the network of CNFs to the Stateful CNF 67 | 2. The Stateful CNF cannot allocate enough quota and ends the charging session, an error is returned through the network of CNFs to the device 68 | 3. The device closes the network session (applications running on the device may prompt the user to act on the failed network request by topping up their account) 69 | - Exceptional steps: 70 | 1. After the charging session has been established with an initial quota request, the device is disconnected from the network (such as if it roams into a different network) 71 | 2. The network informs the Stateful CNF through the network of CNFs that the device is no longer connected, the Stateful CNF calculates the usage based on previously granted quota and generates an event for the charging session 72 | 73 | ### Why do we need to maintain state? 74 | 75 | - A 5G Real-Time Convergent Charging System has to maintain the account balances and usage quotas of active subscriptions 76 | - We need to be able to accurately reflect the Balance for a Subscription across the account's lifetime (Like a bank) 77 | - The state should be preserved in the event of a catastrophic failure 78 | - State should be available across the cluster to all stateless CNFs that need to access it 79 | 80 | ### Why does it need to be low-latency? 81 | 82 | - The Service Provider needs to be able to take action in real-time (ultra low latency: less than a millisecond end-to-end) for whether to allow the device to use a network resource or not 83 | - A liability or opportunity - Service Provider needs to respond quickly to keep their customers happy, these are the expectations of a true digital experience 84 | - Lack of a real-time decision exposes the Service Provider to potential financial loss 85 | - Ultra low latency charging decisions based on stateful data are necessary to inform other policy decisions by the service provider, for example changing the quality of service level 86 | - The low latency requirements may result in a solution where state is stored in-memory 87 | 88 | ### Why do we need to achieve high throughput? 89 | 90 | - A Service Provider may maintain the balances and quotas for millions of devices, many of which may access the Stateful CNF concurrently 91 | - The high throughput requirement may result in a solution that is composed from a set of clustered executor microservices 92 | 93 | ## Expected behaviour 94 | 95 | - The state that is a maintained by the Stateful CNF must remain ACID complaint, a convergent charging system handles real-time financial transactions 96 | - Atomicity: it should be possible to make a binary yes/no decision (at any point in time) for whether a subscriber's device is allowed to use the network, if a subscriber requests quota but they do not have sufficient balance then their account should not modified 97 | - Consistency: if the value of a subscriber's balance is X then it should remain at X if there are no external changes that could affect it, there should be a defined set of business rules that are applied when making changes that may affect a subscriber's balance, such as when a device is requesting new quota 98 | - Isolation: a subscriber may be making charges on their account while at the same time topping up the balance, all of these transactions should be applied independently but should result in a consistent view of the subscriber's account 99 | - Durability: all changes to a subscriber's account should be maintained through any interruptions to the CNF, such as a pod or node failure or cluster maintenance 100 | 101 | - The Stateful CNF should be resilient and scalable 102 | - The Stateful CNF should continue to follow cloud-native principles: 103 | - Node failure should not result in service outage 104 | - The Stateful CNF should be able to scale up and down to handle traffic spikes 105 | - The Stateful CNF should be upgradeable at any time 106 | - The Stateful CNF should be portable across different hosts and clusters 107 | - The Stateful CNF instances should be loosely coupled to the host infrastructure: they should not be tied to a single node, instead taints and tolerations should be used to select nodes with the necessary capabilities 108 | - The Stateful CNF may have specific resource requirements (memory size, CPU %, PV speed) and may also use affinity rules to control the scheduling of the Stateful CNF pods across the kubernetes cluster 109 | 110 | ## Challenges and limitations with Kube-native approach 111 | 112 | In a kube-native environment the Stateful CNF cannot mandate the specific nodes on which it is to execute, therefore it has to rely on the rules provided by the Kubernetes platform to allow the CNF Operator to configure the resource requirements, using affinity rules, tolerations and taints within the deployment to control how the Stateful CNF spreads across the available nodes in the cluster. 113 | 114 | When a node must be taken out of a cluster for planned downtime, in a low-latency environment (where the short-lived state for a Stateful CNF may be held in memory) it can be challenging to move the workload from one node to another. All ongoing charging sessions must be completed before any move can take place, in addition the Stateful CNF should maintain an active cluster of available executors that are able to handle the processing for new charging sessions which must be started on alternate nodes. In the case of a node failure, the short lived state must have been replicated to other nodes in the executor cluster to allow them to take over the management of the charging sessions. 115 | 116 | In a high-throughput environment (where the long lived state for a Stateful CNF may be held in memory), the state must also be backed-up to persistent storage to handle the situation where a failure results in the total loss of the contents of the node's memory. The Stateful CNF should maintain an active cluster of executors that can handle requests from Subscribers to allow the current values of the long-lived state to be retrieved from any node in the cluster. 117 | 118 | ![CCS Cluster](CCS_Cluster.svg) 119 | 120 | ### Established practices to overcome challenges and limitations 121 | 122 | N/A (TBD) 123 | 124 | ### What needs to be done differently in order to overcome challenges and limitations 125 | 126 | A Stateful CNF should exhibit the following cloud native properties: 127 | 128 | - Scalable 129 | - Resilient 130 | - Portable (loosely coupled to the host infrastructure) 131 | -------------------------------------------------------------------------------- /doc/use-case/0004-UC-onboarding-of-CNF-to-the-CSP-platform.md: -------------------------------------------------------------------------------- 1 | # UC-0004: Onboarding of CNF to the CSP platform 2 | 3 | ## Glossary 4 | 5 | > Provide definitions for technical terms and acronyms. 6 | 7 | ## Involved processes 8 | 9 | - [ ] Development 10 | - [x] Deployment 11 | - [x] System integration 12 | - [x] Network integration 13 | - [ ] Lifecycle management 14 | - [ ] Operations 15 | 16 | ## Involved actors / personas 17 | 18 | CNF Vendor 19 | CNF DevOps 20 | Cloud Native Platform DevOps (CNPD) 21 | CSP Network Ops 22 | 23 | ## Involved system entities 24 | 25 | Data Center Network 26 | Wide Area Network 27 | Cloud Native Platform 28 | CNF 29 | CI/CD 30 | 31 | ## Situation 32 | 33 | CNF DevOps teams in CSP are responsible for the introduction and lifecycle of the network functions under their management. They are typically not developing those network functions, but rely on deployment and integration of products of CNF vendors. As the transformation of network functions from PNF/VNFs to CNFs is gaining momentum, CNF DevOps teams and their selected CNF vendors are getting relatively new task - onboarding of CNF onto the generic Kubernetes based platform of CSP. It is relatively new task because in case of PNF entire function was delivered pre-integrated by vendor and in VNF case the prevalent practice was that VNF vendor brings its own NFVi blueprint or requires a third-party blueprint that is certified by that vendor for that CNF. 34 | 35 | In CNF case the assumption is that the platform is already there, that it is in its core Kubernetes based without special forks as well as that different components needed by CNF could be installed via Kubernetes API. 36 | 37 | This creates the situation in which CNF and CSP platform, both of which have their requirements and preconditions, meet each other as such for the first time. The process of getting the CNF deployed on the platform in aforementioned scenario is colloquially called "onboarding". In this process CNF vendor, CNF DevOps and CNPD work together to reach common goal of having production grade deployment of CNF supported with appropriate SLA which factors-in the fact that CNF runs on CSP platform. 38 | 39 | ## Expected behaviour 40 | 41 | CNF vendor expects that CSP platform and environment (e.g. Network) are fulfilling certain requirements and preconditions for CNF to be able to achieve production grade deployment. These requirements are different from CNF to CNF. 42 | 43 | CNPD expects that all requirements are well documented with and provided in such form before onboarding starts. The documentation shall enable CNPD team to verify readiness and prepare the platform (e.g. configure necessary kernel modules, SR-IOV interfaces and similar). In practically all deployments that are meant for production environment the deployment artifacts are stored in private on-prem artifacts and registries. Therefore CNPD expects that CNF vendor provides its artefacts in a form that is typical for cloud native applications (Helm charts, containers, yaml manifests) which could be easily replicated to CSP's on-prem environment (e.g. CNF vendor's registry accessible via internet from CSPs caching registry). 44 | 45 | CNF DevOps team expects to deploy and configure the CNF application on their own, preferably directly via automation pipelines and in interaction and cooperation with CNPD and CSP Network teams. For that they expect to receive well documented installation procedures, references for typical continuous deployment realisations with that particular or similar CNFs as well as comprehensive deployment troubleshooting guide. As already mentioned in [UC0001](0001-UC-lifecycle-of-infrastructure-where-CNF-is-running.md), CNF DevOps team also expects to get suite of tests from CNF vendor. Tests could then be deployed in continuous testing pipeline to automatically verify that CNF is running properly (production grade) on the platform and that the deployment qualifies for SLA. 46 | 47 | The starting point of onboarding shall be extension of platform with any requirements and dependencies that are coming from CNF. When all conditions on the platform level are met, the deployment of CNF shall proceed. 48 | 49 | ## Challenges and limitations with Kube-native approach 50 | 51 | Most of the CNFs today, especially complex ones, are packaged in such way that only trained professional service teams of the CNF vendor can reasonably deploy them. The documentation and manuals that come with those CNFs are giving same impression. The onboarding today usually requires intensive interaction between CNF vendor, CNF DevOps and CNPD teams and lots of trial/error tuning and manual adaptations on both platform and CNF side. 52 | 53 | The CNFs are also regularly packaged to be suitable for opinionated installers that are typically part of vendor's own platform offering or particular platform blueprint. The onboarding on CSP's own platform in such cases frequently involves sort of reverse engineering of CNF vendor's installer to come to basic cloud native artefacts and derive requirements from installer's code with the help of vendor's engineers. 54 | 55 | Finally, the CNFs are frequently delivered as Helm charts. However, these Helm charts often do not cover all dependencies that enables reconciling the deployments in proper way. They also install CRDs which are not managed well by Helm. 56 | 57 | This situation is in most cases not blocking the onboarding, but is making it laborious and not repeatable. 58 | 59 | ### Established practices to overcome challenges and limitations 60 | 61 | The practices already exist outside of CNF world, but are not very well followed by CNF vendors. Some of the practices are here: 62 | 63 | - [The Chart Best Practices Guide](https://helm.sh/docs/chart_best_practices/) 64 | - [GitOps](https://www.gitops.tech/) 65 | - [Kubernetes production best practices](https://learnk8s.io/production-best-practices) 66 | 67 | ### What needs to be done differently in order to overcome challenges and limitations 68 | 69 | CNF vendors need to start preparing CNFs for shipping as regular cloud software that is well documented and that can be deployed and configured by CNF DevOps team. Following established best practices for cloud native software distribution and deployment would improve the situation significantly. 70 | -------------------------------------------------------------------------------- /doc/use-case/NNNN-UC-template.md: -------------------------------------------------------------------------------- 1 | # UC-NNNN: Short descriptive title of your use case (TEMPLATE) 2 | 3 | ## Glossary 4 | 5 | > Provide definitions for technical terms and acronyms. 6 | 7 | ## Involved processes 8 | 9 | > Please uncomment and/or add key processes that are involved in the use case you are presenting. This helps actors to decide if the use case is relevant for them or not. 10 | > 11 | > - Development 12 | > - Deployment 13 | > - System integration 14 | > - Network integration 15 | > - Lifecycle management 16 | > - Operations 17 | > 18 | - [ ] Development 19 | - [ ] Deployment 20 | - [ ] System integration 21 | - [ ] Network integration 22 | - [ ] Lifecycle management 23 | - [ ] Operations 24 | 25 | ## Involved actors / personas 26 | 27 | > List main actors / personas involved in the use case. For example: 28 | Business customer of CSP 29 | Operations Team of CSP 30 | Service team of CNF Vendor 31 | Etc. 32 | 33 | ## Involved system entities 34 | 35 | > List system entities that are involved in the use case. For example: 36 | Data Center Network 37 | Wide Area Network 38 | Cloud Native Platform 39 | CNF X 40 | MANO 41 | OSS 42 | Etc. 43 | 44 | ## Situation 45 | 46 | > Describe situation related to this use case. Indicate what are intents and expectations of different parties as well as the roles and requirements of different system components. Explain how everything is related. Drawings, flow and system diagrams are beneficial here. This part shall give the reader clear overview of the situation as the starting point for evaluation of the use case. It should not contain the detailed technical descriptions and analysis. 47 | 48 | ## Expected behaviour 49 | 50 | > Provide detailed technical descriptions and depict expected behaviour of system entities and components regardless of cloud nativeness. Highlight what are different entities expected to provide. Depending on the use case you can look at it from different angles e.g. CNF angle, Cloud Native Platform angle, Network angle etc. This section should give the reader clear understanding of expected behaviour. 51 | 52 | ## Challenges and limitations with Kube-native approach 53 | 54 | > This section will give us main inputs for discussion and evaluation of best practices. Describe which challenges and limitations kube-native approach presents for this use case. Provide technical details about what is not possible with kube-native approach and elaborate why is that relevant. If there are some established practices that are used to work around these limitations please describe those in the chapter below* 55 | 56 | ### Established practices to overcome challenges and limitations 57 | 58 | > If such practices exist please elaborate them here in sufficient technical details. Provide references and give your opinion weather or not these practices can be considered as candidates for CNF best practices in line with [CBPP description](../cbpps/0001-cnf-best-practice-proposal-process.md). 59 | 60 | ### What needs to be done differently in order to overcome challenges and limitations 61 | 62 | > If there are no established practices and you have ideas or if you believe things related to this use case should be done differently please indicate it shortly here. The purpose of this is to earmark some potential inputs for CBPP process. It is not expected to elaborate CBPPs here. 63 | -------------------------------------------------------------------------------- /doc/use-case/README.md: -------------------------------------------------------------------------------- 1 | # Networking Use Cases 2 | 3 | It's clearer that a best practice is a good course of action when it satisfies the needs of a Kubernetes use case related to one or more actors. 4 | 5 | This folder contains different use cases, their description, references, and other context. The best practices, recorded elsewhere, refer back to these use cases for their justification. 6 | 7 | Goal: 8 | 9 | - Relating to real world use cases. 10 | - Discussion of core features and functionality from use cases and their NFs. 11 | - Examination of existing use cases to discussion of kubenative implementations which provide the core features and functionality. 12 | 13 | ## How to add a new use case 14 | 15 | If you just have a partially-formed idea for a use case feel free to add to the discussion topic [listing networking uses cases](https://github.com/cncf/cnf-wg/discussions/39). This is the best place to go if you aren't sure how to write up the use case. We can help you expand that into a full description. 16 | 17 | Once you have a new use case you would like to discuss in more depth, create a new [GitHub discussion topic](https://github.com/cncf/cnf-wg/discussions) with the details you have including attachments and/or links to any external content (eg. Google docs, hackmd, diagrams). 18 | 19 | If you have enough content to writeup a full use case then feel free to create a pull request in this [use case folder](./) with either a 20 | 21 | - single Markdown file with all content included (including in-line diagrams) for the use case 22 | - new use case folder with all content with a README.md as the starting point describing the use case. 23 | 24 | If you're going to do this via a pull request you must have a complete description of the use case. You should have gathered any information you need and included it by this point. We will then discuss the merits of the use case during review of the pull request, but this is not the time to be writing large sections of it. 25 | -------------------------------------------------------------------------------- /doc/user-stories/air-gapped-environments.md: -------------------------------------------------------------------------------- 1 | # User stories: Employing Air Gapped Environments 2 | 3 | Environments hosting critical infrastructure often deny access to the public internet, even via a proxy. This is done for several reasons such as controlling the artifacts that come into an environment, controlling builds at the source, and ensuring malicious code is not able to call back to remote destinations. Beyond the security implications of air gapping an environment and utilizing private registries, upstream repository management becomes a centralized task that all internal stakeholders can take advantage of. Internal developers and operations teams can rely on their SREs to ensure that the images, packages, etc... they require are available and secure. 4 | 5 | Note: Air gapped environments are not a challenge solely for service providers/network operators. Other operators of heavily regulated environments, such as US DoD networks, seeking to employ CNFs would be beholden to these constraints. 6 | 7 | ## A CNF utilizes a cloud based licensing model 8 | 9 | A CNF running on a K8s cluster is downloaded from a centralized repository. Upon instantiation, the CNF validates that it has the required license tied to a suite of desired functionality. If the license is valid, the functionality is enabled. If a valid license is not available, the desired behavior is often that the next available license in a pool is requested, or a new order is generated in real time. 10 | 11 | In this case, if the operator does not allow for a CNF to access a remote licensing server, this method breaks down. Additionally, if the licensing server is hosted locally in the environment but requires licenses be purchased in advance, then the desired behavior of dynamically deploying CNFs might be hindered due to a lack of available licenses in the pool. 12 | 13 | The goal for licensing best practices should be to find ways that allow operators to safely and securely deploy CNFs on-demand in air gapped environments. This means that CNF vendors and operators must find a way to operate CNFs in this environment - one that breaks a common assumption of cloud software, that it has reachability to internet services. 14 | 15 | ## An operator employs [GitOps](../use-case/0001-UC-lifecycle-of-infrastructure-where-CNF-is-running.md) methodologies with defense in depth 16 | 17 | In an air gapped environment, unique supply chain considerations arise. As discussed in [user-story supply-chain-attacks](supply-chain-attacks.md), securing one's supply chain is paramount to a defense in depth strategy. In an operator's air gapped environment, this supply chain almost always enforces a centralized choke point for pulling, distributing, and versioning artifacts. 18 | 19 | In this case, the operator would seek to ensure that artifacts are signed and deemed "trusted" before being released to the rest of the internal actors. Trust could be established in multiple ways, such as internal pipelines that pull untrusted artifacts and run vulnerability scans and automated testing or via an established software supply chain with a trusted vendor. All declarations supporting an operator's GitOps style approach would strictly build from these privately hosted repositories in fully air gapped development, stage, and production environments. Testing would include validation that a CNF is fully deployable and maintainable with zero reach to the internet. 20 | 21 | ## A vendor employs SaaS based services 22 | 23 | A vendor provides a SaaS based solution hosted in a third-party cloud environment. This service could be related to licensing, streaming telemetry, or automated fault detection. In this instance, the provider and operator must come to an agreement as to how these services will be accessed. 24 | 25 | This could be accomplished via a VPN, by providing operators the ability to host these software solutions within their environments, or convincing operators that fully air gapped approaches are not feasible in the cloud native world. Best practices that explore how to consume SaaS based offers in air gapped environments must be explored in the future. 26 | 27 | ## Glossary 28 | 29 | - Air gapped environment: A physically and logically isolated environment that disallows direct network access/connectivity. These environments are created to protect infrastructure from bad actors either attempting to gain access to the environment remotely, or for malicious software to reach outside of the isolated environment. Given these constraints, standard cloud software consumption models where users and software pull, clone, or download required resources from public repositories are blocked. All new software bound for this environment must first be introduced through the operator’s secure source control mechanisms. 30 | - Source: For the context of this user story, references to an air gapped environment’s "source", with regards to controlling builds, software ingest, and other topics, is specifically referring to a private source repository accessible through private network connectivity. Specifically, the intention is that this entity acts as a single source of truth for all operators in this environment. Any software, tooling, or configuration files leveraged in this environment must be made accessible here. **Note:** The source control tooling can be federated, and this definition does not imply that there is a requirement for a single entity hosting all images and source code. It is a logical construct that acts as a control mechanism to strictly enforce what is made available in an air gapped environment. 31 | - Upstream: For the context of this user story, upstream refers to any repository sitting "North" of the private repository. Air gapped environments, if built to be truly isolated, are prevented from pulling directly from internet repositories and cannot use them as the upstream repository. To access software in an air-gapped environment, software should be validated and then hosted in a private repository within the air-gapped environment. Mechanisms, such as VPNs and proxies can be leveraged as part of a secure source control strategy as well, but implies implicit trust to the additional “sources” you are pulling from and the security of the network mechanism used. 32 | -------------------------------------------------------------------------------- /doc/user-stories/stateful-cnf-use-cases-and-user-stories.md: -------------------------------------------------------------------------------- 1 | # Stateful CNF - Use Cases & User Stories 2 | 3 | 4 | ## Stateful / Persistent data 5 | 6 | ### Use Case 7 | 8 | As a CSP, I must maintain persistent data (such as subscriber information, account balances, quota balances, thresholds, etc.) for the subscribers that use my network and/or services. This persistent data can be static or dynamic in nature. In many cases, the persistent data is maintained for the lifetime of a subscriber’s account. 9 | 10 | ### User Stories 11 | 12 | 1. A subscriber wants to update his/her address with a CSP. To allow for this, a CSP must be able to maintain persistent data (static) such as the subscriber identity, subscribed services, selected features, etc. 13 | 2. A subscriber wants to use the services offered by a CSP. To provide service to a subscriber, a CSP must maintain persistent data (dynamic) regarding the subscriber such as an account balance, quota balance, thresholds, etc. This data is then utilized to trigger a host of business decisions such as QoS changes, moving a call/session from one access network to another, allowing or denying access to a service, etc. 14 | 15 | ## Real-time / Low Latency CRUD actions 16 | 17 | ### Use Case 18 | 19 | As a CSP, I must be able to perform real-time CRUD actions on persistent data (dynamic) to limit financial exposure and to support diverse business, network, and service decisions such as changing a subscriber’s QoS, changing the service access method, granting access to a service, limiting, or denying access to a service, etc. 20 | 21 | ### User Stories 22 | 23 | 1. A subscriber attempts to access a CSP service (e.g., a data service). A real-time request is made to determine if the subscriber has sufficient quota (stored as persistent data) available to allow access to the service. If quota is available, access to the service is granted in real-time. If not, access may be denied, limited, or an alternative proposed. 24 | 2. As the subscriber is using the CSP service, quota is updated (stored as persistent data) in real-time to accurately reflect consumption at any given time. In addition, the quota is maintained in real-time to serve as a basis for decision making (business, service, or financial by the CSP) 25 | 3. An IoT device in a smart factory attempts to access a higher QoS network slice service to accommodate a spike in production. A real-time request is made to determine if the IoT device has utilized its High Bandwidth QoS threshold for the day or not. If the threshold has not yet been reached, access to the higher QoS is granted. Otherwise, the IoT device is denied access. 26 | 27 | ## High Transaction Processing Throughput 28 | 29 | ### Use Case 30 | 31 | As a CSP, I must be able to achieve high throughput for CRUD actions on persistent data. In some scenarios, this could amount to hundreds of thousands of transactions per second. 32 | 33 | ### User Stories 34 | 35 | 1. A subscriber of a large CSP wants to access and use services on the CSP’s network. At any given point in time, millions of subscribers and devices may be attempting to access and use services at the same time. Therefore, persistent data must be created, read, updated, and deleted in real-time to support a wide variety of business, network, and service decisions. The sheer number of subscribers and devices demands that hundreds of thousands of such CRUD actions per second be possible. Without this capability, the CSP is unable to determine if a subscriber should be allowed to use the service without being exposed to financial risk or being unable to make various business, network and services decisions based on real-time data. 36 | 37 | ## ACID Compliant 38 | 39 | ### Use Case 40 | 41 | As a CSP, I must ensure that all financially related transactions impacting (or using) persistent data are ACID compliant (Atomicity, Consistency, Isolation, Durability). 42 | 43 | ### User Stories 44 | 45 | 1. As a subscriber to services provided by a CSP, certain events, transactions, and activities can create financial obligations to the CSP and vice versa. As such, these transactions must be kept as persistent data and must be ACID compliant to guarantee their accuracy at any given time. 46 | 47 | ## Availability 48 | 49 | ### Use Case 50 | 51 | As a CSP, I must continuously provide persistent data to stateless services that need (and have permission) to access the information. 52 | 53 | ### User Stories 54 | 55 | 1. A subscriber of telecommunication services expects services to be available 24/7/365, therefore CSPs must be able to provide high availability of persistent data which is used for business, network, and service decisions. 56 | 57 | ## Continuity 58 | 59 | ### Use Case 60 | 61 | As a CSP, I must be able to ensure “instant and total” recovery of persistent data in the event of a local or even complete site failure. 62 | 63 | ### User Stories 64 | 65 | 1. A subscriber of telecommunication services expects services to be available 24/7/365, therefore CSPs must be able to provide continuity of persistent data in the event of a catastrophic failure. Without continuity, decisions on behalf of the subscriber would not be possible or be based on inaccurate information. The expectation of continuity is expressed as the ability to recover all persistent data instantly and totally. 66 | 67 | 68 | -------------------------------------------------------------------------------- /doc/user-stories/supply-chain-attacks.md: -------------------------------------------------------------------------------- 1 | # User stories: Defense in Depth against Supply Chain Attacks 2 | 3 | Supply chain attacks are a risk at any point in the supply chain. In a supply chain attack, a malicious actor sneaks code into the application that serves their nefarious purposes. It can happen for many reasons: for instance, because they managed to modify the registry, they managed to modify the image before it was placed in the registry, or they managed to get illicit code into an open source project that is built into the container. 4 | 5 | 'Defence in depth' says that we should (a) defend against supply chain attacks but also (b) add mitigations in the case that supply chain attacks happen - that is, we should not assume that a single line of defence will hold. 6 | 7 | ## A CNF downloads compromised updates 8 | 9 | A CNF running on a K8s cluster is downloaded from a centralized registry. Updates also come from this registry. The operator installs new images in the centralised repository as they receive them from the CNF developer. 10 | 11 | In this case, if the operator ensures that processes in the container will be run as a normal container user without extra rights, the malicious code will have a very limited ability to do dangerous things. It will be harder for it to try to escalate its privileges, it has no rights outside of the container, it cannot change files within the container that could cause more damage (such as root-owned settings files in /etc). 12 | 13 | Thus: limiting the privilege of container processes makes the system safer to use. 14 | 15 | ## A CNF succumbs to code injection 16 | 17 | As above, a CNF runs malicious code, in this case introduced into the process through a code injection attack on its internal or external interfaces by a bad actor within the operator's organisation or by anyone on the network, respectively. In this instance, limiting the power wielded by processes in the container again prevents the attack from going further. 18 | 19 | ## A CNF succumbs to malicious instructions 20 | 21 | As an example: the configuration of a CNF might allow the operator to determine paths where a log file is written. If this path is set to a protected file, then the container root user has the rights necessary to overwrite that file. This could be exploited to change configuration and settings files within the container image and subvert it. 22 | 23 | A CNF such as a programmable forwarder might receive rules updates from a central system. Maliciously crafted rules might cause the process to attempt to do operations within its container that would compromise the container's security. While it takes a successful attack to get malicious rules into the feed of updates, this means that such an attack can escalate to container control and perhaps further. This can be limited if the process in the container is an unprivileged process running as a normal user without any capabilities, meaning that it cannot write files owned by other users. 24 | 25 | ## A CNF has a security-compromising bug 26 | 27 | A CNF unwittingly has a bug where an appropriately crafted packet will write a file at a location on the disk dependent on the packet's content. This can lead to compromise. Many possibilities for compromise are prevented if the process in question does not have privileges to write in the filesystem, as is typically available to a container root user. 28 | 29 | A CNF may also have a bug where it attempts to rewrite settings files that keep the CNF secure (e.g. files containing credentials). If the file is not owned by the process and the process is not itself a root process it has no privilege to change those files. 30 | -------------------------------------------------------------------------------- /doc/whitepaper/Accelerating Cloud Native in Telco.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lfn-cnti/bestpractices/4910069b22b6e3c9f4ab9bd3ccaf13df8d2a2d91/doc/whitepaper/Accelerating Cloud Native in Telco.pdf -------------------------------------------------------------------------------- /doc/whitepaper/README.md: -------------------------------------------------------------------------------- 1 | # Whitepapers by the CNF WG 2 | 3 | - [Accelerating Cloud Native in Telco](Accelerating_Cloud_Native_in_Telco.md) ([PDF](Accelerating%20Cloud%20Native%20in%20Telco.pdf)) 4 | - [Best Practices - TBD](../best_cnf_dev.md) 5 | -------------------------------------------------------------------------------- /events/.gitignore: -------------------------------------------------------------------------------- 1 | *.swp -------------------------------------------------------------------------------- /events/2023-cloud-native-telco-day-chicago/README.md: -------------------------------------------------------------------------------- 1 | # Cloud Native Telco Day and related events from Chicago 2023 2 | 3 | This contains notes, summaries, images, etc from CN Telco Day and the LFN+CNCF alignment workshop from November 2023 4 | -------------------------------------------------------------------------------- /events/2023-cloud-native-telco-day-chicago/images/.gitignore: -------------------------------------------------------------------------------- 1 | -- 2 | -------------------------------------------------------------------------------- /events/2023-cloud-native-telco-day-chicago/images/Topic 1 - Whiteboard.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lfn-cnti/bestpractices/4910069b22b6e3c9f4ab9bd3ccaf13df8d2a2d91/events/2023-cloud-native-telco-day-chicago/images/Topic 1 - Whiteboard.jpg -------------------------------------------------------------------------------- /events/2023-cloud-native-telco-day-chicago/images/Topic 1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lfn-cnti/bestpractices/4910069b22b6e3c9f4ab9bd3ccaf13df8d2a2d91/events/2023-cloud-native-telco-day-chicago/images/Topic 1.png -------------------------------------------------------------------------------- /events/2023-cloud-native-telco-day-chicago/images/Topic 2 - Whiteboard.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lfn-cnti/bestpractices/4910069b22b6e3c9f4ab9bd3ccaf13df8d2a2d91/events/2023-cloud-native-telco-day-chicago/images/Topic 2 - Whiteboard.jpg -------------------------------------------------------------------------------- /events/2023-cloud-native-telco-day-chicago/images/Topic 2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lfn-cnti/bestpractices/4910069b22b6e3c9f4ab9bd3ccaf13df8d2a2d91/events/2023-cloud-native-telco-day-chicago/images/Topic 2.png -------------------------------------------------------------------------------- /events/2023-cloud-native-telco-day-chicago/images/Topic 3 - Whiteboard.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lfn-cnti/bestpractices/4910069b22b6e3c9f4ab9bd3ccaf13df8d2a2d91/events/2023-cloud-native-telco-day-chicago/images/Topic 3 - Whiteboard.jpg -------------------------------------------------------------------------------- /events/2023-cloud-native-telco-day-chicago/images/Topic 3.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lfn-cnti/bestpractices/4910069b22b6e3c9f4ab9bd3ccaf13df8d2a2d91/events/2023-cloud-native-telco-day-chicago/images/Topic 3.png -------------------------------------------------------------------------------- /events/2023-cloud-native-telco-day-chicago/images/Topic 4 - Whiteboard.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lfn-cnti/bestpractices/4910069b22b6e3c9f4ab9bd3ccaf13df8d2a2d91/events/2023-cloud-native-telco-day-chicago/images/Topic 4 - Whiteboard.jpg -------------------------------------------------------------------------------- /events/2023-cloud-native-telco-day-chicago/images/Topic 4.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/lfn-cnti/bestpractices/4910069b22b6e3c9f4ab9bd3ccaf13df8d2a2d91/events/2023-cloud-native-telco-day-chicago/images/Topic 4.png -------------------------------------------------------------------------------- /events/LF-Cloud-Native-Networking-Program-Alignment-Workshop.md: -------------------------------------------------------------------------------- 1 | # LF Cloud Native Networking Program Alignment Workshop 2023 2 | 3 | - Tuesday, November 7th, 2023 a 2-4pm CST 4 | - KubeCon North America 2023 5 | - Recording available at: 6 | 7 | ## Agenda 8 | 9 | - (10 min) - Opening Remarks, Ranny 10 | - (10 min) - Collaborative Workshop, Moderator 11 | - (20 min) - Topic 1: High Level Scope & Priorities, Participants 12 | - (20 min) - Topic 2: Positioning / Governance, Participants 13 | - (20 min) - Topic 3: Assets, Participants 14 | - (20 min) - Topic 4: CSP & Vendor Engagement/Demand, Participants 15 | - Next Steps & Closing Remarks, Ranny 16 | 17 | ## Brainstorming Output 18 | 19 | Topic 1: High Level Scope & Priorities 20 | 21 | ![image](2023-cloud-native-telco-day-chicago/images/Topic%201.png) 22 | 23 | Workshop feedback from online and in person participants: 24 | 25 | Platform and Workloads 26 | 27 | - Creating CNF <-> platform tests may not be feasible 28 | - There is too much variance between platforms 29 | - Better to focus on the CNF level tests 30 | - Platform needs to be defined as well as the workloads 31 | - If the scope is e2e, we should include platform testing to the first priority targets 32 | - We could have a layered scope (Sylva specific, Nephio specific, Google specific, etc.) 33 | - CNF vendor layer|cnf compatibility | platform compatibility 34 | - Creating common vendor neutral + upstream based reference for certification and conformance 35 | 36 | Types of tests/coverage 37 | 38 | - Distributed applications 39 | - Distributed workloads 40 | - Help needed with multi-cloud practices 41 | 42 | Community Collaboration and Output 43 | 44 | - Scope is hard to parse. What is the output and how does it relate to other projects? 45 | - Anuket works with GSMA and this is important to maintain to keep up with the industry 46 | - Need to coordinate between initiatives for practices not just definitions 47 | - Avoid having multiple programs for the same thing 48 | - Community interaction must be included (avoid a silo) 49 | - Connection with other implied cloud native telco ecosystems + standards bodies 50 | - Is this an open-source initiative, i.e. alignment within open source communities? 51 | - CSPs can call a referendum and veto proposed principles 52 | 53 | Other feedback/questions 54 | 55 | - Live lab, CI/CD, e2e lab 56 | - Provide local CI runners 57 | - Roles – Need to help define who brings what 58 | - Vendor CNF dashboard 59 | - Concern – how to assure to have good visibility of CNF requirements and specifics in CNCF 60 | - Secondary networking CLIS in K8s for telco network solutions 61 | - Zero trust networking – Security 62 | - Agree 63 | - Should live out CNF internal architecture of the scope (believe the comment is about ignoring internal CNF architecture from scope of the initiative) 64 | - Makes sense – practical & brings concrete value add 65 | - Clear linkage between end users (telco, CSPs) and the work output (especially testing programs) 66 | - Is the anticipated output best practice docs, test plans, and certification or testing program? Something else? 67 | 68 | [Post-it notes for Topic 1](2023-cloud-native-telco-day-chicago/images/Topic%201%20-%20Whiteboard.jpg) 69 | 70 | Topic 2: Positioning & Governance 71 | ![image](2023-cloud-native-telco-day-chicago/images/Topic%202.png) 72 | 73 | Workshop feedback from online and in person participants: 74 | 75 | Merging Scope 76 | 77 | - Less is more 78 | - Challenge - covering LFN and CNCF topics in same meeting 79 | - We should merge the overlapping activities, like CNF WG and RA2, CNF Test Suite and RC2 and testbed 80 | - We need to communicate what happens to Anuket Assured 81 | - Managing many different projects and initiatives will be challenging - initiative is vague 82 | 83 | Structure 84 | 85 | - Contributions by "PowerPoint" people should be possible. Requirements should be in English 86 | - Something new is better than Anuket or CNCF (Pull from multiple sides and buy-in needed) 87 | - Working group sounds natural for continuity 88 | - Agree this should be a new entity so it provides value to many LFN projects 89 | - New work group perhaps? These options need elaboration 90 | - No challenge for having this under LFN board 91 | - New structure preferred but optimize existing outcomes of current projects 92 | - More coherence and focus on c-n transformation would be beneficial instead of fit to all initiatives 93 | - It should not be a "resource" under LFN, but something what is a valid LFN construct like a project 94 | - Do we envision this governed as a project? It needs governance 95 | - TOC? to streamline north star goals and enabling coordination of workgroup(s) 96 | - Seems vague and disconnected which leads to accountability problems 97 | - Should the effort report metrics or accountability up to LFN board 98 | 99 | Output 100 | 101 | - We should be able to produce the GSMA docs like RM, RA1 and RA2 102 | - Should be good enough that Sylva uses it 103 | - Provide similar solutions 104 | - Less focus on tooling, more on engagement with CSP end users 105 | - Getting disconnected from c-n mainstream 106 | 107 | [Post-it notes for Topic 2](2023-cloud-native-telco-day-chicago/images/Topic%202%20-%20Whiteboard.jpg) 108 | 109 | Topic 3: Assets 110 | ![image](2023-cloud-native-telco-day-chicago/images/Topic%203.png) 111 | 112 | Workshop feedback from online and in person participants: 113 | 114 | Assets 115 | 116 | - Need a thorough audit of assets 117 | - The XGVela project is an asset to consider 118 | - Anuket assets are still mixed b/t CNTT and GH mirrors and using GH directly 119 | - No mention of the current test plans from Anuket and CNCF 120 | - Alignment to Sylva is a must 121 | - De-duplication should be higher priority than non interruption 122 | 123 | Onboarding and Contribution 124 | 125 | - Tools should not have high barriers to contribution 126 | - Self service to the extent possible is important 127 | - Any tools we decide upon needs to be well supported by LFN and easy to use/collaborate 128 | - Follow the LFN project processes and build on community agreement 129 | - Is the testing self-testing, in-lab testing, self delegation? 130 | - Autonomy (Independent to start but build bridge) 131 | 132 | Use Case Output 133 | 134 | - Providing guidance on tool output (test output) might make it easier to onboard new or existing tools 135 | - Vertical focus - tooling should be unopinionated 136 | - Is there a plan for functional vs performance testing? 137 | - How is the integrity of results being maintained or preserved (i.e. tool output -> cert or badge)? 138 | - Testing specifications from other organizations, e.g. standards, can these be reused/leveraged? 139 | - Local stand-up (dev workplace) 140 | - All of the ideas mentioned in the slide are good but it will not be easy to realize them. A substantial investment will be needed 141 | - Industry wide plugfests? 142 | 143 | [Post-it notes for Topic 3](2023-cloud-native-telco-day-chicago/images/Topic%203%20-%20Whiteboard.jpg) 144 | 145 | Topic 4: CSP and Vendor Engagement/Demand 146 | 147 | ![image](2023-cloud-native-telco-day-chicago/images/Topic%204.png) 148 | 149 | Workshop feedback from online and in person participants: 150 | 151 | Drive: Use Case, End Users, and Marketing 152 | 153 | - RFP/RFQ 154 | - We need cloud provider engagement as part of the ecosystem 155 | - Hyperscaler telco BU co-sell 156 | - We need to hear from CSPs how this initiative can/will help them 157 | - Select a use case and create a showcase to promote at some CSP forum to create support 158 | - MWC, GSMA 159 | - Confidence building in vertical events (MWC) 160 | - Ensure we capture use case from end users (CSPs) to develop output and drive testing or programs 161 | - Direct interests: Vendors, contributors 162 | - Affected interests: CSP, purchasers, consumers 163 | - Affected interests should have a veto (CSPs) 164 | - Provide success stories to incentivize the participation 165 | - Clear value proposition and focus: Reduce integration costs, more cloud native architecture, easier management 166 | - What would be vendors incentive to comply with cloud native? 167 | - Need a catchy name for the program 168 | - CSPs should incorporate that process in their RFx and vendor selection processes 169 | - Alignment on criteria/expectations from conformance, requirements, best practices, patterns 170 | 171 | Tools, Testing, Ease of Use 172 | 173 | - CSPs will want performance testing which will drive interest and involvement on their side - security as well 174 | - Usage in DevOps by the CSP (i.e. the tools and tests are directly used in their operational pipelines) 175 | - Test suite as a service (SaaS) 176 | 177 | Output, metrics, value 178 | 179 | - Downstream consumption 180 | - For CSPs to embrace any certification it has to bring value 181 | - Reward participation with awards 182 | - Testing conformance validation as a means to unblock CSP road to Cloud Native 183 | - CSPs have requested certification models and when delivered they have abandoned them 184 | - Comparisons between VNF and CNF systems demonstrate the cost savings or returns for end users 185 | - How can CSPs trust the outcomes from certification process? How to measure the quality of the certification? 186 | - Both CNF and platform vendors need to engage 187 | - Software and cloud vendors engagement and support will be crucial for practical outcomes of any certification program 188 | - Community engagement is the key. With the community we should identify a small number of focal points 189 | 190 | [Post-it notes for Topic 4](2023-cloud-native-telco-day-chicago/images/Topic%204%20-%20Whiteboard.jpg) 191 | -------------------------------------------------------------------------------- /events/README.md: -------------------------------------------------------------------------------- 1 | # Events for CNF WG and the CNCF Telecom community 2 | 3 | - [Telco Community Gathering - KubeCon EU 2023](telco-community-gathering-kubecon-eu-20230418.md) 4 | - Tuesday, April 18th, 2023 - In the afternoon following Cloud Native Telco Day 5 | 6 | - [LF Cloud Native Networking Program Alignment Workshop](LF-Cloud-Native-Networking-Program-Alignment-Workshop.md) 7 | - Tuesday, November 7th, 2023 a 2-4pm CST 8 | -------------------------------------------------------------------------------- /events/telco-community-gathering-kubecon-eu-20230418.md: -------------------------------------------------------------------------------- 1 | # Telco Community Gathering - KubeCon EU 2023 2 | 3 | 4 | | visual representation of the telco community gathering with tulips in Amsterdam with people sitting around together | **Telco Community Gathering - KubeCon EU 2023**
**Tuesday, April 18th, 2023**
**2:00-5:00 PM CEST**
**Taki Taki Co-working**
Address: [Coronalab.eu Amsterdam Zuid,
Hogelandplein 1a, 1079 RZ Amsterdam, Netherlands](https://goo.gl/maps/8USfbrArH7g6CjGw8) | | 5 | |:-:|:-:|:-:| 6 | 7 | ## A time (and place) during KubeCon where people can meet with the CNCF Telco community, including 8 | 9 | - [CNF Working Group](https://github.com/cncf/cnf-wg) - Documenting use cases, best practices, and collaborating with complimentary communities 10 | - [CNF Certification](https://github.com/cncf/cnf-certification/#readme) - What it is about and how to get started, including hands-on support 11 | - [CNF Test Suite](https://github.com/cncf/cnf-testsuite#cnf-test-suite) - Using it as a development and ops tool, integrate into CI systems 12 | - Birds of a Feather presentations - short 5-10 minute presentation on topic of interest 13 | - [Open discussion](https://github.com/lfn-cnti/bestpractices/discussions/257) - Time for various topics such as a wider discussion about CNF/Platform certification/interoperability approaches, problems, 14 | processes, vendor(s) involvements, communities such as Sylva, LFN Anuket, etc. 15 | 16 | ## When is it? 17 | 18 | Tuesday, April 18th, 2023 at 2-5pm (Central European Summer Time) 19 | 20 | - In the afternoon following [Cloud Native Telco Day](https://events.linuxfoundation.org/kubecon-cloudnativecon-europe/co-located-events/cloud-native-telco-day/) 21 | 22 | ## Where is it? 23 | 24 | Taki Taki Co-working: 25 | ~~Hogelandplein 1a, 1079 RZ Amsterdam, Netherlands~~ 26 | Google maps shows incorrect location. Please use the following address: [Coronalab.eu Amsterdam Zuid, Hogelandplein 1a, 1079 RZ Amsterdam, Netherlands](https://goo.gl/maps/8USfbrArH7g6CjGw8) 27 | 28 | | ![From Amsterdam RAI Europaplein 24, 1078 GZ Amsterdam, Netherlands, Head east on Europaplein. Turn right toward Europaboulevard. Turn left onto . Sharp right toward Maaslandstraat. Turn left onto Maaslandstraat. Turn right onto Gelrestraat. Turn left onto Zuidelijke Wandelweg. Turn right onto Hogelandplein](https://user-images.githubusercontent.com/26697/232744901-05000d2b-ec78-4665-9392-f899f8af010e.png) | ![map showing walking directions from RAI to Taki Taki Co-Working](https://user-images.githubusercontent.com/26697/230246177-8262bac4-54bd-4feb-bdaf-6a918a013913.png) | 29 | 30 | ## Who should attend? 31 | 32 | Registered KubeCon attendees (up to 50 people) 33 | 34 | ## Who is hosting/facilitating this gathering? 35 | 36 | CNF Working Group (WG) 37 | 38 | ## How do I sign up? 39 | 40 | RSVP at 41 | 42 | ---- 43 | 44 | ## What to do before the event? 45 | 46 | If you have 5 minutes (or less lighting talk), a topic idea for open discussion, or a working session tasks you would like to suggest (such as contributing a telecom term to the CNCF glossary) post a comment to this issue. You can also start a discussion in the [Discussions forum in this thread](https://github.com/cncf/cnf-wg/discussions/257). 47 | 48 | ## What to expect at the event? 49 | 50 | There will be an area designated for our community gathering. You should be prepared to show your badge or some ID related to your RSVP. There is a max of 50 people for this event and those that RSVP have first access with any that were not RSVP allowed as long as there is still room. 51 | 52 | - Topics can be proposed upon arrival at the gathering as well including for the BoF lighting talks until we are full 53 | - There will be drinks (both alcoholic and non-alcoholic including drink tickets for those who RSVPd) and snacks available 54 | - There will be internet and ability to present on a screen 55 | 56 | ---- 57 | 58 | ## Agenda 59 | 60 | See live notes here 61 | 62 | - Birds of Feather - (up to 5 minute talks) 63 | - TBD (add your topic in the comments of this issue) 64 | - Topic open discussions. Ideas suggested so far: 65 | - "telco validation/certification and the flexibility of CNI plugins" 66 | - "Probing CNFs in containerised environments" 67 | - Nephio 68 | - Anuket 69 | - Private 5G with a raspberry pie 70 | - Telco validation/certification and the flexibility of CNI plugins - Cilium / eBPF for Telco workloads 71 | - Open Discussion over CAMARA Exposure Gateway 72 | - Consider CNF on-boarding best practices in WG 73 | - Kubernetes LTS mechanism, the alignment in telco multi-vendor interoperability 74 | - Working session(s) 75 | - Draft + Submit Best practice PRs 76 | - Contribute to [CNCF glossary project](https://github.com/cncf/glossary) 77 | 78 | Note: The open discussion and working session time may overlap 79 | -------------------------------------------------------------------------------- /mlc_config.json: -------------------------------------------------------------------------------- 1 | { 2 | "aliveStatusCodes": [429, 200, 403] 3 | } 4 | -------------------------------------------------------------------------------- /tox.ini: -------------------------------------------------------------------------------- 1 | [tox] 2 | minversion = 3.15 3 | skipsdist = True 4 | envlist = spell 5 | 6 | [testenv] 7 | passenv = http_proxy,HTTP_PROXY,https_proxy,HTTPS_PROXY,no_proxy,NO_PROXY 8 | commands = 9 | find . -type f -name "*.pyc" -delete 10 | 11 | [testenv:spell] 12 | deps = 13 | pyspelling 14 | commands = pyspelling -c .spellcheck.yml 15 | -------------------------------------------------------------------------------- /use-case/5G_RAN/0001-UC-sync-timing-ran.md: -------------------------------------------------------------------------------- 1 | # UC-0001: Synchronization/Timing in Telco CNF world 2 | 3 | 4 | 5 | ## Glossary 6 | 7 | > RAN - Radio Access Network 8 | > DU - Distributed Unit support RLC, MAC, Physical layers 9 | > CU - Centralized Unit support SDAP, RRC, PDCP protocol layers 10 | > RU - Radio Unit 11 | > gNB - gNodeB 12 | - PTP - Precision Time Protocol 13 | 14 | 15 | 16 | ## Involved processes 17 | 18 | - [x] Development 19 | - [x] Deployment 20 | - [x] System integration 21 | - [x] Network integration 22 | - [ ] Lifecycle management 23 | - [x] Operations 24 | 25 | 26 | 27 | ## Involved actors / personas 28 | 29 | 30 | 31 | > Business customer of CSP 32 | Operations Team of CSP 33 | CNF Vendor 34 | Cloud provider 35 | Datacenter Operator/provider 36 | 37 | 38 | 39 | ## Involved system entities 40 | 41 | 42 | 43 | >Data Center Network 44 | Wide Area Network 45 | Cloud Native Platform 46 | CNF X 47 | RAN 48 | MANO 49 | 50 | 51 | 52 | ## Situation 53 | 54 | > 3GPP 5G gNB have a flexible architecture to support a wide range of deployments. The gNB can be split into CU, DU, and RU. The CU provides the support of a higher layer protocol stack(Layer 3 and above). The DU runs a lower-layer protocol stack(L1, L2), and RU does the radio and Low-PHY processing. The DU requires real-time performance and strict timing or synchronization to perform all the Layer 1, Layer 2 processing and scheduling. 55 | 56 | > To meet 5G requirements and processing demands, the network nodes must be synchronized. There are different ways to distribute the timing, like through GPS or any master PTP clock in the network with IEEE 1588v2 PTP. 57 | 58 | > In a non-virtualized network(DU/CU), the entire monolithic stack runs directly on ASIC or hardware at line rates, which aids in maintaining the synchronization/timing. 59 | 60 | > In container-based virtualized systems(vDU/vCU) or CNFs, the platform manages the workloads. In cloud native design, a monolithic vDU or vCU would result in many individual workloads. 5G has critical timing/synchronization requirements because of new services, new architectures and new technologies[1]. A time sync of ~1.5 microseconds or less is required to support the new spectrum and RAN features/applications[2]. The workloads need to maintain tight synchronization to meet such requirements. 61 | 62 | > The nature of the telco workload is compute-network intensive, demanding real-time performance. The orchestration platforms need to support CNFs to meet their timing synchronization requirements. 63 | 64 | 65 | 66 | ## Expected behaviour 67 | 68 | 69 | 70 | > The cloud native platform should be able to maintain the synchronization/timing of the telco workloads. 71 | 72 | 73 | 74 | ## Challenges and limitations with Kube-native approach 75 | 76 | 77 | 78 | > The K8s egress/ingress gateway is limited to web-based traffic. They are not aware of time stamped packets, like PTP packets. These packets are time-critical and they do not identify or process differently. The existing gateways might introduce a delay for many unknown reasons during the distribution of synchronization/timing packets to workloads. This unwanted delay may lead the workloads to lose their synchronization. The IEEE 1588v2 PTP have taken care of such scenarios of loss of synchronization, but the container platform may lack the capabilities internally that can enable the workloads to manage their synchronization/timing. 79 | 80 | 81 | 82 | ### Established practices to overcome challenges and limitations 83 | 84 | 85 | 86 | > To be developed 87 | 88 | 89 | 90 | ### What needs to be done differently in order to overcome challenges and limitations 91 | 92 | 93 | 94 | > Kubernetes could have a time-sensitive gateway or a particular gateway that keeps track of synchronization/timing distributed from the network or GPS to workloads. This would help workload maintain their synchronization across the network. There could also be a correction mechanism internally in the platform to maintain the synchronization/timing. 95 | 96 | 97 | 98 | ### References 99 | 100 | > [1] [https://wsts.atis.org/wp-content/uploads/sites/9/2018/11/2-01_China-Mobile_Li_5G-Synchronization_v5.pdf](https://wsts.atis.org/wp-content/uploads/sites/9/2018/11/2-01_China-Mobile_Li_5G-Synchronization_v5.pdf) 101 | 102 | > [2] [https://www.ericsson.com/en/blog/2019/8/what-you-need-to-know-about-timing-and-sync-in-5g-transport-networks](https://www.ericsson.com/en/blog/2019/8/what-you-need-to-know-about-timing-and-sync-in-5g-transport-networks) 103 | -------------------------------------------------------------------------------- /use-case/5G_RAN/0002-UC-multi-interface.md: -------------------------------------------------------------------------------- 1 | # UC-0002: Support of Multi-Communication Interface in Telco CNF world 2 | 3 | ## Glossary 4 | 5 | > RAN - Radio Access Network 6 | > DU - Distributed Unit support RLC, MAC, Physical layers 7 | > CU - Centralized Unit support SDAP, RRC, PDCP protocol layers 8 | > RU - Radio Unit 9 | > gNB - gNodeB 10 | > IPC - Inter-Process Communication 11 | 12 | ## Involved processes 13 | 14 | - [x] Development 15 | - [x] Deployment 16 | - [x] System integration 17 | - [x] Network integration 18 | - [ ] Lifecycle management 19 | 20 | 21 | ## Involved actors / personas 22 | 23 | > Business customer of CSP 24 | CNF Vendor 25 | Cloud provider 26 | Datacenter Operator/provider 27 | 28 | ## Involved system entities 29 | 30 | >Data Center Network 31 | Wide Area Network 32 | Cloud Native Platform 33 | CNF X 34 | RAN 35 | MANO 36 | 37 | ## Situation 38 | 39 | > 3GPP 5G gNB have a flexible architecture to support a wide range of deployments. The gNB can be split into CU, DU, and RU. The CU provides the support of a higher layer protocol stack(Layer 3 and above). The DU runs a lower-layer protocol stack(L1, L2), and RU does the radio and Low-PHY processing. The DU requires real-time performance and strict timing/synchronization to perform all the Layer 1, Layer 2 processing and scheduling. 40 | > In a non-virtualized networks (DU/CU), the entire monolithic stack(DU), including the baseband processing, runs directly on ASIC or hardware to achieve real-time performance. 41 | > Consider a DU, which runs the PHY, RLC, MAC layers of the protocol stack. In a monolithic black box non-virtualized DU, the interface between RLC and MAC is a proprietary interface and vendor-driven. This interface could be some form of IPC mechanism or a simple function call. The interface between PHY and MAC is FAPI/nFAPI, a open interface standardized by Small Cell Forum (SCF). These proprietary interfaces, mainly designed for high-performance, low overhead, help to achieve real-time performance. 42 | > In virtualized systems(vDU/vCU CNFs), the platform manages the workloads. With a cloud native design, the monolithic DU or CU would be split into many individual components as per the design, like workloads for different layers of the protocol stack, procedures, state management, user context storage, management and administration functionalities, etc. 43 | > The nature of the telco workload is compute-network intensive, demanding real-time performance. An efficient and performance-driven communication interface between CNFs is required to meet the real-time performance. 44 | 45 | > The Kubernetes model of Pod-to-Pod communication is network-based or IPC based. 46 | > Any network-based communication over IP between CNFs could lead to degradation in performance because of the additional communication overhead they create, like building a IP packet, processing through the network stack, in comparison with other IPC mechanisms. This could increase the latency between CNFs communication. 47 | >The native K8s supports IPC within the Pod and IP communication outside the Pod. It is not clear on the communication between Pods over IPC mechanisms. 48 | >There is no support for vendors to plug their methods of communication between CNFs. 49 | 50 | ## Expected behaviour 51 | 52 | > The cloud-native orchestration/container platforms should support the multiple IPC mechanisms and multiple IPC endpoints communication. 53 | ## Challenges and limitations with Kube-native approach 54 | 55 | >Kube-native approach is suitable for IP based networking. The telco workloads might require IPC based mechanism for CNFs communication. 56 | 57 | ### Established practices to overcome challenges and limitations 58 | 59 | > Not applicable 60 | 61 | ### What needs to be done differently in order to overcome challenges and limitations 62 | 63 | >The K8s or container platforms can have plugin for IPC based communication like Multus CNI, enabling a POD to have multiple network interfaces. 64 | >A plugin for IPC might be required, enabling the POD/CNF to support multiple IPC based endpoints and mechanisms. 65 | >There should be some provision for vendors to plug their IPC mechanisms. 66 | --------------------------------------------------------------------------------