├── .gitignore ├── .gitmodules ├── LICENSE ├── README.md ├── images ├── architecture-detail.png ├── architecture-h-and-s.png ├── architecture.drawio ├── firewall-01.drawio ├── firewall-01.png └── front-door │ ├── aks-service-lb.png │ ├── approve-pls.png │ ├── front-door.drawio │ ├── front-door.png │ ├── pls-unapproved.png │ ├── pls-unconnected.png │ ├── resources.png │ └── result.png ├── modules-arm ├── front-door-1.json ├── front-door-2.json └── hub-spoke-aks.json ├── modules ├── aks-vote-app │ ├── aks-vote-app.bicep │ ├── aks-vote-app.yaml │ └── bicepconfig.json ├── aks │ └── aks.bicep ├── front-door │ ├── front-door-1.bicep │ └── front-door-2.bicep ├── hub-spoke-aks │ └── hub-spoke-aks.bicep └── test │ ├── front-door-only-test.bicep │ ├── private-link-test.bicep │ ├── private-link-test.json │ ├── vnet-test.bicep │ └── yaml-bootstrap-test.bicep ├── scenarios ├── confidential-01.md ├── containerinsights.md ├── firewall-01.md ├── firewall-routing.md └── front-door.md └── testPR /.gitignore: -------------------------------------------------------------------------------- 1 | ################################################################################ 2 | # This .gitignore file was automatically created by Microsoft(R) Visual Studio. 3 | ################################################################################ 4 | 5 | /.vs 6 | /bin 7 | /obj 8 | 9 | #VSCODE 10 | .vscode/* 11 | !.vscode/settings.json 12 | !.vscode/tasks.json 13 | !.vscode/launch.json 14 | !.vscode/extensions.json 15 | !.vscode/*.code-snippets 16 | 17 | # Local History for Visual Studio Code 18 | .history/ 19 | 20 | # Built Visual Studio Code Extensions 21 | *.vsix 22 | #END OF VSCODE 23 | #VS 24 | # User-specific files 25 | *.rsuser 26 | *.suo 27 | *.user 28 | *.userosscache 29 | *.sln.docstates 30 | 31 | # User-specific files (MonoDevelop/Xamarin Studio) 32 | *.userprefs 33 | 34 | # Mono auto generated files 35 | mono_crash.* 36 | 37 | # Build results 38 | [Dd]ebug/ 39 | [Dd]ebugPublic/ 40 | [Rr]elease/ 41 | [Rr]eleases/ 42 | x64/ 43 | x86/ 44 | [Ww][Ii][Nn]32/ 45 | [Aa][Rr][Mm]/ 46 | [Aa][Rr][Mm]64/ 47 | bld/ 48 | [Bb]in/ 49 | [Oo]bj/ 50 | [Ll]og/ 51 | [Ll]ogs/ 52 | 53 | # Visual Studio 2015/2017 cache/options directory 54 | .vs/ 55 | # Uncomment if you have tasks that create the project's static files in wwwroot 56 | #wwwroot/ 57 | 58 | # Visual Studio 2017 auto generated files 59 | Generated\ Files/ 60 | 61 | # MSTest test Results 62 | [Tt]est[Rr]esult*/ 63 | [Bb]uild[Ll]og.* 64 | 65 | # NUnit 66 | *.VisualState.xml 67 | TestResult.xml 68 | nunit-*.xml 69 | 70 | # Build Results of an ATL Project 71 | [Dd]ebugPS/ 72 | [Rr]eleasePS/ 73 | dlldata.c 74 | 75 | # Benchmark Results 76 | BenchmarkDotNet.Artifacts/ 77 | 78 | # .NET Core 79 | project.lock.json 80 | project.fragment.lock.json 81 | artifacts/ 82 | 83 | # ASP.NET Scaffolding 84 | ScaffoldingReadMe.txt 85 | 86 | # StyleCop 87 | StyleCopReport.xml 88 | 89 | # Files built by Visual Studio 90 | *_i.c 91 | *_p.c 92 | *_h.h 93 | *.ilk 94 | *.meta 95 | *.obj 96 | *.iobj 97 | *.pch 98 | *.pdb 99 | *.ipdb 100 | *.pgc 101 | *.pgd 102 | *.rsp 103 | *.sbr 104 | *.tlb 105 | *.tli 106 | *.tlh 107 | *.tmp 108 | *.tmp_proj 109 | *_wpftmp.csproj 110 | *.log 111 | *.tlog 112 | *.vspscc 113 | *.vssscc 114 | .builds 115 | *.pidb 116 | *.svclog 117 | *.scc 118 | 119 | # Chutzpah Test files 120 | _Chutzpah* 121 | 122 | # Visual C++ cache files 123 | ipch/ 124 | *.aps 125 | *.ncb 126 | *.opendb 127 | *.opensdf 128 | *.sdf 129 | *.cachefile 130 | *.VC.db 131 | *.VC.VC.opendb 132 | 133 | # Visual Studio profiler 134 | *.psess 135 | *.vsp 136 | *.vspx 137 | *.sap 138 | 139 | # Visual Studio Trace Files 140 | *.e2e 141 | 142 | # TFS 2012 Local Workspace 143 | $tf/ 144 | 145 | # Guidance Automation Toolkit 146 | *.gpState 147 | 148 | # ReSharper is a .NET coding add-in 149 | _ReSharper*/ 150 | *.[Rr]e[Ss]harper 151 | *.DotSettings.user 152 | 153 | # TeamCity is a build add-in 154 | _TeamCity* 155 | 156 | # DotCover is a Code Coverage Tool 157 | *.dotCover 158 | 159 | # AxoCover is a Code Coverage Tool 160 | .axoCover/* 161 | !.axoCover/settings.json 162 | 163 | # Coverlet is a free, cross platform Code Coverage Tool 164 | coverage*.json 165 | coverage*.xml 166 | coverage*.info 167 | 168 | # Visual Studio code coverage results 169 | *.coverage 170 | *.coveragexml 171 | 172 | # NCrunch 173 | _NCrunch_* 174 | .*crunch*.local.xml 175 | nCrunchTemp_* 176 | 177 | # MightyMoose 178 | *.mm.* 179 | AutoTest.Net/ 180 | 181 | # Web workbench (sass) 182 | .sass-cache/ 183 | 184 | # Installshield output folder 185 | [Ee]xpress/ 186 | 187 | # DocProject is a documentation generator add-in 188 | DocProject/buildhelp/ 189 | DocProject/Help/*.HxT 190 | DocProject/Help/*.HxC 191 | DocProject/Help/*.hhc 192 | DocProject/Help/*.hhk 193 | DocProject/Help/*.hhp 194 | DocProject/Help/Html2 195 | DocProject/Help/html 196 | 197 | # Click-Once directory 198 | publish/ 199 | 200 | # Publish Web Output 201 | *.[Pp]ublish.xml 202 | *.azurePubxml 203 | # Note: Comment the next line if you want to checkin your web deploy settings, 204 | # but database connection strings (with potential passwords) will be unencrypted 205 | *.pubxml 206 | *.publishproj 207 | 208 | # Microsoft Azure Web App publish settings. Comment the next line if you want to 209 | # checkin your Azure Web App publish settings, but sensitive information contained 210 | # in these scripts will be unencrypted 211 | PublishScripts/ 212 | 213 | # NuGet Packages 214 | *.nupkg 215 | # NuGet Symbol Packages 216 | *.snupkg 217 | # The packages folder can be ignored because of Package Restore 218 | **/[Pp]ackages/* 219 | # except build/, which is used as an MSBuild target. 220 | !**/[Pp]ackages/build/ 221 | # Uncomment if necessary however generally it will be regenerated when needed 222 | #!**/[Pp]ackages/repositories.config 223 | # NuGet v3's project.json files produces more ignorable files 224 | *.nuget.props 225 | *.nuget.targets 226 | 227 | # Microsoft Azure Build Output 228 | csx/ 229 | *.build.csdef 230 | 231 | # Microsoft Azure Emulator 232 | ecf/ 233 | rcf/ 234 | 235 | # Windows Store app package directories and files 236 | AppPackages/ 237 | BundleArtifacts/ 238 | Package.StoreAssociation.xml 239 | _pkginfo.txt 240 | *.appx 241 | *.appxbundle 242 | *.appxupload 243 | 244 | # Visual Studio cache files 245 | # files ending in .cache can be ignored 246 | *.[Cc]ache 247 | # but keep track of directories ending in .cache 248 | !?*.[Cc]ache/ 249 | 250 | # Others 251 | ClientBin/ 252 | ~$* 253 | *~ 254 | *.dbmdl 255 | *.dbproj.schemaview 256 | *.jfm 257 | *.pfx 258 | *.publishsettings 259 | orleans.codegen.cs 260 | 261 | # Including strong name files can present a security risk 262 | # (https://github.com/github/gitignore/pull/2483#issue-259490424) 263 | #*.snk 264 | 265 | # Since there are multiple workflows, uncomment next line to ignore bower_components 266 | # (https://github.com/github/gitignore/pull/1529#issuecomment-104372622) 267 | #bower_components/ 268 | 269 | # RIA/Silverlight projects 270 | Generated_Code/ 271 | 272 | # Backup & report files from converting an old project file 273 | # to a newer Visual Studio version. Backup files are not needed, 274 | # because we have git ;-) 275 | _UpgradeReport_Files/ 276 | Backup*/ 277 | UpgradeLog*.XML 278 | UpgradeLog*.htm 279 | ServiceFabricBackup/ 280 | *.rptproj.bak 281 | 282 | # SQL Server files 283 | *.mdf 284 | *.ldf 285 | *.ndf 286 | 287 | # Business Intelligence projects 288 | *.rdl.data 289 | *.bim.layout 290 | *.bim_*.settings 291 | *.rptproj.rsuser 292 | *- [Bb]ackup.rdl 293 | *- [Bb]ackup ([0-9]).rdl 294 | *- [Bb]ackup ([0-9][0-9]).rdl 295 | 296 | # Microsoft Fakes 297 | FakesAssemblies/ 298 | 299 | # GhostDoc plugin setting file 300 | *.GhostDoc.xml 301 | 302 | # Node.js Tools for Visual Studio 303 | .ntvs_analysis.dat 304 | node_modules/ 305 | 306 | # Visual Studio 6 build log 307 | *.plg 308 | 309 | # Visual Studio 6 workspace options file 310 | *.opt 311 | 312 | # Visual Studio 6 auto-generated workspace file (contains which files were open etc.) 313 | *.vbw 314 | 315 | # Visual Studio 6 auto-generated project file (contains which files were open etc.) 316 | *.vbp 317 | 318 | # Visual Studio 6 workspace and project file (working project files containing files to include in project) 319 | *.dsw 320 | *.dsp 321 | 322 | # Visual Studio 6 technical files 323 | *.ncb 324 | *.aps 325 | 326 | # Visual Studio LightSwitch build output 327 | **/*.HTMLClient/GeneratedArtifacts 328 | **/*.DesktopClient/GeneratedArtifacts 329 | **/*.DesktopClient/ModelManifest.xml 330 | **/*.Server/GeneratedArtifacts 331 | **/*.Server/ModelManifest.xml 332 | _Pvt_Extensions 333 | 334 | # Paket dependency manager 335 | .paket/paket.exe 336 | paket-files/ 337 | 338 | # FAKE - F# Make 339 | .fake/ 340 | 341 | # CodeRush personal settings 342 | .cr/personal 343 | 344 | # Python Tools for Visual Studio (PTVS) 345 | __pycache__/ 346 | *.pyc 347 | 348 | # Cake - Uncomment if you are using it 349 | # tools/** 350 | # !tools/packages.config 351 | 352 | # Tabs Studio 353 | *.tss 354 | 355 | # Telerik's JustMock configuration file 356 | *.jmconfig 357 | 358 | # BizTalk build output 359 | *.btp.cs 360 | *.btm.cs 361 | *.odx.cs 362 | *.xsd.cs 363 | 364 | # OpenCover UI analysis results 365 | OpenCover/ 366 | 367 | # Azure Stream Analytics local run output 368 | ASALocalRun/ 369 | 370 | # MSBuild Binary and Structured Log 371 | *.binlog 372 | 373 | # NVidia Nsight GPU debugger configuration file 374 | *.nvuser 375 | 376 | # MFractors (Xamarin productivity tool) working folder 377 | .mfractor/ 378 | 379 | # Local History for Visual Studio 380 | .localhistory/ 381 | 382 | # Visual Studio History (VSHistory) files 383 | .vshistory/ 384 | 385 | # BeatPulse healthcheck temp database 386 | healthchecksdb 387 | 388 | # Backup folder for Package Reference Convert tool in Visual Studio 2017 389 | MigrationBackup/ 390 | 391 | # Ionide (cross platform F# VS Code tools) working folder 392 | .ionide/ 393 | 394 | # Fody - auto-generated XML schema 395 | FodyWeavers.xsd 396 | 397 | # VS Code files for those working on multiple tools 398 | .vscode/* 399 | !.vscode/settings.json 400 | !.vscode/tasks.json 401 | !.vscode/launch.json 402 | !.vscode/extensions.json 403 | *.code-workspace 404 | 405 | # Local History for Visual Studio Code 406 | .history/ 407 | 408 | # Windows Installer files from build outputs 409 | *.cab 410 | *.msi 411 | *.msix 412 | *.msm 413 | *.msp 414 | 415 | # JetBrains Rider 416 | *.sln.iml 417 | #END OF VS 418 | 419 | 420 | *.bkp 421 | *.dtmp 422 | 423 | -------------------------------------------------------------------------------- /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "hub-and-spoke-playground"] 2 | path = hub-and-spoke-playground 3 | url = https://github.com/nicolgit/hub-and-spoke-playground 4 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 NicolD 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 |

the AKS (Azure Kubernetes Service) 🤯
playground

2 | 3 |
4 | Built with ❤︎ by 5 | nicolgit, lucapisano and mela125 6 | 7 |
8 |

9 | 10 | ![hub and spoke](images/architecture-h-and-s.png) 11 | 12 | _Download the [draw.io file](images/architecture.drawio) of this schema_ 13 | 14 | This repo contains a preconfigured Azure Kubernetes Service cluster embedded inside an hub-and-spoke network topology, aligned to the Azure enterprise-scale landing zone reference architecture, useful for testing and studying network configurations in a controlled, repeatable environment. 15 | 16 | As bonus many scenarios with step-by-step solutions for studying and learning are also available. 17 | 18 | The "playground" is composed by: 19 | * a hub and spoke network topologies aligned with the Microsoft Enterprise scale landing zone reference architecture 20 | * an AKS cluster deployed in one spoke 21 | * routing table(s) and firewall policy configured so that all the AKS outbound traffic is routed through the firewall 22 | 23 | # Deploy to Azure 24 | 25 | You can use the following button to deploy the demo to your Azure subscription: 26 | 27 | | |   |   | 28 | |---|---|---| 29 | |1| the **AKS** playground
deploys `hub-lab-net` spokes `01`-`02`-`03` an `AKS Cluster` and all the required routing | [![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fnicolgit%2Fthe-aks-playground%2Fmain%2Fmodules-arm%2Fhub-spoke-aks.json) 30 | 31 | 32 | # Architecture 33 | 34 | This diagram shows a detailed version with also all subnets, virtual machines, NVAs, IPs and Firewalls. 35 | 36 | ![detailed architecture](images/architecture-detail.png) 37 | _Download the [draw.io file](images/architecture.drawio) of this schema._ 38 | 39 | the ARM template [hub-spoke-aks.json](modules-arm/hub-spoke-aks.json) deploys: 40 | 41 | * 4 Azure Virtual Networks: 42 | * `hub-lab-net` with 4 subnets: 43 | * an empty `default` subnet 44 | * AzureFirewallSubet: a subnet is used by Azure Firewall 45 | * AzureBastionSubnet: a subnet is used by Azure Bastion 46 | * GatewaySubnet: a subnet ready to deploy an by Azure Virtual Network Gateway 47 | * `spoke-01` with 2 subnets `default` and `services` 48 | * `spoke-02` with 2 subnets `default` and `services` 49 | * `spoke-03` with 2 subnets `default` and `services` 50 | * `lab-firewall`: an Azure Firewall **premium** on the `hub-lab-net` network 51 | * `my-firewall-policy`: a sample policy that implements the any-to-any routing between spokes and all the internet traffic outbound 52 | * `all-to-firewall-we` and `all-to-firewall-ne`: route tables that forward all the outbound traffic through the central Azure Firewall 53 | * `hub-playground-ws`: a log analytics workspace where to collect all firewall logs 54 | * `aks-01`: an Azure Kubernetes Service cluster deployed on `services` `spoke-01` subnet 55 | 56 | `aks-01` cluster has 1 node pool and 2 sample workload deployed: `azure vote front` and `azure vote back` taken from the [Microsoft Artifact Registry](https://mcr.microsoft.com/): a super-simple front-end/back-end application that exposes a sample UI over HTTP. 57 | To test the workload, you need to know the IP of the front-end load balancer. 58 | _Because we're using Azure CNI, you could also have used the pod IP. But keep in mind that pods are volatile, so in a Kubernetes context it is always advisable to use the load balancer IP. _ 59 | 60 | You can find this IP in: 61 | * Azure Portal > `aks-01` > Services and ingresses > `azure-vote-front` > Services > `azure-vote-front` > External IP (something like **10.13.1.y**) 62 | 63 | To test it: access to `hub-vm-01` in RDP/bastion and open in Edge `http://x.x.x.x` (where `x.x.x.x` is the IP found above) 64 | 65 | 66 | ## Playground's scenarios 67 | Here there is a list of tested scenarios usable on this playground. 68 | 69 | For each scenario you have: 70 | 71 | * **prerequisites**: component to deploy required to implement the solution (only the hub, also one on-prem playground or both) 72 | * **solution**: a step-by-step sequence to implement the solution 73 | * **test solution**: a procedure to follow, to verify if the scenario is working as expected 74 | 75 | 76 | | | scenario description | solution | 77 | |---|---|---| 78 | | 1 | Deploy a confidential computing nodes pool | [see the documentation](scenarios/confidential-01.md) | 79 | | 2 | Expose a workload from AKS with Azure Front Door | [see the documentation](scenarios/front-door.md) | 80 | | 3 | Expose a workload from AKS with Azure Firewall | [see the documentation](scenarios/firewall-01.md) | 81 | | 4 | Deploy Container Insights on AKS Cluster | [see the documentation](scenarios/containerinsights.md) | -------------------------------------------------------------------------------- /images/architecture-detail.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolgit/the-aks-playground/9177f9a510c8acf0b28258ca1985f6a317a19db6/images/architecture-detail.png -------------------------------------------------------------------------------- /images/architecture-h-and-s.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolgit/the-aks-playground/9177f9a510c8acf0b28258ca1985f6a317a19db6/images/architecture-h-and-s.png -------------------------------------------------------------------------------- /images/architecture.drawio: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | 107 | 108 | 109 | 110 | 111 | 112 | 113 | 114 | 115 | 116 | 117 | 118 | 119 | 120 | 121 | 122 | 123 | 124 | 125 | 126 | 127 | 128 | 129 | 130 | 131 | 132 | 133 | 134 | 135 | 136 | 137 | 138 | 139 | 140 | 141 | 142 | 143 | 144 | 145 | 146 | 147 | 148 | 149 | 150 | 151 | 152 | 153 | 154 | 155 | 156 | 157 | 158 | 159 | 160 | 161 | 162 | 163 | 164 | 165 | 166 | 167 | 168 | 169 | 170 | 171 | 172 | 173 | 174 | 175 | 176 | 177 | 178 | 179 | 180 | 181 | 182 | 183 | 184 | 185 | 186 | 187 | 188 | 189 | 190 | 191 | 192 | 193 | 194 | 195 | 196 | 197 | 198 | 199 | 200 | 201 | 202 | 203 | 204 | 205 | 206 | 207 | 208 | 209 | 210 | 211 | 212 | 213 | 214 | 215 | 216 | 217 | 218 | 219 | 220 | 221 | 222 | 223 | 224 | 225 | 226 | 227 | 228 | 229 | 230 | 231 | 232 | 233 | 234 | 235 | 236 | 237 | 238 | 239 | 240 | 241 | 242 | 243 | 244 | 245 | 246 | 247 | 248 | 249 | 250 | 251 | 252 | 253 | 254 | 255 | 256 | 257 | 258 | 259 | 260 | 261 | 262 | 263 | 264 | 265 | 266 | 267 | 268 | 269 | 270 | 271 | 272 | 273 | 274 | 275 | 276 | 277 | 278 | 279 | 280 | 281 | 282 | 283 | 284 | 285 | 286 | 287 | 288 | 289 | 290 | 291 | 292 | 293 | 294 | 295 | 296 | 297 | 298 | 299 | 300 | 301 | 302 | 303 | 304 | 305 | 306 | 307 | 308 | 309 | 310 | 311 | 312 | 313 | 314 | 315 | 316 | 317 | 318 | 319 | 320 | 321 | 322 | 323 | 324 | 325 | 326 | 327 | 328 | 329 | 330 | 331 | 332 | 333 | 334 | 335 | 336 | 337 | 338 | 339 | 340 | 341 | 342 | 343 | 344 | 345 | 346 | 347 | 348 | 349 | 350 | 351 | 352 | 353 | 354 | 355 | 356 | 357 | 358 | 359 | 360 | 361 | 362 | 363 | 364 | 365 | 366 | 367 | 368 | 369 | 370 | 371 | 372 | 373 | 374 | 375 | 376 | 377 | 378 | 379 | 380 | 381 | 382 | 383 | 384 | 385 | 386 | 387 | 388 | 389 | 390 | 391 | 392 | 393 | 394 | 395 | 396 | 397 | 398 | -------------------------------------------------------------------------------- /images/firewall-01.drawio: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | 107 | 108 | 109 | 110 | 111 | 112 | 113 | 114 | 115 | 116 | 117 | 118 | 119 | 120 | 121 | 122 | 123 | 124 | 125 | 126 | 127 | 128 | 129 | 130 | 131 | 132 | 133 | 134 | 135 | 136 | 137 | 138 | 139 | 140 | 141 | 142 | 143 | 144 | 145 | 146 | 147 | 148 | 149 | 150 | 151 | 152 | 153 | 154 | 155 | 156 | 157 | 158 | 159 | 160 | 161 | 162 | 163 | 164 | 165 | 166 | 167 | 168 | 169 | 170 | 171 | 172 | 173 | 174 | 175 | 176 | 177 | 178 | 179 | 180 | 181 | 182 | 183 | 184 | 185 | 186 | 187 | 188 | 189 | 190 | 191 | 192 | 193 | 194 | 195 | 196 | 197 | 198 | 199 | 200 | 201 | 202 | 203 | 204 | 205 | 206 | 207 | 208 | 209 | 210 | 211 | 212 | 213 | 214 | 215 | 216 | 217 | 218 | 219 | 220 | 221 | 222 | 223 | 224 | 225 | 226 | 227 | 228 | 229 | -------------------------------------------------------------------------------- /images/firewall-01.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolgit/the-aks-playground/9177f9a510c8acf0b28258ca1985f6a317a19db6/images/firewall-01.png -------------------------------------------------------------------------------- /images/front-door/aks-service-lb.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolgit/the-aks-playground/9177f9a510c8acf0b28258ca1985f6a317a19db6/images/front-door/aks-service-lb.png -------------------------------------------------------------------------------- /images/front-door/approve-pls.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolgit/the-aks-playground/9177f9a510c8acf0b28258ca1985f6a317a19db6/images/front-door/approve-pls.png -------------------------------------------------------------------------------- /images/front-door/front-door.drawio: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 74 | 75 | 76 | 77 | 78 | 79 | 80 | 81 | 82 | 83 | 84 | 85 | 86 | 87 | 88 | 89 | 90 | 91 | 92 | 93 | 94 | 95 | 96 | 97 | 98 | 99 | 100 | 101 | 102 | 103 | 104 | 105 | 106 | 107 | 108 | 109 | 110 | 111 | 112 | 113 | 114 | 115 | 116 | 117 | 118 | 119 | 120 | 121 | 122 | 123 | 124 | 125 | 126 | 127 | 128 | 129 | 130 | 131 | 132 | 133 | 134 | 135 | 136 | 137 | 138 | 139 | 140 | 141 | 142 | 143 | 144 | 145 | 146 | 147 | 148 | 149 | 150 | 151 | 152 | 153 | 154 | 155 | 156 | 157 | 158 | 159 | 160 | 161 | 162 | 163 | 164 | 165 | 166 | 167 | 168 | 169 | 170 | 171 | 172 | 173 | 174 | 175 | 176 | 177 | 178 | 179 | 180 | 181 | 182 | 183 | 184 | 185 | 186 | 187 | 188 | 189 | 190 | 191 | 192 | 193 | -------------------------------------------------------------------------------- /images/front-door/front-door.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolgit/the-aks-playground/9177f9a510c8acf0b28258ca1985f6a317a19db6/images/front-door/front-door.png -------------------------------------------------------------------------------- /images/front-door/pls-unapproved.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolgit/the-aks-playground/9177f9a510c8acf0b28258ca1985f6a317a19db6/images/front-door/pls-unapproved.png -------------------------------------------------------------------------------- /images/front-door/pls-unconnected.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolgit/the-aks-playground/9177f9a510c8acf0b28258ca1985f6a317a19db6/images/front-door/pls-unconnected.png -------------------------------------------------------------------------------- /images/front-door/resources.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolgit/the-aks-playground/9177f9a510c8acf0b28258ca1985f6a317a19db6/images/front-door/resources.png -------------------------------------------------------------------------------- /images/front-door/result.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/nicolgit/the-aks-playground/9177f9a510c8acf0b28258ca1985f6a317a19db6/images/front-door/result.png -------------------------------------------------------------------------------- /modules-arm/front-door-1.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", 3 | "contentVersion": "1.0.0.0", 4 | "metadata": { 5 | "_generator": { 6 | "name": "bicep", 7 | "version": "0.18.4.5664", 8 | "templateHash": "282938552662850812" 9 | } 10 | }, 11 | "variables": { 12 | "clusterName": "aks-01", 13 | "clusterResourceGroup": "[format('cl01-{0}-rg', variables('clusterName'))]", 14 | "privateLinkServiceName": "pl-aks-01", 15 | "location": "westeurope" 16 | }, 17 | "resources": [ 18 | { 19 | "type": "Microsoft.Resources/deployments", 20 | "apiVersion": "2022-09-01", 21 | "name": "hub-spoke-deploy", 22 | "properties": { 23 | "expressionEvaluationOptions": { 24 | "scope": "inner" 25 | }, 26 | "mode": "Incremental", 27 | "parameters": { 28 | "location": { 29 | "value": "[variables('location')]" 30 | }, 31 | "locationSpoke03": { 32 | "value": "[variables('location')]" 33 | }, 34 | "firewallTier": { 35 | "value": "Premium" 36 | }, 37 | "deployBastion": { 38 | "value": false 39 | }, 40 | "deployGateway": { 41 | "value": false 42 | }, 43 | "deployVmHub": { 44 | "value": false 45 | }, 46 | "deployVm01": { 47 | "value": false 48 | }, 49 | "deployVm02": { 50 | "value": false 51 | }, 52 | "deployVm03": { 53 | "value": false 54 | } 55 | }, 56 | "template": { 57 | "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", 58 | "languageVersion": "1.10-experimental", 59 | "contentVersion": "1.0.0.0", 60 | "metadata": { 61 | "_EXPERIMENTAL_WARNING": "Symbolic name support in ARM is experimental, and should be enabled for testing purposes only. Do not enable this setting for any production usage, or you may be unexpectedly broken at any time!", 62 | "_generator": { 63 | "name": "bicep", 64 | "version": "0.18.4.5664", 65 | "templateHash": "3643799897607885012" 66 | } 67 | }, 68 | "parameters": { 69 | "location": { 70 | "type": "string", 71 | "defaultValue": "westeurope" 72 | }, 73 | "locationSpoke03": { 74 | "type": "string", 75 | "defaultValue": "northeurope" 76 | }, 77 | "firewallTier": { 78 | "type": "string", 79 | "defaultValue": "Premium", 80 | "allowedValues": [ 81 | "Basic", 82 | "Standard", 83 | "Premium" 84 | ], 85 | "metadata": { 86 | "description": "Basic, Standard or Premium tier" 87 | } 88 | }, 89 | "deployBastion": { 90 | "type": "bool", 91 | "defaultValue": true 92 | }, 93 | "deployGateway": { 94 | "type": "bool", 95 | "defaultValue": true 96 | }, 97 | "deployVmHub": { 98 | "type": "bool", 99 | "defaultValue": true 100 | }, 101 | "deployVm01": { 102 | "type": "bool", 103 | "defaultValue": true 104 | }, 105 | "deployVm02": { 106 | "type": "bool", 107 | "defaultValue": true 108 | }, 109 | "deployVm03": { 110 | "type": "bool", 111 | "defaultValue": true 112 | }, 113 | "username": { 114 | "type": "string", 115 | "defaultValue": "nicola", 116 | "metadata": { 117 | "description": "username administrator for all VMs" 118 | } 119 | }, 120 | "password": { 121 | "type": "securestring", 122 | "defaultValue": "password.123", 123 | "metadata": { 124 | "description": "username administrator password for all VMs" 125 | } 126 | }, 127 | "virtualMachineSKU": { 128 | "type": "string", 129 | "defaultValue": "Standard_D2s_v3" 130 | } 131 | }, 132 | "variables": { 133 | "firewallName": "lab-firewall", 134 | "bastionName": "lab-bastion", 135 | "vnetGatewayName": "lab-gateway", 136 | "hublabName": "hub-lab-net", 137 | "spoke01Name": "spoke-01", 138 | "spoke02Name": "spoke-02", 139 | "spoke03Name": "spoke-03", 140 | "vmHubName": "hub-vm", 141 | "vm01Name": "[format('{0}-vm', variables('spoke01Name'))]", 142 | "vm02Name": "[format('{0}-vm', variables('spoke02Name'))]", 143 | "vm03Name": "[format('{0}-vm', variables('spoke03Name'))]", 144 | "firewallIPName": "[format('{0}-ip', variables('firewallName'))]", 145 | "firewallManagementIPName": "lab-firewall-mgt-ip", 146 | "bastionIPName": "[format('{0}-ip', variables('bastionName'))]", 147 | "vnetGatewayIPName": "lab-gateway-ip", 148 | "vmHubDiskName": "[format('{0}-disk', variables('vmHubName'))]", 149 | "vmHubNICName": "[format('{0}-nic', variables('vmHubName'))]", 150 | "vmHubAutoshutdownName": "[format('shutdown-computevm-{0}', variables('vmHubName'))]", 151 | "vm01DiskName": "[format('{0}-disk', variables('vm01Name'))]", 152 | "vm01NICName": "[format('{0}-nic', variables('vm01Name'))]", 153 | "vm01AutoshutdownName": "[format('shutdown-computevm-{0}', variables('vm01Name'))]", 154 | "vm02DiskName": "[format('{0}-disk', variables('vm02Name'))]", 155 | "vm02NICName": "[format('{0}-nic', variables('vm02Name'))]", 156 | "vm02AutoshutdownName": "[format('shutdown-computevm-{0}', variables('vm02Name'))]", 157 | "vm03DiskName": "[format('{0}-disk', variables('vm03Name'))]", 158 | "vm03NICName": "[format('{0}-nic', variables('vm03Name'))]", 159 | "vm03AutoshutdownName": "[format('shutdown-computevm-{0}', variables('vm03Name'))]", 160 | "subnets": "[concat(createArray(createObject('name', 'GatewaySubnet', 'properties', createObject('addressPrefix', '10.12.4.0/24')), createObject('name', 'AzureFirewallSubnet', 'properties', createObject('addressPrefix', '10.12.0.0/24')), createObject('name', 'AzureBastionSubnet', 'properties', createObject('addressPrefix', '10.12.2.0/24')), createObject('name', 'DefaultSubnet', 'properties', createObject('addressPrefix', '10.12.1.0/24'))), if(equals(parameters('firewallTier'), 'Basic'), createArray(createObject('name', 'AzureFirewallManagementSubnet', 'properties', createObject('addressPrefix', '10.12.3.0/24', 'privateEndpointNetworkPolicies', 'Enabled', 'privateLinkServiceNetworkPolicies', 'Enabled'))), createArray()))]" 161 | }, 162 | "resources": { 163 | "hubLabVnet": { 164 | "type": "Microsoft.Network/virtualNetworks", 165 | "apiVersion": "2019-09-01", 166 | "name": "[variables('hublabName')]", 167 | "location": "[parameters('location')]", 168 | "properties": { 169 | "addressSpace": { 170 | "addressPrefixes": [ 171 | "10.12.0.0/16" 172 | ] 173 | }, 174 | "subnets": "[variables('subnets')]" 175 | } 176 | }, 177 | "spoke01vnet": { 178 | "type": "Microsoft.Network/virtualNetworks", 179 | "apiVersion": "2019-09-01", 180 | "name": "[variables('spoke01Name')]", 181 | "location": "[parameters('location')]", 182 | "properties": { 183 | "addressSpace": { 184 | "addressPrefixes": [ 185 | "10.13.1.0/24" 186 | ] 187 | }, 188 | "subnets": [ 189 | { 190 | "name": "default", 191 | "properties": { 192 | "addressPrefix": "10.13.1.0/26" 193 | } 194 | }, 195 | { 196 | "name": "services", 197 | "properties": { 198 | "addressPrefix": "10.13.1.64/26" 199 | } 200 | } 201 | ] 202 | } 203 | }, 204 | "spoke02vnet": { 205 | "type": "Microsoft.Network/virtualNetworks", 206 | "apiVersion": "2019-09-01", 207 | "name": "[variables('spoke02Name')]", 208 | "location": "[parameters('location')]", 209 | "properties": { 210 | "addressSpace": { 211 | "addressPrefixes": [ 212 | "10.13.2.0/24" 213 | ] 214 | }, 215 | "subnets": [ 216 | { 217 | "name": "default", 218 | "properties": { 219 | "addressPrefix": "10.13.2.0/26" 220 | } 221 | }, 222 | { 223 | "name": "services", 224 | "properties": { 225 | "addressPrefix": "10.13.2.64/26" 226 | } 227 | } 228 | ] 229 | } 230 | }, 231 | "spoke03vnet": { 232 | "type": "Microsoft.Network/virtualNetworks", 233 | "apiVersion": "2019-09-01", 234 | "name": "[variables('spoke03Name')]", 235 | "location": "[parameters('locationSpoke03')]", 236 | "properties": { 237 | "addressSpace": { 238 | "addressPrefixes": [ 239 | "10.13.3.0/24" 240 | ] 241 | }, 242 | "subnets": [ 243 | { 244 | "name": "default", 245 | "properties": { 246 | "addressPrefix": "10.13.3.0/26" 247 | } 248 | }, 249 | { 250 | "name": "services", 251 | "properties": { 252 | "addressPrefix": "10.13.3.64/26" 253 | } 254 | } 255 | ] 256 | } 257 | }, 258 | "peeringHubSpoke01": { 259 | "type": "Microsoft.Network/virtualNetworks/virtualNetworkPeerings", 260 | "apiVersion": "2019-09-01", 261 | "name": "[format('{0}/{1}', variables('hublabName'), format('{0}-to-{1}', variables('hublabName'), variables('spoke01Name')))]", 262 | "properties": { 263 | "allowVirtualNetworkAccess": true, 264 | "allowForwardedTraffic": true, 265 | "allowGatewayTransit": true, 266 | "useRemoteGateways": false, 267 | "remoteVirtualNetwork": { 268 | "id": "[resourceId('Microsoft.Network/virtualNetworks', variables('spoke01Name'))]" 269 | } 270 | }, 271 | "dependsOn": [ 272 | "hubLabVnet", 273 | "spoke01vnet" 274 | ] 275 | }, 276 | "peeringSpoke01Hub": { 277 | "type": "Microsoft.Network/virtualNetworks/virtualNetworkPeerings", 278 | "apiVersion": "2019-09-01", 279 | "name": "[format('{0}/{1}', variables('spoke01Name'), format('{0}-to-{1}', variables('spoke01Name'), variables('hublabName')))]", 280 | "properties": { 281 | "allowVirtualNetworkAccess": true, 282 | "allowForwardedTraffic": true, 283 | "allowGatewayTransit": false, 284 | "useRemoteGateways": false, 285 | "remoteVirtualNetwork": { 286 | "id": "[resourceId('Microsoft.Network/virtualNetworks', variables('hublabName'))]" 287 | } 288 | }, 289 | "dependsOn": [ 290 | "hubLabVnet", 291 | "spoke01vnet" 292 | ] 293 | }, 294 | "peeringHubSpoke02": { 295 | "type": "Microsoft.Network/virtualNetworks/virtualNetworkPeerings", 296 | "apiVersion": "2019-09-01", 297 | "name": "[format('{0}/{1}', variables('hublabName'), format('{0}-to-{1}', variables('hublabName'), variables('spoke02Name')))]", 298 | "properties": { 299 | "allowVirtualNetworkAccess": true, 300 | "allowForwardedTraffic": true, 301 | "allowGatewayTransit": true, 302 | "useRemoteGateways": false, 303 | "remoteVirtualNetwork": { 304 | "id": "[resourceId('Microsoft.Network/virtualNetworks', variables('spoke02Name'))]" 305 | } 306 | }, 307 | "dependsOn": [ 308 | "hubLabVnet", 309 | "spoke02vnet" 310 | ] 311 | }, 312 | "peeringSpoke02Hub": { 313 | "type": "Microsoft.Network/virtualNetworks/virtualNetworkPeerings", 314 | "apiVersion": "2019-09-01", 315 | "name": "[format('{0}/{1}', variables('spoke02Name'), format('{0}-to-{1}', variables('spoke02Name'), variables('hublabName')))]", 316 | "properties": { 317 | "allowVirtualNetworkAccess": true, 318 | "allowForwardedTraffic": true, 319 | "allowGatewayTransit": false, 320 | "useRemoteGateways": false, 321 | "remoteVirtualNetwork": { 322 | "id": "[resourceId('Microsoft.Network/virtualNetworks', variables('hublabName'))]" 323 | } 324 | }, 325 | "dependsOn": [ 326 | "hubLabVnet", 327 | "spoke02vnet" 328 | ] 329 | }, 330 | "peeringHubSpoke03": { 331 | "type": "Microsoft.Network/virtualNetworks/virtualNetworkPeerings", 332 | "apiVersion": "2019-09-01", 333 | "name": "[format('{0}/{1}', variables('hublabName'), format('{0}-to-{1}', variables('hublabName'), variables('spoke03Name')))]", 334 | "properties": { 335 | "allowVirtualNetworkAccess": true, 336 | "allowForwardedTraffic": true, 337 | "allowGatewayTransit": true, 338 | "useRemoteGateways": false, 339 | "remoteVirtualNetwork": { 340 | "id": "[resourceId('Microsoft.Network/virtualNetworks', variables('spoke03Name'))]" 341 | } 342 | }, 343 | "dependsOn": [ 344 | "hubLabVnet", 345 | "spoke03vnet" 346 | ] 347 | }, 348 | "peeringSpoke03Hub": { 349 | "type": "Microsoft.Network/virtualNetworks/virtualNetworkPeerings", 350 | "apiVersion": "2019-09-01", 351 | "name": "[format('{0}/{1}', variables('spoke03Name'), format('{0}-to-{1}', variables('spoke03Name'), variables('hublabName')))]", 352 | "properties": { 353 | "allowVirtualNetworkAccess": true, 354 | "allowForwardedTraffic": true, 355 | "allowGatewayTransit": false, 356 | "useRemoteGateways": false, 357 | "remoteVirtualNetwork": { 358 | "id": "[resourceId('Microsoft.Network/virtualNetworks', variables('hublabName'))]" 359 | } 360 | }, 361 | "dependsOn": [ 362 | "hubLabVnet", 363 | "spoke03vnet" 364 | ] 365 | }, 366 | "bastionHubIP": { 367 | "condition": "[parameters('deployBastion')]", 368 | "type": "Microsoft.Network/publicIPAddresses", 369 | "apiVersion": "2019-09-01", 370 | "name": "[variables('bastionIPName')]", 371 | "location": "[parameters('location')]", 372 | "sku": { 373 | "name": "Standard" 374 | }, 375 | "properties": { 376 | "publicIPAllocationMethod": "Static" 377 | } 378 | }, 379 | "bastion": { 380 | "condition": "[parameters('deployBastion')]", 381 | "type": "Microsoft.Network/bastionHosts", 382 | "apiVersion": "2019-09-01", 383 | "name": "[variables('bastionName')]", 384 | "location": "[parameters('location')]", 385 | "properties": { 386 | "ipConfigurations": [ 387 | { 388 | "name": "ipconfig1", 389 | "properties": { 390 | "subnet": { 391 | "id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('hublabName'), 'AzureBastionSubnet')]" 392 | }, 393 | "publicIPAddress": { 394 | "id": "[resourceId('Microsoft.Network/publicIPAddresses', variables('bastionIPName'))]" 395 | } 396 | } 397 | } 398 | ] 399 | }, 400 | "dependsOn": [ 401 | "bastionHubIP", 402 | "hubLabVnet" 403 | ] 404 | }, 405 | "workspace": { 406 | "type": "Microsoft.OperationalInsights/workspaces", 407 | "apiVersion": "2022-10-01", 408 | "name": "hub-playground-ws", 409 | "location": "[parameters('location')]", 410 | "properties": { 411 | "sku": { 412 | "name": "PerGB2018" 413 | } 414 | } 415 | }, 416 | "firewallIP": { 417 | "type": "Microsoft.Network/publicIPAddresses", 418 | "apiVersion": "2019-09-01", 419 | "name": "[variables('firewallIPName')]", 420 | "location": "[parameters('location')]", 421 | "sku": { 422 | "name": "Standard" 423 | }, 424 | "properties": { 425 | "publicIPAllocationMethod": "Static" 426 | } 427 | }, 428 | "firewallManagementIP": { 429 | "condition": "[equals(parameters('firewallTier'), 'Basic')]", 430 | "type": "Microsoft.Network/publicIPAddresses", 431 | "apiVersion": "2019-09-01", 432 | "name": "[variables('firewallManagementIPName')]", 433 | "location": "[parameters('location')]", 434 | "sku": { 435 | "name": "Standard" 436 | }, 437 | "properties": { 438 | "publicIPAllocationMethod": "Static" 439 | } 440 | }, 441 | "firewall": { 442 | "type": "Microsoft.Network/azureFirewalls", 443 | "apiVersion": "2022-09-01", 444 | "name": "[variables('firewallName')]", 445 | "location": "[parameters('location')]", 446 | "properties": { 447 | "managementIpConfiguration": "[if(equals(parameters('firewallTier'), 'Basic'), createObject('name', 'ipconfig-mgt', 'properties', createObject('subnet', createObject('id', resourceId('Microsoft.Network/virtualNetworks/subnets', variables('hublabName'), 'AzureFirewallManagementSubnet')), 'publicIPAddress', createObject('id', resourceId('Microsoft.Network/publicIPAddresses', variables('firewallManagementIPName'))))), null())]", 448 | "ipConfigurations": [ 449 | { 450 | "name": "ipconfig1", 451 | "properties": { 452 | "subnet": { 453 | "id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('hublabName'), 'AzureFirewallSubnet')]" 454 | }, 455 | "publicIPAddress": { 456 | "id": "[resourceId('Microsoft.Network/publicIPAddresses', variables('firewallIPName'))]" 457 | } 458 | } 459 | } 460 | ], 461 | "sku": { 462 | "name": "AZFW_VNet", 463 | "tier": "[parameters('firewallTier')]" 464 | } 465 | }, 466 | "dependsOn": [ 467 | "firewallIP", 468 | "firewallManagementIP", 469 | "hubLabVnet", 470 | "vnetGateway" 471 | ] 472 | }, 473 | "firewallDiagnostics": { 474 | "type": "Microsoft.Insights/diagnosticSettings", 475 | "apiVersion": "2021-05-01-preview", 476 | "scope": "[format('Microsoft.Network/azureFirewalls/{0}', variables('firewallName'))]", 477 | "name": "hub-playground-ws", 478 | "properties": { 479 | "workspaceId": "[resourceId('Microsoft.OperationalInsights/workspaces', 'hub-playground-ws')]", 480 | "metrics": [ 481 | { 482 | "category": "AllMetrics", 483 | "enabled": true, 484 | "retentionPolicy": { 485 | "days": 0, 486 | "enabled": true 487 | } 488 | } 489 | ], 490 | "logAnalyticsDestinationType": null, 491 | "logs": [ 492 | { 493 | "categoryGroup": "allLogs", 494 | "enabled": true, 495 | "retentionPolicy": { 496 | "days": 0, 497 | "enabled": true 498 | } 499 | } 500 | ] 501 | }, 502 | "dependsOn": [ 503 | "firewall", 504 | "workspace" 505 | ] 506 | }, 507 | "vnetGatewayIP": { 508 | "condition": "[parameters('deployGateway')]", 509 | "type": "Microsoft.Network/publicIPAddresses", 510 | "apiVersion": "2019-09-01", 511 | "name": "[variables('vnetGatewayIPName')]", 512 | "location": "[parameters('location')]", 513 | "sku": { 514 | "name": "Basic" 515 | }, 516 | "properties": { 517 | "publicIPAllocationMethod": "Dynamic" 518 | } 519 | }, 520 | "vnetGateway": { 521 | "condition": "[parameters('deployGateway')]", 522 | "type": "Microsoft.Network/virtualNetworkGateways", 523 | "apiVersion": "2019-09-01", 524 | "name": "[variables('vnetGatewayName')]", 525 | "location": "[parameters('location')]", 526 | "properties": { 527 | "ipConfigurations": [ 528 | { 529 | "name": "ipconfig1", 530 | "properties": { 531 | "subnet": { 532 | "id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('hublabName'), 'GatewaySubnet')]" 533 | }, 534 | "publicIPAddress": { 535 | "id": "[resourceId('Microsoft.Network/publicIPAddresses', variables('vnetGatewayIPName'))]" 536 | } 537 | } 538 | } 539 | ], 540 | "gatewayType": "Vpn", 541 | "vpnType": "RouteBased", 542 | "enableBgp": false, 543 | "bgpSettings": null, 544 | "sku": { 545 | "name": "VpnGw1", 546 | "tier": "VpnGw1" 547 | } 548 | }, 549 | "dependsOn": [ 550 | "hubLabVnet", 551 | "vnetGatewayIP" 552 | ] 553 | }, 554 | "vmHubDisk": { 555 | "condition": "[parameters('deployVmHub')]", 556 | "type": "Microsoft.Compute/disks", 557 | "apiVersion": "2019-07-01", 558 | "name": "[variables('vmHubDiskName')]", 559 | "location": "[parameters('location')]", 560 | "properties": { 561 | "creationData": { 562 | "createOption": "Empty" 563 | }, 564 | "diskSizeGB": 128 565 | } 566 | }, 567 | "vmHubNIC": { 568 | "condition": "[parameters('deployVmHub')]", 569 | "type": "Microsoft.Network/networkInterfaces", 570 | "apiVersion": "2019-09-01", 571 | "name": "[variables('vmHubNICName')]", 572 | "location": "[parameters('location')]", 573 | "properties": { 574 | "ipConfigurations": [ 575 | { 576 | "name": "ipconfig1", 577 | "properties": { 578 | "subnet": { 579 | "id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('hublabName'), 'DefaultSubnet')]" 580 | }, 581 | "privateIPAllocationMethod": "Dynamic" 582 | } 583 | } 584 | ] 585 | }, 586 | "dependsOn": [ 587 | "hubLabVnet" 588 | ] 589 | }, 590 | "vmHub": { 591 | "condition": "[parameters('deployVmHub')]", 592 | "type": "Microsoft.Compute/virtualMachines", 593 | "apiVersion": "2019-07-01", 594 | "name": "[variables('vmHubName')]", 595 | "location": "[parameters('location')]", 596 | "properties": { 597 | "hardwareProfile": { 598 | "vmSize": "[parameters('virtualMachineSKU')]" 599 | }, 600 | "storageProfile": { 601 | "imageReference": { 602 | "publisher": "MicrosoftWindowsServer", 603 | "offer": "WindowsServer", 604 | "sku": "2019-Datacenter", 605 | "version": "latest" 606 | }, 607 | "dataDisks": [ 608 | { 609 | "lun": 0, 610 | "name": "[variables('vmHubDiskName')]", 611 | "createOption": "Attach", 612 | "managedDisk": { 613 | "id": "[resourceId('Microsoft.Compute/disks', variables('vmHubDiskName'))]" 614 | } 615 | } 616 | ] 617 | }, 618 | "osProfile": { 619 | "computerName": "[variables('vmHubName')]", 620 | "adminUsername": "[parameters('username')]", 621 | "adminPassword": "[parameters('password')]", 622 | "windowsConfiguration": { 623 | "enableAutomaticUpdates": true 624 | } 625 | }, 626 | "networkProfile": { 627 | "networkInterfaces": [ 628 | { 629 | "id": "[resourceId('Microsoft.Network/networkInterfaces', variables('vmHubNICName'))]" 630 | } 631 | ] 632 | } 633 | }, 634 | "dependsOn": [ 635 | "vmHubDisk", 636 | "vmHubNIC" 637 | ] 638 | }, 639 | "vmHubAutoshutdown": { 640 | "condition": "[parameters('deployVmHub')]", 641 | "type": "Microsoft.DevTestLab/schedules", 642 | "apiVersion": "2018-09-15", 643 | "name": "[variables('vmHubAutoshutdownName')]", 644 | "location": "[parameters('location')]", 645 | "properties": { 646 | "status": "Enabled", 647 | "taskType": "ComputeVmShutdownTask", 648 | "timeZoneId": "UTC", 649 | "dailyRecurrence": { 650 | "time": "20:00" 651 | }, 652 | "notificationSettings": { 653 | "status": "Disabled" 654 | }, 655 | "targetResourceId": "[resourceId('Microsoft.Compute/virtualMachines', variables('vmHubName'))]" 656 | }, 657 | "dependsOn": [ 658 | "vmHub" 659 | ] 660 | }, 661 | "vm01Disk": { 662 | "condition": "[parameters('deployVm01')]", 663 | "type": "Microsoft.Compute/disks", 664 | "apiVersion": "2019-07-01", 665 | "name": "[variables('vm01DiskName')]", 666 | "location": "[parameters('location')]", 667 | "properties": { 668 | "creationData": { 669 | "createOption": "Empty" 670 | }, 671 | "diskSizeGB": 128 672 | } 673 | }, 674 | "vm01NIC": { 675 | "condition": "[parameters('deployVm01')]", 676 | "type": "Microsoft.Network/networkInterfaces", 677 | "apiVersion": "2019-09-01", 678 | "name": "[variables('vm01NICName')]", 679 | "location": "[parameters('location')]", 680 | "properties": { 681 | "ipConfigurations": [ 682 | { 683 | "name": "ipconfig1", 684 | "properties": { 685 | "subnet": { 686 | "id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('spoke01Name'), 'default')]" 687 | }, 688 | "privateIPAllocationMethod": "Dynamic" 689 | } 690 | } 691 | ] 692 | }, 693 | "dependsOn": [ 694 | "spoke01vnet" 695 | ] 696 | }, 697 | "vm01": { 698 | "condition": "[parameters('deployVm01')]", 699 | "type": "Microsoft.Compute/virtualMachines", 700 | "apiVersion": "2019-07-01", 701 | "name": "[variables('vm01Name')]", 702 | "location": "[parameters('location')]", 703 | "properties": { 704 | "hardwareProfile": { 705 | "vmSize": "[parameters('virtualMachineSKU')]" 706 | }, 707 | "storageProfile": { 708 | "imageReference": { 709 | "publisher": "MicrosoftWindowsServer", 710 | "offer": "WindowsServer", 711 | "sku": "2019-Datacenter", 712 | "version": "latest" 713 | }, 714 | "dataDisks": [ 715 | { 716 | "lun": 0, 717 | "name": "[variables('vm01DiskName')]", 718 | "createOption": "Attach", 719 | "managedDisk": { 720 | "id": "[resourceId('Microsoft.Compute/disks', variables('vm01DiskName'))]" 721 | } 722 | } 723 | ] 724 | }, 725 | "osProfile": { 726 | "computerName": "[variables('vm01Name')]", 727 | "adminUsername": "[parameters('username')]", 728 | "adminPassword": "[parameters('password')]", 729 | "windowsConfiguration": { 730 | "enableAutomaticUpdates": true 731 | } 732 | }, 733 | "networkProfile": { 734 | "networkInterfaces": [ 735 | { 736 | "id": "[resourceId('Microsoft.Network/networkInterfaces', variables('vm01NICName'))]" 737 | } 738 | ] 739 | } 740 | }, 741 | "dependsOn": [ 742 | "vm01Disk", 743 | "vm01NIC" 744 | ] 745 | }, 746 | "vm01Autoshutdown": { 747 | "condition": "[parameters('deployVm01')]", 748 | "type": "Microsoft.DevTestLab/schedules", 749 | "apiVersion": "2018-09-15", 750 | "name": "[variables('vm01AutoshutdownName')]", 751 | "location": "[parameters('location')]", 752 | "properties": { 753 | "status": "Enabled", 754 | "taskType": "ComputeVmShutdownTask", 755 | "timeZoneId": "UTC", 756 | "dailyRecurrence": { 757 | "time": "20:00" 758 | }, 759 | "notificationSettings": { 760 | "status": "Disabled" 761 | }, 762 | "targetResourceId": "[resourceId('Microsoft.Compute/virtualMachines', variables('vm01Name'))]" 763 | }, 764 | "dependsOn": [ 765 | "vm01" 766 | ] 767 | }, 768 | "vm02Disk": { 769 | "condition": "[parameters('deployVm02')]", 770 | "type": "Microsoft.Compute/disks", 771 | "apiVersion": "2019-07-01", 772 | "name": "[variables('vm02DiskName')]", 773 | "location": "[parameters('location')]", 774 | "properties": { 775 | "creationData": { 776 | "createOption": "Empty" 777 | }, 778 | "diskSizeGB": 128 779 | } 780 | }, 781 | "vm02NIC": { 782 | "condition": "[parameters('deployVm02')]", 783 | "type": "Microsoft.Network/networkInterfaces", 784 | "apiVersion": "2019-09-01", 785 | "name": "[variables('vm02NICName')]", 786 | "location": "[parameters('location')]", 787 | "properties": { 788 | "ipConfigurations": [ 789 | { 790 | "name": "ipconfig1", 791 | "properties": { 792 | "subnet": { 793 | "id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('spoke02Name'), 'default')]" 794 | }, 795 | "privateIPAllocationMethod": "Dynamic" 796 | } 797 | } 798 | ] 799 | }, 800 | "dependsOn": [ 801 | "spoke02vnet" 802 | ] 803 | }, 804 | "vm02": { 805 | "condition": "[parameters('deployVm02')]", 806 | "type": "Microsoft.Compute/virtualMachines", 807 | "apiVersion": "2019-07-01", 808 | "name": "[variables('vm02Name')]", 809 | "location": "[parameters('location')]", 810 | "properties": { 811 | "hardwareProfile": { 812 | "vmSize": "[parameters('virtualMachineSKU')]" 813 | }, 814 | "storageProfile": { 815 | "imageReference": { 816 | "publisher": "MicrosoftWindowsServer", 817 | "offer": "WindowsServer", 818 | "sku": "2019-Datacenter", 819 | "version": "latest" 820 | }, 821 | "dataDisks": [ 822 | { 823 | "lun": 0, 824 | "name": "[variables('vm02DiskName')]", 825 | "createOption": "Attach", 826 | "managedDisk": { 827 | "id": "[resourceId('Microsoft.Compute/disks', variables('vm02DiskName'))]" 828 | } 829 | } 830 | ] 831 | }, 832 | "osProfile": { 833 | "computerName": "[variables('vm02Name')]", 834 | "adminUsername": "[parameters('username')]", 835 | "adminPassword": "[parameters('password')]", 836 | "windowsConfiguration": { 837 | "enableAutomaticUpdates": true 838 | } 839 | }, 840 | "networkProfile": { 841 | "networkInterfaces": [ 842 | { 843 | "id": "[resourceId('Microsoft.Network/networkInterfaces', variables('vm02NICName'))]" 844 | } 845 | ] 846 | } 847 | }, 848 | "dependsOn": [ 849 | "vm02Disk", 850 | "vm02NIC" 851 | ] 852 | }, 853 | "vm02Autoshutdown": { 854 | "condition": "[parameters('deployVm02')]", 855 | "type": "Microsoft.DevTestLab/schedules", 856 | "apiVersion": "2018-09-15", 857 | "name": "[variables('vm02AutoshutdownName')]", 858 | "location": "[parameters('location')]", 859 | "properties": { 860 | "status": "Enabled", 861 | "taskType": "ComputeVmShutdownTask", 862 | "timeZoneId": "UTC", 863 | "dailyRecurrence": { 864 | "time": "20:00" 865 | }, 866 | "notificationSettings": { 867 | "status": "Disabled" 868 | }, 869 | "targetResourceId": "[resourceId('Microsoft.Compute/virtualMachines', variables('vm02Name'))]" 870 | }, 871 | "dependsOn": [ 872 | "vm02" 873 | ] 874 | }, 875 | "vm03Disk": { 876 | "condition": "[parameters('deployVm03')]", 877 | "type": "Microsoft.Compute/disks", 878 | "apiVersion": "2019-07-01", 879 | "name": "[variables('vm03DiskName')]", 880 | "location": "[parameters('locationSpoke03')]", 881 | "properties": { 882 | "creationData": { 883 | "createOption": "Empty" 884 | }, 885 | "diskSizeGB": 128 886 | } 887 | }, 888 | "vm03NIC": { 889 | "condition": "[parameters('deployVm03')]", 890 | "type": "Microsoft.Network/networkInterfaces", 891 | "apiVersion": "2019-09-01", 892 | "name": "[variables('vm03NICName')]", 893 | "location": "[parameters('locationSpoke03')]", 894 | "properties": { 895 | "ipConfigurations": [ 896 | { 897 | "name": "ipconfig1", 898 | "properties": { 899 | "subnet": { 900 | "id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('spoke03Name'), 'default')]" 901 | }, 902 | "privateIPAllocationMethod": "Dynamic" 903 | } 904 | } 905 | ] 906 | }, 907 | "dependsOn": [ 908 | "spoke03vnet" 909 | ] 910 | }, 911 | "vm03": { 912 | "condition": "[parameters('deployVm03')]", 913 | "type": "Microsoft.Compute/virtualMachines", 914 | "apiVersion": "2019-07-01", 915 | "name": "[variables('vm03Name')]", 916 | "location": "[parameters('locationSpoke03')]", 917 | "properties": { 918 | "hardwareProfile": { 919 | "vmSize": "[parameters('virtualMachineSKU')]" 920 | }, 921 | "storageProfile": { 922 | "imageReference": { 923 | "publisher": "MicrosoftWindowsServer", 924 | "offer": "WindowsServer", 925 | "sku": "2019-Datacenter", 926 | "version": "latest" 927 | }, 928 | "dataDisks": [ 929 | { 930 | "lun": 0, 931 | "name": "[variables('vm03DiskName')]", 932 | "createOption": "Attach", 933 | "managedDisk": { 934 | "id": "[resourceId('Microsoft.Compute/disks', variables('vm03DiskName'))]" 935 | } 936 | } 937 | ] 938 | }, 939 | "osProfile": { 940 | "computerName": "[variables('vm03Name')]", 941 | "adminUsername": "[parameters('username')]", 942 | "adminPassword": "[parameters('password')]", 943 | "windowsConfiguration": { 944 | "enableAutomaticUpdates": true 945 | } 946 | }, 947 | "networkProfile": { 948 | "networkInterfaces": [ 949 | { 950 | "id": "[resourceId('Microsoft.Network/networkInterfaces', variables('vm03NICName'))]" 951 | } 952 | ] 953 | } 954 | }, 955 | "dependsOn": [ 956 | "vm03Disk", 957 | "vm03NIC" 958 | ] 959 | }, 960 | "vm03Autoshutdown": { 961 | "condition": "[parameters('deployVm03')]", 962 | "type": "Microsoft.DevTestLab/schedules", 963 | "apiVersion": "2018-09-15", 964 | "name": "[variables('vm03AutoshutdownName')]", 965 | "location": "[parameters('locationSpoke03')]", 966 | "properties": { 967 | "status": "Enabled", 968 | "taskType": "ComputeVmShutdownTask", 969 | "timeZoneId": "UTC", 970 | "dailyRecurrence": { 971 | "time": "20:00" 972 | }, 973 | "notificationSettings": { 974 | "status": "Disabled" 975 | }, 976 | "targetResourceId": "[resourceId('Microsoft.Compute/virtualMachines', variables('vm03Name'))]" 977 | }, 978 | "dependsOn": [ 979 | "vm03" 980 | ] 981 | } 982 | }, 983 | "outputs": { 984 | "hubVnet": { 985 | "type": "object", 986 | "value": "[reference('hubLabVnet', '2019-09-01', 'full')]" 987 | }, 988 | "spoke01Vnet": { 989 | "type": "object", 990 | "value": "[reference('spoke01vnet', '2019-09-01', 'full')]" 991 | }, 992 | "spoke02Vnet": { 993 | "type": "object", 994 | "value": "[reference('spoke02vnet', '2019-09-01', 'full')]" 995 | }, 996 | "spoke03Vnet": { 997 | "type": "object", 998 | "value": "[reference('spoke03vnet', '2019-09-01', 'full')]" 999 | }, 1000 | "firewallTier": { 1001 | "type": "string", 1002 | "allowedValues": [ 1003 | "Basic", 1004 | "Premium", 1005 | "Standard" 1006 | ], 1007 | "value": "[parameters('firewallTier')]" 1008 | } 1009 | } 1010 | } 1011 | } 1012 | }, 1013 | { 1014 | "type": "Microsoft.Resources/deployments", 1015 | "apiVersion": "2022-09-01", 1016 | "name": "aks-deploy", 1017 | "properties": { 1018 | "expressionEvaluationOptions": { 1019 | "scope": "inner" 1020 | }, 1021 | "mode": "Incremental", 1022 | "parameters": { 1023 | "location": { 1024 | "value": "[variables('location')]" 1025 | }, 1026 | "clusterName": { 1027 | "value": "[variables('clusterName')]" 1028 | }, 1029 | "nodeResourceGroup": { 1030 | "value": "[variables('clusterResourceGroup')]" 1031 | }, 1032 | "subnetID": { 1033 | "value": "[reference(resourceId('Microsoft.Resources/deployments', 'hub-spoke-deploy'), '2022-09-01').outputs.spoke01Vnet.value.properties.subnets[1].id]" 1034 | }, 1035 | "availabilityZones": { 1036 | "value": [ 1037 | "3" 1038 | ] 1039 | }, 1040 | "usePrivateApiServer": { 1041 | "value": false 1042 | } 1043 | }, 1044 | "template": { 1045 | "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", 1046 | "contentVersion": "1.0.0.0", 1047 | "metadata": { 1048 | "_generator": { 1049 | "name": "bicep", 1050 | "version": "0.18.4.5664", 1051 | "templateHash": "775211741727374315" 1052 | } 1053 | }, 1054 | "parameters": { 1055 | "clusterName": { 1056 | "type": "string", 1057 | "defaultValue": "aks-01" 1058 | }, 1059 | "usePrivateApiServer": { 1060 | "type": "bool", 1061 | "defaultValue": false 1062 | }, 1063 | "location": { 1064 | "type": "string", 1065 | "defaultValue": "[resourceGroup().location]" 1066 | }, 1067 | "subnetID": { 1068 | "type": "string" 1069 | }, 1070 | "useNetworkOverlay": { 1071 | "type": "bool", 1072 | "defaultValue": false 1073 | }, 1074 | "nodeCount": { 1075 | "type": "int", 1076 | "defaultValue": 1 1077 | }, 1078 | "nodeVMSize": { 1079 | "type": "string", 1080 | "defaultValue": "Standard_D2s_v3" 1081 | }, 1082 | "availabilityZones": { 1083 | "type": "array", 1084 | "defaultValue": [ 1085 | "1", 1086 | "2", 1087 | "3" 1088 | ] 1089 | }, 1090 | "dnsPrefix": { 1091 | "type": "string", 1092 | "defaultValue": "cl01" 1093 | }, 1094 | "nodeResourceGroup": { 1095 | "type": "string", 1096 | "defaultValue": "[format('{0}-{1}-{2}-rg', parameters('dnsPrefix'), parameters('clusterName'), utcNow())]" 1097 | }, 1098 | "tags": { 1099 | "type": "object", 1100 | "defaultValue": { 1101 | "environment": "production", 1102 | "projectCode": "xyz" 1103 | } 1104 | }, 1105 | "kubeVersion": { 1106 | "type": "string", 1107 | "defaultValue": "" 1108 | }, 1109 | "logAnalyticsWorkspace": { 1110 | "type": "string", 1111 | "defaultValue": "" 1112 | }, 1113 | "useAzureADRBAC": { 1114 | "type": "bool", 1115 | "defaultValue": false 1116 | } 1117 | }, 1118 | "variables": { 1119 | "nodePoolName": "systempool", 1120 | "subnetArray": "[split(parameters('subnetID'), '/')]", 1121 | "vnetName": "[variables('subnetArray')[sub(length(variables('subnetArray')), 3)]]" 1122 | }, 1123 | "resources": [ 1124 | { 1125 | "type": "Microsoft.ContainerService/managedClusters", 1126 | "apiVersion": "2023-03-01", 1127 | "name": "[parameters('clusterName')]", 1128 | "location": "[parameters('location')]", 1129 | "tags": "[parameters('tags')]", 1130 | "identity": { 1131 | "type": "SystemAssigned" 1132 | }, 1133 | "properties": { 1134 | "kubernetesVersion": "[if(not(empty(parameters('kubeVersion'))), parameters('kubeVersion'), null())]", 1135 | "enableRBAC": "[parameters('useAzureADRBAC')]", 1136 | "dnsPrefix": "[parameters('dnsPrefix')]", 1137 | "agentPoolProfiles": [ 1138 | { 1139 | "availabilityZones": "[parameters('availabilityZones')]", 1140 | "name": "[variables('nodePoolName')]", 1141 | "count": "[parameters('nodeCount')]", 1142 | "mode": "System", 1143 | "vmSize": "[parameters('nodeVMSize')]", 1144 | "type": "VirtualMachineScaleSets", 1145 | "osType": "Linux", 1146 | "enableAutoScaling": false, 1147 | "vnetSubnetID": "[parameters('subnetID')]" 1148 | } 1149 | ], 1150 | "apiServerAccessProfile": { 1151 | "enablePrivateCluster": "[parameters('usePrivateApiServer')]" 1152 | }, 1153 | "servicePrincipalProfile": { 1154 | "clientId": "msi" 1155 | }, 1156 | "nodeResourceGroup": "[parameters('nodeResourceGroup')]", 1157 | "networkProfile": { 1158 | "networkPlugin": "azure", 1159 | "loadBalancerSku": "standard", 1160 | "networkPluginMode": "[if(parameters('useNetworkOverlay'), 'overlay', null())]" 1161 | }, 1162 | "addonProfiles": { 1163 | "omsagent": { 1164 | "config": { 1165 | "logAnalyticsWorkspaceResourceID": "[if(not(empty(parameters('logAnalyticsWorkspace'))), parameters('logAnalyticsWorkspace'), null())]" 1166 | }, 1167 | "enabled": "[if(not(empty(parameters('logAnalyticsWorkspace'))), true(), false())]" 1168 | } 1169 | } 1170 | } 1171 | }, 1172 | { 1173 | "type": "Microsoft.Authorization/roleAssignments", 1174 | "apiVersion": "2022-04-01", 1175 | "name": "[guid('aks-vnet')]", 1176 | "properties": { 1177 | "roleDefinitionId": "/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c", 1178 | "principalId": "[reference(resourceId('Microsoft.ContainerService/managedClusters', parameters('clusterName')), '2023-03-01', 'full').identity.principalId]", 1179 | "principalType": "ServicePrincipal" 1180 | }, 1181 | "dependsOn": [ 1182 | "[resourceId('Microsoft.ContainerService/managedClusters', parameters('clusterName'))]" 1183 | ] 1184 | } 1185 | ], 1186 | "outputs": { 1187 | "aks": { 1188 | "type": "object", 1189 | "value": "[reference(resourceId('Microsoft.ContainerService/managedClusters', parameters('clusterName')), '2023-03-01', 'full')]" 1190 | }, 1191 | "kubeconfig": { 1192 | "type": "string", 1193 | "value": "[listClusterAdminCredential(resourceId('Microsoft.ContainerService/managedClusters', parameters('clusterName')), '2023-03-01').kubeconfigs[0].value]" 1194 | } 1195 | } 1196 | } 1197 | }, 1198 | "dependsOn": [ 1199 | "[resourceId('Microsoft.Resources/deployments', 'hub-spoke-deploy')]" 1200 | ] 1201 | }, 1202 | { 1203 | "type": "Microsoft.Resources/deployments", 1204 | "apiVersion": "2022-09-01", 1205 | "name": "buildbicep-deploy", 1206 | "properties": { 1207 | "expressionEvaluationOptions": { 1208 | "scope": "inner" 1209 | }, 1210 | "mode": "Incremental", 1211 | "parameters": { 1212 | "kubeConfig": { 1213 | "value": "[reference(resourceId('Microsoft.Resources/deployments', 'aks-deploy'), '2022-09-01').outputs.kubeconfig.value]" 1214 | }, 1215 | "privateLinkServiceName": { 1216 | "value": "[variables('privateLinkServiceName')]" 1217 | }, 1218 | "privateLoadBalancer": { 1219 | "value": true 1220 | } 1221 | }, 1222 | "template": { 1223 | "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", 1224 | "languageVersion": "1.10-experimental", 1225 | "contentVersion": "1.0.0.0", 1226 | "metadata": { 1227 | "_EXPERIMENTAL_WARNING": "Symbolic name support in ARM is experimental, and should be enabled for testing purposes only. Do not enable this setting for any production usage, or you may be unexpectedly broken at any time!", 1228 | "_generator": { 1229 | "name": "bicep", 1230 | "version": "0.18.4.5664", 1231 | "templateHash": "1378555755554142058" 1232 | } 1233 | }, 1234 | "parameters": { 1235 | "privateLoadBalancer": { 1236 | "type": "bool", 1237 | "defaultValue": false 1238 | }, 1239 | "privateLinkServiceName": { 1240 | "type": "string", 1241 | "defaultValue": "" 1242 | }, 1243 | "kubeConfig": { 1244 | "type": "securestring" 1245 | } 1246 | }, 1247 | "variables": { 1248 | "privateLink": "[not(empty(parameters('privateLinkServiceName')))]" 1249 | }, 1250 | "imports": { 1251 | "kubernetes": { 1252 | "provider": "Kubernetes", 1253 | "version": "1.0.0", 1254 | "config": { 1255 | "namespace": "default", 1256 | "kubeConfig": "[parameters('kubeConfig')]" 1257 | } 1258 | } 1259 | }, 1260 | "resources": { 1261 | "appsDeployment_azureVoteBack": { 1262 | "import": "kubernetes", 1263 | "type": "apps/Deployment@v1", 1264 | "properties": { 1265 | "metadata": { 1266 | "name": "azure-vote-back" 1267 | }, 1268 | "spec": { 1269 | "replicas": 1, 1270 | "selector": { 1271 | "matchLabels": { 1272 | "app": "azure-vote-back" 1273 | } 1274 | }, 1275 | "template": { 1276 | "metadata": { 1277 | "labels": { 1278 | "app": "azure-vote-back" 1279 | } 1280 | }, 1281 | "spec": { 1282 | "nodeSelector": { 1283 | "kubernetes.io/os": "linux" 1284 | }, 1285 | "containers": [ 1286 | { 1287 | "name": "azure-vote-back", 1288 | "image": "mcr.microsoft.com/oss/bitnami/redis:6.0.8", 1289 | "env": [ 1290 | { 1291 | "name": "ALLOW_EMPTY_PASSWORD", 1292 | "value": "yes" 1293 | } 1294 | ], 1295 | "resources": { 1296 | "requests": { 1297 | "cpu": "100m", 1298 | "memory": "128Mi" 1299 | }, 1300 | "limits": { 1301 | "cpu": "250m", 1302 | "memory": "256Mi" 1303 | } 1304 | }, 1305 | "ports": [ 1306 | { 1307 | "containerPort": 6379, 1308 | "name": "redis" 1309 | } 1310 | ] 1311 | } 1312 | ] 1313 | } 1314 | } 1315 | } 1316 | } 1317 | }, 1318 | "coreService_azureVoteBack": { 1319 | "import": "kubernetes", 1320 | "type": "core/Service@v1", 1321 | "properties": { 1322 | "metadata": { 1323 | "name": "azure-vote-back" 1324 | }, 1325 | "spec": { 1326 | "ports": [ 1327 | { 1328 | "port": 6379 1329 | } 1330 | ], 1331 | "selector": { 1332 | "app": "azure-vote-back" 1333 | } 1334 | } 1335 | } 1336 | }, 1337 | "appsDeployment_azureVoteFront": { 1338 | "import": "kubernetes", 1339 | "type": "apps/Deployment@v1", 1340 | "properties": { 1341 | "metadata": { 1342 | "name": "azure-vote-front" 1343 | }, 1344 | "spec": { 1345 | "replicas": 1, 1346 | "selector": { 1347 | "matchLabels": { 1348 | "app": "azure-vote-front" 1349 | } 1350 | }, 1351 | "template": { 1352 | "metadata": { 1353 | "labels": { 1354 | "app": "azure-vote-front" 1355 | } 1356 | }, 1357 | "spec": { 1358 | "nodeSelector": { 1359 | "kubernetes.io/os": "linux" 1360 | }, 1361 | "containers": [ 1362 | { 1363 | "name": "azure-vote-front", 1364 | "image": "mcr.microsoft.com/azuredocs/azure-vote-front:v1", 1365 | "resources": { 1366 | "requests": { 1367 | "cpu": "100m", 1368 | "memory": "128Mi" 1369 | }, 1370 | "limits": { 1371 | "cpu": "250m", 1372 | "memory": "256Mi" 1373 | } 1374 | }, 1375 | "ports": [ 1376 | { 1377 | "containerPort": 80 1378 | } 1379 | ], 1380 | "env": [ 1381 | { 1382 | "name": "REDIS", 1383 | "value": "azure-vote-back" 1384 | } 1385 | ] 1386 | } 1387 | ] 1388 | } 1389 | } 1390 | } 1391 | } 1392 | }, 1393 | "coreService_azureVoteFront": { 1394 | "import": "kubernetes", 1395 | "type": "core/Service@v1", 1396 | "properties": { 1397 | "metadata": { 1398 | "name": "azure-vote-front", 1399 | "annotations": { 1400 | "service.beta.kubernetes.io/azure-load-balancer-internal": "[if(parameters('privateLoadBalancer'), 'true', 'false')]", 1401 | "service.beta.kubernetes.io/azure-pls-create": "[if(variables('privateLink'), 'true', 'false')]", 1402 | "service.beta.kubernetes.io/azure-pls-name": "[parameters('privateLinkServiceName')]" 1403 | } 1404 | }, 1405 | "spec": { 1406 | "type": "LoadBalancer", 1407 | "ports": [ 1408 | { 1409 | "port": 80 1410 | } 1411 | ], 1412 | "selector": { 1413 | "app": "azure-vote-front" 1414 | } 1415 | } 1416 | } 1417 | } 1418 | } 1419 | } 1420 | }, 1421 | "dependsOn": [ 1422 | "[resourceId('Microsoft.Resources/deployments', 'aks-deploy')]" 1423 | ] 1424 | } 1425 | ], 1426 | "outputs": { 1427 | "clusterResourceGroupOut": { 1428 | "type": "string", 1429 | "value": "[variables('clusterResourceGroup')]" 1430 | } 1431 | } 1432 | } -------------------------------------------------------------------------------- /modules-arm/front-door-2.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", 3 | "contentVersion": "1.0.0.0", 4 | "metadata": { 5 | "_generator": { 6 | "name": "bicep", 7 | "version": "0.18.4.5664", 8 | "templateHash": "7449078717699916358" 9 | } 10 | }, 11 | "variables": { 12 | "privateLinkServiceName": "pl-aks-01", 13 | "location": "westeurope", 14 | "clusterName": "aks-01", 15 | "clusterResourceGroup": "[format('cl01-{0}-rg', variables('clusterName'))]", 16 | "frontDoorProfileName": "aks-fd", 17 | "frontDoorOriginGroupName": "aks-origin-group", 18 | "frontDoorOriginName": "aks-origin", 19 | "frontDoorRouteName": "aks-route", 20 | "frontDoorSkuName": "Premium_AzureFrontDoor", 21 | "frontDoorEndpointName": "[format('aks-afd-{0}', uniqueString(resourceGroup().id))]" 22 | }, 23 | "resources": [ 24 | { 25 | "type": "Microsoft.Cdn/profiles", 26 | "apiVersion": "2023-05-01", 27 | "name": "[variables('frontDoorProfileName')]", 28 | "location": "global", 29 | "sku": { 30 | "name": "[variables('frontDoorSkuName')]" 31 | } 32 | }, 33 | { 34 | "type": "Microsoft.Cdn/profiles/afdEndpoints", 35 | "apiVersion": "2023-05-01", 36 | "name": "[format('{0}/{1}', variables('frontDoorProfileName'), variables('frontDoorEndpointName'))]", 37 | "location": "global", 38 | "properties": { 39 | "enabledState": "Enabled" 40 | }, 41 | "dependsOn": [ 42 | "[resourceId('Microsoft.Cdn/profiles', variables('frontDoorProfileName'))]" 43 | ] 44 | }, 45 | { 46 | "type": "Microsoft.Cdn/profiles/originGroups", 47 | "apiVersion": "2023-05-01", 48 | "name": "[format('{0}/{1}', variables('frontDoorProfileName'), variables('frontDoorOriginGroupName'))]", 49 | "properties": { 50 | "loadBalancingSettings": { 51 | "sampleSize": 4, 52 | "successfulSamplesRequired": 3 53 | }, 54 | "healthProbeSettings": { 55 | "probePath": "/", 56 | "probeRequestType": "GET", 57 | "probeProtocol": "Http", 58 | "probeIntervalInSeconds": 100 59 | } 60 | }, 61 | "dependsOn": [ 62 | "[resourceId('Microsoft.Cdn/profiles', variables('frontDoorProfileName'))]" 63 | ] 64 | }, 65 | { 66 | "type": "Microsoft.Cdn/profiles/originGroups/origins", 67 | "apiVersion": "2023-05-01", 68 | "name": "[format('{0}/{1}/{2}', variables('frontDoorProfileName'), variables('frontDoorOriginGroupName'), variables('frontDoorOriginName'))]", 69 | "properties": { 70 | "hostName": "[reference(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', subscription().subscriptionId, variables('clusterResourceGroup')), 'Microsoft.Network/privateLinkServices', variables('privateLinkServiceName')), '2022-11-01').alias]", 71 | "httpPort": 80, 72 | "httpsPort": 443, 73 | "originHostHeader": "[reference(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', subscription().subscriptionId, variables('clusterResourceGroup')), 'Microsoft.Network/privateLinkServices', variables('privateLinkServiceName')), '2022-11-01').alias]", 74 | "priority": 1, 75 | "weight": 1000, 76 | "sharedPrivateLinkResource": { 77 | "privateLinkLocation": "[variables('location')]", 78 | "privateLink": { 79 | "id": "[extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', subscription().subscriptionId, variables('clusterResourceGroup')), 'Microsoft.Network/privateLinkServices', variables('privateLinkServiceName'))]" 80 | }, 81 | "requestMessage": "[variables('clusterName')]", 82 | "status": "Approved" 83 | } 84 | }, 85 | "dependsOn": [ 86 | "[resourceId('Microsoft.Cdn/profiles/originGroups', variables('frontDoorProfileName'), variables('frontDoorOriginGroupName'))]" 87 | ] 88 | }, 89 | { 90 | "type": "Microsoft.Cdn/profiles/afdEndpoints/routes", 91 | "apiVersion": "2023-05-01", 92 | "name": "[format('{0}/{1}/{2}', variables('frontDoorProfileName'), variables('frontDoorEndpointName'), variables('frontDoorRouteName'))]", 93 | "properties": { 94 | "originGroup": { 95 | "id": "[resourceId('Microsoft.Cdn/profiles/originGroups', variables('frontDoorProfileName'), variables('frontDoorOriginGroupName'))]" 96 | }, 97 | "supportedProtocols": [ 98 | "Http", 99 | "Https" 100 | ], 101 | "patternsToMatch": [ 102 | "/*" 103 | ], 104 | "forwardingProtocol": "HttpOnly", 105 | "linkToDefaultDomain": "Enabled", 106 | "httpsRedirect": "Enabled" 107 | }, 108 | "dependsOn": [ 109 | "[resourceId('Microsoft.Cdn/profiles/afdEndpoints', variables('frontDoorProfileName'), variables('frontDoorEndpointName'))]", 110 | "[resourceId('Microsoft.Cdn/profiles/originGroups/origins', variables('frontDoorProfileName'), variables('frontDoorOriginGroupName'), variables('frontDoorOriginName'))]", 111 | "[resourceId('Microsoft.Cdn/profiles/originGroups', variables('frontDoorProfileName'), variables('frontDoorOriginGroupName'))]" 112 | ] 113 | } 114 | ], 115 | "outputs": { 116 | "frontDoorEndpointHostName": { 117 | "type": "string", 118 | "value": "[reference(resourceId('Microsoft.Cdn/profiles/afdEndpoints', variables('frontDoorProfileName'), variables('frontDoorEndpointName')), '2023-05-01').hostName]" 119 | } 120 | } 121 | } -------------------------------------------------------------------------------- /modules/aks-vote-app/aks-vote-app.bicep: -------------------------------------------------------------------------------- 1 | param privateLoadBalancer bool = false 2 | param privateLinkServiceName string = '' 3 | var privateLink = !empty(privateLinkServiceName) 4 | 5 | @secure() 6 | param kubeConfig string 7 | 8 | import 'kubernetes@1.0.0' with { 9 | namespace: 'default' 10 | kubeConfig: kubeConfig 11 | } 12 | 13 | resource appsDeployment_azureVoteBack 'apps/Deployment@v1' = { 14 | metadata: { 15 | name: 'azure-vote-back' 16 | } 17 | spec: { 18 | replicas: 1 19 | selector: { 20 | matchLabels: { 21 | app: 'azure-vote-back' 22 | } 23 | } 24 | template: { 25 | metadata: { 26 | labels: { 27 | app: 'azure-vote-back' 28 | } 29 | } 30 | spec: { 31 | nodeSelector: { 32 | 'kubernetes.io/os': 'linux' 33 | } 34 | containers: [ 35 | { 36 | name: 'azure-vote-back' 37 | image: 'mcr.microsoft.com/oss/bitnami/redis:6.0.8' 38 | env: [ 39 | { 40 | name: 'ALLOW_EMPTY_PASSWORD' 41 | value: 'yes' 42 | } 43 | ] 44 | resources: { 45 | requests: { 46 | cpu: '100m' 47 | memory: '128Mi' 48 | } 49 | limits: { 50 | cpu: '250m' 51 | memory: '256Mi' 52 | } 53 | } 54 | ports: [ 55 | { 56 | containerPort: 6379 57 | name: 'redis' 58 | } 59 | ] 60 | } 61 | ] 62 | } 63 | } 64 | } 65 | } 66 | 67 | resource coreService_azureVoteBack 'core/Service@v1' = { 68 | metadata: { 69 | name: 'azure-vote-back' 70 | } 71 | spec: { 72 | ports: [ 73 | { 74 | port: 6379 75 | } 76 | ] 77 | selector: { 78 | app: 'azure-vote-back' 79 | } 80 | } 81 | } 82 | 83 | resource appsDeployment_azureVoteFront 'apps/Deployment@v1' = { 84 | metadata: { 85 | name: 'azure-vote-front' 86 | } 87 | spec: { 88 | replicas: 1 89 | selector: { 90 | matchLabels: { 91 | app: 'azure-vote-front' 92 | } 93 | } 94 | template: { 95 | metadata: { 96 | labels: { 97 | app: 'azure-vote-front' 98 | } 99 | } 100 | spec: { 101 | nodeSelector: { 102 | 'kubernetes.io/os': 'linux' 103 | } 104 | containers: [ 105 | { 106 | name: 'azure-vote-front' 107 | image: 'mcr.microsoft.com/azuredocs/azure-vote-front:v1' 108 | resources: { 109 | requests: { 110 | cpu: '100m' 111 | memory: '128Mi' 112 | } 113 | limits: { 114 | cpu: '250m' 115 | memory: '256Mi' 116 | } 117 | } 118 | ports: [ 119 | { 120 | containerPort: 80 121 | } 122 | ] 123 | env: [ 124 | { 125 | name: 'REDIS' 126 | value: 'azure-vote-back' 127 | } 128 | ] 129 | } 130 | ] 131 | } 132 | } 133 | } 134 | } 135 | 136 | resource coreService_azureVoteFront 'core/Service@v1' = { 137 | metadata: { 138 | name: 'azure-vote-front' 139 | annotations: { 140 | 'service.beta.kubernetes.io/azure-load-balancer-internal': privateLoadBalancer ? 'true' : 'false' 141 | 'service.beta.kubernetes.io/azure-load-balancer-internal-subnet': 'default' 142 | 'service.beta.kubernetes.io/azure-pls-create': privateLink ? 'true' : 'false' 143 | 'service.beta.kubernetes.io/azure-pls-name': privateLinkServiceName 144 | } 145 | } 146 | spec: { 147 | type: 'LoadBalancer' 148 | ports: [ 149 | { 150 | port: 80 151 | } 152 | ] 153 | selector: { 154 | app: 'azure-vote-front' 155 | } 156 | } 157 | } 158 | -------------------------------------------------------------------------------- /modules/aks-vote-app/aks-vote-app.yaml: -------------------------------------------------------------------------------- 1 | apiVersion: apps/v1 2 | kind: Deployment 3 | metadata: 4 | name: azure-vote-back 5 | spec: 6 | replicas: 1 7 | selector: 8 | matchLabels: 9 | app: azure-vote-back 10 | template: 11 | metadata: 12 | labels: 13 | app: azure-vote-back 14 | spec: 15 | nodeSelector: 16 | "kubernetes.io/os": linux 17 | containers: 18 | - name: azure-vote-back 19 | image: mcr.microsoft.com/oss/bitnami/redis:6.0.8 20 | env: 21 | - name: ALLOW_EMPTY_PASSWORD 22 | value: "yes" 23 | resources: 24 | requests: 25 | cpu: 100m 26 | memory: 128Mi 27 | limits: 28 | cpu: 250m 29 | memory: 256Mi 30 | ports: 31 | - containerPort: 6379 32 | name: redis 33 | --- 34 | apiVersion: v1 35 | kind: Service 36 | metadata: 37 | name: azure-vote-back 38 | spec: 39 | ports: 40 | - port: 6379 41 | selector: 42 | app: azure-vote-back 43 | --- 44 | apiVersion: apps/v1 45 | kind: Deployment 46 | metadata: 47 | name: azure-vote-front 48 | spec: 49 | replicas: 1 50 | selector: 51 | matchLabels: 52 | app: azure-vote-front 53 | template: 54 | metadata: 55 | labels: 56 | app: azure-vote-front 57 | spec: 58 | nodeSelector: 59 | "kubernetes.io/os": linux 60 | containers: 61 | - name: azure-vote-front 62 | image: mcr.microsoft.com/azuredocs/azure-vote-front:v1 63 | resources: 64 | requests: 65 | cpu: 100m 66 | memory: 128Mi 67 | limits: 68 | cpu: 250m 69 | memory: 256Mi 70 | ports: 71 | - containerPort: 80 72 | env: 73 | - name: REDIS 74 | value: "azure-vote-back" 75 | --- 76 | apiVersion: v1 77 | kind: Service 78 | metadata: 79 | name: azure-vote-front 80 | spec: 81 | type: LoadBalancer 82 | ports: 83 | - port: 80 84 | selector: 85 | app: azure-vote-front -------------------------------------------------------------------------------- /modules/aks-vote-app/bicepconfig.json: -------------------------------------------------------------------------------- 1 | { 2 | "experimentalFeaturesEnabled": { 3 | "extensibility": true 4 | } 5 | } -------------------------------------------------------------------------------- /modules/aks/aks.bicep: -------------------------------------------------------------------------------- 1 | //az deployment group create --resource-group --template-file default-aks.bicep --parameters @default-aks.parameters.json 2 | // description('The name of the Managed Cluster resource.') 3 | param clusterName string = 'aks-01' 4 | // Whether to deploy an API Server accessible ONLY from the vNET 5 | param usePrivateApiServer bool = false 6 | // description('Specifies the Azure location where the cluster should be created. Defaults to resource group location') 7 | param location string = resourceGroup().location 8 | //NETWORK 9 | // description('The nodes' subnet ID. Make sure to provide the entire subnet ID and not subnet name') 10 | param subnetID string 11 | // Whether to use Overlay network mode (optional) 12 | param useNetworkOverlay bool = false 13 | //END NETWORK 14 | //COMPUTE 15 | // minValue(1), maxValue(50) 16 | // description('The number of nodes for the cluster. 1 Node is enough for Dev/Test and minimum 3 nodes, is recommended for Production') 17 | param nodeCount int = 1 18 | // description('The family and size of the Virtual Machine.') (optional) 19 | param nodeVMSize string = 'Standard_D2s_v3' 20 | param availabilityZones array = ['1','2','3'] 21 | //END COMPUTE 22 | // description('The DNS prefix to use with hosted Kubernetes API server FQDN.') (optional) 23 | param dnsPrefix string = 'cl01' 24 | //OTHERS 25 | // The nodes resource group name (optional) 26 | param nodeResourceGroup string = '${dnsPrefix}-${clusterName}-${utcNow()}-rg' 27 | param tags object = { 28 | environment: 'production' 29 | projectCode: 'xyz' 30 | } 31 | 32 | // The Kubernetes version (optional) 33 | param kubeVersion string = '' 34 | // The logAnalyticsWorkspace version (optional) 35 | param logAnalyticsWorkspace string = '' 36 | // Whether to use the AzureAD RBAC model instead of the native Kubernetes (optional) 37 | param useAzureADRBAC bool = false 38 | //END OTHERS 39 | // vars 40 | var nodePoolName = 'systempool' 41 | 42 | // Create the Azure kubernetes service cluster 43 | resource aks 'Microsoft.ContainerService/managedClusters@2023-03-01' = { 44 | name: clusterName 45 | location: location 46 | tags: tags 47 | identity: { 48 | type: 'SystemAssigned' 49 | } 50 | properties: { 51 | kubernetesVersion: ((!empty(kubeVersion)) ? kubeVersion : null) 52 | enableRBAC: useAzureADRBAC 53 | dnsPrefix: dnsPrefix 54 | agentPoolProfiles: [ 55 | { 56 | availabilityZones: availabilityZones 57 | name: nodePoolName 58 | count: nodeCount 59 | mode: 'System' 60 | vmSize: nodeVMSize 61 | type: 'VirtualMachineScaleSets' 62 | osType: 'Linux' 63 | enableAutoScaling: false 64 | vnetSubnetID: subnetID 65 | } 66 | 67 | ] 68 | apiServerAccessProfile:{ 69 | enablePrivateCluster: usePrivateApiServer 70 | } 71 | servicePrincipalProfile: { 72 | clientId: 'msi' 73 | } 74 | nodeResourceGroup: nodeResourceGroup 75 | networkProfile: { 76 | networkPlugin: 'azure' 77 | loadBalancerSku: 'standard' 78 | networkPluginMode: (useNetworkOverlay ? 'overlay' : null) 79 | } 80 | addonProfiles:{ 81 | omsagent:{ 82 | config:{ 83 | #disable-next-line BCP321 84 | logAnalyticsWorkspaceResourceID: ((!empty(logAnalyticsWorkspace)) ? logAnalyticsWorkspace : null) 85 | } 86 | enabled: ((!empty(logAnalyticsWorkspace)) ? true : false) 87 | } 88 | } 89 | } 90 | } 91 | 92 | var subnetArray = split(subnetID, '/') 93 | var vnetName = subnetArray[length(subnetArray)-3] 94 | resource virtualNetwork 'Microsoft.Network/virtualNetworks@2022-11-01' existing = { 95 | name: vnetName 96 | } 97 | 98 | resource roleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = { 99 | dependsOn: [virtualNetwork] 100 | scope: resourceGroup() 101 | name: guid('aks-vnet') 102 | properties: { 103 | roleDefinitionId: '/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c'//contributor 104 | principalId: aks.identity.principalId 105 | principalType: 'ServicePrincipal' 106 | } 107 | } 108 | output aks object = aks 109 | output kubeconfig string = aks.listClusterAdminCredential().kubeconfigs[0].value //this is the only way (as of now) to pass a reference to the credentials at compile time. Because TYPE CAST in Bicep is still not available 110 | -------------------------------------------------------------------------------- /modules/front-door/front-door-1.bicep: -------------------------------------------------------------------------------- 1 | //MAKE SURE to APPROVE privateLinks requested by FrontDoor, otherwise you will get a 502 error from FrontDoor 2 | var clusterName = 'aks-01' 3 | var clusterResourceGroup = 'cl01-${clusterName}-rg' 4 | var privateLinkServiceName = 'pl-aks-01' 5 | var location = 'westeurope' 6 | 7 | module hubSpokeDeploy '../../hub-and-spoke-playground/hub-01-bicep/hub-01.bicep' = { 8 | name: 'hub-spoke-deploy' 9 | params: { 10 | location: location 11 | locationSpoke03: location 12 | firewallTier: 'Premium' 13 | deployBastion: false 14 | deployGateway: false 15 | deployVmHub: false 16 | deployVm01: false 17 | deployVm02: false 18 | deployVm03: false 19 | } 20 | } 21 | 22 | module aksDeploy '../aks/aks.bicep' = { 23 | name: 'aks-deploy' 24 | params: { 25 | location: location 26 | clusterName: clusterName 27 | nodeResourceGroup: clusterResourceGroup 28 | subnetID: hubSpokeDeploy.outputs.spoke01Vnet.properties.subnets[1].id 29 | availabilityZones: [ '3' ] 30 | usePrivateApiServer: false 31 | } 32 | } 33 | 34 | module kubernetes '../aks-vote-app/aks-vote-app.bicep' = { 35 | name: 'buildbicep-deploy' 36 | params: { 37 | kubeConfig: aksDeploy.outputs.kubeconfig 38 | privateLinkServiceName: privateLinkServiceName 39 | privateLoadBalancer: true 40 | } 41 | } 42 | output clusterResourceGroupOut string = clusterResourceGroup 43 | -------------------------------------------------------------------------------- /modules/front-door/front-door-2.bicep: -------------------------------------------------------------------------------- 1 | //MAKE SURE to APPROVE privateLinks requested by FrontDoor, otherwise you will get a 502 error from FrontDoor 2 | var privateLinkServiceName = 'pl-aks-01' 3 | var location = 'westeurope' 4 | var clusterName = 'aks-01' 5 | var clusterResourceGroup = 'cl01-${clusterName}-rg' 6 | var frontDoorProfileName = 'aks-fd' 7 | var frontDoorOriginGroupName = 'aks-origin-group' 8 | var frontDoorOriginName = 'aks-origin' 9 | var frontDoorRouteName = 'aks-route' 10 | var frontDoorSkuName = 'Premium_AzureFrontDoor' 11 | var frontDoorEndpointName = 'aks-afd-${uniqueString(resourceGroup().id)}' 12 | 13 | 14 | resource privateLinkService 'Microsoft.Network/privateLinkServices@2022-11-01' existing = { 15 | name: privateLinkServiceName 16 | scope: resourceGroup(clusterResourceGroup) 17 | } 18 | 19 | resource frontDoorProfile 'Microsoft.Cdn/profiles@2023-05-01' = { 20 | name: frontDoorProfileName 21 | location: 'global' 22 | sku: { 23 | name: frontDoorSkuName 24 | } 25 | } 26 | 27 | resource frontDoorEndpoint 'Microsoft.Cdn/profiles/afdEndpoints@2023-05-01' = { 28 | name: frontDoorEndpointName 29 | parent: frontDoorProfile 30 | location: 'global' 31 | properties: { 32 | enabledState: 'Enabled' 33 | } 34 | } 35 | 36 | resource frontDoorOriginGroup 'Microsoft.Cdn/profiles/originGroups@2023-05-01' = { 37 | name: frontDoorOriginGroupName 38 | parent: frontDoorProfile 39 | properties: { 40 | loadBalancingSettings: { 41 | sampleSize: 4 42 | successfulSamplesRequired: 3 43 | } 44 | healthProbeSettings: { 45 | probePath: '/' 46 | probeRequestType: 'GET' 47 | probeProtocol: 'Http' 48 | probeIntervalInSeconds: 100 49 | } 50 | } 51 | } 52 | 53 | resource frontDoorOrigin 'Microsoft.Cdn/profiles/originGroups/origins@2023-05-01' = { 54 | name: frontDoorOriginName 55 | parent: frontDoorOriginGroup 56 | properties: { 57 | hostName: privateLinkService.properties.alias 58 | httpPort: 80 59 | httpsPort: 443 60 | originHostHeader: privateLinkService.properties.alias 61 | priority: 1 62 | weight: 1000 63 | sharedPrivateLinkResource:{ 64 | privateLinkLocation: location 65 | privateLink: { 66 | id: privateLinkService.id 67 | } 68 | requestMessage: clusterName 69 | status: 'Approved' 70 | } 71 | } 72 | dependsOn:[ 73 | privateLinkService 74 | ] 75 | } 76 | 77 | resource frontDoorRoute 'Microsoft.Cdn/profiles/afdEndpoints/routes@2023-05-01' = { 78 | name: frontDoorRouteName 79 | parent: frontDoorEndpoint 80 | dependsOn: [ 81 | frontDoorOrigin // This explicit dependency is required to ensure that the origin group is not empty when the route is created. 82 | ] 83 | properties: { 84 | originGroup: { 85 | id: frontDoorOriginGroup.id 86 | } 87 | supportedProtocols: [ 88 | 'Http' 89 | 'Https' 90 | ] 91 | patternsToMatch: [ 92 | '/*' 93 | ] 94 | forwardingProtocol: 'HttpOnly' //because the origin only exposes http 95 | linkToDefaultDomain: 'Enabled' 96 | httpsRedirect: 'Enabled' 97 | } 98 | } 99 | 100 | output frontDoorEndpointHostName string = frontDoorEndpoint.properties.hostName 101 | -------------------------------------------------------------------------------- /modules/hub-spoke-aks/hub-spoke-aks.bicep: -------------------------------------------------------------------------------- 1 | //az deployment group create --resource-group --template-file default-aks.bicep --parameters @default-aks.parameters.json 2 | //https://learn.microsoft.com/en-us/azure/azure-resource-manager/bicep/bicep-extensibility-kubernetes-provider 3 | //PRIVATE clusters are not supported as of now: https://github.com/Azure/bicep-extensibility/issues/130 4 | 5 | var clusterName = 'aks-01' 6 | var location = 'westeurope' 7 | @description('username administrator for all VMs') 8 | param username string = 'nicola' 9 | 10 | @description('username administrator password for all VMs') 11 | @secure() 12 | param password string = 'password.123' 13 | 14 | 15 | module hubSpokeDeploy '../../hub-and-spoke-playground/hub-01-bicep/hub-01.bicep' = { 16 | name: 'hub-spoke-deploy' 17 | params: { 18 | location: location 19 | locationSpoke03: location 20 | firewallTier: 'Premium' 21 | deployBastion: true 22 | deployGateway: false 23 | deployVmHub: true 24 | deployVm01: false 25 | deployVm02: false 26 | deployVm03: false 27 | username: username 28 | password: password 29 | } 30 | } 31 | 32 | module anyToAnyDeploy '../../hub-and-spoke-playground/any-to-any-bicep/any-to-any.bicep' = { 33 | name: 'any-to-any-deploy' 34 | params: { 35 | locationWE: location 36 | locationNE: location 37 | firewallTier: any(hubSpokeDeploy.outputs.firewallTier) 38 | } 39 | } 40 | 41 | 42 | module aksDeploy '../aks/aks.bicep' = { 43 | name: 'aks-deploy' 44 | params: { 45 | location: location 46 | clusterName: clusterName 47 | subnetID: hubSpokeDeploy.outputs.spoke01Vnet.properties.subnets[1].id 48 | availabilityZones: [ '3' ] 49 | usePrivateApiServer: false 50 | } 51 | } 52 | 53 | module kubernetes '../aks-vote-app/aks-vote-app.bicep' = { 54 | name: 'buildbicep-deploy' 55 | params: { 56 | kubeConfig: aksDeploy.outputs.kubeconfig 57 | privateLoadBalancer: true //this is needed in order to test DNAT rule from the firewall to the internal load balancer 58 | } 59 | } 60 | -------------------------------------------------------------------------------- /modules/test/front-door-only-test.bicep: -------------------------------------------------------------------------------- 1 | var clusterName = 'aks-01' 2 | var nodeResourceGroup = 'cl01-aks-01-20230621T083306Z-rg' 3 | var privateLinkServiceName = 'aks-pls' 4 | var location = 'westeurope' 5 | 6 | var frontDoorProfileName = 'aks-fd' 7 | var frontDoorOriginGroupName = 'aks-origin-group' 8 | var frontDoorOriginName = 'aks-origin' 9 | var frontDoorRouteName = 'aks-route' 10 | var frontDoorSkuName = 'Premium_AzureFrontDoor' 11 | var frontDoorEndpointName = 'aks-afd-${uniqueString(resourceGroup().id)}' 12 | 13 | resource privateLinkService 'Microsoft.Network/privateLinkServices@2022-11-01' existing = { 14 | name: privateLinkServiceName 15 | scope: resourceGroup(nodeResourceGroup) 16 | } 17 | 18 | resource frontDoorProfile 'Microsoft.Cdn/profiles@2023-05-01' = { 19 | name: frontDoorProfileName 20 | location: 'global' 21 | sku: { 22 | name: frontDoorSkuName 23 | } 24 | } 25 | 26 | resource frontDoorEndpoint 'Microsoft.Cdn/profiles/afdEndpoints@2023-05-01' = { 27 | name: frontDoorEndpointName 28 | parent: frontDoorProfile 29 | location: 'global' 30 | properties: { 31 | enabledState: 'Enabled' 32 | } 33 | } 34 | 35 | resource frontDoorOriginGroup 'Microsoft.Cdn/profiles/originGroups@2023-05-01' = { 36 | name: frontDoorOriginGroupName 37 | parent: frontDoorProfile 38 | properties: { 39 | loadBalancingSettings: { 40 | sampleSize: 4 41 | successfulSamplesRequired: 3 42 | } 43 | healthProbeSettings: { 44 | probePath: '/' 45 | probeRequestType: 'GET' 46 | probeProtocol: 'Http' 47 | probeIntervalInSeconds: 100 48 | } 49 | } 50 | } 51 | 52 | resource frontDoorOrigin 'Microsoft.Cdn/profiles/originGroups/origins@2023-05-01' = { 53 | name: frontDoorOriginName 54 | parent: frontDoorOriginGroup 55 | properties: { 56 | hostName: privateLinkService.properties.alias 57 | httpPort: 80 58 | httpsPort: 443 59 | originHostHeader: privateLinkService.properties.alias 60 | priority: 1 61 | weight: 1000 62 | sharedPrivateLinkResource:{ 63 | privateLinkLocation: location 64 | privateLink: { 65 | id: privateLinkService.id 66 | } 67 | requestMessage: clusterName 68 | status: 'Approved' 69 | } 70 | } 71 | dependsOn:[ 72 | privateLinkService 73 | ] 74 | } 75 | 76 | resource frontDoorRoute 'Microsoft.Cdn/profiles/afdEndpoints/routes@2023-05-01' = { 77 | name: frontDoorRouteName 78 | parent: frontDoorEndpoint 79 | dependsOn: [ 80 | frontDoorOrigin // This explicit dependency is required to ensure that the origin group is not empty when the route is created. 81 | ] 82 | properties: { 83 | originGroup: { 84 | id: frontDoorOriginGroup.id 85 | } 86 | supportedProtocols: [ 87 | 'Http' 88 | 'Https' 89 | ] 90 | patternsToMatch: [ 91 | '/*' 92 | ] 93 | forwardingProtocol: 'HttpOnly' //because the origin only exposes http 94 | linkToDefaultDomain: 'Enabled' 95 | httpsRedirect: 'Enabled' 96 | } 97 | } 98 | 99 | output frontDoorEndpointHostName string = frontDoorEndpoint.properties.hostName 100 | -------------------------------------------------------------------------------- /modules/test/private-link-test.bicep: -------------------------------------------------------------------------------- 1 | var nodeResourceGroup = 'cl01-aks-01-20230621T083306Z-rg' 2 | var privateLinkServiceName = 'aks-pls' 3 | var location = 'westeurope' 4 | 5 | resource privateLinkService 'Microsoft.Network/privateLinkServices@2022-11-01' existing = { 6 | name: privateLinkServiceName 7 | scope: resourceGroup(nodeResourceGroup) 8 | } 9 | var plAliasVar= privateLinkService.properties.alias 10 | output plAlias string =plAliasVar 11 | -------------------------------------------------------------------------------- /modules/test/private-link-test.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", 3 | "contentVersion": "1.0.0.0", 4 | "metadata": { 5 | "_generator": { 6 | "name": "bicep", 7 | "version": "0.18.4.5664", 8 | "templateHash": "12689372997683497355" 9 | } 10 | }, 11 | "variables": { 12 | "nodeResourceGroup": "cl01-aks-01-20230621T083306Z-rg", 13 | "privateLinkServiceName": "aks-pls", 14 | "location": "westeurope" 15 | }, 16 | "resources": [], 17 | "outputs": { 18 | "plAlias": { 19 | "type": "string", 20 | "value": "[reference(extensionResourceId(format('/subscriptions/{0}/resourceGroups/{1}', subscription().subscriptionId, variables('nodeResourceGroup')), 'Microsoft.Network/privateLinkServices', variables('privateLinkServiceName')), '2022-11-01').alias]" 21 | } 22 | } 23 | } -------------------------------------------------------------------------------- /modules/test/vnet-test.bicep: -------------------------------------------------------------------------------- 1 | param clusterName string 2 | resource aks 'Microsoft.ContainerService/managedClusters@2023-03-01' existing = { 3 | name: clusterName 4 | } 5 | 6 | 7 | var subnetID = aks.properties.agentPoolProfiles[0].vnetSubnetID 8 | var subnetArray = split(subnetID, '/') 9 | var vnetName = subnetArray[length(subnetArray)-3] 10 | resource virtualNetwork 'Microsoft.Network/virtualNetworks@2022-11-01' existing = { 11 | name: vnetName 12 | } 13 | 14 | resource roleAssignment 'Microsoft.Authorization/roleAssignments@2022-04-01' = { 15 | dependsOn: [virtualNetwork] 16 | scope: resourceGroup() 17 | name: guid('aks-vnet') 18 | properties: { 19 | roleDefinitionId: '/providers/Microsoft.Authorization/roleDefinitions/b24988ac-6180-42a0-ab88-20f7382dd24c'//contributor 20 | principalId: aks.identity.principalId 21 | principalType: 'ServicePrincipal' 22 | } 23 | } 24 | 25 | output subnetID string = subnetID 26 | output stringArray array = subnetArray 27 | output vnetName string = vnetName 28 | output vnetNameObj string = virtualNetwork.name 29 | -------------------------------------------------------------------------------- /modules/test/yaml-bootstrap-test.bicep: -------------------------------------------------------------------------------- 1 | param clusterName string 2 | resource aks 'Microsoft.ContainerService/managedClusters@2023-03-01' existing = { 3 | name: clusterName 4 | } 5 | module kubernetes './sample-aks.bicep' = { 6 | name: 'buildbicep-deploy' 7 | params: { 8 | kubeConfig: aks.listClusterAdminCredential().kubeconfigs[0].value 9 | privateLoadBalancer: true 10 | } 11 | } 12 | -------------------------------------------------------------------------------- /scenarios/confidential-01.md: -------------------------------------------------------------------------------- 1 | # SCENARIO: Deploy a confidential computing cluster nodes 2 | 3 | ## Pre-requisites 4 | 5 | To implement this scenario, you need to install the `AKS-playground` that you find in the home of this repo. 6 | 7 | ## Solution 8 | 9 | In this scenario, You'll use the Azure CLI to deploy an enclave-aware (DCsv2/DCSv3) VM node pool on your AKS cluster. You'll then run a simple Hello World application in the enclave. 10 | 11 | * *Secure enclaves* (also known as Trusted Execution Environments or TEE) are at the core of confidential computing. Secure Enclaves are sets of security-related instruction codes built into new CPUs. They protect data in use, because the enclave is decrypted on the fly only within the CPU, and then only for code and data running within the enclave itself. Introduced by Intel as Software Guard Extensions (**SGX**), secure enclaves are based on hardware-level encrypted memory isolation. AMD now offers similar functionality with its SEV technology, built into **Epyc**. 12 | * In a secure enclave, applications run in an environment that is isolated from the host. Memory is completely isolated from anything else on the machine, including the operating system. Private keys are hard-coded at the hardware level. A process called attestation allows enclaves to authenticate the hardware inside which they run as genuine, and to attest to the integrity of enclave memory to a remote party. Secure enclaves protect applications, data, and storage—locally, across the network, and in the cloud—simply and effectively. Application code and data are completely inaccessible to any other entities while running inside a secure enclave. Insiders with root or physical access to the system do not have access to memory. Even privileged users on the guest operating system, hypervisor, or the host operating system are blocked. 13 | 14 | Let's proceed: open a BASH cloud shell as onwer of the subscription that contains the `AKS-playground`. 15 | 16 | Run the following command to enable the confidential computing add-on: 17 | 18 | ``` 19 | myRg="" 20 | # az aks enable-addons --addons monitoring --name "aks-01" --resource-group $myRg 21 | az aks enable-addons --addons confcom --name "aks-01" --resource-group $myRg 22 | ``` 23 | 24 | Run the following command to add a user node pool of `Standard_DC4s_v3` size with two nodes to the AKS cluster. 25 | 26 | ``` 27 | az aks nodepool add --cluster-name "aks-01" --name confcompool1 --resource-group $myRg --node-vm-size Standard_DC4s_v3 --node-count 2 28 | ``` 29 | 30 | ## Test Solution 31 | Get the credentials for your AKS cluster by using the az aks get-credentials command: 32 | 33 | ``` 34 | az aks get-credentials --resource-group $myRg --name "aks-01" 35 | ``` 36 | 37 | Use the kubectl get pods command to verify that the nodes are created properly and the SGX-related DaemonSets are running on DCsv2 node pools: 38 | 39 | kubectl get pods --all-namespaces 40 | 41 | this row in the output means that SGX-related DaemonSets are properly configured. 42 | 43 | ``` 44 | kube-system sgx-device-plugin-xxxx 1/1 Running 45 | 46 | ``` 47 | 48 | Now deploy the `Hello World from an isolated enclave application`: Create a file named `hello.yaml` and paste in the following YAML manifest. 49 | 50 | ``` 51 | apiVersion: batch/v1 52 | kind: Job 53 | metadata: 54 | name: sgx-test 55 | labels: 56 | app: sgx-test 57 | spec: 58 | template: 59 | metadata: 60 | labels: 61 | app: sgx-test 62 | spec: 63 | containers: 64 | - name: sgxtest 65 | image: oeciteam/sgx-test:1.0 66 | resources: 67 | limits: 68 | sgx.intel.com/epc: 5Mi # This limit will automatically place the job into a confidential computing node and mount the required driver volumes. sgx limit setting needs "confcom" AKS Addon as referenced above. 69 | restartPolicy: Never 70 | backoffLimit: 0 71 | ``` 72 | 73 | deploy the sample using: 74 | 75 | ``` 76 | kubectl apply -f hello.yaml 77 | ``` 78 | 79 | wait until the command `kubectl get jobs -l app=sgx-test` gives as result the following: 80 | 81 | ``` 82 | NAME READY STATUS RESTARTS AGE 83 | sgx-test-rchvg 0/1 **Completed** 0 25s 84 | ``` 85 | 86 | `kubectl logs -l app=sgx-test` will show 87 | 88 | ``` 89 | Hello world from the enclave 90 | Enclave called into host to print: Hello World! 91 | 92 | 93 | ``` 94 | # More information 95 | 96 | * https://learn.microsoft.com/en-us/azure/confidential-computing/confidential-enclave-nodes-aks-get-started 97 | * https://github.com/openenclave/openenclave/tree/master/samples/helloworld 98 | -------------------------------------------------------------------------------- /scenarios/containerinsights.md: -------------------------------------------------------------------------------- 1 | # SCENARIO: Deploy Azure Monitor on AKS Cluster 2 | 3 | ## Pre-requisites 4 | 5 | To implement this scenario, you need to install the `AKS-playground` that you find in the home of this repo. 6 | 7 | ## Solution 8 | 9 | In this section we will implement Azure Monitor for Container Insights. 10 | Monitoring is one of the **most important** things that you need to implement, if you want to have a big picture of you infrastructure. 11 | 12 | Let's proceed: open a BASH cloud shell as onwer of the subscription that contains the `AKS-playground`. 13 | 14 | Run the following command to enable the add-on: 15 | 16 | ``` 17 | myRg="" 18 | myworkspaceid="" 19 | 20 | az aks enable-addons -a monitoring -n "aks-01" -g $myRg --workspace-resource-id $myworkspaceid 21 | ``` 22 | 23 | Remember that you need to specify the workspace Resource ID of your Log Analytics Workspace (LAW). An easy way to find that ID is looking in the section properties of your LAW, or using this command line 24 | 25 | ``` 26 | myworkspaceid = az resource list --resource-group "" --name "" --query [*].id --output tsv 27 | ``` 28 | 29 | ## Test Solution 30 | Get the credentials for your AKS cluster by using the az aks get-credentials command: 31 | 32 | ``` 33 | az aks get-credentials --resource-group $myRg --name "aks-01" 34 | ``` 35 | 36 | and after that, you can run 37 | 38 | ``` 39 | kubectl get ds ama-logs --namespace=kube-system 40 | 41 | ``` 42 | The output will be very similar to : 43 | ``` 44 | NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE 45 | ama-logs 1 1 1 1 1 138d 46 | ``` 47 | 48 | Now you can go to Azure Monitor and click on Container section. 49 | 50 | # More information 51 | 52 | * https://learn.microsoft.com/en-us/azure/azure-monitor/containers/container-insights-onboard 53 | -------------------------------------------------------------------------------- /scenarios/firewall-01.md: -------------------------------------------------------------------------------- 1 | # SCENARIO: Expose a workload from AKS with Azure Firewall 2 | 3 | ## Pre-requisites 4 | 5 | To implement this scenario, you need to install the `AKS-playground` that you find in the home of this repo. 6 | 7 | ## Solution 8 | In the context of an hub-and-spoke network, in this scenario we will expose a workload from Azure Kubernetes Service (`azure-vote-front`) externally via the Azure Firewall located in the hub network. 9 | 10 | It is recommended to expose internal services via an Azure Firewall because it provides a centralized and secure way to access them from outside the virtual network. Azure Firewall is a stateful firewall that can filter traffic based on rules and application signatures. It also supports network address translation (NAT) and threat intelligence-based filtering. By using Azure Firewall, you can protect your internal services from unauthorized or malicious access, while allowing authorized users to access them from the internet or other networks. 11 | 12 | When you install Azure Firewall, it has one public IP. We will expose on this IP, on port 80 (HTTP), the internal service `azure-vote-front`. 13 | 14 | In the context of our hub-and-spoke network, already all the traffic from and to any spoke is routed via the Azure Firewall. To expose the internal service on the Azure Firewall public IP, we need to create an additional DNAT Rule on the current firewall policy. 15 | 16 | > DNAT stands for **Destination Network Address Ttranslation** and is used when an external Host with a Public IP, starts a connection towards our Internal/Private Network. Therefore, the DNAT L3 device (in our scenario Azure Firewall), transforms the public IP address of that host to the private IP of the internal Host/Server. 17 | 18 | ![scenario implemented](../images/firewall-01.png) 19 | _Download the [draw.io file](../images/firewall-01.drawio) of this schema._ 20 | 21 | # Step 1 - get the Azure Firewall public IP Address 22 | Go to Azure portal > Firewalls > `lab-firewall` > Firewall public IP > `lab-firewall-ip` > IP Address > IP (**x.x.x.x**) 23 | 24 | # Step 2 - get the the IP of the front-end internal load balancer 25 | Go to Azure Portal > `aks-01` > Services and ingresses > `azure-vote-front` > Services > `azure-vote-front` > External IP (**10.13.1.y**) 26 | _Please note that the IP is a private IP, even if the label states 'External IP'. External in this context refers to the IP given to the Azure load balancer instance, while Internal refers to the Kubernetes assigned IP only reachable from inside the cluster._ 27 | 28 | # Step 3 - configure Azure Firewall DNAT policy 29 | 30 | Go to Azure Portal > `my-firewall-policy` > DNAT Rules > Add Rule Collection 31 | * Name: `my-dnat-rule-collection` 32 | * Collection Type: `DNAT` 33 | * priority: `1000` 34 | * Collection Group: `DefaultDnatCollectionGroup` 35 | * Source Type: `IP` 36 | * source: `*` 37 | * Protocol: `TCP` 38 | * Port: `80` 39 | * Destination Type: `IP` 40 | * Destination IP: **x.x.x.x** (public IP) 41 | * Translated type: `IP` 42 | * Translated IP: **y.y.y.y** (load balancer IP) 43 | * Translated port: `80` 44 | * click **Add** 45 | 46 | ## How test the solution 47 | 48 | From your PC (public internet), open a browser and type: http://**x.x.x.x.x**. You should see the `azure-vote-app`. 49 | 50 | ## More information 51 | 52 | * Azure Firewall: https://learn.microsoft.com/en-us/azure/firewall/overview 53 | * DNAT: https://en.wikipedia.org/wiki/Network_address_translation 54 | 55 | 56 | -------------------------------------------------------------------------------- /scenarios/firewall-routing.md: -------------------------------------------------------------------------------- 1 | Da spiegare come salgono più LB in base alle annotation dello YAML: da verificare se portare come scenario autonomo -------------------------------------------------------------------------------- /scenarios/front-door.md: -------------------------------------------------------------------------------- 1 | # SCENARIO: Deploy a Front Door to expose AKS workload 2 | 3 | ## Pre-requisites 4 | 5 | To implement this scenario, you need 6 | * a resource group called `play-aks` 7 | * a terminal shell with Azure CLI already connected to your subscription 8 | 9 | ## Architecture 10 | 11 | ![architecture](../images/front-door/front-door.png) 12 | 13 | ## Solution 14 | 15 | In this scenario, you'll use 2 .bicep files to deploy the components for the solutions, the [front-door-1.bicep](../front-door/front-door-1.bicep) will deploy the Hub & Spoke networks, the AKS cluster and the Private Link Service. Then the [front-door-2.bicep](../front-door/front-door-2.bicep) will deploy Front Door and connect it to the Private Link Service as an origin. 16 | Therefore the only exposed endpoint is Front Door, then traffic flows privately using the Microsoft backbone up to the pod. 17 | 18 | **Why I need 2 bicep files?** 19 | As of now, the resource _privateLinkService_ does not support dependsOn property with the existing keyword. That's why we need to execute the 2 files sequentially. 20 | Open a Terminal and execute the following command: 21 | 22 | ``` 23 | az deployment group create --resource-group play-aks --template-file front-door-1.bicep 24 | ``` 25 | 26 | You'll see the following resources provisioned to your resource group 27 | 28 | ![resources](../images/front-door/resources.png) 29 | 30 | As of now, you have an AKS cluster up and running with a deployment and a service exposed using a private load balancer. 31 | You can use any VM in the HUB or in the spokes to communicate with the private IP exposed by the load balancer, but in this scenario we're willing to expose it to the Internet via Front Door. 32 | To do it in a secure way, we'll leverage Front Door **Premium** feature that is Private Link origins. That's why we deployed, not only a Private Load Balancer, but a Private Link Service too, attached to the Load Balancer. 33 | If you see the PLS that has been deployed, you'll see it has no connections yet. 34 | 35 | ![pls-unconnected](../images/front-door/pls-unconnected.png) 36 | 37 | As soon as we deploy the following .bicep, you'll see that there will be a new connection to that PLS, initiated by Front Door. 38 | Open a Terminal and execute the following command: 39 | 40 | ``` 41 | az deployment group create --resource-group play-aks --template-file front-door-2.bicep 42 | ``` 43 | You'll now have a Front Door URL you can use to reach your pod. If you try to browse it, you'll surprisingly notice that you'll receive a Gateway Timeout Error (HTTP 502), why? 44 | The reason is that, Front Door requested a new connection to your Private Link Service, but that request must be approved before the connection effectively starts. 45 | 46 | ![pls-unapproved](../images/front-door/pls-unapproved.png) 47 | 48 | This behaviour is required because Front Door is an Azure Managed Service, globally distributed and external to your tenant, so it is treated as an external entity who is requesting to access a link inside your tenant. 49 | To approve the request, go to the Azure Portal -> Private Link Service. There you'll find the request to approve. 50 | 51 | ![approve-pls](../images/front-door/approve-pls.png) 52 | 53 | # Test scenario 54 | 55 | Once approved, go to your Azure Front Door instance `aks-fd`, copy the endpoint hostname (something like `aks-afd-j6nwrw4dffkkq-e8hrbed2ere6gzb8.z01.azurefd.net`), open your local browser and paste the URL: You'll be able to reach you pod, **served by Front Door!** 🎉 56 | 57 | ![result](../images/front-door/result.png) -------------------------------------------------------------------------------- /testPR: -------------------------------------------------------------------------------- 1 | pippo --------------------------------------------------------------------------------