├── .gitignore ├── LICENSE.txt ├── README.md ├── SECURITY.md ├── azure.yaml ├── azuredeploy.json ├── images ├── apim-loadbalancing-active.png ├── apim-loadbalancing-throttling.png └── intro-loadbalance.png ├── infra ├── abbreviations.json ├── core │ ├── ai │ │ └── cognitiveservices.bicep │ ├── database │ │ ├── cosmos │ │ │ ├── cosmos-account.bicep │ │ │ ├── mongo │ │ │ │ ├── cosmos-mongo-account.bicep │ │ │ │ └── cosmos-mongo-db.bicep │ │ │ └── sql │ │ │ │ ├── cosmos-sql-account.bicep │ │ │ │ ├── cosmos-sql-db.bicep │ │ │ │ ├── cosmos-sql-role-assign.bicep │ │ │ │ └── cosmos-sql-role-def.bicep │ │ ├── postgresql │ │ │ └── flexibleserver.bicep │ │ └── sqlserver │ │ │ └── sqlserver.bicep │ ├── gateway │ │ └── apim.bicep │ ├── host │ │ ├── aks-agent-pool.bicep │ │ ├── aks-managed-cluster.bicep │ │ ├── aks.bicep │ │ ├── appservice-appsettings.bicep │ │ ├── appservice.bicep │ │ ├── appserviceplan.bicep │ │ ├── container-app-upsert.bicep │ │ ├── container-app.bicep │ │ ├── container-apps-environment.bicep │ │ ├── container-apps.bicep │ │ ├── container-registry.bicep │ │ ├── functions.bicep │ │ └── staticwebapp.bicep │ ├── monitor │ │ ├── applicationinsights-dashboard.bicep │ │ ├── applicationinsights.bicep │ │ ├── loganalytics.bicep │ │ └── monitoring.bicep │ ├── networking │ │ ├── cdn-endpoint.bicep │ │ ├── cdn-profile.bicep │ │ └── cdn.bicep │ ├── search │ │ └── search-services.bicep │ ├── security │ │ ├── keyvault-access.bicep │ │ ├── keyvault-secret.bicep │ │ ├── keyvault.bicep │ │ ├── registry-access.bicep │ │ └── role.bicep │ ├── storage │ │ └── storage-account.bicep │ └── testing │ │ └── loadtesting.bicep ├── main.bicep ├── main.parameters.json └── web.bicep └── src ├── .dockerignore ├── .gitignore ├── BackendConfig.cs ├── Dockerfile ├── Program.cs ├── Properties └── launchSettings.json ├── RetryMiddleware.cs ├── YarpConfiguration.cs ├── appsettings.Development.json ├── appsettings.json ├── openai-loadbalancer.csproj └── openai-loadbalancer.sln /.gitignore: -------------------------------------------------------------------------------- 1 | .azure -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) Microsoft Corporation. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | > [!CAUTION] 2 | > This repository is moved to https://github.com/Azure-Samples/openai-aca-lb/ 3 | 4 | 5 | ![Smart load balancing](./images/intro-loadbalance.png) 6 | 7 | # :rocket: Smart load balancing for OpenAI endpoints 8 | 9 | Many service providers, including OpenAI, usually set limits on the number of calls that can be made. In the case of Azure OpenAI, there are token limits (TPM or tokens per minute) and limits on the number of requests per minute (RPM). When a server starts running out of resources or the service limits are exhausted, the provider may issue a 429 or TooManyRequests HTTP Status code, and also a Retry-After response header indicating how much time you should wait until you try the next request. 10 | 11 | The solution presented here is part of comprehensive one that takes into consideration things like a good UX/workflow design, adding application resiliency and fault-handling logic, considering service limits, choosing the right model for the job, the API policies, setting up logging and monitoring among other considerations. This solution seamlessly expose a single endpoint to your applications while keeping an efficient logic to consume two or more OpenAI or any API backends based on availability and priority. 12 | 13 | It is built using the high-performance [YARP C# reverse-proxy](https://github.com/microsoft/reverse-proxy) framework from Microsoft. However, you don't need to understand C# to use it, you can just build the provided Docker image. 14 | This is an alternative solution to the [API Management OpenAI smart load balancer](https://github.com/andredewes/apim-aoai-smart-loadbalancing), with the same logic. 15 | 16 | ## :sparkles: Why do you call this "smart" and different from round-robin load balancers? 17 | 18 | One of the key components of handling OpenAI throttling is to be aware of the HTTP status code error 429 (Too Many Requests). There are [Tokens-Per-Minute and a Requests-Per-Minute](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/quota?tabs=rest#understanding-rate-limits) rate limiting in Azure OpenAI. Both situations will return the same error code 429. 19 | 20 | Together with that HTTP status code 429, Azure OpenAI will also return a HTTP response header called "Retry-After", which is the number of seconds that instance will be unavailable before it starts accepting requests again. 21 | 22 | These errors are normally handled in the client-side by SDKs. This works great if you have a single API endpoint. However, for multiple OpenAI endpoints (used for fallback) you would need to manage the list of URLs in the client-side too, which is not ideal. 23 | 24 | What makes this solution different than others is that it is aware of the "Retry-After" and 429 errors and intelligently sends traffic to other OpenAI backends that are not currently throttling. You can even have a priority order in your backends, so the highest priority are the ones being consumed first while they are not throttling. When throttling kicks in, it will fallback to lower priority backends while your highest ones are waiting to recover. 25 | 26 | Another important feature: there is no time interval between attempts to call different backends. Many of other OpenAI load balancers out there configure a waiting internal (often exponential). While this is a good idea doing at the client side, making a server-side load balancer to wait is not a good practice because you hold your client and consume more server and network capacity during this waiting time. Retries on the server-side should be immediate and to a different endpoint. 27 | 28 | Check this diagram for easier understanding: 29 | 30 | ![normal!](/images/apim-loadbalancing-active.png "Normal scenario") 31 | 32 | ![throttling!](/images/apim-loadbalancing-throttling.png "Throttling scenario") 33 | 34 | ## :1234: Priorities 35 | 36 | One thing that stands out in the above images is the concept of "priority groups". Why do we have that? That's because you might want to consume all your available quota in specific instances before falling back to others. For example, in this scenario: 37 | - You have a [PTU (Provisioned Throughput)](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/provisioned-throughput) deployment. You want to consume all its capacity first because you are paying for this either you use it or not. You can set this instance(s) as **Priority 1** 38 | - Then you have extra deployments using the default S0 tier (token-based consumption model) spread in different Azure regions which you would like to fallback in case your PTU instance is fully occupied and returning errors 429. Here you don't have a fixed pricing model like in PTU but you will consume these endpoints only during the period that PTU is not available. You can set these as **Priority 2** 39 | 40 | Another scenario: 41 | - You don't have any PTU (provisioned) deployment but you would like to have many S0 (token-based consumption model) spread in different Azure regions in case you hit throttling. Let's assume your applications are mostly in USA. 42 | - You then deploy one instance of OpenAI in each region of USA that has OpenAI capacity. You can set these instance(s) as **Priority 1** 43 | - However, if all USA instances are getting throttled, you have another set of endpoints in Canada, which is closest region outside of USA. You can set these instance(s) as **Priority 2** 44 | - Even if Canada also gets throttling at the same at as your USA instances, you can fallback to European regions now. You can set these instance(s) as **Priority 3** 45 | - In the last case if all other previous endpoints are still throttling during the same time, you might even consider having OpenAI endpoints in Asia as "last resort". Latency will be little bit higher but still acceptable. You can set these instance(s) as **Priority 4** 46 | 47 | And what happens if I have multiple backends with the same priority? Let's assume I have 3 OpenAI backends in USA with all Priority = 1 and all of them are not throttling? In this case, the algorithm will randomly pick among these 3 URLs. 48 | 49 | ## :gear: Getting started 50 | 51 | The source code provides a Dockerfile, which means you are free to build and deploy to your own service, as long as it supports container images. 52 | 53 | ### [Option 1] Deploy the service directly to an Azure Container Apps 54 | 55 | If you are not comfortable working with container images and you would like a very easy way to test this load balancer in Azure, you can deploy quickly to [Azure Container Apps](https://azure.microsoft.com/products/container-apps): 56 | 57 | [![Deploy to Azure](https://aka.ms/deploytoazurebutton)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2Fazure%2Faoai-smart-loadbalancing%2Fmain%2Fazuredeploy.json) 58 | 59 | - Clicking the Deploy button above, you will be taken to an Azure page with the required parameters. You need to fill the parameters beginning with "Backend_X_" (see below in [Configuring the OpenAI endpoints](#Configuring-the-OpenAI-endpoints) for more information on what they do) 60 | - After the deployment is finished, go to your newly created Container Apps service and from the Overview menu, get the Application Url of your app. The format will be "https://app-[something].[region].azurecontainerapps.io". This is the URL you will call from your client applications 61 | 62 | 63 | ### [Option 2] Build and deploy as a Docker image 64 | 65 | If you want to clone this repository and build your own image instead of using the pre-built public image: 66 | 67 | ` 68 | docker build -t aoai-smart-loadbalancing:v1 . 69 | ` 70 | 71 | This will use the Dockerfile which will build the source code inside the container itself (no need to have .NET build tools in your host machine) and then it will copy the build output to a new runtime image for ASP.NET 8. Just make sure your Docker version supports [multi-stage](https://docs.docker.com/build/building/multi-stage/) builds. The final image will have around 87 MB. 72 | 73 | ### [Option 3] Deploy the pre-built image from Docker hub 74 | 75 | If you don't want to build the container from the source code, you can pull it from the public Docker registry: 76 | 77 | ` 78 | docker pull andredewes/aoai-smart-loadbalancing:v1 79 | ` 80 | 81 | ### Configuring the OpenAI endpoints 82 | 83 | After you deployed your container service using one of the methods above, it is time to adjust your OpenAI backends configuration using environment variables. 84 | This is the expected format you must provide: 85 | 86 | | Environment variable name | Mandatory | Description | Example | 87 | |---------------------------|-----------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------| 88 | | BACKEND_X_URL | Yes | The full Azure OpenAI URL. Replace "_X_" with the number of your backend. For example, "BACKEND_1_URL" or "BACKEND_2_URL" | https://andre-openai.openai.azure.com | 89 | | BACKEND_X_PRIORITY | Yes | The priority of your OpenAI endpoint. Lower numbers means higher priority. Replace "_X_" with the number of your backend. For example, "BACKEND_1_PRIORITY" or "BACKEND_2_PRIORITY" | 1 | 90 | | BACKEND_X_APIKEY | Yes | The API key of your OpenAI endpoint. Replace "_X_" with the number of your backend. For example, "BACKEND_1_APIKEY" or "BACKEND_2_APIKEY" | 761c427c520d40c991299c66d10c6e40 | 91 | | BACKEND_X_DEPLOYMENT_NAME | No | If this setting is set, the incoming client URL will have the deployment name overridden. For example, an incoming HTTP request is:

https://andre-openai-eastus.openai.azure.com/openai/deployments/gpt35turbo/chat/completions?api-version=2023-07-01-preview

The **gpt35turbo** will be replaced by this setting when the request goes to your backend. In this way, you don't need to have all the deployment names as the same across your endpoints. You can even mix different GPT models (like GPT35 and GPT4), even though this is not recommended.

If nothing is set, the incoming URL coming from your client applications will be kept the same when sending to the backend, it will not be overridden. That means your deployment names must be the same in your OpenAI endpoints. | your-openai-deployment-name | 92 | 93 | For example, let's say you would like to configure the load balancer to have a main endpoint (we call it here BACKEND_1). We set it with the highest priority 1. And then we have two more endpoints as fallback in case the main one is throttling... we define then BACKEND_2 and BACKEND_3 with the same priority of 2: 94 | 95 | 96 | | Environment variable name | Value | 97 | |---------------------------|-----------------------------------------| 98 | | BACKEND_1_URL | https://andre-openai.openai.azure.com | 99 | | BACKEND_1_PRIORITY | 1 | 100 | | BACKEND_1_APIKEY | 33b9996ce4644bc0893c7988bae349af | 101 | | BACKEND_2_URL | https://andre-openai-2.openai.azure.com | 102 | | BACKEND_2_PRIORITY | 2 | 103 | | BACKEND_2_APIKEY | 412ceac74dde451e9ac12581ca50b5c5 | 104 | | BACKEND_3_URL | https://andre-openai-3.openai.azure.com | 105 | | BACKEND_3_PRIORITY | 2 | 106 | | BACKEND_3_APIKEY | 326ec768694d4d469eda2fe2c582ef8b | 107 | 108 | It is important to always create 3 environment variables for each new OpenAI endpoiny that you would like to use. 109 | 110 | ### Load balancer settings 111 | 112 | This is the list of environment variables that are used to configure the load balancer in global scope, not only for specific backend endpoints: 113 | 114 | | Environment variable name | Mandatory | Description | Example | 115 | |---------------------------|-----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------| 116 | | HTTP_TIMEOUT_SECONDS | No | If set, it will change the default 100 seconds timeout when waiting for OpenAI responses to something else.
If a timeout is reached, the endpoint it will be marked unhealthy for 10 seconds and the request will fallback to another backend. | 120 | 117 | 118 | ### Testing the solution 119 | 120 | To test if everything works by running some code of your choice, e.g., this code with OpenAI Python SDK: 121 | ```python 122 | from openai import AzureOpenAI 123 | 124 | client = AzureOpenAI( 125 | azure_endpoint="https://", #if you deployed to Azure Container Apps, it will be 'https://app-[something].[region].azurecontainerapps.io' 126 | api_key="does-not-matter", #The api-key sent by the client SDKs will be overriden by the ones configured in the backend environment variables 127 | api_version="2023-12-01-preview" 128 | ) 129 | 130 | response = client.chat.completions.create( 131 | model="", 132 | messages=[ 133 | {"role": "system", "content": "You are a helpful assistant."}, 134 | {"role": "user", "content": "What is the first letter of the alphabet?"} 135 | ] 136 | ) 137 | print(response) 138 | ``` 139 | 140 | ### Scalability vs Reliability 141 | This solution addresses both scalability and reliability concerns by allowing your total Azure OpenAI quotas to increase and providing server-side failovers transparently for your applications. However, if you are looking purely for a way to increase default quotas, I still would recommend that you follow the official guidance to [request a quota increase](https://learn.microsoft.com/azure/ai-services/openai/quotas-limits#how-to-request-increases-to-the-default-quotas-and-limits). 142 | 143 | ### Multiple load balancer instances 144 | This solution uses the local memory to store the endpoints health state. That means each instance will have its own view of the throttling state of each OpenAI endpoint. What might happen during runtime is this: 145 | - Instance 1 receives a customer request and gets a 429 error from backend 1. It marks that backend as unavailable for X seconds and then reroute that customer request to next backend 146 | - Instance 2 receives a customer request and sends that request again to backend 1 (since its local cached list of backends didn't have the information from instance 1 when it marked as throttled). Backend 1 will respond with error 429 again and instance 2 will also mark it as unavailable and reroutes the request to next backend 147 | 148 | So, it might occur that internally, different instances will try route to throttled backends and will need to retry to another backend. Eventually, all instances will be in sync again at a small cost of unnecessary roundtrips to throttled endpoints. 149 | I honestly think this is a very small price to pay, but if you want to solve that you can always change the source code to use an external shared cache such as Redis, so all instances will share the same cached object. 150 | 151 | Having this in mind, be careful when you configure your hosting container service when it comes to scalability. For instance, the default scaling rules in a Azure Container Apps is the number of concurrent HTTP requests: if it is higher than 10, it will create another container instance. This effect is undesirable for the load balancer as it will create many instances, and that's why the Quick Deploy button in this repo changes that default behavior to only scale the container when CPU usage is higher than 50%. 152 | 153 | ### Logging 154 | The default logging features coming from [YARP](https://microsoft.github.io/reverse-proxy/articles/diagnosing-yarp-issues.html) are not changed here, it is still applicable. For example, you should see these log lines being print in the container console (Stdout) when requests are sucesfully redirected to the backends: 155 | 156 | ``` 157 | info: Yarp.ReverseProxy.Forwarder.HttpForwarder[9] 158 | info: Proxying to https://andre-openai-eastus.openai.azure.com/openai/deployments/gpt35turbo/chat/completions?api-version=2023-07-01-preview HTTP/2 RequestVersionOrLower 159 | info: Yarp.ReverseProxy.Forwarder.HttpForwarder[56] 160 | info: Received HTTP/2.0 response 200. 161 | info: Yarp.ReverseProxy.Forwarder.HttpForwarder[9] 162 | info: Proxying to https://andre-openai-eastus.openai.azure.com/openai/deployments/gpt35turbo/chat/completions?api-version=2023-07-01-preview HTTP/2 RequestVersionOrLower 163 | info: Yarp.ReverseProxy.Forwarder.HttpForwarder[56] 164 | info: Received HTTP/2.0 response 200. 165 | ``` 166 | 167 | Now, these are example of logs generated when the load balancer receives a 429 error from the OpenAI backend: 168 | 169 | ``` 170 | info: Yarp.ReverseProxy.Forwarder.HttpForwarder[56] 171 | info: Received HTTP/2.0 response 429. 172 | warn: Yarp.ReverseProxy.Health.DestinationHealthUpdater[19] 173 | warn: Destination `BACKEND_4` marked as 'Unhealthy` by the passive health check is scheduled for a reactivation in `00:00:07`. 174 | ``` 175 | Notice that it reads the value coming in the "Retry-After" header from OpenAI response and marks that backend as Unhealthy and it also prints how much time it will take to be reactivated. In this case, 7 seconds. 176 | 177 | And this is the log line that appears after that time is elapsed: 178 | 179 | ``` 180 | info: Yarp.ReverseProxy.Health.DestinationHealthUpdater[20] 181 | info: Passive health state of the destination `BACKEND_4` is reset to 'Unknown`. 182 | ``` 183 | 184 | It is OK that it says the status is reset to "Unknown". That means that backend will be actively receiving HTTP requests and its internal state will be updated to Healthy if it receives a 200 response from the OpenAI backend next time. This is called Passive health check and is a [YARP feature](https://microsoft.github.io/reverse-proxy/articles/dests-health-checks.html#passive-health-checks). 185 | 186 | 187 | ## :question: FAQ 188 | 189 | ### What happens if all backends are throttling at the same time? 190 | In that case, the load balancer will route a random backend in the list. Since that endpoint is throttling, it will return the same 429 error as the OpenAI backend. That's why it is **still important for your client application/SDKs to have a logic to handle retries**, even though it should be much less frequent. 191 | 192 | ### Reading the C# logic is hard for me. Can you describe it in plain english? 193 | Sure. That's how it works when the load balancer gets a new incoming request: 194 | 195 | 1. From the list of backends defined in the environment variables, it will pick one backend using this logic: 196 | 1. Selects the highest priority (lower number) that is not currently throttling. If it finds more than one healthy backend with the same priority, it will randomly select one of them 197 | 2. Sends the request to the chosen backend URL 198 | 1. If the backend responds with success (HTTP status code 200), the response is passed to the client and the flow ends here 199 | 2. If the backend responds with error 429 or 5xx 200 | 1. It will read the HTTP response header "Retry-After" to see when it will become available again 201 | 2. Then, it marks that specific backend URL as throttling and also saves what time it will be healthy again 202 | 3. If there are still other available backends in the pool, it runs again the logic to select another backend (go to the point 1. again and so on) 203 | 4. If there are no backends available (all are throttling), it will send the customer request to the first backend defined in the list and return its response 204 | 205 | ## :link: Related articles 206 | - The same load balancer logic but using Azure API Management: [Smart Load-Balancing for Azure OpenAI with Azure API Management](https://github.com/andredewes/apim-aoai-smart-loadbalancing) 207 | -------------------------------------------------------------------------------- /SECURITY.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | ## Security 4 | 5 | Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/Microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet) and [Xamarin](https://github.com/xamarin). 6 | 7 | If you believe you have found a security vulnerability in any Microsoft-owned repository that meets [Microsoft's definition of a security vulnerability](https://aka.ms/security.md/definition), please report it to us as described below. 8 | 9 | ## Reporting Security Issues 10 | 11 | **Please do not report security vulnerabilities through public GitHub issues.** 12 | 13 | Instead, please report them to the Microsoft Security Response Center (MSRC) at [https://msrc.microsoft.com/create-report](https://aka.ms/security.md/msrc/create-report). 14 | 15 | If you prefer to submit without logging in, send email to [secure@microsoft.com](mailto:secure@microsoft.com). If possible, encrypt your message with our PGP key; please download it from the [Microsoft Security Response Center PGP Key page](https://aka.ms/security.md/msrc/pgp). 16 | 17 | You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://www.microsoft.com/msrc). 18 | 19 | Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue: 20 | 21 | * Type of issue (e.g. buffer overflow, SQL injection, cross-site scripting, etc.) 22 | * Full paths of source file(s) related to the manifestation of the issue 23 | * The location of the affected source code (tag/branch/commit or direct URL) 24 | * Any special configuration required to reproduce the issue 25 | * Step-by-step instructions to reproduce the issue 26 | * Proof-of-concept or exploit code (if possible) 27 | * Impact of the issue, including how an attacker might exploit the issue 28 | 29 | This information will help us triage your report more quickly. 30 | 31 | If you are reporting for a bug bounty, more complete reports can contribute to a higher bounty award. Please visit our [Microsoft Bug Bounty Program](https://aka.ms/security.md/msrc/bounty) page for more details about our active programs. 32 | 33 | ## Preferred Languages 34 | 35 | We prefer all communications to be in English. 36 | 37 | ## Policy 38 | 39 | Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://aka.ms/security.md/cvd). 40 | 41 | 42 | -------------------------------------------------------------------------------- /azure.yaml: -------------------------------------------------------------------------------- 1 | # yaml-language-server: $schema=https://raw.githubusercontent.com/Azure/azure-dev/main/schemas/v1.0/azure.yaml.json 2 | 3 | name: aoai-smart-loadbalancing 4 | metadata: 5 | template: aoai-smart-loadbalancing@0.0.1 6 | infra: 7 | provider: "bicep" 8 | -------------------------------------------------------------------------------- /azuredeploy.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", 3 | "contentVersion": "1.0.0.0", 4 | "parameters": { 5 | "containerAppName": { 6 | "type": "string", 7 | "defaultValue": "[format('app-{0}', uniqueString(resourceGroup().id))]", 8 | "metadata": { 9 | "description": "Specifies the name of the container app." 10 | } 11 | }, 12 | "containerAppEnvName": { 13 | "type": "string", 14 | "defaultValue": "[format('env-{0}', uniqueString(resourceGroup().id))]", 15 | "metadata": { 16 | "description": "Specifies the name of the container app environment." 17 | } 18 | }, 19 | "location": { 20 | "type": "string", 21 | "defaultValue": "[resourceGroup().location]", 22 | "metadata": { 23 | "description": "Specifies the location for all resources." 24 | } 25 | }, 26 | "backend_1_url": { 27 | "type": "string", 28 | "minLength": 1, 29 | "metadata": { 30 | "description": "The URL of your first Azure OpenAI endpoint in the following format: https://[name].openai.azure.com" 31 | } 32 | }, 33 | "backend_1_priority": { 34 | "type": "int", 35 | "defaultValue": 1, 36 | "metadata": { 37 | "description": "The priority of your first OpenAI endpoint (lower number means higher priority)" 38 | } 39 | }, 40 | "backend_1_api_key": { 41 | "type": "string", 42 | "minLength": 1, 43 | "metadata": { 44 | "description": "The API key your secibd OpenAI endpoint" 45 | } 46 | }, 47 | "backend_2_url": { 48 | "type": "string", 49 | "minLength": 1, 50 | "metadata": { 51 | "description": "The URL of your second Azure OpenAI endpoint in the following format: https://[name].openai.azure.com" 52 | } 53 | }, 54 | "backend_2_priority": { 55 | "type": "int", 56 | "defaultValue": 2, 57 | "metadata": { 58 | "description": "The priority of your second OpenAI endpoint (lower number means higher priority)" 59 | } 60 | }, 61 | "backend_2_api_key": { 62 | "type": "string", 63 | "minLength": 1, 64 | "metadata": { 65 | "description": "The API key your second OpenAI endpoint" 66 | } 67 | } 68 | }, 69 | "resources": [ 70 | { 71 | "type": "Microsoft.App/managedEnvironments", 72 | "apiVersion": "2022-06-01-preview", 73 | "name": "[parameters('containerAppEnvName')]", 74 | "location": "[parameters('location')]", 75 | "sku": { 76 | "name": "Consumption" 77 | }, 78 | "properties": {} 79 | }, 80 | { 81 | "type": "Microsoft.App/containerApps", 82 | "apiVersion": "2022-06-01-preview", 83 | "name": "[parameters('containerAppName')]", 84 | "location": "[parameters('location')]", 85 | "properties": { 86 | "managedEnvironmentId": "[resourceId('Microsoft.App/managedEnvironments', parameters('containerAppEnvName'))]", 87 | "configuration": { 88 | "ingress": { 89 | "external": true, 90 | "targetPort": 8080, 91 | "allowInsecure": false, 92 | "traffic": [ 93 | { 94 | "latestRevision": true, 95 | "weight": 100 96 | } 97 | ] 98 | } 99 | }, 100 | "template": { 101 | "revisionSuffix": "firstrevision", 102 | "containers": [ 103 | { 104 | "name": "[parameters('containerAppName')]", 105 | "image": "andredewes/aoai-smart-loadbalancing:v1", 106 | "resources": { 107 | "cpu": 1, 108 | "memory": "2Gi" 109 | }, 110 | "env": [ 111 | { 112 | "name": "BACKEND_1_URL", 113 | "value": "[parameters('backend_1_url')]" 114 | }, 115 | { 116 | "name": "BACKEND_1_PRIORITY", 117 | "value": "[string(parameters('backend_1_priority'))]" 118 | }, 119 | { 120 | "name": "BACKEND_1_APIKEY", 121 | "value": "[parameters('backend_1_api_key')]" 122 | }, 123 | { 124 | "name": "BACKEND_2_URL", 125 | "value": "[parameters('backend_2_url')]" 126 | }, 127 | { 128 | "name": "BACKEND_2_PRIORITY", 129 | "value": "[string(parameters('backend_2_priority'))]" 130 | }, 131 | { 132 | "name": "BACKEND_2_APIKEY", 133 | "value": "[parameters('backend_2_api_key')]" 134 | } 135 | ] 136 | } 137 | ], 138 | "scale": { 139 | "minReplicas": 0, 140 | "maxReplicas": 10, 141 | "rules": [ 142 | { 143 | "name": "cpu-50", 144 | "custom": { 145 | "type": "cpu", 146 | "metadata": { 147 | "type": "Utilization", 148 | "value": "50" 149 | } 150 | } 151 | } 152 | ] 153 | } 154 | } 155 | }, 156 | "dependsOn": [ 157 | "[resourceId('Microsoft.App/managedEnvironments', parameters('containerAppEnvName'))]" 158 | ] 159 | } 160 | ], 161 | "outputs": { 162 | "containerAppFQDN": { 163 | "type": "string", 164 | "value": "[reference(resourceId('Microsoft.App/containerApps', parameters('containerAppName'))).configuration.ingress.fqdn]" 165 | } 166 | } 167 | } -------------------------------------------------------------------------------- /images/apim-loadbalancing-active.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Azure/aoai-smart-loadbalancing/8f3d5cf30eadaa4233e2a22856b1169cec58a157/images/apim-loadbalancing-active.png -------------------------------------------------------------------------------- /images/apim-loadbalancing-throttling.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Azure/aoai-smart-loadbalancing/8f3d5cf30eadaa4233e2a22856b1169cec58a157/images/apim-loadbalancing-throttling.png -------------------------------------------------------------------------------- /images/intro-loadbalance.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Azure/aoai-smart-loadbalancing/8f3d5cf30eadaa4233e2a22856b1169cec58a157/images/intro-loadbalance.png -------------------------------------------------------------------------------- /infra/abbreviations.json: -------------------------------------------------------------------------------- 1 | { 2 | "analysisServicesServers": "as", 3 | "apiManagementService": "apim-", 4 | "appConfigurationConfigurationStores": "appcs-", 5 | "appManagedEnvironments": "cae-", 6 | "appContainerApps": "ca-", 7 | "authorizationPolicyDefinitions": "policy-", 8 | "automationAutomationAccounts": "aa-", 9 | "blueprintBlueprints": "bp-", 10 | "blueprintBlueprintsArtifacts": "bpa-", 11 | "cacheRedis": "redis-", 12 | "cdnProfiles": "cdnp-", 13 | "cdnProfilesEndpoints": "cdne-", 14 | "cognitiveServicesAccounts": "cog-", 15 | "cognitiveServicesFormRecognizer": "cog-fr-", 16 | "cognitiveServicesTextAnalytics": "cog-ta-", 17 | "computeAvailabilitySets": "avail-", 18 | "computeCloudServices": "cld-", 19 | "computeDiskEncryptionSets": "des", 20 | "computeDisks": "disk", 21 | "computeDisksOs": "osdisk", 22 | "computeGalleries": "gal", 23 | "computeSnapshots": "snap-", 24 | "computeVirtualMachines": "vm", 25 | "computeVirtualMachineScaleSets": "vmss-", 26 | "containerInstanceContainerGroups": "ci", 27 | "containerRegistryRegistries": "cr", 28 | "containerServiceManagedClusters": "aks-", 29 | "databricksWorkspaces": "dbw-", 30 | "dataFactoryFactories": "adf-", 31 | "dataLakeAnalyticsAccounts": "dla", 32 | "dataLakeStoreAccounts": "dls", 33 | "dataMigrationServices": "dms-", 34 | "dBforMySQLServers": "mysql-", 35 | "dBforPostgreSQLServers": "psql-", 36 | "devicesIotHubs": "iot-", 37 | "devicesProvisioningServices": "provs-", 38 | "devicesProvisioningServicesCertificates": "pcert-", 39 | "documentDBDatabaseAccounts": "cosmos-", 40 | "eventGridDomains": "evgd-", 41 | "eventGridDomainsTopics": "evgt-", 42 | "eventGridEventSubscriptions": "evgs-", 43 | "eventHubNamespaces": "evhns-", 44 | "eventHubNamespacesEventHubs": "evh-", 45 | "hdInsightClustersHadoop": "hadoop-", 46 | "hdInsightClustersHbase": "hbase-", 47 | "hdInsightClustersKafka": "kafka-", 48 | "hdInsightClustersMl": "mls-", 49 | "hdInsightClustersSpark": "spark-", 50 | "hdInsightClustersStorm": "storm-", 51 | "hybridComputeMachines": "arcs-", 52 | "insightsActionGroups": "ag-", 53 | "insightsComponents": "appi-", 54 | "keyVaultVaults": "kv-", 55 | "kubernetesConnectedClusters": "arck", 56 | "kustoClusters": "dec", 57 | "kustoClustersDatabases": "dedb", 58 | "logicIntegrationAccounts": "ia-", 59 | "logicWorkflows": "logic-", 60 | "machineLearningServicesWorkspaces": "mlw-", 61 | "managedIdentityUserAssignedIdentities": "id-", 62 | "managementManagementGroups": "mg-", 63 | "migrateAssessmentProjects": "migr-", 64 | "networkApplicationGateways": "agw-", 65 | "networkApplicationSecurityGroups": "asg-", 66 | "networkAzureFirewalls": "afw-", 67 | "networkBastionHosts": "bas-", 68 | "networkConnections": "con-", 69 | "networkDnsZones": "dnsz-", 70 | "networkExpressRouteCircuits": "erc-", 71 | "networkFirewallPolicies": "afwp-", 72 | "networkFirewallPoliciesWebApplication": "waf", 73 | "networkFirewallPoliciesRuleGroups": "wafrg", 74 | "networkFrontDoors": "fd-", 75 | "networkFrontdoorWebApplicationFirewallPolicies": "fdfp-", 76 | "networkLoadBalancersExternal": "lbe-", 77 | "networkLoadBalancersInternal": "lbi-", 78 | "networkLoadBalancersInboundNatRules": "rule-", 79 | "networkLocalNetworkGateways": "lgw-", 80 | "networkNatGateways": "ng-", 81 | "networkNetworkInterfaces": "nic-", 82 | "networkNetworkSecurityGroups": "nsg-", 83 | "networkNetworkSecurityGroupsSecurityRules": "nsgsr-", 84 | "networkNetworkWatchers": "nw-", 85 | "networkPrivateDnsZones": "pdnsz-", 86 | "networkPrivateLinkServices": "pl-", 87 | "networkPublicIPAddresses": "pip-", 88 | "networkPublicIPPrefixes": "ippre-", 89 | "networkRouteFilters": "rf-", 90 | "networkRouteTables": "rt-", 91 | "networkRouteTablesRoutes": "udr-", 92 | "networkTrafficManagerProfiles": "traf-", 93 | "networkVirtualNetworkGateways": "vgw-", 94 | "networkVirtualNetworks": "vnet-", 95 | "networkVirtualNetworksSubnets": "snet-", 96 | "networkVirtualNetworksVirtualNetworkPeerings": "peer-", 97 | "networkVirtualWans": "vwan-", 98 | "networkVpnGateways": "vpng-", 99 | "networkVpnGatewaysVpnConnections": "vcn-", 100 | "networkVpnGatewaysVpnSites": "vst-", 101 | "notificationHubsNamespaces": "ntfns-", 102 | "notificationHubsNamespacesNotificationHubs": "ntf-", 103 | "operationalInsightsWorkspaces": "log-", 104 | "portalDashboards": "dash-", 105 | "powerBIDedicatedCapacities": "pbi-", 106 | "purviewAccounts": "pview-", 107 | "recoveryServicesVaults": "rsv-", 108 | "resourcesResourceGroups": "rg-", 109 | "searchSearchServices": "srch-", 110 | "serviceBusNamespaces": "sb-", 111 | "serviceBusNamespacesQueues": "sbq-", 112 | "serviceBusNamespacesTopics": "sbt-", 113 | "serviceEndPointPolicies": "se-", 114 | "serviceFabricClusters": "sf-", 115 | "signalRServiceSignalR": "sigr", 116 | "sqlManagedInstances": "sqlmi-", 117 | "sqlServers": "sql-", 118 | "sqlServersDataWarehouse": "sqldw-", 119 | "sqlServersDatabases": "sqldb-", 120 | "sqlServersDatabasesStretch": "sqlstrdb-", 121 | "storageStorageAccounts": "st", 122 | "storageStorageAccountsVm": "stvm", 123 | "storSimpleManagers": "ssimp", 124 | "streamAnalyticsCluster": "asa-", 125 | "synapseWorkspaces": "syn", 126 | "synapseWorkspacesAnalyticsWorkspaces": "synw", 127 | "synapseWorkspacesSqlPoolsDedicated": "syndp", 128 | "synapseWorkspacesSqlPoolsSpark": "synsp", 129 | "timeSeriesInsightsEnvironments": "tsi-", 130 | "webServerFarms": "plan-", 131 | "webSitesAppService": "app-", 132 | "webSitesAppServiceEnvironment": "ase-", 133 | "webSitesFunctions": "func-", 134 | "webStaticSites": "stapp-" 135 | } -------------------------------------------------------------------------------- /infra/core/ai/cognitiveservices.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure Cognitive Services instance.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | @description('The custom subdomain name used to access the API. Defaults to the value of the name parameter.') 6 | param customSubDomainName string = name 7 | param deployments array = [] 8 | param kind string = 'OpenAI' 9 | 10 | @allowed([ 'Enabled', 'Disabled' ]) 11 | param publicNetworkAccess string = 'Enabled' 12 | param sku object = { 13 | name: 'S0' 14 | } 15 | 16 | param allowedIpRules array = [] 17 | param networkAcls object = empty(allowedIpRules) ? { 18 | defaultAction: 'Allow' 19 | } : { 20 | ipRules: allowedIpRules 21 | defaultAction: 'Deny' 22 | } 23 | 24 | resource account 'Microsoft.CognitiveServices/accounts@2023-05-01' = { 25 | name: name 26 | location: location 27 | tags: tags 28 | kind: kind 29 | properties: { 30 | customSubDomainName: customSubDomainName 31 | publicNetworkAccess: publicNetworkAccess 32 | networkAcls: networkAcls 33 | } 34 | sku: sku 35 | } 36 | 37 | @batchSize(1) 38 | resource deployment 'Microsoft.CognitiveServices/accounts/deployments@2023-05-01' = [for deployment in deployments: { 39 | parent: account 40 | name: deployment.name 41 | properties: { 42 | model: deployment.model 43 | raiPolicyName: contains(deployment, 'raiPolicyName') ? deployment.raiPolicyName : null 44 | } 45 | sku: contains(deployment, 'sku') ? deployment.sku : { 46 | name: 'Standard' 47 | capacity: 20 48 | } 49 | }] 50 | 51 | output endpoint string = account.properties.endpoint 52 | output id string = account.id 53 | output name string = account.name 54 | -------------------------------------------------------------------------------- /infra/core/database/cosmos/cosmos-account.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure Cosmos DB account.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | param connectionStringKey string = 'AZURE-COSMOS-CONNECTION-STRING' 7 | param keyVaultName string 8 | 9 | @allowed([ 'GlobalDocumentDB', 'MongoDB', 'Parse' ]) 10 | param kind string 11 | 12 | resource cosmos 'Microsoft.DocumentDB/databaseAccounts@2022-08-15' = { 13 | name: name 14 | kind: kind 15 | location: location 16 | tags: tags 17 | properties: { 18 | consistencyPolicy: { defaultConsistencyLevel: 'Session' } 19 | locations: [ 20 | { 21 | locationName: location 22 | failoverPriority: 0 23 | isZoneRedundant: false 24 | } 25 | ] 26 | databaseAccountOfferType: 'Standard' 27 | enableAutomaticFailover: false 28 | enableMultipleWriteLocations: false 29 | apiProperties: (kind == 'MongoDB') ? { serverVersion: '4.2' } : {} 30 | capabilities: [ { name: 'EnableServerless' } ] 31 | } 32 | } 33 | 34 | resource cosmosConnectionString 'Microsoft.KeyVault/vaults/secrets@2022-07-01' = { 35 | parent: keyVault 36 | name: connectionStringKey 37 | properties: { 38 | value: cosmos.listConnectionStrings().connectionStrings[0].connectionString 39 | } 40 | } 41 | 42 | resource keyVault 'Microsoft.KeyVault/vaults@2022-07-01' existing = { 43 | name: keyVaultName 44 | } 45 | 46 | output connectionStringKey string = connectionStringKey 47 | output endpoint string = cosmos.properties.documentEndpoint 48 | output id string = cosmos.id 49 | output name string = cosmos.name 50 | -------------------------------------------------------------------------------- /infra/core/database/cosmos/mongo/cosmos-mongo-account.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure Cosmos DB for MongoDB account.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | param keyVaultName string 7 | param connectionStringKey string = 'AZURE-COSMOS-CONNECTION-STRING' 8 | 9 | module cosmos '../../cosmos/cosmos-account.bicep' = { 10 | name: 'cosmos-account' 11 | params: { 12 | name: name 13 | location: location 14 | connectionStringKey: connectionStringKey 15 | keyVaultName: keyVaultName 16 | kind: 'MongoDB' 17 | tags: tags 18 | } 19 | } 20 | 21 | output connectionStringKey string = cosmos.outputs.connectionStringKey 22 | output endpoint string = cosmos.outputs.endpoint 23 | output id string = cosmos.outputs.id 24 | -------------------------------------------------------------------------------- /infra/core/database/cosmos/mongo/cosmos-mongo-db.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure Cosmos DB for MongoDB account with a database.' 2 | param accountName string 3 | param databaseName string 4 | param location string = resourceGroup().location 5 | param tags object = {} 6 | 7 | param collections array = [] 8 | param connectionStringKey string = 'AZURE-COSMOS-CONNECTION-STRING' 9 | param keyVaultName string 10 | 11 | module cosmos 'cosmos-mongo-account.bicep' = { 12 | name: 'cosmos-mongo-account' 13 | params: { 14 | name: accountName 15 | location: location 16 | keyVaultName: keyVaultName 17 | tags: tags 18 | connectionStringKey: connectionStringKey 19 | } 20 | } 21 | 22 | resource database 'Microsoft.DocumentDB/databaseAccounts/mongodbDatabases@2022-08-15' = { 23 | name: '${accountName}/${databaseName}' 24 | tags: tags 25 | properties: { 26 | resource: { id: databaseName } 27 | } 28 | 29 | resource list 'collections' = [for collection in collections: { 30 | name: collection.name 31 | properties: { 32 | resource: { 33 | id: collection.id 34 | shardKey: { _id: collection.shardKey } 35 | indexes: [ { key: { keys: [ collection.indexKey ] } } ] 36 | } 37 | } 38 | }] 39 | 40 | dependsOn: [ 41 | cosmos 42 | ] 43 | } 44 | 45 | output connectionStringKey string = connectionStringKey 46 | output databaseName string = databaseName 47 | output endpoint string = cosmos.outputs.endpoint 48 | -------------------------------------------------------------------------------- /infra/core/database/cosmos/sql/cosmos-sql-account.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure Cosmos DB for NoSQL account.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | param keyVaultName string 7 | 8 | module cosmos '../../cosmos/cosmos-account.bicep' = { 9 | name: 'cosmos-account' 10 | params: { 11 | name: name 12 | location: location 13 | tags: tags 14 | keyVaultName: keyVaultName 15 | kind: 'GlobalDocumentDB' 16 | } 17 | } 18 | 19 | output connectionStringKey string = cosmos.outputs.connectionStringKey 20 | output endpoint string = cosmos.outputs.endpoint 21 | output id string = cosmos.outputs.id 22 | output name string = cosmos.outputs.name 23 | -------------------------------------------------------------------------------- /infra/core/database/cosmos/sql/cosmos-sql-db.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure Cosmos DB for NoSQL account with a database.' 2 | param accountName string 3 | param databaseName string 4 | param location string = resourceGroup().location 5 | param tags object = {} 6 | 7 | param containers array = [] 8 | param keyVaultName string 9 | param principalIds array = [] 10 | 11 | module cosmos 'cosmos-sql-account.bicep' = { 12 | name: 'cosmos-sql-account' 13 | params: { 14 | name: accountName 15 | location: location 16 | tags: tags 17 | keyVaultName: keyVaultName 18 | } 19 | } 20 | 21 | resource database 'Microsoft.DocumentDB/databaseAccounts/sqlDatabases@2022-05-15' = { 22 | name: '${accountName}/${databaseName}' 23 | properties: { 24 | resource: { id: databaseName } 25 | } 26 | 27 | resource list 'containers' = [for container in containers: { 28 | name: container.name 29 | properties: { 30 | resource: { 31 | id: container.id 32 | partitionKey: { paths: [ container.partitionKey ] } 33 | } 34 | options: {} 35 | } 36 | }] 37 | 38 | dependsOn: [ 39 | cosmos 40 | ] 41 | } 42 | 43 | module roleDefinition 'cosmos-sql-role-def.bicep' = { 44 | name: 'cosmos-sql-role-definition' 45 | params: { 46 | accountName: accountName 47 | } 48 | dependsOn: [ 49 | cosmos 50 | database 51 | ] 52 | } 53 | 54 | // We need batchSize(1) here because sql role assignments have to be done sequentially 55 | @batchSize(1) 56 | module userRole 'cosmos-sql-role-assign.bicep' = [for principalId in principalIds: if (!empty(principalId)) { 57 | name: 'cosmos-sql-user-role-${uniqueString(principalId)}' 58 | params: { 59 | accountName: accountName 60 | roleDefinitionId: roleDefinition.outputs.id 61 | principalId: principalId 62 | } 63 | dependsOn: [ 64 | cosmos 65 | database 66 | ] 67 | }] 68 | 69 | output accountId string = cosmos.outputs.id 70 | output accountName string = cosmos.outputs.name 71 | output connectionStringKey string = cosmos.outputs.connectionStringKey 72 | output databaseName string = databaseName 73 | output endpoint string = cosmos.outputs.endpoint 74 | output roleDefinitionId string = roleDefinition.outputs.id 75 | -------------------------------------------------------------------------------- /infra/core/database/cosmos/sql/cosmos-sql-role-assign.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates a SQL role assignment under an Azure Cosmos DB account.' 2 | param accountName string 3 | 4 | param roleDefinitionId string 5 | param principalId string = '' 6 | 7 | resource role 'Microsoft.DocumentDB/databaseAccounts/sqlRoleAssignments@2022-05-15' = { 8 | parent: cosmos 9 | name: guid(roleDefinitionId, principalId, cosmos.id) 10 | properties: { 11 | principalId: principalId 12 | roleDefinitionId: roleDefinitionId 13 | scope: cosmos.id 14 | } 15 | } 16 | 17 | resource cosmos 'Microsoft.DocumentDB/databaseAccounts@2022-08-15' existing = { 18 | name: accountName 19 | } 20 | -------------------------------------------------------------------------------- /infra/core/database/cosmos/sql/cosmos-sql-role-def.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates a SQL role definition under an Azure Cosmos DB account.' 2 | param accountName string 3 | 4 | resource roleDefinition 'Microsoft.DocumentDB/databaseAccounts/sqlRoleDefinitions@2022-08-15' = { 5 | parent: cosmos 6 | name: guid(cosmos.id, accountName, 'sql-role') 7 | properties: { 8 | assignableScopes: [ 9 | cosmos.id 10 | ] 11 | permissions: [ 12 | { 13 | dataActions: [ 14 | 'Microsoft.DocumentDB/databaseAccounts/readMetadata' 15 | 'Microsoft.DocumentDB/databaseAccounts/sqlDatabases/containers/items/*' 16 | 'Microsoft.DocumentDB/databaseAccounts/sqlDatabases/containers/*' 17 | ] 18 | notDataActions: [] 19 | } 20 | ] 21 | roleName: 'Reader Writer' 22 | type: 'CustomRole' 23 | } 24 | } 25 | 26 | resource cosmos 'Microsoft.DocumentDB/databaseAccounts@2022-08-15' existing = { 27 | name: accountName 28 | } 29 | 30 | output id string = roleDefinition.id 31 | -------------------------------------------------------------------------------- /infra/core/database/postgresql/flexibleserver.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure Database for PostgreSQL - Flexible Server.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | param sku object 7 | param storage object 8 | param administratorLogin string 9 | @secure() 10 | param administratorLoginPassword string 11 | param databaseNames array = [] 12 | param allowAzureIPsFirewall bool = false 13 | param allowAllIPsFirewall bool = false 14 | param allowedSingleIPs array = [] 15 | 16 | // PostgreSQL version 17 | param version string 18 | 19 | // Latest official version 2022-12-01 does not have Bicep types available 20 | resource postgresServer 'Microsoft.DBforPostgreSQL/flexibleServers@2022-12-01' = { 21 | location: location 22 | tags: tags 23 | name: name 24 | sku: sku 25 | properties: { 26 | version: version 27 | administratorLogin: administratorLogin 28 | administratorLoginPassword: administratorLoginPassword 29 | storage: storage 30 | highAvailability: { 31 | mode: 'Disabled' 32 | } 33 | } 34 | 35 | resource database 'databases' = [for name in databaseNames: { 36 | name: name 37 | }] 38 | 39 | resource firewall_all 'firewallRules' = if (allowAllIPsFirewall) { 40 | name: 'allow-all-IPs' 41 | properties: { 42 | startIpAddress: '0.0.0.0' 43 | endIpAddress: '255.255.255.255' 44 | } 45 | } 46 | 47 | resource firewall_azure 'firewallRules' = if (allowAzureIPsFirewall) { 48 | name: 'allow-all-azure-internal-IPs' 49 | properties: { 50 | startIpAddress: '0.0.0.0' 51 | endIpAddress: '0.0.0.0' 52 | } 53 | } 54 | 55 | resource firewall_single 'firewallRules' = [for ip in allowedSingleIPs: { 56 | name: 'allow-single-${replace(ip, '.', '')}' 57 | properties: { 58 | startIpAddress: ip 59 | endIpAddress: ip 60 | } 61 | }] 62 | 63 | } 64 | 65 | output POSTGRES_DOMAIN_NAME string = postgresServer.properties.fullyQualifiedDomainName 66 | -------------------------------------------------------------------------------- /infra/core/database/sqlserver/sqlserver.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure SQL Server instance.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | param appUser string = 'appUser' 7 | param databaseName string 8 | param keyVaultName string 9 | param sqlAdmin string = 'sqlAdmin' 10 | param connectionStringKey string = 'AZURE-SQL-CONNECTION-STRING' 11 | 12 | @secure() 13 | param sqlAdminPassword string 14 | @secure() 15 | param appUserPassword string 16 | 17 | resource sqlServer 'Microsoft.Sql/servers@2022-05-01-preview' = { 18 | name: name 19 | location: location 20 | tags: tags 21 | properties: { 22 | version: '12.0' 23 | minimalTlsVersion: '1.2' 24 | publicNetworkAccess: 'Enabled' 25 | administratorLogin: sqlAdmin 26 | administratorLoginPassword: sqlAdminPassword 27 | } 28 | 29 | resource database 'databases' = { 30 | name: databaseName 31 | location: location 32 | } 33 | 34 | resource firewall 'firewallRules' = { 35 | name: 'Azure Services' 36 | properties: { 37 | // Allow all clients 38 | // Note: range [0.0.0.0-0.0.0.0] means "allow all Azure-hosted clients only". 39 | // This is not sufficient, because we also want to allow direct access from developer machine, for debugging purposes. 40 | startIpAddress: '0.0.0.1' 41 | endIpAddress: '255.255.255.254' 42 | } 43 | } 44 | } 45 | 46 | resource sqlDeploymentScript 'Microsoft.Resources/deploymentScripts@2020-10-01' = { 47 | name: '${name}-deployment-script' 48 | location: location 49 | kind: 'AzureCLI' 50 | properties: { 51 | azCliVersion: '2.37.0' 52 | retentionInterval: 'PT1H' // Retain the script resource for 1 hour after it ends running 53 | timeout: 'PT5M' // Five minutes 54 | cleanupPreference: 'OnSuccess' 55 | environmentVariables: [ 56 | { 57 | name: 'APPUSERNAME' 58 | value: appUser 59 | } 60 | { 61 | name: 'APPUSERPASSWORD' 62 | secureValue: appUserPassword 63 | } 64 | { 65 | name: 'DBNAME' 66 | value: databaseName 67 | } 68 | { 69 | name: 'DBSERVER' 70 | value: sqlServer.properties.fullyQualifiedDomainName 71 | } 72 | { 73 | name: 'SQLCMDPASSWORD' 74 | secureValue: sqlAdminPassword 75 | } 76 | { 77 | name: 'SQLADMIN' 78 | value: sqlAdmin 79 | } 80 | ] 81 | 82 | scriptContent: ''' 83 | wget https://github.com/microsoft/go-sqlcmd/releases/download/v0.8.1/sqlcmd-v0.8.1-linux-x64.tar.bz2 84 | tar x -f sqlcmd-v0.8.1-linux-x64.tar.bz2 -C . 85 | 86 | cat < ./initDb.sql 87 | drop user if exists ${APPUSERNAME} 88 | go 89 | create user ${APPUSERNAME} with password = '${APPUSERPASSWORD}' 90 | go 91 | alter role db_owner add member ${APPUSERNAME} 92 | go 93 | SCRIPT_END 94 | 95 | ./sqlcmd -S ${DBSERVER} -d ${DBNAME} -U ${SQLADMIN} -i ./initDb.sql 96 | ''' 97 | } 98 | } 99 | 100 | resource sqlAdminPasswordSecret 'Microsoft.KeyVault/vaults/secrets@2022-07-01' = { 101 | parent: keyVault 102 | name: 'sqlAdminPassword' 103 | properties: { 104 | value: sqlAdminPassword 105 | } 106 | } 107 | 108 | resource appUserPasswordSecret 'Microsoft.KeyVault/vaults/secrets@2022-07-01' = { 109 | parent: keyVault 110 | name: 'appUserPassword' 111 | properties: { 112 | value: appUserPassword 113 | } 114 | } 115 | 116 | resource sqlAzureConnectionStringSercret 'Microsoft.KeyVault/vaults/secrets@2022-07-01' = { 117 | parent: keyVault 118 | name: connectionStringKey 119 | properties: { 120 | value: '${connectionString}; Password=${appUserPassword}' 121 | } 122 | } 123 | 124 | resource keyVault 'Microsoft.KeyVault/vaults@2022-07-01' existing = { 125 | name: keyVaultName 126 | } 127 | 128 | var connectionString = 'Server=${sqlServer.properties.fullyQualifiedDomainName}; Database=${sqlServer::database.name}; User=${appUser}' 129 | output connectionStringKey string = connectionStringKey 130 | output databaseName string = sqlServer::database.name 131 | -------------------------------------------------------------------------------- /infra/core/gateway/apim.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure API Management instance.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | @description('The email address of the owner of the service') 7 | @minLength(1) 8 | param publisherEmail string = 'noreply@microsoft.com' 9 | 10 | @description('The name of the owner of the service') 11 | @minLength(1) 12 | param publisherName string = 'n/a' 13 | 14 | @description('The pricing tier of this API Management service') 15 | @allowed([ 16 | 'Consumption' 17 | 'Developer' 18 | 'Standard' 19 | 'Premium' 20 | ]) 21 | param sku string = 'Consumption' 22 | 23 | @description('The instance size of this API Management service.') 24 | @allowed([ 0, 1, 2 ]) 25 | param skuCount int = 0 26 | 27 | @description('Azure Application Insights Name') 28 | param applicationInsightsName string 29 | 30 | resource apimService 'Microsoft.ApiManagement/service@2021-08-01' = { 31 | name: name 32 | location: location 33 | tags: union(tags, { 'azd-service-name': name }) 34 | sku: { 35 | name: sku 36 | capacity: (sku == 'Consumption') ? 0 : ((sku == 'Developer') ? 1 : skuCount) 37 | } 38 | properties: { 39 | publisherEmail: publisherEmail 40 | publisherName: publisherName 41 | // Custom properties are not supported for Consumption SKU 42 | customProperties: sku == 'Consumption' ? {} : { 43 | 'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA': 'false' 44 | 'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA': 'false' 45 | 'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_RSA_WITH_AES_128_GCM_SHA256': 'false' 46 | 'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_RSA_WITH_AES_256_CBC_SHA256': 'false' 47 | 'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_RSA_WITH_AES_128_CBC_SHA256': 'false' 48 | 'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_RSA_WITH_AES_256_CBC_SHA': 'false' 49 | 'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TLS_RSA_WITH_AES_128_CBC_SHA': 'false' 50 | 'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Ciphers.TripleDes168': 'false' 51 | 'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Protocols.Tls10': 'false' 52 | 'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Protocols.Tls11': 'false' 53 | 'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Protocols.Ssl30': 'false' 54 | 'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Backend.Protocols.Tls10': 'false' 55 | 'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Backend.Protocols.Tls11': 'false' 56 | 'Microsoft.WindowsAzure.ApiManagement.Gateway.Security.Backend.Protocols.Ssl30': 'false' 57 | } 58 | } 59 | } 60 | 61 | resource apimLogger 'Microsoft.ApiManagement/service/loggers@2021-12-01-preview' = if (!empty(applicationInsightsName)) { 62 | name: 'app-insights-logger' 63 | parent: apimService 64 | properties: { 65 | credentials: { 66 | instrumentationKey: applicationInsights.properties.InstrumentationKey 67 | } 68 | description: 'Logger to Azure Application Insights' 69 | isBuffered: false 70 | loggerType: 'applicationInsights' 71 | resourceId: applicationInsights.id 72 | } 73 | } 74 | 75 | resource applicationInsights 'Microsoft.Insights/components@2020-02-02' existing = if (!empty(applicationInsightsName)) { 76 | name: applicationInsightsName 77 | } 78 | 79 | output apimServiceName string = apimService.name 80 | -------------------------------------------------------------------------------- /infra/core/host/aks-agent-pool.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Adds an agent pool to an Azure Kubernetes Service (AKS) cluster.' 2 | param clusterName string 3 | 4 | @description('The agent pool name') 5 | param name string 6 | 7 | @description('The agent pool configuration') 8 | param config object 9 | 10 | resource aksCluster 'Microsoft.ContainerService/managedClusters@2023-03-02-preview' existing = { 11 | name: clusterName 12 | } 13 | 14 | resource nodePool 'Microsoft.ContainerService/managedClusters/agentPools@2023-03-02-preview' = { 15 | parent: aksCluster 16 | name: name 17 | properties: config 18 | } 19 | -------------------------------------------------------------------------------- /infra/core/host/aks-managed-cluster.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure Kubernetes Service (AKS) cluster with a system agent pool.' 2 | @description('The name for the AKS managed cluster') 3 | param name string 4 | 5 | @description('The name of the resource group for the managed resources of the AKS cluster') 6 | param nodeResourceGroupName string = '' 7 | 8 | @description('The Azure region/location for the AKS resources') 9 | param location string = resourceGroup().location 10 | 11 | @description('Custom tags to apply to the AKS resources') 12 | param tags object = {} 13 | 14 | @description('Kubernetes Version') 15 | param kubernetesVersion string = '1.25.5' 16 | 17 | @description('Whether RBAC is enabled for local accounts') 18 | param enableRbac bool = true 19 | 20 | // Add-ons 21 | @description('Whether web app routing (preview) add-on is enabled') 22 | param webAppRoutingAddon bool = true 23 | 24 | // AAD Integration 25 | @description('Enable Azure Active Directory integration') 26 | param enableAad bool = false 27 | 28 | @description('Enable RBAC using AAD') 29 | param enableAzureRbac bool = false 30 | 31 | @description('The Tenant ID associated to the Azure Active Directory') 32 | param aadTenantId string = '' 33 | 34 | @description('The load balancer SKU to use for ingress into the AKS cluster') 35 | @allowed([ 'basic', 'standard' ]) 36 | param loadBalancerSku string = 'standard' 37 | 38 | @description('Network plugin used for building the Kubernetes network.') 39 | @allowed([ 'azure', 'kubenet', 'none' ]) 40 | param networkPlugin string = 'azure' 41 | 42 | @description('Network policy used for building the Kubernetes network.') 43 | @allowed([ 'azure', 'calico' ]) 44 | param networkPolicy string = 'azure' 45 | 46 | @description('If set to true, getting static credentials will be disabled for this cluster.') 47 | param disableLocalAccounts bool = false 48 | 49 | @description('The managed cluster SKU.') 50 | @allowed([ 'Free', 'Paid', 'Standard' ]) 51 | param sku string = 'Free' 52 | 53 | @description('Configuration of AKS add-ons') 54 | param addOns object = {} 55 | 56 | @description('The log analytics workspace id used for logging & monitoring') 57 | param workspaceId string = '' 58 | 59 | @description('The node pool configuration for the System agent pool') 60 | param systemPoolConfig object 61 | 62 | @description('The DNS prefix to associate with the AKS cluster') 63 | param dnsPrefix string = '' 64 | 65 | resource aks 'Microsoft.ContainerService/managedClusters@2023-03-02-preview' = { 66 | name: name 67 | location: location 68 | tags: tags 69 | identity: { 70 | type: 'SystemAssigned' 71 | } 72 | sku: { 73 | name: 'Base' 74 | tier: sku 75 | } 76 | properties: { 77 | nodeResourceGroup: !empty(nodeResourceGroupName) ? nodeResourceGroupName : 'rg-mc-${name}' 78 | kubernetesVersion: kubernetesVersion 79 | dnsPrefix: empty(dnsPrefix) ? '${name}-dns' : dnsPrefix 80 | enableRBAC: enableRbac 81 | aadProfile: enableAad ? { 82 | managed: true 83 | enableAzureRBAC: enableAzureRbac 84 | tenantID: aadTenantId 85 | } : null 86 | agentPoolProfiles: [ 87 | systemPoolConfig 88 | ] 89 | networkProfile: { 90 | loadBalancerSku: loadBalancerSku 91 | networkPlugin: networkPlugin 92 | networkPolicy: networkPolicy 93 | } 94 | disableLocalAccounts: disableLocalAccounts && enableAad 95 | addonProfiles: addOns 96 | ingressProfile: { 97 | webAppRouting: { 98 | enabled: webAppRoutingAddon 99 | } 100 | } 101 | } 102 | } 103 | 104 | var aksDiagCategories = [ 105 | 'cluster-autoscaler' 106 | 'kube-controller-manager' 107 | 'kube-audit-admin' 108 | 'guard' 109 | ] 110 | 111 | // TODO: Update diagnostics to be its own module 112 | // Blocking issue: https://github.com/Azure/bicep/issues/622 113 | // Unable to pass in a `resource` scope or unable to use string interpolation in resource types 114 | resource diagnostics 'Microsoft.Insights/diagnosticSettings@2021-05-01-preview' = if (!empty(workspaceId)) { 115 | name: 'aks-diagnostics' 116 | scope: aks 117 | properties: { 118 | workspaceId: workspaceId 119 | logs: [for category in aksDiagCategories: { 120 | category: category 121 | enabled: true 122 | }] 123 | metrics: [ 124 | { 125 | category: 'AllMetrics' 126 | enabled: true 127 | } 128 | ] 129 | } 130 | } 131 | 132 | @description('The resource name of the AKS cluster') 133 | output clusterName string = aks.name 134 | 135 | @description('The AKS cluster identity') 136 | output clusterIdentity object = { 137 | clientId: aks.properties.identityProfile.kubeletidentity.clientId 138 | objectId: aks.properties.identityProfile.kubeletidentity.objectId 139 | resourceId: aks.properties.identityProfile.kubeletidentity.resourceId 140 | } 141 | -------------------------------------------------------------------------------- /infra/core/host/aks.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure Kubernetes Service (AKS) cluster with a system agent pool as well as an additional user agent pool.' 2 | @description('The name for the AKS managed cluster') 3 | param name string 4 | 5 | @description('The name for the Azure container registry (ACR)') 6 | param containerRegistryName string 7 | 8 | @description('The name of the connected log analytics workspace') 9 | param logAnalyticsName string = '' 10 | 11 | @description('The name of the keyvault to grant access') 12 | param keyVaultName string 13 | 14 | @description('The Azure region/location for the AKS resources') 15 | param location string = resourceGroup().location 16 | 17 | @description('Custom tags to apply to the AKS resources') 18 | param tags object = {} 19 | 20 | @description('AKS add-ons configuration') 21 | param addOns object = { 22 | azurePolicy: { 23 | enabled: true 24 | config: { 25 | version: 'v2' 26 | } 27 | } 28 | keyVault: { 29 | enabled: true 30 | config: { 31 | enableSecretRotation: 'true' 32 | rotationPollInterval: '2m' 33 | } 34 | } 35 | openServiceMesh: { 36 | enabled: false 37 | config: {} 38 | } 39 | omsAgent: { 40 | enabled: true 41 | config: {} 42 | } 43 | applicationGateway: { 44 | enabled: false 45 | config: {} 46 | } 47 | } 48 | 49 | @allowed([ 50 | 'CostOptimised' 51 | 'Standard' 52 | 'HighSpec' 53 | 'Custom' 54 | ]) 55 | @description('The System Pool Preset sizing') 56 | param systemPoolType string = 'CostOptimised' 57 | 58 | @allowed([ 59 | '' 60 | 'CostOptimised' 61 | 'Standard' 62 | 'HighSpec' 63 | 'Custom' 64 | ]) 65 | @description('The User Pool Preset sizing') 66 | param agentPoolType string = '' 67 | 68 | // Configure system / user agent pools 69 | @description('Custom configuration of system node pool') 70 | param systemPoolConfig object = {} 71 | @description('Custom configuration of user node pool') 72 | param agentPoolConfig object = {} 73 | 74 | // Configure AKS add-ons 75 | var omsAgentConfig = (!empty(logAnalyticsName) && !empty(addOns.omsAgent) && addOns.omsAgent.enabled) ? union( 76 | addOns.omsAgent, 77 | { 78 | config: { 79 | logAnalyticsWorkspaceResourceID: logAnalytics.id 80 | } 81 | } 82 | ) : {} 83 | 84 | var addOnsConfig = union( 85 | (!empty(addOns.azurePolicy) && addOns.azurePolicy.enabled) ? { azurepolicy: addOns.azurePolicy } : {}, 86 | (!empty(addOns.keyVault) && addOns.keyVault.enabled) ? { azureKeyvaultSecretsProvider: addOns.keyVault } : {}, 87 | (!empty(addOns.openServiceMesh) && addOns.openServiceMesh.enabled) ? { openServiceMesh: addOns.openServiceMesh } : {}, 88 | (!empty(addOns.omsAgent) && addOns.omsAgent.enabled) ? { omsagent: omsAgentConfig } : {}, 89 | (!empty(addOns.applicationGateway) && addOns.applicationGateway.enabled) ? { ingressApplicationGateway: addOns.applicationGateway } : {} 90 | ) 91 | 92 | // Link to existing log analytics workspace when available 93 | resource logAnalytics 'Microsoft.OperationalInsights/workspaces@2021-12-01-preview' existing = if (!empty(logAnalyticsName)) { 94 | name: logAnalyticsName 95 | } 96 | 97 | var systemPoolSpec = !empty(systemPoolConfig) ? systemPoolConfig : nodePoolPresets[systemPoolType] 98 | 99 | // Create the primary AKS cluster resources and system node pool 100 | module managedCluster 'aks-managed-cluster.bicep' = { 101 | name: 'managed-cluster' 102 | params: { 103 | name: name 104 | location: location 105 | tags: tags 106 | systemPoolConfig: union( 107 | { name: 'npsystem', mode: 'System' }, 108 | nodePoolBase, 109 | systemPoolSpec 110 | ) 111 | addOns: addOnsConfig 112 | workspaceId: !empty(logAnalyticsName) ? logAnalytics.id : '' 113 | } 114 | } 115 | 116 | var hasAgentPool = !empty(agentPoolConfig) || !empty(agentPoolType) 117 | var agentPoolSpec = hasAgentPool && !empty(agentPoolConfig) ? agentPoolConfig : empty(agentPoolType) ? {} : nodePoolPresets[agentPoolType] 118 | 119 | // Create additional user agent pool when specified 120 | module agentPool 'aks-agent-pool.bicep' = if (hasAgentPool) { 121 | name: 'aks-node-pool' 122 | params: { 123 | clusterName: managedCluster.outputs.clusterName 124 | name: 'npuserpool' 125 | config: union({ name: 'npuser', mode: 'User' }, nodePoolBase, agentPoolSpec) 126 | } 127 | } 128 | 129 | // Creates container registry (ACR) 130 | module containerRegistry 'container-registry.bicep' = { 131 | name: 'container-registry' 132 | params: { 133 | name: containerRegistryName 134 | location: location 135 | tags: tags 136 | workspaceId: !empty(logAnalyticsName) ? logAnalytics.id : '' 137 | } 138 | } 139 | 140 | // Grant ACR Pull access from cluster managed identity to container registry 141 | module containerRegistryAccess '../security/registry-access.bicep' = { 142 | name: 'cluster-container-registry-access' 143 | params: { 144 | containerRegistryName: containerRegistry.outputs.name 145 | principalId: managedCluster.outputs.clusterIdentity.objectId 146 | } 147 | } 148 | 149 | // Give the AKS Cluster access to KeyVault 150 | module clusterKeyVaultAccess '../security/keyvault-access.bicep' = { 151 | name: 'cluster-keyvault-access' 152 | params: { 153 | keyVaultName: keyVaultName 154 | principalId: managedCluster.outputs.clusterIdentity.objectId 155 | } 156 | } 157 | 158 | // Helpers for node pool configuration 159 | var nodePoolBase = { 160 | osType: 'Linux' 161 | maxPods: 30 162 | type: 'VirtualMachineScaleSets' 163 | upgradeSettings: { 164 | maxSurge: '33%' 165 | } 166 | } 167 | 168 | var nodePoolPresets = { 169 | CostOptimised: { 170 | vmSize: 'Standard_B4ms' 171 | count: 1 172 | minCount: 1 173 | maxCount: 3 174 | enableAutoScaling: true 175 | availabilityZones: [] 176 | } 177 | Standard: { 178 | vmSize: 'Standard_DS2_v2' 179 | count: 3 180 | minCount: 3 181 | maxCount: 5 182 | enableAutoScaling: true 183 | availabilityZones: [ 184 | '1' 185 | '2' 186 | '3' 187 | ] 188 | } 189 | HighSpec: { 190 | vmSize: 'Standard_D4s_v3' 191 | count: 3 192 | minCount: 3 193 | maxCount: 5 194 | enableAutoScaling: true 195 | availabilityZones: [ 196 | '1' 197 | '2' 198 | '3' 199 | ] 200 | } 201 | } 202 | 203 | // Module outputs 204 | @description('The resource name of the AKS cluster') 205 | output clusterName string = managedCluster.outputs.clusterName 206 | 207 | @description('The AKS cluster identity') 208 | output clusterIdentity object = managedCluster.outputs.clusterIdentity 209 | 210 | @description('The resource name of the ACR') 211 | output containerRegistryName string = containerRegistry.outputs.name 212 | 213 | @description('The login server for the container registry') 214 | output containerRegistryLoginServer string = containerRegistry.outputs.loginServer 215 | -------------------------------------------------------------------------------- /infra/core/host/appservice-appsettings.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Updates app settings for an Azure App Service.' 2 | @description('The name of the app service resource within the current resource group scope') 3 | param name string 4 | 5 | @description('The app settings to be applied to the app service') 6 | @secure() 7 | param appSettings object 8 | 9 | resource appService 'Microsoft.Web/sites@2022-03-01' existing = { 10 | name: name 11 | } 12 | 13 | resource settings 'Microsoft.Web/sites/config@2022-03-01' = { 14 | name: 'appsettings' 15 | parent: appService 16 | properties: appSettings 17 | } 18 | -------------------------------------------------------------------------------- /infra/core/host/appservice.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure App Service in an existing Azure App Service plan.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | // Reference Properties 7 | param applicationInsightsName string = '' 8 | param appServicePlanId string 9 | param keyVaultName string = '' 10 | param managedIdentity bool = !empty(keyVaultName) 11 | 12 | // Runtime Properties 13 | @allowed([ 14 | 'dotnet', 'dotnetcore', 'dotnet-isolated', 'node', 'python', 'java', 'powershell', 'custom' 15 | ]) 16 | param runtimeName string 17 | param runtimeNameAndVersion string = '${runtimeName}|${runtimeVersion}' 18 | param runtimeVersion string 19 | 20 | // Microsoft.Web/sites Properties 21 | param kind string = 'app,linux' 22 | 23 | // Microsoft.Web/sites/config 24 | param allowedOrigins array = [] 25 | param alwaysOn bool = true 26 | param appCommandLine string = '' 27 | @secure() 28 | param appSettings object = {} 29 | param clientAffinityEnabled bool = false 30 | param enableOryxBuild bool = contains(kind, 'linux') 31 | param functionAppScaleLimit int = -1 32 | param linuxFxVersion string = runtimeNameAndVersion 33 | param minimumElasticInstanceCount int = -1 34 | param numberOfWorkers int = -1 35 | param scmDoBuildDuringDeployment bool = false 36 | param use32BitWorkerProcess bool = false 37 | param ftpsState string = 'FtpsOnly' 38 | param healthCheckPath string = '' 39 | 40 | resource appService 'Microsoft.Web/sites@2022-03-01' = { 41 | name: name 42 | location: location 43 | tags: tags 44 | kind: kind 45 | properties: { 46 | serverFarmId: appServicePlanId 47 | siteConfig: { 48 | linuxFxVersion: linuxFxVersion 49 | alwaysOn: alwaysOn 50 | ftpsState: ftpsState 51 | minTlsVersion: '1.2' 52 | appCommandLine: appCommandLine 53 | numberOfWorkers: numberOfWorkers != -1 ? numberOfWorkers : null 54 | minimumElasticInstanceCount: minimumElasticInstanceCount != -1 ? minimumElasticInstanceCount : null 55 | use32BitWorkerProcess: use32BitWorkerProcess 56 | functionAppScaleLimit: functionAppScaleLimit != -1 ? functionAppScaleLimit : null 57 | healthCheckPath: healthCheckPath 58 | cors: { 59 | allowedOrigins: union([ 'https://portal.azure.com', 'https://ms.portal.azure.com' ], allowedOrigins) 60 | } 61 | } 62 | clientAffinityEnabled: clientAffinityEnabled 63 | httpsOnly: true 64 | } 65 | 66 | identity: { type: managedIdentity ? 'SystemAssigned' : 'None' } 67 | 68 | resource basicPublishingCredentialsPoliciesFtp 'basicPublishingCredentialsPolicies' = { 69 | name: 'ftp' 70 | properties: { 71 | allow: false 72 | } 73 | } 74 | 75 | resource basicPublishingCredentialsPoliciesScm 'basicPublishingCredentialsPolicies' = { 76 | name: 'scm' 77 | properties: { 78 | allow: false 79 | } 80 | } 81 | } 82 | 83 | // Updates to the single Microsoft.sites/web/config resources that need to be performed sequentially 84 | // sites/web/config 'appsettings' 85 | module configAppSettings 'appservice-appsettings.bicep' = { 86 | name: '${name}-appSettings' 87 | params: { 88 | name: appService.name 89 | appSettings: union(appSettings, 90 | { 91 | SCM_DO_BUILD_DURING_DEPLOYMENT: string(scmDoBuildDuringDeployment) 92 | ENABLE_ORYX_BUILD: string(enableOryxBuild) 93 | }, 94 | runtimeName == 'python' && appCommandLine == '' ? { PYTHON_ENABLE_GUNICORN_MULTIWORKERS: 'true'} : {}, 95 | !empty(applicationInsightsName) ? { APPLICATIONINSIGHTS_CONNECTION_STRING: applicationInsights.properties.ConnectionString } : {}, 96 | !empty(keyVaultName) ? { AZURE_KEY_VAULT_ENDPOINT: keyVault.properties.vaultUri } : {}) 97 | } 98 | } 99 | 100 | // sites/web/config 'logs' 101 | resource configLogs 'Microsoft.Web/sites/config@2022-03-01' = { 102 | name: 'logs' 103 | parent: appService 104 | properties: { 105 | applicationLogs: { fileSystem: { level: 'Verbose' } } 106 | detailedErrorMessages: { enabled: true } 107 | failedRequestsTracing: { enabled: true } 108 | httpLogs: { fileSystem: { enabled: true, retentionInDays: 1, retentionInMb: 35 } } 109 | } 110 | dependsOn: [configAppSettings] 111 | } 112 | 113 | resource keyVault 'Microsoft.KeyVault/vaults@2022-07-01' existing = if (!(empty(keyVaultName))) { 114 | name: keyVaultName 115 | } 116 | 117 | resource applicationInsights 'Microsoft.Insights/components@2020-02-02' existing = if (!empty(applicationInsightsName)) { 118 | name: applicationInsightsName 119 | } 120 | 121 | output identityPrincipalId string = managedIdentity ? appService.identity.principalId : '' 122 | output name string = appService.name 123 | output uri string = 'https://${appService.properties.defaultHostName}' 124 | -------------------------------------------------------------------------------- /infra/core/host/appserviceplan.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure App Service plan.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | param kind string = '' 7 | param reserved bool = true 8 | param sku object 9 | 10 | resource appServicePlan 'Microsoft.Web/serverfarms@2022-03-01' = { 11 | name: name 12 | location: location 13 | tags: tags 14 | sku: sku 15 | kind: kind 16 | properties: { 17 | reserved: reserved 18 | } 19 | } 20 | 21 | output id string = appServicePlan.id 22 | output name string = appServicePlan.name 23 | -------------------------------------------------------------------------------- /infra/core/host/container-app-upsert.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates or updates an existing Azure Container App.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | @description('The environment name for the container apps') 7 | param containerAppsEnvironmentName string 8 | 9 | @description('The number of CPU cores allocated to a single container instance, e.g., 0.5') 10 | param containerCpuCoreCount string = '0.5' 11 | 12 | @description('The maximum number of replicas to run. Must be at least 1.') 13 | @minValue(1) 14 | param containerMaxReplicas int = 10 15 | 16 | @description('The amount of memory allocated to a single container instance, e.g., 1Gi') 17 | param containerMemory string = '1.0Gi' 18 | 19 | @description('The minimum number of replicas to run. Must be at least 1.') 20 | @minValue(1) 21 | param containerMinReplicas int = 1 22 | 23 | @description('The name of the container') 24 | param containerName string = 'main' 25 | 26 | @description('The name of the container registry') 27 | param containerRegistryName string = '' 28 | 29 | @allowed([ 'http', 'grpc' ]) 30 | @description('The protocol used by Dapr to connect to the app, e.g., HTTP or gRPC') 31 | param daprAppProtocol string = 'http' 32 | 33 | @description('Enable or disable Dapr for the container app') 34 | param daprEnabled bool = false 35 | 36 | @description('The Dapr app ID') 37 | param daprAppId string = containerName 38 | 39 | @description('Specifies if the resource already exists') 40 | param exists bool = false 41 | 42 | @description('Specifies if Ingress is enabled for the container app') 43 | param ingressEnabled bool = true 44 | 45 | @description('The type of identity for the resource') 46 | @allowed([ 'None', 'SystemAssigned', 'UserAssigned' ]) 47 | param identityType string = 'None' 48 | 49 | @description('The name of the user-assigned identity') 50 | param identityName string = '' 51 | 52 | @description('The name of the container image') 53 | param imageName string = '' 54 | 55 | @description('The secrets required for the container') 56 | param secrets array = [] 57 | 58 | @description('The environment variables for the container') 59 | param env array = [] 60 | 61 | @description('Specifies if the resource ingress is exposed externally') 62 | param external bool = true 63 | 64 | @description('The service binds associated with the container') 65 | param serviceBinds array = [] 66 | 67 | @description('The target port for the container') 68 | param targetPort int = 80 69 | 70 | resource existingApp 'Microsoft.App/containerApps@2023-04-01-preview' existing = if (exists) { 71 | name: name 72 | } 73 | 74 | module app 'container-app.bicep' = { 75 | name: '${deployment().name}-update' 76 | params: { 77 | name: name 78 | location: location 79 | tags: tags 80 | identityType: identityType 81 | identityName: identityName 82 | ingressEnabled: ingressEnabled 83 | containerName: containerName 84 | containerAppsEnvironmentName: containerAppsEnvironmentName 85 | containerRegistryName: containerRegistryName 86 | containerCpuCoreCount: containerCpuCoreCount 87 | containerMemory: containerMemory 88 | containerMinReplicas: containerMinReplicas 89 | containerMaxReplicas: containerMaxReplicas 90 | daprEnabled: daprEnabled 91 | daprAppId: daprAppId 92 | daprAppProtocol: daprAppProtocol 93 | secrets: secrets 94 | external: external 95 | env: env 96 | imageName: !empty(imageName) ? imageName : exists ? existingApp.properties.template.containers[0].image : '' 97 | targetPort: targetPort 98 | serviceBinds: serviceBinds 99 | } 100 | } 101 | 102 | output defaultDomain string = app.outputs.defaultDomain 103 | output imageName string = app.outputs.imageName 104 | output name string = app.outputs.name 105 | output uri string = app.outputs.uri 106 | -------------------------------------------------------------------------------- /infra/core/host/container-app.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates a container app in an Azure Container App environment.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | @description('Allowed origins') 7 | param allowedOrigins array = [] 8 | 9 | @description('Name of the environment for container apps') 10 | param containerAppsEnvironmentName string 11 | 12 | @description('CPU cores allocated to a single container instance, e.g., 0.5') 13 | param containerCpuCoreCount string = '0.5' 14 | 15 | @description('The maximum number of replicas to run. Must be at least 1.') 16 | @minValue(1) 17 | param containerMaxReplicas int = 10 18 | 19 | @description('Memory allocated to a single container instance, e.g., 1Gi') 20 | param containerMemory string = '1.0Gi' 21 | 22 | @description('The minimum number of replicas to run. Must be at least 1.') 23 | param containerMinReplicas int = 1 24 | 25 | @description('The name of the container') 26 | param containerName string = 'main' 27 | 28 | @description('The name of the container registry') 29 | param containerRegistryName string = '' 30 | 31 | @description('The protocol used by Dapr to connect to the app, e.g., http or grpc') 32 | @allowed([ 'http', 'grpc' ]) 33 | param daprAppProtocol string = 'http' 34 | 35 | @description('The Dapr app ID') 36 | param daprAppId string = containerName 37 | 38 | @description('Enable Dapr') 39 | param daprEnabled bool = false 40 | 41 | @description('The environment variables for the container') 42 | param env array = [] 43 | 44 | @description('Specifies if the resource ingress is exposed externally') 45 | param external bool = true 46 | 47 | @description('The name of the user-assigned identity') 48 | param identityName string = '' 49 | 50 | @description('The type of identity for the resource') 51 | @allowed([ 'None', 'SystemAssigned', 'UserAssigned' ]) 52 | param identityType string = 'None' 53 | 54 | @description('The name of the container image') 55 | param imageName string = '' 56 | 57 | @description('Specifies if Ingress is enabled for the container app') 58 | param ingressEnabled bool = true 59 | 60 | param revisionMode string = 'Single' 61 | 62 | @description('The secrets required for the container') 63 | param secrets array = [] 64 | 65 | @description('The service binds associated with the container') 66 | param serviceBinds array = [] 67 | 68 | @description('The name of the container apps add-on to use. e.g. redis') 69 | param serviceType string = '' 70 | 71 | @description('The target port for the container') 72 | param targetPort int = 80 73 | 74 | resource userIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' existing = if (!empty(identityName)) { 75 | name: identityName 76 | } 77 | 78 | // Private registry support requires both an ACR name and a User Assigned managed identity 79 | var usePrivateRegistry = !empty(identityName) && !empty(containerRegistryName) 80 | 81 | // Automatically set to `UserAssigned` when an `identityName` has been set 82 | var normalizedIdentityType = !empty(identityName) ? 'UserAssigned' : identityType 83 | 84 | module containerRegistryAccess '../security/registry-access.bicep' = if (usePrivateRegistry) { 85 | name: '${deployment().name}-registry-access' 86 | params: { 87 | containerRegistryName: containerRegistryName 88 | principalId: usePrivateRegistry ? userIdentity.properties.principalId : '' 89 | } 90 | } 91 | 92 | resource app 'Microsoft.App/containerApps@2023-04-01-preview' = { 93 | name: name 94 | location: location 95 | tags: tags 96 | // It is critical that the identity is granted ACR pull access before the app is created 97 | // otherwise the container app will throw a provision error 98 | // This also forces us to use an user assigned managed identity since there would no way to 99 | // provide the system assigned identity with the ACR pull access before the app is created 100 | dependsOn: usePrivateRegistry ? [ containerRegistryAccess ] : [] 101 | identity: { 102 | type: normalizedIdentityType 103 | userAssignedIdentities: !empty(identityName) && normalizedIdentityType == 'UserAssigned' ? { '${userIdentity.id}': {} } : null 104 | } 105 | properties: { 106 | managedEnvironmentId: containerAppsEnvironment.id 107 | configuration: { 108 | activeRevisionsMode: revisionMode 109 | ingress: ingressEnabled ? { 110 | external: external 111 | targetPort: targetPort 112 | transport: 'auto' 113 | corsPolicy: { 114 | allowedOrigins: union([ 'https://portal.azure.com', 'https://ms.portal.azure.com' ], allowedOrigins) 115 | } 116 | } : null 117 | dapr: daprEnabled ? { 118 | enabled: true 119 | appId: daprAppId 120 | appProtocol: daprAppProtocol 121 | appPort: ingressEnabled ? targetPort : 0 122 | } : { enabled: false } 123 | secrets: secrets 124 | service: !empty(serviceType) ? { type: serviceType } : null 125 | registries: usePrivateRegistry ? [ 126 | { 127 | server: '${containerRegistryName}.azurecr.io' 128 | identity: userIdentity.id 129 | } 130 | ] : [] 131 | } 132 | template: { 133 | serviceBinds: !empty(serviceBinds) ? serviceBinds : null 134 | containers: [ 135 | { 136 | image: !empty(imageName) ? imageName : 'mcr.microsoft.com/azuredocs/containerapps-helloworld:latest' 137 | name: containerName 138 | env: env 139 | resources: { 140 | cpu: json(containerCpuCoreCount) 141 | memory: containerMemory 142 | } 143 | } 144 | ] 145 | scale: { 146 | minReplicas: containerMinReplicas 147 | maxReplicas: containerMaxReplicas 148 | } 149 | } 150 | } 151 | } 152 | 153 | resource containerAppsEnvironment 'Microsoft.App/managedEnvironments@2023-04-01-preview' existing = { 154 | name: containerAppsEnvironmentName 155 | } 156 | 157 | output defaultDomain string = containerAppsEnvironment.properties.defaultDomain 158 | output identityPrincipalId string = normalizedIdentityType == 'None' ? '' : (empty(identityName) ? app.identity.principalId : userIdentity.properties.principalId) 159 | output imageName string = imageName 160 | output name string = app.name 161 | output serviceBind object = !empty(serviceType) ? { serviceId: app.id, name: name } : {} 162 | output uri string = ingressEnabled ? 'https://${app.properties.configuration.ingress.fqdn}' : '' 163 | -------------------------------------------------------------------------------- /infra/core/host/container-apps-environment.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure Container Apps environment.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | @description('Name of the Application Insights resource') 7 | param applicationInsightsName string = '' 8 | 9 | @description('Specifies if Dapr is enabled') 10 | param daprEnabled bool = false 11 | 12 | @description('Name of the Log Analytics workspace') 13 | param logAnalyticsWorkspaceName string 14 | 15 | resource containerAppsEnvironment 'Microsoft.App/managedEnvironments@2023-04-01-preview' = { 16 | name: name 17 | location: location 18 | tags: tags 19 | properties: { 20 | appLogsConfiguration: { 21 | destination: 'log-analytics' 22 | logAnalyticsConfiguration: { 23 | customerId: logAnalyticsWorkspace.properties.customerId 24 | sharedKey: logAnalyticsWorkspace.listKeys().primarySharedKey 25 | } 26 | } 27 | daprAIInstrumentationKey: daprEnabled && !empty(applicationInsightsName) ? applicationInsights.properties.InstrumentationKey : '' 28 | } 29 | } 30 | 31 | resource logAnalyticsWorkspace 'Microsoft.OperationalInsights/workspaces@2022-10-01' existing = { 32 | name: logAnalyticsWorkspaceName 33 | } 34 | 35 | resource applicationInsights 'Microsoft.Insights/components@2020-02-02' existing = if (daprEnabled && !empty(applicationInsightsName)) { 36 | name: applicationInsightsName 37 | } 38 | 39 | output defaultDomain string = containerAppsEnvironment.properties.defaultDomain 40 | output id string = containerAppsEnvironment.id 41 | output name string = containerAppsEnvironment.name 42 | -------------------------------------------------------------------------------- /infra/core/host/container-apps.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure Container Registry and an Azure Container Apps environment.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | param containerAppsEnvironmentName string 7 | param containerRegistryName string 8 | param containerRegistryResourceGroupName string = '' 9 | param logAnalyticsWorkspaceName string 10 | param applicationInsightsName string = '' 11 | 12 | module containerAppsEnvironment 'container-apps-environment.bicep' = { 13 | name: '${name}-container-apps-environment' 14 | params: { 15 | name: containerAppsEnvironmentName 16 | location: location 17 | tags: tags 18 | logAnalyticsWorkspaceName: logAnalyticsWorkspaceName 19 | applicationInsightsName: applicationInsightsName 20 | } 21 | } 22 | 23 | module containerRegistry 'container-registry.bicep' = { 24 | name: '${name}-container-registry' 25 | scope: !empty(containerRegistryResourceGroupName) ? resourceGroup(containerRegistryResourceGroupName) : resourceGroup() 26 | params: { 27 | name: containerRegistryName 28 | location: location 29 | tags: tags 30 | } 31 | } 32 | 33 | output defaultDomain string = containerAppsEnvironment.outputs.defaultDomain 34 | output environmentName string = containerAppsEnvironment.outputs.name 35 | output environmentId string = containerAppsEnvironment.outputs.id 36 | 37 | output registryLoginServer string = containerRegistry.outputs.loginServer 38 | output registryName string = containerRegistry.outputs.name 39 | -------------------------------------------------------------------------------- /infra/core/host/container-registry.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure Container Registry.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | @description('Indicates whether admin user is enabled') 7 | param adminUserEnabled bool = false 8 | 9 | @description('Indicates whether anonymous pull is enabled') 10 | param anonymousPullEnabled bool = false 11 | 12 | @description('Indicates whether data endpoint is enabled') 13 | param dataEndpointEnabled bool = false 14 | 15 | @description('Encryption settings') 16 | param encryption object = { 17 | status: 'disabled' 18 | } 19 | 20 | @description('Options for bypassing network rules') 21 | param networkRuleBypassOptions string = 'AzureServices' 22 | 23 | @description('Public network access setting') 24 | param publicNetworkAccess string = 'Enabled' 25 | 26 | @description('SKU settings') 27 | param sku object = { 28 | name: 'Basic' 29 | } 30 | 31 | @description('Zone redundancy setting') 32 | param zoneRedundancy string = 'Disabled' 33 | 34 | @description('The log analytics workspace ID used for logging and monitoring') 35 | param workspaceId string = '' 36 | 37 | // 2022-02-01-preview needed for anonymousPullEnabled 38 | resource containerRegistry 'Microsoft.ContainerRegistry/registries@2022-02-01-preview' = { 39 | name: name 40 | location: location 41 | tags: tags 42 | sku: sku 43 | properties: { 44 | adminUserEnabled: adminUserEnabled 45 | anonymousPullEnabled: anonymousPullEnabled 46 | dataEndpointEnabled: dataEndpointEnabled 47 | encryption: encryption 48 | networkRuleBypassOptions: networkRuleBypassOptions 49 | publicNetworkAccess: publicNetworkAccess 50 | zoneRedundancy: zoneRedundancy 51 | } 52 | } 53 | 54 | // TODO: Update diagnostics to be its own module 55 | // Blocking issue: https://github.com/Azure/bicep/issues/622 56 | // Unable to pass in a `resource` scope or unable to use string interpolation in resource types 57 | resource diagnostics 'Microsoft.Insights/diagnosticSettings@2021-05-01-preview' = if (!empty(workspaceId)) { 58 | name: 'registry-diagnostics' 59 | scope: containerRegistry 60 | properties: { 61 | workspaceId: workspaceId 62 | logs: [ 63 | { 64 | category: 'ContainerRegistryRepositoryEvents' 65 | enabled: true 66 | } 67 | { 68 | category: 'ContainerRegistryLoginEvents' 69 | enabled: true 70 | } 71 | ] 72 | metrics: [ 73 | { 74 | category: 'AllMetrics' 75 | enabled: true 76 | timeGrain: 'PT1M' 77 | } 78 | ] 79 | } 80 | } 81 | 82 | output loginServer string = containerRegistry.properties.loginServer 83 | output name string = containerRegistry.name 84 | -------------------------------------------------------------------------------- /infra/core/host/functions.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure Function in an existing Azure App Service plan.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | // Reference Properties 7 | param applicationInsightsName string = '' 8 | param appServicePlanId string 9 | param keyVaultName string = '' 10 | param managedIdentity bool = !empty(keyVaultName) 11 | param storageAccountName string 12 | 13 | // Runtime Properties 14 | @allowed([ 15 | 'dotnet', 'dotnetcore', 'dotnet-isolated', 'node', 'python', 'java', 'powershell', 'custom' 16 | ]) 17 | param runtimeName string 18 | param runtimeNameAndVersion string = '${runtimeName}|${runtimeVersion}' 19 | param runtimeVersion string 20 | 21 | // Function Settings 22 | @allowed([ 23 | '~4', '~3', '~2', '~1' 24 | ]) 25 | param extensionVersion string = '~4' 26 | 27 | // Microsoft.Web/sites Properties 28 | param kind string = 'functionapp,linux' 29 | 30 | // Microsoft.Web/sites/config 31 | param allowedOrigins array = [] 32 | param alwaysOn bool = true 33 | param appCommandLine string = '' 34 | @secure() 35 | param appSettings object = {} 36 | param clientAffinityEnabled bool = false 37 | param enableOryxBuild bool = contains(kind, 'linux') 38 | param functionAppScaleLimit int = -1 39 | param linuxFxVersion string = runtimeNameAndVersion 40 | param minimumElasticInstanceCount int = -1 41 | param numberOfWorkers int = -1 42 | param scmDoBuildDuringDeployment bool = true 43 | param use32BitWorkerProcess bool = false 44 | param healthCheckPath string = '' 45 | 46 | module functions 'appservice.bicep' = { 47 | name: '${name}-functions' 48 | params: { 49 | name: name 50 | location: location 51 | tags: tags 52 | allowedOrigins: allowedOrigins 53 | alwaysOn: alwaysOn 54 | appCommandLine: appCommandLine 55 | applicationInsightsName: applicationInsightsName 56 | appServicePlanId: appServicePlanId 57 | appSettings: union(appSettings, { 58 | AzureWebJobsStorage: 'DefaultEndpointsProtocol=https;AccountName=${storage.name};AccountKey=${storage.listKeys().keys[0].value};EndpointSuffix=${environment().suffixes.storage}' 59 | FUNCTIONS_EXTENSION_VERSION: extensionVersion 60 | FUNCTIONS_WORKER_RUNTIME: runtimeName 61 | }) 62 | clientAffinityEnabled: clientAffinityEnabled 63 | enableOryxBuild: enableOryxBuild 64 | functionAppScaleLimit: functionAppScaleLimit 65 | healthCheckPath: healthCheckPath 66 | keyVaultName: keyVaultName 67 | kind: kind 68 | linuxFxVersion: linuxFxVersion 69 | managedIdentity: managedIdentity 70 | minimumElasticInstanceCount: minimumElasticInstanceCount 71 | numberOfWorkers: numberOfWorkers 72 | runtimeName: runtimeName 73 | runtimeVersion: runtimeVersion 74 | runtimeNameAndVersion: runtimeNameAndVersion 75 | scmDoBuildDuringDeployment: scmDoBuildDuringDeployment 76 | use32BitWorkerProcess: use32BitWorkerProcess 77 | } 78 | } 79 | 80 | resource storage 'Microsoft.Storage/storageAccounts@2021-09-01' existing = { 81 | name: storageAccountName 82 | } 83 | 84 | output identityPrincipalId string = managedIdentity ? functions.outputs.identityPrincipalId : '' 85 | output name string = functions.outputs.name 86 | output uri string = functions.outputs.uri 87 | -------------------------------------------------------------------------------- /infra/core/host/staticwebapp.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure Static Web Apps instance.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | param sku object = { 7 | name: 'Free' 8 | tier: 'Free' 9 | } 10 | 11 | resource web 'Microsoft.Web/staticSites@2022-03-01' = { 12 | name: name 13 | location: location 14 | tags: tags 15 | sku: sku 16 | properties: { 17 | provider: 'Custom' 18 | } 19 | } 20 | 21 | output name string = web.name 22 | output uri string = 'https://${web.properties.defaultHostname}' 23 | -------------------------------------------------------------------------------- /infra/core/monitor/applicationinsights-dashboard.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates a dashboard for an Application Insights instance.' 2 | param name string 3 | param applicationInsightsName string 4 | param location string = resourceGroup().location 5 | param tags object = {} 6 | 7 | // 2020-09-01-preview because that is the latest valid version 8 | resource applicationInsightsDashboard 'Microsoft.Portal/dashboards@2020-09-01-preview' = { 9 | name: name 10 | location: location 11 | tags: tags 12 | properties: { 13 | lenses: [ 14 | { 15 | order: 0 16 | parts: [ 17 | { 18 | position: { 19 | x: 0 20 | y: 0 21 | colSpan: 2 22 | rowSpan: 1 23 | } 24 | metadata: { 25 | inputs: [ 26 | { 27 | name: 'id' 28 | value: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 29 | } 30 | { 31 | name: 'Version' 32 | value: '1.0' 33 | } 34 | ] 35 | #disable-next-line BCP036 36 | type: 'Extension/AppInsightsExtension/PartType/AspNetOverviewPinnedPart' 37 | asset: { 38 | idInputName: 'id' 39 | type: 'ApplicationInsights' 40 | } 41 | defaultMenuItemId: 'overview' 42 | } 43 | } 44 | { 45 | position: { 46 | x: 2 47 | y: 0 48 | colSpan: 1 49 | rowSpan: 1 50 | } 51 | metadata: { 52 | inputs: [ 53 | { 54 | name: 'ComponentId' 55 | value: { 56 | Name: applicationInsights.name 57 | SubscriptionId: subscription().subscriptionId 58 | ResourceGroup: resourceGroup().name 59 | } 60 | } 61 | { 62 | name: 'Version' 63 | value: '1.0' 64 | } 65 | ] 66 | #disable-next-line BCP036 67 | type: 'Extension/AppInsightsExtension/PartType/ProactiveDetectionAsyncPart' 68 | asset: { 69 | idInputName: 'ComponentId' 70 | type: 'ApplicationInsights' 71 | } 72 | defaultMenuItemId: 'ProactiveDetection' 73 | } 74 | } 75 | { 76 | position: { 77 | x: 3 78 | y: 0 79 | colSpan: 1 80 | rowSpan: 1 81 | } 82 | metadata: { 83 | inputs: [ 84 | { 85 | name: 'ComponentId' 86 | value: { 87 | Name: applicationInsights.name 88 | SubscriptionId: subscription().subscriptionId 89 | ResourceGroup: resourceGroup().name 90 | } 91 | } 92 | { 93 | name: 'ResourceId' 94 | value: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 95 | } 96 | ] 97 | #disable-next-line BCP036 98 | type: 'Extension/AppInsightsExtension/PartType/QuickPulseButtonSmallPart' 99 | asset: { 100 | idInputName: 'ComponentId' 101 | type: 'ApplicationInsights' 102 | } 103 | } 104 | } 105 | { 106 | position: { 107 | x: 4 108 | y: 0 109 | colSpan: 1 110 | rowSpan: 1 111 | } 112 | metadata: { 113 | inputs: [ 114 | { 115 | name: 'ComponentId' 116 | value: { 117 | Name: applicationInsights.name 118 | SubscriptionId: subscription().subscriptionId 119 | ResourceGroup: resourceGroup().name 120 | } 121 | } 122 | { 123 | name: 'TimeContext' 124 | value: { 125 | durationMs: 86400000 126 | endTime: null 127 | createdTime: '2018-05-04T01:20:33.345Z' 128 | isInitialTime: true 129 | grain: 1 130 | useDashboardTimeRange: false 131 | } 132 | } 133 | { 134 | name: 'Version' 135 | value: '1.0' 136 | } 137 | ] 138 | #disable-next-line BCP036 139 | type: 'Extension/AppInsightsExtension/PartType/AvailabilityNavButtonPart' 140 | asset: { 141 | idInputName: 'ComponentId' 142 | type: 'ApplicationInsights' 143 | } 144 | } 145 | } 146 | { 147 | position: { 148 | x: 5 149 | y: 0 150 | colSpan: 1 151 | rowSpan: 1 152 | } 153 | metadata: { 154 | inputs: [ 155 | { 156 | name: 'ComponentId' 157 | value: { 158 | Name: applicationInsights.name 159 | SubscriptionId: subscription().subscriptionId 160 | ResourceGroup: resourceGroup().name 161 | } 162 | } 163 | { 164 | name: 'TimeContext' 165 | value: { 166 | durationMs: 86400000 167 | endTime: null 168 | createdTime: '2018-05-08T18:47:35.237Z' 169 | isInitialTime: true 170 | grain: 1 171 | useDashboardTimeRange: false 172 | } 173 | } 174 | { 175 | name: 'ConfigurationId' 176 | value: '78ce933e-e864-4b05-a27b-71fd55a6afad' 177 | } 178 | ] 179 | #disable-next-line BCP036 180 | type: 'Extension/AppInsightsExtension/PartType/AppMapButtonPart' 181 | asset: { 182 | idInputName: 'ComponentId' 183 | type: 'ApplicationInsights' 184 | } 185 | } 186 | } 187 | { 188 | position: { 189 | x: 0 190 | y: 1 191 | colSpan: 3 192 | rowSpan: 1 193 | } 194 | metadata: { 195 | inputs: [] 196 | type: 'Extension/HubsExtension/PartType/MarkdownPart' 197 | settings: { 198 | content: { 199 | settings: { 200 | content: '# Usage' 201 | title: '' 202 | subtitle: '' 203 | } 204 | } 205 | } 206 | } 207 | } 208 | { 209 | position: { 210 | x: 3 211 | y: 1 212 | colSpan: 1 213 | rowSpan: 1 214 | } 215 | metadata: { 216 | inputs: [ 217 | { 218 | name: 'ComponentId' 219 | value: { 220 | Name: applicationInsights.name 221 | SubscriptionId: subscription().subscriptionId 222 | ResourceGroup: resourceGroup().name 223 | } 224 | } 225 | { 226 | name: 'TimeContext' 227 | value: { 228 | durationMs: 86400000 229 | endTime: null 230 | createdTime: '2018-05-04T01:22:35.782Z' 231 | isInitialTime: true 232 | grain: 1 233 | useDashboardTimeRange: false 234 | } 235 | } 236 | ] 237 | #disable-next-line BCP036 238 | type: 'Extension/AppInsightsExtension/PartType/UsageUsersOverviewPart' 239 | asset: { 240 | idInputName: 'ComponentId' 241 | type: 'ApplicationInsights' 242 | } 243 | } 244 | } 245 | { 246 | position: { 247 | x: 4 248 | y: 1 249 | colSpan: 3 250 | rowSpan: 1 251 | } 252 | metadata: { 253 | inputs: [] 254 | type: 'Extension/HubsExtension/PartType/MarkdownPart' 255 | settings: { 256 | content: { 257 | settings: { 258 | content: '# Reliability' 259 | title: '' 260 | subtitle: '' 261 | } 262 | } 263 | } 264 | } 265 | } 266 | { 267 | position: { 268 | x: 7 269 | y: 1 270 | colSpan: 1 271 | rowSpan: 1 272 | } 273 | metadata: { 274 | inputs: [ 275 | { 276 | name: 'ResourceId' 277 | value: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 278 | } 279 | { 280 | name: 'DataModel' 281 | value: { 282 | version: '1.0.0' 283 | timeContext: { 284 | durationMs: 86400000 285 | createdTime: '2018-05-04T23:42:40.072Z' 286 | isInitialTime: false 287 | grain: 1 288 | useDashboardTimeRange: false 289 | } 290 | } 291 | isOptional: true 292 | } 293 | { 294 | name: 'ConfigurationId' 295 | value: '8a02f7bf-ac0f-40e1-afe9-f0e72cfee77f' 296 | isOptional: true 297 | } 298 | ] 299 | #disable-next-line BCP036 300 | type: 'Extension/AppInsightsExtension/PartType/CuratedBladeFailuresPinnedPart' 301 | isAdapter: true 302 | asset: { 303 | idInputName: 'ResourceId' 304 | type: 'ApplicationInsights' 305 | } 306 | defaultMenuItemId: 'failures' 307 | } 308 | } 309 | { 310 | position: { 311 | x: 8 312 | y: 1 313 | colSpan: 3 314 | rowSpan: 1 315 | } 316 | metadata: { 317 | inputs: [] 318 | type: 'Extension/HubsExtension/PartType/MarkdownPart' 319 | settings: { 320 | content: { 321 | settings: { 322 | content: '# Responsiveness\r\n' 323 | title: '' 324 | subtitle: '' 325 | } 326 | } 327 | } 328 | } 329 | } 330 | { 331 | position: { 332 | x: 11 333 | y: 1 334 | colSpan: 1 335 | rowSpan: 1 336 | } 337 | metadata: { 338 | inputs: [ 339 | { 340 | name: 'ResourceId' 341 | value: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 342 | } 343 | { 344 | name: 'DataModel' 345 | value: { 346 | version: '1.0.0' 347 | timeContext: { 348 | durationMs: 86400000 349 | createdTime: '2018-05-04T23:43:37.804Z' 350 | isInitialTime: false 351 | grain: 1 352 | useDashboardTimeRange: false 353 | } 354 | } 355 | isOptional: true 356 | } 357 | { 358 | name: 'ConfigurationId' 359 | value: '2a8ede4f-2bee-4b9c-aed9-2db0e8a01865' 360 | isOptional: true 361 | } 362 | ] 363 | #disable-next-line BCP036 364 | type: 'Extension/AppInsightsExtension/PartType/CuratedBladePerformancePinnedPart' 365 | isAdapter: true 366 | asset: { 367 | idInputName: 'ResourceId' 368 | type: 'ApplicationInsights' 369 | } 370 | defaultMenuItemId: 'performance' 371 | } 372 | } 373 | { 374 | position: { 375 | x: 12 376 | y: 1 377 | colSpan: 3 378 | rowSpan: 1 379 | } 380 | metadata: { 381 | inputs: [] 382 | type: 'Extension/HubsExtension/PartType/MarkdownPart' 383 | settings: { 384 | content: { 385 | settings: { 386 | content: '# Browser' 387 | title: '' 388 | subtitle: '' 389 | } 390 | } 391 | } 392 | } 393 | } 394 | { 395 | position: { 396 | x: 15 397 | y: 1 398 | colSpan: 1 399 | rowSpan: 1 400 | } 401 | metadata: { 402 | inputs: [ 403 | { 404 | name: 'ComponentId' 405 | value: { 406 | Name: applicationInsights.name 407 | SubscriptionId: subscription().subscriptionId 408 | ResourceGroup: resourceGroup().name 409 | } 410 | } 411 | { 412 | name: 'MetricsExplorerJsonDefinitionId' 413 | value: 'BrowserPerformanceTimelineMetrics' 414 | } 415 | { 416 | name: 'TimeContext' 417 | value: { 418 | durationMs: 86400000 419 | createdTime: '2018-05-08T12:16:27.534Z' 420 | isInitialTime: false 421 | grain: 1 422 | useDashboardTimeRange: false 423 | } 424 | } 425 | { 426 | name: 'CurrentFilter' 427 | value: { 428 | eventTypes: [ 429 | 4 430 | 1 431 | 3 432 | 5 433 | 2 434 | 6 435 | 13 436 | ] 437 | typeFacets: {} 438 | isPermissive: false 439 | } 440 | } 441 | { 442 | name: 'id' 443 | value: { 444 | Name: applicationInsights.name 445 | SubscriptionId: subscription().subscriptionId 446 | ResourceGroup: resourceGroup().name 447 | } 448 | } 449 | { 450 | name: 'Version' 451 | value: '1.0' 452 | } 453 | ] 454 | #disable-next-line BCP036 455 | type: 'Extension/AppInsightsExtension/PartType/MetricsExplorerBladePinnedPart' 456 | asset: { 457 | idInputName: 'ComponentId' 458 | type: 'ApplicationInsights' 459 | } 460 | defaultMenuItemId: 'browser' 461 | } 462 | } 463 | { 464 | position: { 465 | x: 0 466 | y: 2 467 | colSpan: 4 468 | rowSpan: 3 469 | } 470 | metadata: { 471 | inputs: [ 472 | { 473 | name: 'options' 474 | value: { 475 | chart: { 476 | metrics: [ 477 | { 478 | resourceMetadata: { 479 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 480 | } 481 | name: 'sessions/count' 482 | aggregationType: 5 483 | namespace: 'microsoft.insights/components/kusto' 484 | metricVisualization: { 485 | displayName: 'Sessions' 486 | color: '#47BDF5' 487 | } 488 | } 489 | { 490 | resourceMetadata: { 491 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 492 | } 493 | name: 'users/count' 494 | aggregationType: 5 495 | namespace: 'microsoft.insights/components/kusto' 496 | metricVisualization: { 497 | displayName: 'Users' 498 | color: '#7E58FF' 499 | } 500 | } 501 | ] 502 | title: 'Unique sessions and users' 503 | visualization: { 504 | chartType: 2 505 | legendVisualization: { 506 | isVisible: true 507 | position: 2 508 | hideSubtitle: false 509 | } 510 | axisVisualization: { 511 | x: { 512 | isVisible: true 513 | axisType: 2 514 | } 515 | y: { 516 | isVisible: true 517 | axisType: 1 518 | } 519 | } 520 | } 521 | openBladeOnClick: { 522 | openBlade: true 523 | destinationBlade: { 524 | extensionName: 'HubsExtension' 525 | bladeName: 'ResourceMenuBlade' 526 | parameters: { 527 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 528 | menuid: 'segmentationUsers' 529 | } 530 | } 531 | } 532 | } 533 | } 534 | } 535 | { 536 | name: 'sharedTimeRange' 537 | isOptional: true 538 | } 539 | ] 540 | #disable-next-line BCP036 541 | type: 'Extension/HubsExtension/PartType/MonitorChartPart' 542 | settings: {} 543 | } 544 | } 545 | { 546 | position: { 547 | x: 4 548 | y: 2 549 | colSpan: 4 550 | rowSpan: 3 551 | } 552 | metadata: { 553 | inputs: [ 554 | { 555 | name: 'options' 556 | value: { 557 | chart: { 558 | metrics: [ 559 | { 560 | resourceMetadata: { 561 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 562 | } 563 | name: 'requests/failed' 564 | aggregationType: 7 565 | namespace: 'microsoft.insights/components' 566 | metricVisualization: { 567 | displayName: 'Failed requests' 568 | color: '#EC008C' 569 | } 570 | } 571 | ] 572 | title: 'Failed requests' 573 | visualization: { 574 | chartType: 3 575 | legendVisualization: { 576 | isVisible: true 577 | position: 2 578 | hideSubtitle: false 579 | } 580 | axisVisualization: { 581 | x: { 582 | isVisible: true 583 | axisType: 2 584 | } 585 | y: { 586 | isVisible: true 587 | axisType: 1 588 | } 589 | } 590 | } 591 | openBladeOnClick: { 592 | openBlade: true 593 | destinationBlade: { 594 | extensionName: 'HubsExtension' 595 | bladeName: 'ResourceMenuBlade' 596 | parameters: { 597 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 598 | menuid: 'failures' 599 | } 600 | } 601 | } 602 | } 603 | } 604 | } 605 | { 606 | name: 'sharedTimeRange' 607 | isOptional: true 608 | } 609 | ] 610 | #disable-next-line BCP036 611 | type: 'Extension/HubsExtension/PartType/MonitorChartPart' 612 | settings: {} 613 | } 614 | } 615 | { 616 | position: { 617 | x: 8 618 | y: 2 619 | colSpan: 4 620 | rowSpan: 3 621 | } 622 | metadata: { 623 | inputs: [ 624 | { 625 | name: 'options' 626 | value: { 627 | chart: { 628 | metrics: [ 629 | { 630 | resourceMetadata: { 631 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 632 | } 633 | name: 'requests/duration' 634 | aggregationType: 4 635 | namespace: 'microsoft.insights/components' 636 | metricVisualization: { 637 | displayName: 'Server response time' 638 | color: '#00BCF2' 639 | } 640 | } 641 | ] 642 | title: 'Server response time' 643 | visualization: { 644 | chartType: 2 645 | legendVisualization: { 646 | isVisible: true 647 | position: 2 648 | hideSubtitle: false 649 | } 650 | axisVisualization: { 651 | x: { 652 | isVisible: true 653 | axisType: 2 654 | } 655 | y: { 656 | isVisible: true 657 | axisType: 1 658 | } 659 | } 660 | } 661 | openBladeOnClick: { 662 | openBlade: true 663 | destinationBlade: { 664 | extensionName: 'HubsExtension' 665 | bladeName: 'ResourceMenuBlade' 666 | parameters: { 667 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 668 | menuid: 'performance' 669 | } 670 | } 671 | } 672 | } 673 | } 674 | } 675 | { 676 | name: 'sharedTimeRange' 677 | isOptional: true 678 | } 679 | ] 680 | #disable-next-line BCP036 681 | type: 'Extension/HubsExtension/PartType/MonitorChartPart' 682 | settings: {} 683 | } 684 | } 685 | { 686 | position: { 687 | x: 12 688 | y: 2 689 | colSpan: 4 690 | rowSpan: 3 691 | } 692 | metadata: { 693 | inputs: [ 694 | { 695 | name: 'options' 696 | value: { 697 | chart: { 698 | metrics: [ 699 | { 700 | resourceMetadata: { 701 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 702 | } 703 | name: 'browserTimings/networkDuration' 704 | aggregationType: 4 705 | namespace: 'microsoft.insights/components' 706 | metricVisualization: { 707 | displayName: 'Page load network connect time' 708 | color: '#7E58FF' 709 | } 710 | } 711 | { 712 | resourceMetadata: { 713 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 714 | } 715 | name: 'browserTimings/processingDuration' 716 | aggregationType: 4 717 | namespace: 'microsoft.insights/components' 718 | metricVisualization: { 719 | displayName: 'Client processing time' 720 | color: '#44F1C8' 721 | } 722 | } 723 | { 724 | resourceMetadata: { 725 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 726 | } 727 | name: 'browserTimings/sendDuration' 728 | aggregationType: 4 729 | namespace: 'microsoft.insights/components' 730 | metricVisualization: { 731 | displayName: 'Send request time' 732 | color: '#EB9371' 733 | } 734 | } 735 | { 736 | resourceMetadata: { 737 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 738 | } 739 | name: 'browserTimings/receiveDuration' 740 | aggregationType: 4 741 | namespace: 'microsoft.insights/components' 742 | metricVisualization: { 743 | displayName: 'Receiving response time' 744 | color: '#0672F1' 745 | } 746 | } 747 | ] 748 | title: 'Average page load time breakdown' 749 | visualization: { 750 | chartType: 3 751 | legendVisualization: { 752 | isVisible: true 753 | position: 2 754 | hideSubtitle: false 755 | } 756 | axisVisualization: { 757 | x: { 758 | isVisible: true 759 | axisType: 2 760 | } 761 | y: { 762 | isVisible: true 763 | axisType: 1 764 | } 765 | } 766 | } 767 | } 768 | } 769 | } 770 | { 771 | name: 'sharedTimeRange' 772 | isOptional: true 773 | } 774 | ] 775 | #disable-next-line BCP036 776 | type: 'Extension/HubsExtension/PartType/MonitorChartPart' 777 | settings: {} 778 | } 779 | } 780 | { 781 | position: { 782 | x: 0 783 | y: 5 784 | colSpan: 4 785 | rowSpan: 3 786 | } 787 | metadata: { 788 | inputs: [ 789 | { 790 | name: 'options' 791 | value: { 792 | chart: { 793 | metrics: [ 794 | { 795 | resourceMetadata: { 796 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 797 | } 798 | name: 'availabilityResults/availabilityPercentage' 799 | aggregationType: 4 800 | namespace: 'microsoft.insights/components' 801 | metricVisualization: { 802 | displayName: 'Availability' 803 | color: '#47BDF5' 804 | } 805 | } 806 | ] 807 | title: 'Average availability' 808 | visualization: { 809 | chartType: 3 810 | legendVisualization: { 811 | isVisible: true 812 | position: 2 813 | hideSubtitle: false 814 | } 815 | axisVisualization: { 816 | x: { 817 | isVisible: true 818 | axisType: 2 819 | } 820 | y: { 821 | isVisible: true 822 | axisType: 1 823 | } 824 | } 825 | } 826 | openBladeOnClick: { 827 | openBlade: true 828 | destinationBlade: { 829 | extensionName: 'HubsExtension' 830 | bladeName: 'ResourceMenuBlade' 831 | parameters: { 832 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 833 | menuid: 'availability' 834 | } 835 | } 836 | } 837 | } 838 | } 839 | } 840 | { 841 | name: 'sharedTimeRange' 842 | isOptional: true 843 | } 844 | ] 845 | #disable-next-line BCP036 846 | type: 'Extension/HubsExtension/PartType/MonitorChartPart' 847 | settings: {} 848 | } 849 | } 850 | { 851 | position: { 852 | x: 4 853 | y: 5 854 | colSpan: 4 855 | rowSpan: 3 856 | } 857 | metadata: { 858 | inputs: [ 859 | { 860 | name: 'options' 861 | value: { 862 | chart: { 863 | metrics: [ 864 | { 865 | resourceMetadata: { 866 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 867 | } 868 | name: 'exceptions/server' 869 | aggregationType: 7 870 | namespace: 'microsoft.insights/components' 871 | metricVisualization: { 872 | displayName: 'Server exceptions' 873 | color: '#47BDF5' 874 | } 875 | } 876 | { 877 | resourceMetadata: { 878 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 879 | } 880 | name: 'dependencies/failed' 881 | aggregationType: 7 882 | namespace: 'microsoft.insights/components' 883 | metricVisualization: { 884 | displayName: 'Dependency failures' 885 | color: '#7E58FF' 886 | } 887 | } 888 | ] 889 | title: 'Server exceptions and Dependency failures' 890 | visualization: { 891 | chartType: 2 892 | legendVisualization: { 893 | isVisible: true 894 | position: 2 895 | hideSubtitle: false 896 | } 897 | axisVisualization: { 898 | x: { 899 | isVisible: true 900 | axisType: 2 901 | } 902 | y: { 903 | isVisible: true 904 | axisType: 1 905 | } 906 | } 907 | } 908 | } 909 | } 910 | } 911 | { 912 | name: 'sharedTimeRange' 913 | isOptional: true 914 | } 915 | ] 916 | #disable-next-line BCP036 917 | type: 'Extension/HubsExtension/PartType/MonitorChartPart' 918 | settings: {} 919 | } 920 | } 921 | { 922 | position: { 923 | x: 8 924 | y: 5 925 | colSpan: 4 926 | rowSpan: 3 927 | } 928 | metadata: { 929 | inputs: [ 930 | { 931 | name: 'options' 932 | value: { 933 | chart: { 934 | metrics: [ 935 | { 936 | resourceMetadata: { 937 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 938 | } 939 | name: 'performanceCounters/processorCpuPercentage' 940 | aggregationType: 4 941 | namespace: 'microsoft.insights/components' 942 | metricVisualization: { 943 | displayName: 'Processor time' 944 | color: '#47BDF5' 945 | } 946 | } 947 | { 948 | resourceMetadata: { 949 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 950 | } 951 | name: 'performanceCounters/processCpuPercentage' 952 | aggregationType: 4 953 | namespace: 'microsoft.insights/components' 954 | metricVisualization: { 955 | displayName: 'Process CPU' 956 | color: '#7E58FF' 957 | } 958 | } 959 | ] 960 | title: 'Average processor and process CPU utilization' 961 | visualization: { 962 | chartType: 2 963 | legendVisualization: { 964 | isVisible: true 965 | position: 2 966 | hideSubtitle: false 967 | } 968 | axisVisualization: { 969 | x: { 970 | isVisible: true 971 | axisType: 2 972 | } 973 | y: { 974 | isVisible: true 975 | axisType: 1 976 | } 977 | } 978 | } 979 | } 980 | } 981 | } 982 | { 983 | name: 'sharedTimeRange' 984 | isOptional: true 985 | } 986 | ] 987 | #disable-next-line BCP036 988 | type: 'Extension/HubsExtension/PartType/MonitorChartPart' 989 | settings: {} 990 | } 991 | } 992 | { 993 | position: { 994 | x: 12 995 | y: 5 996 | colSpan: 4 997 | rowSpan: 3 998 | } 999 | metadata: { 1000 | inputs: [ 1001 | { 1002 | name: 'options' 1003 | value: { 1004 | chart: { 1005 | metrics: [ 1006 | { 1007 | resourceMetadata: { 1008 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 1009 | } 1010 | name: 'exceptions/browser' 1011 | aggregationType: 7 1012 | namespace: 'microsoft.insights/components' 1013 | metricVisualization: { 1014 | displayName: 'Browser exceptions' 1015 | color: '#47BDF5' 1016 | } 1017 | } 1018 | ] 1019 | title: 'Browser exceptions' 1020 | visualization: { 1021 | chartType: 2 1022 | legendVisualization: { 1023 | isVisible: true 1024 | position: 2 1025 | hideSubtitle: false 1026 | } 1027 | axisVisualization: { 1028 | x: { 1029 | isVisible: true 1030 | axisType: 2 1031 | } 1032 | y: { 1033 | isVisible: true 1034 | axisType: 1 1035 | } 1036 | } 1037 | } 1038 | } 1039 | } 1040 | } 1041 | { 1042 | name: 'sharedTimeRange' 1043 | isOptional: true 1044 | } 1045 | ] 1046 | #disable-next-line BCP036 1047 | type: 'Extension/HubsExtension/PartType/MonitorChartPart' 1048 | settings: {} 1049 | } 1050 | } 1051 | { 1052 | position: { 1053 | x: 0 1054 | y: 8 1055 | colSpan: 4 1056 | rowSpan: 3 1057 | } 1058 | metadata: { 1059 | inputs: [ 1060 | { 1061 | name: 'options' 1062 | value: { 1063 | chart: { 1064 | metrics: [ 1065 | { 1066 | resourceMetadata: { 1067 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 1068 | } 1069 | name: 'availabilityResults/count' 1070 | aggregationType: 7 1071 | namespace: 'microsoft.insights/components' 1072 | metricVisualization: { 1073 | displayName: 'Availability test results count' 1074 | color: '#47BDF5' 1075 | } 1076 | } 1077 | ] 1078 | title: 'Availability test results count' 1079 | visualization: { 1080 | chartType: 2 1081 | legendVisualization: { 1082 | isVisible: true 1083 | position: 2 1084 | hideSubtitle: false 1085 | } 1086 | axisVisualization: { 1087 | x: { 1088 | isVisible: true 1089 | axisType: 2 1090 | } 1091 | y: { 1092 | isVisible: true 1093 | axisType: 1 1094 | } 1095 | } 1096 | } 1097 | } 1098 | } 1099 | } 1100 | { 1101 | name: 'sharedTimeRange' 1102 | isOptional: true 1103 | } 1104 | ] 1105 | #disable-next-line BCP036 1106 | type: 'Extension/HubsExtension/PartType/MonitorChartPart' 1107 | settings: {} 1108 | } 1109 | } 1110 | { 1111 | position: { 1112 | x: 4 1113 | y: 8 1114 | colSpan: 4 1115 | rowSpan: 3 1116 | } 1117 | metadata: { 1118 | inputs: [ 1119 | { 1120 | name: 'options' 1121 | value: { 1122 | chart: { 1123 | metrics: [ 1124 | { 1125 | resourceMetadata: { 1126 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 1127 | } 1128 | name: 'performanceCounters/processIOBytesPerSecond' 1129 | aggregationType: 4 1130 | namespace: 'microsoft.insights/components' 1131 | metricVisualization: { 1132 | displayName: 'Process IO rate' 1133 | color: '#47BDF5' 1134 | } 1135 | } 1136 | ] 1137 | title: 'Average process I/O rate' 1138 | visualization: { 1139 | chartType: 2 1140 | legendVisualization: { 1141 | isVisible: true 1142 | position: 2 1143 | hideSubtitle: false 1144 | } 1145 | axisVisualization: { 1146 | x: { 1147 | isVisible: true 1148 | axisType: 2 1149 | } 1150 | y: { 1151 | isVisible: true 1152 | axisType: 1 1153 | } 1154 | } 1155 | } 1156 | } 1157 | } 1158 | } 1159 | { 1160 | name: 'sharedTimeRange' 1161 | isOptional: true 1162 | } 1163 | ] 1164 | #disable-next-line BCP036 1165 | type: 'Extension/HubsExtension/PartType/MonitorChartPart' 1166 | settings: {} 1167 | } 1168 | } 1169 | { 1170 | position: { 1171 | x: 8 1172 | y: 8 1173 | colSpan: 4 1174 | rowSpan: 3 1175 | } 1176 | metadata: { 1177 | inputs: [ 1178 | { 1179 | name: 'options' 1180 | value: { 1181 | chart: { 1182 | metrics: [ 1183 | { 1184 | resourceMetadata: { 1185 | id: '/subscriptions/${subscription().subscriptionId}/resourceGroups/${resourceGroup().name}/providers/Microsoft.Insights/components/${applicationInsights.name}' 1186 | } 1187 | name: 'performanceCounters/memoryAvailableBytes' 1188 | aggregationType: 4 1189 | namespace: 'microsoft.insights/components' 1190 | metricVisualization: { 1191 | displayName: 'Available memory' 1192 | color: '#47BDF5' 1193 | } 1194 | } 1195 | ] 1196 | title: 'Average available memory' 1197 | visualization: { 1198 | chartType: 2 1199 | legendVisualization: { 1200 | isVisible: true 1201 | position: 2 1202 | hideSubtitle: false 1203 | } 1204 | axisVisualization: { 1205 | x: { 1206 | isVisible: true 1207 | axisType: 2 1208 | } 1209 | y: { 1210 | isVisible: true 1211 | axisType: 1 1212 | } 1213 | } 1214 | } 1215 | } 1216 | } 1217 | } 1218 | { 1219 | name: 'sharedTimeRange' 1220 | isOptional: true 1221 | } 1222 | ] 1223 | #disable-next-line BCP036 1224 | type: 'Extension/HubsExtension/PartType/MonitorChartPart' 1225 | settings: {} 1226 | } 1227 | } 1228 | ] 1229 | } 1230 | ] 1231 | } 1232 | } 1233 | 1234 | resource applicationInsights 'Microsoft.Insights/components@2020-02-02' existing = { 1235 | name: applicationInsightsName 1236 | } 1237 | -------------------------------------------------------------------------------- /infra/core/monitor/applicationinsights.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Application Insights instance based on an existing Log Analytics workspace.' 2 | param name string 3 | param dashboardName string = '' 4 | param location string = resourceGroup().location 5 | param tags object = {} 6 | param logAnalyticsWorkspaceId string 7 | 8 | resource applicationInsights 'Microsoft.Insights/components@2020-02-02' = { 9 | name: name 10 | location: location 11 | tags: tags 12 | kind: 'web' 13 | properties: { 14 | Application_Type: 'web' 15 | WorkspaceResourceId: logAnalyticsWorkspaceId 16 | } 17 | } 18 | 19 | module applicationInsightsDashboard 'applicationinsights-dashboard.bicep' = if (!empty(dashboardName)) { 20 | name: 'application-insights-dashboard' 21 | params: { 22 | name: dashboardName 23 | location: location 24 | applicationInsightsName: applicationInsights.name 25 | } 26 | } 27 | 28 | output connectionString string = applicationInsights.properties.ConnectionString 29 | output instrumentationKey string = applicationInsights.properties.InstrumentationKey 30 | output name string = applicationInsights.name 31 | -------------------------------------------------------------------------------- /infra/core/monitor/loganalytics.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates a Log Analytics workspace.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | resource logAnalytics 'Microsoft.OperationalInsights/workspaces@2021-12-01-preview' = { 7 | name: name 8 | location: location 9 | tags: tags 10 | properties: any({ 11 | retentionInDays: 30 12 | features: { 13 | searchVersion: 1 14 | } 15 | sku: { 16 | name: 'PerGB2018' 17 | } 18 | }) 19 | } 20 | 21 | output id string = logAnalytics.id 22 | output name string = logAnalytics.name 23 | -------------------------------------------------------------------------------- /infra/core/monitor/monitoring.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Application Insights instance and a Log Analytics workspace.' 2 | param logAnalyticsName string 3 | param applicationInsightsName string 4 | param applicationInsightsDashboardName string = '' 5 | param location string = resourceGroup().location 6 | param tags object = {} 7 | 8 | module logAnalytics 'loganalytics.bicep' = { 9 | name: 'loganalytics' 10 | params: { 11 | name: logAnalyticsName 12 | location: location 13 | tags: tags 14 | } 15 | } 16 | 17 | module applicationInsights 'applicationinsights.bicep' = { 18 | name: 'applicationinsights' 19 | params: { 20 | name: applicationInsightsName 21 | location: location 22 | tags: tags 23 | dashboardName: applicationInsightsDashboardName 24 | logAnalyticsWorkspaceId: logAnalytics.outputs.id 25 | } 26 | } 27 | 28 | output applicationInsightsConnectionString string = applicationInsights.outputs.connectionString 29 | output applicationInsightsInstrumentationKey string = applicationInsights.outputs.instrumentationKey 30 | output applicationInsightsName string = applicationInsights.outputs.name 31 | output logAnalyticsWorkspaceId string = logAnalytics.outputs.id 32 | output logAnalyticsWorkspaceName string = logAnalytics.outputs.name 33 | -------------------------------------------------------------------------------- /infra/core/networking/cdn-endpoint.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Adds an endpoint to an Azure CDN profile.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | @description('The name of the CDN profile resource') 7 | @minLength(1) 8 | param cdnProfileName string 9 | 10 | @description('Delivery policy rules') 11 | param deliveryPolicyRules array = [] 12 | 13 | @description('The origin URL for the endpoint') 14 | @minLength(1) 15 | param originUrl string 16 | 17 | resource endpoint 'Microsoft.Cdn/profiles/endpoints@2022-05-01-preview' = { 18 | parent: cdnProfile 19 | name: name 20 | location: location 21 | tags: tags 22 | properties: { 23 | originHostHeader: originUrl 24 | isHttpAllowed: false 25 | isHttpsAllowed: true 26 | queryStringCachingBehavior: 'UseQueryString' 27 | optimizationType: 'GeneralWebDelivery' 28 | origins: [ 29 | { 30 | name: replace(originUrl, '.', '-') 31 | properties: { 32 | hostName: originUrl 33 | originHostHeader: originUrl 34 | priority: 1 35 | weight: 1000 36 | enabled: true 37 | } 38 | } 39 | ] 40 | deliveryPolicy: { 41 | rules: deliveryPolicyRules 42 | } 43 | } 44 | } 45 | 46 | resource cdnProfile 'Microsoft.Cdn/profiles@2022-05-01-preview' existing = { 47 | name: cdnProfileName 48 | } 49 | 50 | output id string = endpoint.id 51 | output name string = endpoint.name 52 | output uri string = 'https://${endpoint.properties.hostName}' 53 | -------------------------------------------------------------------------------- /infra/core/networking/cdn-profile.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure CDN profile.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | @description('The pricing tier of this CDN profile') 7 | @allowed([ 8 | 'Custom_Verizon' 9 | 'Premium_AzureFrontDoor' 10 | 'Premium_Verizon' 11 | 'StandardPlus_955BandWidth_ChinaCdn' 12 | 'StandardPlus_AvgBandWidth_ChinaCdn' 13 | 'StandardPlus_ChinaCdn' 14 | 'Standard_955BandWidth_ChinaCdn' 15 | 'Standard_Akamai' 16 | 'Standard_AvgBandWidth_ChinaCdn' 17 | 'Standard_AzureFrontDoor' 18 | 'Standard_ChinaCdn' 19 | 'Standard_Microsoft' 20 | 'Standard_Verizon' 21 | ]) 22 | param sku string = 'Standard_Microsoft' 23 | 24 | resource profile 'Microsoft.Cdn/profiles@2022-05-01-preview' = { 25 | name: name 26 | location: location 27 | tags: tags 28 | sku: { 29 | name: sku 30 | } 31 | } 32 | 33 | output id string = profile.id 34 | output name string = profile.name 35 | -------------------------------------------------------------------------------- /infra/core/networking/cdn.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure CDN profile with a single endpoint.' 2 | param location string = resourceGroup().location 3 | param tags object = {} 4 | 5 | @description('Name of the CDN endpoint resource') 6 | param cdnEndpointName string 7 | 8 | @description('Name of the CDN profile resource') 9 | param cdnProfileName string 10 | 11 | @description('Delivery policy rules') 12 | param deliveryPolicyRules array = [] 13 | 14 | @description('Origin URL for the CDN endpoint') 15 | param originUrl string 16 | 17 | module cdnProfile 'cdn-profile.bicep' = { 18 | name: 'cdn-profile' 19 | params: { 20 | name: cdnProfileName 21 | location: location 22 | tags: tags 23 | } 24 | } 25 | 26 | module cdnEndpoint 'cdn-endpoint.bicep' = { 27 | name: 'cdn-endpoint' 28 | params: { 29 | name: cdnEndpointName 30 | location: location 31 | tags: tags 32 | cdnProfileName: cdnProfile.outputs.name 33 | originUrl: originUrl 34 | deliveryPolicyRules: deliveryPolicyRules 35 | } 36 | } 37 | 38 | output endpointName string = cdnEndpoint.outputs.name 39 | output endpointId string = cdnEndpoint.outputs.id 40 | output profileName string = cdnProfile.outputs.name 41 | output profileId string = cdnProfile.outputs.id 42 | output uri string = cdnEndpoint.outputs.uri 43 | -------------------------------------------------------------------------------- /infra/core/search/search-services.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure AI Search instance.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | param sku object = { 7 | name: 'standard' 8 | } 9 | 10 | param authOptions object = {} 11 | param disableLocalAuth bool = false 12 | param disabledDataExfiltrationOptions array = [] 13 | param encryptionWithCmk object = { 14 | enforcement: 'Unspecified' 15 | } 16 | @allowed([ 17 | 'default' 18 | 'highDensity' 19 | ]) 20 | param hostingMode string = 'default' 21 | param networkRuleSet object = { 22 | bypass: 'None' 23 | ipRules: [] 24 | } 25 | param partitionCount int = 1 26 | @allowed([ 27 | 'enabled' 28 | 'disabled' 29 | ]) 30 | param publicNetworkAccess string = 'enabled' 31 | param replicaCount int = 1 32 | @allowed([ 33 | 'disabled' 34 | 'free' 35 | 'standard' 36 | ]) 37 | param semanticSearch string = 'disabled' 38 | 39 | resource search 'Microsoft.Search/searchServices@2021-04-01-preview' = { 40 | name: name 41 | location: location 42 | tags: tags 43 | identity: { 44 | type: 'SystemAssigned' 45 | } 46 | properties: { 47 | authOptions: authOptions 48 | disableLocalAuth: disableLocalAuth 49 | disabledDataExfiltrationOptions: disabledDataExfiltrationOptions 50 | encryptionWithCmk: encryptionWithCmk 51 | hostingMode: hostingMode 52 | networkRuleSet: networkRuleSet 53 | partitionCount: partitionCount 54 | publicNetworkAccess: publicNetworkAccess 55 | replicaCount: replicaCount 56 | semanticSearch: semanticSearch 57 | } 58 | sku: sku 59 | } 60 | 61 | output id string = search.id 62 | output endpoint string = 'https://${name}.search.windows.net/' 63 | output name string = search.name 64 | -------------------------------------------------------------------------------- /infra/core/security/keyvault-access.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Assigns an Azure Key Vault access policy.' 2 | param name string = 'add' 3 | 4 | param keyVaultName string 5 | param permissions object = { secrets: [ 'get', 'list' ] } 6 | param principalId string 7 | 8 | resource keyVaultAccessPolicies 'Microsoft.KeyVault/vaults/accessPolicies@2022-07-01' = { 9 | parent: keyVault 10 | name: name 11 | properties: { 12 | accessPolicies: [ { 13 | objectId: principalId 14 | tenantId: subscription().tenantId 15 | permissions: permissions 16 | } ] 17 | } 18 | } 19 | 20 | resource keyVault 'Microsoft.KeyVault/vaults@2022-07-01' existing = { 21 | name: keyVaultName 22 | } 23 | -------------------------------------------------------------------------------- /infra/core/security/keyvault-secret.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates or updates a secret in an Azure Key Vault.' 2 | param name string 3 | param tags object = {} 4 | param keyVaultName string 5 | param contentType string = 'string' 6 | @description('The value of the secret. Provide only derived values like blob storage access, but do not hard code any secrets in your templates') 7 | @secure() 8 | param secretValue string 9 | 10 | param enabled bool = true 11 | param exp int = 0 12 | param nbf int = 0 13 | 14 | resource keyVaultSecret 'Microsoft.KeyVault/vaults/secrets@2022-07-01' = { 15 | name: name 16 | tags: tags 17 | parent: keyVault 18 | properties: { 19 | attributes: { 20 | enabled: enabled 21 | exp: exp 22 | nbf: nbf 23 | } 24 | contentType: contentType 25 | value: secretValue 26 | } 27 | } 28 | 29 | resource keyVault 'Microsoft.KeyVault/vaults@2022-07-01' existing = { 30 | name: keyVaultName 31 | } 32 | -------------------------------------------------------------------------------- /infra/core/security/keyvault.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure Key Vault.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | param principalId string = '' 7 | 8 | resource keyVault 'Microsoft.KeyVault/vaults@2022-07-01' = { 9 | name: name 10 | location: location 11 | tags: tags 12 | properties: { 13 | tenantId: subscription().tenantId 14 | sku: { family: 'A', name: 'standard' } 15 | accessPolicies: !empty(principalId) ? [ 16 | { 17 | objectId: principalId 18 | permissions: { secrets: [ 'get', 'list' ] } 19 | tenantId: subscription().tenantId 20 | } 21 | ] : [] 22 | } 23 | } 24 | 25 | output endpoint string = keyVault.properties.vaultUri 26 | output name string = keyVault.name 27 | -------------------------------------------------------------------------------- /infra/core/security/registry-access.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Assigns ACR Pull permissions to access an Azure Container Registry.' 2 | param containerRegistryName string 3 | param principalId string 4 | 5 | var acrPullRole = subscriptionResourceId('Microsoft.Authorization/roleDefinitions', '7f951dda-4ed3-4680-a7ca-43fe172d538d') 6 | 7 | resource aksAcrPull 'Microsoft.Authorization/roleAssignments@2022-04-01' = { 8 | scope: containerRegistry // Use when specifying a scope that is different than the deployment scope 9 | name: guid(subscription().id, resourceGroup().id, principalId, acrPullRole) 10 | properties: { 11 | roleDefinitionId: acrPullRole 12 | principalType: 'ServicePrincipal' 13 | principalId: principalId 14 | } 15 | } 16 | 17 | resource containerRegistry 'Microsoft.ContainerRegistry/registries@2022-02-01-preview' existing = { 18 | name: containerRegistryName 19 | } 20 | -------------------------------------------------------------------------------- /infra/core/security/role.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates a role assignment for a service principal.' 2 | param principalId string 3 | 4 | @allowed([ 5 | 'Device' 6 | 'ForeignGroup' 7 | 'Group' 8 | 'ServicePrincipal' 9 | 'User' 10 | ]) 11 | param principalType string = 'ServicePrincipal' 12 | param roleDefinitionId string 13 | 14 | resource role 'Microsoft.Authorization/roleAssignments@2022-04-01' = { 15 | name: guid(subscription().id, resourceGroup().id, principalId, roleDefinitionId) 16 | properties: { 17 | principalId: principalId 18 | principalType: principalType 19 | roleDefinitionId: resourceId('Microsoft.Authorization/roleDefinitions', roleDefinitionId) 20 | } 21 | } 22 | -------------------------------------------------------------------------------- /infra/core/storage/storage-account.bicep: -------------------------------------------------------------------------------- 1 | metadata description = 'Creates an Azure storage account.' 2 | param name string 3 | param location string = resourceGroup().location 4 | param tags object = {} 5 | 6 | @allowed([ 7 | 'Cool' 8 | 'Hot' 9 | 'Premium' ]) 10 | param accessTier string = 'Hot' 11 | param allowBlobPublicAccess bool = true 12 | param allowCrossTenantReplication bool = true 13 | param allowSharedKeyAccess bool = true 14 | param containers array = [] 15 | param defaultToOAuthAuthentication bool = false 16 | param deleteRetentionPolicy object = {} 17 | @allowed([ 'AzureDnsZone', 'Standard' ]) 18 | param dnsEndpointType string = 'Standard' 19 | param kind string = 'StorageV2' 20 | param minimumTlsVersion string = 'TLS1_2' 21 | param supportsHttpsTrafficOnly bool = true 22 | param networkAcls object = { 23 | bypass: 'AzureServices' 24 | defaultAction: 'Allow' 25 | } 26 | @allowed([ 'Enabled', 'Disabled' ]) 27 | param publicNetworkAccess string = 'Enabled' 28 | param sku object = { name: 'Standard_LRS' } 29 | 30 | resource storage 'Microsoft.Storage/storageAccounts@2022-05-01' = { 31 | name: name 32 | location: location 33 | tags: tags 34 | kind: kind 35 | sku: sku 36 | properties: { 37 | accessTier: accessTier 38 | allowBlobPublicAccess: allowBlobPublicAccess 39 | allowCrossTenantReplication: allowCrossTenantReplication 40 | allowSharedKeyAccess: allowSharedKeyAccess 41 | defaultToOAuthAuthentication: defaultToOAuthAuthentication 42 | dnsEndpointType: dnsEndpointType 43 | minimumTlsVersion: minimumTlsVersion 44 | networkAcls: networkAcls 45 | publicNetworkAccess: publicNetworkAccess 46 | supportsHttpsTrafficOnly: supportsHttpsTrafficOnly 47 | } 48 | 49 | resource blobServices 'blobServices' = if (!empty(containers)) { 50 | name: 'default' 51 | properties: { 52 | deleteRetentionPolicy: deleteRetentionPolicy 53 | } 54 | resource container 'containers' = [for container in containers: { 55 | name: container.name 56 | properties: { 57 | publicAccess: contains(container, 'publicAccess') ? container.publicAccess : 'None' 58 | } 59 | }] 60 | } 61 | } 62 | 63 | output name string = storage.name 64 | output primaryEndpoints object = storage.properties.primaryEndpoints 65 | -------------------------------------------------------------------------------- /infra/core/testing/loadtesting.bicep: -------------------------------------------------------------------------------- 1 | param name string 2 | param location string = resourceGroup().location 3 | param managedIdentity bool = false 4 | param tags object = {} 5 | 6 | resource loadTest 'Microsoft.LoadTestService/loadTests@2022-12-01' = { 7 | name: name 8 | location: location 9 | tags: tags 10 | identity: { type: managedIdentity ? 'SystemAssigned' : 'None' } 11 | properties: { 12 | } 13 | } 14 | 15 | output loadTestingName string = loadTest.name 16 | -------------------------------------------------------------------------------- /infra/main.bicep: -------------------------------------------------------------------------------- 1 | targetScope = 'subscription' 2 | 3 | @description('Specifies the location for all resources.') 4 | param location string 5 | 6 | @minLength(1) 7 | @description('The URL of your first Azure OpenAI endpoint in the following format: https://[name].openai.azure.com') 8 | param backend_1_url string 9 | 10 | @description('The priority of your first OpenAI endpoint (lower number means higher priority)') 11 | param backend_1_priority int 12 | 13 | @minLength(1) 14 | @description('The API key your first OpenAI endpoint') 15 | param backend_1_api_key string 16 | 17 | @minLength(1) 18 | @description('The URL of your second Azure OpenAI endpoint in the following format: https://[name].openai.azure.com') 19 | param backend_2_url string 20 | 21 | @description('The priority of your second OpenAI endpoint (lower number means higher priority)') 22 | param backend_2_priority int 23 | 24 | @minLength(1) 25 | @description('The API key your second OpenAI endpoint') 26 | param backend_2_api_key string 27 | 28 | @minLength(1) 29 | @maxLength(64) 30 | @description('Name which is used to generate a short unique hash for each resource') 31 | param name string 32 | 33 | var resourceToken = toLower(uniqueString(subscription().id, name, location)) 34 | var prefix = '${name}-${resourceToken}' 35 | var tags = { 'azd-env-name': name } 36 | 37 | resource resourceGroup 'Microsoft.Resources/resourceGroups@2021-04-01' = { 38 | name: '${name}-rg' 39 | location: location 40 | tags: tags 41 | } 42 | 43 | // Monitor application with Azure Monitor 44 | module monitoring 'core/monitor/monitoring.bicep' = { 45 | name: 'monitoring' 46 | scope: resourceGroup 47 | params: { 48 | location: location 49 | tags: tags 50 | applicationInsightsDashboardName: '${prefix}-appinsights-dashboard' 51 | applicationInsightsName: '${prefix}-appinsights' 52 | logAnalyticsName: '${take(prefix, 50)}-loganalytics' // Max 63 chars 53 | } 54 | } 55 | 56 | // Web frontend 57 | module web 'web.bicep' = { 58 | name: 'web' 59 | scope: resourceGroup 60 | params: { 61 | name: replace('${take(prefix, 19)}-ca', '--', '-') 62 | location: location 63 | tags: tags 64 | applicationInsightsName: monitoring.outputs.applicationInsightsName 65 | logAnalyticsWorkspaceName: monitoring.outputs.logAnalyticsWorkspaceName 66 | identityName: '${prefix}-id-web' 67 | containerAppsEnvironmentName: '${prefix}-containerapps-env' 68 | containerRegistryName: '${replace(prefix, '-', '')}registry' 69 | backend_1_url: backend_1_url 70 | backend_1_priority: backend_1_priority 71 | backend_1_api_key: backend_1_api_key 72 | backend_2_url: backend_2_url 73 | backend_2_priority: backend_2_priority 74 | backend_2_api_key: backend_2_api_key 75 | } 76 | } 77 | -------------------------------------------------------------------------------- /infra/main.parameters.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#", 3 | "contentVersion": "1.0.0.0", 4 | "parameters": { 5 | "name": { 6 | "value": "${AZURE_ENV_NAME}", 7 | "metadata": { 8 | "description": "Specifies the name of the app." 9 | } 10 | }, 11 | "location": { 12 | "value": "${AZURE_LOCATION}", 13 | "metadata": { 14 | "description": "Specifies the location for all resources." 15 | } 16 | }, 17 | "backend_1_url": { 18 | "value": "", 19 | "metadata": { 20 | "description": "The URL of your first Azure OpenAI endpoint in the following format: https://[name].openai.azure.com" 21 | } 22 | }, 23 | "backend_1_priority": { 24 | "value": 1, 25 | "metadata": { 26 | "description": "The priority of your first OpenAI endpoint (lower number means higher priority)" 27 | } 28 | }, 29 | "backend_1_api_key": { 30 | "value": "", 31 | "metadata": { 32 | "description": "The API key of your first OpenAI endpoint" 33 | } 34 | }, 35 | "backend_2_url": { 36 | "value": "", 37 | "metadata": { 38 | "description": "The URL of your second Azure OpenAI endpoint in the following format: https://[name].openai.azure.com" 39 | } 40 | }, 41 | "backend_2_priority": { 42 | "value": 2, 43 | "metadata": { 44 | "description": "The priority of your second OpenAI endpoint (lower number means higher priority)" 45 | } 46 | }, 47 | "backend_2_api_key": { 48 | "value": "", 49 | "metadata": { 50 | "description": "The API key of your second OpenAI endpoint" 51 | } 52 | } 53 | } 54 | } -------------------------------------------------------------------------------- /infra/web.bicep: -------------------------------------------------------------------------------- 1 | param name string 2 | param location string = resourceGroup().location 3 | param tags object = {} 4 | 5 | param applicationInsightsName string 6 | param containerAppsEnvironmentName string 7 | param containerRegistryName string 8 | param identityName string 9 | param logAnalyticsWorkspaceName string 10 | 11 | @minLength(1) 12 | @description('The URL of your first Azure OpenAI endpoint in the following format: https://[name].openai.azure.com') 13 | param backend_1_url string 14 | 15 | @description('The priority of your first OpenAI endpoint (lower number means higher priority)') 16 | param backend_1_priority int 17 | 18 | @minLength(1) 19 | @description('The API key your first OpenAI endpoint') 20 | param backend_1_api_key string 21 | 22 | @minLength(1) 23 | @description('The URL of your second Azure OpenAI endpoint in the following format: https://[name].openai.azure.com') 24 | param backend_2_url string 25 | 26 | @description('The priority of your second OpenAI endpoint (lower number means higher priority)') 27 | param backend_2_priority int 28 | 29 | @minLength(1) 30 | @description('The API key your second OpenAI endpoint') 31 | param backend_2_api_key string 32 | 33 | resource webIdentity 'Microsoft.ManagedIdentity/userAssignedIdentities@2023-01-31' = { 34 | name: identityName 35 | location: location 36 | } 37 | 38 | 39 | // module containerAppsEnvironment 'core/host/container-apps-environment.bicep' = { 40 | // name: containerAppsEnvironmentName 41 | // params: { 42 | // name: containerAppsEnvironmentName 43 | // location: location 44 | // tags: tags 45 | // logAnalyticsWorkspaceName: logAnalyticsWorkspaceName 46 | // applicationInsightsName: applicationInsightsName 47 | // } 48 | // } 49 | 50 | // module containerRegistry 'core/host/container-registry.bicep' = { 51 | // name: containerRegistryName 52 | // params: { 53 | // name: containerRegistryName 54 | // location: location 55 | // tags: tags 56 | // } 57 | // } 58 | 59 | // Container apps host (including container registry) 60 | module containerApps 'core/host/container-apps.bicep' = { 61 | name: 'container-apps' 62 | params: { 63 | name: 'app' 64 | location: location 65 | containerAppsEnvironmentName: containerAppsEnvironmentName 66 | containerRegistryName: containerRegistryName 67 | logAnalyticsWorkspaceName: logAnalyticsWorkspaceName 68 | } 69 | } 70 | 71 | module app 'core/host/container-app.bicep' = { 72 | name: '${deployment().name}-update' 73 | params: { 74 | name: name 75 | location: location 76 | tags: tags 77 | identityName: identityName 78 | ingressEnabled: true 79 | containerName: 'main' 80 | containerAppsEnvironmentName: containerAppsEnvironmentName 81 | containerRegistryName: containerRegistryName 82 | containerCpuCoreCount: '1' 83 | containerMemory: '2Gi' 84 | containerMinReplicas: 1 85 | containerMaxReplicas: 10 86 | external: true 87 | env: [ 88 | { 89 | name: 'RUNNING_IN_PRODUCTION' 90 | value: 'true' 91 | } 92 | { 93 | name: 'APPLICATIONINSIGHTS_CONNECTION_STRING' 94 | value: applicationInsights.properties.ConnectionString 95 | } 96 | { 97 | name: 'BACKEND_1_URL' 98 | value: backend_1_url 99 | // Continue with the rest of the environment variables 100 | } 101 | { 102 | name: 'BACKEND_1_PRIORITY' 103 | value: string(backend_1_priority) 104 | } 105 | { 106 | name: 'BACKEND_1_APIKEY' 107 | value: backend_1_api_key 108 | } 109 | { 110 | name: 'BACKEND_2_URL' 111 | value: backend_2_url 112 | } 113 | { 114 | name: 'BACKEND_2_PRIORITY' 115 | value: string(backend_2_priority) 116 | } 117 | { 118 | name: 'BACKEND_2_APIKEY' 119 | value: backend_2_api_key 120 | } 121 | ] 122 | imageName: 'andredewes/aoai-smart-loadbalancing:v1' 123 | targetPort: 8080 124 | } 125 | dependsOn: [ 126 | containerApps 127 | ] 128 | } 129 | 130 | resource applicationInsights 'Microsoft.Insights/components@2020-02-02' existing = { 131 | name: applicationInsightsName 132 | } 133 | 134 | output SERVICE_WEB_NAME string = app.outputs.name 135 | output SERVICE_WEB_URI string = app.outputs.uri 136 | output SERVICE_WEB_IMAGE_NAME string = app.outputs.imageName 137 | 138 | output uri string = app.outputs.uri 139 | -------------------------------------------------------------------------------- /src/.dockerignore: -------------------------------------------------------------------------------- 1 | **/.classpath 2 | **/.dockerignore 3 | **/.env 4 | **/.git 5 | **/.gitignore 6 | **/.project 7 | **/.settings 8 | **/.toolstarget 9 | **/.vs 10 | **/.vscode 11 | **/*.*proj.user 12 | **/*.dbmdl 13 | **/*.jfm 14 | **/azds.yaml 15 | **/bin 16 | **/charts 17 | **/docker-compose* 18 | **/Dockerfile* 19 | **/node_modules 20 | **/npm-debug.log 21 | **/obj 22 | **/secrets.dev.yaml 23 | **/values.dev.yaml 24 | LICENSE 25 | README.md 26 | !**/.gitignore 27 | !.git/HEAD 28 | !.git/config 29 | !.git/packed-refs 30 | !.git/refs/heads/** -------------------------------------------------------------------------------- /src/.gitignore: -------------------------------------------------------------------------------- 1 | ## Ignore Visual Studio temporary files, build results, and 2 | ## files generated by popular Visual Studio add-ons. 3 | ## 4 | ## Get latest from `dotnet new gitignore` 5 | 6 | # dotenv files 7 | .env 8 | 9 | # User-specific files 10 | *.rsuser 11 | *.suo 12 | *.user 13 | *.userosscache 14 | *.sln.docstates 15 | 16 | # User-specific files (MonoDevelop/Xamarin Studio) 17 | *.userprefs 18 | 19 | # Mono auto generated files 20 | mono_crash.* 21 | 22 | # Build results 23 | [Dd]ebug/ 24 | [Dd]ebugPublic/ 25 | [Rr]elease/ 26 | [Rr]eleases/ 27 | x64/ 28 | x86/ 29 | [Ww][Ii][Nn]32/ 30 | [Aa][Rr][Mm]/ 31 | [Aa][Rr][Mm]64/ 32 | bld/ 33 | [Bb]in/ 34 | [Oo]bj/ 35 | [Ll]og/ 36 | [Ll]ogs/ 37 | 38 | # Visual Studio 2015/2017 cache/options directory 39 | .vs/ 40 | # Uncomment if you have tasks that create the project's static files in wwwroot 41 | #wwwroot/ 42 | 43 | # Visual Studio 2017 auto generated files 44 | Generated\ Files/ 45 | 46 | # MSTest test Results 47 | [Tt]est[Rr]esult*/ 48 | [Bb]uild[Ll]og.* 49 | 50 | # NUnit 51 | *.VisualState.xml 52 | TestResult.xml 53 | nunit-*.xml 54 | 55 | # Build Results of an ATL Project 56 | [Dd]ebugPS/ 57 | [Rr]eleasePS/ 58 | dlldata.c 59 | 60 | # Benchmark Results 61 | BenchmarkDotNet.Artifacts/ 62 | 63 | # .NET 64 | project.lock.json 65 | project.fragment.lock.json 66 | artifacts/ 67 | 68 | # Tye 69 | .tye/ 70 | 71 | # ASP.NET Scaffolding 72 | ScaffoldingReadMe.txt 73 | 74 | # StyleCop 75 | StyleCopReport.xml 76 | 77 | # Files built by Visual Studio 78 | *_i.c 79 | *_p.c 80 | *_h.h 81 | *.ilk 82 | *.meta 83 | *.obj 84 | *.iobj 85 | *.pch 86 | *.pdb 87 | *.ipdb 88 | *.pgc 89 | *.pgd 90 | *.rsp 91 | *.sbr 92 | *.tlb 93 | *.tli 94 | *.tlh 95 | *.tmp 96 | *.tmp_proj 97 | *_wpftmp.csproj 98 | *.log 99 | *.tlog 100 | *.vspscc 101 | *.vssscc 102 | .builds 103 | *.pidb 104 | *.svclog 105 | *.scc 106 | 107 | # Chutzpah Test files 108 | _Chutzpah* 109 | 110 | # Visual C++ cache files 111 | ipch/ 112 | *.aps 113 | *.ncb 114 | *.opendb 115 | *.opensdf 116 | *.sdf 117 | *.cachefile 118 | *.VC.db 119 | *.VC.VC.opendb 120 | 121 | # Visual Studio profiler 122 | *.psess 123 | *.vsp 124 | *.vspx 125 | *.sap 126 | 127 | # Visual Studio Trace Files 128 | *.e2e 129 | 130 | # TFS 2012 Local Workspace 131 | $tf/ 132 | 133 | # Guidance Automation Toolkit 134 | *.gpState 135 | 136 | # ReSharper is a .NET coding add-in 137 | _ReSharper*/ 138 | *.[Rr]e[Ss]harper 139 | *.DotSettings.user 140 | 141 | # TeamCity is a build add-in 142 | _TeamCity* 143 | 144 | # DotCover is a Code Coverage Tool 145 | *.dotCover 146 | 147 | # AxoCover is a Code Coverage Tool 148 | .axoCover/* 149 | !.axoCover/settings.json 150 | 151 | # Coverlet is a free, cross platform Code Coverage Tool 152 | coverage*.json 153 | coverage*.xml 154 | coverage*.info 155 | 156 | # Visual Studio code coverage results 157 | *.coverage 158 | *.coveragexml 159 | 160 | # NCrunch 161 | _NCrunch_* 162 | .*crunch*.local.xml 163 | nCrunchTemp_* 164 | 165 | # MightyMoose 166 | *.mm.* 167 | AutoTest.Net/ 168 | 169 | # Web workbench (sass) 170 | .sass-cache/ 171 | 172 | # Installshield output folder 173 | [Ee]xpress/ 174 | 175 | # DocProject is a documentation generator add-in 176 | DocProject/buildhelp/ 177 | DocProject/Help/*.HxT 178 | DocProject/Help/*.HxC 179 | DocProject/Help/*.hhc 180 | DocProject/Help/*.hhk 181 | DocProject/Help/*.hhp 182 | DocProject/Help/Html2 183 | DocProject/Help/html 184 | 185 | # Click-Once directory 186 | publish/ 187 | 188 | # Publish Web Output 189 | *.[Pp]ublish.xml 190 | *.azurePubxml 191 | # Note: Comment the next line if you want to checkin your web deploy settings, 192 | # but database connection strings (with potential passwords) will be unencrypted 193 | *.pubxml 194 | *.publishproj 195 | 196 | # Microsoft Azure Web App publish settings. Comment the next line if you want to 197 | # checkin your Azure Web App publish settings, but sensitive information contained 198 | # in these scripts will be unencrypted 199 | PublishScripts/ 200 | 201 | # NuGet Packages 202 | *.nupkg 203 | # NuGet Symbol Packages 204 | *.snupkg 205 | # The packages folder can be ignored because of Package Restore 206 | **/[Pp]ackages/* 207 | # except build/, which is used as an MSBuild target. 208 | !**/[Pp]ackages/build/ 209 | # Uncomment if necessary however generally it will be regenerated when needed 210 | #!**/[Pp]ackages/repositories.config 211 | # NuGet v3's project.json files produces more ignorable files 212 | *.nuget.props 213 | *.nuget.targets 214 | 215 | # Microsoft Azure Build Output 216 | csx/ 217 | *.build.csdef 218 | 219 | # Microsoft Azure Emulator 220 | ecf/ 221 | rcf/ 222 | 223 | # Windows Store app package directories and files 224 | AppPackages/ 225 | BundleArtifacts/ 226 | Package.StoreAssociation.xml 227 | _pkginfo.txt 228 | *.appx 229 | *.appxbundle 230 | *.appxupload 231 | 232 | # Visual Studio cache files 233 | # files ending in .cache can be ignored 234 | *.[Cc]ache 235 | # but keep track of directories ending in .cache 236 | !?*.[Cc]ache/ 237 | 238 | # Others 239 | ClientBin/ 240 | ~$* 241 | *~ 242 | *.dbmdl 243 | *.dbproj.schemaview 244 | *.jfm 245 | *.pfx 246 | *.publishsettings 247 | orleans.codegen.cs 248 | 249 | # Including strong name files can present a security risk 250 | # (https://github.com/github/gitignore/pull/2483#issue-259490424) 251 | #*.snk 252 | 253 | # Since there are multiple workflows, uncomment next line to ignore bower_components 254 | # (https://github.com/github/gitignore/pull/1529#issuecomment-104372622) 255 | #bower_components/ 256 | 257 | # RIA/Silverlight projects 258 | Generated_Code/ 259 | 260 | # Backup & report files from converting an old project file 261 | # to a newer Visual Studio version. Backup files are not needed, 262 | # because we have git ;-) 263 | _UpgradeReport_Files/ 264 | Backup*/ 265 | UpgradeLog*.XML 266 | UpgradeLog*.htm 267 | ServiceFabricBackup/ 268 | *.rptproj.bak 269 | 270 | # SQL Server files 271 | *.mdf 272 | *.ldf 273 | *.ndf 274 | 275 | # Business Intelligence projects 276 | *.rdl.data 277 | *.bim.layout 278 | *.bim_*.settings 279 | *.rptproj.rsuser 280 | *- [Bb]ackup.rdl 281 | *- [Bb]ackup ([0-9]).rdl 282 | *- [Bb]ackup ([0-9][0-9]).rdl 283 | 284 | # Microsoft Fakes 285 | FakesAssemblies/ 286 | 287 | # GhostDoc plugin setting file 288 | *.GhostDoc.xml 289 | 290 | # Node.js Tools for Visual Studio 291 | .ntvs_analysis.dat 292 | node_modules/ 293 | 294 | # Visual Studio 6 build log 295 | *.plg 296 | 297 | # Visual Studio 6 workspace options file 298 | *.opt 299 | 300 | # Visual Studio 6 auto-generated workspace file (contains which files were open etc.) 301 | *.vbw 302 | 303 | # Visual Studio 6 auto-generated project file (contains which files were open etc.) 304 | *.vbp 305 | 306 | # Visual Studio 6 workspace and project file (working project files containing files to include in project) 307 | *.dsw 308 | *.dsp 309 | 310 | # Visual Studio 6 technical files 311 | *.ncb 312 | *.aps 313 | 314 | # Visual Studio LightSwitch build output 315 | **/*.HTMLClient/GeneratedArtifacts 316 | **/*.DesktopClient/GeneratedArtifacts 317 | **/*.DesktopClient/ModelManifest.xml 318 | **/*.Server/GeneratedArtifacts 319 | **/*.Server/ModelManifest.xml 320 | _Pvt_Extensions 321 | 322 | # Paket dependency manager 323 | .paket/paket.exe 324 | paket-files/ 325 | 326 | # FAKE - F# Make 327 | .fake/ 328 | 329 | # CodeRush personal settings 330 | .cr/personal 331 | 332 | # Python Tools for Visual Studio (PTVS) 333 | __pycache__/ 334 | *.pyc 335 | 336 | # Cake - Uncomment if you are using it 337 | # tools/** 338 | # !tools/packages.config 339 | 340 | # Tabs Studio 341 | *.tss 342 | 343 | # Telerik's JustMock configuration file 344 | *.jmconfig 345 | 346 | # BizTalk build output 347 | *.btp.cs 348 | *.btm.cs 349 | *.odx.cs 350 | *.xsd.cs 351 | 352 | # OpenCover UI analysis results 353 | OpenCover/ 354 | 355 | # Azure Stream Analytics local run output 356 | ASALocalRun/ 357 | 358 | # MSBuild Binary and Structured Log 359 | *.binlog 360 | 361 | # NVidia Nsight GPU debugger configuration file 362 | *.nvuser 363 | 364 | # MFractors (Xamarin productivity tool) working folder 365 | .mfractor/ 366 | 367 | # Local History for Visual Studio 368 | .localhistory/ 369 | 370 | # Visual Studio History (VSHistory) files 371 | .vshistory/ 372 | 373 | # BeatPulse healthcheck temp database 374 | healthchecksdb 375 | 376 | # Backup folder for Package Reference Convert tool in Visual Studio 2017 377 | MigrationBackup/ 378 | 379 | # Ionide (cross platform F# VS Code tools) working folder 380 | .ionide/ 381 | 382 | # Fody - auto-generated XML schema 383 | FodyWeavers.xsd 384 | 385 | # VS Code files for those working on multiple tools 386 | .vscode/* 387 | !.vscode/settings.json 388 | !.vscode/tasks.json 389 | !.vscode/launch.json 390 | !.vscode/extensions.json 391 | *.code-workspace 392 | 393 | # Local History for Visual Studio Code 394 | .history/ 395 | 396 | # Windows Installer files from build outputs 397 | *.cab 398 | *.msi 399 | *.msix 400 | *.msm 401 | *.msp 402 | 403 | # JetBrains Rider 404 | *.sln.iml 405 | .idea 406 | 407 | ## 408 | ## Visual studio for Mac 409 | ## 410 | 411 | 412 | # globs 413 | Makefile.in 414 | *.userprefs 415 | *.usertasks 416 | config.make 417 | config.status 418 | aclocal.m4 419 | install-sh 420 | autom4te.cache/ 421 | *.tar.gz 422 | tarballs/ 423 | test-results/ 424 | 425 | # Mac bundle stuff 426 | *.dmg 427 | *.app 428 | 429 | # content below from: https://github.com/github/gitignore/blob/master/Global/macOS.gitignore 430 | # General 431 | .DS_Store 432 | .AppleDouble 433 | .LSOverride 434 | 435 | # Icon must end with two \r 436 | Icon 437 | 438 | 439 | # Thumbnails 440 | ._* 441 | 442 | # Files that might appear in the root of a volume 443 | .DocumentRevisions-V100 444 | .fseventsd 445 | .Spotlight-V100 446 | .TemporaryItems 447 | .Trashes 448 | .VolumeIcon.icns 449 | .com.apple.timemachine.donotpresent 450 | 451 | # Directories potentially created on remote AFP share 452 | .AppleDB 453 | .AppleDesktop 454 | Network Trash Folder 455 | Temporary Items 456 | .apdisk 457 | 458 | # content below from: https://github.com/github/gitignore/blob/master/Global/Windows.gitignore 459 | # Windows thumbnail cache files 460 | Thumbs.db 461 | ehthumbs.db 462 | ehthumbs_vista.db 463 | 464 | # Dump file 465 | *.stackdump 466 | 467 | # Folder config file 468 | [Dd]esktop.ini 469 | 470 | # Recycle Bin used on file shares 471 | $RECYCLE.BIN/ 472 | 473 | # Windows Installer files 474 | *.cab 475 | *.msi 476 | *.msix 477 | *.msm 478 | *.msp 479 | 480 | # Windows shortcuts 481 | *.lnk 482 | 483 | # Vim temporary swap files 484 | *.swp 485 | -------------------------------------------------------------------------------- /src/BackendConfig.cs: -------------------------------------------------------------------------------- 1 | namespace openai_loadbalancer; 2 | 3 | public class BackendConfig 4 | { 5 | public static int HttpTimeoutSeconds = 100; 6 | 7 | public required string Url { get; set; } 8 | public string? DeploymentName { get; set; } 9 | public int Priority { get; set; } 10 | public required string ApiKey { get; set; } 11 | 12 | public static IReadOnlyDictionary LoadConfig(IConfiguration config) 13 | { 14 | var returnDictionary = new Dictionary(); 15 | 16 | var environmentVariables = config.AsEnumerable().Where(x => x.Key.ToUpperInvariant().StartsWith("BACKEND_")).ToDictionary(x => x.Key, x => x.Value, StringComparer.OrdinalIgnoreCase); 17 | 18 | var numberOfBackends = environmentVariables.Select(x => x.Key.Split('_')[1]).Distinct(); 19 | 20 | if (environmentVariables.Count() == 0 || numberOfBackends.Count() == 0) 21 | { 22 | throw new Exception("Could not find any environment variable starting with 'BACKEND_[x]'... please define your backend endpoints"); 23 | } 24 | 25 | foreach (var backendIndex in numberOfBackends) 26 | { 27 | var key = $"BACKEND_{backendIndex}"; 28 | var url = LoadEnvironmentVariable(environmentVariables, backendIndex, "URL"); 29 | var deploymentName = LoadEnvironmentVariable(environmentVariables, backendIndex, "DEPLOYMENT_NAME", isMandatory: false); 30 | var apiKey = LoadEnvironmentVariable(environmentVariables, backendIndex, "APIKEY"); 31 | var priority = Convert.ToInt32(LoadEnvironmentVariable(environmentVariables, backendIndex, "PRIORITY")); 32 | 33 | returnDictionary.Add(key, new BackendConfig { Url = url, ApiKey = apiKey, Priority = priority, DeploymentName = deploymentName }); 34 | } 35 | 36 | //Load the general settings not in scope only for specific backends 37 | var httpTimeout = Environment.GetEnvironmentVariable("HTTP_TIMEOUT_SECONDS"); 38 | 39 | if (httpTimeout != null) 40 | { 41 | HttpTimeoutSeconds = Convert.ToInt32(httpTimeout); 42 | } 43 | 44 | return returnDictionary; 45 | } 46 | 47 | private static string? LoadEnvironmentVariable(IDictionary variables, string backendIndex, string property, bool isMandatory = true) 48 | { 49 | var key = $"BACKEND_{backendIndex}_{property}"; 50 | 51 | if (!variables.TryGetValue(key, out var value) && isMandatory) 52 | { 53 | throw new Exception($"Missing environment variable {key}"); 54 | } 55 | 56 | if (value != null) 57 | { 58 | return value.Trim(); 59 | } 60 | else 61 | { 62 | return null; 63 | } 64 | } 65 | } 66 | 67 | -------------------------------------------------------------------------------- /src/Dockerfile: -------------------------------------------------------------------------------- 1 | #See https://aka.ms/customizecontainer to learn how to customize your debug container and how Visual Studio uses this Dockerfile to build your images for faster debugging. 2 | 3 | FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS base 4 | USER app 5 | WORKDIR /app 6 | EXPOSE 8080 7 | EXPOSE 8081 8 | 9 | FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build 10 | ARG BUILD_CONFIGURATION=Release 11 | WORKDIR /src 12 | COPY ["openai-loadbalancer.csproj", "."] 13 | RUN dotnet restore "./././openai-loadbalancer.csproj" 14 | COPY . . 15 | WORKDIR "/src/." 16 | RUN dotnet build "./openai-loadbalancer.csproj" -c $BUILD_CONFIGURATION -o /app/build 17 | 18 | FROM build AS publish 19 | ARG BUILD_CONFIGURATION=Release 20 | RUN dotnet publish "./openai-loadbalancer.csproj" -c $BUILD_CONFIGURATION -o /app/publish /p:UseAppHost=false 21 | 22 | FROM base AS final 23 | WORKDIR /app 24 | COPY --from=publish /app/publish . 25 | ENTRYPOINT ["dotnet", "openai-loadbalancer.dll"] -------------------------------------------------------------------------------- /src/Program.cs: -------------------------------------------------------------------------------- 1 | using Yarp.ReverseProxy.Health; 2 | using Yarp.ReverseProxy.Transforms; 3 | 4 | namespace openai_loadbalancer; 5 | 6 | public class Program 7 | { 8 | public static void Main(string[] args) 9 | { 10 | var builder = WebApplication.CreateBuilder(args); 11 | 12 | var backendConfiguration = BackendConfig.LoadConfig(builder.Configuration); 13 | var yarpConfiguration = new YarpConfiguration(backendConfiguration); 14 | builder.Services.AddSingleton(); 15 | builder.Services.AddReverseProxy().AddTransforms(m => 16 | { 17 | m.AddRequestTransform(yarpConfiguration.TransformRequest()); 18 | m.AddResponseTransform(yarpConfiguration.TransformResponse()); 19 | }).LoadFromMemory(yarpConfiguration.GetRoutes(), yarpConfiguration.GetClusters()); 20 | 21 | builder.Services.AddHealthChecks(); 22 | var app = builder.Build(); 23 | 24 | app.MapHealthChecks("/healthz"); 25 | app.MapReverseProxy(m => 26 | { 27 | m.UseMiddleware(backendConfiguration); 28 | m.UsePassiveHealthChecks(); 29 | }); 30 | 31 | app.Run(); 32 | } 33 | } 34 | -------------------------------------------------------------------------------- /src/Properties/launchSettings.json: -------------------------------------------------------------------------------- 1 | { 2 | "profiles": { 3 | "https": { 4 | "commandName": "Project", 5 | "launchBrowser": true, 6 | "environmentVariables": { 7 | "ASPNETCORE_ENVIRONMENT": "Development", 8 | "BACKEND_1_URL": "https://andre-openai-eastus.openai.azure.com", 9 | "BACKEND_1_PRIORITY": "1", 10 | "BACKEND_1_APIKEY": "your-api-key", 11 | "BACKEND_2_URL": "https://andre-openai-eastus-2.openai.azure.com", 12 | "BACKEND_2_PRIORITY": "1", 13 | "BACKEND_2_APIKEY": "your-api-key", 14 | "BACKEND_3_URL": "https://andre-openai-canadaeast.openai.azure.com", 15 | "BACKEND_3_PRIORITY": "2", 16 | "BACKEND_3_APIKEY": "your-api-key", 17 | "BACKEND_4_URL": "https://andre-openai-francecentral.openai.azure.com/", 18 | "BACKEND_4_PRIORITY": "3", 19 | "BACKEND_4_APIKEY": "your-api-key", 20 | "BACKEND_5_URL": "https://andre-openai-uksouth.openai.azure.com/", 21 | "BACKEND_5_PRIORITY": "3", 22 | "BACKEND_5_APIKEY": "your-api-key", 23 | "BACKEND_6_URL": "https://andre-openai-westeurope.openai.azure.com/", 24 | "BACKEND_6_PRIORITY": "3", 25 | "BACKEND_6_APIKEY": "your-api-key", 26 | "BACKEND_7_URL": "https://andre-openai-australia.openai.azure.com/", 27 | "BACKEND_7_PRIORITY": "4", 28 | "BACKEND_7_APIKEY": "your-api-key" 29 | }, 30 | "dotnetRunMessages": true, 31 | "applicationUrl": "https://localhost:7151" 32 | } 33 | }, 34 | "$schema": "https://json.schemastore.org/launchsettings.json" 35 | } -------------------------------------------------------------------------------- /src/RetryMiddleware.cs: -------------------------------------------------------------------------------- 1 | using Yarp.ReverseProxy.Model; 2 | 3 | namespace openai_loadbalancer; 4 | 5 | public class RetryMiddleware 6 | { 7 | private readonly RequestDelegate _next; 8 | private readonly Dictionary _backends; 9 | private readonly ILogger _logger; 10 | 11 | public RetryMiddleware(RequestDelegate next, Dictionary backends, ILoggerFactory loggerFactory) 12 | { 13 | _next = next; 14 | _backends = backends; 15 | _logger = loggerFactory.CreateLogger(); 16 | } 17 | 18 | /// 19 | /// The code in this method is based on comments from https://github.com/microsoft/reverse-proxy/issues/56 20 | /// When YARP natively supports retries, this will probably be greatly simplified. 21 | /// 22 | public async Task InvokeAsync(HttpContext context) 23 | { 24 | context.Request.EnableBuffering(); 25 | 26 | var shouldRetry = true; 27 | var retryCount = 0; 28 | 29 | while (shouldRetry) 30 | { 31 | var reverseProxyFeature = context.GetReverseProxyFeature(); 32 | var destination = PickOneDestination(context); 33 | 34 | reverseProxyFeature.AvailableDestinations = new List(destination); 35 | 36 | if (retryCount > 0) 37 | { 38 | //If this is a retry, we must reset the request body to initial position and clear the current response 39 | context.Request.Body.Position = 0; 40 | reverseProxyFeature.ProxiedDestination = null; 41 | context.Response.Clear(); 42 | } 43 | 44 | await _next(context); 45 | 46 | var statusCode = context.Response.StatusCode; 47 | var atLeastOneBackendHealthy = GetNumberHealthyEndpoints(context) > 0; 48 | retryCount++; 49 | 50 | shouldRetry = (statusCode is 429 or >= 500) && atLeastOneBackendHealthy; 51 | } 52 | } 53 | 54 | private static int GetNumberHealthyEndpoints(HttpContext context) 55 | { 56 | return context.GetReverseProxyFeature().AllDestinations.Count(m => m.Health.Passive is DestinationHealth.Healthy or DestinationHealth.Unknown); 57 | } 58 | 59 | 60 | /// 61 | /// The native YARP ILoadBalancingPolicy interface does not play well with HTTP retries, that's why we're adding this custom load-balancing code. 62 | /// This needs to be reevaluated to a ILoadBalancingPolicy implementation when YARP supports natively HTTP retries. 63 | /// 64 | private DestinationState PickOneDestination(HttpContext context) 65 | { 66 | var reverseProxyFeature = context.GetReverseProxyFeature(); 67 | var allDestinations = reverseProxyFeature.AllDestinations; 68 | 69 | var selectedPriority = int.MaxValue; 70 | var availableBackends = new List(); 71 | 72 | for (var i = 0; i < allDestinations.Count; i++) 73 | { 74 | var destination = allDestinations[i]; 75 | 76 | if (destination.Health.Passive != DestinationHealth.Unhealthy) 77 | { 78 | var destinationPriority = _backends[destination.DestinationId].Priority; 79 | 80 | if (destinationPriority < selectedPriority) 81 | { 82 | selectedPriority = destinationPriority; 83 | availableBackends.Clear(); 84 | availableBackends.Add(i); 85 | } 86 | else if (destinationPriority == selectedPriority) 87 | { 88 | availableBackends.Add(i); 89 | } 90 | } 91 | } 92 | 93 | int backendIndex; 94 | 95 | if (availableBackends.Count == 1) 96 | { 97 | //Returns the only available backend if we have only one available 98 | backendIndex = availableBackends[0]; 99 | } 100 | else 101 | if (availableBackends.Count > 0) 102 | { 103 | //Returns a random backend from the list if we have more than one available with the same priority 104 | backendIndex = availableBackends[Random.Shared.Next(0, availableBackends.Count)]; 105 | } 106 | else 107 | { 108 | //Returns a random backend if all backends are unhealthy 109 | _logger.LogWarning($"All backends are unhealthy. Picking a random backend..."); 110 | backendIndex = Random.Shared.Next(0, allDestinations.Count); 111 | } 112 | 113 | var pickedDestination = allDestinations[backendIndex]; 114 | 115 | return pickedDestination; 116 | } 117 | } 118 | -------------------------------------------------------------------------------- /src/YarpConfiguration.cs: -------------------------------------------------------------------------------- 1 | using System.Net; 2 | using Yarp.ReverseProxy.Configuration; 3 | using Yarp.ReverseProxy.Forwarder; 4 | using Yarp.ReverseProxy.Health; 5 | using Yarp.ReverseProxy.Model; 6 | using Yarp.ReverseProxy.Transforms; 7 | 8 | namespace openai_loadbalancer; 9 | 10 | public class YarpConfiguration 11 | { 12 | private readonly IReadOnlyDictionary backends; 13 | 14 | public YarpConfiguration(IReadOnlyDictionary backends) 15 | { 16 | this.backends = backends; 17 | } 18 | 19 | public RouteConfig[] GetRoutes() 20 | { 21 | return 22 | [ 23 | new RouteConfig() 24 | { 25 | RouteId = "route", 26 | ClusterId = "cluster", 27 | Match = new RouteMatch 28 | { 29 | Path = "{**catch-all}" 30 | } 31 | } 32 | ]; 33 | } 34 | public ClusterConfig[] GetClusters() 35 | { 36 | 37 | var destinations = new Dictionary(StringComparer.OrdinalIgnoreCase); 38 | 39 | foreach (var backend in backends) 40 | { 41 | var metadata = new Dictionary 42 | { 43 | { "url", backend.Value.Url }, 44 | { "priority", backend.Value.Priority.ToString() } 45 | }; 46 | destinations.Add(backend.Key, new DestinationConfig { Address = backend.Value.Url, Metadata = metadata }); 47 | } 48 | 49 | return 50 | [ 51 | new ClusterConfig() 52 | { 53 | ClusterId = "cluster", 54 | HealthCheck = new HealthCheckConfig 55 | { 56 | Passive = new PassiveHealthCheckConfig 57 | { 58 | Enabled = true, 59 | Policy = ThrottlingHealthPolicy.ThrottlingPolicyName, 60 | } 61 | }, 62 | Destinations = destinations, 63 | HttpRequest = new ForwarderRequestConfig { ActivityTimeout = TimeSpan.FromSeconds(BackendConfig.HttpTimeoutSeconds) } 64 | } 65 | ]; 66 | } 67 | 68 | public Func TransformResponse() 69 | { 70 | return context => 71 | { 72 | if (context.ProxyResponse?.StatusCode is HttpStatusCode.TooManyRequests or >= HttpStatusCode.InternalServerError) 73 | { 74 | var reverseProxyContext = context.HttpContext.GetReverseProxyFeature(); 75 | 76 | var canRetry = reverseProxyContext.AllDestinations.Count(m => m.Health.Passive != DestinationHealth.Unhealthy && m.DestinationId != reverseProxyContext?.ProxiedDestination?.DestinationId) > 0; 77 | 78 | if (canRetry) 79 | { 80 | // Suppress the response body from being written when we will retry 81 | context.SuppressResponseBody = true; 82 | } 83 | } 84 | 85 | return default; 86 | }; 87 | } 88 | 89 | internal Func TransformRequest() 90 | { 91 | return context => 92 | { 93 | var proxyHeaders = context.ProxyRequest.Headers; 94 | var reverseProxyFeature = context.HttpContext.GetReverseProxyFeature(); 95 | 96 | var backendConfig = backends[reverseProxyFeature.AvailableDestinations[0].DestinationId]; 97 | proxyHeaders.Remove("api-key"); 98 | proxyHeaders.Add("api-key", backendConfig.ApiKey); 99 | 100 | if (backendConfig.DeploymentName != null) 101 | { 102 | var pathSegments = context.Path.Value!.Split('/'); 103 | 104 | if (pathSegments.Length >= 4) 105 | { 106 | //Incoming path should be coming in format "/openai/deployments/{deploymentName}/*" 107 | //We must grab the {deploynameName} from in the incoming request (array position [3]) and replace it by the one specified in the configuration 108 | context.Path = new PathString(context.Path.Value.Replace(pathSegments[3], backendConfig.DeploymentName)); 109 | } 110 | } 111 | 112 | return default; 113 | }; 114 | } 115 | } 116 | 117 | public class ThrottlingHealthPolicy : IPassiveHealthCheckPolicy 118 | { 119 | public static string ThrottlingPolicyName = "ThrottlingPolicy"; 120 | private readonly IDestinationHealthUpdater _healthUpdater; 121 | 122 | public ThrottlingHealthPolicy(IDestinationHealthUpdater healthUpdater) 123 | { 124 | _healthUpdater = healthUpdater; 125 | } 126 | 127 | public string Name => ThrottlingPolicyName; 128 | 129 | public void RequestProxied(HttpContext context, ClusterState cluster, DestinationState destination) 130 | { 131 | var headers = context.Response.Headers; 132 | 133 | if (context.Response.StatusCode is 429 or >= 500) 134 | { 135 | var retryAfterSeconds = 10; 136 | 137 | if (headers.TryGetValue("Retry-After", out var retryAfterHeader) && retryAfterHeader.Count > 0 && int.TryParse(retryAfterHeader[0], out var retryAfter)) 138 | { 139 | retryAfterSeconds = retryAfter; 140 | } 141 | else 142 | if (headers.TryGetValue("x-ratelimit-reset-requests", out var ratelimiResetRequests) && ratelimiResetRequests.Count > 0 && int.TryParse(ratelimiResetRequests[0], out var ratelimiResetRequest)) 143 | { 144 | retryAfterSeconds = ratelimiResetRequest; 145 | } 146 | else 147 | if (headers.TryGetValue("x-ratelimit-reset-tokens", out var ratelimitResetTokens) && ratelimitResetTokens.Count > 0 && int.TryParse(ratelimitResetTokens[0], out var ratelimitResetToken)) 148 | { 149 | retryAfterSeconds = ratelimitResetToken; 150 | } 151 | 152 | _healthUpdater.SetPassive(cluster, destination, DestinationHealth.Unhealthy, TimeSpan.FromSeconds(retryAfterSeconds)); 153 | } 154 | } 155 | } 156 | -------------------------------------------------------------------------------- /src/appsettings.Development.json: -------------------------------------------------------------------------------- 1 | { 2 | "Logging": { 3 | "LogLevel": { 4 | "Default": "Information", 5 | "Microsoft.AspNetCore": "Warning" 6 | } 7 | } 8 | } 9 | -------------------------------------------------------------------------------- /src/appsettings.json: -------------------------------------------------------------------------------- 1 | { 2 | "Logging": { 3 | "LogLevel": { 4 | "Default": "Information", 5 | "Microsoft.AspNetCore": "Warning" 6 | } 7 | }, 8 | "AllowedHosts": "*" 9 | } 10 | -------------------------------------------------------------------------------- /src/openai-loadbalancer.csproj: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | net8.0 5 | enable 6 | enable 7 | openai_loadbalancer 8 | 06475c31-d32a-4a5b-9c8d-0f290a8dbd75 9 | Linux 10 | . 11 | false 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | -------------------------------------------------------------------------------- /src/openai-loadbalancer.sln: -------------------------------------------------------------------------------- 1 |  2 | Microsoft Visual Studio Solution File, Format Version 12.00 3 | # Visual Studio Version 17 4 | VisualStudioVersion = 17.5.002.0 5 | MinimumVisualStudioVersion = 10.0.40219.1 6 | Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "openai-loadbalancer", "openai-loadbalancer.csproj", "{A4DD3578-4F24-436C-871E-8C1E630C7370}" 7 | EndProject 8 | Global 9 | GlobalSection(SolutionConfigurationPlatforms) = preSolution 10 | Debug|Any CPU = Debug|Any CPU 11 | Release|Any CPU = Release|Any CPU 12 | EndGlobalSection 13 | GlobalSection(ProjectConfigurationPlatforms) = postSolution 14 | {A4DD3578-4F24-436C-871E-8C1E630C7370}.Debug|Any CPU.ActiveCfg = Debug|Any CPU 15 | {A4DD3578-4F24-436C-871E-8C1E630C7370}.Debug|Any CPU.Build.0 = Debug|Any CPU 16 | {A4DD3578-4F24-436C-871E-8C1E630C7370}.Release|Any CPU.ActiveCfg = Release|Any CPU 17 | {A4DD3578-4F24-436C-871E-8C1E630C7370}.Release|Any CPU.Build.0 = Release|Any CPU 18 | EndGlobalSection 19 | GlobalSection(SolutionProperties) = preSolution 20 | HideSolutionNode = FALSE 21 | EndGlobalSection 22 | GlobalSection(ExtensibilityGlobals) = postSolution 23 | SolutionGuid = {8289027B-BF39-495C-8A1F-E096BCA22903} 24 | EndGlobalSection 25 | EndGlobal 26 | --------------------------------------------------------------------------------