├── .gitignore ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── SECURITY.md ├── SUPPORT.md ├── partials ├── interactive_prompts.sh └── setup_ado.sh ├── pipelines ├── deploy-pipeline-adf.yml └── deploy-pipeline-syn.yml ├── readme_assets ├── 1_verify_workspaces.png ├── 2_start_setup.png ├── 3_setup_git_repository.png ├── 4_setup_git_integration.png ├── 5_review_pipelines.png ├── architecture.drawio ├── architecture.drawio.png └── fta.png └── setup.sh /.gitignore: -------------------------------------------------------------------------------- 1 | .$* -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Microsoft Open Source Code of Conduct 2 | 3 | This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). 4 | 5 | Resources: 6 | 7 | - [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/) 8 | - [Microsoft Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) 9 | - Contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with questions or concerns 10 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing 2 | 3 | Welcome, and thank you for your interest in contributing. There are many ways to contribute: 4 | 5 | - Submit issues to report bugs and make suggestions. Please search the existing issues before filing new issues to avoid duplicates. For new issues, file your bug or feature request as a new Issue and select the appropriate label from the available labels. 6 | 7 | - Review the source code changes. 8 | 9 | - Contribute features and fixes by forking the repository and creating a pull request. 10 | 11 | - The naming convention for new features branches is _feature(fix)/issueNumber-description_. Example: _feature/13-doc-contribution_ 12 | 13 | ## Contributor License Agreement 14 | 15 | This project welcomes contributions and suggestions. Most contributions require you to agree to a 16 | Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us 17 | the rights to use your contribution. For details, visit . 18 | 19 | When you submit a pull request, a CLA bot will automatically determine whether you need to provide 20 | a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions 21 | provided by the bot. You will only need to do this once across all repos using our CLA. 22 | 23 | ## Microsoft Open Source Code of Conduct 24 | 25 | This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) Microsoft Corporation. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ![Fast Track for Azure](./readme_assets/fta.png) 2 | Move to Azure efficiently with customized guidance from Azure engineering. [FastTrack for Azure – Benefits and FAQ | Microsoft Azure](https://azure.microsoft.com/en-us/programs/azure-fasttrack/) 3 | 4 | # Analytics CI/CD CLI Utility 5 | 6 | Command line tool for quickly setting up CI/CD for Azure Data Factory and Synapse Analytics. 7 | 8 | ## Table of Contents 9 | - [Overview](#overview) 10 | - [Prerequisites](#prerequisites) 11 | - [Solution Architecture](#solution-architecture) 12 | - [Usage](#usage) 13 | - [Result](#result) 14 | - [Contributing](#contributing) 15 | - [Trademarks](#trademarks) 16 | 17 | ## Overview 18 | 19 | Azure Data Factory and Synapse Analytics both have built-in Git integration. With a few clicks, a user may link their workspace with a Git repository (in either GitHub or Azure DevOps) and start applying version control. 20 | 21 | However, there is some additional complexity when we consider multiple environments - development, test and production, for example. There are multiple solutions to this problem, and the project team must carefully choose and set up a CI/CD mechanism that fits their needs. 22 | 23 | This repository provides an interactive command line tool that helps you set up CI/CD for Data Factory and Synapse Analytics. This template is simple, yet effective and adaptable to most use-case scenarios, while following best practices around DevOps for Analytics. 24 | 25 | ## Prerequisites 26 | 27 | - Active Azure Subscription 28 | - Write access to an Azure DevOps project 29 | - Azure Data Factory or Synapse Workspaces (or permission to create them) 30 | 31 | ## Solution Architecture 32 | 33 | Below is the architecture diagram of this solution and how it interacts with your existing subscriptions. 34 | 35 | ![Solution Architecture](./readme_assets/architecture.drawio.png) 36 | 37 | The Analytics CI/CD Setup Tool will interact with the user, requesting them to choose the target subscriptions and workspaces. Then, it will collect the git configuration from the development workspace, and use the provided information to create the Deploy pipeline. 38 | 39 | Triggering the Deploy pipeline will publish the development environment to production. The tooling will allow you to deploy the desired version, even if new features have been published and are under active development. 40 | 41 | ## Usage 42 | 43 | | Step | Description | Detail | 44 | | --- | ------------ | ----- | 45 | | 1 | Create Data Factory or Synapse workspaces | ![Identify workspaces](./readme_assets/1_verify_workspaces.png) | 46 | | 2 | Set up Azure DevOps project with a git repository | ![Setup Git Repository](./readme_assets/3_setup_git_repository.png) | 47 | | 3 | Connect your development workspace with the git repository | ![Setup Git Integration](./readme_assets/4_setup_git_integration.png) | 48 | | 4 | Set up Azure CLI
- Install az cli
- Log in to the correct tenant
- Make sure the `datafactory` or `synapse`, as well as `devops` plugins are installed | ----- | 49 | | 5 | Clone this repository | ```git clone https://github.com/microsoft/ftatoolkit-analytics-cicd-cli.git``` | 50 | | 6 | Run `sh setup.sh` | ![Start setup](./readme_assets/2_start_setup.png) | 51 | | 7 | Follow the interactive prompts | ----- | 52 | | 8 | Review newly created pipeline | ![Start setup](./readme_assets/5_review_pipelines.png) | 53 | 54 | ## Result 55 | 56 | After executing the setup script, you may run the Workspace Deploy Pipeline to replicate your Development environment to Production. You may also customize the pipeline to better suit your use case, if necessary. 57 | 58 | ## Contributing 59 | 60 | This project welcomes contributions and suggestions. Most contributions require you to agree to a 61 | Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us 62 | the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com. 63 | 64 | When you submit a pull request, a CLA bot will automatically determine whether you need to provide 65 | a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions 66 | provided by the bot. You will only need to do this once across all repos using our CLA. 67 | 68 | This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). 69 | For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or 70 | contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. 71 | 72 | ## Trademarks 73 | 74 | This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft 75 | trademarks or logos is subject to and must follow 76 | [Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general). 77 | Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. 78 | Any use of third-party trademarks or logos are subject to those third-party's policies. 79 | -------------------------------------------------------------------------------- /SECURITY.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | ## Security 4 | 5 | Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet), [Xamarin](https://github.com/xamarin), and [our GitHub organizations](https://opensource.microsoft.com/). 6 | 7 | If you believe you have found a security vulnerability in any Microsoft-owned repository that meets [Microsoft's definition of a security vulnerability](https://aka.ms/opensource/security/definition), please report it to us as described below. 8 | 9 | ## Reporting Security Issues 10 | 11 | **Please do not report security vulnerabilities through public GitHub issues.** 12 | 13 | Instead, please report them to the Microsoft Security Response Center (MSRC) at [https://msrc.microsoft.com/create-report](https://aka.ms/opensource/security/create-report). 14 | 15 | If you prefer to submit without logging in, send email to [secure@microsoft.com](mailto:secure@microsoft.com). If possible, encrypt your message with our PGP key; please download it from the [Microsoft Security Response Center PGP Key page](https://aka.ms/opensource/security/pgpkey). 16 | 17 | You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://aka.ms/opensource/security/msrc). 18 | 19 | Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue: 20 | 21 | * Type of issue (e.g. buffer overflow, SQL injection, cross-site scripting, etc.) 22 | * Full paths of source file(s) related to the manifestation of the issue 23 | * The location of the affected source code (tag/branch/commit or direct URL) 24 | * Any special configuration required to reproduce the issue 25 | * Step-by-step instructions to reproduce the issue 26 | * Proof-of-concept or exploit code (if possible) 27 | * Impact of the issue, including how an attacker might exploit the issue 28 | 29 | This information will help us triage your report more quickly. 30 | 31 | If you are reporting for a bug bounty, more complete reports can contribute to a higher bounty award. Please visit our [Microsoft Bug Bounty Program](https://aka.ms/opensource/security/bounty) page for more details about our active programs. 32 | 33 | ## Preferred Languages 34 | 35 | We prefer all communications to be in English. 36 | 37 | ## Policy 38 | 39 | Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://aka.ms/opensource/security/cvd). 40 | 41 | 42 | -------------------------------------------------------------------------------- /SUPPORT.md: -------------------------------------------------------------------------------- 1 | # Support 2 | 3 | ## How to file issues and get help 4 | 5 | This project uses GitHub Issues to track bugs and feature requests. Please search the existing 6 | issues before filing new issues to avoid duplicates. For new issues, file your bug or 7 | feature request as a new Issue. 8 | 9 | ## Microsoft Support Policy 10 | 11 | Support for this project is limited to the resources listed above. 12 | -------------------------------------------------------------------------------- /partials/interactive_prompts.sh: -------------------------------------------------------------------------------- 1 | # Copyright (c) Microsoft Corporation. 2 | # Licensed under the MIT License. 3 | 4 | # Requests key press so user can read prompts 5 | wait_for_prompt () { 6 | read -n1 -s -r -p $'Press any key to continue, Ctrl+C to exit.\n' key 7 | echo 8 | } 9 | 10 | # Outputs: $output JSON 11 | api_selection () { 12 | echo "Listing resources ($1)..." 13 | items=${2} 14 | select item_name in $(echo $items | jq '.[].name'); do break; done; 15 | if [ -z "$item_name" ] 16 | then 17 | echo "Invalid $1 choice. Exiting..." 18 | exit 1 19 | fi 20 | echo "You chose $item_name\n" 21 | output=$(echo $items | jq " .[] | select(.name==$item_name)") 22 | } 23 | 24 | # Outputs: $subscription JSON, configures CLI 25 | choose_subscription_interactive () { 26 | wait_for_prompt 27 | api_selection subscription """$(echo $(az account list) | jq 'map(.name |= gsub(" ";"__"))')""" 28 | az account set -n $(echo $output | jq .id -r) 29 | subscription=$output 30 | } 31 | 32 | # Outputs: $service string 33 | choose_service_interactive () { 34 | echo "Please select an Analytics service." 35 | api_selection service '[{"name":"Azure_Data_Factory"}, {"name":"Azure_Synapse_Analytics"}]' 36 | service=$(echo $output | jq .name -r) 37 | } 38 | 39 | # Outputs: $workspace JSON 40 | choose_workspace_interactive () { 41 | if [[ $service == "Azure_Data_Factory" ]]; then 42 | api_selection factory """$(az datafactory list)""" 43 | workspace=$output 44 | elif [[ $service == "Azure_Synapse_Analytics" ]]; then 45 | api_selection synapse """$(az synapse workspace list)""" 46 | workspace=$(az synapse workspace show --name $(echo $output | jq .name -r) --resource-group $(echo $output | jq .resourceGroup -r)) 47 | else 48 | echo "This Analytics service is not currently supported" 49 | fi 50 | } 51 | 52 | # Outputs: $service_connection JSON 53 | choose_service_connection_interactive() { 54 | echo "Select Service Connection for deploying workspace (must have Contributor access)." 55 | api_selection service_connection """$(az devops service-endpoint list | jq 'map(select(.type == "azurerm"))')""" 56 | service_connection=$output 57 | } -------------------------------------------------------------------------------- /partials/setup_ado.sh: -------------------------------------------------------------------------------- 1 | # Copyright (c) Microsoft Corporation. 2 | # Licensed under the MIT License. 3 | 4 | # Set up Azure DevOps repository for CI/CD 5 | configure_ado() { 6 | echo "Azure Devops source control detected. 7 | We will create a pipeline named Workspace Deploy, which publishes the Development version to Production." 8 | 9 | echo "Enter ADO authentication" 10 | read -p "DevOps URL [dev.azure.com]" ORG_URL 11 | ORG_URL=${ORG_URL:-dev.azure.com} 12 | read -p "DevOps Username: " ADO_USER 13 | read -s -p "DevOps Personal Access Token: " ADO_PAT 14 | echo 15 | [[ $1 = "adf" ]] && DEFAULT_PUBLISH_BRANCH=adf_publish || DEFAULT_PUBLISH_BRANCH=workspace_publish 16 | read -p "Publish branch name [$DEFAULT_PUBLISH_BRANCH]: " PUBLISH_BRANCH 17 | PUBLISH_BRANCH=${PUBLISH_BRANCH:-${DEFAULT_PUBLISH_BRANCH}} 18 | 19 | if [[ $1 = "adf" ]]; then 20 | repo_configuration=$(echo $SOURCE_WORKSPACE | jq .repoConfiguration) 21 | else 22 | repo_configuration=$(echo $SOURCE_WORKSPACE | jq .workspaceRepositoryConfiguration) 23 | fi 24 | ORG_NAME=$(echo $repo_configuration | jq " .accountName" -r) 25 | PROJECT_NAME=$(echo $repo_configuration | jq " .projectName" -r) 26 | REPO_NAME=$(echo $repo_configuration | jq " .repositoryName" -r) 27 | LOCATION=$(echo $TARGET_WORKSPACE | jq .location -r) 28 | echo $ADO_PAT | az devops login 29 | az devops configure --defaults organization=https://${ORG_URL}/${ORG_NAME} project=${PROJECT_NAME} 30 | } 31 | 32 | add_pipelines_to_git() { 33 | set +e 34 | rm -rf ./"${REPO_NAME}" 35 | git clone https://${ADO_USER}:${ADO_PAT}@${ORG_URL}/${ORG_NAME}/${PROJECT_NAME}/_git/${REPO_NAME} --branch ${PUBLISH_BRANCH} --single-branch 36 | cd "${REPO_NAME}" 37 | mkdir .ado 38 | mkdir .ado/workflows 39 | cp ../pipelines/deploy-pipeline-$1.yml ./.ado/workflows/deploy-pipeline.yml 40 | git add . 41 | git commit -m "Add CI/CD Pipelines" 42 | git push 43 | set -e 44 | } 45 | 46 | create_deploy_pipeline() { 47 | TARGET_SUBSCRIPTION_ID="$(echo $TARGET_SUBSCRIPTION | jq .id -r)" 48 | SERVICE_CONNECTION_NAME=$(echo $service_connection | jq " .name" -r) 49 | az pipelines create --name "Workspace Deploy" --description "Deploy Workspace resources to production environment" --repository ${REPO_NAME} --branch ${PUBLISH_BRANCH} --repository-type tfsgit --yml-path .ado/workflows/deploy-pipeline.yml --skip-run > /dev/null 50 | az pipelines variable create --pipeline-name "Workspace Deploy" --name ARM_CONNECTION --value ${SERVICE_CONNECTION_NAME} > /dev/null 51 | az pipelines variable create --pipeline-name "Workspace Deploy" --name LOCATION --value ${LOCATION} > /dev/null 52 | az pipelines variable create --pipeline-name "Workspace Deploy" --name RESOURCE_GROUP --value ${TARGET_RESOURCE_GROUP} > /dev/null 53 | az pipelines variable create --pipeline-name "Workspace Deploy" --name SUBSCRIPTION_ID --value ${TARGET_SUBSCRIPTION_ID} > /dev/null 54 | az pipelines variable create --pipeline-name "Workspace Deploy" --name SOURCE_WORKSPACE_NAME --value ${SOURCE_WORKSPACE_NAME} > /dev/null 55 | az pipelines variable create --pipeline-name "Workspace Deploy" --name WORKSPACE_NAME --value ${TARGET_WORKSPACE_NAME} > /dev/null 56 | } 57 | 58 | 59 | setup_ado () { 60 | configure_ado $1 61 | echo "Creating Service Principal..." 62 | set +e 63 | app=$(az ad app create --display-name "$PROJECT_NAME-cicd") 64 | appid=$(echo $app | jq " .appId" -r) 65 | subid=$(echo $subscription | jq " .id" -r) 66 | subname=$(echo $subscription | jq " .name" -r) 67 | tenantid=$(echo $subscription | jq " .tenantId" -r) 68 | rg=$(echo $TARGET_WORKSPACE | jq " .resourceGroup" -r) 69 | credential=$(az ad app credential reset --id $appid) 70 | service_principal=$(az ad sp create --id $appid) 71 | service_principal=$(az ad sp show --id $appid) 72 | spid=$(echo $service_principal | jq " .id" -r) 73 | echo "Assigning required permissions to Service Principal..." 74 | az role assignment create --assignee $spid \ 75 | --role "Contributor" \ 76 | --scope "/subscriptions/$subid/resourcegroups/$rg" > /dev/null 77 | if [[ $1 = "syn" ]]; then 78 | az synapse role assignment create --workspace-name $TARGET_WORKSPACE_NAME --scope workspaces/$TARGET_WORKSPACE_NAME --role "Synapse Administrator" --assignee $spid > /dev/null 79 | fi 80 | set -e 81 | echo "Creating service connection..." 82 | service_connection=$(AZURE_DEVOPS_EXT_AZURE_RM_SERVICE_PRINCIPAL_KEY=$(echo $credential | jq " .password" -r) az devops service-endpoint azurerm create --azure-rm-service-principal-id $appid --azure-rm-subscription-id $subid --azure-rm-subscription-name "$subname" --azure-rm-tenant-id $tenantid --name cicdtool) 83 | SERVICE_CONNECTION_NAME=$(echo $service_connection | jq " .name" -r) 84 | add_pipelines_to_git $1 85 | create_deploy_pipeline 86 | echo " 87 | All done! Check your Azure DevOps project for a pipeline named Workspace Deploy. 88 | It will replicate your SOURCE workspace to your TARGET workspace" 89 | } -------------------------------------------------------------------------------- /pipelines/deploy-pipeline-adf.yml: -------------------------------------------------------------------------------- 1 | trigger: none 2 | 3 | pool: 4 | vmImage: ubuntu-latest 5 | 6 | steps: 7 | - task: AzureResourceManagerTemplateDeployment@3 8 | inputs: 9 | deploymentScope: 'Resource Group' 10 | azureResourceManagerConnection: '$(ARM_CONNECTION)' 11 | subscriptionId: '$(SUBSCRIPTION_ID)' 12 | action: 'Create Or Update Resource Group' 13 | resourceGroupName: '$(RESOURCE_GROUP)' 14 | location: '$(LOCATION)' 15 | templateLocation: 'Linked artifact' 16 | csmFile: '$(SOURCE_WORKSPACE_NAME)/ARMTemplateForFactory.json' 17 | csmParametersFile: '$(SOURCE_WORKSPACE_NAME)/ARMTemplateParametersForFactory.json' 18 | overrideParameters: '-factoryName $(WORKSPACE_NAME)' 19 | deploymentMode: 'Incremental' -------------------------------------------------------------------------------- /pipelines/deploy-pipeline-syn.yml: -------------------------------------------------------------------------------- 1 | trigger: none 2 | 3 | pool: 4 | vmImage: ubuntu-latest 5 | 6 | steps: 7 | - task: Synapse workspace deployment@2 8 | inputs: 9 | operation: 'deploy' 10 | TemplateFile: '$(SOURCE_WORKSPACE_NAME)/TemplateForWorkspace.json' 11 | ParametersFile: '$(SOURCE_WORKSPACE_NAME)/TemplateParametersForWorkspace.json' 12 | azureSubscription: '$(ARM_CONNECTION)' 13 | ResourceGroupName: '$(RESOURCE_GROUP)' 14 | TargetWorkspaceName: '$(WORKSPACE_NAME)' 15 | DeleteArtifactsNotInTemplate: true 16 | DeployManagedPrivateEndpoints: false 17 | FailOnMissingOverrides: false 18 | Environment: 'prod' 19 | npmpackage: 'prod' -------------------------------------------------------------------------------- /readme_assets/1_verify_workspaces.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/ftatoolkit-analytics-cicd-cli/85094811ab985a65079e9f409c77c21fac55de9a/readme_assets/1_verify_workspaces.png -------------------------------------------------------------------------------- /readme_assets/2_start_setup.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/ftatoolkit-analytics-cicd-cli/85094811ab985a65079e9f409c77c21fac55de9a/readme_assets/2_start_setup.png -------------------------------------------------------------------------------- /readme_assets/3_setup_git_repository.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/ftatoolkit-analytics-cicd-cli/85094811ab985a65079e9f409c77c21fac55de9a/readme_assets/3_setup_git_repository.png -------------------------------------------------------------------------------- /readme_assets/4_setup_git_integration.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/ftatoolkit-analytics-cicd-cli/85094811ab985a65079e9f409c77c21fac55de9a/readme_assets/4_setup_git_integration.png -------------------------------------------------------------------------------- /readme_assets/5_review_pipelines.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/ftatoolkit-analytics-cicd-cli/85094811ab985a65079e9f409c77c21fac55de9a/readme_assets/5_review_pipelines.png -------------------------------------------------------------------------------- /readme_assets/architecture.drawio: -------------------------------------------------------------------------------- 1 | 7Vxbc9o4FP4t+8DM7gOM78BjCCFJp9vNNN1p9ykjbGHU2JZXFgT661eyJLAtc0swJJukMw06kuXj8527RVr2Zby4JiCd/okDGLUsI1i07GHLsvqOx/7nhKUgeF1LEEKCAkEy14R79AtKoiGpMxTArLSQYhxRlJaJPk4S6NMSDRCCn8rLJjgq3zUFIdQI9z6IdOp3FNCppJpefz1xA1E4lbfuWV0xEQO1WD5JNgUBfiqQ7KuWfUkwpuJTvLiEEZedkou4brRhdsUYgQnd54Iv0eLrp0/J30+94WLg49ubsD1sWxKNjC7VE8OACUAOMaFTHOIERFdr6oDgWRJAvq3BRus1nzFOGdFkxJ+Q0qVEE8woZqQpjSM5yzgmyx9s0DY6hrWi/MN37FhqOFzIO4jRsji6gwTFkEIiibowlLIAEkK6TQJiHX/qwoVShNcQs7uQJVtAYAQompfVAkjtClfrVpfeYcRYsQxpCLZSaGkHdtcob5HhGfGhvGoNI/tQYGNNysE9AGjJ8BxEM/kIF0GMEkaaZUyKVSWYQ0IRs4LPYAyjO5whinDCpsaYUhyX0VRrLyIU8jWUa8EAyJHPEOEwDTJK8CO8xBHmoCU44ao0QVGkSC3LNozB1YgrADOWlLMSL0LuVjrg14zATs7qFrA5J3CxFUY5a3kVONx+xxWUp7WZO11FnBZM3DU2g1+C7VCMLOccxrhRmic3ib5b3kLYrmYS2kaOsWOjhm3L0m2LAbBkRpEx8uUt2/SSURmTXsREPBgze/NC/uke0lnKZr6xkPZGbDDzCUrpA7uIX5tymUJyNWd3yHYp1P7mqSGqzPV8xuloIN/PxkIYDJZMQw9kqchHJmjBzbOIVi61nEF30HKHdUihOE9MBhOcUGm6prWmD1EcsseI0Jj9nwPDHy+ECSSAPcqoxFsnm4fHwcWruE2zV4OLo6PS3eIcXoSKe9b0hWcvdjl7cbr95+QvcIHoD7U9+yw2c+VovRMfLAuDvfMg4QJ3O7Gd+ZJzmuDgVoODCo47fLq2kWdYyk+st+rsF2kuCAHLwjJpt5uZrrqtnruVt+p6y3QqFiE4OG60MjRP9hllNIdMCFV3ZhztPAyVDUHzWtUwFKMgEDYHM/QLjPP9jFoPqGvuNoPXXNaqApQ3aRWLrDpXZnRsz3XK3kyMXpzWlDZtm3Z5BzyZZLCZPMTVkL3lwACfaogy6dHmwOQ+M5NRa5+0Q5KOEKFsu1pnWVqEUkGsGKGsxpJ6TwPlkkBAIX9SlMIIsUd/L+i4Vr+Ejmfq+YNpnBSergbPNbdO4xrR2pSduZkJCmcE5Nn4+wGuWi/r1XKvBje7KdxsPYqZGhpsH5RmXARPU0ThfQryFOiJlTMVkKrJuhCt6vhZMhEfgRhFXALfwBTHgK9jW6IkHMiSbOisaV+lFCwNKVZkBfm/I2FTSZXMvm5U1iltytabTTVtxneJje2eGxu9WWF/YLMxGJ0UG7evYaNBE7LqNd3/4Q9Oiz3XKyusXVPh1yVQjtOUWEzdnewrlu1ifr6wzieM56vIdtm+RVm4vd3CKDd7drizPYW2SX82GlXbLHcgVjV4qZ1Z081cCfTosnv1ois5YVsXjWkeQTRfP/9c/swesti6vZkMvvz1/WI8Xb0TLchmCChglBGrpLG8w9693kKHTubjeltevRrmgg5ANl1J/ZA2cJz5gJWSI87sg2RVtn4j3rsZAP8xzEEtBNRJ/qO/BXCG/b5R31E8jk1YRsco/JhmuRdTYyFujXNprN2vV+33ywTwpMgyCm93DlGEhpr+YM3N6IITHySnDys+N74AOA6UZqU4VK3aonerCwxeU9jZNc5N1PHsBtyQ/SkgmWxze//O+BGMwYxOuFdUQ1XvqytZdpqUwFYLfWE0F2yShOPf2YMyrg316498C4Mj2p7INJgvvYHRHPI+QGFelPZ81jbSRXFC3JTPJJjEICrMzQFBgP1mugQoQz/bsc4H6aYlTxIcPulI4zciSJlytmVyrl+JSToFidzSEjTeBWlL7ebklYKrOcQCSyLvtHrUfIYSttmE7a/uJDsYXJnywzyF2zxhEpQZW+3FnmX8iNh2fE9RRbSlOpbWjVc+sV3B0XJ6AkLL6csPrkIz3zWAPhZdoDadIv8xgZlkDyWIIiWf6toCllvXFdgprZtEGNCqcAKUpRFYquWip2j8huIUEwoSqceaZo+UcrOPQr/VTH1ba0cOUHBbdpOho7/T3dinDBUqESnmC3AOI5zGMH8xAJM5IjgRo7fZMDyseNgc9M3KuzizrgF80v68qWd7Gkgnri6VsBxdNvWpcGNdVrMukr6k3LReLJyT1AX1zOtHQDRhNFEzrUB4JUVTPZP6u8ePoumIyL6Nqsl8x2XT4WC+urqpLhR+1E0fddNH3fQa66bDHc5rK5z0sxd3BAczXxyt+P/WTZuy4M2nmvaqm+qCRWN1k7PHW7kmsuGN9dXeWm9Ymuhq46xlNCY7PdDmKQgjDeH8r1TPkLJHSP2pFOQJ0yWVHosMSTB3WFp0OF5mr3pe98xZrdPV8DjL93K2ynfnAW9bpo07T3grt3y8I94vk753Dumvz/i3Suf73dZRvp24FcqdCCkomz6EbxqVA9LVRtO+h/BNp1KnrgPakY/gV1m23G5FAxs4Um/rZe8QphHmjG884/s6XPqdZK95r14Nw+ok2rm8+lmc+nPdig7Lsb/X0zuNS3EramD3Kvhu8ANH+x50/wP2ujS+adw9p4y7W+1ivZKvYHnSLT0zXrDh+i86iOXrP4thX/0H -------------------------------------------------------------------------------- /readme_assets/architecture.drawio.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/ftatoolkit-analytics-cicd-cli/85094811ab985a65079e9f409c77c21fac55de9a/readme_assets/architecture.drawio.png -------------------------------------------------------------------------------- /readme_assets/fta.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/microsoft/ftatoolkit-analytics-cicd-cli/85094811ab985a65079e9f409c77c21fac55de9a/readme_assets/fta.png -------------------------------------------------------------------------------- /setup.sh: -------------------------------------------------------------------------------- 1 | # Copyright (c) Microsoft Corporation. 2 | # Licensed under the MIT License. 3 | 4 | set -e 5 | 6 | source ./partials/interactive_prompts.sh 7 | source ./partials/setup_ado.sh 8 | 9 | echo " 10 | ############################################################ 11 | Welcome to the Analytics CI/CD Setup Tool. 12 | ############################################################ 13 | 14 | Use this to quickly and interactively set up a CI/CD template for your Analytics environment 15 | Before you continue, make sure you: 16 | 17 | - Are logged in with the Azure CLI in the desired Tenant 18 | - Have both a development and a production environment to set up CI/CD between them 19 | - Have configured Git integration in the development environment 20 | - We currently support: 21 | - Azure Data Factory with Azure DevOps 22 | - Azure Synapse Analytics with Azure DevOps 23 | 24 | " 25 | choose_service_interactive # --> $service 26 | echo "Please select the subscription that contains SOURCE workspace." 27 | choose_subscription_interactive # --> $subscription 28 | SOURCE_SUBSCRIPTION=$subscription 29 | echo "Select SOURCE workspace." 30 | choose_workspace_interactive # --> $workspace 31 | SOURCE_WORKSPACE=$workspace 32 | 33 | echo "Select the subscription that contains TARGET workspace." 34 | choose_subscription_interactive # --> $subscription 35 | TARGET_SUBSCRIPTION=$subscription 36 | echo "Select TARGET workspace." 37 | choose_workspace_interactive # --> $workspace 38 | TARGET_WORKSPACE=$workspace 39 | 40 | SOURCE_WORKSPACE_NAME=$(echo $SOURCE_WORKSPACE | jq " .name" -r) 41 | TARGET_WORKSPACE_NAME=$(echo $TARGET_WORKSPACE | jq " .name" -r) 42 | TARGET_RESOURCE_GROUP=$(echo $TARGET_WORKSPACE | jq " .resourceGroup" -r) 43 | 44 | if [[ $(echo $SOURCE_WORKSPACE | jq .repoConfiguration.type -r) == "FactoryVSTSConfiguration" ]]; then 45 | setup_ado adf 46 | elif [[ $(echo $SOURCE_WORKSPACE | jq .workspaceRepositoryConfiguration.type -r) == "WorkspaceVSTSConfiguration" ]]; then 47 | setup_ado syn 48 | else 49 | echo "This repo configuration is not currently supported" 50 | fi --------------------------------------------------------------------------------