├── .github ├── CONTRIBUTING.md ├── ISSUE_TEMPLATE.md └── PULL_REQUEST_TEMPLATE.md ├── .vscode └── settings.json ├── Instructions ├── 02-Explore-Azure-Machine-Learning.md ├── 02-Explore-developer-tools.md ├── 03-Make-data-available.md ├── 04-Work-with-compute.md ├── 04-Work-with-environments.md ├── 05-Designer-train-model.md ├── 06-AutoML-classification-model.md ├── 07-Notebooks-mlflow-tracking.md ├── 08-Script-command-job.md ├── 08-Script-mlflow-tracking.md ├── 09-Hyperparameter-tuning.md ├── 09-Run-pipelines.md ├── 10-Compare-models.md ├── 10-Log-mlflow-models.md ├── 11-Deploy-batch-endpoint.md ├── 11-Deploy-online-endpoint.md └── images │ ├── 03-01-01-setup.png │ ├── 04-01-completed-pipeline.png │ └── 04-02-submitted-jobs.png ├── LICENSE ├── Labs ├── 10 │ ├── Create Responsible AI dashboard.ipynb │ ├── Log models with MLflow.ipynb │ ├── model │ │ ├── MLmodel │ │ ├── conda.yaml │ │ ├── model.pkl │ │ ├── python_env.yaml │ │ └── requirements.txt │ ├── setup.sh │ ├── src │ │ └── diabetes.csv │ ├── test-data │ │ ├── MLTable │ │ └── diabetes-test.csv │ └── train-data │ │ ├── MLTable │ │ └── diabetes.csv ├── 11 │ ├── Deploy to batch endpoint.ipynb │ ├── Deploy to online endpoint.ipynb │ ├── data │ │ ├── patient-data-0.csv │ │ ├── patient-data-1.csv │ │ ├── patient-data-2.csv │ │ └── patient-data-3.csv │ ├── model │ │ ├── MLmodel │ │ ├── conda.yaml │ │ ├── model.pkl │ │ ├── python_env.yaml │ │ └── requirements.txt │ ├── sample-data.json │ └── setup.sh ├── 02 │ ├── Run training script.ipynb │ ├── diabetes-data.zip │ └── src │ │ └── diabetes.csv ├── 03 │ ├── Work with data.ipynb │ ├── data │ │ ├── MLTable │ │ └── diabetes.csv │ └── setup.sh ├── 04 │ ├── Work with compute.ipynb │ ├── Work with environments.ipynb │ ├── images │ │ └── screenshot-logs.png │ ├── setup.sh │ └── src │ │ └── diabetes.csv ├── 05 │ ├── data │ │ └── diabetes.csv │ ├── fix-missing-data.yml │ ├── normalize-data.yml │ ├── setup.sh │ ├── src │ │ ├── fix-missing-data.py │ │ ├── normalize-data.py │ │ ├── train-decision-tree.py │ │ └── train-logistic-regression.py │ ├── train-decision-tree.yml │ └── train-logistic-regression.yml ├── 06 │ ├── Classification with Automated Machine Learning.ipynb │ ├── diabetes-data │ │ ├── MLTable │ │ └── diabetes.csv │ ├── orange-juice-data │ │ ├── MLTable │ │ └── orange-juice-data.csv │ └── setup.sh ├── 07 │ ├── Track model training with MLflow.ipynb │ ├── data │ │ └── diabetes.csv │ ├── images │ │ └── output-logs.png │ └── setup.sh ├── 08 │ ├── Run script as a command job.ipynb │ ├── Use MLflow to track jobs.ipynb │ ├── setup.sh │ └── src │ │ ├── Train classification model.ipynb │ │ ├── diabetes.csv │ │ └── train-model-parameters.py └── 09 │ ├── Hyperparameter tuning.ipynb │ ├── Run a pipeline job.ipynb │ ├── data │ └── diabetes.csv │ └── setup.sh ├── Policies ├── DP-100 Base + LP11.json ├── DP-100 Base.json └── Required Resource Providers.md ├── _build.yml ├── _config.yml ├── index.md ├── readme.md └── typings └── __builtins__.pyi /.github/CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing to Microsoft Learning Repositories 2 | 3 | MCT contributions are a key part of keeping the lab and demo content current as the Azure platform changes. We want to make it as easy as possible for you to contribute changes to the lab files. Here are a few guidelines to keep in mind as you contribute changes. 4 | 5 | ## GitHub Use & Purpose 6 | 7 | Microsoft Learning is using GitHub to publish the lab steps and lab scripts for courses that cover cloud services like Azure. Using GitHub allows the course’s authors and MCTs to keep the lab content current with Azure platform changes. Using GitHub allows the MCTs to provide feedback and suggestions for lab changes, and then the course authors can update lab steps and scripts quickly and relatively easily. 8 | 9 | > When you prepare to teach these courses, you should ensure that you are using the latest lab steps and scripts by downloading the appropriate files from GitHub. GitHub should not be used to discuss technical content in the course, or how to prep. It should only be used to address changes in the labs. 10 | 11 | It is strongly recommended that MCTs and Partners access these materials and in turn, provide them separately to students. Pointing students directly to GitHub to access Lab steps as part of an ongoing class will require them to access yet another UI as part of the course, contributing to a confusing experience for the student. An explanation to the student regarding why they are receiving separate Lab instructions can highlight the nature of an always-changing cloud-based interface and platform. Microsoft Learning support for accessing files on GitHub and support for navigation of the GitHub site is limited to MCTs teaching this course only. 12 | 13 | > As an alternative to pointing students directly to the GitHub repository, you can point students to the GitHub Pages website to view the lab instructions. The URL for the GitHub Pages website can be found at the top of the repository. 14 | 15 | To address general comments about the course and demos, or how to prepare for a course delivery, please use the existing MCT forums. 16 | 17 | ## Additional Resources 18 | 19 | A user guide has been provided for MCTs who are new to GitHub. It provides steps for connecting to GitHub, downloading and printing course materials, updating the scripts that students use in labs, and explaining how you can help ensure that this course’s content remains current. 20 | 21 | 22 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | # Module: 00 2 | ## Lab/Demo: 00 3 | ### Task: 00 4 | #### Step: 00 5 | 6 | Description of issue 7 | 8 | Repro steps: 9 | 10 | 1. 11 | 1. 12 | 1. -------------------------------------------------------------------------------- /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | # Module: 00 2 | ## Lab/Demo: 00 3 | 4 | Fixes # . 5 | 6 | Changes proposed in this pull request: 7 | 8 | - 9 | - 10 | - -------------------------------------------------------------------------------- /.vscode/settings.json: -------------------------------------------------------------------------------- 1 | { 2 | "python.envFile": "${workspaceFolder}\\.databricks\\.databricks.env", 3 | "databricks.python.envFile": "${workspaceFolder}/.env" 4 | } -------------------------------------------------------------------------------- /Instructions/02-Explore-Azure-Machine-Learning.md: -------------------------------------------------------------------------------- 1 | --- 2 | lab: 3 | title: 'Explore the Azure Machine Learning workspace' 4 | --- 5 | 6 | # Explore the Azure Machine Learning workspace 7 | 8 | Azure Machine Learning provides a data science platform to train and manage machine learning models. In this lab, you'll create an Azure Machine Learning workspace and explore the various ways to work with the workspace. The lab is designed as an introduction of the various core capabilities of Azure Machine Learning and the developer tools. If you want to learn about the capabilities in more depth, there are other labs to explore. 9 | 10 | ## Before you start 11 | 12 | You'll need an [Azure subscription](https://azure.microsoft.com/free?azure-portal=true) in which you have administrative-level access. 13 | 14 | ## Provision an Azure Machine Learning workspace 15 | 16 | An Azure Machine Learning **workspace** provides a central place for managing all resources and assets you need to train and manage your models. You can provision a workspace using the interactive interface in the Azure portal, or you can use the Azure CLI with the Azure Machine Learning extension. In most production scenarios, it's best to automate provisioning with the CLI so that you can incorporate resource deployment into a repeatable development and operations (*DevOps*) process. 17 | 18 | In this exercise, you'll use the Azure portal to provision Azure Machine Learning to explore all options. 19 | 20 | 1. Sign into the `https://portal.azure.com/`. 21 | 2. Create a new **Azure Machine Learning** resource with the following settings: 22 | - **Subscription**: *Your Azure subscription* 23 | - **Resource group**: `rg-dp100-labs` 24 | - **Workspace name**: `mlw-dp100-labs` 25 | - **Region**: *Select the geographical region closest to you* 26 | - **Storage account**: *Note the default new storage account that will be created for your workspace* 27 | - **Key vault**: *Note the default new key vault that will be created for your workspace* 28 | - **Application insights**: *Note the default new application insights resource that will be created for your workspace* 29 | - **Container registry**: None (*one will be created automatically the first time you deploy a model to a container*) 30 | 3. Wait for the workspace and its associated resources to be created - this typically takes around 5 minutes. 31 | 32 | > **Note**: When you create an Azure Machine Learning workspace, you can use some advanced options to restrict access through a *private endpoint* and specify custom keys for data encryption. We won't use these options in this exercise - but you should be aware of them! 33 | 34 | ## Explore the Azure Machine Learning studio 35 | 36 | *Azure Machine Learning studio* is a web-based portal through which you can access the Azure Machine Learning workspace. You can use the Azure Machine Learning studio to manage all assets and resources within your workspace. 37 | 38 | 1. Go to the resource group named **rg-dp100-labs**. 39 | 1. Confirm that the resource group contains your Azure Machine Learning workspace, an Application Insights, a Key Vault, and a Storage Account. 40 | 1. Select your Azure Machine Learning workspace. 41 | 1. Select **Launch studio** from the **Overview** page. Another tab will open in your browser to open the Azure Machine Learning studio. 42 | 1. Close any pop-ups that appear in the studio. 43 | 1. Note the different pages shown on the left side of the studio. If only the symbols are visible in the menu, select the ☰ icon to expand the menu and explore the names of the pages. 44 | 1. Note the **Authoring** section, which includes **Notebooks**, **Automated ML**, and **Designer**. These are the three ways you can create your own machine learning models within the Azure Machine Learning studio. 45 | 1. Note the **Assets** section, which includes **Data**, **Jobs**, and **Models** among other things. Assets are either consumed or created when training or scoring a model. Assets are used to train, deploy, and manage your models and can be versioned to keep track of your history. 46 | 1. Note the **Manage** section, which includes **Compute** among other things. These are infrastructural resources needed to train or deploy a machine learning model. 47 | 48 | ## Train a model using AutoML 49 | 50 | To explore the use of the assets and resources in the Azure Machine Learning workspace, let's try and train a model. 51 | 52 | A quick way to train and find the best model for a task based on your data is by using the **AutoML** option. 53 | 54 | > **Note**: Pop-ups may appear throughout to guide you through the studio. You can close and ignore all pop-ups and focus on the instructions of this lab. 55 | 56 | 1. Download the training data that will be used at `https://github.com/MicrosoftLearning/mslearn-azure-ml/raw/refs/heads/main/Labs/02/diabetes-data.zip` and extract the compressed files. 57 | 1. Back in the Azure Machine Learning studio, select the **AutoML** page from the menu on the left side of the studio. 58 | 1. Select **+ New Automated ML job**. 59 | 1. In the **Basic settings** step, give a unique name to your training job and experiment or use the default values assigned. Select **Next**. 60 | 1. In the **Task type & data** step, select **Classification** as the task type, and select **+ Create** to add your training data. 61 | 2. On the **Create data asset** page, in the **Data type** step, give a name to your data asset (e.g `training-data`) and select **Next**. 62 | 1. In the **Data source** step, select **From local files** to upload the training data you download previously. Select **Next**. 63 | 1. In the **Destination storage type** step, verify that **Azure Blob Storage** is selected as the datastore type and that **workspaceblobstore** is the datastore selected. Select **Next**. 64 | 1. In the **MLTable selection** step, select **Upload folder** and select the folder you extracted from the compressed file downloaded earlier. Select **Next**. 65 | 1. Review the settings for your data asset and select **Create**. 66 | 1. Back in the **Task type & data** step, select the data you just uploaded and select **Next**. 67 | 68 | > **Tip**: You may need to select the **Classification** task type again before moving to the next step. 69 | 70 | 1. In the **Task settings** step, select **Diabetic (Boolean)** as your target column, then open the **View additional configuration settings** option. 71 | 1. In the **Additional configuration** pane, change the primary metric to **Accuracy**, then select **Save**. 72 | 1. Expand the **Limits** option and set the following properties: 73 | * **Max trials**: 10 74 | * **Experiment timeout (minutes)**: 60 75 | * **Iteration timeout (minutes)**: 15 76 | * **Enable early termination**: Checked 77 | 78 | 1. For **Test data**, select **Train-test split** and verify that the **Percentage test of data** is 10. Select **Next**. 79 | 1. In the **Compute** step, verify that the compute type is **Serveless** and the virtual machine size selected is **Standard_DS3_v2**. Select **Next**. 80 | 81 | > **Note**: Compute instances and clusters are based on standard Azure virtual machine images. For this exercise, the *Standard_DS3_v2* image is recommended to achieve the optimal balance of cost and performance. If your subscription has a quota that does not include this image, choose an alternative image; but bear in mind that a larger image may incur higher cost and a smaller image may not be sufficient to complete the tasks. Alternatively, ask your Azure administrator to extend your quota. 82 | 83 | 1. Review all your settings and select **Submit training job**. 84 | 85 | ## Use jobs to view your history 86 | 87 | After submitting the job, you'll be redirected to the job's page. Jobs allow you to keep track of the workloads you ran and compare them with each other. Jobs belong to an **experiment**, which allows you to group job runs together. 88 | 89 | 1. Note that in the **Overview** parameters, you can find the job's status, who created it, when it was created and how long it took to run (among other things). 90 | 1. It should take 10-20 minutes for the training job to finish. When it is completed, you can also view the details of each individual component run, including the output. Feel free to explore the job page to understand how the models are trained. 91 | 92 | Azure Machine Learning automatically keeps track of your job's properties. By using jobs, you can easily view your history to understand what you or your colleagues have already done. 93 | During experimentation, jobs help keep track of the different models you train to compare and identify the best model. During production, jobs allow you to check whether automated workloads ran as expected. 94 | 95 | ## Delete Azure resources 96 | 97 | When you finish exploring Azure Machine Learning, you should delete the resources you've created to avoid unnecessary Azure costs. 98 | 99 | 1. Close the Azure Machine Learning studio tab and return to the Azure portal. 100 | 1. In the Azure portal, on the **Home** page, select **Resource groups**. 101 | 1. Select the **rg-dp100-labs** resource group. 102 | 1. At the top of the **Overview** page for your resource group, select **Delete resource group**. 103 | 1. Enter the resource group name to confirm you want to delete it, and select **Delete**. 104 | -------------------------------------------------------------------------------- /Instructions/03-Make-data-available.md: -------------------------------------------------------------------------------- 1 | --- 2 | lab: 3 | title: 'Make data available in Azure Machine Learning' 4 | --- 5 | 6 | # Make data available in Azure Machine Learning 7 | 8 | Although it's fairly common to work with data on their local file system, in an enterprise environment it can be more effective to store the data in a central location where multiple data scientists and machine learning engineers can access it. 9 | 10 | In this exercise, you'll explore *datastores* and *data assets*, which are the primary objects used to abstract data access in Azure Machine Learning. 11 | 12 | ## Before you start 13 | 14 | You'll need an [Azure subscription](https://azure.microsoft.com/free?azure-portal=true) in which you have administrative-level access. 15 | 16 | ## Provision an Azure Machine Learning workspace 17 | 18 | An Azure Machine Learning *workspace* provides a central place for managing all resources and assets you need to train and manage your models. You can interact with the Azure Machine Learning workspace through the studio, Python SDK, and Azure CLI. 19 | 20 | You'll use a Shell script which uses the Azure CLI to provision the workspace and necessary resources. Next, you'll use the Designer in the Azure Machine Learning studio to train and compare models. 21 | 22 | ### Create the workspace and compute resources 23 | 24 | To create the Azure Machine Learning workspace and compute resources, you'll use the Azure CLI. All necessary commands are grouped in a Shell script for you to execute. 25 | 26 | 1. In a browser, open the Azure portal at `https://portal.azure.com/`, signing in with your Microsoft account. 27 | 1. Select the \[>_] (*Cloud Shell*) button at the top of the page to the right of the search box. This opens a Cloud Shell pane at the bottom of the portal. 28 | 1. Select **Bash** if asked. The first time you open the cloud shell, you will be asked to choose the type of shell you want to use (*Bash* or *PowerShell*). 29 | 1. Check that the correct subscription is specified and that **No storage account required** is selected. Select **Apply**. 30 | 1. Enter the following commands in the terminal to clone this repo: 31 | 32 | ```azurecli 33 | rm -r azure-ml-labs -f 34 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 35 | ``` 36 | 37 | > Use `SHIFT + INSERT` to paste your copied code into the Cloud Shell. 38 | 39 | 1. Enter the following commands after the repo has been cloned, to change to the folder for this lab and run the **setup.sh** script it contains: 40 | 41 | ```azurecli 42 | cd azure-ml-labs/Labs/03 43 | ./setup.sh 44 | ``` 45 | 46 | > Ignore any (error) messages that say that the extensions were not installed. 47 | 48 | 1. Wait for the script to complete - this typically takes around 5-10 minutes. 49 | 50 |
51 | Troubleshooting tip: Workspace creation error
52 |

If you receive an error when running the setup script through the CLI, you need to provision the resources manually:

53 |
    54 |
  1. In the Azure portal home page, select + Create a resource.
    2. 55 |
  2. Search for machine learning and then select Azure Machine Learning. Select Create.
    3. 56 |
  3. Create a new Azure Machine Learning resource with the following settings: 57 |
      58 |
    • Subscription: Your Azure subscription
      • 59 |
    • Resource group: rg-dp100-labs
      • 60 |
    • Workspace name: mlw-dp100-labs
      • 61 |
    • Region: Select the geographical region closest to you
      • 62 |
    • Storage account: Note the default new storage account that will be created for your workspace
      • 63 |
    • Key vault: Note the default new key vault that will be created for your workspace
      • 64 |
    • Application insights: Note the default new application insights resource that will be created for your workspace
      • 65 |
    • Container registry: None (one will be created automatically the first time you deploy a model to a container)
      • 66 |
    67 |
  4. Select Review + create and wait for the workspace and its associated resources to be created - this typically takes around 5 minutes.
    5. 68 |
  5. Select Go to resource and in its Overview page, select Launch studio. Another tab will open in your browser to open the Azure Machine Learning studio.
    6. 69 |
  6. Close any pop-ups that appear in the studio.
    7. 70 |
  7. Within the Azure Machine Learning studio, navigate to the Compute page and select + New under the Compute instances tab.
    8. 71 |
  8. Give the compute instance a unique name and then select Standard_DS11_v2 as the virtual machine size.
    9. 72 |
  9. Select Review + create and then select Create.
    10. 73 |
  10. Next, select the Compute clusters tab and select + New.
    11. 74 |
  11. Choose the same region as the one where you created your workspace and then select Standard_DS11_v2 as the virtual machine size. Select Next
    12. 75 |
  12. Give the cluster a unique name and then select Create.
    13. 76 |
77 |
78 | 79 | ## Explore the default datastores 80 | 81 | When you create an Azure Machine Learning workspace, a Storage Account is automatically created and connected to your workspace. You'll explore how the Storage Account is connected. 82 | 83 | 1. In the Azure portal, navigate to the new resource group named **rg-dp100-...**. 84 | 1. Select the Storage Account in the resource group. The name often starts with the name you provided for the workspace (without hyphens). 85 | 1. Review the **Overview** page of the Storage Account. Note that the Storage Account has several options for **Data storage** as shown in the Overview pane, and in the left menu. 86 | 1. Select **Containers** to explore the Blob storage part of the Storage Account. 87 | 1. Note the **azureml-blobstore-...** container. The default datastore for data assets uses this container to store data. 88 | 1. Using the + **Container** button at the top of the screen, create a new container and name it `training-data`. 89 | 1. Select **File shares** from the left menu to explore the File share part of the Storage Account. 90 | 1. Note the **code-...** file share. Any notebooks in the workspace are stored here. After cloning the lab materials, you can find the files in this file share, in the **code-.../Users/*your-user-name*/azure-ml-labs** folder. 91 | 92 | ## Copy the access key 93 | 94 | To create a datastore in the Azure Machine Learning workspace, you need to provide some credentials. An easy way to provide the workspace with access to a Blob storage is to use the account key. 95 | 96 | 1. In the Storage Account, select the **Access keys** tab from the left menu. 97 | 1. Note that two keys are provided: key1 and key2. Each key has the same functionality. 98 | 1. Select **Show** for the **Key** field under **key1**. 99 | 1. Copy the value of the **Key** field to a notepad. You'll need to paste this value into the notebook later. 100 | 1. Copy the name of your storage account from the top of the page. The name should start with **mlwdp100storage...** You'll need to paste this value into the notebook later too. 101 | 102 | > **Note**: 103 | > Copy the account key and name to a notepad to avoid automatic capitalization (which happens in Word). The key is case-sensitive. 104 | 105 | ## Clone the lab materials 106 | 107 | To create a datastore and data assets with the Python SDK, you'll need to clone the lab materials into the workspace. 108 | 109 | 1. In the Azure portal, navigate to the Azure Machine Learning workspace named **mlw-dp100-labs**. 110 | 1. Select the Azure Machine Learning workspace, and in its **Overview** page, select **Launch studio**. Another tab will open in your browser to open the Azure Machine Learning studio. 111 | 1. Close any pop-ups that appear in the studio. 112 | 1. Within the Azure Machine Learning studio, navigate to the **Compute** page and verify that the compute instance and cluster you created in the previous section exist. The compute instance should be running, the cluster should be idle and have 0 nodes running. 113 | 1. In the **Compute instances** tab, find your compute instance, and select the **Terminal** application. 114 | 1. In the terminal, install the Python SDK on the compute instance by running the following commands in the terminal: 115 | 116 | ```azurecli 117 | pip uninstall azure-ai-ml 118 | pip install azure-ai-ml 119 | pip install mltable 120 | ``` 121 | 122 | > Ignore any (error) messages that say that the packages were not installed. 123 | 124 | 1. Run the following command to clone a Git repository containing notebooks, data, and other files to your workspace: 125 | 126 | ```azurecli 127 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 128 | ``` 129 | 130 | 1. When the command has completed, in the **Files** pane, click **↻** to refresh the view and verify that a new **Users/*your-user-name*/azure-ml-labs** folder has been created. 131 | 132 | **Optionally**, in another browser tab, navigate back to the [Azure portal](https://portal.azure.com?azure-portal=true). Explore the file share **code-...** in the Storage account again to find the cloned lab materials in the newly created **azure-ml-labs** folder. 133 | 134 | ## Create a datastore and data assets 135 | 136 | The code to create a datastore and data assets with the Python SDK is provided in a notebook. 137 | 138 | 1. Open the **Labs/03/Work with data.ipynb** notebook. 139 | 140 | > Select **Authenticate** and follow the necessary steps if a notification appears asking you to authenticate. 141 | 142 | 1. Verify that the notebook uses the **Python 3.10 - AzureML** kernel. 143 | 1. Run all cells in the notebook. 144 | 145 | ## Optional: Explore the data assets 146 | 147 | **Optionally**, you can explore how the data assets are stored in the associated Storage Account. 148 | 149 | 1. Navigate to the **Data** tab in the Azure Machine Learning studio to explore the data assets. 150 | 1. Select the **diabetes-local** data asset name to explore its details. 151 | 152 | Under **Data sources** for the **diabetes-local** data asset, you'll find where the file has been uploaded to. The path starting with **LocalUpload/...** shows the path within the Storage Account container **azureml-blobstore-...**. You can verify the file exists by navigating to that path in the Azure portal. 153 | 154 | ## Delete Azure resources 155 | 156 | When you finish exploring Azure Machine Learning, you should delete the resources you've created to avoid unnecessary Azure costs. 157 | 158 | 1. Close the Azure Machine Learning studio tab and return to the Azure portal. 159 | 1. In the Azure portal, on the **Home** page, select **Resource groups**. 160 | 1. Select the **rg-dp100-...** resource group. 161 | 1. At the top of the **Overview** page for your resource group, select **Delete resource group**. 162 | 1. Enter the resource group name to confirm you want to delete it, and select **Delete**. 163 | -------------------------------------------------------------------------------- /Instructions/04-Work-with-compute.md: -------------------------------------------------------------------------------- 1 | --- 2 | lab: 3 | title: 'Work with compute resources in Azure Machine Learning' 4 | --- 5 | 6 | # Work with compute resources in Azure Machine Learning 7 | 8 | One of the key benefits of the cloud is the ability to use scalable, on-demand compute resources for cost-effective processing of large data. 9 | 10 | In this exercise, you'll learn how to use cloud compute in Azure Machine Learning to run experiments and production code at scale. 11 | 12 | ## Before you start 13 | 14 | You'll need an [Azure subscription](https://azure.microsoft.com/free?azure-portal=true) in which you have administrative-level access. 15 | 16 | ## Provision an Azure Machine Learning workspace 17 | 18 | An Azure Machine Learning *workspace* provides a central place for managing all resources and assets you need to train and manage your models. You can interact with the Azure Machine Learning workspace through the studio, Python SDK, and Azure CLI. 19 | 20 | To create the Azure Machine Learning workspace, you'll use the Azure CLI. All necessary commands are grouped in a Shell script for you to execute. 21 | 22 | 1. In a browser, open the Azure portal at `https://portal.azure.com/`, signing in with your Microsoft account. 23 | 1. Select the \[>_] (*Cloud Shell*) button at the top of the page to the right of the search box. This opens a Cloud Shell pane at the bottom of the portal. 24 | 1. Select **Bash** if asked. The first time you open the cloud shell, you will be asked to choose the type of shell you want to use (*Bash* or *PowerShell*). 25 | 1. Check that the correct subscription is specified and that **No storage account required** is selected. Select **Apply**. 26 | 1. To avoid any conflicts with previous versions, remove any ML CLI extensions (both version 1 and 2) by running this command in the terminal: 27 | 28 | ```azurecli 29 | az extension remove -n azure-cli-ml 30 | az extension remove -n ml 31 | ``` 32 | 33 | > Use `SHIFT + INSERT` to paste your copied code into the Cloud Shell. 34 | 35 | > Ignore any (error) messages that say that the extensions were not installed. 36 | 37 | 1. Install the Azure Machine Learning (v2) extension with the following command: 38 | 39 | ```azurecli 40 | az extension add -n ml -y 41 | ``` 42 | 43 | 1. Create a resource group. Choose a location close to you. 44 | 45 | ```azurecli 46 | az group create --name "rg-dp100-labs" --location "eastus" 47 | ``` 48 | 49 | 1. Create a workspace: 50 | 51 | ```azurecli 52 | az ml workspace create --name "mlw-dp100-labs" -g "rg-dp100-labs" 53 | ``` 54 | 55 | 1. Wait for the command to complete - this typically takes around 5-10 minutes. 56 | 57 |
58 | Troubleshooting tip: Workspace creation error
59 |

If you receive an error when creating a workspace through the CLI, you need to provision the resource manually:

60 |
    61 |
  1. In the Azure portal home page, select + Create a resource.
    2. 62 |
  2. Search for machine learning and then select Azure Machine Learning. Select Create.
    3. 63 |
  3. Create a new Azure Machine Learning resource with the following settings: 64 |
      65 |
    • Subscription: Your Azure subscription
      • 66 |
    • Resource group: rg-dp100-labs
      • 67 |
    • Workspace name: mlw-dp100-labs
      • 68 |
    • Region: Select the geographical region closest to you
      • 69 |
    • Storage account: Note the default new storage account that will be created for your workspace
      • 70 |
    • Key vault: Note the default new key vault that will be created for your workspace
      • 71 |
    • Application insights: Note the default new application insights resource that will be created for your workspace
      • 72 |
    • Container registry: None (one will be created automatically the first time you deploy a model to a container)
      • 73 |
    74 |
  4. Select Review + create and wait for the workspace and its associated resources to be created - this typically takes around 5 minutes.
    5. 75 |
76 |
77 | 78 | ## Create the compute setup script 79 | 80 | To run notebooks within the Azure Machine Learning workspace, you'll need a compute instance. You can use a setup script to configure the compute instance on creation. 81 | 82 | 1. In the Azure portal, navigate to the Azure Machine Learning workspace named **mlw-dp100-labs**. 83 | 1. Select the Azure Machine Learning workspace, and in its **Overview** page, select **Launch studio**. Another tab will open in your browser to open the Azure Machine Learning studio. 84 | 1. Close any pop-ups that appear in the studio. 85 | 1. Within the Azure Machine Learning studio, navigate to the **Notebooks** page. 86 | 1. In the **Files** pane, select the ⨁ icon to **Add files**. 87 | 1. Select **Create new file**. 88 | 1. Verify that the file location is **Users/*your-user-name***. 89 | 1. Change the file type to **Bash (*.sh)**. 90 | 1. Change the file name to `compute-setup.sh`. 91 | 1. Open the newly created **compute-setup.sh** file and paste the following to its content: 92 | 93 | ```azurecli 94 | #!/bin/bash 95 | 96 | # clone repository 97 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 98 | ``` 99 | 100 | 1. Save the **compute-setup.sh** file. 101 | 102 | ## Create the compute instance 103 | 104 | To create the compute instance, you can use the studio, Python SDK, or Azure CLI. You'll use the studio to create the compute instance with the setup script you just created. 105 | 106 | 1. Navigate to the **Compute** page, using the menu on the left. 107 | 1. In the **Compute instances** tab, select **New**. 108 | 1. Configure (don't create yet) the compute instance with the following settings: 109 | - **Compute name**: *enter a unique name* 110 | - **Virtual machine type**: *CPU* 111 | - **Virtual machine size**: *Standard_DS11_v2* 112 | 1. Select **Next**. 113 | 1. Select **Add schedule** and configure the schedule to **stop** the compute instance every day at **18:00** or **6:00 PM**. 114 | 1. Select **Next**. 115 | 1. Review the security settings but do **not** select them: 116 | - **Enable SSH access**: *You can use this to enable direct access to the virtual machine using an SSH client.* 117 | - **Enable virtual network**: *You would typically use this in an enterprise environment to enhance network security.* 118 | - **Assign to another user**: *You can use this to assign a compute instance to another data scientist.* 119 | 1. Select **Next**. 120 | 1. Select the toggle for **Provision with a creation script**. 121 | 1. Select the **compute-setup.sh** script you created previously. 122 | 1. Select **Review + Create** to create the compute instance and wait for it to start and its state to change to **Running**. 123 | 1. When the compute instance is running, navigate to the **Notebooks** page. In the **Files** pane, click **↻** to refresh the view and verify that a new **Users/*your-user-name*/dp100-azure-ml-labs** folder has been created. 124 | 125 | ## Configure the compute instance 126 | 127 | When you've created the compute instance, you can run notebooks on it. You may need to install certain packages to run the code you want. You can include packages in the setup script, or install them using the terminal. 128 | 129 | 1. In the **Compute instances** tab, find your compute instance, and select the **Terminal** application. 130 | 1. In the terminal, install the Python SDK on the compute instance by running the following commands in the terminal: 131 | 132 | ``` 133 | pip uninstall azure-ai-ml 134 | pip install azure-ai-ml 135 | ``` 136 | 137 | > Ignore any (error) messages that say that the packages were not installed. 138 | 139 | 1. When the packages are installed, you can close the tab to terminate the terminal. 140 | 141 | ## Create a compute cluster 142 | 143 | Notebooks are ideal for development or iterative work during experimentation. When experimenting, you'll want to run notebooks on a compute instance to quickly test and review code. When moving to production, you'll want to run scripts on a compute cluster. You'll create a compute cluster with the Python SDK, and then use it to run a script as a job. 144 | 145 | 1. Open the **Labs/04/Work with compute.ipynb** notebook. 146 | 147 | > Select **Authenticate** and follow the necessary steps if a notification appears asking you to authenticate. 148 | 149 | 1. Verify that the notebook uses the **Python 3.10 - AzureML** kernel. 150 | 1. Run all cells in the notebook. 151 | 152 | ## Delete Azure resources 153 | 154 | When you finish exploring Azure Machine Learning, you should delete the resources you've created to avoid unnecessary Azure costs. 155 | 156 | 1. Close the Azure Machine Learning studio tab and return to the Azure portal. 157 | 1. In the Azure portal, on the **Home** page, select **Resource groups**. 158 | 1. Select the **rg-dp100-labs** resource group. 159 | 1. At the top of the **Overview** page for your resource group, select **Delete resource group**. 160 | 1. Enter the resource group name to confirm you want to delete it, and select **Delete**. 161 | -------------------------------------------------------------------------------- /Instructions/04-Work-with-environments.md: -------------------------------------------------------------------------------- 1 | --- 2 | lab: 3 | title: 'Work with environments in Azure Machine Learning' 4 | --- 5 | 6 | # Work with environments in Azure Machine Learning 7 | 8 | To run notebooks and scripts, you must ensure that the required packages are installed. Environments allow you to specify the runtimes and Python packages that must be used by your compute to run your code. 9 | 10 | In this exercise, you will learn about environments and how to use them when training machine learning models with Azure Machine Learning compute. 11 | 12 | ## Before you start 13 | 14 | You'll need an [Azure subscription](https://azure.microsoft.com/free?azure-portal=true) in which you have administrative-level access. 15 | 16 | ## Provision an Azure Machine Learning workspace 17 | 18 | An Azure Machine Learning *workspace* provides a central place for managing all resources and assets you need to train and manage your models. You can interact with the Azure Machine Learning workspace through the studio, Python SDK, and Azure CLI. 19 | 20 | You'll use the Azure CLI to provision the workspace and necessary compute, and you'll use the Python SDK to train a classification model with Automated Machine Learning. 21 | 22 | ### Create the workspace and compute resources 23 | 24 | To create the Azure Machine Learning workspace and compute resources, you'll use the Azure CLI. All necessary commands are grouped in a Shell script for you to execute. 25 | 26 | 1. In a browser, open the Azure portal at `https://portal.azure.com/`, signing in with your Microsoft account. 27 | 1. Select the \[>_] (*Cloud Shell*) button at the top of the page to the right of the search box. This opens a Cloud Shell pane at the bottom of the portal. 28 | 1. Select **Bash** if asked. The first time you open the cloud shell, you will be asked to choose the type of shell you want to use (*Bash* or *PowerShell*). 29 | 1. Check that the correct subscription is specified and that **No storage account required** is selected. Select **Apply**. 30 | 1. In the terminal, enter the following commands to clone this repo: 31 | 32 | ```azurecli 33 | rm -r azure-ml-labs -f 34 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 35 | ``` 36 | 37 | > Use `SHIFT + INSERT` to paste your copied code into the Cloud Shell. 38 | 39 | 1. After the repo has been cloned, enter the following commands to change to the folder for this lab and run the **setup.sh** script it contains: 40 | 41 | ```azurecli 42 | cd azure-ml-labs/Labs/04 43 | ./setup.sh 44 | ``` 45 | 46 | > Ignore any (error) messages that say that the extensions were not installed. 47 | 48 | 1. Wait for the script to complete - this typically takes around 5-10 minutes. 49 | 50 |
51 | Troubleshooting tip: Workspace creation error
52 |

If you receive an error when running the setup script through the CLI, you need to provision the resources manually:

53 |
    54 |
  1. In the Azure portal home page, select + Create a resource.
    2. 55 |
  2. Search for machine learning and then select Azure Machine Learning. Select Create.
    3. 56 |
  3. Create a new Azure Machine Learning resource with the following settings: 57 |
      58 |
    • Subscription: Your Azure subscription
      • 59 |
    • Resource group: rg-dp100-labs
      • 60 |
    • Workspace name: mlw-dp100-labs
      • 61 |
    • Region: Select the geographical region closest to you
      • 62 |
    • Storage account: Note the default new storage account that will be created for your workspace
      • 63 |
    • Key vault: Note the default new key vault that will be created for your workspace
      • 64 |
    • Application insights: Note the default new application insights resource that will be created for your workspace
      • 65 |
    • Container registry: None (one will be created automatically the first time you deploy a model to a container)
      • 66 |
    67 |
  4. Select Review + create and wait for the workspace and its associated resources to be created - this typically takes around 5 minutes.
    5. 68 |
  5. Select Go to resource and in its Overview page, select Launch studio. Another tab will open in your browser to open the Azure Machine Learning studio.
    6. 69 |
  6. Close any pop-ups that appear in the studio.
    7. 70 |
  7. Within the Azure Machine Learning studio, navigate to the Compute page and select + New under the Compute instances tab.
    8. 71 |
  8. Give the compute instance a unique name and then select Standard_DS11_v2 as the virtual machine size.
    9. 72 |
  9. Select Review + create and then select Create.
    10. 73 |
  10. Next, select the Compute clusters tab and select + New.
    11. 74 |
  11. Choose the same region as the one where you created your workspace and then select Standard_DS11_v2 as the virtual machine size. Select Next
    12. 75 |
  12. Give the cluster a unique name and then select Create.
    13. 76 |
77 |
78 | 79 | ## Clone the lab materials 80 | 81 | When you've created the workspace and necessary compute resources, you can open the Azure Machine Learning studio and clone the lab materials. 82 | 83 | 1. In the Azure portal, navigate to the Azure Machine Learning workspace named **mlw-dp100-...**. 84 | 1. Select the Azure Machine Learning workspace, and in its **Overview** page, select **Launch studio**. Another tab will open in your browser to open the Azure Machine Learning studio. 85 | 1. Close any pop-ups that appear in the studio. 86 | 1. Within the Azure Machine Learning studio, navigate to the **Compute** page and verify that the compute instance and cluster you created in the previous section exist. The compute instance should be running, the cluster should be idle and have 0 nodes running. 87 | 1. In the **Compute instances** tab, find your compute instance, and select the **Terminal** application. 88 | 1. In the terminal, install the Python SDK on the compute instance by running the following commands in the terminal: 89 | 90 | ``` 91 | pip uninstall azure-ai-ml 92 | pip install azure-ai-ml 93 | ``` 94 | 95 | > Ignore any (error) messages that say that the packages couldn't be found and uninstalled. 96 | 97 | 1. Run the following command to clone a Git repository containing notebooks, data, and other files to your workspace: 98 | 99 | ``` 100 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 101 | ``` 102 | 103 | 1. When the command has completed, in the **Files** pane, click **↻** to refresh the view and verify that a new **Users/*your-user-name*/azure-ml-labs** folder has been created. 104 | 105 | ## Work with environments 106 | 107 | The code to create and manage environments with the Python SDK is provided in a notebook. 108 | 109 | 1. Open the **Labs/04/Work with environments.ipynb** notebook. 110 | 111 | > Select **Authenticate** and follow the necessary steps if a notification appears asking you to authenticate. 112 | 113 | 1. Verify that the notebook uses the **Python 3.10 - AzureML** kernel. 114 | 1. Run all cells in the notebook. 115 | 116 | ## Delete Azure resources 117 | 118 | When you finish exploring Azure Machine Learning, you should delete the resources you've created to avoid unnecessary Azure costs. 119 | 120 | 1. Close the Azure Machine Learning studio tab and return to the Azure portal. 121 | 1. In the Azure portal, on the **Home** page, select **Resource groups**. 122 | 1. Select the **rg-dp100-...** resource group. 123 | 1. At the top of the **Overview** page for your resource group, select **Delete resource group**. 124 | 1. Enter the resource group name to confirm you want to delete it, and select **Delete**. 125 | -------------------------------------------------------------------------------- /Instructions/05-Designer-train-model.md: -------------------------------------------------------------------------------- 1 | --- 2 | lab: 3 | title: 'Train a model with the Azure Machine Learning Designer' 4 | --- 5 | 6 | # Train a model with the Azure Machine Learning Designer 7 | 8 | Azure Machine Learning Designer provides a drag and drop interface with which you can define a workflow. You can create a workflow to train a model, testing and comparing multiple algorithms with ease. 9 | 10 | In this exercise, you'll use the Designer to quickly train and compare two classification algorithms. 11 | 12 | ## Before you start 13 | 14 | You'll need an [Azure subscription](https://azure.microsoft.com/free?azure-portal=true) in which you have administrative-level access. 15 | 16 | ## Provision an Azure Machine Learning workspace 17 | 18 | An Azure Machine Learning *workspace* provides a central place for managing all resources and assets you need to train and manage your models. You can interact with the Azure Machine Learning workspace through the studio, Python SDK, and Azure CLI. 19 | 20 | You'll use a Shell script which uses the Azure CLI to provision the workspace and necessary resources. Next, you'll use the Designer in the Azure Machine Learning studio to train and compare models. 21 | 22 | ### Create the workspace and compute cluster 23 | 24 | To create the Azure Machine Learning workspace and a compute cluster, you'll use the Azure CLI. All necessary commands are grouped in a Shell script for you to execute. 25 | 26 | 1. In a browser, open the Azure portal at `https://portal.azure.com/`, signing in with your Microsoft account. 27 | 1. Select the \[>_] (*Cloud Shell*) button at the top of the page to the right of the search box. This opens a Cloud Shell pane at the bottom of the portal. 28 | 1. Select **Bash** if asked. The first time you open the cloud shell, you will be asked to choose the type of shell you want to use (*Bash* or *PowerShell*). 29 | 1. Check that the correct subscription is specified and that **No storage account required** is selected. Select **Apply**. 30 | 1. In the terminal, enter the following commands to clone this repo: 31 | 32 | ```azurecli 33 | rm -r azure-ml-labs -f 34 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 35 | ``` 36 | 37 | > Use `SHIFT + INSERT` to paste your copied code into the Cloud Shell. 38 | 39 | 1. After the repo has been cloned, enter the following commands to change to the folder for this lab and run the setup.sh script it contains: 40 | 41 | ```azurecli 42 | cd azure-ml-labs/Labs/05 43 | ./setup.sh 44 | ``` 45 | 46 | > Ignore any (error) messages that say that the extensions were not installed. 47 | 48 | 1. Wait for the script to complete - this typically takes around 5-10 minutes. 49 | 50 |
51 | Troubleshooting tip: Workspace creation error
52 |

If you receive an error when running the setup script through the CLI, you need to provision the resources manually:

53 |
    54 |
  1. In the Azure portal home page, select + Create a resource.
    2. 55 |
  2. Search for machine learning and then select Azure Machine Learning. Select Create.
    3. 56 |
  3. Create a new Azure Machine Learning resource with the following settings: 57 |
      58 |
    • Subscription: Your Azure subscription
      • 59 |
    • Resource group: rg-dp100-labs
      • 60 |
    • Workspace name: mlw-dp100-labs
      • 61 |
    • Region: Select the geographical region closest to you
      • 62 |
    • Storage account: Note the default new storage account that will be created for your workspace
      • 63 |
    • Key vault: Note the default new key vault that will be created for your workspace
      • 64 |
    • Application insights: Note the default new application insights resource that will be created for your workspace
      • 65 |
    • Container registry: None (one will be created automatically the first time you deploy a model to a container)
      • 66 |
    67 |
  4. Select Review + create and wait for the workspace and its associated resources to be created - this typically takes around 5 minutes.
    5. 68 |
  5. Select Go to resource and in its Overview page, select Launch studio. Another tab will open in your browser to open the Azure Machine Learning studio.
    6. 69 |
  6. Close any pop-ups that appear in the studio.
    7. 70 |
  7. Within the Azure Machine Learning studio, navigate to the Compute page and select + New under the Compute instances tab.
    8. 71 |
  8. Give the compute instance a unique name and then select Standard_DS11_v2 as the virtual machine size.
    9. 72 |
  9. Select Review + create and then select Create.
    10. 73 |
  10. Next, select the Compute clusters tab and select + New.
    11. 74 |
  11. Choose the same region as the one where you created your workspace and then select Standard_DS11_v2 as the virtual machine size. Select Next
    12. 75 |
  12. Give the cluster a unique name and then select Create.
    13. 76 |
77 |
78 | 79 | ## Configure a new pipeline 80 | 81 | When you've created the workspace and necessary compute cluster, you can open the Azure Machine Learning studio and create a training pipeline with the Designer. 82 | 83 | 1. In the Azure portal, navigate to the Azure Machine Learning workspace named **mlw-dp100-...**. 84 | 1. Select the Azure Machine Learning workspace, and in its **Overview** page, select **Launch studio**. Another tab will open in your browser to open the Azure Machine Learning studio. 85 | 1. Close any pop-ups that appear in the studio. 86 | 1. Within the Azure Machine Learning studio, navigate to the **Compute** page and verify that the compute cluster you created in the previous section exist. The cluster should be idle and have 0 nodes running. 87 | 1. Navigate to the **Designer** page. 88 | 1. Select the **Custom** tab at the top of the page. 89 | 1. Create a new empty pipeline using custom components. 90 | 1. Change the default pipeline name (**Pipeline-Created-on-*date***) to `Train-Diabetes-Classifier` by selecting the pencil icon on its right. 91 | 92 | 93 | ## Create a new pipeline 94 | 95 | To train a model, you'll need data. You can use any data stored in a datastore or use a publicly accessible URL. 96 | 97 | 1. In the left menu, select the **Data** tab. 98 | 1. Drag and drop the **diabetes-folder** component to the canvas. 99 | 100 | Now that you have your data, you can continue by creating a pipeline using custom components that already exist within the workspace (were created for you during setup). 101 | 102 | 1. In the left menu, select the **Components** tab. 103 | 1. Drag and drop the **Remove Empty Rows** component on to the canvas, below the **diabetes-folder**. 104 | 1. Connect the output of the data to the input of the new component. 105 | 1. Drag and drop the **Normalize Numerical Columns** component on to the canvas, below the **Remove Empty Rows**. 106 | 1. Connect the output of the previous component to the input of the new component. 107 | 1. Drag and drop the **Train a Decision Tree Classifier Model** component on to the canvas, below the **Normalize Numerical Columns**. 108 | 1. Connect the output of the previous component to the input of the new component. 109 | 1. Select **Configure & Submit** and in the **Set up pipeline job** page create a new experiment and name it `diabetes-designer-pipeline`, then select **Next**. 110 | 1. On the **Inputs & Outputs** make no changes and select **Next**. 111 | 1. On the **Runtime settings** select **Compute Cluster**, and under the **Select Azure ML compute cluster** select your *aml-cluster*. 112 | 1. Select **Review + Submit** and then select **Submit** to start the pipeline run. 113 | 1. You can check the status of the run by going to the **Pipelines** page and selecting the **Train-Diabetes-Classifier** pipeline. 114 | 1. Wait until all components have successfully completed. 115 | 116 | Submitting the job will initialize the compute cluster. As the compute cluster was idle up until now, it may take some time for the cluster to resize to more than 0 nodes. Once the cluster has resized, it will automatically start running the pipeline. 117 | 118 | You'll be able to track the run of each component. When the pipeline fails, you'll be able to explore which component failed and why it failed. Error messages will show in the **Outputs + logs** tab of the job overview. 119 | 120 | ## Train a second model to compare 121 | 122 | To compare between algorithms and evaluate which performs better, you can train two models within one pipeline and compare. 123 | 124 | 1. Return to the **Designer** and select the **Train-Diabetes-Classifier** pipeline draft. 125 | 1. Add the **Train a Logistic Regression Classifier Model** component to the canvas, next to the other training component. 126 | 1. Connect the output of the **Normalize Numerical Columns** component to the input of the new training component. 127 | 1. At the top, select **Configure & Submit**. 128 | 1. On the **Basics** page, create a new experiment named `designer-compare-classification`, and run it. 129 | 1. Select **Review + Submit** and then select **Submit** to start the pipeline run. 130 | 1. You can check the status of the run by going to the **Pipelines** page and selecting the **Train-Diabetes-Classifier** pipeline with the **designer-compare-classification** experiment. 131 | 1. Wait until all components have successfully completed. 132 | 1. Select **Job overview**, then select the **Metrics** tab to review the results for both training components. 133 | 1. Try and determine which model performed better. 134 | 135 | ## Delete Azure resources 136 | 137 | When you finish exploring Azure Machine Learning, you should delete the resources you've created to avoid unnecessary Azure costs. 138 | 139 | 1. Close the Azure Machine Learning studio tab and return to the Azure portal. 140 | 1. In the Azure portal, on the **Home** page, select **Resource groups**. 141 | 1. Select the **rg-dp100-...** resource group. 142 | 1. At the top of the **Overview** page for your resource group, select **Delete resource group**. 143 | 1. Enter the resource group name to confirm you want to delete it, and select **Delete**. 144 | -------------------------------------------------------------------------------- /Instructions/06-AutoML-classification-model.md: -------------------------------------------------------------------------------- 1 | --- 2 | lab: 3 | title: 'Find the best classification model with Automated Machine Learning' 4 | --- 5 | 6 | # Find the best classification model with Automated Machine Learning 7 | 8 | Determining the right algorithm and preprocessing transformations for model training can involve a lot of guesswork and experimentation. 9 | 10 | In this exercise, you'll use automated machine learning to determine the optimal algorithm and preprocessing steps for a model by performing multiple training runs in parallel. 11 | 12 | ## Before you start 13 | 14 | You'll need an [Azure subscription](https://azure.microsoft.com/free?azure-portal=true) in which you have administrative-level access. 15 | 16 | ## Provision an Azure Machine Learning workspace 17 | 18 | An Azure Machine Learning *workspace* provides a central place for managing all resources and assets you need to train and manage your models. You can interact with the Azure Machine Learning workspace through the studio, Python SDK, and Azure CLI. 19 | 20 | You'll use the Azure CLI to provision the workspace and necessary compute, and you'll use the Python SDK to train a classification model with Automated Machine Learning. 21 | 22 | ### Create the workspace and compute resources 23 | 24 | To create the Azure Machine Learning workspace, a compute instance, and a compute cluster, you'll use the Azure CLI. All necessary commands are grouped in a Shell script for you to execute. 25 | 26 | 1. In a browser, open the Azure portal at `https://portal.azure.com/`, signing in with your Microsoft account. 27 | 1. Select the \[>_] (*Cloud Shell*) button at the top of the page to the right of the search box. This opens a Cloud Shell pane at the bottom of the portal. 28 | 1. Select **Bash** if asked. The first time you open the cloud shell, you will be asked to choose the type of shell you want to use (*Bash* or *PowerShell*). 29 | 1. Check that the correct subscription is specified and that **No storage account required** is selected. Select **Apply**. 30 | 1. In the terminal, enter the following commands to clone this repo: 31 | 32 | ```azurecli 33 | rm -r azure-ml-labs -f 34 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 35 | ``` 36 | 37 | > Use `SHIFT + INSERT` to paste your copied code into the Cloud Shell. 38 | 39 | 1. After the repo has been cloned, enter the following commands to change to the folder for this lab and run the **setup.sh** script it contains: 40 | 41 | ```azurecli 42 | cd azure-ml-labs/Labs/06 43 | ./setup.sh 44 | ``` 45 | 46 | > Ignore any (error) messages that say that the extensions were not installed. 47 | 48 | 1. Wait for the script to complete - this typically takes around 5-10 minutes. 49 | 50 |
51 | Troubleshooting tip: Workspace creation error
52 |

If you receive an error when running the setup script through the CLI, you need to provision the resources manually:

53 |
    54 |
  1. In the Azure portal home page, select + Create a resource.
    2. 55 |
  2. Search for machine learning and then select Azure Machine Learning. Select Create.
    3. 56 |
  3. Create a new Azure Machine Learning resource with the following settings: 57 |
      58 |
    • Subscription: Your Azure subscription
      • 59 |
    • Resource group: rg-dp100-labs
      • 60 |
    • Workspace name: mlw-dp100-labs
      • 61 |
    • Region: Select the geographical region closest to you
      • 62 |
    • Storage account: Note the default new storage account that will be created for your workspace
      • 63 |
    • Key vault: Note the default new key vault that will be created for your workspace
      • 64 |
    • Application insights: Note the default new application insights resource that will be created for your workspace
      • 65 |
    • Container registry: None (one will be created automatically the first time you deploy a model to a container)
      • 66 |
    67 |
  4. Select Review + create and wait for the workspace and its associated resources to be created - this typically takes around 5 minutes.
    5. 68 |
  5. Select Go to resource and in its Overview page, select Launch studio. Another tab will open in your browser to open the Azure Machine Learning studio.
    6. 69 |
  6. Close any pop-ups that appear in the studio.
    7. 70 |
  7. Within the Azure Machine Learning studio, navigate to the Compute page and select + New under the Compute instances tab.
    8. 71 |
  8. Give the compute instance a unique name and then select Standard_DS11_v2 as the virtual machine size.
    9. 72 |
  9. Select Review + create and then select Create.
    10. 73 |
  10. Next, select the Compute clusters tab and select + New.
    11. 74 |
  11. Choose the same region as the one where you created your workspace and then select Standard_DS11_v2 as the virtual machine size. Select Next
    12. 75 |
  12. Give the cluster a unique name and then select Create.
    13. 76 |
77 |
78 | 79 | ## Clone the lab materials 80 | 81 | When you've created the workspace and necessary compute resources, you can open the Azure Machine Learning studio and clone the lab materials into the workspace. 82 | 83 | 1. In the Azure portal, navigate to the Azure Machine Learning workspace named **mlw-dp100-...**. 84 | 1. Select the Azure Machine Learning workspace, and in its **Overview** page, select **Launch studio**. Another tab will open in your browser to open the Azure Machine Learning studio. 85 | 1. Close any pop-ups that appear in the studio. 86 | 1. Within the Azure Machine Learning studio, navigate to the **Compute** page and verify that the compute instance and cluster you created in the previous section exist. The compute instance should be running, the cluster should be idle and have 0 nodes running. 87 | 1. In the **Compute instances** tab, find your compute instance, and select the **Terminal** application. 88 | 1. In the terminal, install the Python SDK on the compute instance by running the following commands in the terminal: 89 | 90 | ``` 91 | pip uninstall azure-ai-ml 92 | pip install azure-ai-ml 93 | ``` 94 | 95 | > Ignore any (error) messages that say that the packages couldn't be found and uninstalled. 96 | 97 | 1. Run the following command to clone a Git repository containing notebooks, data, and other files to your workspace: 98 | 99 | ``` 100 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 101 | ``` 102 | 103 | 1. When the command has completed, in the **Files** pane, click **↻** to refresh the view and verify that a new **Users/*your-user-name*/azure-ml-labs** folder has been created. 104 | 105 | ## Train a classification model with automated machine learning 106 | 107 | Now that you have all the necessary resources, you can run the notebook to configure and submit the Automated Machine Learning job. 108 | 109 | 1. Open the **Labs/06/Classification with Automated Machine Learning.ipynb** notebook. 110 | 111 | > Select **Authenticate** and follow the necessary steps if a notification appears asking you to authenticate. 112 | 113 | 1. Verify that the notebook uses the **Python 3.10 - AzureML** kernel. 114 | 1. Run all cells in the notebook. 115 | 116 | A new job will be created in the Azure Machine Learning workspace. The job tracks the inputs defined in the job configuration, the data asset used, and the outputs like metrics to evaluate the models. 117 | 118 | Note that the Automated Machine Learning jobs contains child jobs, which represent individual models that have been trained and other tasks needed to execute. 119 | 1. Go to **Jobs** and select the **auto-ml-class-dev** experiment. 120 | 1. Select the job under the **Display name** column. 121 | 1. Wait for its status to change to **Completed**. 122 | 1. When the Automate Machine Learning job status has changed to **Completed**, explore the job details in the studio: 123 | - The **Data guardrails** tab shows whether your training data had any issues. 124 | - The **Models + child jobs** tab will show all models that have been trained. Select **Explain model** for the best model and create the explanation job run using the **aml-cluster**. 125 | - Wait until a new column **Explained** appears next to the **Algorithm name** column and select **View explanation**. You may need to refresh the algorithm list for this option to appear. 126 | - Review the dashboard created to understand which features influenced the target value the most. 127 | 128 | ## Delete Azure resources 129 | 130 | When you finish exploring Azure Machine Learning, you should delete the resources you've created to avoid unnecessary Azure costs. 131 | 132 | 1. Close the Azure Machine Learning studio tab and return to the Azure portal. 133 | 1. In the Azure portal, on the **Home** page, select **Resource groups**. 134 | 1. Select the **rg-dp100-...** resource group. 135 | 1. At the top of the **Overview** page for your resource group, select **Delete resource group**. 136 | 1. Enter the resource group name to confirm you want to delete it, and select **Delete**. 137 | -------------------------------------------------------------------------------- /Instructions/07-Notebooks-mlflow-tracking.md: -------------------------------------------------------------------------------- 1 | --- 2 | lab: 3 | title: 'Track model training in notebooks with MLflow' 4 | --- 5 | 6 | # Track model training in notebooks with MLflow 7 | 8 | Often, you'll start a new data science project by experimenting and training multiple models. To track your work and keep an overview of the models you train and how they perform, you can use MLflow tracking. 9 | 10 | In this exercise, you'll use MLflow within a notebook running on a compute instance to log model training. 11 | 12 | ## Before you start 13 | 14 | You'll need an [Azure subscription](https://azure.microsoft.com/free) in which you have administrative-level access. 15 | 16 | ## Provision an Azure Machine Learning workspace 17 | 18 | An Azure Machine Learning *workspace* provides a central place for managing all resources and assets you need to train and manage your models. You can interact with the Azure Machine Learning workspace through the studio, Python SDK, and Azure CLI. 19 | 20 | You'll use the Azure CLI to provision the workspace and necessary compute, and you'll use the Python SDK to train a classification model with Automated Machine Learning. 21 | 22 | ### Create the workspace and compute resources 23 | 24 | To create the Azure Machine Learning workspace and a compute instance, you'll use the Azure CLI. All necessary commands are grouped in a Shell script for you to execute. 25 | 1. In a browser, open the Azure portal at `https://portal.azure.com/`, signing in with your Microsoft account. 26 | 1. Select the \[>_] (*Cloud Shell*) button at the top of the page to the right of the search box. This opens a Cloud Shell pane at the bottom of the portal. 27 | 1. Select **Bash** if asked. The first time you open the cloud shell, you will be asked to choose the type of shell you want to use (*Bash* or *PowerShell*). 28 | 1. Check that the correct subscription is specified and that **No storage account required** is selected. Select **Apply**. 29 | 1. In the terminal, enter the following commands to clone this repo: 30 | 31 | ```azurecli 32 | rm -r azure-ml-labs -f 33 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 34 | ``` 35 | 36 | > Use `SHIFT + INSERT` to paste your copied code into the Cloud Shell. 37 | 38 | 1. After the repo has been cloned, enter the following commands to change to the folder for this lab and run the **setup.sh** script it contains: 39 | 40 | ```azurecli 41 | cd azure-ml-labs/Labs/07 42 | ./setup.sh 43 | ``` 44 | 45 | > Ignore any (error) messages that say that the extensions were not installed. 46 | 47 | 1. Wait for the script to complete - this typically takes around 5-10 minutes. 48 | 49 |
50 | Troubleshooting tip: Workspace creation error
51 |

If you receive an error when running the setup script through the CLI, you need to provision the resources manually:

52 |
    53 |
  1. In the Azure portal home page, select + Create a resource.
    2. 54 |
  2. Search for machine learning and then select Azure Machine Learning. Select Create.
    3. 55 |
  3. Create a new Azure Machine Learning resource with the following settings: 56 |
      57 |
    • Subscription: Your Azure subscription
      • 58 |
    • Resource group: rg-dp100-labs
      • 59 |
    • Workspace name: mlw-dp100-labs
      • 60 |
    • Region: Select the geographical region closest to you
      • 61 |
    • Storage account: Note the default new storage account that will be created for your workspace
      • 62 |
    • Key vault: Note the default new key vault that will be created for your workspace
      • 63 |
    • Application insights: Note the default new application insights resource that will be created for your workspace
      • 64 |
    • Container registry: None (one will be created automatically the first time you deploy a model to a container)
      • 65 |
    66 |
  4. Select Review + create and wait for the workspace and its associated resources to be created - this typically takes around 5 minutes.
    5. 67 |
  5. Select Go to resource and in its Overview page, select Launch studio. Another tab will open in your browser to open the Azure Machine Learning studio.
    6. 68 |
  6. Close any pop-ups that appear in the studio.
    7. 69 |
  7. Within the Azure Machine Learning studio, navigate to the Compute page and select + New under the Compute instances tab.
    8. 70 |
  8. Give the compute instance a unique name and then select Standard_DS11_v2 as the virtual machine size.
    9. 71 |
  9. Select Review + create and then select Create.
    10. 72 |
  10. Next, select the Compute clusters tab and select + New.
    11. 73 |
  11. Choose the same region as the one where you created your workspace and then select Standard_DS11_v2 as the virtual machine size. Select Next
    12. 74 |
  12. Give the cluster a unique name and then select Create.
    13. 75 |
76 |
77 | 78 | ## Clone the lab materials 79 | 80 | When you've created the workspace and necessary compute resources, you can open the Azure Machine Learning studio and clone the lab materials into the workspace. 81 | 82 | 1. In the Azure portal, navigate to the Azure Machine Learning workspace named **mlw-dp100-...**. 83 | 1. Select the Azure Machine Learning workspace, and in its **Overview** page, select **Launch studio**. Another tab will open in your browser to open the Azure Machine Learning studio. 84 | 1. Close any pop-ups that appear in the studio. 85 | 1. Within the Azure Machine Learning studio, navigate to the **Compute** page and verify that the compute instance you created in the previous section exist. The compute instance should be running. 86 | 1. In the **Compute instances** tab, find your compute instance, and select the **Terminal** application. 87 | 1. In the terminal, install the Python SDK and the MLflow library on the compute instance by running the following commands in the terminal: 88 | 89 | ``` 90 | pip uninstall azure-ai-ml 91 | pip install azure-ai-ml 92 | pip install mlflow 93 | ``` 94 | 95 | > Ignore any (error) messages that say that the packages couldn't be found and uninstalled. 96 | 97 | 1. Run the following command to clone a Git repository containing a notebook, data, and other files to your workspace: 98 | 99 | ``` 100 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 101 | ``` 102 | 103 | 1. When the command has completed, in the **Files** pane, click **↻** to refresh the view and verify that a new **Users/*your-user-name*/azure-ml-labs** folder has been created. 104 | 105 | ## Track model training with MLflow 106 | 107 | Now that you have all the necessary resources, you can run the notebook to configure and use MLflow when training models in a notebook. 108 | 109 | 1. Open the **Labs/07/Track model training with MLflow.ipynb** notebook. 110 | 111 | > Select **Authenticate** and follow the necessary steps if a notification appears asking you to authenticate. 112 | 113 | 1. Verify that the notebook uses the **Python 3.10 - AzureML** kernel. 114 | 1. Run all cells in the notebook. 115 | 1. Review the new job that's created every time you train a model. 116 | 117 | > **Note:** When you train a model, the cell's output will show a link to the job run. If the link returns an error, you can still review the job run by selecting **Jobs** on the left side panel. 118 | 119 | ## Delete Azure resources 120 | 121 | When you finish exploring Azure Machine Learning, you should delete the resources you've created to avoid unnecessary Azure costs. 122 | 123 | 1. Close the Azure Machine Learning studio tab and return to the Azure portal. 124 | 1. In the Azure portal, on the **Home** page, select **Resource groups**. 125 | 1. Select the **rg-dp100-...** resource group. 126 | 1. At the top of the **Overview** page for your resource group, select **Delete resource group**. 127 | 1. Enter the resource group name to confirm you want to delete it, and select **Delete**. 128 | -------------------------------------------------------------------------------- /Instructions/08-Script-mlflow-tracking.md: -------------------------------------------------------------------------------- 1 | --- 2 | lab: 3 | title: 'Use MLflow to track training jobs' 4 | --- 5 | 6 | # Use MLflow to track training jobs 7 | 8 | MLflow is an open-source platform for managing the end-to-end machine learning lifecycle. MLflow Tracking is a component that logs and tracks your training job metrics, parameters and model artifacts. 9 | 10 | In this exercise, you'll use MLflow to track model training run as a command job. 11 | 12 | ## Before you start 13 | 14 | You'll need an [Azure subscription](https://azure.microsoft.com/free?azure-portal=true) in which you have administrative-level access. 15 | 16 | ## Provision an Azure Machine Learning workspace 17 | 18 | An Azure Machine Learning *workspace* provides a central place for managing all resources and assets you need to train and manage your models. You can interact with the Azure Machine Learning workspace through the studio, Python SDK, and Azure CLI. 19 | 20 | You'll use the Azure CLI to provision the workspace and necessary compute, and you'll use the Python SDK to run a command job. 21 | 22 | ### Create the workspace and compute resources 23 | 24 | To create the Azure Machine Learning workspace, a compute instance, and a compute cluster, you'll use the Azure CLI. All necessary commands are grouped in a Shell script for you to execute. 25 | 26 | 1. In a browser, open the Azure portal at `https://portal.azure.com/`, signing in with your Microsoft account. 27 | 1. Select the \[>_] (*Cloud Shell*) button at the top of the page to the right of the search box. This opens a Cloud Shell pane at the bottom of the portal. 28 | 1. Select **Bash** if asked. The first time you open the cloud shell, you will be asked to choose the type of shell you want to use (*Bash* or *PowerShell*). 29 | 1. Check that the correct subscription is specified and that **No storage account required** is selected. Select **Apply**. 30 | 1. In the terminal, enter the following commands to clone this repo: 31 | 32 | ```azurecli 33 | rm -r azure-ml-labs -f 34 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 35 | ``` 36 | 37 | > Use `SHIFT + INSERT` to paste your copied code into the Cloud Shell. 38 | 39 | 1. After the repo has been cloned, enter the following commands to change to the folder for this lab and run the **setup.sh** script it contains: 40 | 41 | ```azurecli 42 | cd azure-ml-labs/Labs/08 43 | ./setup.sh 44 | ``` 45 | 46 | > Ignore any (error) messages that say that the extensions were not installed. 47 | 48 | 1. Wait for the script to complete - this typically takes around 5-10 minutes. 49 | 50 |
51 | Troubleshooting tip: Workspace creation error
52 |

If you receive an error when running the setup script through the CLI, you need to provision the resources manually:

53 |
    54 |
  1. In the Azure portal home page, select + Create a resource.
    2. 55 |
  2. Search for machine learning and then select Azure Machine Learning. Select Create.
    3. 56 |
  3. Create a new Azure Machine Learning resource with the following settings: 57 |
      58 |
    • Subscription: Your Azure subscription
      • 59 |
    • Resource group: rg-dp100-labs
      • 60 |
    • Workspace name: mlw-dp100-labs
      • 61 |
    • Region: Select the geographical region closest to you
      • 62 |
    • Storage account: Note the default new storage account that will be created for your workspace
      • 63 |
    • Key vault: Note the default new key vault that will be created for your workspace
      • 64 |
    • Application insights: Note the default new application insights resource that will be created for your workspace
      • 65 |
    • Container registry: None (one will be created automatically the first time you deploy a model to a container)
      • 66 |
    67 |
  4. Select Review + create and wait for the workspace and its associated resources to be created - this typically takes around 5 minutes.
    5. 68 |
  5. Select Go to resource and in its Overview page, select Launch studio. Another tab will open in your browser to open the Azure Machine Learning studio.
    6. 69 |
  6. Close any pop-ups that appear in the studio.
    7. 70 |
  7. Within the Azure Machine Learning studio, navigate to the Compute page and select + New under the Compute instances tab.
    8. 71 |
  8. Give the compute instance a unique name and then select Standard_DS11_v2 as the virtual machine size.
    9. 72 |
  9. Select Review + create and then select Create.
    10. 73 |
  10. Next, select the Compute clusters tab and select + New.
    11. 74 |
  11. Choose the same region as the one where you created your workspace and then select Standard_DS11_v2 as the virtual machine size. Select Next
    12. 75 |
  12. Give the cluster a unique name and then select Create.
    13. 76 |
77 |
78 | 79 | ## Clone the lab materials 80 | 81 | When you've created the workspace and necessary compute resources, you can open the Azure Machine Learning studio and clone the lab materials into the workspace. 82 | 83 | 1. In the Azure portal, navigate to the Azure Machine Learning workspace named **mlw-dp100-...**. 84 | 1. Select the Azure Machine Learning workspace, and in its **Overview** page, select **Launch studio**. Another tab will open in your browser to open the Azure Machine Learning studio. 85 | 1. Close any pop-ups that appear in the studio. 86 | 1. Within the Azure Machine Learning studio, navigate to the **Compute** page and verify that the compute instance and cluster you created in the previous section exist. The compute instance should be running, the cluster should be idle and have 0 nodes running. 87 | 1. In the **Compute instances** tab, find your compute instance, and select the **Terminal** application. 88 | 1. In the terminal, install the Python SDK on the compute instance by running the following commands in the terminal: 89 | 90 | ``` 91 | pip uninstall azure-ai-ml 92 | pip install azure-ai-ml 93 | ``` 94 | 95 | > Ignore any (error) messages that say that the packages couldn't be found and uninstalled. 96 | 97 | 1. Run the following command to clone a Git repository containing notebooks, data, and other files to your workspace: 98 | 99 | ``` 100 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 101 | ``` 102 | 103 | 1. When the command has completed, in the **Files** pane, click **↻** to refresh the view and verify that a new **Users/*your-user-name*/azure-ml-labs** folder has been created. 104 | 105 | ## Submit MLflow jobs from a notebook 106 | 107 | Now that you have all the necessary resources, you can run the notebook to submit the job, which uses MLflow to track model parameters, metrics, and artifacts. 108 | 109 | 1. Open the **Labs/08/Use MLflow to track jobs.ipynb** notebook. 110 | 111 | > Select **Authenticate** and follow the necessary steps if a notification appears asking you to authenticate. 112 | 113 | 1. Verify that the notebook uses the **Python 3.10 - AzureML** kernel. 114 | 1. Run all cells in the notebook. 115 | 116 | ## Delete Azure resources 117 | 118 | When you finish exploring Azure Machine Learning, you should delete the resources you've created to avoid unnecessary Azure costs. 119 | 120 | 1. Close the Azure Machine Learning studio tab and return to the Azure portal. 121 | 1. In the Azure portal, on the **Home** page, select **Resource groups**. 122 | 1. Select the **rg-dp100-...** resource group. 123 | 1. At the top of the **Overview** page for your resource group, select **Delete resource group**. 124 | 1. Enter the resource group name to confirm you want to delete it, and select **Delete**. 125 | -------------------------------------------------------------------------------- /Instructions/09-Hyperparameter-tuning.md: -------------------------------------------------------------------------------- 1 | --- 2 | lab: 3 | title: 'Perform hyperparameter tuning with a sweep job' 4 | --- 5 | 6 | # Perform hyperparameter tuning with a sweep job 7 | 8 | Hyperparameters are variables that affect how a model is trained, but which can't be derived from the training data. Choosing the optimal hyperparameter values for model training can be difficult, and usually involved a great deal of trial and error. 9 | 10 | In this exercise, you'll use Azure Machine Learning to tune hyperparameters by performing multiple training trials in parallel. 11 | 12 | ## Before you start 13 | 14 | You'll need an [Azure subscription](https://azure.microsoft.com/free?azure-portal=true) in which you have administrative-level access. 15 | 16 | ## Provision an Azure Machine Learning workspace 17 | 18 | An Azure Machine Learning *workspace* provides a central place for managing all resources and assets you need to train and manage your models. You can interact with the Azure Machine Learning workspace through the studio, Python SDK, and Azure CLI. 19 | 20 | You'll use the Azure CLI to provision the workspace and necessary compute, and you'll use the Python SDK to run a command job. 21 | 22 | ### Create the workspace and compute resources 23 | 24 | To create the Azure Machine Learning workspace, a compute instance, and a compute cluster, you'll use the Azure CLI. All necessary commands are grouped in a Shell script for you to execute. 25 | 26 | 1. In a browser, open the Azure portal at `https://portal.azure.com/`, signing in with your Microsoft account. 27 | 1. Select the \[>_] (*Cloud Shell*) button at the top of the page to the right of the search box. This opens a Cloud Shell pane at the bottom of the portal. 28 | 1. Select **Bash** if asked. The first time you open the cloud shell, you will be asked to choose the type of shell you want to use (*Bash* or *PowerShell*). 29 | 1. Check that the correct subscription is specified and that **No storage account required** is selected. Select **Apply**. 30 | 1. In the terminal, enter the following commands to clone this repo: 31 | 32 | ```azurecli 33 | rm -r azure-ml-labs -f 34 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 35 | ``` 36 | 37 | > Use `SHIFT + INSERT` to paste your copied code into the Cloud Shell. 38 | 39 | 1. After the repo has been cloned, enter the following commands to change to the folder for this lab and run the **setup.sh** script it contains: 40 | 41 | ```azurecli 42 | cd azure-ml-labs/Labs/09 43 | ./setup.sh 44 | ``` 45 | 46 | > Ignore any (error) messages that say that the extensions were not installed. 47 | 48 | 1. Wait for the script to complete - this typically takes around 5-10 minutes. 49 | 50 |
51 | Troubleshooting tip: Workspace creation error
52 |

If you receive an error when running the setup script through the CLI, you need to provision the resources manually:

53 |
    54 |
  1. In the Azure portal home page, select + Create a resource.
    2. 55 |
  2. Search for machine learning and then select Azure Machine Learning. Select Create.
    3. 56 |
  3. Create a new Azure Machine Learning resource with the following settings: 57 |
      58 |
    • Subscription: Your Azure subscription
      • 59 |
    • Resource group: rg-dp100-labs
      • 60 |
    • Workspace name: mlw-dp100-labs
      • 61 |
    • Region: Select the geographical region closest to you
      • 62 |
    • Storage account: Note the default new storage account that will be created for your workspace
      • 63 |
    • Key vault: Note the default new key vault that will be created for your workspace
      • 64 |
    • Application insights: Note the default new application insights resource that will be created for your workspace
      • 65 |
    • Container registry: None (one will be created automatically the first time you deploy a model to a container)
      • 66 |
    67 |
  4. Select Review + create and wait for the workspace and its associated resources to be created - this typically takes around 5 minutes.
    5. 68 |
  5. Select Go to resource and in its Overview page, select Launch studio. Another tab will open in your browser to open the Azure Machine Learning studio.
    6. 69 |
  6. Close any pop-ups that appear in the studio.
    7. 70 |
  7. Within the Azure Machine Learning studio, navigate to the Compute page and select + New under the Compute instances tab.
    8. 71 |
  8. Give the compute instance a unique name and then select Standard_DS11_v2 as the virtual machine size.
    9. 72 |
  9. Select Review + create and then select Create.
    10. 73 |
  10. Next, select the Compute clusters tab and select + New.
    11. 74 |
  11. Choose the same region as the one where you created your workspace and then select Standard_DS11_v2 as the virtual machine size. Select Next
    12. 75 |
  12. Give the cluster a unique name and then select Create.
    13. 76 |
77 |
78 | 79 | ## Clone the lab materials 80 | 81 | When you've created the workspace and necessary compute resources, you can open the Azure Machine Learning studio and clone the lab materials into the workspace. 82 | 83 | 1. In the Azure portal, navigate to the Azure Machine Learning workspace named **mlw-dp100-...**. 84 | 1. Select the Azure Machine Learning workspace, and in its **Overview** page, select **Launch studio**. Another tab will open in your browser to open the Azure Machine Learning studio. 85 | 1. Close any pop-ups that appear in the studio. 86 | 1. Within the Azure Machine Learning studio, navigate to the **Compute** page and verify that the compute instance and cluster you created in the previous section exist. The compute instance should be running, the cluster should be idle and have 0 nodes running. 87 | 1. In the **Compute instances** tab, find your compute instance, and select the **Terminal** application. 88 | 1. In the terminal, install the Python SDK on the compute instance by running the following commands in the terminal: 89 | 90 | ``` 91 | pip uninstall azure-ai-ml 92 | pip install azure-ai-ml 93 | ``` 94 | 95 | > Ignore any (error) messages that say that the packages couldn't be found and uninstalled. 96 | 97 | 1. Run the following command to clone a Git repository containing notebooks, data, and other files to your workspace: 98 | 99 | ``` 100 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 101 | ``` 102 | 103 | 1. When the command has completed, in the **Files** pane, click **↻** to refresh the view and verify that a new **Users/*your-user-name*/azure-ml-labs** folder has been created. 104 | 105 | ## Tune hyperparameters with a sweep job 106 | 107 | Now that you have all the necessary resources, you can run the notebook to submit a sweep job. 108 | 109 | 1. Open the **Labs/09/Hyperparameter tuning.ipynb** notebook. 110 | 111 | > Select **Authenticate** and follow the necessary steps if a notification appears asking you to authenticate. 112 | 113 | 1. Verify that the notebook uses the **Python 3.10 - AzureML** kernel. 114 | 1. Run all cells in the notebook. 115 | 116 | ## Delete Azure resources 117 | 118 | When you finish exploring Azure Machine Learning, you should delete the resources you've created to avoid unnecessary Azure costs. 119 | 120 | 1. Close the Azure Machine Learning studio tab and return to the Azure portal. 121 | 1. In the Azure portal, on the **Home** page, select **Resource groups**. 122 | 1. Select the **rg-dp100-...** resource group. 123 | 1. At the top of the **Overview** page for your resource group, select **Delete resource group**. 124 | 1. Enter the resource group name to confirm you want to delete it, and select **Delete**. 125 | -------------------------------------------------------------------------------- /Instructions/09-Run-pipelines.md: -------------------------------------------------------------------------------- 1 | --- 2 | lab: 3 | title: 'Run pipelines in Azure Machine Learning' 4 | --- 5 | 6 | # Run pipelines in Azure Machine Learning 7 | 8 | You can use the Python SDK to perform all of the tasks required to create and operate a machine learning solution in Azure. Rather than perform these tasks individually, you can use pipelines to orchestrate the steps required to prepare data, run training scripts, and other tasks. 9 | 10 | In this exercise, you'll run multiple scripts as a pipeline job. 11 | 12 | ## Before you start 13 | 14 | You'll need an [Azure subscription](https://azure.microsoft.com/free?azure-portal=true) in which you have administrative-level access. 15 | 16 | ## Provision an Azure Machine Learning workspace 17 | 18 | An Azure Machine Learning *workspace* provides a central place for managing all resources and assets you need to train and manage your models. You can interact with the Azure Machine Learning workspace through the studio, Python SDK, and Azure CLI. 19 | 20 | You'll use the Azure CLI to provision the workspace and necessary compute, and you'll use the Python SDK to run a command job. 21 | 22 | ### Create the workspace and compute resources 23 | 24 | To create the Azure Machine Learning workspace, a compute instance, and a compute cluster, you'll use the Azure CLI. All necessary commands are grouped in a Shell script for you to execute. 25 | 26 | 1. In a browser, open the Azure portal at `https://portal.azure.com/`, signing in with your Microsoft account. 27 | 1. Select the \[>_] (*Cloud Shell*) button at the top of the page to the right of the search box. This opens a Cloud Shell pane at the bottom of the portal. 28 | 1. Select **Bash** if asked. The first time you open the cloud shell, you will be asked to choose the type of shell you want to use (*Bash* or *PowerShell*). 29 | 1. Check that the correct subscription is specified and that **No storage account required** is selected. Select **Apply**. 30 | 1. In the terminal, enter the following commands to clone this repo: 31 | 32 | ```azurecli 33 | rm -r azure-ml-labs -f 34 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 35 | ``` 36 | 37 | > Use `SHIFT + INSERT` to paste your copied code into the Cloud Shell. 38 | 39 | 1. After the repo has been cloned, enter the following commands to change to the folder for this lab and run the **setup.sh** script it contains: 40 | 41 | ```azurecli 42 | cd azure-ml-labs/Labs/09 43 | ./setup.sh 44 | ``` 45 | 46 | > Ignore any (error) messages that say that the extensions were not installed. 47 | 48 | 1. Wait for the script to complete - this typically takes around 5-10 minutes. 49 | 50 |
51 | Troubleshooting tip: Workspace creation error
52 |

If you receive an error when running the setup script through the CLI, you need to provision the resources manually:

53 |
    54 |
  1. In the Azure portal home page, select + Create a resource.
    2. 55 |
  2. Search for machine learning and then select Azure Machine Learning. Select Create.
    3. 56 |
  3. Create a new Azure Machine Learning resource with the following settings: 57 |
      58 |
    • Subscription: Your Azure subscription
      • 59 |
    • Resource group: rg-dp100-labs
      • 60 |
    • Workspace name: mlw-dp100-labs
      • 61 |
    • Region: Select the geographical region closest to you
      • 62 |
    • Storage account: Note the default new storage account that will be created for your workspace
      • 63 |
    • Key vault: Note the default new key vault that will be created for your workspace
      • 64 |
    • Application insights: Note the default new application insights resource that will be created for your workspace
      • 65 |
    • Container registry: None (one will be created automatically the first time you deploy a model to a container)
      • 66 |
    67 |
  4. Select Review + create and wait for the workspace and its associated resources to be created - this typically takes around 5 minutes.
    5. 68 |
  5. Select Go to resource and in its Overview page, select Launch studio. Another tab will open in your browser to open the Azure Machine Learning studio.
    6. 69 |
  6. Close any pop-ups that appear in the studio.
    7. 70 |
  7. Within the Azure Machine Learning studio, navigate to the Compute page and select + New under the Compute instances tab.
    8. 71 |
  8. Give the compute instance a unique name and then select Standard_DS11_v2 as the virtual machine size.
    9. 72 |
  9. Select Review + create and then select Create.
    10. 73 |
  10. Next, select the Compute clusters tab and select + New.
    11. 74 |
  11. Choose the same region as the one where you created your workspace and then select Standard_DS11_v2 as the virtual machine size. Select Next.
    12. 75 |
  12. Give the cluster a unique name and then select Create.
    13. 76 |
  13. Download the training data from https://github.com/MicrosoftLearning/mslearn-azure-ml/raw/refs/heads/main/Labs/09/data/diabetes.csv
    14. 77 |
  14. In the Azure Machine Learning studio, navigate to the Data page and select + Create.
    15. 78 |
  15. Name the data asset diabetes-data and verify that the type File (uri_file) is selected. Select Next.
    16. 79 |
  16. Select From local files as your data source and then select Next.
    17. 80 |
  17. Verify that Azure Blob Storage and workspaceblobstore are selected as your destination storage type and datastore respectively. Select Next.
    18. 81 |
  18. Upload the .csv file you downloaded previously and then select Next.
    19. 82 |
  19. Review the settings for your data asset and then select Create.
    20. 83 |
84 |
85 | 86 | ## Clone the lab materials 87 | 88 | When you've created the workspace and necessary compute resources, you can open the Azure Machine Learning studio and clone the lab materials into the workspace. 89 | 90 | 1. In the Azure portal, navigate to the Azure Machine Learning workspace named **mlw-dp100-...**. 91 | 1. Select the Azure Machine Learning workspace, and in its **Overview** page, select **Launch studio**. Another tab will open in your browser to open the Azure Machine Learning studio. 92 | 1. Close any pop-ups that appear in the studio. 93 | 1. Within the Azure Machine Learning studio, navigate to the **Compute** page and verify that the compute instance and cluster you created in the previous section exist. The compute instance should be running, the cluster should be idle and have 0 nodes running. 94 | 1. In the **Compute instances** tab, find your compute instance, and select the **Terminal** application. 95 | 1. In the terminal, install the Python SDK on the compute instance by running the following commands in the terminal: 96 | 97 | ``` 98 | pip uninstall azure-ai-ml 99 | pip install azure-ai-ml 100 | ``` 101 | 102 | > Ignore any (error) messages that say that the packages couldn't be found and uninstalled. 103 | 104 | 1. Run the following command to clone a Git repository containing notebooks, data, and other files to your workspace: 105 | 106 | ``` 107 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 108 | ``` 109 | 110 | 1. When the command has completed, in the **Files** pane, click **↻** to refresh the view and verify that a new **Users/*your-user-name*/azure-ml-labs** folder has been created. 111 | 112 | ## Run scripts as a pipeline job 113 | 114 | The code to build and submit a pipeline with the Python SDK is provided in a notebook. 115 | 116 | 1. Open the **Labs/09/Run a pipeline job.ipynb** notebook. 117 | 118 | > Select **Authenticate** and follow the necessary steps if a notification appears asking you to authenticate. 119 | 120 | 1. Verify that the notebook uses the **Python 3.10 - AzureML** kernel. 121 | 1. Run all cells in the notebook. 122 | 123 | ## Delete Azure resources 124 | 125 | When you finish exploring Azure Machine Learning, you should delete the resources you've created to avoid unnecessary Azure costs. 126 | 127 | 1. Close the Azure Machine Learning studio tab and return to the Azure portal. 128 | 1. In the Azure portal, on the **Home** page, select **Resource groups**. 129 | 1. Select the **rg-dp100-...** resource group. 130 | 1. At the top of the **Overview** page for your resource group, select **Delete resource group**. 131 | 1. Enter the resource group name to confirm you want to delete it, and select **Delete**. 132 | -------------------------------------------------------------------------------- /Instructions/10-Compare-models.md: -------------------------------------------------------------------------------- 1 | --- 2 | lab: 3 | title: 'Create and explore the Responsible AI dashboard' 4 | --- 5 | 6 | # Create and explore the Responsible AI dashboard 7 | 8 | After you train your model, you'll want to evaluate your model to explore whether it's performing as expected. Next to performance metrics, there are other factors you can take into consideration. The Responsible AI dashboard in Azure Machine Learning allows you to analyze the data and the model's predictions to identify any bias or unfairness. 9 | 10 | In this exercise, you'll prepare your data and create a Responsible AI dashboard in Azure Machine Learning. 11 | 12 | ## Before you start 13 | 14 | You'll need an [Azure subscription](https://azure.microsoft.com/free?azure-portal=true) in which you have administrative-level access. 15 | 16 | ## Provision an Azure Machine Learning workspace 17 | 18 | An Azure Machine Learning *workspace* provides a central place for managing all resources and assets you need to train and manage your models. You can interact with the Azure Machine Learning workspace through the studio, Python SDK, and Azure CLI. 19 | 20 | You'll use the Azure CLI to provision the workspace and necessary compute, and you'll use the Python SDK to run a command job. 21 | 22 | ### Create the workspace and compute resources 23 | 24 | To create the Azure Machine Learning workspace, a compute instance, and a compute cluster, you'll use the Azure CLI. All necessary commands are grouped in a Shell script for you to execute. 25 | 26 | 1. In a browser, open the Azure portal at `https://portal.azure.com/`, signing in with your Microsoft account. 27 | 1. Select the \[>_] (*Cloud Shell*) button at the top of the page to the right of the search box. This opens a Cloud Shell pane at the bottom of the portal. 28 | 1. Select **Bash** if asked. The first time you open the cloud shell, you will be asked to choose the type of shell you want to use (*Bash* or *PowerShell*). 29 | 1. Check that the correct subscription is specified and that **No storage account required** is selected. Select **Apply**. 30 | 1. In the terminal, enter the following commands to clone this repo: 31 | 32 | ```azurecli 33 | rm -r azure-ml-labs -f 34 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 35 | ``` 36 | 37 | > Use `SHIFT + INSERT` to paste your copied code into the Cloud Shell. 38 | 39 | 1. After the repo has been cloned, enter the following commands to change to the folder for this lab and run the **setup.sh** script it contains: 40 | 41 | ```azurecli 42 | cd azure-ml-labs/Labs/10 43 | ./setup.sh 44 | ``` 45 | 46 | > Ignore any (error) messages that say that the extensions were not installed. 47 | 48 | 1. Wait for the script to complete - this typically takes around 5-10 minutes. 49 | 50 |
51 | Troubleshooting tip: Workspace creation error
52 |

If you receive an error when running the setup script through the CLI, you need to provision the resources manually:

53 |
    54 |
  1. In the Azure portal home page, select + Create a resource.
    2. 55 |
  2. Search for machine learning and then select Azure Machine Learning. Select Create.
    3. 56 |
  3. Create a new Azure Machine Learning resource with the following settings: 57 |
      58 |
    • Subscription: Your Azure subscription
      • 59 |
    • Resource group: rg-dp100-labs
      • 60 |
    • Workspace name: mlw-dp100-labs
      • 61 |
    • Region: Select the geographical region closest to you
      • 62 |
    • Storage account: Note the default new storage account that will be created for your workspace
      • 63 |
    • Key vault: Note the default new key vault that will be created for your workspace
      • 64 |
    • Application insights: Note the default new application insights resource that will be created for your workspace
      • 65 |
    • Container registry: None (one will be created automatically the first time you deploy a model to a container)
      • 66 |
    67 |
  4. Select Review + create and wait for the workspace and its associated resources to be created - this typically takes around 5 minutes.
    5. 68 |
  5. Select Go to resource and in its Overview page, select Launch studio. Another tab will open in your browser to open the Azure Machine Learning studio.
    6. 69 |
  6. Close any pop-ups that appear in the studio.
    7. 70 |
  7. Within the Azure Machine Learning studio, navigate to the Compute page and select + New under the Compute instances tab.
    8. 71 |
  8. Give the compute instance a unique name and then select Standard_DS11_v2 as the virtual machine size.
    9. 72 |
  9. Select Review + create and then select Create.
    10. 73 |
  10. Next, select the Compute clusters tab and select + New.
    11. 74 |
  11. Choose the same region as the one where you created your workspace and then select Standard_DS11_v2 as the virtual machine size. Select Next
    12. 75 |
  12. Give the cluster a unique name and then select Create.
    13. 76 |
77 |
78 | 79 | ## Clone the lab materials 80 | 81 | When you've created the workspace and necessary compute resources, you can open the Azure Machine Learning studio and clone the lab materials into the workspace. 82 | 83 | 1. In the Azure portal, navigate to the Azure Machine Learning workspace named **mlw-dp100-...**. 84 | 1. Select the Azure Machine Learning workspace, and in its **Overview** page, select **Launch studio**. Another tab will open in your browser to open the Azure Machine Learning studio. 85 | 1. Close any pop-ups that appear in the studio. 86 | 1. Within the Azure Machine Learning studio, navigate to the **Compute** page and verify that the compute instance and cluster you created in the previous section exist. The compute instance should be running, the cluster should be idle and have 0 nodes running. 87 | 1. In the **Compute instances** tab, find your compute instance, and select the **Terminal** application. 88 | 1. In the terminal, install the Python SDK on the compute instance by running the following commands in the terminal: 89 | 90 | ``` 91 | pip uninstall azure-ai-ml 92 | pip install azure-ai-ml 93 | ``` 94 | 95 | > Ignore any (error) messages that say that the packages couldn't be found and uninstalled. 96 | 97 | 1. Run the following command to clone a Git repository containing notebooks, data, and other files to your workspace: 98 | 99 | ``` 100 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 101 | ``` 102 | 103 | 1. When the command has completed, in the **Files** pane, click **↻** to refresh the view and verify that a new **Users/*your-user-name*/azure-ml-labs** folder has been created. 104 | 105 | ## Create a pipeline to evaluate models and submit from a notebook 106 | 107 | Now that you have all the necessary resources, you can run the notebook to retrieve the built-in responsible AI components, create a pipeline, and submit the pipeline to generate a responsible AI dashboard. 108 | 109 | 1. Open the **Labs/10/Create Responsible AI dashboard.ipynb** notebook. 110 | 111 | > Select **Authenticate** and follow the necessary steps if a notification appears asking you to authenticate. 112 | 113 | 1. Verify that the notebook uses the **Python 3.10 - AzureML** kernel. 114 | 1. Run all cells in the notebook. 115 | 116 | ## Delete Azure resources 117 | 118 | When you finish exploring Azure Machine Learning, you should delete the resources you've created to avoid unnecessary Azure costs. 119 | 120 | 1. Close the Azure Machine Learning studio tab and return to the Azure portal. 121 | 1. In the Azure portal, on the **Home** page, select **Resource groups**. 122 | 1. Select the **rg-dp100-...** resource group. 123 | 1. At the top of the **Overview** page for your resource group, select **Delete resource group**. 124 | 1. Enter the resource group name to confirm you want to delete it, and select **Delete**. 125 | -------------------------------------------------------------------------------- /Instructions/10-Log-mlflow-models.md: -------------------------------------------------------------------------------- 1 | --- 2 | lab: 3 | title: 'Log and register models with MLflow' 4 | --- 5 | 6 | # Log and register models with MLflow 7 | 8 | MLflow is an open-source platform for managing the end-to-end machine learning lifecycle. When you log models with MLflow, you can easily move the model across platforms and workloads. 9 | 10 | In this exercise, you'll use MLflow to log machine learning models. 11 | 12 | ## Before you start 13 | 14 | You'll need an [Azure subscription](https://azure.microsoft.com/free?azure-portal=true) in which you have administrative-level access. 15 | 16 | ## Provision an Azure Machine Learning workspace 17 | 18 | An Azure Machine Learning *workspace* provides a central place for managing all resources and assets you need to train and manage your models. You can interact with the Azure Machine Learning workspace through the studio, Python SDK, and Azure CLI. 19 | 20 | You'll use the Azure CLI to provision the workspace and necessary compute, and you'll use the Python SDK to run a command job. 21 | 22 | ### Create the workspace and compute resources 23 | 24 | To create the Azure Machine Learning workspace, a compute instance, and a compute cluster, you'll use the Azure CLI. All necessary commands are grouped in a Shell script for you to execute. 25 | 26 | 1. In a browser, open the Azure portal at `https://portal.azure.com/`, signing in with your Microsoft account. 27 | 1. Select the \[>_] (*Cloud Shell*) button at the top of the page to the right of the search box. This opens a Cloud Shell pane at the bottom of the portal. 28 | 1. Select **Bash** if asked. The first time you open the cloud shell, you will be asked to choose the type of shell you want to use (*Bash* or *PowerShell*). 29 | 1. Check that the correct subscription is specified and that **No storage account required** is selected. Select **Apply**. 30 | 1. In the terminal, enter the following commands to clone this repo: 31 | 32 | ```azurecli 33 | rm -r azure-ml-labs -f 34 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 35 | ``` 36 | 37 | > Use `SHIFT + INSERT` to paste your copied code into the Cloud Shell. 38 | 39 | 1. After the repo has been cloned, enter the following commands to change to the folder for this lab and run the **setup.sh** script it contains: 40 | 41 | ```azurecli 42 | cd azure-ml-labs/Labs/10 43 | ./setup.sh 44 | ``` 45 | 46 | > Ignore any (error) messages that say that the extensions were not installed. 47 | 48 | 1. Wait for the script to complete - this typically takes around 5-10 minutes. 49 | 50 |
51 | Troubleshooting tip: Workspace creation error
52 |

If you receive an error when running the setup script through the CLI, you need to provision the resources manually:

53 |
    54 |
  1. In the Azure portal home page, select + Create a resource.
    2. 55 |
  2. Search for machine learning and then select Azure Machine Learning. Select Create.
    3. 56 |
  3. Create a new Azure Machine Learning resource with the following settings: 57 |
      58 |
    • Subscription: Your Azure subscription
      • 59 |
    • Resource group: rg-dp100-labs
      • 60 |
    • Workspace name: mlw-dp100-labs
      • 61 |
    • Region: Select the geographical region closest to you
      • 62 |
    • Storage account: Note the default new storage account that will be created for your workspace
      • 63 |
    • Key vault: Note the default new key vault that will be created for your workspace
      • 64 |
    • Application insights: Note the default new application insights resource that will be created for your workspace
      • 65 |
    • Container registry: None (one will be created automatically the first time you deploy a model to a container)
      • 66 |
    67 |
  4. Select Review + create and wait for the workspace and its associated resources to be created - this typically takes around 5 minutes.
    5. 68 |
  5. Select Go to resource and in its Overview page, select Launch studio. Another tab will open in your browser to open the Azure Machine Learning studio.
    6. 69 |
  6. Close any pop-ups that appear in the studio.
    7. 70 |
  7. Within the Azure Machine Learning studio, navigate to the Compute page and select + New under the Compute instances tab.
    8. 71 |
  8. Give the compute instance a unique name and then select Standard_DS11_v2 as the virtual machine size.
    9. 72 |
  9. Select Review + create and then select Create.
    10. 73 |
  10. Next, select the Compute clusters tab and select + New.
    11. 74 |
  11. Choose the same region as the one where you created your workspace and then select Standard_DS11_v2 as the virtual machine size. Select Next
    12. 75 |
  12. Give the cluster a unique name and then select Create.
    13. 76 |
77 |
78 | 79 | ## Clone the lab materials 80 | 81 | When you've created the workspace and necessary compute resources, you can open the Azure Machine Learning studio and clone the lab materials into the workspace. 82 | 83 | 1. In the Azure portal, navigate to the Azure Machine Learning workspace named **mlw-dp100-...**. 84 | 1. Select the Azure Machine Learning workspace, and in its **Overview** page, select **Launch studio**. Another tab will open in your browser to open the Azure Machine Learning studio. 85 | 1. Close any pop-ups that appear in the studio. 86 | 1. Within the Azure Machine Learning studio, navigate to the **Compute** page and verify that the compute instance and cluster you created in the previous section exist. The compute instance should be running, the cluster should be idle and have 0 nodes running. 87 | 1. In the **Compute instances** tab, find your compute instance, and select the **Terminal** application. 88 | 1. In the terminal, install the Python SDK on the compute instance by running the following commands in the terminal: 89 | 90 | ``` 91 | pip uninstall azure-ai-ml 92 | pip install azure-ai-ml 93 | ``` 94 | 95 | > Ignore any (error) messages that say that the packages couldn't be found and uninstalled. 96 | 97 | 1. Run the following command to clone a Git repository containing notebooks, data, and other files to your workspace: 98 | 99 | ``` 100 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 101 | ``` 102 | 103 | 1. When the command has completed, in the **Files** pane, click **↻** to refresh the view and verify that a new **Users/*your-user-name*/azure-ml-labs** folder has been created. 104 | 105 | ## Submit MLflow jobs from a notebook 106 | 107 | Now that you have all the necessary resources, you can run the notebook to train and log models with MLflow. 108 | 109 | 1. Open the **Labs\10\Log models with MLflow.ipynb** notebook. 110 | 111 | > Select **Authenticate** and follow the necessary steps if a notification appears asking you to authenticate. 112 | 113 | 1. Verify that the notebook uses the **Python 3.10 - AzureML** kernel. 114 | 1. Run all cells in the notebook. 115 | 116 | ## Delete Azure resources 117 | 118 | When you finish exploring Azure Machine Learning, you should delete the resources you've created to avoid unnecessary Azure costs. 119 | 120 | 1. Close the Azure Machine Learning studio tab and return to the Azure portal. 121 | 1. In the Azure portal, on the **Home** page, select **Resource groups**. 122 | 1. Select the **rg-dp100-...** resource group. 123 | 1. At the top of the **Overview** page for your resource group, select **Delete resource group**. 124 | 1. Enter the resource group name to confirm you want to delete it, and select **Delete**. 125 | -------------------------------------------------------------------------------- /Instructions/11-Deploy-batch-endpoint.md: -------------------------------------------------------------------------------- 1 | --- 2 | lab: 3 | title: 'Deploy a model to a batch endpoint' 4 | --- 5 | 6 | # Deploy a model to a batch endpoint 7 | 8 | In many scenarios, inferencing is performed as a batch process that uses a predictive model to score a large number of cases. To implement this kind of inferencing solution in Azure Machine Learning, you can create a batch endpoint. 9 | 10 | In this exercise, you'll deploy an MLflow model to a batch endpoint, and test it on sample data by submitting a job. 11 | 12 | ## Before you start 13 | 14 | You'll need an [Azure subscription](https://azure.microsoft.com/free?azure-portal=true) in which you have administrative-level access. 15 | 16 | ## Provision an Azure Machine Learning workspace 17 | 18 | An Azure Machine Learning *workspace* provides a central place for managing all resources and assets you need to train and manage your models. You can interact with the Azure Machine Learning workspace through the studio, Python SDK, and Azure CLI. 19 | 20 | You'll use the Azure CLI to provision the workspace and necessary compute, and you'll use the Python SDK to run a command job. 21 | 22 | ### Create the workspace and compute resources 23 | 24 | To create the Azure Machine Learning workspace, a compute instance, and a compute cluster, you'll use the Azure CLI. All necessary commands are grouped in a Shell script for you to execute. 25 | 26 | 1. In a browser, open the Azure portal at `https://portal.azure.com/`, signing in with your Microsoft account. 27 | 1. Select the \[>_] (*Cloud Shell*) button at the top of the page to the right of the search box. This opens a Cloud Shell pane at the bottom of the portal. 28 | 1. Select **Bash** if asked. The first time you open the cloud shell, you will be asked to choose the type of shell you want to use (*Bash* or *PowerShell*). 29 | 1. Check that the correct subscription is specified and that **No storage account required** is selected. Select **Apply**. 30 | 1. In the terminal, enter the following commands to clone this repo: 31 | 32 | ```azurecli 33 | rm -r azure-ml-labs -f 34 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 35 | ``` 36 | 37 | > Use `SHIFT + INSERT` to paste your copied code into the Cloud Shell. 38 | 39 | 1. After the repo has been cloned, enter the following commands to change to the folder for this lab and run the **setup.sh** script it contains: 40 | 41 | ```azurecli 42 | cd azure-ml-labs/Labs/11 43 | ./setup.sh 44 | ``` 45 | 46 | > Ignore any (error) messages that say that the extensions were not installed. 47 | 48 | 1. Wait for the script to complete - this typically takes around 5-10 minutes. 49 | 50 |
51 | Troubleshooting tip: Workspace creation error
52 |

If you receive an error when running the setup script through the CLI, you need to provision the resources manually:

53 |
    54 |
  1. In the Azure portal home page, select + Create a resource.
    2. 55 |
  2. Search for machine learning and then select Azure Machine Learning. Select Create.
    3. 56 |
  3. Create a new Azure Machine Learning resource with the following settings: 57 |
      58 |
    • Subscription: Your Azure subscription
      • 59 |
    • Resource group: rg-dp100-labs
      • 60 |
    • Workspace name: mlw-dp100-labs
      • 61 |
    • Region: Select the geographical region closest to you
      • 62 |
    • Storage account: Note the default new storage account that will be created for your workspace
      • 63 |
    • Key vault: Note the default new key vault that will be created for your workspace
      • 64 |
    • Application insights: Note the default new application insights resource that will be created for your workspace
      • 65 |
    • Container registry: None (one will be created automatically the first time you deploy a model to a container)
      • 66 |
    67 |
  4. Select Review + create and wait for the workspace and its associated resources to be created - this typically takes around 5 minutes.
    5. 68 |
  5. Select Go to resource and in its Overview page, select Launch studio. Another tab will open in your browser to open the Azure Machine Learning studio.
    6. 69 |
  6. Close any pop-ups that appear in the studio.
    7. 70 |
  7. Within the Azure Machine Learning studio, navigate to the Compute page and select + New under the Compute instances tab.
    8. 71 |
  8. Give the compute instance a unique name and then select Standard_DS11_v2 as the virtual machine size.
    9. 72 |
  9. Select Review + create and then select Create.
    10. 73 |
  10. Next, select the Compute clusters tab and select + New.
    11. 74 |
  11. Choose the same region as the one where you created your workspace and then select Standard_DS11_v2 as the virtual machine size. Select Next
    12. 75 |
  12. Give the cluster a unique name and then select Create.
    13. 76 |
77 |
78 | 79 | ## Clone the lab materials 80 | 81 | When you've created the workspace and necessary compute resources, you can open the Azure Machine Learning studio and clone the lab materials into the workspace. 82 | 83 | 1. In the Azure portal, navigate to the Azure Machine Learning workspace named **mlw-dp100-...**. 84 | 1. Select the Azure Machine Learning workspace, and in its **Overview** page, select **Launch studio**. Another tab will open in your browser to open the Azure Machine Learning studio. 85 | 1. Close any pop-ups that appear in the studio. 86 | 1. Within the Azure Machine Learning studio, navigate to the **Compute** page and verify that the compute instance and cluster you created in the previous section exist. The compute instance should be running, the cluster should be idle and have 0 nodes running. 87 | 1. In the **Compute instances** tab, find your compute instance, and select the **Terminal** application. 88 | 1. In the terminal, install the Python SDK on the compute instance by running the following commands in the terminal: 89 | 90 | ``` 91 | 92 | pip uninstall azure-ai-ml 93 | pip install azure-ai-ml 94 | ``` 95 | 96 | > Ignore any (error) messages that say that the packages couldn't be found and uninstalled. 97 | 98 | 1. Run the following command to clone a Git repository containing a notebook, data, and other files to your workspace: 99 | 100 | ``` 101 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 102 | ``` 103 | 104 | 1. When the command has completed, in the **Files** pane, click **↻** to refresh the view and verify that a new **Users/*your-user-name*/azure-ml-labs** folder has been created. 105 | 106 | ## Deploy a model to an batch endpoint 107 | 108 | The code to create the endpoint and deploy an MLflow model with the Python SDK is provided in a notebook. 109 | 110 | 1. Open the **Labs/11/Deploy to batch endpoint.ipynb** notebook. 111 | 112 | > Select **Authenticate** and follow the necessary steps if a notification appears asking you to authenticate. 113 | 114 | 1. Verify that the notebook uses the **Python 3.10 - AzureML** kernel. 115 | 1. Run all cells in the notebook. 116 | 117 | ## Delete Azure resources 118 | 119 | When you finish exploring Azure Machine Learning, you should delete the resources you've created to avoid unnecessary Azure costs. 120 | 121 | 1. Close the Azure Machine Learning studio tab and return to the Azure portal. 122 | 1. In the Azure portal, on the **Home** page, select **Resource groups**. 123 | 1. Select the **rg-dp100-...** resource group. 124 | 1. At the top of the **Overview** page for your resource group, select **Delete resource group**. 125 | 1. Enter the resource group name to confirm you want to delete it, and select **Delete**. 126 | -------------------------------------------------------------------------------- /Instructions/11-Deploy-online-endpoint.md: -------------------------------------------------------------------------------- 1 | --- 2 | lab: 3 | title: 'Deploy a model to a managed online endpoint' 4 | --- 5 | 6 | # Deploy a model to a managed online endpoint 7 | 8 | To consume a model in an application, and get real-time predictions, you'll want to deploy the model to a managed online endpoint. An MLflow model is easily deployed since you won't need to define the environment or create the scoring script. 9 | 10 | In this exercise, you'll deploy an MLflow model to a managed online endpoint, and test it on sample data. 11 | 12 | ## Before you start 13 | 14 | You'll need an [Azure subscription](https://azure.microsoft.com/free?azure-portal=true) in which you have administrative-level access. 15 | 16 | ## Provision an Azure Machine Learning workspace 17 | 18 | An Azure Machine Learning *workspace* provides a central place for managing all resources and assets you need to train and manage your models. You can interact with the Azure Machine Learning workspace through the studio, Python SDK, and Azure CLI. 19 | 20 | You'll use the Azure CLI to provision the workspace and necessary compute, and you'll use the Python SDK to run a command job. 21 | 22 | ### Create the workspace and compute resources 23 | 24 | To create the Azure Machine Learning workspace, a compute instance, and a compute cluster, you'll use the Azure CLI. All necessary commands are grouped in a Shell script for you to execute. 25 | 26 | 1. In a browser, open the Azure portal at `https://portal.azure.com/`, signing in with your Microsoft account. 27 | 1. Select the \[>_] (*Cloud Shell*) button at the top of the page to the right of the search box. This opens a Cloud Shell pane at the bottom of the portal. 28 | 1. Select **Bash** if asked. The first time you open the cloud shell, you will be asked to choose the type of shell you want to use (*Bash* or *PowerShell*). 29 | 1. Check that the correct subscription is specified and that **No storage account required** is selected. Select **Apply**. 30 | 1. In the terminal, enter the following commands to clone this repo: 31 | 32 | ```azurecli 33 | rm -r azure-ml-labs -f 34 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 35 | ``` 36 | 37 | > Use `SHIFT + INSERT` to paste your copied code into the Cloud Shell. 38 | 39 | 1. After the repo has been cloned, enter the following commands to change to the folder for this lab and run the **setup.sh** script it contains: 40 | 41 | ```azurecli 42 | cd azure-ml-labs/Labs/11 43 | ./setup.sh 44 | ``` 45 | 46 | > Ignore any (error) messages that say that the extensions were not installed. 47 | 48 | 1. Wait for the script to complete - this typically takes around 5-10 minutes. 49 | 50 |
51 | Troubleshooting tip: Workspace creation error
52 |

If you receive an error when running the setup script through the CLI, you need to provision the resources manually:

53 |
    54 |
  1. In the Azure portal home page, select + Create a resource.
    2. 55 |
  2. Search for machine learning and then select Azure Machine Learning. Select Create.
    3. 56 |
  3. Create a new Azure Machine Learning resource with the following settings: 57 |
      58 |
    • Subscription: Your Azure subscription
      • 59 |
    • Resource group: rg-dp100-labs
      • 60 |
    • Workspace name: mlw-dp100-labs
      • 61 |
    • Region: Select the geographical region closest to you
      • 62 |
    • Storage account: Note the default new storage account that will be created for your workspace
      • 63 |
    • Key vault: Note the default new key vault that will be created for your workspace
      • 64 |
    • Application insights: Note the default new application insights resource that will be created for your workspace
      • 65 |
    • Container registry: None (one will be created automatically the first time you deploy a model to a container)
      • 66 |
    67 |
  4. Select Review + create and wait for the workspace and its associated resources to be created - this typically takes around 5 minutes.
    5. 68 |
  5. Select Go to resource and in its Overview page, select Launch studio. Another tab will open in your browser to open the Azure Machine Learning studio.
    6. 69 |
  6. Close any pop-ups that appear in the studio.
    7. 70 |
  7. Within the Azure Machine Learning studio, navigate to the Compute page and select + New under the Compute instances tab.
    8. 71 |
  8. Give the compute instance a unique name and then select Standard_DS11_v2 as the virtual machine size.
    9. 72 |
  9. Select Review + create and then select Create.
    10. 73 |
  10. Next, select the Compute clusters tab and select + New.
    11. 74 |
  11. Choose the same region as the one where you created your workspace and then select Standard_DS11_v2 as the virtual machine size. Select Next
    12. 75 |
  12. Give the cluster a unique name and then select Create.
    13. 76 |
77 |
78 | 79 | ## Clone the lab materials 80 | 81 | When you've created the workspace and necessary compute resources, you can open the Azure Machine Learning studio and clone the lab materials into the workspace. 82 | 83 | 1. In the Azure portal, navigate to the Azure Machine Learning workspace named **mlw-dp100-...**. 84 | 1. Select the Azure Machine Learning workspace, and in its **Overview** page, select **Launch studio**. Another tab will open in your browser to open the Azure Machine Learning studio. 85 | 1. Close any pop-ups that appear in the studio. 86 | 1. Within the Azure Machine Learning studio, navigate to the **Compute** page and verify that the compute instance and cluster you created in the previous section exist. The compute instance should be running, the cluster should be idle and have 0 nodes running. 87 | 1. In the **Compute instances** tab, find your compute instance, and select the **Terminal** application. 88 | 1. In the terminal, install the Python SDK on the compute instance by running the following commands in the terminal: 89 | 90 | ``` 91 | pip uninstall azure-ai-ml 92 | pip install azure-ai-ml 93 | ``` 94 | 95 | > Ignore any (error) messages that say that the packages couldn't be found and uninstalled. 96 | 97 | 1. Run the following command to clone a Git repository containing a notebook, data, and other files to your workspace: 98 | 99 | ``` 100 | git clone https://github.com/MicrosoftLearning/mslearn-azure-ml.git azure-ml-labs 101 | ``` 102 | 103 | 1. When the command has completed, in the **Files** pane, click **↻** to refresh the view and verify that a new **Users/*your-user-name*/azure-ml-labs** folder has been created. 104 | 105 | ## Deploy a model to an online endpoint 106 | 107 | The code to create the endpoint and deploy an MLflow model with the Python SDK is provided in a notebook. 108 | 109 | 1. Open the **Labs/11/Deploy to online endpoint.ipynb** notebook. 110 | 111 | > Select **Authenticate** and follow the necessary steps if a notification appears asking you to authenticate. 112 | 113 | 1. Verify that the notebook uses the **Python 3.10 - AzureML** kernel. 114 | 1. Run all cells in the notebook. 115 | 116 | ## Delete Azure resources 117 | 118 | When you finish exploring Azure Machine Learning, you should delete the resources you've created to avoid unnecessary Azure costs. 119 | 120 | 1. Close the Azure Machine Learning studio tab and return to the Azure portal. 121 | 1. In the Azure portal, on the **Home** page, select **Resource groups**. 122 | 1. Select the **rg-dp100-...** resource group. 123 | 1. At the top of the **Overview** page for your resource group, select **Delete resource group**. 124 | 1. Enter the resource group name to confirm you want to delete it, and select **Delete**. 125 | -------------------------------------------------------------------------------- /Instructions/images/03-01-01-setup.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrosoftLearning/mslearn-azure-ml/9e0bfc3bfda27097e2c744952f1f33205fed6723/Instructions/images/03-01-01-setup.png -------------------------------------------------------------------------------- /Instructions/images/04-01-completed-pipeline.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrosoftLearning/mslearn-azure-ml/9e0bfc3bfda27097e2c744952f1f33205fed6723/Instructions/images/04-01-completed-pipeline.png -------------------------------------------------------------------------------- /Instructions/images/04-02-submitted-jobs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrosoftLearning/mslearn-azure-ml/9e0bfc3bfda27097e2c744952f1f33205fed6723/Instructions/images/04-02-submitted-jobs.png -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 Sidney Andrews 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /Labs/02/Run training script.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "attachments": {}, 5 | "cell_type": "markdown", 6 | "metadata": {}, 7 | "source": [ 8 | "# Run a training script with the Python SDK\n", 9 | "\n", 10 | "You can use the Python SDK for Azure Machine Learning to submit scripts as jobs. By using jobs, you can easily keep track of the input parameters and outputs when training a machine learning model.\n", 11 | "\n", 12 | "## Before you start\n", 13 | "\n", 14 | "You'll need the latest version of the **azure-ai-ml** package to run the code in this notebook. Run the cell below to verify that it is installed.\n", 15 | "\n", 16 | "> **Note**:\n", 17 | "> If the **azure-ai-ml** package is not installed, run `pip install azure-ai-ml` to install it." 18 | ] 19 | }, 20 | { 21 | "cell_type": "code", 22 | "execution_count": null, 23 | "metadata": { 24 | "vscode": { 25 | "languageId": "python" 26 | } 27 | }, 28 | "outputs": [], 29 | "source": [ 30 | "pip show azure-ai-ml" 31 | ] 32 | }, 33 | { 34 | "attachments": {}, 35 | "cell_type": "markdown", 36 | "metadata": {}, 37 | "source": [ 38 | "## Connect to your workspace\n", 39 | "\n", 40 | "With the required SDK packages installed, now you're ready to connect to your workspace.\n", 41 | "\n", 42 | "To connect to a workspace, we need identifier parameters - a subscription ID, resource group name, and workspace name. Since you're working with a compute instance, managed by Azure Machine Learning, you can use the default values to connect to the workspace." 43 | ] 44 | }, 45 | { 46 | "cell_type": "code", 47 | "execution_count": null, 48 | "metadata": { 49 | "gather": { 50 | "logged": 1663753569264 51 | }, 52 | "vscode": { 53 | "languageId": "python" 54 | } 55 | }, 56 | "outputs": [], 57 | "source": [ 58 | "from azure.identity import DefaultAzureCredential, InteractiveBrowserCredential\n", 59 | "from azure.ai.ml import MLClient\n", 60 | "\n", 61 | "try:\n", 62 | " credential = DefaultAzureCredential()\n", 63 | " # Check if given credential can get token successfully.\n", 64 | " credential.get_token(\"https://management.azure.com/.default\")\n", 65 | "except Exception as ex:\n", 66 | " # Fall back to InteractiveBrowserCredential in case DefaultAzureCredential not work\n", 67 | " credential = InteractiveBrowserCredential()" 68 | ] 69 | }, 70 | { 71 | "cell_type": "code", 72 | "execution_count": null, 73 | "metadata": { 74 | "vscode": { 75 | "languageId": "python" 76 | } 77 | }, 78 | "outputs": [], 79 | "source": [ 80 | "# Get a handle to workspace\n", 81 | "ml_client = MLClient.from_config(credential=credential)" 82 | ] 83 | }, 84 | { 85 | "attachments": {}, 86 | "cell_type": "markdown", 87 | "metadata": {}, 88 | "source": [ 89 | "## Use the Python SDK to train a model\n", 90 | "\n", 91 | "To train a model, you'll first create the **diabetes_training.py** script in the **src** folder. The script uses the **diabetes.csv** file in the same folder as the training data." 92 | ] 93 | }, 94 | { 95 | "cell_type": "code", 96 | "execution_count": null, 97 | "metadata": { 98 | "vscode": { 99 | "languageId": "python" 100 | } 101 | }, 102 | "outputs": [], 103 | "source": [ 104 | "%%writefile src/diabetes-training.py\n", 105 | "# import libraries\n", 106 | "import pandas as pd\n", 107 | "import numpy as np\n", 108 | "from sklearn.model_selection import train_test_split\n", 109 | "from sklearn.linear_model import LogisticRegression\n", 110 | "from sklearn.metrics import roc_auc_score\n", 111 | "from sklearn.metrics import roc_curve\n", 112 | "\n", 113 | "# load the diabetes dataset\n", 114 | "print(\"Loading Data...\")\n", 115 | "diabetes = pd.read_csv('diabetes.csv')\n", 116 | "\n", 117 | "# separate features and labels\n", 118 | "X, y = diabetes[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, diabetes['Diabetic'].values\n", 119 | "\n", 120 | "# split data into training set and test set\n", 121 | "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)\n", 122 | "\n", 123 | "# set regularization hyperparameter\n", 124 | "reg = 0.01\n", 125 | "\n", 126 | "# train a logistic regression model\n", 127 | "print('Training a logistic regression model with regularization rate of', reg)\n", 128 | "model = LogisticRegression(C=1/reg, solver=\"liblinear\").fit(X_train, y_train)\n", 129 | "\n", 130 | "# calculate accuracy\n", 131 | "y_hat = model.predict(X_test)\n", 132 | "acc = np.average(y_hat == y_test)\n", 133 | "print('Accuracy:', acc)\n", 134 | "\n", 135 | "# calculate AUC\n", 136 | "y_scores = model.predict_proba(X_test)\n", 137 | "auc = roc_auc_score(y_test,y_scores[:,1])\n", 138 | "print('AUC: ' + str(auc))\n" 139 | ] 140 | }, 141 | { 142 | "attachments": {}, 143 | "cell_type": "markdown", 144 | "metadata": {}, 145 | "source": [ 146 | "Run the cell below to submit the job that trains a classification model to predict diabetes. " 147 | ] 148 | }, 149 | { 150 | "cell_type": "code", 151 | "execution_count": null, 152 | "metadata": { 153 | "vscode": { 154 | "languageId": "python" 155 | } 156 | }, 157 | "outputs": [], 158 | "source": [ 159 | "from azure.ai.ml import command\n", 160 | "\n", 161 | "# configure job\n", 162 | "job = command(\n", 163 | " code=\"./src\",\n", 164 | " command=\"python diabetes-training.py\",\n", 165 | " environment=\"AzureML-sklearn-0.24-ubuntu18.04-py37-cpu@latest\",\n", 166 | " compute=\"aml-cluster\",\n", 167 | " display_name=\"diabetes-pythonv2-train\",\n", 168 | " experiment_name=\"diabetes-training\"\n", 169 | ")\n", 170 | "\n", 171 | "# submit job\n", 172 | "returned_job = ml_client.create_or_update(job)\n", 173 | "aml_url = returned_job.studio_url\n", 174 | "print(\"Monitor your job at\", aml_url)" 175 | ] 176 | } 177 | ], 178 | "metadata": { 179 | "kernel_info": { 180 | "name": "python38-azureml" 181 | }, 182 | "kernelspec": { 183 | "display_name": "Python 3.8 - AzureML", 184 | "language": "python", 185 | "name": "python38-azureml" 186 | }, 187 | "microsoft": { 188 | "host": { 189 | "AzureML": { 190 | "notebookHasBeenCompleted": true 191 | } 192 | } 193 | }, 194 | "nteract": { 195 | "version": "nteract-front-end@1.0.0" 196 | }, 197 | "vscode": { 198 | "interpreter": { 199 | "hash": "f2b2cd046deda8eabef1e765a11d0ec9aa9bd1d31d56ce79c815a38c323e14ec" 200 | } 201 | } 202 | }, 203 | "nbformat": 4, 204 | "nbformat_minor": 2 205 | } 206 | -------------------------------------------------------------------------------- /Labs/02/diabetes-data.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrosoftLearning/mslearn-azure-ml/9e0bfc3bfda27097e2c744952f1f33205fed6723/Labs/02/diabetes-data.zip -------------------------------------------------------------------------------- /Labs/03/data/MLTable: -------------------------------------------------------------------------------- 1 | # MLTable definition file 2 | 3 | paths: 4 | - file: ./diabetes.csv 5 | transformations: 6 | - read_delimited: 7 | delimiter: ',' 8 | encoding: 'ascii' -------------------------------------------------------------------------------- /Labs/03/setup.sh: -------------------------------------------------------------------------------- 1 | #! /usr/bin/sh 2 | 3 | # Create random string 4 | guid=$(cat /proc/sys/kernel/random/uuid) 5 | suffix=${guid//[-]/} 6 | suffix=${suffix:0:18} 7 | 8 | # Set the necessary variables 9 | RESOURCE_GROUP="rg-dp100-l${suffix}" 10 | RESOURCE_PROVIDER="Microsoft.MachineLearningServices" 11 | REGIONS=("eastus" "westus" "centralus" "northeurope" "westeurope") 12 | RANDOM_REGION=${REGIONS[$RANDOM % ${#REGIONS[@]}]} 13 | WORKSPACE_NAME="mlw-dp100-l${suffix}" 14 | COMPUTE_INSTANCE="ci${suffix}" 15 | COMPUTE_CLUSTER="aml-cluster" 16 | 17 | # Register the Azure Machine Learning resource provider in the subscription 18 | echo "Register the Machine Learning resource provider:" 19 | az provider register --namespace $RESOURCE_PROVIDER 20 | 21 | # Create the resource group and workspace and set to default 22 | echo "Create a resource group and set as default:" 23 | az group create --name $RESOURCE_GROUP --location $RANDOM_REGION 24 | az configure --defaults group=$RESOURCE_GROUP 25 | 26 | echo "Create an Azure Machine Learning workspace:" 27 | az ml workspace create --name $WORKSPACE_NAME 28 | az configure --defaults workspace=$WORKSPACE_NAME 29 | 30 | # Create compute instance 31 | echo "Creating a compute instance with name: " $COMPUTE_INSTANCE 32 | az ml compute create --name ${COMPUTE_INSTANCE} --size STANDARD_DS11_V2 --type ComputeInstance 33 | 34 | # Create compute cluster 35 | echo "Creating a compute cluster with name: " $COMPUTE_CLUSTER 36 | az ml compute create --name ${COMPUTE_CLUSTER} --size STANDARD_DS11_V2 --max-instances 2 --type AmlCompute 37 | -------------------------------------------------------------------------------- /Labs/04/images/screenshot-logs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrosoftLearning/mslearn-azure-ml/9e0bfc3bfda27097e2c744952f1f33205fed6723/Labs/04/images/screenshot-logs.png -------------------------------------------------------------------------------- /Labs/04/setup.sh: -------------------------------------------------------------------------------- 1 | #! /usr/bin/sh 2 | 3 | # Create random string 4 | guid=$(cat /proc/sys/kernel/random/uuid) 5 | suffix=${guid//[-]/} 6 | suffix=${suffix:0:18} 7 | 8 | # Set the necessary variables 9 | RESOURCE_GROUP="rg-dp100-l${suffix}" 10 | RESOURCE_PROVIDER="Microsoft.MachineLearningServices" 11 | REGIONS=("eastus" "westus" "centralus" "northeurope" "westeurope") 12 | RANDOM_REGION=${REGIONS[$RANDOM % ${#REGIONS[@]}]} 13 | WORKSPACE_NAME="mlw-dp100-l${suffix}" 14 | COMPUTE_INSTANCE="ci${suffix}" 15 | COMPUTE_CLUSTER="aml-cluster" 16 | 17 | # Register the Azure Machine Learning resource provider in the subscription 18 | echo "Register the Machine Learning resource provider:" 19 | az provider register --namespace $RESOURCE_PROVIDER 20 | 21 | # Create the resource group and workspace and set to default 22 | echo "Create a resource group and set as default:" 23 | az group create --name $RESOURCE_GROUP --location $RANDOM_REGION 24 | az configure --defaults group=$RESOURCE_GROUP 25 | 26 | echo "Create an Azure Machine Learning workspace:" 27 | az ml workspace create --name $WORKSPACE_NAME 28 | az configure --defaults workspace=$WORKSPACE_NAME 29 | 30 | # Create compute instance 31 | echo "Creating a compute instance with name: " $COMPUTE_INSTANCE 32 | az ml compute create --name ${COMPUTE_INSTANCE} --size STANDARD_DS11_V2 --type ComputeInstance 33 | 34 | # Create compute cluster 35 | echo "Creating a compute cluster with name: " $COMPUTE_CLUSTER 36 | az ml compute create --name ${COMPUTE_CLUSTER} --size STANDARD_DS11_V2 --max-instances 2 --type AmlCompute 37 | -------------------------------------------------------------------------------- /Labs/05/fix-missing-data.yml: -------------------------------------------------------------------------------- 1 | $schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json 2 | name: remove_empty_rows 3 | display_name: Remove Empty Rows 4 | version: 1 5 | type: command 6 | inputs: 7 | input_data: 8 | type: uri_folder 9 | outputs: 10 | output_data: 11 | type: uri_folder 12 | code: ./src 13 | environment: azureml:AzureML-sklearn-0.24-ubuntu18.04-py37-cpu@latest 14 | command: >- 15 | python fix-missing-data.py 16 | --input_data ${{inputs.input_data}} 17 | --output_data ${{outputs.output_data}} 18 | 19 | -------------------------------------------------------------------------------- /Labs/05/normalize-data.yml: -------------------------------------------------------------------------------- 1 | # 2 | $schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json 3 | name: normalize_data 4 | display_name: Normalize Numerical Columns 5 | version: 1 6 | type: command 7 | inputs: 8 | input_data: 9 | type: uri_folder 10 | outputs: 11 | output_data: 12 | type: uri_folder 13 | code: ./src 14 | environment: azureml:AzureML-sklearn-0.24-ubuntu18.04-py37-cpu@latest 15 | command: >- 16 | python normalize-data.py 17 | --input_data ${{inputs.input_data}} 18 | --output_data ${{outputs.output_data}} 19 | # 20 | -------------------------------------------------------------------------------- /Labs/05/setup.sh: -------------------------------------------------------------------------------- 1 | #! /usr/bin/sh 2 | 3 | # Create random string 4 | guid=$(cat /proc/sys/kernel/random/uuid) 5 | suffix=${guid//[-]/} 6 | suffix=${suffix:0:18} 7 | 8 | # Set the necessary variables 9 | RESOURCE_GROUP="rg-dp100-l${suffix}" 10 | RESOURCE_PROVIDER="Microsoft.MachineLearningServices" 11 | REGIONS=("eastus" "westus" "centralus" "northeurope" "westeurope") 12 | RANDOM_REGION=${REGIONS[$RANDOM % ${#REGIONS[@]}]} 13 | WORKSPACE_NAME="mlw-dp100-l${suffix}" 14 | COMPUTE_CLUSTER="aml-cluster" 15 | 16 | # Register the Azure Machine Learning resource provider in the subscription 17 | echo "Register the Machine Learning resource provider:" 18 | az provider register --namespace $RESOURCE_PROVIDER 19 | 20 | # Create the resource group and workspace and set to default 21 | echo "Create a resource group and set as default:" 22 | az group create --name $RESOURCE_GROUP --location $RANDOM_REGION 23 | az configure --defaults group=$RESOURCE_GROUP 24 | 25 | echo "Create an Azure Machine Learning workspace:" 26 | az ml workspace create --name $WORKSPACE_NAME 27 | az configure --defaults workspace=$WORKSPACE_NAME 28 | 29 | # Create compute cluster 30 | echo "Creating a compute cluster with name: " $COMPUTE_CLUSTER 31 | az ml compute create --name ${COMPUTE_CLUSTER} --size STANDARD_DS11_V2 --max-instances 2 --type AmlCompute 32 | 33 | # Create data asset 34 | echo "Creating a data asset with name: diabetes-folder" 35 | az ml data create --name diabetes-folder --path ./data 36 | 37 | # Create components 38 | echo "Creating components" 39 | az ml component create --file ./fix-missing-data.yml 40 | az ml component create --file ./normalize-data.yml 41 | az ml component create --file ./train-decision-tree.yml 42 | az ml component create --file ./train-logistic-regression.yml 43 | -------------------------------------------------------------------------------- /Labs/05/src/fix-missing-data.py: -------------------------------------------------------------------------------- 1 | # import libraries 2 | import argparse 3 | import glob 4 | from pathlib import Path 5 | import pandas as pd 6 | import mlflow 7 | 8 | # get parameters 9 | parser = argparse.ArgumentParser() 10 | parser.add_argument("--input_data", type=str, help='Path to input data') 11 | parser.add_argument('--output_data', type=str, help='Path of output data') 12 | args = parser.parse_args() 13 | 14 | # load the data (passed as an input dataset) 15 | data_path = args.input_data 16 | all_files = glob.glob(data_path + "/*.csv") 17 | df = pd.concat((pd.read_csv(f) for f in all_files), sort=False) 18 | 19 | # log row count input data 20 | row_count = (len(df)) 21 | mlflow.log_metric('row count input data', row_count) 22 | 23 | # remove nulls 24 | df = df.dropna() 25 | 26 | # log processed rows 27 | row_count_processed = (len(df)) 28 | mlflow.log_metric('row count output data', row_count_processed) 29 | 30 | # set the processed data as output 31 | output_df = df.to_csv((Path(args.output_data) / "output_data.csv")) -------------------------------------------------------------------------------- /Labs/05/src/normalize-data.py: -------------------------------------------------------------------------------- 1 | # import libraries 2 | import argparse 3 | import os 4 | import glob 5 | from pathlib import Path 6 | import pandas as pd 7 | import mlflow 8 | from sklearn.preprocessing import MinMaxScaler 9 | 10 | # get parameters 11 | parser = argparse.ArgumentParser() 12 | parser.add_argument("--input_data", type=str, help='Path to input data') 13 | parser.add_argument('--output_data', type=str, help='Path of output data') 14 | args = parser.parse_args() 15 | 16 | # load the data (passed as an input dataset) 17 | print("files in input_data path: ") 18 | arr = os.listdir(args.input_data) 19 | print(arr) 20 | 21 | for filename in arr: 22 | print("reading file: %s ..." % filename) 23 | with open(os.path.join(args.input_data, filename), "r") as handle: 24 | print(handle.read()) 25 | 26 | data_path = args.input_data 27 | all_files = glob.glob(data_path + "/*.csv") 28 | df = pd.concat((pd.read_csv(f) for f in all_files), sort=False) 29 | 30 | # log row count input data 31 | row_count = (len(df)) 32 | mlflow.log_metric('row count input data', row_count) 33 | 34 | # normalize the numeric columns 35 | scaler = MinMaxScaler() 36 | num_cols = ['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree'] 37 | df[num_cols] = scaler.fit_transform(df[num_cols]) 38 | 39 | # log processed rows 40 | row_count_processed = (len(df)) 41 | mlflow.log_metric('row count output data', row_count_processed) 42 | 43 | # set the processed data as output 44 | output_df = df.to_csv((Path(args.output_data) / "output_data.csv")) -------------------------------------------------------------------------------- /Labs/05/src/train-decision-tree.py: -------------------------------------------------------------------------------- 1 | # Import libraries 2 | import argparse 3 | import glob 4 | import pickle 5 | from pathlib import Path 6 | 7 | import matplotlib.pyplot as plt 8 | import mlflow 9 | import numpy as np 10 | import pandas as pd 11 | from sklearn.metrics import confusion_matrix, roc_auc_score, roc_curve 12 | from sklearn.model_selection import train_test_split 13 | from sklearn.tree import DecisionTreeClassifier 14 | 15 | # get parameters 16 | parser = argparse.ArgumentParser("train") 17 | parser.add_argument("--training_data", type=str, help="Path to training data") 18 | parser.add_argument("--model_output", type=str, help="Path of output model") 19 | 20 | args = parser.parse_args() 21 | 22 | training_data = args.training_data 23 | 24 | # load the prepared data file in the training folder 25 | print("Loading Data...") 26 | data_path = args.training_data 27 | all_files = glob.glob(data_path + "/*.csv") 28 | df = pd.concat((pd.read_csv(f) for f in all_files), sort=False) 29 | 30 | # Separate features and labels 31 | X, y = ( 32 | df[ 33 | [ 34 | "Pregnancies", 35 | "PlasmaGlucose", 36 | "DiastolicBloodPressure", 37 | "TricepsThickness", 38 | "SerumInsulin", 39 | "BMI", 40 | "DiabetesPedigree", 41 | "Age", 42 | ] 43 | ].values, 44 | df["Diabetic"].values, 45 | ) 46 | 47 | # Split data into training set and test set 48 | X_train, X_test, y_train, y_test = train_test_split( 49 | X, y, test_size=0.30, random_state=0 50 | ) 51 | 52 | # Train a decision tree model 53 | print('Training a decision tree model...') 54 | model = DecisionTreeClassifier().fit(X_train, y_train) 55 | 56 | # Calculate accuracy 57 | y_pred = model.predict(X_test) 58 | acc = np.average(y_pred == y_test) 59 | print("Accuracy:", acc) 60 | mlflow.log_metric("Accuracy", np.float(acc)) 61 | 62 | # Calculate AUC 63 | y_pred_proba = model.predict_proba(X_test) 64 | auc = roc_auc_score(y_test, y_pred_proba[:, 1]) 65 | print("AUC: " + str(auc)) 66 | mlflow.log_metric("AUC", np.float(auc)) 67 | 68 | # Plot ROC curve 69 | fpr, tpr, thresholds = roc_curve(y_test, y_pred_proba[:, 1]) 70 | fig = plt.figure(figsize=(6, 4)) 71 | # Plot the diagonal 50% line 72 | plt.plot([0, 1], [0, 1], "k--") 73 | # Plot the FPR and TPR achieved by our model 74 | plt.plot(fpr, tpr) 75 | plt.xlabel("False Positive Rate") 76 | plt.ylabel("True Positive Rate") 77 | plt.title("ROC Curve") 78 | plt.savefig("ROCcurve.png") 79 | mlflow.log_artifact("ROCcurve.png") 80 | 81 | # Create confusion matrix 82 | conf_matrix = confusion_matrix(y_true=y_test, y_pred=y_pred) 83 | fig, ax = plt.subplots(figsize=(7.5, 7.5)) 84 | ax.matshow(conf_matrix, cmap=plt.cm.Blues, alpha=0.3) 85 | for i in range(conf_matrix.shape[0]): 86 | for j in range(conf_matrix.shape[1]): 87 | ax.text( 88 | x=j, y=i, s=conf_matrix[i, j], va="center", ha="center", 89 | size="xx-large" 90 | ) 91 | 92 | plt.xlabel("Predictions", fontsize=18) 93 | plt.ylabel("Actuals", fontsize=18) 94 | plt.title("Confusion Matrix", fontsize=18) 95 | plt.savefig("ConfusionMatrix.png") 96 | mlflow.log_artifact("ConfusionMatrix.png") 97 | 98 | # Output the model and test data 99 | pickle.dump(model, open((Path(args.model_output) / "model.sav"), "wb")) -------------------------------------------------------------------------------- /Labs/05/src/train-logistic-regression.py: -------------------------------------------------------------------------------- 1 | # Import libraries 2 | import argparse 3 | import glob 4 | import pickle 5 | from pathlib import Path 6 | 7 | import matplotlib.pyplot as plt 8 | import mlflow 9 | import numpy as np 10 | import pandas as pd 11 | from sklearn.linear_model import LogisticRegression 12 | from sklearn.metrics import confusion_matrix, roc_auc_score, roc_curve 13 | from sklearn.model_selection import train_test_split 14 | 15 | # get parameters 16 | parser = argparse.ArgumentParser("train") 17 | parser.add_argument("--training_data", type=str, help="Path to training data") 18 | parser.add_argument("--reg_rate", type=float, default=0.01) 19 | parser.add_argument("--model_output", type=str, help="Path of output model") 20 | 21 | args = parser.parse_args() 22 | 23 | training_data = args.training_data 24 | 25 | # load the prepared data file in the training folder 26 | print("Loading Data...") 27 | data_path = args.training_data 28 | all_files = glob.glob(data_path + "/*.csv") 29 | df = pd.concat((pd.read_csv(f) for f in all_files), sort=False) 30 | 31 | # Separate features and labels 32 | X, y = ( 33 | df[ 34 | [ 35 | "Pregnancies", 36 | "PlasmaGlucose", 37 | "DiastolicBloodPressure", 38 | "TricepsThickness", 39 | "SerumInsulin", 40 | "BMI", 41 | "DiabetesPedigree", 42 | "Age", 43 | ] 44 | ].values, 45 | df["Diabetic"].values, 46 | ) 47 | 48 | # Split data into training set and test set 49 | X_train, X_test, y_train, y_test = train_test_split( 50 | X, y, test_size=0.30, random_state=0 51 | ) 52 | 53 | # Train a logistic regression model 54 | print('Training a logistic regression model...') 55 | model = LogisticRegression(C=1 / args.reg_rate, solver="liblinear").fit( 56 | X_train, y_train 57 | ) 58 | 59 | # Calculate accuracy 60 | y_pred = model.predict(X_test) 61 | acc = np.average(y_pred == y_test) 62 | print("Accuracy:", acc) 63 | mlflow.log_metric("Accuracy", np.float(acc)) 64 | 65 | # Calculate AUC 66 | y_pred_proba = model.predict_proba(X_test) 67 | auc = roc_auc_score(y_test, y_pred_proba[:, 1]) 68 | print("AUC: " + str(auc)) 69 | mlflow.log_metric("AUC", np.float(auc)) 70 | 71 | # Plot ROC curve 72 | fpr, tpr, thresholds = roc_curve(y_test, y_pred_proba[:, 1]) 73 | fig = plt.figure(figsize=(6, 4)) 74 | # Plot the diagonal 50% line 75 | plt.plot([0, 1], [0, 1], "k--") 76 | # Plot the FPR and TPR achieved by our model 77 | plt.plot(fpr, tpr) 78 | plt.xlabel("False Positive Rate") 79 | plt.ylabel("True Positive Rate") 80 | plt.title("ROC Curve") 81 | plt.savefig("ROCcurve.png") 82 | mlflow.log_artifact("ROCcurve.png") 83 | 84 | # Create confusion matrix 85 | conf_matrix = confusion_matrix(y_true=y_test, y_pred=y_pred) 86 | fig, ax = plt.subplots(figsize=(7.5, 7.5)) 87 | ax.matshow(conf_matrix, cmap=plt.cm.Blues, alpha=0.3) 88 | for i in range(conf_matrix.shape[0]): 89 | for j in range(conf_matrix.shape[1]): 90 | ax.text( 91 | x=j, y=i, s=conf_matrix[i, j], va="center", ha="center", 92 | size="xx-large" 93 | ) 94 | 95 | plt.xlabel("Predictions", fontsize=18) 96 | plt.ylabel("Actuals", fontsize=18) 97 | plt.title("Confusion Matrix", fontsize=18) 98 | plt.savefig("ConfusionMatrix.png") 99 | mlflow.log_artifact("ConfusionMatrix.png") 100 | 101 | # Output the model and test data 102 | pickle.dump(model, open((Path(args.model_output) / "model.sav"), "wb")) -------------------------------------------------------------------------------- /Labs/05/train-decision-tree.yml: -------------------------------------------------------------------------------- 1 | # 2 | $schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json 3 | name: train_decision_tree_classifier_model 4 | display_name: Train a Decision Tree Classifier Model 5 | version: 1 6 | type: command 7 | inputs: 8 | training_data: 9 | type: uri_folder 10 | outputs: 11 | model_output: 12 | type: uri_folder 13 | code: ./src 14 | environment: azureml:AzureML-sklearn-0.24-ubuntu18.04-py37-cpu@latest 15 | command: >- 16 | python train-decision-tree.py 17 | --training_data ${{inputs.training_data}} 18 | --model_output ${{outputs.model_output}} 19 | # 20 | -------------------------------------------------------------------------------- /Labs/05/train-logistic-regression.yml: -------------------------------------------------------------------------------- 1 | # 2 | $schema: https://azuremlschemas.azureedge.net/latest/commandComponent.schema.json 3 | name: train_logistic_regression_classifier_model 4 | display_name: Train a Logistic Regression Classifier Model 5 | version: 1 6 | type: command 7 | inputs: 8 | training_data: 9 | type: uri_folder 10 | regularization_rate: 11 | type: number 12 | default: 0.01 13 | outputs: 14 | model_output: 15 | type: uri_folder 16 | code: ./src 17 | environment: azureml:AzureML-sklearn-0.24-ubuntu18.04-py37-cpu@latest 18 | command: >- 19 | python train-logistic-regression.py 20 | --training_data ${{inputs.training_data}} 21 | --reg_rate ${{inputs.regularization_rate}} 22 | --model_output ${{outputs.model_output}} 23 | # 24 | -------------------------------------------------------------------------------- /Labs/06/Classification with Automated Machine Learning.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "attachments": {}, 5 | "cell_type": "markdown", 6 | "metadata": {}, 7 | "source": [ 8 | "# Train a classification model with Automated Machine Learning\n", 9 | "\n", 10 | "There are many kinds of machine learning algorithm that you can use to train a model, and sometimes it's not easy to determine the most effective algorithm for your particular data and prediction requirements. Additionally, you can significantly affect the predictive performance of a model by preprocessing the training data, using techniques such as normalization, missing feature imputation, and others. In your quest to find the best model for your requirements, you may need to try many combinations of algorithms and preprocessing transformations; which takes a lot of time and compute resources.\n", 11 | "\n", 12 | "Azure Machine Learning enables you to automate the comparison of models trained using different algorithms and preprocessing options. You can use the visual interface in [Azure Machine Learning Studio](https://ml/azure.com) or the Python SDK (v2) to leverage this capability. The Python SDK gives you greater control over the settings for the automated machine learning job, but the visual interface is easier to use.\n", 13 | "\n", 14 | "## Before you start\n", 15 | "\n", 16 | "You'll need the latest version of the **azure-ai-ml** package to run the code in this notebook. Run the cell below to verify that it is installed.\n", 17 | "\n", 18 | "> **Note**:\n", 19 | "> If the **azure-ai-ml** package is not installed, run `pip install azure-ai-ml` to install it." 20 | ] 21 | }, 22 | { 23 | "cell_type": "code", 24 | "execution_count": null, 25 | "metadata": { 26 | "vscode": { 27 | "languageId": "python" 28 | } 29 | }, 30 | "outputs": [], 31 | "source": [ 32 | "pip show azure-ai-ml" 33 | ] 34 | }, 35 | { 36 | "attachments": {}, 37 | "cell_type": "markdown", 38 | "metadata": {}, 39 | "source": [ 40 | "## Connect to your workspace\n", 41 | "\n", 42 | "With the required SDK packages installed, now you're ready to connect to your workspace.\n", 43 | "\n", 44 | "To connect to a workspace, we need identifier parameters - a subscription ID, resource group name, and workspace name. Since you're working with a compute instance, managed by Azure Machine Learning, you can use the default values to connect to the workspace." 45 | ] 46 | }, 47 | { 48 | "cell_type": "code", 49 | "execution_count": null, 50 | "metadata": { 51 | "gather": { 52 | "logged": 1664965651163 53 | }, 54 | "jupyter": { 55 | "outputs_hidden": false, 56 | "source_hidden": false 57 | }, 58 | "nteract": { 59 | "transient": { 60 | "deleting": false 61 | } 62 | }, 63 | "vscode": { 64 | "languageId": "python" 65 | } 66 | }, 67 | "outputs": [], 68 | "source": [ 69 | "from azure.identity import DefaultAzureCredential, InteractiveBrowserCredential\n", 70 | "from azure.ai.ml import MLClient\n", 71 | "\n", 72 | "try:\n", 73 | " credential = DefaultAzureCredential()\n", 74 | " # Check if given credential can get token successfully.\n", 75 | " credential.get_token(\"https://management.azure.com/.default\")\n", 76 | "except Exception as ex:\n", 77 | " # Fall back to InteractiveBrowserCredential in case DefaultAzureCredential not work\n", 78 | " credential = InteractiveBrowserCredential()" 79 | ] 80 | }, 81 | { 82 | "cell_type": "code", 83 | "execution_count": null, 84 | "metadata": { 85 | "vscode": { 86 | "languageId": "python" 87 | } 88 | }, 89 | "outputs": [], 90 | "source": [ 91 | "# Get a handle to workspace\n", 92 | "ml_client = MLClient.from_config(credential=credential)" 93 | ] 94 | }, 95 | { 96 | "attachments": {}, 97 | "cell_type": "markdown", 98 | "metadata": {}, 99 | "source": [ 100 | "## Prepare data\n", 101 | "\n", 102 | "You don't need to create a training script for automated machine learning, but you do need to load the training data. \n", 103 | "\n", 104 | "In this case, you'll use a dataset containing details of diabetes patients. \n", 105 | "\n", 106 | "To pass a dataset as an input to an automated machine learning job, the data must be in tabular form and include a target column. For the data to be interpreted as a tabular dataset, the input dataset must be a **MLTable**.\n", 107 | "\n", 108 | "A MLTable data asset has already been created for you during set-up. You can explore the data asset by navigating to the **Data** page. You'll retrieve the data asset here by specifying its name `diabetes-training-table` and version `1`. " 109 | ] 110 | }, 111 | { 112 | "cell_type": "code", 113 | "execution_count": null, 114 | "metadata": { 115 | "gather": { 116 | "logged": 1664965655212 117 | }, 118 | "jupyter": { 119 | "outputs_hidden": false, 120 | "source_hidden": false 121 | }, 122 | "nteract": { 123 | "transient": { 124 | "deleting": false 125 | } 126 | }, 127 | "vscode": { 128 | "languageId": "python" 129 | } 130 | }, 131 | "outputs": [], 132 | "source": [ 133 | "from azure.ai.ml.constants import AssetTypes\n", 134 | "from azure.ai.ml import Input\n", 135 | "\n", 136 | "# creates a dataset based on the files in the local data folder\n", 137 | "my_training_data_input = Input(type=AssetTypes.MLTABLE, path=\"azureml:diabetes-training:1\")" 138 | ] 139 | }, 140 | { 141 | "attachments": {}, 142 | "cell_type": "markdown", 143 | "metadata": {}, 144 | "source": [ 145 | "## Configure automated machine learning job\n", 146 | "\n", 147 | "Now, you're ready to configure the automated machine learning experiment.\n", 148 | "\n", 149 | "When you run the code below, it will create an automated machine learning job that:\n", 150 | "\n", 151 | "- Uses the compute cluster named `aml-cluster`\n", 152 | "- Sets `Diabetic` as the target column\n", 153 | "- Sets `accuracy` as the primary metric\n", 154 | "- Times out after `60` minutes of total training time \n", 155 | "- Trains a maximum of `5` models\n", 156 | "- No model will be trained with the `LogisticRegression` algorithm" 157 | ] 158 | }, 159 | { 160 | "cell_type": "code", 161 | "execution_count": null, 162 | "metadata": { 163 | "gather": { 164 | "logged": 1664965734226 165 | }, 166 | "jupyter": { 167 | "outputs_hidden": false, 168 | "source_hidden": false 169 | }, 170 | "nteract": { 171 | "transient": { 172 | "deleting": false 173 | } 174 | }, 175 | "vscode": { 176 | "languageId": "python" 177 | } 178 | }, 179 | "outputs": [], 180 | "source": [ 181 | "from azure.ai.ml import automl\n", 182 | "\n", 183 | "# configure the classification job\n", 184 | "classification_job = automl.classification(\n", 185 | " compute=\"aml-cluster\",\n", 186 | " experiment_name=\"auto-ml-class-dev\",\n", 187 | " training_data=my_training_data_input,\n", 188 | " target_column_name=\"Diabetic\",\n", 189 | " primary_metric=\"accuracy\",\n", 190 | " n_cross_validations=5,\n", 191 | " enable_model_explainability=True\n", 192 | ")\n", 193 | "\n", 194 | "# set the limits (optional)\n", 195 | "classification_job.set_limits(\n", 196 | " timeout_minutes=60, \n", 197 | " trial_timeout_minutes=20, \n", 198 | " max_trials=5,\n", 199 | " enable_early_termination=True,\n", 200 | ")\n", 201 | "\n", 202 | "# set the training properties (optional)\n", 203 | "classification_job.set_training(\n", 204 | " blocked_training_algorithms=[\"LogisticRegression\"], \n", 205 | " enable_onnx_compatible_models=True\n", 206 | ")" 207 | ] 208 | }, 209 | { 210 | "attachments": {}, 211 | "cell_type": "markdown", 212 | "metadata": {}, 213 | "source": [ 214 | "## Run an automated machine learning job\n", 215 | "\n", 216 | "OK, you're ready to go. Let's run the automated machine learning experiment.\n", 217 | "\n", 218 | "> **Note**: This may take some time!" 219 | ] 220 | }, 221 | { 222 | "cell_type": "code", 223 | "execution_count": null, 224 | "metadata": { 225 | "gather": { 226 | "logged": 1664965739262 227 | }, 228 | "jupyter": { 229 | "outputs_hidden": false, 230 | "source_hidden": false 231 | }, 232 | "nteract": { 233 | "transient": { 234 | "deleting": false 235 | } 236 | }, 237 | "vscode": { 238 | "languageId": "python" 239 | } 240 | }, 241 | "outputs": [], 242 | "source": [ 243 | "# Submit the AutoML job\n", 244 | "returned_job = ml_client.jobs.create_or_update(\n", 245 | " classification_job\n", 246 | ") \n", 247 | "\n", 248 | "# submit the job to the backend\n", 249 | "aml_url = returned_job.studio_url\n", 250 | "print(\"Monitor your job at\", aml_url)" 251 | ] 252 | }, 253 | { 254 | "attachments": {}, 255 | "cell_type": "markdown", 256 | "metadata": {}, 257 | "source": [ 258 | "While the job is running, you can monitor it in the Studio." 259 | ] 260 | } 261 | ], 262 | "metadata": { 263 | "kernel_info": { 264 | "name": "python38-azureml" 265 | }, 266 | "kernelspec": { 267 | "display_name": "Python 3.8 - AzureML", 268 | "language": "python", 269 | "name": "python38-azureml" 270 | }, 271 | "microsoft": { 272 | "host": { 273 | "AzureML": { 274 | "notebookHasBeenCompleted": true 275 | } 276 | } 277 | }, 278 | "nteract": { 279 | "version": "nteract-front-end@1.0.0" 280 | }, 281 | "vscode": { 282 | "interpreter": { 283 | "hash": "f2b2cd046deda8eabef1e765a11d0ec9aa9bd1d31d56ce79c815a38c323e14ec" 284 | } 285 | } 286 | }, 287 | "nbformat": 4, 288 | "nbformat_minor": 2 289 | } 290 | -------------------------------------------------------------------------------- /Labs/06/diabetes-data/MLTable: -------------------------------------------------------------------------------- 1 | # MLTable definition file 2 | 3 | paths: 4 | - file: ./diabetes.csv 5 | transformations: 6 | - read_delimited: 7 | delimiter: ',' 8 | encoding: 'ascii' 9 | -------------------------------------------------------------------------------- /Labs/06/orange-juice-data/MLTable: -------------------------------------------------------------------------------- 1 | # MLTable definition file 2 | 3 | paths: 4 | - file: ./orange-juice-data.csv 5 | transformations: 6 | - read_delimited: 7 | delimiter: ',' 8 | encoding: 'ascii' 9 | -------------------------------------------------------------------------------- /Labs/06/setup.sh: -------------------------------------------------------------------------------- 1 | #! /usr/bin/sh 2 | 3 | # Create random string 4 | guid=$(cat /proc/sys/kernel/random/uuid) 5 | suffix=${guid//[-]/} 6 | suffix=${suffix:0:18} 7 | 8 | # Set the necessary variables 9 | RESOURCE_GROUP="rg-dp100-l${suffix}" 10 | RESOURCE_PROVIDER="Microsoft.MachineLearningServices" 11 | REGIONS=("eastus" "westus" "centralus" "northeurope" "westeurope") 12 | RANDOM_REGION=${REGIONS[$RANDOM % ${#REGIONS[@]}]} 13 | WORKSPACE_NAME="mlw-dp100-l${suffix}" 14 | COMPUTE_INSTANCE="ci${suffix}" 15 | COMPUTE_CLUSTER="aml-cluster" 16 | 17 | # Register the Azure Machine Learning resource provider in the subscription 18 | echo "Register the Machine Learning resource provider:" 19 | az provider register --namespace $RESOURCE_PROVIDER 20 | 21 | # Create the resource group and workspace and set to default 22 | echo "Create a resource group and set as default:" 23 | az group create --name $RESOURCE_GROUP --location $RANDOM_REGION 24 | az configure --defaults group=$RESOURCE_GROUP 25 | 26 | echo "Create an Azure Machine Learning workspace:" 27 | az ml workspace create --name $WORKSPACE_NAME 28 | az configure --defaults workspace=$WORKSPACE_NAME 29 | 30 | # Create compute instance 31 | echo "Creating a compute instance with name: " $COMPUTE_INSTANCE 32 | az ml compute create --name ${COMPUTE_INSTANCE} --size STANDARD_DS11_V2 --type ComputeInstance 33 | 34 | # Create compute cluster 35 | echo "Creating a compute cluster with name: " $COMPUTE_CLUSTER 36 | az ml compute create --name ${COMPUTE_CLUSTER} --size STANDARD_DS11_V2 --max-instances 2 --type AmlCompute 37 | 38 | # Create data assets 39 | echo "Create training data asset:" 40 | az ml data create --type mltable --name "diabetes-training" --path ./diabetes-data 41 | az ml data create --type mltable --name "oj-training" --path ./orange-juice-data 42 | -------------------------------------------------------------------------------- /Labs/07/images/output-logs.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrosoftLearning/mslearn-azure-ml/9e0bfc3bfda27097e2c744952f1f33205fed6723/Labs/07/images/output-logs.png -------------------------------------------------------------------------------- /Labs/07/setup.sh: -------------------------------------------------------------------------------- 1 | #! /usr/bin/sh 2 | 3 | # Create random string 4 | guid=$(cat /proc/sys/kernel/random/uuid) 5 | suffix=${guid//[-]/} 6 | suffix=${suffix:0:18} 7 | 8 | # Set the necessary variables 9 | RESOURCE_GROUP="rg-dp100-l${suffix}" 10 | RESOURCE_PROVIDER="Microsoft.MachineLearningServices" 11 | REGIONS=("eastus" "westus" "centralus" "northeurope" "westeurope") 12 | RANDOM_REGION=${REGIONS[$RANDOM % ${#REGIONS[@]}]} 13 | WORKSPACE_NAME="mlw-dp100-l${suffix}" 14 | COMPUTE_INSTANCE="ci${suffix}" 15 | 16 | # Register the Azure Machine Learning resource provider in the subscription 17 | echo "Register the Machine Learning resource provider:" 18 | az provider register --namespace $RESOURCE_PROVIDER 19 | 20 | # Create the resource group and workspace and set to default 21 | echo "Create a resource group and set as default:" 22 | az group create --name $RESOURCE_GROUP --location $RANDOM_REGION 23 | az configure --defaults group=$RESOURCE_GROUP 24 | 25 | echo "Create an Azure Machine Learning workspace:" 26 | az ml workspace create --name $WORKSPACE_NAME 27 | az configure --defaults workspace=$WORKSPACE_NAME 28 | 29 | # Create compute instance 30 | echo "Creating a compute instance with name: " $COMPUTE_INSTANCE 31 | az ml compute create --name ${COMPUTE_INSTANCE} --size STANDARD_DS11_V2 --type ComputeInstance 32 | 33 | -------------------------------------------------------------------------------- /Labs/08/Run script as a command job.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "attachments": {}, 5 | "cell_type": "markdown", 6 | "metadata": {}, 7 | "source": [ 8 | "# Run a training script as a command job\n", 9 | "\n", 10 | "You can use the Python SDK for Azure Machine Learning to submit scripts as command jobs. By using jobs, you can easily keep track of the input parameters and outputs when training a machine learning model.\n", 11 | "\n", 12 | "## Before you start\n", 13 | "\n", 14 | "You'll need the latest version of the **azure-ai-ml** package to run the code in this notebook. Run the cell below to verify that it is installed.\n", 15 | "\n", 16 | "> **Note**:\n", 17 | "> If the **azure-ai-ml** package is not installed, run `pip install azure-ai-ml` to install it." 18 | ] 19 | }, 20 | { 21 | "cell_type": "code", 22 | "execution_count": null, 23 | "metadata": { 24 | "vscode": { 25 | "languageId": "python" 26 | } 27 | }, 28 | "outputs": [], 29 | "source": [ 30 | "pip show azure-ai-ml" 31 | ] 32 | }, 33 | { 34 | "attachments": {}, 35 | "cell_type": "markdown", 36 | "metadata": {}, 37 | "source": [ 38 | "## Connect to your workspace\n", 39 | "\n", 40 | "With the required SDK packages installed, now you're ready to connect to your workspace.\n", 41 | "\n", 42 | "To connect to a workspace, we need identifier parameters - a subscription ID, resource group name, and workspace name. Since you're working with a compute instance, managed by Azure Machine Learning, you can use the default values to connect to the workspace." 43 | ] 44 | }, 45 | { 46 | "cell_type": "code", 47 | "execution_count": null, 48 | "metadata": { 49 | "gather": { 50 | "logged": 1663753569264 51 | }, 52 | "vscode": { 53 | "languageId": "python" 54 | } 55 | }, 56 | "outputs": [], 57 | "source": [ 58 | "from azure.identity import DefaultAzureCredential, InteractiveBrowserCredential\n", 59 | "from azure.ai.ml import MLClient\n", 60 | "\n", 61 | "try:\n", 62 | " credential = DefaultAzureCredential()\n", 63 | " # Check if given credential can get token successfully.\n", 64 | " credential.get_token(\"https://management.azure.com/.default\")\n", 65 | "except Exception as ex:\n", 66 | " # Fall back to InteractiveBrowserCredential in case DefaultAzureCredential not work\n", 67 | " credential = InteractiveBrowserCredential()" 68 | ] 69 | }, 70 | { 71 | "cell_type": "code", 72 | "execution_count": null, 73 | "metadata": { 74 | "vscode": { 75 | "languageId": "python" 76 | } 77 | }, 78 | "outputs": [], 79 | "source": [ 80 | "# Get a handle to workspace\n", 81 | "ml_client = MLClient.from_config(credential=credential)" 82 | ] 83 | }, 84 | { 85 | "attachments": {}, 86 | "cell_type": "markdown", 87 | "metadata": {}, 88 | "source": [ 89 | "## Initiate a command job\n", 90 | "\n", 91 | "Run the cell below to train a classification model to predict diabetes. The model is trained by running the **train-model-parameters.py** script that can be found in the **src** folder. It uses the **diabetes.csv** file as the training data. \n", 92 | "\n", 93 | "- `code`: specifies the folder that includes the script to run.\n", 94 | "- `command`: specifies what to run exactly.\n", 95 | "- `environment`: specifies the necessary packages to be installed on the compute before running the command.\n", 96 | "- `compute`: specifies the compute to use to run the command.\n", 97 | "- `display_name`: the name of the individual job.\n", 98 | "- `experiment_name`: the name of the experiment the job belongs to.\n", 99 | "\n", 100 | "Note that the command used to test the script in the terminal is the same as the command in the configuration of the job below. " 101 | ] 102 | }, 103 | { 104 | "cell_type": "code", 105 | "execution_count": null, 106 | "metadata": { 107 | "vscode": { 108 | "languageId": "python" 109 | } 110 | }, 111 | "outputs": [], 112 | "source": [ 113 | "from azure.ai.ml import command\n", 114 | "\n", 115 | "# configure job\n", 116 | "\n", 117 | "job = command(\n", 118 | " code=\"./src\",\n", 119 | " command=\"python train-model-parameters.py --training_data diabetes.csv\",\n", 120 | " environment=\"AzureML-sklearn-0.24-ubuntu18.04-py37-cpu@latest\",\n", 121 | " compute=\"aml-cluster\",\n", 122 | " display_name=\"diabetes-train-script\",\n", 123 | " experiment_name=\"diabetes-training\"\n", 124 | " )\n", 125 | "\n", 126 | "# submit job\n", 127 | "returned_job = ml_client.create_or_update(job)\n", 128 | "aml_url = returned_job.studio_url\n", 129 | "print(\"Monitor your job at\", aml_url)" 130 | ] 131 | } 132 | ], 133 | "metadata": { 134 | "kernel_info": { 135 | "name": "python38-azureml" 136 | }, 137 | "kernelspec": { 138 | "display_name": "Python 3.8 - AzureML", 139 | "language": "python", 140 | "name": "python38-azureml" 141 | }, 142 | "microsoft": { 143 | "host": { 144 | "AzureML": { 145 | "notebookHasBeenCompleted": true 146 | } 147 | } 148 | }, 149 | "nteract": { 150 | "version": "nteract-front-end@1.0.0" 151 | }, 152 | "vscode": { 153 | "interpreter": { 154 | "hash": "f2b2cd046deda8eabef1e765a11d0ec9aa9bd1d31d56ce79c815a38c323e14ec" 155 | } 156 | } 157 | }, 158 | "nbformat": 4, 159 | "nbformat_minor": 2 160 | } 161 | -------------------------------------------------------------------------------- /Labs/08/setup.sh: -------------------------------------------------------------------------------- 1 | #! /usr/bin/sh 2 | 3 | # Create random string 4 | guid=$(cat /proc/sys/kernel/random/uuid) 5 | suffix=${guid//[-]/} 6 | suffix=${suffix:0:18} 7 | 8 | # Set the necessary variables 9 | RESOURCE_GROUP="rg-dp100-l${suffix}" 10 | RESOURCE_PROVIDER="Microsoft.MachineLearningServices" 11 | REGIONS=("eastus" "westus" "centralus" "northeurope" "westeurope") 12 | RANDOM_REGION=${REGIONS[$RANDOM % ${#REGIONS[@]}]} 13 | WORKSPACE_NAME="mlw-dp100-l${suffix}" 14 | COMPUTE_INSTANCE="ci${suffix}" 15 | COMPUTE_CLUSTER="aml-cluster" 16 | 17 | # Register the Azure Machine Learning resource provider in the subscription 18 | echo "Register the Machine Learning resource provider:" 19 | az provider register --namespace $RESOURCE_PROVIDER 20 | 21 | # Create the resource group and workspace and set to default 22 | echo "Create a resource group and set as default:" 23 | az group create --name $RESOURCE_GROUP --location $RANDOM_REGION 24 | az configure --defaults group=$RESOURCE_GROUP 25 | 26 | echo "Create an Azure Machine Learning workspace:" 27 | az ml workspace create --name $WORKSPACE_NAME 28 | az configure --defaults workspace=$WORKSPACE_NAME 29 | 30 | # Create compute instance 31 | echo "Creating a compute instance with name: " $COMPUTE_INSTANCE 32 | az ml compute create --name ${COMPUTE_INSTANCE} --size STANDARD_DS11_V2 --type ComputeInstance 33 | 34 | # Create compute cluster 35 | echo "Creating a compute cluster with name: " $COMPUTE_CLUSTER 36 | az ml compute create --name ${COMPUTE_CLUSTER} --size STANDARD_DS11_V2 --max-instances 2 --type AmlCompute 37 | -------------------------------------------------------------------------------- /Labs/08/src/Train classification model.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Train diabetes classification model\n", 8 | "\n", 9 | "This notebook reads a CSV file and trains a model to predict diabetes in patients. The data is already preprocessed and requires no feature engineering.\n", 10 | "\n", 11 | "The evaluation methods were used during experimentation to decide whether the model was accurate enough. Moving forward, there's a preference to use the autolog feature of MLflow to more easily deploy the model later on." 12 | ] 13 | }, 14 | { 15 | "cell_type": "markdown", 16 | "metadata": {}, 17 | "source": [ 18 | "## Read data from local file\n", 19 | "\n" 20 | ] 21 | }, 22 | { 23 | "cell_type": "code", 24 | "execution_count": null, 25 | "metadata": { 26 | "gather": { 27 | "logged": 1666350530680 28 | } 29 | }, 30 | "outputs": [], 31 | "source": [ 32 | "import pandas as pd\n", 33 | "\n", 34 | "print(\"Reading data...\")\n", 35 | "df = pd.read_csv('diabetes.csv')\n", 36 | "df.head()" 37 | ] 38 | }, 39 | { 40 | "cell_type": "markdown", 41 | "metadata": {}, 42 | "source": [ 43 | "## Split data" 44 | ] 45 | }, 46 | { 47 | "cell_type": "code", 48 | "execution_count": null, 49 | "metadata": { 50 | "gather": { 51 | "logged": 1666350573655 52 | } 53 | }, 54 | "outputs": [], 55 | "source": [ 56 | "print(\"Splitting data...\")\n", 57 | "X, y = df[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, df['Diabetic'].values" 58 | ] 59 | }, 60 | { 61 | "cell_type": "code", 62 | "execution_count": null, 63 | "metadata": { 64 | "gather": { 65 | "logged": 1666350577729 66 | } 67 | }, 68 | "outputs": [], 69 | "source": [ 70 | "from sklearn.model_selection import train_test_split\n", 71 | "\n", 72 | "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)" 73 | ] 74 | }, 75 | { 76 | "cell_type": "markdown", 77 | "metadata": {}, 78 | "source": [ 79 | "## Train model" 80 | ] 81 | }, 82 | { 83 | "cell_type": "code", 84 | "execution_count": null, 85 | "metadata": { 86 | "gather": { 87 | "logged": 1666350591209 88 | } 89 | }, 90 | "outputs": [], 91 | "source": [ 92 | "from sklearn.linear_model import LogisticRegression\n", 93 | "\n", 94 | "print(\"Training model...\")\n", 95 | "model = LogisticRegression(C=1/0.1, solver=\"liblinear\").fit(X_train, y_train)" 96 | ] 97 | }, 98 | { 99 | "cell_type": "markdown", 100 | "metadata": {}, 101 | "source": [ 102 | "## Evaluate model" 103 | ] 104 | }, 105 | { 106 | "cell_type": "code", 107 | "execution_count": null, 108 | "metadata": { 109 | "gather": { 110 | "logged": 1666350714479 111 | } 112 | }, 113 | "outputs": [], 114 | "source": [ 115 | "import numpy as np\n", 116 | "\n", 117 | "y_hat = model.predict(X_test)\n", 118 | "acc = np.average(y_hat == y_test)\n", 119 | "\n", 120 | "print('Accuracy:', acc)" 121 | ] 122 | }, 123 | { 124 | "cell_type": "code", 125 | "execution_count": null, 126 | "metadata": { 127 | "gather": { 128 | "logged": 1666350716289 129 | } 130 | }, 131 | "outputs": [], 132 | "source": [ 133 | "from sklearn.metrics import roc_auc_score\n", 134 | "\n", 135 | "y_scores = model.predict_proba(X_test)\n", 136 | "auc = roc_auc_score(y_test,y_scores[:,1])\n", 137 | "\n", 138 | "print('AUC: ' + str(auc))" 139 | ] 140 | }, 141 | { 142 | "cell_type": "code", 143 | "execution_count": null, 144 | "metadata": { 145 | "gather": { 146 | "logged": 1666350747845 147 | } 148 | }, 149 | "outputs": [], 150 | "source": [ 151 | "from sklearn.metrics import roc_curve\n", 152 | "import matplotlib.pyplot as plt\n", 153 | "\n", 154 | "# plot ROC curve\n", 155 | "fpr, tpr, thresholds = roc_curve(y_test, y_scores[:,1])\n", 156 | "fig = plt.figure(figsize=(6, 4))\n", 157 | "# Plot the diagonal 50% line\n", 158 | "plt.plot([0, 1], [0, 1], 'k--')\n", 159 | "# Plot the FPR and TPR achieved by our model\n", 160 | "plt.plot(fpr, tpr)\n", 161 | "plt.xlabel('False Positive Rate')\n", 162 | "plt.ylabel('True Positive Rate')\n", 163 | "plt.title('ROC Curve')" 164 | ] 165 | } 166 | ], 167 | "metadata": { 168 | "kernel_info": { 169 | "name": "python3" 170 | }, 171 | "kernelspec": { 172 | "display_name": "base", 173 | "language": "python", 174 | "name": "python3" 175 | }, 176 | "language_info": { 177 | "name": "python", 178 | "version": "3.9.5 (default, May 18 2021, 14:42:02) [MSC v.1916 64 bit (AMD64)]" 179 | }, 180 | "microsoft": { 181 | "host": { 182 | "AzureML": { 183 | "notebookHasBeenCompleted": true 184 | } 185 | } 186 | }, 187 | "nteract": { 188 | "version": "nteract-front-end@1.0.0" 189 | }, 190 | "vscode": { 191 | "interpreter": { 192 | "hash": "f2b2cd046deda8eabef1e765a11d0ec9aa9bd1d31d56ce79c815a38c323e14ec" 193 | } 194 | } 195 | }, 196 | "nbformat": 4, 197 | "nbformat_minor": 0 198 | } 199 | -------------------------------------------------------------------------------- /Labs/08/src/train-model-parameters.py: -------------------------------------------------------------------------------- 1 | # import libraries 2 | import argparse 3 | import pandas as pd 4 | import numpy as np 5 | from sklearn.model_selection import train_test_split 6 | from sklearn.linear_model import LogisticRegression 7 | from sklearn.metrics import roc_auc_score 8 | from sklearn.metrics import roc_curve 9 | import matplotlib.pyplot as plt 10 | 11 | def main(args): 12 | # read data 13 | df = get_data(args.training_data) 14 | 15 | # split data 16 | X_train, X_test, y_train, y_test = split_data(df) 17 | 18 | # train model 19 | model = train_model(args.reg_rate, X_train, X_test, y_train, y_test) 20 | 21 | # evaluate model 22 | eval_model(model, X_test, y_test) 23 | 24 | # function that reads the data 25 | def get_data(path): 26 | print("Reading data...") 27 | df = pd.read_csv(path) 28 | 29 | return df 30 | 31 | # function that splits the data 32 | def split_data(df): 33 | print("Splitting data...") 34 | X, y = df[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness', 35 | 'SerumInsulin','BMI','DiabetesPedigree','Age']].values, df['Diabetic'].values 36 | 37 | X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0) 38 | 39 | return X_train, X_test, y_train, y_test 40 | 41 | # function that trains the model 42 | def train_model(reg_rate, X_train, X_test, y_train, y_test): 43 | print("Training model...") 44 | model = LogisticRegression(C=1/reg_rate, solver="liblinear").fit(X_train, y_train) 45 | 46 | return model 47 | 48 | # function that evaluates the model 49 | def eval_model(model, X_test, y_test): 50 | # calculate accuracy 51 | y_hat = model.predict(X_test) 52 | acc = np.average(y_hat == y_test) 53 | print('Accuracy:', acc) 54 | 55 | # calculate AUC 56 | y_scores = model.predict_proba(X_test) 57 | auc = roc_auc_score(y_test,y_scores[:,1]) 58 | print('AUC: ' + str(auc)) 59 | 60 | # plot ROC curve 61 | fpr, tpr, thresholds = roc_curve(y_test, y_scores[:,1]) 62 | fig = plt.figure(figsize=(6, 4)) 63 | # Plot the diagonal 50% line 64 | plt.plot([0, 1], [0, 1], 'k--') 65 | # Plot the FPR and TPR achieved by our model 66 | plt.plot(fpr, tpr) 67 | plt.xlabel('False Positive Rate') 68 | plt.ylabel('True Positive Rate') 69 | plt.title('ROC Curve') 70 | 71 | def parse_args(): 72 | # setup arg parser 73 | parser = argparse.ArgumentParser() 74 | 75 | # add arguments 76 | parser.add_argument("--training_data", dest='training_data', 77 | type=str) 78 | parser.add_argument("--reg_rate", dest='reg_rate', 79 | type=float, default=0.01) 80 | 81 | # parse args 82 | args = parser.parse_args() 83 | 84 | # return args 85 | return args 86 | 87 | # run script 88 | if __name__ == "__main__": 89 | # add space in logs 90 | print("\n\n") 91 | print("*" * 60) 92 | 93 | # parse args 94 | args = parse_args() 95 | 96 | # run main function 97 | main(args) 98 | 99 | # add space in logs 100 | print("*" * 60) 101 | print("\n\n") 102 | -------------------------------------------------------------------------------- /Labs/09/setup.sh: -------------------------------------------------------------------------------- 1 | #! /usr/bin/sh 2 | 3 | # Create random string 4 | guid=$(cat /proc/sys/kernel/random/uuid) 5 | suffix=${guid//[-]/} 6 | suffix=${suffix:0:18} 7 | 8 | # Set the necessary variables 9 | RESOURCE_GROUP="rg-dp100-l${suffix}" 10 | RESOURCE_PROVIDER="Microsoft.MachineLearningServices" 11 | REGIONS=("eastus" "westus" "centralus" "northeurope" "westeurope") 12 | RANDOM_REGION=${REGIONS[$RANDOM % ${#REGIONS[@]}]} 13 | WORKSPACE_NAME="mlw-dp100-l${suffix}" 14 | COMPUTE_INSTANCE="ci${suffix}" 15 | COMPUTE_CLUSTER="aml-cluster" 16 | 17 | # Register the Azure Machine Learning resource provider in the subscription 18 | echo "Register the Machine Learning resource provider:" 19 | az provider register --namespace $RESOURCE_PROVIDER 20 | 21 | # Create the resource group and workspace and set to default 22 | echo "Create a resource group and set as default:" 23 | az group create --name $RESOURCE_GROUP --location $RANDOM_REGION 24 | az configure --defaults group=$RESOURCE_GROUP 25 | 26 | echo "Create an Azure Machine Learning workspace:" 27 | az ml workspace create --name $WORKSPACE_NAME 28 | az configure --defaults workspace=$WORKSPACE_NAME 29 | 30 | # Create compute instance 31 | echo "Creating a compute instance with name: " $COMPUTE_INSTANCE 32 | az ml compute create --name ${COMPUTE_INSTANCE} --size STANDARD_DS11_V2 --type ComputeInstance 33 | 34 | # Create compute cluster 35 | echo "Creating a compute cluster with name: " $COMPUTE_CLUSTER 36 | az ml compute create --name ${COMPUTE_CLUSTER} --size STANDARD_DS11_V2 --max-instances 2 --type AmlCompute 37 | 38 | # Create data assets 39 | echo "Create training data asset:" 40 | az ml data create --type uri_file --name "diabetes-data" --path ./data/diabetes.csv 41 | -------------------------------------------------------------------------------- /Labs/10/model/MLmodel: -------------------------------------------------------------------------------- 1 | artifact_path: model 2 | flavors: 3 | python_function: 4 | env: conda.yaml 5 | loader_module: mlflow.sklearn 6 | model_path: model.pkl 7 | predict_fn: predict 8 | python_version: 3.7.15 9 | sklearn: 10 | code: null 11 | pickled_model: model.pkl 12 | serialization_format: cloudpickle 13 | sklearn_version: 0.24.1 14 | mlflow_version: 1.30.0 15 | model_uuid: 580862f780fb48eda0a137857d12c9f9 16 | run_id: calm_garden_gzd94mfzcr 17 | signature: 18 | inputs: '[{"name": "PatientID", "type": "integer"}, {"name": "Pregnancies", "type": 19 | "integer"}, {"name": "PlasmaGlucose", "type": "integer"}, {"name": "DiastolicBloodPressure", 20 | "type": "integer"}, {"name": "TricepsThickness", "type": "integer"}, {"name": 21 | "DiastolicBloodPressure", "type": "integer"}, {"name": "SerumInsulin", "type": 22 | "integer"}, {"name": "BMI", "type": "double"}, {"name": "DiabetesPedigree", "type": 23 | "double"}, {"name": "Age", "type": "integer"}]' 24 | outputs: '[{"type": "boolean"}]' 25 | utc_time_created: '2023-02-15 12:46:19.119720' 26 | -------------------------------------------------------------------------------- /Labs/10/model/conda.yaml: -------------------------------------------------------------------------------- 1 | channels: 2 | - conda-forge 3 | dependencies: 4 | - python=3.7.15 5 | - pip<=20.2.4 6 | - pip: 7 | - mlflow 8 | - cloudpickle==2.2.0 9 | - psutil==5.8.0 10 | - scikit-learn==0.24.1 11 | name: mlflow-env 12 | -------------------------------------------------------------------------------- /Labs/10/model/model.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrosoftLearning/mslearn-azure-ml/9e0bfc3bfda27097e2c744952f1f33205fed6723/Labs/10/model/model.pkl -------------------------------------------------------------------------------- /Labs/10/model/python_env.yaml: -------------------------------------------------------------------------------- 1 | python: 3.7.15 2 | build_dependencies: 3 | - pip==20.2.4 4 | - setuptools==65.6.3 5 | - wheel==0.37.1 6 | dependencies: 7 | - -r requirements.txt 8 | -------------------------------------------------------------------------------- /Labs/10/model/requirements.txt: -------------------------------------------------------------------------------- 1 | mlflow 2 | cloudpickle==2.2.0 3 | psutil==5.8.0 4 | scikit-learn==0.24.1 -------------------------------------------------------------------------------- /Labs/10/setup.sh: -------------------------------------------------------------------------------- 1 | #! /usr/bin/sh 2 | 3 | # Create random string 4 | guid=$(cat /proc/sys/kernel/random/uuid) 5 | suffix=${guid//[-]/} 6 | suffix=${suffix:0:18} 7 | 8 | # Set the necessary variables 9 | RESOURCE_GROUP="rg-dp100-l${suffix}" 10 | RESOURCE_PROVIDER="Microsoft.MachineLearningServices" 11 | REGIONS=("eastus" "westus" "centralus" "northeurope" "westeurope") 12 | RANDOM_REGION=${REGIONS[$RANDOM % ${#REGIONS[@]}]} 13 | WORKSPACE_NAME="mlw-dp100-l${suffix}" 14 | COMPUTE_INSTANCE="ci${suffix}" 15 | COMPUTE_CLUSTER="aml-cluster" 16 | 17 | # Register the Azure Machine Learning resource provider in the subscription 18 | echo "Register the Machine Learning resource provider:" 19 | az provider register --namespace $RESOURCE_PROVIDER 20 | 21 | # Create the resource group and workspace and set to default 22 | echo "Create a resource group and set as default:" 23 | az group create --name $RESOURCE_GROUP --location $RANDOM_REGION 24 | az configure --defaults group=$RESOURCE_GROUP 25 | 26 | echo "Create an Azure Machine Learning workspace:" 27 | az ml workspace create --name $WORKSPACE_NAME 28 | az configure --defaults workspace=$WORKSPACE_NAME 29 | 30 | # Create compute instance 31 | echo "Creating a compute instance with name: " $COMPUTE_INSTANCE 32 | az ml compute create --name ${COMPUTE_INSTANCE} --size STANDARD_DS11_V2 --type ComputeInstance 33 | 34 | # Create compute cluster 35 | echo "Creating a compute cluster with name: " $COMPUTE_CLUSTER 36 | az ml compute create --name ${COMPUTE_CLUSTER} --size STANDARD_DS11_V2 --max-instances 2 --type AmlCompute 37 | -------------------------------------------------------------------------------- /Labs/10/test-data/MLTable: -------------------------------------------------------------------------------- 1 | type: mltable 2 | paths: 3 | - file: ./diabetes-test.parquet 4 | transformations: 5 | - read_parquet -------------------------------------------------------------------------------- /Labs/10/train-data/MLTable: -------------------------------------------------------------------------------- 1 | type: mltable 2 | paths: 3 | - file: ./diabetes-training.parquet 4 | transformations: 5 | - read_parquet -------------------------------------------------------------------------------- /Labs/11/data/patient-data-0.csv: -------------------------------------------------------------------------------- 1 | Pregnancies,PlasmaGlucose,DiastolicBloodPressure,TricepsThickness,SerumInsulin,BMI,DiabetesPedigree,Age 2 | 1,78,41,33,311,50.79639151,0.420803683,24 3 | 0,116,92,16,184,18.60362975,0.131156495,22 4 | 8,171,42,29,160,35.48224692,0.082671083,22 5 | 3,108,63,45,297,49.37516891,0.100979095,46 6 | 8,153,99,15,41,35.06213922,0.116191245,22 7 | 0,81,90,17,38,41.65802578,0.478648501,21 8 | 0,82,52,45,42,40.75754211,0.082351573,26 9 | 4,96,83,26,34,52.94533137,0.160199188,53 10 | 3,103,54,47,94,55.14942833,0.316474755,21 11 | 11,99,60,12,737,32.89819759,0.108512148,46 12 | 2,172,87,89,159,34.47680708,0.088046993,23 13 | 6,142,63,10,264,45.06192863,0.100248331,21 14 | 1,124,77,34,222,37.47587301,0.11803065,24 15 | 0,93,54,15,48,19.3063246,0.200975356,35 16 | 7,116,104,9,34,21.1130511,0.163600466,22 17 | 0,172,87,31,229,20.41148409,0.161431534,21 18 | 0,86,68,8,670,20.65361156,0.478544924,21 19 | 0,97,63,45,50,21.3390726,0.165936334,24 20 | 2,114,82,53,290,32.2006567,1.007866423,23 21 | 0,156,75,22,82,45.85308577,0.142359764,23 22 | 1,67,92,27,35,36.34790041,0.164559761,30 23 | 3,99,74,45,192,20.99543755,0.101987243,44 24 | 0,112,57,33,159,37.46997343,0.112575863,21 25 | 3,125,66,36,473,50.57783819,0.628785695,23 26 | 2,139,54,41,369,39.72906082,0.100530003,21 27 | 1,100,81,10,221,40.22464804,0.715535153,21 28 | 1,68,86,7,19,20.07449283,0.257963043,38 29 | 0,147,51,17,32,20.16086129,0.231691106,44 30 | 2,55,71,10,162,42.97015979,0.095980491,25 31 | 9,66,84,11,55,29.63401172,0.17048918,46 32 | 0,78,54,43,195,21.64862431,0.134370303,34 33 | 6,110,70,72,75,26.1909532,0.719460579,38 34 | 10,160,93,7,79,28.76166776,0.175174954,22 35 | 1,86,52,46,43,36.02104105,0.754510199,24 36 | 0,149,85,9,194,21.73084171,0.171645331,23 37 | 11,171,57,25,18,19.92907168,0.150670553,24 38 | 13,68,100,28,355,29.62325889,0.625536868,46 39 | 4,105,68,28,54,25.11457874,0.108611667,53 40 | 11,101,53,42,85,19.63506134,0.135484093,61 41 | 1,88,85,7,239,20.31830551,0.48330106,22 42 | 0,108,48,33,29,37.46424652,0.445595024,23 43 | 1,136,68,41,36,20.75334152,0.108459069,26 44 | 0,122,78,40,42,22.05531052,0.803052147,22 45 | 0,155,64,7,31,37.2230851,0.164810692,25 46 | 5,156,85,47,56,43.39424506,0.182227321,21 47 | 1,114,96,29,371,43.25856917,0.186726643,26 48 | 0,71,45,32,25,21.30856774,0.132384145,23 49 | 9,71,59,44,36,39.31578651,0.161326308,21 50 | 0,113,87,31,173,45.46612637,1.078221375,26 51 | 3,82,60,41,83,38.29601059,0.266818381,21 52 | 13,141,69,27,96,40.89745344,1.543839617,21 53 | 4,96,67,13,96,30.40576185,1.05350731,42 54 | 5,132,30,23,203,32.89243173,0.700750318,46 55 | 1,86,42,35,28,40.63583552,0.092081607,34 56 | 4,113,63,44,172,39.60714714,0.517234662,56 57 | 10,58,66,45,220,29.44670129,0.114716464,23 58 | 2,54,60,43,76,27.0740572,0.269633414,22 59 | 0,153,70,17,190,42.85058516,0.136981162,23 60 | 0,123,42,43,189,25.47899808,0.081786567,21 61 | 5,176,86,12,73,43.80220819,0.176042207,47 62 | 0,116,44,12,43,20.22885805,0.166925586,24 63 | 1,113,60,11,69,35.96867309,0.148976217,21 64 | 1,164,78,46,341,18.76872846,0.127150255,23 65 | 0,125,98,36,39,37.63793959,0.079968382,43 66 | 9,135,77,55,81,23.31746262,0.092185598,44 67 | 6,116,85,45,188,20.56394448,0.124906282,21 68 | 1,141,45,10,23,35.98996017,0.649486097,71 69 | 5,103,52,47,65,31.49180363,0.497997418,58 70 | 2,103,92,26,182,30.37900919,0.103994203,21 71 | 5,98,62,45,153,22.97355432,0.097565885,21 72 | 0,91,76,16,14,21.3532444,0.440126063,22 73 | 7,81,86,30,67,41.77868053,0.872753938,22 74 | 7,122,49,23,35,44.64366598,0.330684449,21 75 | 0,89,52,35,266,37.43879763,0.141168659,59 76 | 7,82,94,35,30,36.19994044,0.719006459,21 77 | 1,157,47,33,24,18.6788066,0.552695167,24 78 | 0,116,49,19,212,21.86496011,1.04348649,26 79 | 1,48,46,32,51,50.90953804,0.146570527,49 80 | 9,102,60,43,80,27.19579155,0.153326997,21 81 | 2,46,78,31,68,28.61516386,0.482734934,39 82 | 3,140,81,10,76,37.94697587,0.771212408,43 83 | 1,151,89,33,155,20.96904836,0.170772195,51 84 | 9,96,80,33,86,23.58027967,0.125940453,33 85 | 3,116,68,15,24,36.66076997,0.649216856,42 86 | 4,98,90,52,91,34.23682296,1.095994039,57 87 | 3,116,60,21,236,32.40036715,0.319827884,55 88 | 1,82,97,53,137,18.88106285,0.679509583,21 89 | 5,68,77,45,161,20.99035924,0.832492432,47 90 | 10,183,57,44,159,51.76502485,0.17736259,21 91 | 9,126,57,31,159,19.97422504,0.78706667,31 92 | 0,87,81,11,14,20.46528284,0.201481075,21 93 | 0,74,45,11,69,20.6363498,0.191635484,22 94 | 1,100,85,38,34,20.28612712,0.103249336,25 95 | 1,85,68,10,39,38.09640961,0.817364971,23 96 | 0,124,81,42,25,19.98349782,0.208457902,22 97 | 2,105,61,35,63,31.62174025,0.131901471,45 98 | 0,93,56,19,35,21.46739359,0.605473453,21 99 | 5,119,64,14,65,40.61138513,0.143062887,22 100 | 0,83,100,7,183,40.92813374,0.156912786,33 101 | 0,141,50,9,32,18.67678177,0.130955457,26 102 | -------------------------------------------------------------------------------- /Labs/11/data/patient-data-1.csv: -------------------------------------------------------------------------------- 1 | Pregnancies,PlasmaGlucose,DiastolicBloodPressure,TricepsThickness,SerumInsulin,BMI,DiabetesPedigree,Age 2 | 0,141,50,9,32,18.67678177,0.130955457,26 3 | 1,101,63,50,80,30.88806127,0.249992755,56 4 | 7,96,63,31,20,20.27025933,0.109792372,23 5 | 6,92,96,10,37,38.86686317,0.955710632,22 6 | 0,104,82,10,189,19.87682775,0.139418018,21 7 | 0,91,54,42,179,21.9367905,0.445162587,21 8 | 5,92,51,11,46,18.65281177,0.830471708,26 9 | 1,152,64,11,29,27.07931456,0.238992802,24 10 | 0,82,78,42,269,42.41801153,0.505928555,53 11 | 0,72,50,33,17,19.6444375,0.563380915,23 12 | 1,143,41,43,259,39.25485017,0.300255121,22 13 | 2,93,92,13,58,34.3270297,0.184667714,45 14 | 0,146,54,33,192,20.37541191,0.1706651,23 15 | 1,69,80,31,29,19.16402427,0.245485214,21 16 | 1,88,84,31,48,20.91855142,0.139349808,22 17 | 1,87,64,32,141,45.11701561,0.08319965,23 18 | 0,104,44,12,46,18.87313653,0.206046249,35 19 | 0,127,63,33,22,46.25528885,0.138105181,25 20 | 1,132,85,8,29,41.02698629,0.08570251,26 21 | 7,86,49,10,39,30.28164216,0.10399271,26 22 | 3,90,91,13,59,26.70539445,0.177958481,22 23 | 0,77,83,46,50,19.70601858,0.151746782,25 24 | 9,102,52,36,53,40.37831438,0.18550978,22 25 | 0,70,64,8,244,20.66290074,0.129434462,21 26 | 0,115,63,18,180,18.2083711,0.46710402,22 27 | 1,125,83,41,235,19.65795152,0.150529189,23 28 | 0,141,72,54,203,50.11057402,0.625267152,54 29 | 2,119,55,43,19,29.48678102,0.185100135,22 30 | 10,156,41,8,85,46.29465247,0.270549322,21 31 | 1,81,82,25,37,43.77850539,0.580747696,23 32 | 9,106,64,7,43,20.14591479,0.153048495,21 33 | 0,170,77,42,71,19.76523872,0.273872213,32 34 | 1,120,64,35,43,40.93275814,0.120392424,76 35 | 7,144,97,20,168,42.21064493,1.046933141,49 36 | 3,153,60,34,37,19.76704351,0.163243448,25 37 | 1,126,82,42,49,35.8754255,0.103204944,34 38 | 1,137,76,32,31,55.49053906,1.58171469,53 39 | 4,64,80,15,35,34.56993161,0.73126265,55 40 | 10,119,71,14,398,27.04504242,0.975376187,47 41 | 0,118,41,17,184,42.44204188,0.432746806,21 42 | 0,55,103,44,19,42.85779275,0.131546392,21 43 | 6,117,66,12,229,25.32054727,1.553289911,21 44 | 1,74,44,7,29,21.38845039,1.125380762,26 45 | 8,119,46,46,158,33.05348747,0.112634739,25 46 | 0,80,95,34,36,19.13577156,0.66512333,22 47 | 3,106,82,12,256,26.7809188,0.865475397,22 48 | 0,51,88,41,131,33.21079913,0.084454895,23 49 | 10,155,81,42,164,21.83580208,0.243732554,21 50 | 10,169,94,23,81,28.25038417,0.102593749,44 51 | 9,82,49,11,191,37.8410018,0.588516683,24 52 | 3,141,62,34,43,21.09664512,0.143043718,45 53 | 0,72,90,9,49,21.71979572,0.156716799,22 54 | 0,120,87,33,134,30.92945282,0.159651465,24 55 | 0,114,71,14,32,36.83764781,0.264874109,25 56 | 3,101,66,8,66,20.74751125,0.096349269,58 57 | 2,99,73,13,33,36.77663375,1.051766111,28 58 | 0,125,48,34,136,20.3212516,0.416760886,25 59 | 0,93,86,43,44,19.72650131,0.137773974,31 60 | 8,131,67,34,95,37.4622037,0.99493165,45 61 | 4,106,80,12,196,45.65831803,0.135800817,58 62 | 0,82,80,33,129,21.33949822,0.158019435,26 63 | 3,136,68,27,235,48.63051144,0.602097843,21 64 | 3,72,45,7,92,27.30002973,1.59301825,43 65 | 3,71,65,37,28,48.7348335,0.103440909,25 66 | 7,50,56,45,42,18.70978882,0.51009117,21 67 | 1,123,81,42,359,41.71560287,0.152179447,24 68 | 1,108,50,50,236,33.49924567,0.302601369,47 69 | 0,68,60,33,168,43.15842754,0.08591777,25 70 | 1,80,83,8,39,44.05207321,0.686604355,24 71 | 7,83,62,15,187,32.11289732,0.113991343,48 72 | 0,93,90,27,23,33.63104346,0.986559366,26 73 | 0,69,86,16,38,18.57971721,0.738061716,23 74 | 11,46,63,34,224,44.17777381,0.130871677,26 75 | 7,150,102,28,36,18.72943175,0.245625457,21 76 | 8,98,52,45,51,23.80642408,0.120329825,21 77 | 4,120,60,15,31,45.89173119,0.457400008,43 78 | 9,98,55,43,52,25.48622826,0.56292994,57 79 | 1,140,53,23,15,21.30569858,0.663714017,69 80 | 1,95,81,49,198,21.05623621,0.19580385,22 81 | 3,106,83,39,223,31.77645097,0.877332438,22 82 | 8,106,86,14,81,34.10031142,0.715467158,23 83 | 7,88,80,32,216,19.26501808,0.147303928,25 84 | 0,122,52,28,247,37.53181477,0.290160277,21 85 | 2,98,99,14,93,40.05996882,0.117392433,22 86 | 0,107,39,43,26,20.92566713,0.562194037,50 87 | 6,48,97,32,63,33.67670834,0.29105506,23 88 | 7,56,54,11,34,29.30427736,0.162785876,23 89 | 1,146,42,20,129,21.84669127,0.081411723,22 90 | 7,73,56,34,189,41.59406812,0.224010423,24 91 | 0,127,89,38,157,50.94556737,0.112911889,22 92 | 0,115,66,25,241,27.40515383,0.23382395,44 93 | 8,171,97,52,87,41.39808451,0.150851834,21 94 | 1,126,89,33,146,37.62803667,0.13579614,47 95 | 3,106,88,33,251,36.86891308,0.594567798,46 96 | 2,108,47,43,72,21.71484573,0.097413117,22 97 | 1,151,65,34,156,38.30942642,0.164821164,58 98 | 2,53,79,41,24,35.40465077,0.774385476,22 99 | 6,83,50,19,19,38.2270337,0.142189843,22 100 | 5,114,55,21,67,46.29867316,0.540335711,22 101 | 0,87,50,23,36,21.34182799,0.118283508,33 102 | -------------------------------------------------------------------------------- /Labs/11/data/patient-data-2.csv: -------------------------------------------------------------------------------- 1 | Pregnancies,PlasmaGlucose,DiastolicBloodPressure,TricepsThickness,SerumInsulin,BMI,DiabetesPedigree,Age 2 | 3,180,66,25,28,31.9753005,0.105521172,39 3 | 0,146,84,12,16,44.71526895,0.153820909,22 4 | 1,110,85,33,664,38.58862004,0.530840493,24 5 | 1,157,94,24,193,21.14497072,0.101123649,24 6 | 0,156,52,31,56,21.75134651,0.195947069,25 7 | 3,76,56,28,70,31.59009627,0.887729937,21 8 | 8,176,88,47,87,36.02188745,0.786877606,39 9 | 1,125,95,31,18,36.77443276,0.127156051,25 10 | 0,52,86,11,31,40.72368362,0.489861405,21 11 | 0,73,58,43,223,28.63814103,0.198343895,25 12 | 3,96,92,27,20,42.39535484,0.19751416,38 13 | 2,114,91,43,184,32.83578517,1.559307025,22 14 | 0,146,83,11,160,20.43910622,0.965408775,24 15 | 1,150,54,10,201,20.35356391,0.220736049,32 16 | 1,87,79,24,85,19.90836015,0.160852801,26 17 | 1,93,102,11,86,21.05937142,0.247044813,26 18 | 4,130,87,26,74,44.87461202,1.495534191,22 19 | 1,77,40,23,182,19.34757075,0.257241681,22 20 | 1,95,95,30,21,39.91504282,0.130111937,25 21 | 4,87,75,21,74,45.51473954,0.101194533,55 22 | 0,73,26,43,48,34.76637597,0.806276976,31 23 | 3,172,64,42,36,31.45138382,0.675905094,22 24 | 3,102,54,56,76,23.18808214,1.167706544,22 25 | 0,68,66,33,151,20.21184816,0.217224573,45 26 | 10,111,85,22,74,31.78307221,0.47349237,39 27 | 1,102,48,20,48,18.22113978,0.737338642,22 28 | 3,103,62,28,50,43.19491984,1.100660788,23 29 | 3,80,92,23,47,36.43048855,0.65009028,26 30 | 4,135,79,11,47,32.33475753,0.583211768,49 31 | 1,70,50,36,170,39.33740507,0.172863706,24 32 | 1,133,90,11,182,45.94471294,0.079790728,25 33 | 1,94,77,34,49,21.27893791,0.156431348,26 34 | 0,145,80,34,84,36.87668924,0.511967481,25 35 | 0,71,57,11,251,19.35259645,0.098715631,22 36 | 1,156,65,7,39,20.99706957,0.081388525,26 37 | 3,91,64,34,194,35.14976774,0.236028198,22 38 | 1,68,82,8,27,21.64481707,0.123128576,21 39 | 1,96,63,44,190,35.65222431,1.570058393,46 40 | 5,96,86,13,208,31.23734273,0.165695648,22 41 | 0,97,81,9,176,21.47174219,0.225348628,25 42 | 9,102,69,38,152,46.17970943,0.101577776,47 43 | 1,69,82,31,233,41.83892776,0.284987613,23 44 | 6,153,66,43,795,35.68387318,0.143845509,23 45 | 7,85,88,9,168,37.7664514,0.182130016,46 46 | 0,73,42,20,193,37.24441224,0.278737714,22 47 | 4,116,84,40,73,51.15473712,1.387268364,22 48 | 8,154,95,23,94,26.19315106,0.116903699,45 49 | 3,119,44,14,93,27.48963684,0.489944815,22 50 | 0,124,45,27,202,39.39558144,0.108749441,25 51 | 1,124,82,35,48,37.18064149,0.138980602,21 52 | 0,91,61,34,139,33.66070769,0.150724607,25 53 | 3,147,91,48,79,30.00224539,0.895372919,40 54 | 1,68,93,35,45,19.75136427,0.135232281,23 55 | 0,94,54,26,48,21.58314624,0.512815662,35 56 | 0,95,62,7,36,38.99237681,0.599176265,26 57 | 0,126,50,42,25,42.03616717,0.096775244,30 58 | 5,97,66,36,55,35.31924114,1.106091492,22 59 | 6,127,81,21,71,37.27052046,1.603479067,21 60 | 9,93,64,44,694,22.06865641,0.136618975,21 61 | 0,150,79,17,28,43.39886289,0.18593283,24 62 | 3,136,63,26,82,44.93158388,0.695173448,21 63 | 8,104,66,7,35,19.6069102,0.776094295,24 64 | 6,88,51,43,190,20.84201248,0.1688357,22 65 | 2,115,46,35,286,30.53166003,0.126307957,22 66 | 9,139,55,44,21,38.39411011,0.11596792,26 67 | 3,114,73,57,513,27.44447214,0.090859123,23 68 | 0,138,81,21,47,37.52252036,0.101688337,21 69 | 0,99,64,7,141,31.82124795,0.160847412,24 70 | 0,55,86,10,228,20.88296657,0.078668224,69 71 | 0,85,62,33,201,37.56251464,0.490328466,22 72 | 6,174,80,14,93,33.02058974,0.11820229,46 73 | 8,61,61,54,172,26.64056668,1.157358978,59 74 | 5,106,50,14,176,19.38769264,0.543332657,38 75 | 2,115,73,13,157,29.72687381,0.808857565,22 76 | 0,80,70,33,14,21.114874,0.172161093,26 77 | 0,93,83,12,30,22.00756189,0.085734931,23 78 | 10,88,70,33,15,36.12803784,0.183883631,30 79 | 1,86,83,17,43,30.62831572,0.209720153,21 80 | 10,126,62,11,29,32.7673833,0.115578137,21 81 | 3,110,65,36,58,28.97599894,1.363331424,47 82 | 0,120,93,44,30,33.19092335,0.088665179,24 83 | 0,126,79,46,135,21.92328794,0.758143519,21 84 | 4,105,90,14,90,23.69861275,0.080416541,45 85 | 3,101,61,54,42,18.70264438,0.259772831,35 86 | 10,107,90,39,250,36.72873538,1.044598005,65 87 | 6,150,59,10,342,18.85849552,0.915795873,31 88 | 7,107,105,33,16,20.76216273,0.086863405,25 89 | 1,115,69,16,44,21.50214824,0.081095183,22 90 | 0,121,97,23,39,21.78152431,0.135898959,35 91 | 1,169,51,30,381,21.2618465,0.262161937,77 92 | 0,129,64,31,205,22.04325249,0.102775442,24 93 | 0,131,60,32,43,39.21948144,0.14070583,22 94 | 0,86,76,36,184,45.27517435,0.135202349,24 95 | 0,142,51,57,150,22.03782572,0.21140775,24 96 | 9,89,57,41,50,32.9320502,0.116770162,25 97 | 0,69,58,9,173,31.61438474,0.167809523,24 98 | 3,125,95,24,35,37.98891178,0.103637313,42 99 | 0,150,80,33,39,47.58612407,0.265330183,21 100 | 0,75,64,10,246,37.10230443,0.169710016,22 101 | 1,71,61,17,63,45.10012367,0.293331682,25 102 | -------------------------------------------------------------------------------- /Labs/11/data/patient-data-3.csv: -------------------------------------------------------------------------------- 1 | Pregnancies,PlasmaGlucose,DiastolicBloodPressure,TricepsThickness,SerumInsulin,BMI,DiabetesPedigree,Age 2 | 1,150,53,32,41,21.11109895,0.124346302,21 3 | 3,152,58,44,223,50.1103144,0.201925039,22 4 | 6,129,51,53,82,22.03744388,0.501260076,21 5 | 1,81,84,42,20,21.32030769,0.118671642,26 6 | 0,160,77,17,32,36.6499553,0.170770834,21 7 | 10,61,80,15,159,27.12459826,0.107820016,22 8 | 0,78,56,28,161,18.55609745,0.669052783,26 9 | 6,140,75,46,221,38.28753442,0.164744894,21 10 | 4,91,69,24,60,30.6249752,0.117302527,21 11 | 0,80,76,22,44,18.58271552,0.751714866,26 12 | 1,88,45,10,53,38.43643314,0.218580467,23 13 | 9,67,66,27,473,30.43275121,0.672883554,23 14 | 8,117,69,41,373,39.06607581,1.083593936,43 15 | 7,84,73,49,172,37.02676107,0.2099995,26 16 | 0,83,80,44,29,33.37643126,0.100267858,22 17 | 1,146,91,11,70,20.60670999,0.07962115,57 18 | 1,70,84,45,27,20.17704832,0.199814096,25 19 | 9,163,64,9,15,21.58993265,0.177244275,26 20 | 0,92,68,45,25,20.06331838,0.64710914,23 21 | 5,47,58,29,154,37.83000883,0.139509144,23 22 | 1,124,47,10,77,38.3679624,0.138341105,25 23 | 0,136,80,15,19,41.64101949,0.112958221,22 24 | 6,50,52,31,87,19.74350813,0.215737686,22 25 | 0,169,94,9,218,21.47797507,0.848945304,26 26 | 4,143,82,22,88,43.61829174,0.177981072,22 27 | 3,109,79,28,37,45.01625092,0.629432857,22 28 | 0,155,67,28,26,32.8024288,0.699522753,22 29 | 3,105,91,55,81,45.06410197,0.084067537,31 30 | 1,116,83,43,655,18.90869512,0.166555691,21 31 | 1,91,56,34,28,36.7585946,0.767226635,42 32 | 3,141,46,10,43,20.77454877,0.117479656,24 33 | 0,78,87,7,180,36.95627959,0.533561147,25 34 | 3,94,69,15,40,24.51395526,0.108441342,47 35 | 1,136,59,10,183,20.89254107,0.572050497,31 36 | 4,99,80,46,332,30.96630182,0.544531133,46 37 | 2,120,66,27,242,29.59869892,0.502384748,21 38 | 8,57,84,33,545,28.73987419,0.08449522,39 39 | 1,126,97,52,333,21.00090436,0.144526057,23 40 | 2,139,82,49,378,48.80105533,1.578507349,21 41 | 5,107,84,13,363,33.53635535,0.19553318,44 42 | 9,108,71,27,179,37.93808268,1.264161711,45 43 | 7,140,58,11,79,20.58109677,0.101238482,24 44 | 9,125,99,15,56,29.49681388,0.839731503,22 45 | 0,57,46,34,177,39.8965653,0.503718944,51 46 | 8,118,69,46,50,45.62556486,0.575291802,35 47 | 0,78,86,9,171,19.17637449,0.086726718,24 48 | 10,93,49,7,14,21.38981218,0.183691966,24 49 | 3,117,69,23,74,30.75070716,0.118022927,22 50 | 5,145,88,13,92,38.52810506,0.167398799,22 51 | 0,69,39,31,21,36.61127519,0.184056779,23 52 | 6,126,84,11,22,41.71725647,0.301171921,21 53 | 3,122,71,15,508,18.80360075,0.816036628,47 54 | 2,98,81,43,91,26.20461947,0.189788525,21 55 | 0,156,56,10,162,33.23793212,0.423135629,34 56 | 3,97,94,15,36,22.04703735,1.042192978,37 57 | 0,138,78,50,434,21.02103284,0.761013732,32 58 | 10,166,57,47,47,41.34484998,0.814850694,21 59 | 0,95,61,10,46,22.00962758,0.606673422,23 60 | 5,87,57,12,135,22.08514162,0.163714903,25 61 | 6,88,51,7,18,21.58110679,0.111186493,21 62 | 0,79,112,42,38,38.43922388,0.156768679,22 63 | 0,105,81,23,156,19.18611158,0.871304557,25 64 | 3,156,78,40,240,30.23771332,0.106470482,21 65 | 9,106,92,41,515,48.8729443,0.174026657,22 66 | 1,70,53,11,27,19.21177292,0.710496734,67 67 | 6,105,82,21,180,21.31648956,0.168994672,35 68 | 6,115,80,15,46,27.94900639,0.110965751,22 69 | 0,127,56,10,269,28.8077053,0.658301856,21 70 | 0,84,82,42,164,37.11979822,0.142465629,26 71 | 9,120,66,29,60,40.83582698,0.154494437,38 72 | 3,117,86,34,165,20.20043078,0.185173902,23 73 | 6,107,59,41,90,46.46172929,0.165244202,62 74 | 0,93,86,29,173,40.23758866,0.155210233,52 75 | 6,88,86,24,269,36.84113961,0.087223097,22 76 | 7,161,75,8,28,21.614417,0.148145849,43 77 | 0,115,80,28,40,36.36483531,0.664069363,25 78 | 9,84,50,34,31,36.86026983,0.721319566,26 79 | 0,125,63,35,19,19.09655159,0.211167981,21 80 | 1,68,49,39,222,19.44043724,0.165348184,25 81 | 1,141,50,44,168,22.0026449,0.513994082,67 82 | 8,106,64,40,188,43.56737336,0.110866791,53 83 | 6,74,90,44,47,39.52042249,0.256792959,23 84 | 1,132,52,35,250,20.49213875,2.162005419,24 85 | 1,79,86,23,71,33.02417479,0.559244075,62 86 | 0,173,62,44,29,21.51223891,0.8055696,21 87 | 2,109,57,25,333,36.23289963,0.974987886,41 88 | 2,99,87,29,604,31.66189632,0.114690788,42 89 | 6,105,82,42,61,46.49009236,0.271713209,22 90 | 1,177,63,22,40,19.15959142,0.921040715,23 91 | 1,141,53,11,31,41.26306402,0.114287539,23 92 | 2,180,82,47,65,29.66365103,0.183305467,67 93 | 5,76,72,56,180,25.49197081,0.520842914,32 94 | 0,117,81,29,24,21.25593005,0.301746808,21 95 | 7,83,60,34,135,45.11345606,0.083354607,21 96 | 1,169,50,42,171,36.18303579,0.179361844,24 97 | 0,134,82,7,40,21.15955185,0.20242626,48 98 | 1,113,55,38,47,30.2973127,0.19665291,22 99 | 1,138,84,12,52,19.2379161,0.174464275,22 100 | 3,152,83,42,261,36.42350233,0.178088012,53 101 | 1,151,40,14,32,45.60017248,0.164846732,35 102 | -------------------------------------------------------------------------------- /Labs/11/model/MLmodel: -------------------------------------------------------------------------------- 1 | artifact_path: model 2 | flavors: 3 | python_function: 4 | env: conda.yaml 5 | loader_module: mlflow.sklearn 6 | model_path: model.pkl 7 | python_version: 3.8.17 8 | sklearn: 9 | pickled_model: model.pkl 10 | serialization_format: cloudpickle 11 | sklearn_version: 1.3.0 12 | mlflow_version: 1.30.0 13 | model_uuid: ba8abd76e4cb479a81145f1bb76472e4 14 | run_id: 36b85079-2e17-43da-b4b4-775944b9a2d3 15 | signature: 16 | inputs: '[{"name": "Pregnancies", "type": "double"}, {"name": "PlasmaGlucose", "type": "double"}, 17 | {"name": "DiastolicBloodPressure", "type": "double"}, {"name": "TricepsThickness", "type": "double"}, {"name": 18 | "SerumInsulin", "type": "double"}, {"name": "BMI", "type": "double"}, {"name": "DiabetesPedigree", "type": 19 | "double"}, {"name": "Age", "type": "double"}]' 20 | outputs: '[{"type": "double"}]' 21 | utc_time_created: '2022-11-01 20:46:17.296023' 22 | -------------------------------------------------------------------------------- /Labs/11/model/conda.yaml: -------------------------------------------------------------------------------- 1 | channels: 2 | - conda-forge 3 | dependencies: 4 | - python=3.8.17 5 | - pip<=20.2.4 6 | - pip: 7 | - mlflow 8 | - cloudpickle==2.2.1 9 | - psutil==5.9.5 10 | - scikit-learn==1.3.0 11 | name: mlflow-env 12 | -------------------------------------------------------------------------------- /Labs/11/model/model.pkl: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/MicrosoftLearning/mslearn-azure-ml/9e0bfc3bfda27097e2c744952f1f33205fed6723/Labs/11/model/model.pkl -------------------------------------------------------------------------------- /Labs/11/model/python_env.yaml: -------------------------------------------------------------------------------- 1 | python: 3.8.17 2 | build_dependencies: 3 | - pip==20.2.4 4 | - setuptools==65.5.0 5 | - wheel==0.37.1 6 | dependencies: 7 | - -r requirements.txt 8 | -------------------------------------------------------------------------------- /Labs/11/model/requirements.txt: -------------------------------------------------------------------------------- 1 | mlflow 2 | cloudpickle==2.2.1 3 | psutil==5.9.5 4 | scikit-learn==1.3.0 -------------------------------------------------------------------------------- /Labs/11/sample-data.json: -------------------------------------------------------------------------------- 1 | { 2 | "input_data": { 3 | "columns": [ 4 | "Pregnancies", 5 | "PlasmaGlucose", 6 | "DiastolicBloodPressure", 7 | "TricepsThickness", 8 | "SerumInsulin", 9 | "BMI", 10 | "DiabetesPedigree", 11 | "Age" 12 | ], 13 | "index": [1], 14 | "data": [ 15 | [ 16 | 0,148,58,11,179,39.19207553,0.160829008,45 17 | ] 18 | ] 19 | } 20 | } -------------------------------------------------------------------------------- /Labs/11/setup.sh: -------------------------------------------------------------------------------- 1 | #! /usr/bin/sh 2 | 3 | # Create random string 4 | guid=$(cat /proc/sys/kernel/random/uuid) 5 | suffix=${guid//[-]/} 6 | suffix=${suffix:0:18} 7 | 8 | # Set the necessary variables 9 | RESOURCE_GROUP="rg-dp100-l${suffix}" 10 | REGIONS=("eastus" "westus" "centralus" "northeurope" "westeurope") 11 | RANDOM_REGION=${REGIONS[$RANDOM % ${#REGIONS[@]}]} 12 | WORKSPACE_NAME="mlw-dp100-l${suffix}" 13 | COMPUTE_INSTANCE="ci${suffix}" 14 | COMPUTE_CLUSTER="aml-cluster" 15 | 16 | # Register the Azure Machine Learning and additional resource providers in the subscription 17 | echo "Register the required resource providers:" 18 | az provider register --namespace "Microsoft.MachineLearningServices" 19 | az provider register --namespace "Microsoft.PolicyInsights" 20 | az provider register --namespace "Microsoft.Cdn" 21 | 22 | # Create the resource group and workspace and set to default 23 | echo "Create a resource group and set as default:" 24 | az group create --name $RESOURCE_GROUP --location $RANDOM_REGION 25 | az configure --defaults group=$RESOURCE_GROUP 26 | 27 | echo "Create an Azure Machine Learning workspace:" 28 | az ml workspace create --name $WORKSPACE_NAME 29 | az configure --defaults workspace=$WORKSPACE_NAME 30 | 31 | # Create compute instance 32 | echo "Creating a compute instance with name: " $COMPUTE_INSTANCE 33 | az ml compute create --name ${COMPUTE_INSTANCE} --size STANDARD_DS11_V2 --type ComputeInstance 34 | 35 | # Create compute cluster 36 | echo "Creating a compute cluster with name: " $COMPUTE_CLUSTER 37 | az ml compute create --name ${COMPUTE_CLUSTER} --size STANDARD_DS11_V2 --max-instances 2 --type AmlCompute 38 | -------------------------------------------------------------------------------- /Policies/DP-100 Base + LP11.json: -------------------------------------------------------------------------------- 1 | { 2 | "if": { 3 | "not": { 4 | "anyOf": [ 5 | { 6 | "field": "type", 7 | "contains": "Microsoft.Storage/storageAccounts" 8 | }, 9 | { 10 | "field": "type", 11 | "contains": "Microsoft.insights/" 12 | }, 13 | { 14 | "field": "type", 15 | "contains": "Microsoft.ContainerInstance/containerGroups" 16 | }, 17 | { 18 | "field": "type", 19 | "contains": "Microsoft.ContainerRegistry/registries" 20 | }, 21 | { 22 | "field": "type", 23 | "contains": "Microsoft.KeyVault/vaults" 24 | }, 25 | { 26 | "field": "type", 27 | "contains": "Microsoft.AlertsManagement/smartDetectorAlertRules" 28 | }, 29 | { 30 | "field": "type", 31 | "contains": "Microsoft.OperationalInsights/workspaces" 32 | }, 33 | { 34 | "field":"type", 35 | "equals":"Microsoft.MachineLearningServices/workspaces" 36 | }, 37 | { 38 | "field":"type", 39 | "contains":"Microsoft.MachineLearningServices/workspaces/onlineEndpoints" 40 | }, 41 | { 42 | "field":"type", 43 | "contains":"Microsoft.MachineLearningServices/workspaces/batchEndpoints" 44 | } 45 | ] 46 | 47 | } 48 | }, 49 | "then": { 50 | "effect": "Deny" 51 | } 52 | } 53 | -------------------------------------------------------------------------------- /Policies/DP-100 Base.json: -------------------------------------------------------------------------------- 1 | { 2 | "if": { 3 | "not": { 4 | "anyOf": [ 5 | { 6 | "field": "type", 7 | "contains": "Microsoft.Storage/storageAccounts" 8 | }, 9 | { 10 | "field": "type", 11 | "contains": "Microsoft.insights/" 12 | }, 13 | { 14 | "field": "type", 15 | "contains": "Microsoft.ContainerInstance/containerGroups" 16 | }, 17 | { 18 | "field": "type", 19 | "contains": "Microsoft.ContainerRegistry/registries" 20 | }, 21 | { 22 | "field": "type", 23 | "contains": "Microsoft.KeyVault/vaults" 24 | }, 25 | { 26 | "field": "type", 27 | "contains": "Microsoft.AlertsManagement/smartDetectorAlertRules" 28 | }, 29 | { 30 | "field": "type", 31 | "contains": "Microsoft.OperationalInsights/workspaces" 32 | }, 33 | { 34 | "field":"type", 35 | "equals":"Microsoft.MachineLearningServices/workspaces" 36 | } 37 | ] 38 | 39 | } 40 | }, 41 | "then": { 42 | "effect": "Deny" 43 | } 44 | } 45 | -------------------------------------------------------------------------------- /Policies/Required Resource Providers.md: -------------------------------------------------------------------------------- 1 | ## Required Resource Providers 2 | 3 | Microsoft.Insights 4 | 5 | Microsoft.ContainerInstance 6 | 7 | Microsoft.ContainerRegistry 8 | 9 | Microsoft.KeyVault 10 | 11 | Microsoft.AlertsManagement 12 | 13 | Microsoft.OperationalInsights 14 | 15 | Microsoft.MachineLearningServices 16 | -------------------------------------------------------------------------------- /_build.yml: -------------------------------------------------------------------------------- 1 | name: '$(Date:yyyyMMdd)$(Rev:.rr)' 2 | jobs: 3 | - job: build_markdown_content 4 | displayName: 'Build Markdown Content' 5 | workspace: 6 | clean: all 7 | pool: 8 | vmImage: 'Ubuntu 16.04' 9 | container: 10 | image: 'microsoftlearning/markdown-build:latest' 11 | steps: 12 | - task: Bash@3 13 | displayName: 'Build Content' 14 | inputs: 15 | targetType: inline 16 | script: | 17 | cp /{attribution.md,template.docx,package.json,package.js} . 18 | npm install 19 | node package.js --version $(Build.BuildNumber) 20 | - task: GitHubRelease@0 21 | displayName: 'Create GitHub Release' 22 | inputs: 23 | gitHubConnection: 'github-microsoftlearning-organization' 24 | repositoryName: '$(Build.Repository.Name)' 25 | tagSource: manual 26 | tag: 'v$(Build.BuildNumber)' 27 | title: 'Version $(Build.BuildNumber)' 28 | releaseNotesSource: input 29 | releaseNotes: '# Version $(Build.BuildNumber) Release' 30 | assets: '$(Build.SourcesDirectory)/out/*.zip' 31 | assetUploadMode: replace 32 | - task: PublishBuildArtifacts@1 33 | displayName: 'Publish Output Files' 34 | inputs: 35 | pathtoPublish: '$(Build.SourcesDirectory)/out/' 36 | artifactName: 'Lab Files' 37 | -------------------------------------------------------------------------------- /_config.yml: -------------------------------------------------------------------------------- 1 | remote_theme: MicrosoftLearning/Jekyll-Theme 2 | exclude: 3 | - readme.md 4 | - .github/ 5 | header_pages: 6 | - index.html 7 | author: Microsoft Learning 8 | twitter_username: mslearning 9 | github_username: MicrosoftLearning 10 | plugins: 11 | - jekyll-sitemap 12 | - jekyll-mentions 13 | - jemoji 14 | markdown: kramdown 15 | kramdown: 16 | syntax_highlighter_opts: 17 | disable : true 18 | -------------------------------------------------------------------------------- /index.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: Online Hosted Instructions 3 | permalink: index.html 4 | layout: home 5 | --- 6 | 7 | # Microsoft Learn - Hands On Exercises 8 | 9 | The following hands-on exercises are designed to support [Microsoft Learn](https://docs.microsoft.com/training/) training. 10 | 11 | {% assign labs = site.pages | where_exp:"page", "page.url contains '/Instructions'" %} 12 | | | 13 | | --- | --- | 14 | {% for activity in labs %}| [{{ activity.lab.title }}{% if activity.lab.type %} - {{ activity.lab.type }}{% endif %}]({{ site.github.url }}{{ activity.url }}) | 15 | {% endfor %} 16 | -------------------------------------------------------------------------------- /readme.md: -------------------------------------------------------------------------------- 1 | ## Hands-on exercises for Azure Machine Learning (v2) 2 | 3 | This repository contains the hands-on lab exercises for the Microsoft Learning Paths exploring Azure Machine Learning. The Learning Paths consists of self-paced modules on Microsoft Learn. The labs are designed to accompany the learning materials and enable you to practice using the technologies described them. 4 | 5 | You can view the instructions for the lab exercises at [https://microsoftlearning.github.io/mslearn-azure-ml/](https://microsoftlearning.github.io/mslearn-azure-ml/). 6 | 7 | ## What are we doing? 8 | 9 | - To support this course, we will need to make frequent updates to the course content to keep it current with the Azure services used in the course. We are publishing the lab instructions and lab files on GitHub to allow for open contributions between the course authors and MCTs to keep the content current with changes in the Azure platform. 10 | 11 | - We hope that this brings a sense of collaboration to the labs like we've never had before - when Azure changes and you find it first during a live delivery, go ahead and make an enhancement right in the lab source. 12 | 13 | ## How do I contribute? 14 | 15 | - Anyone can submit a pull request to the code or content in the GitHub repository, Microsoft and the course author will triage and include content and lab code changes as needed. 16 | 17 | - You can submit bugs, changes, improvement and ideas. Find a new Azure feature before we have? Submit a new demo! 18 | -------------------------------------------------------------------------------- /typings/__builtins__.pyi: -------------------------------------------------------------------------------- 1 | 2 | from databricks.sdk.runtime import * 3 | --------------------------------------------------------------------------------