├── .cloud └── .azure │ ├── registermodel.json │ ├── run.json │ ├── compute.json │ └── workspace.json ├── docs └── images │ ├── aml.png │ ├── actions.png │ ├── secrets.png │ ├── actions_tab.png │ ├── ml-lifecycle.png │ └── ML Ops Workflow (2).png ├── code ├── train │ ├── environment.yaml │ ├── run_config.py │ └── train.py ├── deploy │ ├── environment.yml │ └── score.py └── test │ └── test.py ├── CODE_OF_CONDUCT.md ├── .github └── workflows │ ├── slash_command.yml │ ├── train.yml │ └── deploy.yml ├── LICENSE ├── SECURITY.md ├── .gitignore └── README.md /.cloud/.azure/registermodel.json: -------------------------------------------------------------------------------- 1 | { 2 | "model_file_name": "model.pkl" 3 | } -------------------------------------------------------------------------------- /.cloud/.azure/run.json: -------------------------------------------------------------------------------- 1 | { 2 | "experiment_name": "GitHubActionExperiment" 3 | } 4 | -------------------------------------------------------------------------------- /docs/images/aml.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/retkowsky/aml-template/master/docs/images/aml.png -------------------------------------------------------------------------------- /.cloud/.azure/compute.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "githubcluster", 3 | "compute_type": "amlcluster" 4 | } 5 | -------------------------------------------------------------------------------- /docs/images/actions.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/retkowsky/aml-template/master/docs/images/actions.png -------------------------------------------------------------------------------- /docs/images/secrets.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/retkowsky/aml-template/master/docs/images/secrets.png -------------------------------------------------------------------------------- /.cloud/.azure/workspace.json: -------------------------------------------------------------------------------- 1 | { 2 | "name": "same-demo-wus2", 3 | "resource_group": "gha_and_aml_rg" 4 | } 5 | -------------------------------------------------------------------------------- /docs/images/actions_tab.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/retkowsky/aml-template/master/docs/images/actions_tab.png -------------------------------------------------------------------------------- /docs/images/ml-lifecycle.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/retkowsky/aml-template/master/docs/images/ml-lifecycle.png -------------------------------------------------------------------------------- /docs/images/ML Ops Workflow (2).png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/retkowsky/aml-template/master/docs/images/ML Ops Workflow (2).png -------------------------------------------------------------------------------- /code/train/environment.yaml: -------------------------------------------------------------------------------- 1 | # Conda environment specification. Details about the Conda environment file format: 2 | # https://conda.io/docs/user-guide/tasks/manage-environments.html#create-env-file-manually 3 | 4 | name: training_env 5 | dependencies: 6 | - python=3.6.2 7 | - scikit-learn=0.22.2 8 | - pandas=1.0.3 9 | - matplotlib=3.2.1 10 | - pip: 11 | - azureml-defaults==1.1.5 12 | - azureml-dataprep[pandas,fuse]==1.4.0 13 | channels: 14 | - conda-forge -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Microsoft Open Source Code of Conduct 2 | 3 | This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). 4 | 5 | Resources: 6 | 7 | - [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/) 8 | - [Microsoft Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) 9 | - Contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with questions or concerns 10 | -------------------------------------------------------------------------------- /code/deploy/environment.yml: -------------------------------------------------------------------------------- 1 | # Conda environment specification. Details about the Conda environment file format: 2 | # https://conda.io/docs/user-guide/tasks/manage-environments.html#create-env-file-manually 3 | 4 | name: deployment_env 5 | dependencies: 6 | - python=3.8.2 7 | - scikit-learn=0.22.2 8 | - numpy=1.18.1 9 | - pip: 10 | - azureml-defaults==1.1.5 11 | - azureml-monitoring==0.1.0a18 12 | - inference-schema==1.0.2 13 | - inference-schema[numpy-support]==1.0.2 14 | channels: 15 | - conda-forge 16 | -------------------------------------------------------------------------------- /code/test/test.py: -------------------------------------------------------------------------------- 1 | import json 2 | 3 | from azureml.core import Webservice 4 | 5 | 6 | def main(service): 7 | # Creating input data 8 | print("Creating input data") 9 | data = {"data": [[ 1,2,3,4 ], [ 10,9,8,7 ]]} 10 | input_data = json.dumps(data) 11 | 12 | # Calling webservice 13 | print("Calling webservice") 14 | output_data = service.run(input_data) 15 | predictions = output_data.get("predict") 16 | assert type(predictions) == list 17 | 18 | 19 | if __name__ == "__main__": 20 | main() 21 | -------------------------------------------------------------------------------- /.github/workflows/slash_command.yml: -------------------------------------------------------------------------------- 1 | name: slash-command 2 | 3 | on: 4 | issue_comment: 5 | types: [created] 6 | 7 | jobs: 8 | # This is the dispatch for slash commands in the repo 9 | slash_command: 10 | runs-on: ubuntu-latest 11 | steps: 12 | - name: Slash Command Dispatch 13 | id: slash_command 14 | uses: peter-evans/slash-command-dispatch@v1.1.2 15 | with: 16 | token: ${{ secrets.REPO_ACCESS_TOKEN }} 17 | reaction-token: ${{ secrets.GITHUB_TOKEN }} 18 | commands: deploy 19 | reaction-type: hooray 20 | permission: write 21 | issue-type: pull-request 22 | named-args: true -------------------------------------------------------------------------------- /code/train/run_config.py: -------------------------------------------------------------------------------- 1 | from azureml.core import ComputeTarget 2 | from azureml.train.estimator import Estimator 3 | 4 | 5 | def main(workspace): 6 | # Load compute target 7 | print("Loading compute target") 8 | compute_target = ComputeTarget( 9 | workspace=workspace, 10 | name="githubcluster" 11 | ) 12 | 13 | # Load script parameters 14 | print("Loading script parameters") 15 | script_params = { 16 | "--kernel": "linear", 17 | "--penalty": 1.0 18 | } 19 | 20 | # Create experiment config 21 | print("Creating experiment config") 22 | estimator = Estimator( 23 | source_directory="code/train", 24 | entry_script="train.py", 25 | script_params=script_params, 26 | compute_target=compute_target, 27 | pip_packages=["azureml-dataprep[pandas,fuse]", "scikit-learn", "pandas", "matplotlib"] 28 | ) 29 | return estimator 30 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) Microsoft Corporation. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE 22 | -------------------------------------------------------------------------------- /code/deploy/score.py: -------------------------------------------------------------------------------- 1 | import os 2 | import joblib 3 | import numpy as np 4 | 5 | from sklearn.svm import SVC 6 | from azureml.core import Model 7 | from azureml.monitoring import ModelDataCollector 8 | from inference_schema.schema_decorators import input_schema, output_schema 9 | from inference_schema.parameter_types.numpy_parameter_type import NumpyParameterType 10 | from inference_schema.parameter_types.standard_py_parameter_type import StandardPythonParameterType 11 | 12 | 13 | # The init() method is called once, when the web service starts up. 14 | # Typically you would deserialize the model file, as shown here using joblib, 15 | # and store it in a global variable so your run() method can access it later. 16 | def init(): 17 | global model 18 | global inputs_dc, prediction_dc 19 | # The AZUREML_MODEL_DIR environment variable indicates 20 | # a directory containing the model file you registered. 21 | model_path = Model.get_model_path(model_name="mymodel") 22 | model = joblib.load(model_path) 23 | inputs_dc = ModelDataCollector("sample-model", designation="inputs", feature_names=["feat1", "feat2", "feat3", "feat4"]) 24 | prediction_dc = ModelDataCollector("sample-model", designation="predictions", feature_names=["prediction"]) 25 | 26 | 27 | # The run() method is called each time a request is made to the scoring API. 28 | # Shown here are the optional input_schema and output_schema decorators 29 | # from the inference-schema pip package. Using these decorators on your 30 | # run() method parses and validates the incoming payload against 31 | # the example input you provide here. This will also generate a Swagger 32 | # API document for your web service. 33 | @input_schema('data', NumpyParameterType(np.array([[0.1, 1.2, 2.3, 3.4]]))) 34 | @output_schema(StandardPythonParameterType({'predict': [['Iris-virginica']]})) 35 | def run(data): 36 | # Use the model object loaded by init(). 37 | result = model.predict(data) 38 | inputs_dc.collect(data) #this call is saving our input data into Azure Blob 39 | prediction_dc.collect(result) #this call is saving our input data into Azure Blob 40 | 41 | # You can return any JSON-serializable object. 42 | return { "predict": result.tolist() } 43 | -------------------------------------------------------------------------------- /.github/workflows/train.yml: -------------------------------------------------------------------------------- 1 | # Actions train a model on Azure Machine Learning 2 | name: aml-train 3 | on: [pull_request] 4 | 5 | jobs: 6 | train: 7 | runs-on: ubuntu-latest 8 | steps: 9 | # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it 10 | - name: Check Out Repository 11 | id: checkout_repository 12 | uses: actions/checkout@v2 13 | 14 | # Connect or Create the Azure Machine Learning Workspace 15 | - name: Connect/Create Azure Machine Learning Workspace 16 | id: aml_workspace 17 | uses: Azure/aml-workspace@v1 18 | with: 19 | azure_credentials: ${{ secrets.AZURE_CREDENTIALS }} 20 | 21 | # Connect or Create a Compute Target in Azure Machine Learning 22 | - name: Connect/Create Azure Machine Learning Compute Target 23 | id: aml_compute_training 24 | uses: Azure/aml-compute@v1 25 | with: 26 | azure_credentials: ${{ secrets.AZURE_CREDENTIALS }} 27 | 28 | # Submit a training run to the Azure Machine Learning 29 | - name: Submit training run 30 | id: aml_run 31 | uses: Azure/aml-run@v1 32 | with: 33 | azure_credentials: ${{ secrets.AZURE_CREDENTIALS }} 34 | 35 | - name: comment PR 36 | uses: unsplash/comment-on-pr@master 37 | env: 38 | GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} 39 | with: 40 | msg: "${{ steps.aml_run.outputs.run_metrics }}" 41 | 42 | # Register model in Azure Machine Learning model registry 43 | - name: Register model 44 | id: aml_registermodel 45 | uses: Azure/aml-registermodel@v1 46 | with: 47 | azure_credentials: ${{ secrets.AZURE_CREDENTIALS }} 48 | run_id: ${{ steps.aml_run.outputs.run_id }} 49 | experiment_name: ${{ steps.aml_run.outputs.experiment_name }} 50 | 51 | - name: comment PR 52 | uses: unsplash/comment-on-pr@master 53 | env: 54 | GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} 55 | with: 56 | msg: "${{ steps.aml_registermodel.outputs.model_id }}" 57 | 58 | # - uses: actions/upload-artifact@v1 59 | # with: 60 | # name: ${{ steps.aml_run.outputs.model }} 61 | # path: SpatialSimOutputs 62 | 63 | -------------------------------------------------------------------------------- /SECURITY.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | ## Security 4 | 5 | Microsoft takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organizations, which include [Microsoft](https://github.com/Microsoft), [Azure](https://github.com/Azure), [DotNet](https://github.com/dotnet), [AspNet](https://github.com/aspnet), [Xamarin](https://github.com/xamarin), and [our GitHub organizations](https://opensource.microsoft.com/). 6 | 7 | If you believe you have found a security vulnerability in any Microsoft-owned repository that meets Microsoft's [Microsoft's definition of a security vulnerability](https://docs.microsoft.com/en-us/previous-versions/tn-archive/cc751383(v=technet.10)) of a security vulnerability, please report it to us as described below. 8 | 9 | ## Reporting Security Issues 10 | 11 | **Please do not report security vulnerabilities through public GitHub issues.** 12 | 13 | Instead, please report them to the Microsoft Security Response Center (MSRC) at [https://msrc.microsoft.com/create-report](https://msrc.microsoft.com/create-report). 14 | 15 | If you prefer to submit without logging in, send email to [secure@microsoft.com](mailto:secure@microsoft.com). If possible, encrypt your message with our PGP key; please download it from the the [Microsoft Security Response Center PGP Key page](https://www.microsoft.com/en-us/msrc/pgp-key-msrc). 16 | 17 | You should receive a response within 24 hours. If for some reason you do not, please follow up via email to ensure we received your original message. Additional information can be found at [microsoft.com/msrc](https://www.microsoft.com/msrc). 18 | 19 | Please include the requested information listed below (as much as you can provide) to help us better understand the nature and scope of the possible issue: 20 | 21 | * Type of issue (e.g. buffer overflow, SQL injection, cross-site scripting, etc.) 22 | * Full paths of source file(s) related to the manifestation of the issue 23 | * The location of the affected source code (tag/branch/commit or direct URL) 24 | * Any special configuration required to reproduce the issue 25 | * Step-by-step instructions to reproduce the issue 26 | * Proof-of-concept or exploit code (if possible) 27 | * Impact of the issue, including how an attacker might exploit the issue 28 | 29 | This information will help us triage your report more quickly. 30 | 31 | If you are reporting for a bug bounty, more complete reports can contribute to a higher bounty award. Please visit our [Microsoft Bug Bounty Program](https://microsoft.com/msrc/bounty) page for more details about our active programs. 32 | 33 | ## Preferred Languages 34 | 35 | We prefer all communications to be in English. 36 | 37 | ## Policy 38 | 39 | Microsoft follows the principle of [Coordinated Vulnerability Disclosure](https://www.microsoft.com/en-us/msrc/cvd). 40 | 41 | 42 | -------------------------------------------------------------------------------- /.github/workflows/deploy.yml: -------------------------------------------------------------------------------- 1 | name: aml-deploy 2 | 3 | on: 4 | repository_dispatch: 5 | types: [deploy-command] 6 | 7 | jobs: 8 | deploy: 9 | runs-on: ubuntu-latest 10 | steps: 11 | - name: Check variables 12 | env: 13 | PAYLOAD_CONTEXT: ${{ toJson(github.event.client_payload.slash_command) }} 14 | run: | 15 | if [ -z "${{ github.event.client_payload.slash_command.model_name }}"]; then echo "model_name is NULL"; exit 1; else echo "model_name is ${{ toJson(github.event.client_payload.slash_command.model_name) }}"; fi; 16 | if [ -z "${{ github.event.client_payload.slash_command.model_version }}"]; then echo "model_version is NULL"; exit 1; else echo "model_version is ${{ toJson(github.event.client_payload.slash_command.model_version) }}"; fi; 17 | 18 | # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it 19 | - name: Check Out Repository 20 | id: checkout_repository 21 | uses: actions/checkout@v2 22 | 23 | - name: Create URL to the run output 24 | id: vars 25 | run: echo ::set-output name=run-url::https://github.com/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID 26 | 27 | - name: Create comment 28 | uses: peter-evans/create-or-update-comment@v1 29 | with: 30 | token: ${{ secrets.GITHUB_TOKEN }} 31 | issue-number: ${{ github.event.client_payload.github.payload.issue.number }} 32 | body: | 33 | @${{ github.event.client_payload.github.actor }} Just submit a deploy request. 34 | [Click here to see the deploy run output](${{ steps.vars.outputs.run-url }}) 35 | 36 | # Connect or Create the Azure Machine Learning Workspace 37 | - name: Connect/Create Azure Machine Learning Workspace 38 | id: aml_workspace 39 | uses: Azure/aml-workspace@v1 40 | with: 41 | azure_credentials: ${{ secrets.AZURE_CREDENTIALS }} 42 | 43 | # # Profile your model (TODO) 44 | # - name: Profile Model 45 | # if: github.event.client_payload.slash_command.profile == 'true' 46 | # id: aml_profile 47 | # uses: Azure/aml-profile@master 48 | # with: 49 | # azureCredentials: ${{ secrets.AZURE_CREDENTIALS }} 50 | 51 | # Test your model (this is optional) 52 | - name: Test Model 53 | if: github.event.client_payload.slash_command.test == 'true' 54 | id: aml_test 55 | uses: Azure/aml-deploy@v1 56 | with: 57 | azure_credentials: ${{ secrets.AZURE_CREDENTIALS }} 58 | model_name: ${{ github.event.client_payload.slash_command.model_name }} 59 | model_version: ${{ github.event.client_payload.slash_command.model_version }} 60 | 61 | # Deploy your model to dev (this is optional) 62 | - name: Dev Deploy 63 | if: github.event.client_payload.slash_command.dev == 'true' 64 | id: aml_dev_deploy 65 | uses: Azure/aml-deploy@v1 66 | with: 67 | azure_credentials: ${{ secrets.AZURE_CREDENTIALS }} 68 | model_name: ${{ github.event.client_payload.slash_command.model_name }} 69 | model_version: ${{ github.event.client_payload.slash_command.model_version }} 70 | parameters_file: "dev_aks.json" 71 | 72 | # Deploy your model to production 73 | - name: Prod Deploy 74 | if: github.event.client_payload.slash_command.prod == 'true' 75 | id: aml_prod_deploy 76 | uses: Azure/aml-deploy@v1 77 | with: 78 | azure_credentials: ${{ secrets.AZURE_CREDENTIALS }} 79 | model_name: ${{ github.event.client_payload.slash_command.model_name }} 80 | model_version: ${{ github.event.client_payload.slash_command.model_version }} 81 | parameters_file: "prod_aks.json" 82 | 83 | - name: Create comment 84 | uses: peter-evans/create-or-update-comment@v1 85 | with: 86 | token: ${{ secrets.GITHUB_TOKEN }} 87 | issue-number: ${{ github.event.client_payload.github.payload.issue.number }} 88 | body: | 89 | scoring-endpoint: ${{ steps.aml_prod_deploy.outputs.service_scoring_uri}} 90 | swagger-uri: ${{ steps.aml_prod_deploy.outputs.service_swagger_uri}} 91 | -------------------------------------------------------------------------------- /code/train/train.py: -------------------------------------------------------------------------------- 1 | import os 2 | import argparse 3 | import itertools 4 | import numpy as np 5 | import pandas as pd 6 | import joblib 7 | import matplotlib.pyplot as plt 8 | 9 | from sklearn import datasets 10 | from sklearn.svm import SVC 11 | from sklearn.metrics import confusion_matrix, precision_score, recall_score, f1_score 12 | from sklearn.model_selection import train_test_split 13 | 14 | from azureml.core import Dataset, Run 15 | run = Run.get_context() 16 | 17 | 18 | def log_confusion_matrix_image(cm, labels, normalize=False, log_name='confusion_matrix', title='Confusion matrix', cmap=plt.cm.Blues): 19 | ''' 20 | This function prints and plots the confusion matrix. 21 | Normalization can be applied by setting `normalize=True`. 22 | ''' 23 | if normalize: 24 | cm = cm.astype('float') / cm.sum(axis = 1)[:, np.newaxis] 25 | print('Normalized confusion matrix') 26 | else: 27 | print('Confusion matrix, without normalization') 28 | print(cm) 29 | 30 | plt.figure() 31 | plt.imshow(cm, interpolation = 'nearest', cmap = cmap) 32 | plt.title(title) 33 | plt.colorbar() 34 | tick_marks = np.arange(len(labels)) 35 | plt.xticks(tick_marks, labels, rotation = 45) 36 | plt.yticks(tick_marks, labels) 37 | 38 | fmt = '.2f' if normalize else 'd' 39 | thresh = cm.max() / 2. 40 | for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])): 41 | plt.text(j, i, format(cm[i, j], fmt), horizontalalignment = "center", color = 'white' if cm[i, j] > thresh else 'black') 42 | 43 | plt.ylabel('True label') 44 | plt.xlabel('Predicted label') 45 | plt.tight_layout() 46 | run.log_image(log_name, plot=plt) 47 | plt.savefig(os.path.join('outputs', '{0}.png'.format(log_name))) 48 | 49 | 50 | def log_confusion_matrix(cm, labels): 51 | # log confusion matrix as object 52 | cm_json = { 53 | 'schema_type': 'confusion_matrix', 54 | 'schema_version': 'v1', 55 | 'data': { 56 | 'class_labels': labels, 57 | 'matrix': cm.tolist() 58 | } 59 | } 60 | run.log_confusion_matrix('confusion_matrix', cm_json) 61 | 62 | # log confusion matrix as image 63 | log_confusion_matrix_image(cm, labels, normalize=False, log_name='confusion_matrix_unnormalized', title='Confusion matrix') 64 | 65 | # log normalized confusion matrix as image 66 | log_confusion_matrix_image(cm, labels, normalize=True, log_name='confusion_matrix_normalized', title='Normalized confusion matrix') 67 | 68 | 69 | def main(args): 70 | # create the outputs folder 71 | os.makedirs('outputs', exist_ok=True) 72 | 73 | # Log arguments 74 | run.log('Kernel type', np.str(args.kernel)) 75 | run.log('Penalty', np.float(args.penalty)) 76 | 77 | # Load iris dataset 78 | X, y = datasets.load_iris(return_X_y=True) 79 | 80 | #dividing X,y into train and test data 81 | x_train, x_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=223) 82 | data = {'train': {'X': x_train, 'y': y_train}, 83 | 'test': {'X': x_test, 'y': y_test}} 84 | 85 | # train a SVM classifier 86 | svm_model = SVC(kernel=args.kernel, C=args.penalty, gamma='scale').fit(data['train']['X'], data['train']['y']) 87 | svm_predictions = svm_model.predict(data['test']['X']) 88 | 89 | # accuracy for X_test 90 | accuracy = svm_model.score(data['test']['X'], data['test']['y']) 91 | print('Accuracy of SVM classifier on test set: {:.2f}'.format(accuracy)) 92 | run.log('Accuracy', np.float(accuracy)) 93 | 94 | # precision for X_test 95 | precision = precision_score(svm_predictions, data["test"]["y"], average='weighted') 96 | print('Precision of SVM classifier on test set: {:.2f}'.format(precision)) 97 | run.log('precision', precision) 98 | 99 | # recall for X_test 100 | recall = recall_score(svm_predictions, data["test"]["y"], average='weighted') 101 | print('Recall of SVM classifier on test set: {:.2f}'.format(recall)) 102 | run.log('recall', recall) 103 | 104 | # f1-score for X_test 105 | f1 = f1_score(svm_predictions, data["test"]["y"], average='weighted') 106 | print('F1-Score of SVM classifier on test set: {:.2f}'.format(f1)) 107 | run.log('f1-score', f1) 108 | 109 | # create a confusion matrix 110 | labels = ['Iris-setosa', 'Iris-versicolor', 'Iris-virginica'] 111 | labels_numbers = [0, 1, 2] 112 | cm = confusion_matrix(y_test, svm_predictions, labels_numbers) 113 | log_confusion_matrix(cm, labels) 114 | 115 | # files saved in the "outputs" folder are automatically uploaded into run history 116 | joblib.dump(svm_model, os.path.join('outputs', args.modelname)) 117 | run.log('Model Name', np.str(args.modelname)) 118 | 119 | 120 | def parse_args(): 121 | parser = argparse.ArgumentParser() 122 | parser.add_argument('--kernel', type=str, default='rbf', help='Kernel type to be used in the algorithm') 123 | parser.add_argument('--penalty', type=float, default=1.0, help='Penalty parameter of the error term') 124 | parser.add_argument('--modelname', type=str, default='model.pkl', help='Name of the model file') 125 | args = parser.parse_args() 126 | return args 127 | 128 | 129 | if __name__ == '__main__': 130 | args = parse_args() 131 | main(args=args) 132 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | ## Ignore Visual Studio temporary files, build results, and 2 | ## files generated by popular Visual Studio add-ons. 3 | ## 4 | ## Get latest from https://github.com/github/gitignore/blob/master/VisualStudio.gitignore 5 | 6 | # User-specific files 7 | *.rsuser 8 | *.suo 9 | *.user 10 | *.userosscache 11 | *.sln.docstates 12 | 13 | # User-specific files (MonoDevelop/Xamarin Studio) 14 | *.userprefs 15 | 16 | # Build results 17 | [Dd]ebug/ 18 | [Dd]ebugPublic/ 19 | [Rr]elease/ 20 | [Rr]eleases/ 21 | x64/ 22 | x86/ 23 | [Aa][Rr][Mm]/ 24 | [Aa][Rr][Mm]64/ 25 | bld/ 26 | [Bb]in/ 27 | [Oo]bj/ 28 | [Ll]og/ 29 | 30 | # Visual Studio 2015/2017 cache/options directory 31 | .vs/ 32 | # Uncomment if you have tasks that create the project's static files in wwwroot 33 | #wwwroot/ 34 | 35 | # Visual Studio 2017 auto generated files 36 | Generated\ Files/ 37 | 38 | # MSTest test Results 39 | [Tt]est[Rr]esult*/ 40 | [Bb]uild[Ll]og.* 41 | 42 | # NUNIT 43 | *.VisualState.xml 44 | TestResult.xml 45 | 46 | # Build Results of an ATL Project 47 | [Dd]ebugPS/ 48 | [Rr]eleasePS/ 49 | dlldata.c 50 | 51 | # Benchmark Results 52 | BenchmarkDotNet.Artifacts/ 53 | 54 | # .NET Core 55 | project.lock.json 56 | project.fragment.lock.json 57 | artifacts/ 58 | 59 | # StyleCop 60 | StyleCopReport.xml 61 | 62 | # Files built by Visual Studio 63 | *_i.c 64 | *_p.c 65 | *_h.h 66 | *.ilk 67 | *.meta 68 | *.obj 69 | *.iobj 70 | *.pch 71 | *.pdb 72 | *.ipdb 73 | *.pgc 74 | *.pgd 75 | *.rsp 76 | *.sbr 77 | *.tlb 78 | *.tli 79 | *.tlh 80 | *.tmp 81 | *.tmp_proj 82 | *_wpftmp.csproj 83 | *.log 84 | *.vspscc 85 | *.vssscc 86 | .builds 87 | *.pidb 88 | *.svclog 89 | *.scc 90 | 91 | # Chutzpah Test files 92 | _Chutzpah* 93 | 94 | # Visual C++ cache files 95 | ipch/ 96 | *.aps 97 | *.ncb 98 | *.opendb 99 | *.opensdf 100 | *.sdf 101 | *.cachefile 102 | *.VC.db 103 | *.VC.VC.opendb 104 | 105 | # Visual Studio profiler 106 | *.psess 107 | *.vsp 108 | *.vspx 109 | *.sap 110 | 111 | # Visual Studio Trace Files 112 | *.e2e 113 | 114 | # TFS 2012 Local Workspace 115 | $tf/ 116 | 117 | # Guidance Automation Toolkit 118 | *.gpState 119 | 120 | # ReSharper is a .NET coding add-in 121 | _ReSharper*/ 122 | *.[Rr]e[Ss]harper 123 | *.DotSettings.user 124 | 125 | # JustCode is a .NET coding add-in 126 | .JustCode 127 | 128 | # TeamCity is a build add-in 129 | _TeamCity* 130 | 131 | # DotCover is a Code Coverage Tool 132 | *.dotCover 133 | 134 | # AxoCover is a Code Coverage Tool 135 | .axoCover/* 136 | !.axoCover/settings.json 137 | 138 | # Visual Studio code coverage results 139 | *.coverage 140 | *.coveragexml 141 | 142 | # NCrunch 143 | _NCrunch_* 144 | .*crunch*.local.xml 145 | nCrunchTemp_* 146 | 147 | # MightyMoose 148 | *.mm.* 149 | AutoTest.Net/ 150 | 151 | # Web workbench (sass) 152 | .sass-cache/ 153 | 154 | # Installshield output folder 155 | [Ee]xpress/ 156 | 157 | # DocProject is a documentation generator add-in 158 | DocProject/buildhelp/ 159 | DocProject/Help/*.HxT 160 | DocProject/Help/*.HxC 161 | DocProject/Help/*.hhc 162 | DocProject/Help/*.hhk 163 | DocProject/Help/*.hhp 164 | DocProject/Help/Html2 165 | DocProject/Help/html 166 | 167 | # Click-Once directory 168 | publish/ 169 | 170 | # Publish Web Output 171 | *.[Pp]ublish.xml 172 | *.azurePubxml 173 | # Note: Comment the next line if you want to checkin your web deploy settings, 174 | # but database connection strings (with potential passwords) will be unencrypted 175 | *.pubxml 176 | *.publishproj 177 | 178 | # Microsoft Azure Web App publish settings. Comment the next line if you want to 179 | # checkin your Azure Web App publish settings, but sensitive information contained 180 | # in these scripts will be unencrypted 181 | PublishScripts/ 182 | 183 | # NuGet Packages 184 | *.nupkg 185 | # The packages folder can be ignored because of Package Restore 186 | **/[Pp]ackages/* 187 | # except build/, which is used as an MSBuild target. 188 | !**/[Pp]ackages/build/ 189 | # Uncomment if necessary however generally it will be regenerated when needed 190 | #!**/[Pp]ackages/repositories.config 191 | # NuGet v3's project.json files produces more ignorable files 192 | *.nuget.props 193 | *.nuget.targets 194 | 195 | # Microsoft Azure Build Output 196 | csx/ 197 | *.build.csdef 198 | 199 | # Microsoft Azure Emulator 200 | ecf/ 201 | rcf/ 202 | 203 | # Windows Store app package directories and files 204 | AppPackages/ 205 | BundleArtifacts/ 206 | Package.StoreAssociation.xml 207 | _pkginfo.txt 208 | *.appx 209 | 210 | # Visual Studio cache files 211 | # files ending in .cache can be ignored 212 | *.[Cc]ache 213 | # but keep track of directories ending in .cache 214 | !?*.[Cc]ache/ 215 | 216 | # Others 217 | ClientBin/ 218 | ~$* 219 | *~ 220 | *.dbmdl 221 | *.dbproj.schemaview 222 | *.jfm 223 | *.pfx 224 | *.publishsettings 225 | orleans.codegen.cs 226 | 227 | # Including strong name files can present a security risk 228 | # (https://github.com/github/gitignore/pull/2483#issue-259490424) 229 | #*.snk 230 | 231 | # Since there are multiple workflows, uncomment next line to ignore bower_components 232 | # (https://github.com/github/gitignore/pull/1529#issuecomment-104372622) 233 | #bower_components/ 234 | 235 | # RIA/Silverlight projects 236 | Generated_Code/ 237 | 238 | # Backup & report files from converting an old project file 239 | # to a newer Visual Studio version. Backup files are not needed, 240 | # because we have git ;-) 241 | _UpgradeReport_Files/ 242 | Backup*/ 243 | UpgradeLog*.XML 244 | UpgradeLog*.htm 245 | ServiceFabricBackup/ 246 | *.rptproj.bak 247 | 248 | # SQL Server files 249 | *.mdf 250 | *.ldf 251 | *.ndf 252 | 253 | # Business Intelligence projects 254 | *.rdl.data 255 | *.bim.layout 256 | *.bim_*.settings 257 | *.rptproj.rsuser 258 | *- Backup*.rdl 259 | 260 | # Microsoft Fakes 261 | FakesAssemblies/ 262 | 263 | # GhostDoc plugin setting file 264 | *.GhostDoc.xml 265 | 266 | # Node.js Tools for Visual Studio 267 | .ntvs_analysis.dat 268 | node_modules/ 269 | 270 | # Visual Studio 6 build log 271 | *.plg 272 | 273 | # Visual Studio 6 workspace options file 274 | *.opt 275 | 276 | # Visual Studio 6 auto-generated workspace file (contains which files were open etc.) 277 | *.vbw 278 | 279 | # Visual Studio LightSwitch build output 280 | **/*.HTMLClient/GeneratedArtifacts 281 | **/*.DesktopClient/GeneratedArtifacts 282 | **/*.DesktopClient/ModelManifest.xml 283 | **/*.Server/GeneratedArtifacts 284 | **/*.Server/ModelManifest.xml 285 | _Pvt_Extensions 286 | 287 | # Paket dependency manager 288 | .paket/paket.exe 289 | paket-files/ 290 | 291 | # FAKE - F# Make 292 | .fake/ 293 | 294 | # JetBrains Rider 295 | .idea/ 296 | *.sln.iml 297 | 298 | # CodeRush personal settings 299 | .cr/personal 300 | 301 | # Python Tools for Visual Studio (PTVS) 302 | __pycache__/ 303 | *.pyc 304 | 305 | # Cake - Uncomment if you are using it 306 | # tools/** 307 | # !tools/packages.config 308 | 309 | # Tabs Studio 310 | *.tss 311 | 312 | # Telerik's JustMock configuration file 313 | *.jmconfig 314 | 315 | # BizTalk build output 316 | *.btp.cs 317 | *.btm.cs 318 | *.odx.cs 319 | *.xsd.cs 320 | 321 | # OpenCover UI analysis results 322 | OpenCover/ 323 | 324 | # Azure Stream Analytics local run output 325 | ASALocalRun/ 326 | 327 | # MSBuild Binary and Structured Log 328 | *.binlog 329 | 330 | # NVidia Nsight GPU debugger configuration file 331 | *.nvuser 332 | 333 | # MFractors (Xamarin productivity tool) working folder 334 | .mfractor/ 335 | 336 | # Local History for Visual Studio 337 | .localhistory/ 338 | 339 | # BeatPulse healthcheck temp database 340 | healthchecksdb -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # ML Ops with GitHub Actions and AML 2 | 3 |

4 | 5 | plus 6 | Azure Machine Learning + Actions 7 |

8 | 9 | This template shows the more extensive capabilities of using [GitHub Actions](https://github.com/features/actions) with [Azure Machine Learning](https://docs.microsoft.com/en-us/azure/machine-learning/) managing a machine learning project with automated training and deployment. For a more simplified version of this automated pipeline, see the [ml-template-azure](https://github.com/machine-learning-apps/ml-template-azure) repository. 10 | 11 | # Getting started 12 | 13 | ### 1. Prerequisites 14 | 15 | The following prerequisites are required to make this repository work: 16 | - Azure subscription 17 | - Contributor access to the Azure subscription 18 | - Access to [GitHub Actions](https://github.com/features/actions) 19 | 20 | If you don’t have an Azure subscription, create a free account before you begin. Try the [free or paid version of Azure Machine Learning](https://aka.ms/AMLFree) today. 21 | 22 | ### 2. Create repository 23 | 24 | To get started with ML Ops, simply create a new repo based off this template, by clicking on the green "Use this template" button: 25 | 26 |

27 | GitHub Template repository 28 |

29 | 30 | ### 3. Setting up the required secrets 31 | 32 | A service principal needs to be generated for authentication and getting access to your Azure subscription. We suggest adding a service principal with contributor rights to a new resource group or to the one where you have deployed your existing Azure Machine Learning workspace. Just go to the Azure Portal to find the details of your resource group or workspace. Then start the Cloud CLI or install the [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest) on your computer and execute the following command to generate the required credentials: 33 | 34 | ```sh 35 | # Replace {service-principal-name}, {subscription-id} and {resource-group} with your 36 | # Azure subscription id and resource group name and any name for your service principle 37 | az ad sp create-for-rbac --name {service-principal-name} \ 38 | --role contributor \ 39 | --scopes /subscriptions/{subscription-id}/resourceGroups/{resource-group} \ 40 | --sdk-auth 41 | ``` 42 | 43 | This will generate the following JSON output: 44 | 45 | ```sh 46 | { 47 | "clientId": "", 48 | "clientSecret": "", 49 | "subscriptionId": "", 50 | "tenantId": "", 51 | (...) 52 | } 53 | ``` 54 | 55 | Add this JSON output as [a secret](https://help.github.com/en/actions/configuring-and-managing-workflows/creating-and-storing-encrypted-secrets#creating-encrypted-secrets) with the name `AZURE_CREDENTIALS` in your GitHub repository: 56 | 57 |

58 | GitHub Template repository 59 |

60 | 61 | To do so, click on the Settings tab in your repository, then click on Secrets and finally add the new secret with the name `AZURE_CREDENTIALS` to your repository. 62 | 63 | Please follow [this link](https://help.github.com/en/actions/configuring-and-managing-workflows/creating-and-storing-encrypted-secrets#creating-encrypted-secrets) for more details. 64 | 65 | ### 4. Define your workspace parameters 66 | 67 | You have to modify the parameters in the `/.cloud/.azure/workspace.json"` file in your repository, so that the GitHub Actions create or connect to the desired Azure Machine Learning workspace. Just click on the link and edit the file. 68 | 69 | Please use the same value for the `resource_group` parameter that you have used when generating the azure credentials. If you already have an Azure ML Workspace under that resource group, change the `name` parameter in the JSON file to the name of your workspace, if you want the Action to create a new workspace in that resource group, pick a name for your new workspace, and assign it to the `name` parameter. You can also delete the `name` parameter, if you want the action to use the default value, which is the repository name. Additionally, if the workspace doesn't exist you will need to enable the `create_workspace` property (set it to `true`) in the same file. 70 | 71 | Once you save your changes to the file, the predefined GitHub workflow that trains and deploys a model on Azure Machine Learning gets triggered. Check the actions tab to view if your actions have successfully run. 72 | 73 |

74 | GitHub Actions Tab 75 |

76 | 77 | ### 5. Modify the code 78 | 79 | Now you can start modifying the code in the `code` folder, so that your model and not the provided sample model gets trained on Azure. Where required, modify the environment yaml so that the training and deployment environments will have the correct packages installed in the conda environment for your training and deployment. 80 | Upon pushing the changes, actions will kick off your training and deployment run. Check the actions tab to view if your actions have successfully run. 81 | 82 | Comment lines 39 to 55 in your `"/.github/workflows/train_deploy.yml"` file if you only want to train the model. Uncomment line 7 to 8, if you only want to kick off the workflow when pushing changes to the `"/code/"` file. 83 | 84 | ### 6. Viewing your AML resources and runs 85 | 86 | The log outputs of your action will provide URLs for you to view the resources that have been created in AML. Alternatively, you can visit the [Machine Learning Studio](https://ml.azure.com/) to view the progress of your runs, etc. For more details, read the documentation below. 87 | 88 | # Documentation 89 | 90 | ## Code structure 91 | 92 | | File/folder | Description | 93 | | ----------------------------- | ------------------------------------------ | 94 | | `code` | Sample data science source code that will be submitted to Azure Machine Learning to train and deploy machine learning models. | 95 | | `code/train` | Sample code that is required for training a model on Azure Machine Learning. | 96 | | `code/train/train.py` | Training script that gets executed on a cluster on Azure Machine Learning. | 97 | | `code/train/environment.yml` | Conda environment specification, which describes the dependencies of `train.py`. These packages will be installed inside a Docker image on the Azure Machine Learning compute cluster, when executing your `train.py`. | 98 | | `code/train/run_config.yml` | YAML files, which describes the execution of your training run on Azure Machine Learning. This file also references your `environment.yml`. Please look at the comments in the file for more details. | 99 | | `code/deploy` | Sample code that is required for deploying a model on Azure Machine Learning. | 100 | | `code/deploy/score.py` | Inference script that is used to build a Docker image and that gets executed within the container when you send data to the deployed model on Azure Machine Learning. | 101 | | `code/deploy/environment.yml` | Conda environment specification, which describes the dependencies of `score.py`. These packages will be installed inside the Docker image that will be used for deploying your model. | 102 | | `code/test/test.py` | Test script that can be used for testing your deployed webservice. Add a `deploy.json` to the `.cloud/.azure` folder and add the following code `{ "test_enabled": true }` to enable tests of your webservice. Change the code according to the tests that zou would like to execute. | 103 | | `.cloud/.azure` | Configuration files for the Azure Machine Learning GitHub Actions. Please visit the repositories of the respective actions and read the documentation for more details. | 104 | | `.github/workflows` | Folder for GitHub workflows. The `train_deploy.yml` sample workflow shows you how your can use the Azure Machine Learning GitHub Actions to automate the machine learning process. | 105 | | `docs` | Resources for this README. | 106 | | `CODE_OF_CONDUCT.md` | Microsoft Open Source Code of Conduct. | 107 | | `LICENSE` | The license for the sample. | 108 | | `README.md` | This README file. | 109 | | `SECURITY.md` | Microsoft Security README. | 110 | 111 | ## Documentation of Azure Machine Learning GitHub Actions 112 | 113 | The template uses the open source Azure certified Actions listed below. Click on the links and read the README files for more details. 114 | - [aml-workspace](https://github.com/Azure/aml-workspace) - Connects to or creates a new workspace 115 | - [aml-compute](https://github.com/Azure/aml-compute) - Connects to or creates a new compute target in Azure Machine Learning 116 | - [aml-run](https://github.com/Azure/aml-run) - Submits a ScriptRun, an Estimator or a Pipeline to Azure Machine Learning 117 | - [aml-registermodel](https://github.com/Azure/aml-registermodel) - Registers a model to Azure Machine Learning 118 | - [aml-deploy](https://github.com/Azure/aml-deploy) - Deploys a model and creates an endpoint for the model 119 | 120 | ## Known issues 121 | 122 | ### Error: MissingSubscriptionRegistration 123 | 124 | Error message: 125 | ```sh 126 | Message: ***'error': ***'code': 'MissingSubscriptionRegistration', 'message': "The subscription is not registered to use namespace 'Microsoft.KeyVault'. See https://aka.ms/rps-not-found for how to register subscriptions.", 'details': [***'code': 'MissingSubscriptionRegistration', 'target': 'Microsoft.KeyVault', 'message': "The subscription is not registered to use namespace 'Microsoft.KeyVault'. See https://aka.ms/rps-not-found for how to register subscriptions 127 | ``` 128 | Solution: 129 | 130 | This error message appears, in case the `Azure/aml-workspace` action tries to create a new Azure Machine Learning workspace in your resource group and you have never deployed a Key Vault in the subscription before. We recommend to create an Azure Machine Learning workspace manually in the Azure Portal. Follow the [steps on this website](https://docs.microsoft.com/en-us/azure/machine-learning/tutorial-1st-experiment-sdk-setup#create-a-workspace) to create a new workspace with the desired name. After ou have successfully completed the steps, you have to make sure, that your Service Principal has access to the resource group and that the details in your `/.cloud/.azure/workspace.json"` file are correct and point to the right workspace and resource group. 131 | 132 | # What is MLOps? 133 | 134 |

135 | Azure Machine Learning Lifecycle 136 |

137 | 138 | MLOps empowers data scientists and machine learning engineers to bring together their knowledge and skills to simplify the process of going from model development to release/deployment. ML Ops enables you to track, version, test, certify and reuse assets in every part of the machine learning lifecycle and provides orchestration services to streamline managing this lifecycle. This allows practitioners to automate the end to end machine Learning lifecycle to frequently update models, test new models, and continuously roll out new ML models alongside your other applications and services. 139 | 140 | This repository enables Data Scientists to focus on the training and deployment code of their machine learning project (`code` folder of this repository). Once new code is checked into the `code` folder of the master branch of this repository the GitHub workflow is triggered and open source Azure Machine Learning actions are used to automatically manage the training through to deployment phases. 141 | 142 | # Contributing 143 | 144 | This project welcomes contributions and suggestions. Most contributions require you to agree to a 145 | Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us 146 | the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com. 147 | 148 | When you submit a pull request, a CLA bot will automatically determine whether you need to provide 149 | a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions 150 | provided by the bot. You will only need to do this once across all repos using our CLA. 151 | 152 | This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). 153 | For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or 154 | contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. 155 | 156 | --------------------------------------------------------------------------------