├── .amlignore ├── .gitignore ├── README.md ├── data └── german_credit_data.csv ├── media └── stages.png ├── models └── german-credit-basic │ ├── .amlignore │ ├── README.md │ ├── config │ ├── deployment-config-aci.yml │ ├── deployment-config-aks.yml │ ├── inference-conda.yml │ ├── inference-config.yml │ ├── train-amlcompute.runconfig │ ├── train-conda.yml │ └── train-local.runconfig │ ├── notebooks │ ├── .aml_ignore │ ├── deploy_webservices.ipynb │ ├── german-credit-amlcompute.ipynb │ ├── german-credit-local.ipynb │ ├── german-credit-training-pipeline.ipynb │ └── test_webservices.ipynb │ ├── pipelines │ ├── register.py │ └── training.py │ ├── score.py │ └── train.py ├── pipelines-approval ├── german-credit-config.yml ├── german-credit-deploy-model.yml ├── german-credit-full-approval-flow.yml └── german-credit-publish-pipeline.yml ├── pipelines-staged ├── README.md ├── german-credit-config-dev.yml ├── german-credit-config-prod.yml ├── german-credit-rollout.yml ├── template-deploy.yml └── template-train-and-register.yml ├── pipelines ├── german-credit-config.yml ├── german-credit-deploy.yml └── german-credit-train-and-register.yml └── templates ├── connect-to-workspace.yml ├── deploy-model-by-name.yml ├── deploy-model.yml ├── deploy-pipeline.yml ├── download-artifacts.yml ├── install-aml-cli.yml ├── install-aml-sdk.yml ├── publish-artifacts.yml ├── register-model.yml └── train-model.yml /.amlignore: -------------------------------------------------------------------------------- 1 | .ipynb_checkpoints 2 | azureml-logs 3 | .azureml 4 | .git 5 | outputs 6 | azureml-setup 7 | docs 8 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | .vscode/ 7 | 8 | # AML/notebooks specific 9 | model.json 10 | run.json 11 | .azureml/ 12 | .ipynb_aml_checkpoints/ 13 | explanation/ 14 | *.amltmp 15 | model.pkl 16 | 17 | # C extensions 18 | *.so 19 | 20 | # Distribution / packaging 21 | .Python 22 | build/ 23 | develop-eggs/ 24 | dist/ 25 | downloads/ 26 | eggs/ 27 | .eggs/ 28 | lib/ 29 | lib64/ 30 | parts/ 31 | sdist/ 32 | var/ 33 | wheels/ 34 | pip-wheel-metadata/ 35 | share/python-wheels/ 36 | *.egg-info/ 37 | .installed.cfg 38 | *.egg 39 | MANIFEST 40 | temp/ 41 | 42 | # PyInstaller 43 | # Usually these files are written by a python script from a template 44 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 45 | *.manifest 46 | *.spec 47 | 48 | # Installer logs 49 | pip-log.txt 50 | pip-delete-this-directory.txt 51 | 52 | # Unit test / coverage reports 53 | htmlcov/ 54 | .tox/ 55 | .nox/ 56 | .coverage 57 | .coverage.* 58 | .cache 59 | nosetests.xml 60 | coverage.xml 61 | *.cover 62 | *.py,cover 63 | .hypothesis/ 64 | .pytest_cache/ 65 | 66 | # Translations 67 | *.mo 68 | *.pot 69 | 70 | # Django stuff: 71 | *.log 72 | local_settings.py 73 | db.sqlite3 74 | db.sqlite3-journal 75 | 76 | # Flask stuff: 77 | instance/ 78 | .webassets-cache 79 | 80 | # Scrapy stuff: 81 | .scrapy 82 | 83 | # Sphinx documentation 84 | docs/_build/ 85 | 86 | # PyBuilder 87 | target/ 88 | 89 | # Jupyter Notebook 90 | .ipynb_checkpoints 91 | 92 | # IPython 93 | profile_default/ 94 | ipython_config.py 95 | 96 | # pyenv 97 | .python-version 98 | 99 | # pipenv 100 | # According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control. 101 | # However, in case of collaboration, if having platform-specific dependencies or dependencies 102 | # having no cross-platform support, pipenv may install dependencies that don't work, or not 103 | # install all needed dependencies. 104 | #Pipfile.lock 105 | 106 | # PEP 582; used by e.g. github.com/David-OConnor/pyflow 107 | __pypackages__/ 108 | 109 | # Celery stuff 110 | celerybeat-schedule 111 | celerybeat.pid 112 | 113 | # SageMath parsed files 114 | *.sage.py 115 | 116 | # Environments 117 | .env 118 | .venv 119 | env/ 120 | venv/ 121 | ENV/ 122 | env.bak/ 123 | venv.bak/ 124 | 125 | # Spyder project settings 126 | .spyderproject 127 | .spyproject 128 | 129 | # Rope project settings 130 | .ropeproject 131 | 132 | # mkdocs documentation 133 | /site 134 | 135 | # mypy 136 | .mypy_cache/ 137 | .dmypy.json 138 | dmypy.json 139 | 140 | # Pyre type checker 141 | .pyre/ 142 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # mlops-demo 2 | 3 | This repo shows some introduction examples to Azure Machine Learning and a simple MLOps implemenation for automating model training and deployment. 4 | 5 | ## Setup & Demo Flow 6 | 7 | This gives a short, high-level overview of how this repo may be used. 8 | 9 | ### Interactive demo part 10 | 11 | 1. If required, create an Azure Machine Learning workspace 12 | 1. Create a tabular dataset from [`data/german_credit_data.csv`](data/german_credit_data.csv) and name it `german_credit_dataset` (download the file to your machine and select `From Local File` when creating a new Dataset) 13 | 1. Create a file dataset from [`data/german_credit_data.csv`](data/german_credit_data.csv) and name it `german_credit_file` (use `From Datastore` and point to the same file as in the prior step) 14 | 1. Clone the whole repo into a Compute Instance 15 | 1. Walk through the following notebooks 16 | * [`models/german-credit-basic/notebooks/german-credit-local.ipynb`](models/german-credit-basic/notebooks/german-credit-local.ipynb) - Shows how to run local training inside the Compute Instance, registers the model with data linage, and calculates the model explainability 17 | * [`models/german-credit-basic/notebooks/german-credit-amlcompute.ipynb`](models/german-credit-basic/notebooks/german-credit-amlcompute.ipynb) - Shows how to train the same model on a Compute Cluster 18 | * [`models/german-credit-basic/notebooks/deploy_webservices.ipynb`](models/german-credit-basic/notebooks/deploy_webservices.ipynb) - Shows how to deploy the trained model to an Azure Container Instance 19 | 1. For deployment to AKS, make sure to create an AKS cluster in AML that has `SSL enabled` (using the Microsoft certificate) 20 | 21 | ### MLOps demo part 22 | 23 | #### Simple MLOps pipelines 24 | 25 | 1. Create a new project in Azure DevOps 26 | 1. Fork this repo or import it into Azure DevOps (so that you can make changes to the repo) 27 | 1. Create a service connection to your Azure Machine Learning workspace and use the name `aml-workspace-connection` 28 | 1. Edit [`pipelines/german-credit-config.yml`](pipelines/german-credit-config.yml) and adapt the values to point to your workspace 29 | 1. Import the following pipelines into DevOps 30 | * [`pipelines/german-credit-train-and-register.yml`](pipelines/german-credit-train-and-register.yml) - Trains and registers the model automatically 31 | * [`pipelines/german-credit-deploy.yml`](pipelines/german-credit-deploy.yml) - Deploys the trained model to AKS 32 | 1. Run the pipelines 33 | 34 | #### Staged MLOps pipelines 35 | 36 | 1. First, get the simple pipelines from [`pipelines/`](pipelines/) running 37 | 1. Edit [`pipelines-staged/german-credit-config-dev.yml`](pipelines-staged/german-credit-config-dev.yml) and [`pipelines-staged/german-credit-config-prod.yml`](pipelines-staged/german-credit-config-prod.yml) and adapt the values to point to your dev and prod workspaces 38 | 1. Import the following pipeline into DevOps 39 | * [`pipelines-staged/german-credit-rollout.yml`](pipelines-staged/german-credit-rollout.yml) - Trains, registers, and deploys the model automatically to a Dev environment, once successful, the same is repeated in a Prod environment 40 | 1. Run the pipeline 41 | 42 | ## Conventions 43 | 44 | This repo is fully based on conventions in order to make MLOps reusable and easily scaleable. 45 | The directory structure is as follows: 46 | 47 | ``` 48 | pipelines 49 | \- german-credit-config.yml - Configuration for german credit model 50 | \- german-credit-deploy.yml - Deployment pipeline for german credit model 51 | \- german-credit-train-and-register.yml - Pipline for training and registering the base german credit model 52 | models 53 | \- model1 54 | train.py (entry file for training) 55 | score.py (entry file for scoring) 56 | \- config 57 | deployment-config-aks.yml - Deployment infrastructure definition (e.g., AKS configuration) 58 | inference-conda.yml - Conda environement definition for inferencing/scoring 59 | inference-config.yml - Azure Machine Learning config for inferencing 60 | train-conda.yml - Conda environement definition for training 61 | \- model2 62 | ...same file and folder structure... 63 | ``` 64 | 65 | ## Testing 66 | 67 | This snipped can be used to manually showcase/test the deployed model on ACI: 68 | 69 | ```python 70 | import requests 71 | import json 72 | 73 | url = '' 74 | key = '' 75 | 76 | test_data = { 77 | 'data': [{ 78 | "Age": 20, 79 | "Sex": "male", 80 | "Job": 0, 81 | "Housing": "own", 82 | "Saving accounts": "little", 83 | "Checking account": "little", 84 | "Credit amount": 100, 85 | "Duration": 48, 86 | "Purpose": "radio/TV" 87 | }] 88 | } 89 | 90 | headers = {'Content-Type': 'application/json', 'Authorization': 'Bearer ' + key} 91 | resp = requests.post(url, json=test_data, headers=headers) 92 | 93 | print("Prediction (good, bad):", resp.text) 94 | ``` 95 | 96 | ## Further Work 97 | 98 | :star: A fully documented starting template for Azure Machine Leraning with MLOps can be found here: [microsoft/aml-acceleration-template](https://github.com/microsoft/aml-acceleration-template/). This includes model training, validation, testing, deployment, pipelines, and several other production-grade capabilties. -------------------------------------------------------------------------------- /data/german_credit_data.csv: -------------------------------------------------------------------------------- 1 | Sno,Age,Sex,Job,Housing,Saving accounts,Checking account,Credit amount,Duration,Purpose,Risk 2 | 0,67,male,2,own,NA,little,1169,6,radio/TV,good 3 | 1,22,female,2,own,little,moderate,5951,48,radio/TV,bad 4 | 2,49,male,1,own,little,NA,2096,12,education,good 5 | 3,45,male,2,free,little,little,7882,42,furniture/equipment,good 6 | 4,53,male,2,free,little,little,4870,24,car,bad 7 | 5,35,male,1,free,NA,NA,9055,36,education,good 8 | 6,53,male,2,own,quite rich,NA,2835,24,furniture/equipment,good 9 | 7,35,male,3,rent,little,moderate,6948,36,car,good 10 | 8,61,male,1,own,rich,NA,3059,12,radio/TV,good 11 | 9,28,male,3,own,little,moderate,5234,30,car,bad 12 | 10,25,female,2,rent,little,moderate,1295,12,car,bad 13 | 11,24,female,2,rent,little,little,4308,48,business,bad 14 | 12,22,female,2,own,little,moderate,1567,12,radio/TV,good 15 | 13,60,male,1,own,little,little,1199,24,car,bad 16 | 14,28,female,2,rent,little,little,1403,15,car,good 17 | 15,32,female,1,own,moderate,little,1282,24,radio/TV,bad 18 | 16,53,male,2,own,NA,NA,2424,24,radio/TV,good 19 | 17,25,male,2,own,NA,little,8072,30,business,good 20 | 18,44,female,3,free,little,moderate,12579,24,car,bad 21 | 19,31,male,2,own,quite rich,NA,3430,24,radio/TV,good 22 | 20,48,male,2,own,little,NA,2134,9,car,good 23 | 21,44,male,2,rent,quite rich,little,2647,6,radio/TV,good 24 | 22,48,male,1,rent,little,little,2241,10,car,good 25 | 23,44,male,2,own,moderate,moderate,1804,12,car,good 26 | 24,26,male,2,own,NA,NA,2069,10,furniture/equipment,good 27 | 25,36,male,1,own,little,little,1374,6,furniture/equipment,good 28 | 26,39,male,1,own,little,NA,426,6,radio/TV,good 29 | 27,42,female,2,rent,rich,rich,409,12,radio/TV,good 30 | 28,34,male,2,own,little,moderate,2415,7,radio/TV,good 31 | 29,63,male,2,own,little,little,6836,60,business,bad 32 | 30,36,male,2,own,rich,moderate,1913,18,business,good 33 | 31,27,male,2,own,little,little,4020,24,furniture/equipment,good 34 | 32,30,male,2,own,moderate,moderate,5866,18,car,good 35 | 33,57,male,1,rent,NA,NA,1264,12,business,good 36 | 34,33,female,3,own,little,rich,1474,12,furniture/equipment,good 37 | 35,25,male,1,own,little,moderate,4746,45,radio/TV,bad 38 | 36,31,male,2,free,little,NA,6110,48,education,good 39 | 37,37,male,2,own,little,rich,2100,18,radio/TV,bad 40 | 38,37,male,2,own,little,rich,1225,10,domestic appliances,good 41 | 39,24,male,2,own,little,moderate,458,9,radio/TV,good 42 | 40,30,male,3,own,quite rich,NA,2333,30,radio/TV,good 43 | 41,26,male,2,own,quite rich,moderate,1158,12,radio/TV,good 44 | 42,44,male,1,own,little,moderate,6204,18,repairs,good 45 | 43,24,male,2,rent,moderate,little,6187,30,car,good 46 | 44,58,female,1,free,little,little,6143,48,car,bad 47 | 45,35,female,3,own,little,NA,1393,11,car,good 48 | 46,39,male,2,own,quite rich,NA,2299,36,radio/TV,good 49 | 47,23,female,0,rent,quite rich,little,1352,6,car,good 50 | 48,39,male,1,own,little,NA,7228,11,car,good 51 | 49,28,female,2,own,moderate,NA,2073,12,radio/TV,good 52 | 50,29,male,1,own,NA,moderate,2333,24,furniture/equipment,good 53 | 51,30,male,3,own,little,moderate,5965,27,car,good 54 | 52,25,male,2,own,little,NA,1262,12,radio/TV,good 55 | 53,31,male,2,own,NA,NA,3378,18,car,good 56 | 54,57,male,2,free,little,moderate,2225,36,car,bad 57 | 55,26,male,1,own,NA,NA,783,6,car,good 58 | 56,52,male,3,own,NA,moderate,6468,12,radio/TV,bad 59 | 57,31,female,2,own,little,NA,9566,36,radio/TV,good 60 | 58,23,female,3,own,little,rich,1961,18,car,good 61 | 59,23,female,1,rent,little,little,6229,36,furniture/equipment,bad 62 | 60,27,male,2,own,little,moderate,1391,9,business,good 63 | 61,50,male,2,own,NA,moderate,1537,15,radio/TV,good 64 | 62,61,male,3,free,little,moderate,1953,36,business,bad 65 | 63,25,male,2,own,little,moderate,14421,48,business,bad 66 | 64,26,female,2,own,little,NA,3181,24,radio/TV,good 67 | 65,48,male,2,own,NA,NA,5190,27,repairs,good 68 | 66,29,female,2,own,little,NA,2171,12,radio/TV,good 69 | 67,22,male,2,own,rich,moderate,1007,12,car,good 70 | 68,37,male,2,free,little,NA,1819,36,education,bad 71 | 69,25,female,2,own,NA,NA,2394,36,radio/TV,good 72 | 70,30,female,2,own,little,NA,8133,36,car,good 73 | 71,46,male,1,rent,NA,NA,730,7,radio/TV,good 74 | 72,51,male,3,free,little,little,1164,8,vacation/others,good 75 | 73,41,female,1,own,little,moderate,5954,42,business,good 76 | 74,40,male,3,own,NA,little,1977,36,education,bad 77 | 75,66,male,3,free,little,little,1526,12,car,good 78 | 76,34,male,2,own,little,little,3965,42,radio/TV,bad 79 | 77,51,male,2,own,little,moderate,4771,11,radio/TV,good 80 | 78,39,male,1,own,NA,NA,9436,54,car,good 81 | 79,22,male,2,own,little,moderate,3832,30,furniture/equipment,good 82 | 80,44,female,2,own,NA,NA,5943,24,radio/TV,bad 83 | 81,47,male,2,own,quite rich,NA,1213,15,radio/TV,good 84 | 82,24,female,1,rent,moderate,NA,1568,18,business,good 85 | 83,58,female,1,own,little,little,1755,24,vacation/others,good 86 | 84,52,male,1,own,little,little,2315,10,radio/TV,good 87 | 85,29,female,3,own,little,NA,1412,12,business,good 88 | 86,27,female,2,own,little,moderate,1295,18,furniture/equipment,good 89 | 87,47,male,2,free,moderate,moderate,12612,36,education,bad 90 | 88,30,male,3,own,moderate,little,2249,18,car,good 91 | 89,28,male,2,own,little,little,1108,12,repairs,bad 92 | 90,56,male,2,own,little,NA,618,12,radio/TV,good 93 | 91,54,male,2,own,little,little,1409,12,car,good 94 | 92,33,female,1,own,NA,NA,797,12,radio/TV,bad 95 | 93,20,male,2,rent,NA,rich,3617,24,furniture/equipment,good 96 | 94,54,male,2,own,rich,moderate,1318,12,car,good 97 | 95,58,male,2,rent,little,moderate,15945,54,business,bad 98 | 96,61,female,2,own,NA,NA,2012,12,education,good 99 | 97,34,male,2,own,moderate,moderate,2622,18,business,good 100 | 98,36,male,2,own,little,moderate,2337,36,radio/TV,good 101 | 99,36,male,3,rent,NA,moderate,7057,20,car,good 102 | 100,41,male,1,rent,moderate,NA,1469,24,car,good 103 | 101,24,male,2,rent,little,moderate,2323,36,radio/TV,good 104 | 102,24,female,2,own,little,NA,932,6,radio/TV,good 105 | 103,35,male,2,rent,little,moderate,1919,9,furniture/equipment,good 106 | 104,26,male,2,rent,NA,NA,2445,12,car,good 107 | 105,39,male,3,own,little,moderate,11938,24,vacation/others,bad 108 | 106,39,male,3,own,little,NA,6458,18,car,bad 109 | 107,32,male,2,own,little,moderate,6078,12,car,good 110 | 108,30,female,2,own,NA,little,7721,24,furniture/equipment,good 111 | 109,35,male,2,own,quite rich,moderate,1410,14,business,good 112 | 110,31,male,2,own,moderate,moderate,1449,6,business,good 113 | 111,23,female,2,rent,little,rich,392,15,education,good 114 | 112,28,male,1,rent,little,moderate,6260,18,car,good 115 | 113,25,female,2,own,little,NA,7855,36,car,bad 116 | 114,35,male,2,own,quite rich,little,1680,12,radio/TV,good 117 | 115,47,male,2,own,NA,NA,3578,48,radio/TV,good 118 | 116,30,female,3,own,NA,little,7174,42,radio/TV,bad 119 | 117,27,female,2,rent,NA,little,2132,10,furniture/equipment,good 120 | 118,23,female,2,own,quite rich,little,4281,33,furniture/equipment,bad 121 | 119,36,male,3,own,quite rich,moderate,2366,12,car,good 122 | 120,25,female,2,own,little,little,1835,21,radio/TV,bad 123 | 121,41,female,3,rent,little,NA,3868,24,car,good 124 | 122,24,male,1,rent,little,NA,1768,12,furniture/equipment,good 125 | 123,63,male,2,free,little,rich,781,10,car,good 126 | 124,27,female,2,rent,NA,moderate,1924,18,furniture/equipment,bad 127 | 125,30,male,2,own,little,little,2121,12,car,good 128 | 126,40,male,1,own,little,little,701,12,radio/TV,good 129 | 127,30,male,2,own,little,moderate,639,12,repairs,bad 130 | 128,34,male,3,own,little,moderate,1860,12,car,good 131 | 129,29,female,2,own,little,little,3499,12,car,bad 132 | 130,24,female,2,own,NA,moderate,8487,48,car,good 133 | 131,29,male,2,own,little,little,6887,36,education,bad 134 | 132,27,male,1,own,little,NA,2708,15,furniture/equipment,good 135 | 133,47,male,2,free,little,NA,1984,18,furniture/equipment,good 136 | 134,21,female,2,own,moderate,NA,10144,60,radio/TV,good 137 | 135,38,female,2,own,NA,NA,1240,12,radio/TV,good 138 | 136,27,male,2,own,rich,NA,8613,27,car,good 139 | 137,66,male,1,own,quite rich,moderate,766,12,radio/TV,bad 140 | 138,35,male,2,own,NA,moderate,2728,15,radio/TV,good 141 | 139,44,female,1,rent,little,rich,1881,12,radio/TV,good 142 | 140,27,male,0,own,rich,rich,709,6,car,good 143 | 141,30,female,3,own,little,moderate,4795,36,radio/TV,good 144 | 142,27,male,3,own,little,little,3416,27,radio/TV,good 145 | 143,22,male,2,own,little,little,2462,18,furniture/equipment,bad 146 | 144,23,female,2,own,little,NA,2288,21,furniture/equipment,good 147 | 145,30,male,2,own,moderate,moderate,3566,48,business,good 148 | 146,39,female,2,own,little,little,860,6,car,good 149 | 147,51,female,2,own,moderate,NA,682,12,car,good 150 | 148,28,male,2,own,little,little,5371,36,furniture/equipment,good 151 | 149,46,male,2,own,rich,NA,1582,18,radio/TV,good 152 | 150,42,male,2,free,moderate,NA,1346,6,radio/TV,good 153 | 151,38,male,2,own,little,NA,1924,10,radio/TV,good 154 | 152,24,male,2,own,little,rich,5848,36,radio/TV,good 155 | 153,29,female,2,rent,rich,moderate,7758,24,car,good 156 | 154,36,male,3,rent,moderate,moderate,6967,24,business,good 157 | 155,20,female,2,rent,little,little,1282,12,furniture/equipment,bad 158 | 156,48,male,2,own,moderate,little,1288,9,repairs,good 159 | 157,45,male,1,own,little,little,339,12,education,good 160 | 158,38,male,2,own,moderate,moderate,3512,24,car,good 161 | 159,34,male,1,own,NA,NA,1898,6,radio/TV,good 162 | 160,36,male,2,own,moderate,NA,2872,24,radio/TV,good 163 | 161,30,female,2,own,little,NA,1055,18,car,good 164 | 162,36,male,2,own,quite rich,NA,1262,15,domestic appliances,good 165 | 163,70,male,3,free,little,moderate,7308,10,car,good 166 | 164,36,male,2,own,quite rich,NA,909,36,car,good 167 | 165,32,male,2,own,quite rich,NA,2978,6,furniture/equipment,good 168 | 166,33,female,2,own,little,little,1131,18,furniture/equipment,bad 169 | 167,20,female,2,own,rich,moderate,1577,11,furniture/equipment,good 170 | 168,25,female,2,rent,little,NA,3972,24,furniture/equipment,good 171 | 169,31,male,2,own,little,moderate,1935,24,business,bad 172 | 170,33,male,2,rent,little,little,950,15,car,bad 173 | 171,26,female,2,own,little,NA,763,12,furniture/equipment,good 174 | 172,34,female,3,own,little,moderate,2064,24,furniture/equipment,bad 175 | 173,33,male,2,own,little,moderate,1414,8,radio/TV,good 176 | 174,26,male,2,own,little,little,3414,21,education,bad 177 | 175,53,female,3,own,NA,NA,7485,30,car,bad 178 | 176,42,male,2,own,little,little,2577,12,furniture/equipment,good 179 | 177,52,male,2,own,quite rich,little,338,6,radio/TV,good 180 | 178,31,male,3,rent,little,NA,1963,12,radio/TV,good 181 | 179,65,male,2,own,little,little,571,21,car,good 182 | 180,28,male,2,own,little,NA,9572,36,business,bad 183 | 181,30,male,3,own,little,moderate,4455,36,business,bad 184 | 182,40,male,1,own,NA,little,1647,21,car,bad 185 | 183,50,male,2,own,rich,NA,3777,24,furniture/equipment,good 186 | 184,36,male,2,own,little,moderate,884,18,car,bad 187 | 185,31,male,2,own,little,NA,1360,15,radio/TV,good 188 | 186,74,female,3,free,little,moderate,5129,9,car,bad 189 | 187,68,male,0,free,little,moderate,1175,16,car,good 190 | 188,20,male,2,own,moderate,little,674,12,radio/TV,bad 191 | 189,33,female,2,own,little,moderate,3244,18,furniture/equipment,good 192 | 190,54,male,3,own,rich,NA,4591,24,business,bad 193 | 191,34,male,1,free,moderate,moderate,3844,48,business,bad 194 | 192,36,male,2,own,little,moderate,3915,27,business,bad 195 | 193,29,male,2,rent,little,NA,2108,6,radio/TV,good 196 | 194,21,male,2,rent,moderate,moderate,3031,45,radio/TV,bad 197 | 195,34,female,3,own,little,moderate,1501,9,education,bad 198 | 196,28,female,2,own,little,NA,1382,6,radio/TV,good 199 | 197,27,female,2,rent,moderate,moderate,951,12,furniture/equipment,bad 200 | 198,36,male,2,free,NA,moderate,2760,24,car,good 201 | 199,40,male,3,own,little,moderate,4297,18,furniture/equipment,bad 202 | 200,52,male,2,own,quite rich,NA,936,9,education,good 203 | 201,27,male,1,own,little,little,1168,12,car,good 204 | 202,26,male,2,own,little,NA,5117,27,business,good 205 | 203,21,male,2,rent,little,little,902,12,education,bad 206 | 204,38,male,1,own,little,NA,1495,12,car,good 207 | 205,38,male,3,free,little,little,10623,30,car,good 208 | 206,43,male,2,own,little,NA,1935,12,furniture/equipment,good 209 | 207,26,male,2,own,little,moderate,1424,12,domestic appliances,good 210 | 208,21,male,1,own,little,little,6568,24,business,good 211 | 209,55,male,2,own,rich,NA,1413,12,car,good 212 | 210,33,male,2,own,NA,NA,3074,9,radio/TV,good 213 | 211,45,female,1,own,NA,NA,3835,36,radio/TV,good 214 | 212,50,male,2,own,little,little,5293,27,business,bad 215 | 213,66,male,3,own,little,rich,1908,30,business,bad 216 | 214,51,male,2,own,NA,NA,3342,36,radio/TV,good 217 | 215,39,female,1,own,NA,moderate,932,6,education,good 218 | 216,31,male,2,own,little,little,3104,18,business,good 219 | 217,23,male,2,own,little,rich,3913,36,radio/TV,good 220 | 218,24,male,1,rent,little,little,3021,24,furniture/equipment,good 221 | 219,64,female,2,own,little,NA,1364,10,car,good 222 | 220,26,male,1,own,little,moderate,625,12,radio/TV,good 223 | 221,23,female,2,rent,NA,little,1200,12,education,good 224 | 222,30,male,2,own,little,NA,707,12,radio/TV,good 225 | 223,32,male,2,own,NA,NA,2978,24,business,good 226 | 224,30,male,2,own,little,NA,4657,15,car,good 227 | 225,27,male,2,own,little,NA,2613,36,repairs,good 228 | 226,27,male,2,own,rich,moderate,10961,48,radio/TV,bad 229 | 227,53,male,3,free,little,little,7865,12,furniture/equipment,bad 230 | 228,22,male,2,own,little,NA,1478,9,radio/TV,bad 231 | 229,22,male,2,free,little,little,3149,24,furniture/equipment,good 232 | 230,26,male,2,own,little,rich,4210,36,radio/TV,bad 233 | 231,51,male,1,free,quite rich,NA,2507,9,car,good 234 | 232,35,male,2,own,moderate,NA,2141,12,radio/TV,good 235 | 233,25,male,1,own,little,moderate,866,18,radio/TV,good 236 | 234,42,male,1,own,little,NA,1544,4,radio/TV,good 237 | 235,30,male,3,own,little,little,1823,24,radio/TV,bad 238 | 236,23,male,0,own,NA,moderate,14555,6,car,bad 239 | 237,61,male,1,rent,moderate,moderate,2767,21,business,bad 240 | 238,35,female,2,own,little,NA,1291,12,radio/TV,good 241 | 239,39,male,2,own,little,little,2522,30,radio/TV,good 242 | 240,29,female,2,own,NA,little,915,24,car,bad 243 | 241,51,male,2,own,little,NA,1595,6,radio/TV,good 244 | 242,24,male,2,free,little,little,4605,48,car,bad 245 | 243,27,female,2,own,little,NA,1185,12,business,good 246 | 244,35,female,1,own,quite rich,NA,3447,12,education,good 247 | 245,25,male,2,own,little,NA,1258,24,business,good 248 | 246,52,male,2,own,little,NA,717,12,radio/TV,good 249 | 247,35,male,2,rent,moderate,NA,1204,6,car,good 250 | 248,26,male,2,own,little,rich,1925,24,furniture/equipment,good 251 | 249,22,female,2,rent,little,NA,433,18,radio/TV,bad 252 | 250,39,female,1,own,rich,little,666,6,car,good 253 | 251,46,female,1,own,little,rich,2251,12,furniture/equipment,good 254 | 252,24,female,2,own,little,moderate,2150,30,car,bad 255 | 253,35,male,2,own,moderate,NA,4151,24,furniture/equipment,good 256 | 254,24,male,2,own,NA,moderate,2030,9,furniture/equipment,good 257 | 255,27,male,1,own,NA,moderate,7418,60,radio/TV,good 258 | 256,35,male,1,own,little,NA,2684,24,radio/TV,good 259 | 257,29,male,2,free,little,little,2149,12,radio/TV,bad 260 | 258,23,female,2,own,moderate,NA,3812,15,car,good 261 | 259,57,female,1,own,moderate,NA,1154,11,radio/TV,good 262 | 260,27,male,2,own,little,little,1657,12,furniture/equipment,good 263 | 261,55,female,2,own,little,little,1603,24,radio/TV,good 264 | 262,36,male,3,free,little,little,5302,18,car,good 265 | 263,57,female,1,free,little,NA,2748,12,education,good 266 | 264,32,male,1,own,little,NA,1231,10,car,good 267 | 265,37,male,2,own,little,moderate,802,15,radio/TV,bad 268 | 266,36,male,2,own,NA,NA,6304,36,business,good 269 | 267,38,female,2,own,little,NA,1533,24,radio/TV,good 270 | 268,45,male,3,own,little,little,8978,14,car,bad 271 | 269,25,male,2,own,NA,NA,999,24,radio/TV,good 272 | 270,32,male,2,own,NA,NA,2662,18,car,good 273 | 271,37,female,2,rent,quite rich,NA,1402,12,furniture/equipment,good 274 | 272,36,male,3,free,NA,moderate,12169,48,car,good 275 | 273,28,male,2,own,little,moderate,3060,48,radio/TV,bad 276 | 274,34,male,1,own,little,little,11998,30,repairs,bad 277 | 275,32,male,2,own,little,NA,2697,9,radio/TV,good 278 | 276,26,female,2,own,little,NA,2404,18,radio/TV,good 279 | 277,49,male,1,own,NA,little,1262,12,furniture/equipment,good 280 | 278,32,female,2,own,little,NA,4611,6,furniture/equipment,bad 281 | 279,29,male,3,rent,moderate,NA,1901,24,radio/TV,good 282 | 280,23,male,2,rent,rich,NA,3368,15,car,good 283 | 281,50,male,2,own,little,NA,1574,12,furniture/equipment,good 284 | 282,49,male,1,own,NA,rich,1445,18,radio/TV,good 285 | 283,63,male,2,own,NA,NA,1520,15,furniture/equipment,good 286 | 284,37,male,2,own,moderate,moderate,3878,24,car,good 287 | 285,35,female,1,own,little,little,10722,47,car,good 288 | 286,26,male,2,own,little,little,4788,48,car,good 289 | 287,31,male,3,free,moderate,moderate,7582,48,vacation/others,good 290 | 288,49,female,2,own,little,moderate,1092,12,radio/TV,good 291 | 289,48,male,2,own,little,little,1024,24,radio/TV,bad 292 | 290,26,male,2,own,little,NA,1076,12,business,good 293 | 291,28,male,3,rent,little,moderate,9398,36,car,bad 294 | 292,44,female,3,free,little,little,6419,24,car,good 295 | 293,56,male,2,free,little,rich,4796,42,car,good 296 | 294,46,male,3,own,NA,NA,7629,48,business,good 297 | 295,26,female,2,own,little,moderate,9960,48,furniture/equipment,bad 298 | 296,20,female,2,rent,NA,NA,4675,12,car,good 299 | 297,45,male,1,own,NA,NA,1287,10,car,good 300 | 298,43,male,2,own,little,NA,2515,18,furniture/equipment,good 301 | 299,32,male,2,own,rich,moderate,2745,21,furniture/equipment,good 302 | 300,54,female,0,own,little,NA,672,6,car,good 303 | 301,42,female,2,own,little,moderate,3804,36,radio/TV,bad 304 | 302,37,male,1,own,NA,rich,1344,24,car,bad 305 | 303,49,male,2,own,little,little,1038,10,car,good 306 | 304,44,male,2,free,quite rich,NA,10127,48,car,bad 307 | 305,33,male,2,own,rich,NA,1543,6,furniture/equipment,good 308 | 306,24,female,1,rent,NA,NA,4811,30,car,good 309 | 307,33,male,1,own,moderate,little,727,12,radio/TV,bad 310 | 308,24,female,2,own,little,moderate,1237,8,furniture/equipment,bad 311 | 309,22,male,1,rent,little,moderate,276,9,car,good 312 | 310,40,male,0,free,NA,moderate,5381,48,vacation/others,good 313 | 311,25,male,2,own,moderate,NA,5511,24,furniture/equipment,good 314 | 312,26,female,2,own,little,rich,3749,24,furniture/equipment,good 315 | 313,25,male,1,own,little,moderate,685,12,car,bad 316 | 314,29,male,1,own,NA,rich,1494,4,car,good 317 | 315,31,male,2,own,little,little,2746,36,furniture/equipment,bad 318 | 316,38,male,1,own,little,little,708,12,furniture/equipment,good 319 | 317,48,female,1,own,NA,moderate,4351,24,furniture/equipment,good 320 | 318,32,male,2,own,little,NA,701,12,education,good 321 | 319,27,female,1,own,little,little,3643,15,furniture/equipment,good 322 | 320,28,male,3,own,little,moderate,4249,30,car,bad 323 | 321,32,male,2,own,little,little,1938,24,radio/TV,bad 324 | 322,34,male,3,free,little,little,2910,24,car,good 325 | 323,28,male,2,own,rich,little,2659,18,furniture/equipment,good 326 | 324,36,female,2,own,little,NA,1028,18,car,good 327 | 325,39,male,1,own,little,little,3398,8,car,good 328 | 326,49,male,2,rent,NA,NA,5801,12,furniture/equipment,good 329 | 327,34,female,2,own,rich,NA,1525,24,car,good 330 | 328,31,male,2,own,little,rich,4473,36,radio/TV,good 331 | 329,28,male,2,own,little,moderate,1068,6,radio/TV,good 332 | 330,75,male,3,free,little,little,6615,24,car,good 333 | 331,30,female,2,own,moderate,NA,1864,18,education,bad 334 | 332,24,female,3,own,moderate,moderate,7408,60,car,bad 335 | 333,24,female,1,rent,moderate,NA,11590,48,car,bad 336 | 334,23,male,2,rent,little,little,4110,24,furniture/equipment,bad 337 | 335,44,male,3,rent,little,little,3384,6,furniture/equipment,bad 338 | 336,23,female,1,own,little,moderate,2101,13,radio/TV,good 339 | 337,24,female,2,rent,NA,little,1275,15,domestic appliances,bad 340 | 338,28,male,2,own,little,little,4169,24,furniture/equipment,good 341 | 339,31,male,1,own,little,moderate,1521,10,furniture/equipment,good 342 | 340,24,female,2,free,little,moderate,5743,24,education,good 343 | 341,26,female,1,rent,little,little,3599,21,furniture/equipment,good 344 | 342,25,male,2,rent,quite rich,moderate,3213,18,radio/TV,good 345 | 343,33,male,3,own,little,moderate,4439,18,business,good 346 | 344,37,male,1,own,little,rich,3949,10,car,good 347 | 345,43,female,1,own,little,NA,1459,15,radio/TV,good 348 | 346,23,male,2,own,little,moderate,882,13,radio/TV,good 349 | 347,23,female,0,rent,quite rich,moderate,3758,24,radio/TV,good 350 | 348,34,male,1,own,moderate,NA,1743,6,business,good 351 | 349,32,male,2,free,rich,moderate,1136,9,education,bad 352 | 350,23,female,2,rent,little,NA,1236,9,domestic appliances,good 353 | 351,29,female,2,own,little,moderate,959,9,furniture/equipment,bad 354 | 352,38,male,3,own,NA,NA,3229,18,car,good 355 | 353,28,male,2,rent,little,little,6199,12,radio/TV,bad 356 | 354,46,male,2,free,quite rich,NA,727,10,education,good 357 | 355,23,male,1,own,little,moderate,1246,24,car,bad 358 | 356,49,male,2,own,NA,NA,2331,12,radio/TV,good 359 | 357,26,male,3,own,little,NA,4463,36,radio/TV,bad 360 | 358,28,male,2,own,little,NA,776,12,radio/TV,good 361 | 359,23,female,2,rent,little,little,2406,30,furniture/equipment,bad 362 | 360,61,male,2,free,NA,moderate,1239,18,education,good 363 | 361,37,male,3,own,NA,rich,3399,12,radio/TV,good 364 | 362,36,female,2,own,little,rich,2247,12,car,good 365 | 363,21,male,2,rent,little,NA,1766,6,furniture/equipment,good 366 | 364,25,male,0,own,little,little,2473,18,furniture/equipment,bad 367 | 365,36,male,2,own,little,NA,1542,12,business,good 368 | 366,27,male,2,own,little,NA,3850,18,car,good 369 | 367,22,female,2,rent,little,little,3650,18,furniture/equipment,good 370 | 368,42,male,2,own,little,little,3446,36,furniture/equipment,bad 371 | 369,40,female,2,rent,little,moderate,3001,18,furniture/equipment,good 372 | 370,36,male,2,own,NA,NA,3079,36,car,good 373 | 371,33,male,2,own,little,NA,6070,18,radio/TV,good 374 | 372,23,female,2,rent,little,NA,2146,10,furniture/equipment,good 375 | 373,63,male,3,free,NA,NA,13756,60,car,good 376 | 374,60,female,3,free,moderate,moderate,14782,60,vacation/others,bad 377 | 375,37,female,2,rent,little,little,7685,48,business,bad 378 | 376,34,male,2,own,little,NA,2320,18,radio/TV,good 379 | 377,36,male,2,free,NA,NA,846,7,radio/TV,good 380 | 378,57,male,3,free,little,moderate,14318,36,car,bad 381 | 379,52,female,1,own,moderate,NA,362,6,car,good 382 | 380,39,male,2,own,NA,little,2212,20,furniture/equipment,good 383 | 381,38,female,3,free,little,moderate,12976,18,car,bad 384 | 382,25,female,2,rent,NA,NA,1283,22,car,good 385 | 383,26,male,2,own,little,rich,1330,12,car,good 386 | 384,26,male,1,own,moderate,NA,4272,30,business,good 387 | 385,25,female,2,own,little,NA,2238,18,radio/TV,good 388 | 386,21,female,2,rent,NA,NA,1126,18,radio/TV,good 389 | 387,40,male,3,own,little,moderate,7374,18,furniture/equipment,good 390 | 388,27,male,2,own,quite rich,moderate,2326,15,business,good 391 | 389,27,female,2,own,little,NA,1449,9,business,good 392 | 390,30,male,3,own,little,NA,1820,18,car,good 393 | 391,19,female,1,rent,rich,moderate,983,12,furniture/equipment,good 394 | 392,39,male,3,free,little,little,3249,36,car,good 395 | 393,31,female,2,own,little,little,1957,6,radio/TV,good 396 | 394,31,male,3,own,little,NA,2406,9,furniture/equipment,good 397 | 395,32,male,2,rent,moderate,moderate,11760,39,education,good 398 | 396,55,female,3,free,little,little,2578,12,furniture/equipment,good 399 | 397,46,male,2,own,little,little,2348,36,furniture/equipment,good 400 | 398,46,male,2,rent,little,moderate,1223,12,car,bad 401 | 399,43,female,1,own,rich,NA,1516,24,radio/TV,good 402 | 400,39,male,2,own,little,NA,1473,18,radio/TV,good 403 | 401,28,male,2,own,NA,moderate,1887,18,business,good 404 | 402,27,male,2,own,little,NA,8648,24,business,bad 405 | 403,27,male,1,own,little,NA,802,14,car,good 406 | 404,43,male,2,own,NA,moderate,2899,18,car,good 407 | 405,22,male,2,own,little,moderate,2039,24,radio/TV,bad 408 | 406,43,male,2,own,NA,NA,2197,24,car,good 409 | 407,27,male,2,own,little,little,1053,15,radio/TV,good 410 | 408,26,male,3,own,quite rich,NA,3235,24,radio/TV,good 411 | 409,28,male,2,own,quite rich,rich,939,12,car,bad 412 | 410,20,female,2,own,little,moderate,1967,24,radio/TV,good 413 | 411,35,male,3,own,little,NA,7253,33,car,good 414 | 412,42,male,3,own,little,NA,2292,12,business,bad 415 | 413,40,male,1,rent,quite rich,NA,1597,10,car,good 416 | 414,35,female,2,own,NA,little,1381,24,car,bad 417 | 415,35,male,2,own,little,NA,5842,36,car,good 418 | 416,33,male,1,own,little,little,2579,12,car,bad 419 | 417,23,female,2,rent,NA,little,8471,18,education,good 420 | 418,31,female,3,own,quite rich,NA,2782,21,car,good 421 | 419,33,female,2,own,NA,moderate,1042,18,car,bad 422 | 420,20,female,2,rent,rich,NA,3186,15,car,good 423 | 421,30,male,2,own,NA,moderate,2028,12,car,good 424 | 422,47,male,1,own,little,moderate,958,12,car,good 425 | 423,34,male,3,own,moderate,NA,1591,21,furniture/equipment,good 426 | 424,25,female,2,own,NA,moderate,2762,12,furniture/equipment,bad 427 | 425,21,male,2,rent,little,moderate,2779,18,car,good 428 | 426,29,male,2,own,little,NA,2743,28,radio/TV,good 429 | 427,46,male,2,own,rich,NA,1149,18,radio/TV,good 430 | 428,20,male,2,own,little,NA,1313,9,furniture/equipment,good 431 | 429,55,female,0,free,little,little,1190,18,repairs,bad 432 | 430,74,male,1,own,little,NA,3448,5,business,good 433 | 431,29,male,3,own,little,moderate,11328,24,vacation/others,bad 434 | 432,36,male,3,free,little,little,1872,6,furniture/equipment,good 435 | 433,33,male,2,own,little,NA,2058,24,repairs,good 436 | 434,25,male,2,own,little,little,2136,9,furniture/equipment,good 437 | 435,25,male,2,own,NA,moderate,1484,12,radio/TV,bad 438 | 436,23,male,1,rent,quite rich,NA,660,6,repairs,good 439 | 437,37,female,2,own,rich,NA,1287,24,car,good 440 | 438,65,male,0,own,little,little,3394,42,repairs,good 441 | 439,26,female,0,own,little,rich,609,12,business,bad 442 | 440,39,male,3,own,little,NA,1884,12,car,good 443 | 441,30,female,2,own,little,little,1620,12,furniture/equipment,good 444 | 442,29,male,2,own,little,moderate,2629,20,vacation/others,good 445 | 443,41,male,1,own,little,NA,719,12,education,bad 446 | 444,30,female,3,own,little,moderate,5096,48,furniture/equipment,bad 447 | 445,41,female,1,rent,NA,NA,1244,9,education,good 448 | 446,34,female,2,own,little,little,1842,36,car,bad 449 | 447,35,male,2,own,little,moderate,2576,7,radio/TV,good 450 | 448,55,female,3,own,NA,rich,1424,12,furniture/equipment,good 451 | 449,61,male,2,own,rich,moderate,1512,15,repairs,bad 452 | 450,30,male,3,own,NA,NA,11054,36,car,good 453 | 451,29,female,2,own,little,NA,518,6,radio/TV,good 454 | 452,34,male,2,own,little,NA,2759,12,furniture/equipment,good 455 | 453,35,male,3,own,little,NA,2670,24,car,good 456 | 454,31,male,2,own,little,little,4817,24,car,bad 457 | 455,29,female,3,own,little,NA,2679,24,car,good 458 | 456,36,male,2,rent,little,little,3905,11,car,good 459 | 457,35,male,2,free,little,little,3386,12,car,bad 460 | 458,27,female,2,own,little,little,343,6,domestic appliances,good 461 | 459,32,male,2,own,little,NA,4594,18,radio/TV,good 462 | 460,37,male,2,own,little,little,3620,36,furniture/equipment,good 463 | 461,36,male,2,own,little,little,1721,15,car,good 464 | 462,34,female,3,rent,little,moderate,3017,12,furniture/equipment,good 465 | 463,38,male,2,own,NA,moderate,754,12,education,good 466 | 464,34,male,2,own,little,NA,1950,18,business,good 467 | 465,63,male,2,own,little,little,2924,24,car,good 468 | 466,29,female,1,rent,little,little,1659,24,radio/TV,bad 469 | 467,32,male,2,own,NA,NA,7238,48,radio/TV,good 470 | 468,26,female,2,own,little,NA,2764,33,business,good 471 | 469,35,male,1,own,little,NA,4679,24,car,good 472 | 470,22,male,2,rent,moderate,moderate,3092,24,radio/TV,bad 473 | 471,23,female,2,own,little,little,448,6,education,bad 474 | 472,28,male,1,own,little,little,654,9,car,bad 475 | 473,36,male,3,own,NA,NA,1238,6,education,good 476 | 474,33,male,2,own,little,moderate,1245,18,radio/TV,bad 477 | 475,26,female,2,rent,little,little,3114,18,furniture/equipment,bad 478 | 476,24,male,2,own,quite rich,NA,2569,39,car,good 479 | 477,25,male,2,own,little,rich,5152,24,radio/TV,good 480 | 478,39,male,1,own,moderate,moderate,1037,12,business,good 481 | 479,44,male,2,own,little,little,1478,15,furniture/equipment,good 482 | 480,23,female,1,own,little,moderate,3573,12,radio/TV,good 483 | 481,26,male,2,own,little,moderate,1201,24,car,good 484 | 482,57,female,2,rent,rich,little,3622,30,furniture/equipment,good 485 | 483,30,female,2,own,rich,NA,960,15,furniture/equipment,good 486 | 484,44,male,2,own,quite rich,NA,1163,12,car,good 487 | 485,47,male,3,own,little,moderate,1209,6,car,bad 488 | 486,52,male,2,own,little,NA,3077,12,radio/TV,good 489 | 487,62,female,2,free,little,NA,3757,24,car,good 490 | 488,35,male,1,rent,moderate,NA,1418,10,car,good 491 | 489,26,male,2,rent,little,NA,3518,6,car,good 492 | 490,26,male,2,own,little,NA,1934,12,radio/TV,good 493 | 491,42,female,3,free,little,moderate,8318,27,business,bad 494 | 492,27,female,2,own,moderate,NA,1237,6,radio/TV,good 495 | 493,38,male,2,own,NA,moderate,368,6,radio/TV,good 496 | 494,39,male,1,rent,little,little,2122,12,car,good 497 | 495,20,male,2,own,NA,little,2996,24,furniture/equipment,bad 498 | 496,29,male,3,rent,moderate,moderate,9034,36,furniture/equipment,bad 499 | 497,40,male,2,own,little,NA,1585,24,furniture/equipment,good 500 | 498,32,male,1,own,little,moderate,1301,18,radio/TV,good 501 | 499,28,male,2,own,moderate,rich,1323,6,car,good 502 | 500,27,female,2,own,little,little,3123,24,car,bad 503 | 501,42,male,2,free,little,little,5493,36,car,good 504 | 502,49,male,2,own,moderate,rich,1126,9,radio/TV,good 505 | 503,38,male,2,own,moderate,moderate,1216,24,radio/TV,bad 506 | 504,24,female,2,rent,little,little,1207,24,car,bad 507 | 505,27,male,1,own,NA,NA,1309,10,car,bad 508 | 506,36,male,2,own,quite rich,rich,2360,15,car,good 509 | 507,34,male,3,own,moderate,moderate,6850,15,car,bad 510 | 508,28,male,2,own,little,NA,1413,24,radio/TV,good 511 | 509,45,male,3,own,moderate,NA,8588,39,car,good 512 | 510,26,male,2,own,little,little,759,12,car,bad 513 | 511,32,male,3,free,little,NA,4686,36,car,good 514 | 512,26,male,2,rent,little,rich,2687,15,business,good 515 | 513,20,male,2,rent,little,moderate,585,12,radio/TV,good 516 | 514,54,male,2,own,NA,NA,2255,24,car,good 517 | 515,37,female,2,own,little,little,609,6,car,good 518 | 516,40,male,1,own,little,little,1361,6,car,good 519 | 517,23,female,2,rent,little,NA,7127,36,furniture/equipment,bad 520 | 518,43,male,2,own,moderate,little,1203,6,car,good 521 | 519,36,male,2,free,NA,NA,700,6,radio/TV,good 522 | 520,44,male,2,free,little,NA,5507,24,repairs,good 523 | 521,24,female,2,own,little,little,3190,18,radio/TV,bad 524 | 522,53,male,2,free,little,little,7119,48,furniture/equipment,bad 525 | 523,23,female,2,own,moderate,NA,3488,24,car,good 526 | 524,26,female,1,own,little,moderate,1113,18,radio/TV,good 527 | 525,30,male,2,own,little,moderate,7966,26,car,good 528 | 526,31,female,2,own,moderate,NA,1532,15,education,good 529 | 527,42,male,1,own,little,NA,1503,4,radio/TV,good 530 | 528,31,male,2,rent,little,little,2302,36,radio/TV,bad 531 | 529,41,male,1,own,little,little,662,6,car,good 532 | 530,32,male,2,own,little,moderate,2273,36,education,good 533 | 531,28,female,2,rent,moderate,moderate,2631,15,car,bad 534 | 532,41,male,2,rent,little,NA,1503,12,car,good 535 | 533,26,male,2,own,moderate,NA,1311,24,radio/TV,good 536 | 534,25,male,2,own,NA,NA,3105,24,radio/TV,good 537 | 535,33,male,2,rent,little,rich,2319,21,education,bad 538 | 536,75,female,3,own,NA,little,1374,6,car,good 539 | 537,37,female,2,own,little,moderate,3612,18,furniture/equipment,good 540 | 538,42,male,3,free,little,little,7763,48,car,bad 541 | 539,45,female,1,own,little,rich,3049,18,furniture/equipment,good 542 | 540,23,male,2,rent,little,moderate,1534,12,radio/TV,bad 543 | 541,60,male,2,free,little,NA,2032,24,car,good 544 | 542,31,male,2,own,NA,little,6350,30,furniture/equipment,bad 545 | 543,34,male,1,own,little,rich,2864,18,furniture/equipment,bad 546 | 544,61,male,1,own,little,NA,1255,12,car,good 547 | 545,43,male,2,free,little,little,1333,24,car,bad 548 | 546,37,female,2,own,little,NA,2022,24,car,good 549 | 547,32,male,2,own,little,NA,1552,24,radio/TV,good 550 | 548,24,female,1,own,little,little,626,12,radio/TV,bad 551 | 549,35,male,2,free,NA,NA,8858,48,car,good 552 | 550,23,female,2,own,NA,NA,996,12,repairs,good 553 | 551,45,male,1,own,quite rich,NA,1750,6,radio/TV,good 554 | 552,34,male,2,own,little,little,6999,48,radio/TV,bad 555 | 553,27,male,2,own,moderate,moderate,1995,12,car,good 556 | 554,67,female,3,own,little,moderate,1199,9,education,good 557 | 555,22,male,2,own,little,moderate,1331,12,radio/TV,bad 558 | 556,28,female,2,own,moderate,moderate,2278,18,car,bad 559 | 557,29,female,2,own,NA,NA,5003,21,car,bad 560 | 558,27,male,2,own,little,little,3552,24,furniture/equipment,bad 561 | 559,31,male,1,own,little,moderate,1928,18,furniture/equipment,bad 562 | 560,49,male,2,free,NA,little,2964,24,car,good 563 | 561,24,male,1,rent,little,little,1546,24,radio/TV,bad 564 | 562,29,female,2,own,little,rich,683,6,radio/TV,good 565 | 563,37,male,2,free,NA,moderate,12389,36,car,bad 566 | 564,37,male,3,own,NA,moderate,4712,24,business,good 567 | 565,23,female,2,rent,moderate,moderate,1553,24,radio/TV,good 568 | 566,36,male,2,own,little,little,1372,12,car,bad 569 | 567,34,male,2,own,rich,NA,2578,24,radio/TV,good 570 | 568,41,male,2,own,NA,moderate,3979,48,radio/TV,good 571 | 569,31,female,2,own,little,little,6758,48,radio/TV,bad 572 | 570,23,female,1,rent,little,little,3234,24,furniture/equipment,bad 573 | 571,38,male,2,own,little,NA,5954,30,radio/TV,good 574 | 572,26,female,3,rent,NA,NA,5433,24,car,good 575 | 573,22,female,1,own,little,little,806,15,business,good 576 | 574,27,male,1,own,little,moderate,1082,9,radio/TV,good 577 | 575,24,female,2,own,little,NA,2788,15,furniture/equipment,good 578 | 576,27,female,2,own,little,moderate,2930,12,radio/TV,good 579 | 577,33,female,2,own,NA,NA,1927,24,education,good 580 | 578,27,male,2,own,little,moderate,2820,36,car,bad 581 | 579,27,male,1,own,little,NA,937,24,education,good 582 | 580,30,male,2,own,little,moderate,1056,18,car,bad 583 | 581,49,male,1,own,little,moderate,3124,12,car,good 584 | 582,26,female,2,rent,little,NA,1388,9,furniture/equipment,good 585 | 583,33,male,1,rent,little,moderate,2384,36,repairs,bad 586 | 584,52,female,3,free,NA,NA,2133,12,car,good 587 | 585,20,female,2,rent,little,little,2039,18,furniture/equipment,bad 588 | 586,36,male,2,rent,little,little,2799,9,car,good 589 | 587,21,male,1,own,little,little,1289,12,furniture/equipment,good 590 | 588,47,male,1,own,little,little,1217,18,domestic appliances,bad 591 | 589,60,male,2,own,little,little,2246,12,furniture/equipment,bad 592 | 590,58,female,1,own,little,little,385,12,radio/TV,good 593 | 591,42,female,2,rent,NA,moderate,1965,24,car,good 594 | 592,36,female,1,own,rich,NA,1572,21,business,good 595 | 593,20,female,1,rent,little,moderate,2718,24,car,bad 596 | 594,40,male,3,own,NA,little,1358,24,vacation/others,bad 597 | 595,32,female,1,own,moderate,moderate,931,6,car,bad 598 | 596,23,female,2,rent,little,little,1442,24,car,bad 599 | 597,36,male,1,own,little,moderate,4241,24,business,bad 600 | 598,31,male,2,own,little,NA,2775,18,car,bad 601 | 599,32,male,2,free,little,NA,3863,24,business,good 602 | 600,45,female,2,own,little,moderate,2329,7,radio/TV,good 603 | 601,30,female,2,own,little,moderate,918,9,furniture/equipment,bad 604 | 602,34,female,1,free,little,moderate,1837,24,education,bad 605 | 603,28,female,3,own,little,NA,3349,36,furniture/equipment,bad 606 | 604,23,female,2,own,little,rich,1275,10,furniture/equipment,good 607 | 605,22,male,2,own,quite rich,little,2828,24,furniture/equipment,good 608 | 606,74,male,3,own,little,NA,4526,24,business,good 609 | 607,50,female,2,free,moderate,moderate,2671,36,radio/TV,bad 610 | 608,33,male,2,own,little,NA,2051,18,radio/TV,good 611 | 609,45,male,2,free,NA,NA,1300,15,car,good 612 | 610,22,female,2,own,moderate,little,741,12,domestic appliances,bad 613 | 611,48,female,1,free,moderate,rich,1240,10,car,bad 614 | 612,29,female,2,own,rich,little,3357,21,radio/TV,good 615 | 613,22,female,2,rent,little,little,3632,24,car,good 616 | 614,22,female,2,own,little,NA,1808,18,furniture/equipment,bad 617 | 615,48,male,3,own,NA,moderate,12204,48,business,good 618 | 616,27,male,3,free,NA,moderate,9157,60,radio/TV,good 619 | 617,37,male,2,rent,little,little,3676,6,car,good 620 | 618,21,female,2,rent,moderate,moderate,3441,30,furniture/equipment,bad 621 | 619,49,male,1,own,little,NA,640,12,car,good 622 | 620,27,male,2,own,little,moderate,3652,21,business,good 623 | 621,32,male,2,own,little,NA,1530,18,car,bad 624 | 622,38,male,2,own,NA,NA,3914,48,business,bad 625 | 623,22,female,2,rent,little,little,1858,12,furniture/equipment,good 626 | 624,65,male,2,free,little,little,2600,18,radio/TV,bad 627 | 625,35,male,2,own,NA,NA,1979,15,radio/TV,good 628 | 626,41,male,2,own,little,rich,2116,6,furniture/equipment,good 629 | 627,29,male,2,own,moderate,moderate,1437,9,car,bad 630 | 628,36,male,2,own,quite rich,NA,4042,42,furniture/equipment,good 631 | 629,64,male,1,own,NA,NA,3832,9,education,good 632 | 630,28,female,2,own,little,little,3660,24,radio/TV,good 633 | 631,44,male,2,own,little,little,1553,18,furniture/equipment,bad 634 | 632,23,male,2,own,NA,moderate,1444,15,radio/TV,good 635 | 633,19,female,2,rent,little,NA,1980,9,furniture/equipment,bad 636 | 634,25,female,1,own,little,moderate,1355,24,car,bad 637 | 635,47,male,2,own,little,NA,1393,12,education,good 638 | 636,28,female,2,own,quite rich,NA,1376,24,radio/TV,good 639 | 637,21,male,2,own,little,NA,15653,60,radio/TV,good 640 | 638,34,female,2,own,little,NA,1493,12,radio/TV,good 641 | 639,26,male,2,own,little,little,4370,42,radio/TV,bad 642 | 640,27,female,0,own,little,little,750,18,education,bad 643 | 641,38,male,1,own,little,moderate,1308,15,repairs,good 644 | 642,40,male,3,own,moderate,NA,4623,15,education,bad 645 | 643,33,male,2,own,little,NA,1851,24,radio/TV,good 646 | 644,32,male,3,own,little,little,1880,18,radio/TV,good 647 | 645,27,male,2,rent,NA,NA,7980,36,business,bad 648 | 646,32,male,2,own,little,little,4583,30,furniture/equipment,good 649 | 647,26,female,2,own,quite rich,NA,1386,12,car,bad 650 | 648,38,male,2,free,little,rich,947,24,car,bad 651 | 649,40,male,1,rent,little,little,684,12,education,bad 652 | 650,50,male,3,free,little,little,7476,48,education,good 653 | 651,37,male,1,own,little,moderate,1922,12,furniture/equipment,bad 654 | 652,45,male,2,own,little,little,2303,24,car,bad 655 | 653,42,male,3,own,moderate,moderate,8086,36,car,bad 656 | 654,35,male,2,own,little,NA,2346,24,car,good 657 | 655,22,male,2,free,little,little,3973,14,car,good 658 | 656,41,male,1,own,little,moderate,888,12,car,bad 659 | 657,37,male,2,own,NA,NA,10222,48,radio/TV,good 660 | 658,28,female,2,own,little,moderate,4221,30,business,good 661 | 659,41,male,2,own,little,moderate,6361,18,furniture/equipment,good 662 | 660,23,male,2,rent,little,rich,1297,12,radio/TV,good 663 | 661,23,male,2,own,NA,little,900,12,car,bad 664 | 662,50,male,2,own,little,NA,2241,21,furniture/equipment,good 665 | 663,35,male,3,own,little,moderate,1050,6,furniture/equipment,good 666 | 664,50,female,1,own,little,rich,1047,6,education,good 667 | 665,27,male,3,own,little,NA,6314,24,vacation/others,good 668 | 666,34,male,2,own,rich,moderate,3496,30,furniture/equipment,good 669 | 667,27,female,2,own,little,NA,3609,48,business,good 670 | 668,43,male,2,rent,little,little,4843,12,car,bad 671 | 669,47,male,2,own,little,rich,3017,30,radio/TV,good 672 | 670,27,male,1,own,moderate,NA,4139,24,business,good 673 | 671,31,male,2,own,moderate,NA,5742,36,business,good 674 | 672,42,male,3,own,little,NA,10366,60,car,good 675 | 673,24,male,2,own,quite rich,NA,2080,6,car,good 676 | 674,41,male,1,own,quite rich,NA,2580,21,business,bad 677 | 675,26,female,3,rent,little,NA,4530,30,radio/TV,good 678 | 676,33,male,2,own,little,NA,5150,24,furniture/equipment,good 679 | 677,24,male,2,own,moderate,moderate,5595,72,radio/TV,bad 680 | 678,64,male,1,rent,little,little,2384,24,radio/TV,good 681 | 679,26,female,2,own,little,NA,1453,18,radio/TV,good 682 | 680,56,female,2,own,little,NA,1538,6,education,good 683 | 681,37,male,2,free,NA,NA,2279,12,radio/TV,good 684 | 682,33,male,2,own,little,NA,1478,15,radio/TV,good 685 | 683,47,male,2,free,little,NA,5103,24,radio/TV,good 686 | 684,31,male,1,own,moderate,moderate,9857,36,business,good 687 | 685,34,male,2,free,NA,NA,6527,60,car,good 688 | 686,27,male,2,own,NA,rich,1347,10,radio/TV,good 689 | 687,30,male,2,free,moderate,moderate,2862,36,car,good 690 | 688,35,male,2,own,moderate,NA,2753,9,radio/TV,good 691 | 689,31,male,2,own,rich,little,3651,12,car,good 692 | 690,25,male,2,own,little,little,975,15,furniture/equipment,good 693 | 691,25,female,1,own,moderate,moderate,2631,15,repairs,good 694 | 692,29,male,2,own,moderate,moderate,2896,24,radio/TV,good 695 | 693,44,male,1,own,NA,little,4716,6,car,good 696 | 694,28,male,2,own,little,NA,2284,24,radio/TV,good 697 | 695,50,male,2,rent,quite rich,NA,1236,6,car,good 698 | 696,29,male,2,own,little,moderate,1103,12,radio/TV,good 699 | 697,38,female,0,own,little,NA,926,12,car,good 700 | 698,24,male,2,own,little,NA,1800,18,radio/TV,good 701 | 699,40,male,3,rent,little,rich,1905,15,education,good 702 | 700,29,female,1,rent,quite rich,NA,1123,12,furniture/equipment,bad 703 | 701,46,male,2,free,little,little,6331,48,car,bad 704 | 702,47,female,2,free,moderate,rich,1377,24,radio/TV,good 705 | 703,41,male,2,own,moderate,moderate,2503,30,business,good 706 | 704,32,female,2,own,little,moderate,2528,27,business,good 707 | 705,35,female,2,free,quite rich,NA,5324,15,car,good 708 | 706,24,male,2,own,moderate,moderate,6560,48,car,bad 709 | 707,25,female,2,rent,little,moderate,2969,12,furniture/equipment,bad 710 | 708,25,female,2,own,little,moderate,1206,9,radio/TV,good 711 | 709,37,male,1,own,little,moderate,2118,9,radio/TV,good 712 | 710,32,male,3,own,quite rich,NA,629,18,radio/TV,good 713 | 711,35,female,2,free,little,little,1198,6,education,bad 714 | 712,46,male,3,own,NA,NA,2476,21,car,good 715 | 713,25,male,1,own,little,little,1138,9,radio/TV,good 716 | 714,27,male,3,own,little,moderate,14027,60,car,bad 717 | 715,63,male,2,own,NA,NA,7596,30,car,good 718 | 716,40,male,2,own,NA,NA,3077,30,radio/TV,good 719 | 717,32,male,3,free,little,NA,1505,18,radio/TV,good 720 | 718,31,male,2,own,NA,rich,3148,24,radio/TV,good 721 | 719,31,male,2,own,moderate,moderate,6148,20,car,good 722 | 720,34,male,3,own,little,rich,1337,9,radio/TV,bad 723 | 721,24,female,2,rent,rich,moderate,433,6,education,bad 724 | 722,24,female,1,own,little,little,1228,12,car,bad 725 | 723,66,female,1,own,quite rich,moderate,790,9,radio/TV,good 726 | 724,21,female,2,rent,little,NA,2570,27,car,bad 727 | 725,41,female,1,own,rich,NA,250,6,car,good 728 | 726,47,male,1,own,quite rich,NA,1316,15,radio/TV,good 729 | 727,25,female,2,rent,little,little,1882,18,radio/TV,bad 730 | 728,59,female,2,rent,little,moderate,6416,48,business,bad 731 | 729,36,male,2,own,rich,rich,1275,24,business,good 732 | 730,33,male,2,own,little,moderate,6403,24,radio/TV,good 733 | 731,21,male,1,rent,little,little,1987,24,radio/TV,bad 734 | 732,44,female,1,own,little,moderate,760,8,radio/TV,good 735 | 733,28,female,2,rent,rich,NA,2603,24,car,good 736 | 734,37,female,2,own,little,NA,3380,4,car,good 737 | 735,29,female,0,own,NA,moderate,3990,36,domestic appliances,good 738 | 736,23,female,3,rent,little,moderate,11560,24,car,bad 739 | 737,35,male,1,own,moderate,little,4380,18,car,good 740 | 738,45,male,3,own,little,NA,6761,6,car,good 741 | 739,26,female,1,rent,moderate,moderate,4280,30,business,bad 742 | 740,32,male,2,own,moderate,little,2325,24,car,good 743 | 741,23,male,1,own,little,moderate,1048,10,radio/TV,good 744 | 742,41,male,2,own,NA,NA,3160,21,radio/TV,good 745 | 743,22,male,2,own,quite rich,little,2483,24,furniture/equipment,good 746 | 744,30,male,3,own,NA,little,14179,39,furniture/equipment,good 747 | 745,28,male,1,own,little,little,1797,13,business,good 748 | 746,23,female,2,rent,little,little,2511,15,car,good 749 | 747,37,female,1,own,little,little,1274,12,car,bad 750 | 748,26,male,2,own,NA,NA,5248,21,car,good 751 | 749,33,male,2,own,little,NA,3029,15,car,good 752 | 750,49,female,2,own,little,little,428,6,furniture/equipment,good 753 | 751,23,female,1,own,little,little,976,18,car,bad 754 | 752,23,female,1,rent,moderate,moderate,841,12,business,good 755 | 753,25,female,2,own,little,NA,5771,30,radio/TV,good 756 | 754,55,male,2,free,rich,NA,1555,12,repairs,bad 757 | 755,32,female,2,rent,NA,little,1285,24,car,bad 758 | 756,74,male,0,own,little,rich,1299,6,car,good 759 | 757,39,male,2,free,NA,rich,1271,15,radio/TV,bad 760 | 758,31,male,2,own,little,NA,1393,24,car,good 761 | 759,35,male,2,own,little,little,691,12,car,bad 762 | 760,59,female,2,own,NA,NA,5045,15,car,good 763 | 761,24,female,2,rent,little,little,2124,18,furniture/equipment,bad 764 | 762,24,male,1,own,little,little,2214,12,radio/TV,good 765 | 763,30,male,3,free,NA,NA,12680,21,car,bad 766 | 764,27,male,2,own,moderate,NA,2463,24,car,good 767 | 765,40,male,1,own,little,moderate,1155,12,radio/TV,good 768 | 766,31,male,1,own,little,little,3108,30,furniture/equipment,bad 769 | 767,31,female,2,rent,NA,NA,2901,10,car,good 770 | 768,28,male,2,rent,little,moderate,3617,12,furniture/equipment,good 771 | 769,63,male,1,own,little,NA,1655,12,radio/TV,good 772 | 770,26,female,2,rent,NA,little,2812,24,car,good 773 | 771,25,female,3,own,little,little,8065,36,education,bad 774 | 772,36,male,3,own,little,NA,3275,21,car,good 775 | 773,52,male,2,own,moderate,NA,2223,24,radio/TV,good 776 | 774,66,male,0,free,quite rich,rich,1480,12,car,good 777 | 775,25,female,2,rent,NA,little,1371,24,car,bad 778 | 776,37,male,2,own,little,NA,3535,36,car,good 779 | 777,25,female,2,own,little,little,3509,18,radio/TV,good 780 | 778,38,male,3,own,rich,NA,5711,36,car,good 781 | 779,67,female,2,own,little,moderate,3872,18,repairs,good 782 | 780,25,male,2,own,little,moderate,4933,39,radio/TV,bad 783 | 781,60,male,2,own,rich,NA,1940,24,car,good 784 | 782,31,male,1,own,little,moderate,1410,12,education,good 785 | 783,23,female,1,own,moderate,moderate,836,12,car,bad 786 | 784,60,male,3,own,NA,moderate,6468,20,car,good 787 | 785,35,male,1,own,rich,moderate,1941,18,business,good 788 | 786,40,male,2,own,quite rich,NA,2675,22,radio/TV,good 789 | 787,38,male,2,own,NA,NA,2751,48,car,good 790 | 788,50,male,2,free,little,moderate,6224,48,education,bad 791 | 789,27,male,2,own,little,little,5998,40,education,bad 792 | 790,39,female,2,own,little,moderate,1188,21,business,bad 793 | 791,41,male,3,own,NA,NA,6313,24,car,good 794 | 792,27,male,2,own,NA,NA,1221,6,furniture/equipment,good 795 | 793,51,male,2,free,little,rich,2892,24,furniture/equipment,good 796 | 794,32,male,2,rent,quite rich,NA,3062,24,furniture/equipment,good 797 | 795,22,female,2,rent,moderate,NA,2301,9,furniture/equipment,good 798 | 796,51,male,2,free,NA,little,7511,18,car,bad 799 | 797,22,female,1,rent,little,NA,1258,12,furniture/equipment,good 800 | 798,54,male,2,own,NA,NA,717,24,car,good 801 | 799,35,male,0,own,NA,moderate,1549,9,car,good 802 | 800,54,male,2,free,little,NA,1597,24,education,good 803 | 801,48,female,1,rent,little,moderate,1795,18,radio/TV,good 804 | 802,24,female,2,own,little,little,4272,20,furniture/equipment,good 805 | 803,35,male,2,own,NA,NA,976,12,radio/TV,good 806 | 804,24,female,0,rent,NA,moderate,7472,12,car,good 807 | 805,24,male,2,own,little,little,9271,36,car,bad 808 | 806,26,male,1,own,little,moderate,590,6,radio/TV,good 809 | 807,65,male,2,own,NA,NA,930,12,radio/TV,good 810 | 808,55,male,3,free,little,moderate,9283,42,car,good 811 | 809,26,female,0,rent,little,moderate,1778,15,car,bad 812 | 810,26,male,2,own,little,moderate,907,8,business,good 813 | 811,28,male,1,own,little,moderate,484,6,radio/TV,good 814 | 812,24,male,2,own,little,little,9629,36,car,bad 815 | 813,54,male,2,own,little,little,3051,48,domestic appliances,bad 816 | 814,46,male,2,free,little,little,3931,48,car,bad 817 | 815,54,female,2,rent,little,moderate,7432,36,car,good 818 | 816,62,male,2,own,quite rich,NA,1338,6,domestic appliances,good 819 | 817,24,female,2,rent,little,NA,1554,6,radio/TV,good 820 | 818,43,male,3,own,little,little,15857,36,vacation/others,good 821 | 819,26,male,2,own,little,little,1345,18,radio/TV,bad 822 | 820,27,male,2,own,little,NA,1101,12,car,good 823 | 821,24,male,2,own,little,rich,3016,12,radio/TV,good 824 | 822,41,male,2,own,little,little,2712,36,furniture/equipment,bad 825 | 823,47,male,1,own,little,little,731,8,car,good 826 | 824,35,male,3,own,little,NA,3780,18,furniture/equipment,good 827 | 825,30,male,2,own,little,little,1602,21,car,good 828 | 826,33,female,2,rent,little,little,3966,18,car,bad 829 | 827,36,male,2,own,little,NA,4165,18,business,bad 830 | 828,47,male,2,free,NA,little,8335,36,car,bad 831 | 829,38,male,2,free,NA,moderate,6681,48,business,good 832 | 830,44,male,2,own,quite rich,NA,2375,24,business,good 833 | 831,23,female,2,rent,little,little,1216,18,car,bad 834 | 832,29,male,2,rent,little,little,11816,45,business,bad 835 | 833,42,female,2,own,NA,moderate,5084,24,radio/TV,good 836 | 834,25,female,1,own,little,rich,2327,15,radio/TV,bad 837 | 835,48,male,2,own,little,little,1082,12,car,bad 838 | 836,21,female,2,own,NA,NA,886,12,radio/TV,good 839 | 837,23,female,1,rent,little,NA,601,4,furniture/equipment,good 840 | 838,63,male,2,own,little,little,2957,24,car,good 841 | 839,46,male,2,own,little,NA,2611,24,radio/TV,good 842 | 840,29,male,2,own,little,little,5179,36,furniture/equipment,bad 843 | 841,28,male,1,own,little,NA,2993,21,car,good 844 | 842,23,female,2,own,little,NA,1943,18,repairs,bad 845 | 843,50,male,2,own,little,NA,1559,24,business,good 846 | 844,47,male,2,own,little,NA,3422,18,furniture/equipment,good 847 | 845,35,male,2,own,NA,moderate,3976,21,furniture/equipment,good 848 | 846,68,male,2,rent,NA,NA,6761,18,car,bad 849 | 847,28,male,2,own,little,NA,1249,24,car,good 850 | 848,59,male,2,own,little,little,1364,9,radio/TV,good 851 | 849,57,male,1,own,little,little,709,12,radio/TV,bad 852 | 850,33,male,2,rent,little,little,2235,20,car,bad 853 | 851,43,male,2,own,NA,NA,4042,24,car,good 854 | 852,35,male,2,free,little,NA,1471,15,radio/TV,good 855 | 853,32,male,1,free,little,little,1442,18,car,bad 856 | 854,45,male,2,own,little,NA,10875,36,car,good 857 | 855,33,male,2,own,moderate,NA,1474,24,car,good 858 | 856,40,female,2,own,NA,NA,894,10,education,good 859 | 857,28,male,2,free,little,NA,3343,15,furniture/equipment,good 860 | 858,29,female,2,own,little,little,3959,15,car,bad 861 | 859,26,male,2,rent,moderate,NA,3577,9,car,good 862 | 860,27,male,2,own,rich,NA,5804,24,car,good 863 | 861,28,male,2,own,little,NA,2169,18,business,bad 864 | 862,35,female,2,own,little,little,2439,24,radio/TV,bad 865 | 863,32,male,1,own,rich,NA,4526,27,furniture/equipment,good 866 | 864,25,male,1,rent,little,NA,2210,10,furniture/equipment,bad 867 | 865,20,female,2,rent,quite rich,NA,2221,15,furniture/equipment,good 868 | 866,27,female,2,own,little,little,2389,18,radio/TV,good 869 | 867,42,male,2,own,little,NA,3331,12,furniture/equipment,good 870 | 868,37,male,2,own,NA,NA,7409,36,business,good 871 | 869,24,female,2,rent,little,little,652,12,furniture/equipment,good 872 | 870,40,female,2,own,quite rich,NA,7678,36,furniture/equipment,good 873 | 871,46,male,2,own,little,rich,1343,6,car,good 874 | 872,26,male,2,own,moderate,little,1382,24,business,good 875 | 873,24,female,2,own,NA,NA,874,15,domestic appliances,good 876 | 874,29,male,1,own,little,little,3590,12,furniture/equipment,good 877 | 875,40,female,2,own,rich,moderate,1322,11,car,good 878 | 876,36,male,3,free,little,little,1940,18,radio/TV,good 879 | 877,28,male,2,own,little,NA,3595,36,radio/TV,good 880 | 878,27,male,3,free,little,little,1422,9,car,bad 881 | 879,36,male,2,own,NA,NA,6742,30,radio/TV,good 882 | 880,38,male,3,own,little,NA,7814,24,car,good 883 | 881,48,male,2,free,NA,NA,9277,24,car,good 884 | 882,36,male,2,own,NA,moderate,2181,30,car,good 885 | 883,65,female,0,own,little,NA,1098,18,radio/TV,good 886 | 884,43,male,2,own,little,moderate,4057,24,furniture/equipment,bad 887 | 885,53,female,2,own,little,little,795,12,education,bad 888 | 886,34,male,2,own,NA,moderate,2825,24,business,good 889 | 887,23,male,2,own,little,moderate,15672,48,business,bad 890 | 888,34,male,3,own,little,NA,6614,36,car,good 891 | 889,40,male,2,rent,NA,NA,7824,28,car,good 892 | 890,43,male,3,own,little,little,2442,27,business,good 893 | 891,46,male,2,own,little,NA,1829,15,radio/TV,good 894 | 892,38,male,1,own,little,little,2171,12,car,good 895 | 893,34,male,2,own,little,moderate,5800,36,car,good 896 | 894,29,male,2,own,NA,NA,1169,18,radio/TV,good 897 | 895,31,male,3,own,NA,NA,8947,36,car,good 898 | 896,28,female,3,rent,little,little,2606,21,radio/TV,good 899 | 897,35,female,2,own,rich,NA,1592,12,furniture/equipment,good 900 | 898,33,female,1,rent,NA,NA,2186,15,furniture/equipment,good 901 | 899,42,male,2,own,little,little,4153,18,furniture/equipment,bad 902 | 900,43,male,2,rent,little,little,2625,16,car,bad 903 | 901,44,male,2,own,NA,NA,3485,20,car,good 904 | 902,42,male,2,free,NA,NA,10477,36,car,good 905 | 903,40,male,2,rent,NA,NA,1386,15,radio/TV,good 906 | 904,36,male,3,own,little,NA,1278,24,radio/TV,good 907 | 905,20,male,3,rent,little,little,1107,12,radio/TV,good 908 | 906,24,male,1,own,NA,little,3763,21,car,good 909 | 907,27,male,2,own,NA,moderate,3711,36,education,good 910 | 908,46,female,1,own,little,NA,3594,15,car,good 911 | 909,33,female,1,own,NA,moderate,3195,9,car,good 912 | 910,34,female,2,own,little,NA,4454,36,radio/TV,good 913 | 911,25,female,1,own,little,moderate,4736,24,furniture/equipment,bad 914 | 912,25,female,2,own,NA,moderate,2991,30,radio/TV,good 915 | 913,28,male,2,own,rich,NA,2142,11,business,good 916 | 914,31,male,2,rent,little,little,3161,24,business,bad 917 | 915,32,female,3,own,little,moderate,18424,48,vacation/others,bad 918 | 916,32,male,2,own,moderate,NA,2848,10,car,good 919 | 917,68,male,3,own,little,little,14896,6,car,bad 920 | 918,33,male,2,own,moderate,little,2359,24,furniture/equipment,bad 921 | 919,39,male,3,rent,little,little,3345,24,furniture/equipment,bad 922 | 920,28,female,2,own,little,NA,1817,18,furniture/equipment,good 923 | 921,37,male,3,own,quite rich,NA,12749,48,radio/TV,good 924 | 922,22,female,2,rent,little,little,1366,9,radio/TV,bad 925 | 923,30,male,2,rent,little,moderate,2002,12,car,good 926 | 924,55,male,2,own,little,little,6872,24,furniture/equipment,bad 927 | 925,46,male,2,own,little,little,697,12,car,bad 928 | 926,21,female,2,rent,little,little,1049,18,furniture/equipment,good 929 | 927,39,male,2,free,little,little,10297,48,car,bad 930 | 928,58,male,2,own,NA,NA,1867,30,radio/TV,good 931 | 929,43,male,1,own,little,little,1344,12,car,good 932 | 930,24,male,1,own,little,little,1747,24,furniture/equipment,good 933 | 931,22,female,2,own,little,moderate,1670,9,radio/TV,bad 934 | 932,30,male,2,own,little,NA,1224,9,car,good 935 | 933,42,male,2,own,quite rich,NA,522,12,radio/TV,good 936 | 934,23,female,2,own,little,little,1498,12,radio/TV,good 937 | 935,30,male,3,own,moderate,moderate,1919,30,radio/TV,bad 938 | 936,28,female,1,own,little,rich,745,9,radio/TV,bad 939 | 937,30,male,3,rent,little,moderate,2063,6,radio/TV,good 940 | 938,42,male,2,free,little,moderate,6288,60,education,bad 941 | 939,46,male,3,own,NA,NA,6842,24,car,good 942 | 940,45,male,3,own,NA,NA,3527,12,car,good 943 | 941,31,male,1,own,little,NA,1546,10,car,good 944 | 942,31,male,2,own,NA,NA,929,24,furniture/equipment,good 945 | 943,42,male,1,own,little,NA,1455,4,car,good 946 | 944,46,female,2,rent,little,little,1845,15,furniture/equipment,good 947 | 945,30,female,2,own,quite rich,moderate,8358,48,car,good 948 | 946,30,male,2,free,quite rich,little,3349,24,furniture/equipment,bad 949 | 947,38,male,3,own,NA,NA,2859,12,car,good 950 | 948,43,male,1,own,little,NA,1533,18,furniture/equipment,bad 951 | 949,31,male,2,own,moderate,NA,3621,24,radio/TV,bad 952 | 950,40,male,0,own,little,moderate,3590,18,business,good 953 | 951,24,male,2,own,little,little,2145,36,business,bad 954 | 952,28,female,2,rent,quite rich,moderate,4113,24,car,bad 955 | 953,26,female,3,own,little,NA,10974,36,furniture/equipment,bad 956 | 954,29,female,2,own,little,little,1893,12,car,good 957 | 955,57,female,3,rent,rich,little,1231,24,radio/TV,good 958 | 956,49,male,1,own,NA,rich,3656,30,radio/TV,good 959 | 957,37,male,1,own,little,moderate,1154,9,radio/TV,good 960 | 958,45,male,1,own,little,little,4006,28,car,bad 961 | 959,30,male,2,free,moderate,moderate,3069,24,furniture/equipment,good 962 | 960,30,male,2,rent,little,NA,1740,6,radio/TV,good 963 | 961,47,male,2,own,little,moderate,2353,21,car,good 964 | 962,29,male,2,own,NA,NA,3556,15,car,good 965 | 963,35,male,2,own,quite rich,NA,2397,24,radio/TV,bad 966 | 964,22,male,1,own,little,moderate,454,6,repairs,good 967 | 965,26,female,2,own,NA,moderate,1715,30,radio/TV,good 968 | 966,23,male,1,own,quite rich,moderate,2520,27,radio/TV,bad 969 | 967,54,female,3,rent,little,NA,3568,15,radio/TV,good 970 | 968,29,male,2,rent,NA,NA,7166,42,radio/TV,good 971 | 969,40,male,1,own,little,little,3939,11,car,good 972 | 970,22,male,2,own,moderate,moderate,1514,15,repairs,good 973 | 971,43,male,1,own,little,NA,7393,24,car,good 974 | 972,29,female,0,rent,little,little,1193,24,car,bad 975 | 973,36,male,2,rent,little,little,7297,60,business,bad 976 | 974,33,female,2,own,little,NA,2831,30,radio/TV,good 977 | 975,57,female,1,own,quite rich,rich,1258,24,radio/TV,good 978 | 976,64,female,2,own,little,moderate,753,6,radio/TV,good 979 | 977,42,male,2,own,NA,moderate,2427,18,business,good 980 | 978,47,male,1,own,little,NA,2538,24,car,bad 981 | 979,25,male,2,rent,moderate,moderate,1264,15,car,bad 982 | 980,49,male,2,own,little,moderate,8386,30,furniture/equipment,bad 983 | 981,33,male,3,rent,little,NA,4844,48,business,bad 984 | 982,28,female,3,own,moderate,rich,2923,21,car,good 985 | 983,26,male,2,own,little,little,8229,36,car,bad 986 | 984,30,male,1,own,little,NA,2028,24,furniture/equipment,good 987 | 985,25,female,2,rent,little,little,1433,15,furniture/equipment,good 988 | 986,33,male,2,own,little,rich,6289,42,business,good 989 | 987,64,female,2,own,moderate,NA,1409,13,radio/TV,good 990 | 988,29,male,3,free,little,little,6579,24,car,good 991 | 989,48,male,1,own,little,moderate,1743,24,radio/TV,good 992 | 990,37,male,1,own,NA,NA,3565,12,education,good 993 | 991,34,male,1,own,moderate,NA,1569,15,radio/TV,good 994 | 992,23,male,1,rent,NA,little,1936,18,radio/TV,good 995 | 993,30,male,3,own,little,little,3959,36,furniture/equipment,good 996 | 994,50,male,2,own,NA,NA,2390,12,car,good 997 | 995,31,female,1,own,little,NA,1736,12,furniture/equipment,good 998 | 996,40,male,3,own,little,little,3857,30,car,good 999 | 997,38,male,2,own,little,NA,804,12,radio/TV,good 1000 | 998,23,male,2,free,little,little,1845,45,radio/TV,bad 1001 | 999,27,male,2,own,moderate,moderate,4576,45,car,good 1002 | -------------------------------------------------------------------------------- /media/stages.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/csiebler/mlops-demo/8cd833b0878fb84ddd90af19f22af91e3e904f47/media/stages.png -------------------------------------------------------------------------------- /models/german-credit-basic/.amlignore: -------------------------------------------------------------------------------- 1 | .ipynb_checkpoints 2 | azureml-logs 3 | .azureml 4 | .git 5 | outputs 6 | azureml-setup 7 | docs 8 | __pycache__/ 9 | notebooks -------------------------------------------------------------------------------- /models/german-credit-basic/README.md: -------------------------------------------------------------------------------- 1 | # Running this code from your machine 2 | 3 | ## Running this code locally using Azure Machine Learning in Docker 4 | 5 | We can fully run this example locally on our machine, while still logging the results to Azure Machine Learning. 6 | 7 | Let's make sure we are in the model folder: 8 | 9 | ```console 10 | cd models/german-credit-basic 11 | ``` 12 | 13 | Now, we can attach this local folder to the resource group with our Azure Machine Learning workspace: 14 | 15 | ```console 16 | az ml folder attach -g csamlfsi-rg -w csamlfsi-ws 17 | ``` 18 | (`-w` refers to the workspace name and `-g` to the resource group of the workspace) 19 | 20 | Next, we can run the `train.py` locally in a Docker container on our machine: 21 | 22 | ```console 23 | az ml run submit-script -c config/train-local -e german-credit-train-local 24 | ``` 25 | 26 | This will create a Docker container with the conda enviroment from [`config/train-conda.yml`](config/train-conda.yml), inject our local code from this directory, and then execute [`train.py`](train.py) in it. The full configuration can be found in [`config/train-local.runconfig`](config/train-local.runconfig). All training will happen locally on your machine, however, the training run, metrics, artifacts, etc. will still show up in Azure Machine Learning. The data will come from the `Data Set` in Azure Machine Learning. 27 | 28 | ## Running this code on Azure Machine Learning Compute 29 | 30 | Finally, we can run this example directly on Azure Machine Learning compute by using a different `runconfig` file. Let's make sure we are still in the `models/german-credit-basic` folder and that we executed `az ml folder attach -g csamlfsi-rg -w csamlfsi-ws` beforehand. 31 | 32 | To run the `train.py` on Azure Machine Learning compute, we just use the [`config/train-amlcompute.runconfig`](config/train-amlcompute.runconfig) configuration: 33 | 34 | ```console 35 | az ml run submit-script -c config/train-amlcompute -e german-credit-train-amlcompute -t run.json 36 | ``` 37 | 38 | This will do exactly the same as locally, but everything will happen in Azure. The `-t` option outputs a reference to the experiment run, which can be used to directly register the model from it. The full configuration can be found in [`config/train-amlcompute.runconfig`](config/train-amlcompute.runconfig). 39 | 40 | ## Model registration 41 | 42 | Given the `run-metadata-file` from the experiment run, we can now register the model in AML. In this case, the `asset-path` points to the path structure within the experiment run: 43 | 44 | ```console 45 | az ml model register --name german-credit-basic-model --asset-path outputs/credit-prediction.pkl --run-metadata-file run.json 46 | ``` 47 | 48 | ## Running inferencing locally 49 | 50 | We can deploy the model locally using Docker: 51 | 52 | ```console 53 | az ml model deploy --name german-credit-qa-local --model german-credit-basic-model:1 --inference-config-file config/inference-config.yml --deploy-config-file config/deployment-config-aci-qa.yml --runtime python --compute-type local --port 32000 --overwrite 54 | ``` 55 | 56 | We can test the local endpoint using the following request: 57 | 58 | ``` 59 | POST http://localhost:32000/score HTTP/1.1 60 | Content-Type: application/json 61 | 62 | { 63 | "data": [ 64 | { 65 | "Age": 20, 66 | "Sex": "male", 67 | "Job": 0, 68 | "Housing": "own", 69 | "Saving accounts": "little", 70 | "Checking account": "little", 71 | "Credit amount": 100, 72 | "Duration": 48, 73 | "Purpose": "radio/TV" 74 | } 75 | ] 76 | } 77 | ``` 78 | 79 | Once we know it is working, we can remove the local service via: 80 | 81 | ```console 82 | az ml service delete --name german-credit-qa-local 83 | ``` 84 | 85 | ## Running inferencing on Azure 86 | 87 | Finally, we can deploy to Azure. First, we can try deploying to Azure Container Instances: 88 | 89 | ```console 90 | az ml model deploy -n german-credit-qa-aci --model german-credit-basic-model:1 --inference-config-file config/inference-config.yml --deploy-config-file config/deployment-config-aci-qa.yml --overwrite 91 | ``` 92 | 93 | We can test the ACI endpoint using the following request (make sure to replace the URL with your container's URL): 94 | 95 | ``` 96 | POST http://xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx.xxxxxx.azurecontainer.io/score HTTP/1.1 97 | Content-Type: application/json 98 | 99 | { 100 | "data": [ 101 | { 102 | "Age": 20, 103 | "Sex": "male", 104 | "Job": 0, 105 | "Housing": "own", 106 | "Saving accounts": "little", 107 | "Checking account": "little", 108 | "Credit amount": 100, 109 | "Duration": 48, 110 | "Purpose": "radio/TV" 111 | } 112 | ] 113 | } 114 | ``` 115 | 116 | We can remove the ACI-based service via: 117 | 118 | ```console 119 | az ml service delete --name german-credit-qa-aci 120 | ``` -------------------------------------------------------------------------------- /models/german-credit-basic/config/deployment-config-aci.yml: -------------------------------------------------------------------------------- 1 | computeType: ACI 2 | 3 | containerResourceRequirements: 4 | cpu: 1 5 | memoryInGB: 0.5 6 | 7 | authEnabled: True 8 | 9 | appInsightsEnabled: True 10 | sslEnabled: False -------------------------------------------------------------------------------- /models/german-credit-basic/config/deployment-config-aks.yml: -------------------------------------------------------------------------------- 1 | computeType: AKS 2 | 3 | autoScaler: 4 | autoscaleEnabled: True 5 | minReplicas: 1 6 | maxReplicas: 2 7 | refreshPeriodInSeconds: 10 8 | targetUtilization: 70 9 | 10 | containerResourceRequirements: 11 | cpu: 0.25 12 | memoryInGB: 0.25 13 | 14 | # Only one can be True 15 | authEnabled: True 16 | tokenAuthEnabled: False 17 | 18 | appInsightsEnabled: True 19 | sslEnabled: True 20 | 21 | dataCollection: 22 | storageEnabled: True -------------------------------------------------------------------------------- /models/german-credit-basic/config/inference-conda.yml: -------------------------------------------------------------------------------- 1 | name: german-credit-score 2 | channels: 3 | - conda-forge 4 | - defaults 5 | dependencies: 6 | - python=3.7.9 7 | - pip: 8 | # Azure ML 9 | - azureml-defaults==1.40.0 10 | - inference-schema==1.3.0 11 | - azureml-monitoring 12 | # Other 13 | - scikit-learn==0.24.1 14 | - joblib==1.0.0 15 | - pandas==1.2.1 16 | -------------------------------------------------------------------------------- /models/german-credit-basic/config/inference-config.yml: -------------------------------------------------------------------------------- 1 | entryScript: score.py 2 | runtime: python 3 | condaFile: config/inference-conda.yml 4 | extraDockerfileSteps: 5 | schemaFile: 6 | sourceDirectory: . 7 | enableGpu: False 8 | baseImage: 9 | baseImageRegistry: 10 | -------------------------------------------------------------------------------- /models/german-credit-basic/config/train-amlcompute.runconfig: -------------------------------------------------------------------------------- 1 | script: train.py 2 | arguments: [--data_path, /data, --model_name, credit-prediction.pkl] 3 | target: cpu-cluster 4 | framework: Python 5 | nodeCount: 1 6 | environment: 7 | environmentVariables: 8 | EXAMPLE_ENV_VAR: EXAMPLE_VALUE 9 | python: 10 | userManagedDependencies: false 11 | interpreterPath: python 12 | condaDependenciesFile: config/train-conda.yml 13 | docker: 14 | enabled: true 15 | baseImage: mcr.microsoft.com/azureml/base:intelmpi2018.3-ubuntu16.04 16 | sharedVolumes: true 17 | data: 18 | training_dataset: 19 | environmentVariableName: training_dataset 20 | dataLocation: 21 | dataset: 22 | name: german_credit_file 23 | version: 1 24 | mechanism: download 25 | pathOnCompute: /data 26 | overwrite: true -------------------------------------------------------------------------------- /models/german-credit-basic/config/train-conda.yml: -------------------------------------------------------------------------------- 1 | name: german-credit-train 2 | channels: 3 | - conda-forge 4 | - defaults 5 | dependencies: 6 | - python=3.7.9 7 | - pip: 8 | # Azure ML 9 | - azureml-defaults==1.33.0 10 | - azureml-sdk==1.33.0 11 | - azureml-interpret==1.33.0 12 | # Other 13 | - scikit-learn==0.24.1 14 | - joblib==1.0.0 15 | - pandas==1.2.1 16 | -------------------------------------------------------------------------------- /models/german-credit-basic/config/train-local.runconfig: -------------------------------------------------------------------------------- 1 | script: train.py 2 | arguments: [--data_path, /data, --model_name, credit-prediction.pkl] 3 | target: local 4 | framework: Python 5 | communicator: None 6 | environment: 7 | environmentVariables: 8 | EXAMPLE_ENV_VAR: EXAMPLE_VALUE 9 | python: 10 | userManagedDependencies: false 11 | interpreterPath: python 12 | condaDependenciesFile: config/train-conda.yml 13 | docker: 14 | enabled: true 15 | baseImage: mcr.microsoft.com/azureml/base:intelmpi2018.3-ubuntu16.04 16 | arguments: [] 17 | data: 18 | training_dataset: 19 | environmentVariableName: training_dataset 20 | dataLocation: 21 | dataset: 22 | name: german_credit_file 23 | version: 1 24 | mechanism: download 25 | pathOnCompute: /data 26 | overwrite: true -------------------------------------------------------------------------------- /models/german-credit-basic/notebooks/.aml_ignore: -------------------------------------------------------------------------------- 1 | explanation 2 | outputs 3 | download_explanation 4 | .git -------------------------------------------------------------------------------- /models/german-credit-basic/notebooks/deploy_webservices.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Deploy webservice" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": null, 13 | "metadata": {}, 14 | "outputs": [], 15 | "source": [ 16 | "from azureml.core.model import Model, InferenceConfig\n", 17 | "from azureml.core import Workspace, Environment \n", 18 | "from azureml.core.webservice import AciWebservice\n", 19 | "\n", 20 | "ws = Workspace.from_config()\n", 21 | "model = Model(ws, 'german-credit-basic-model')" 22 | ] 23 | }, 24 | { 25 | "cell_type": "code", 26 | "execution_count": null, 27 | "metadata": {}, 28 | "outputs": [], 29 | "source": [ 30 | "# Define deployment setup\n", 31 | "\n", 32 | "# Conda enviroment (if we want to use additional Python packages)\n", 33 | "env = Environment.from_conda_specification(\"inference-env\", \"../config/inference-conda.yml\")\n", 34 | "\n", 35 | "inference_config = InferenceConfig(entry_script='../score.py', environment=env)\n", 36 | "\n", 37 | "# Define how our webservice should look like (resources, security, etc.)\n", 38 | "deployment_config = AciWebservice.deploy_configuration(cpu_cores=1, memory_gb=1)" 39 | ] 40 | }, 41 | { 42 | "cell_type": "code", 43 | "execution_count": null, 44 | "metadata": {}, 45 | "outputs": [], 46 | "source": [ 47 | "service = Model.deploy(ws, \"model-test-deployment\", [model], inference_config, deployment_config)\n", 48 | "service.wait_for_deployment(show_output=True)" 49 | ] 50 | }, 51 | { 52 | "cell_type": "code", 53 | "execution_count": null, 54 | "metadata": {}, 55 | "outputs": [], 56 | "source": [ 57 | "service.scoring_uri" 58 | ] 59 | }, 60 | { 61 | "cell_type": "code", 62 | "execution_count": null, 63 | "metadata": {}, 64 | "outputs": [], 65 | "source": [ 66 | "import requests\n", 67 | "import json\n", 68 | "\n", 69 | "url = service.scoring_uri\n", 70 | "\n", 71 | "test_data = {\n", 72 | " 'data': [{\n", 73 | " \"Age\": 20,\n", 74 | " \"Sex\": \"male\",\n", 75 | " \"Job\": 0,\n", 76 | " \"Housing\": \"own\",\n", 77 | " \"Saving accounts\": \"little\",\n", 78 | " \"Checking account\": \"little\",\n", 79 | " \"Credit amount\": 100,\n", 80 | " \"Duration\": 48,\n", 81 | " \"Purpose\": \"radio/TV\"\n", 82 | " }]\n", 83 | "}\n", 84 | "\n", 85 | "headers = {'Content-Type':'application/json'}\n", 86 | "resp = requests.post(url, json=test_data, headers=headers)\n", 87 | "\n", 88 | "print(\"Prediction (good, bad):\", resp.text)" 89 | ] 90 | }, 91 | { 92 | "cell_type": "code", 93 | "execution_count": null, 94 | "metadata": {}, 95 | "outputs": [], 96 | "source": [] 97 | } 98 | ], 99 | "metadata": { 100 | "kernelspec": { 101 | "display_name": "Python 3.6 - AzureML", 102 | "language": "python", 103 | "name": "python3-azureml" 104 | }, 105 | "language_info": { 106 | "codemirror_mode": { 107 | "name": "ipython", 108 | "version": 3 109 | }, 110 | "file_extension": ".py", 111 | "mimetype": "text/x-python", 112 | "name": "python", 113 | "nbconvert_exporter": "python", 114 | "pygments_lexer": "ipython3", 115 | "version": "3.6.9" 116 | } 117 | }, 118 | "nbformat": 4, 119 | "nbformat_minor": 4 120 | } 121 | -------------------------------------------------------------------------------- /models/german-credit-basic/notebooks/german-credit-amlcompute.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Connect to workspace" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": null, 13 | "metadata": { 14 | "tags": [] 15 | }, 16 | "outputs": [], 17 | "source": [ 18 | "import azureml.core\n", 19 | "from azureml.core import Workspace, Dataset\n", 20 | "\n", 21 | "ws = Workspace.from_config()\n", 22 | "\n", 23 | "print(azureml.core.VERSION)" 24 | ] 25 | }, 26 | { 27 | "cell_type": "markdown", 28 | "metadata": {}, 29 | "source": [ 30 | "# Create compute cluster" 31 | ] 32 | }, 33 | { 34 | "cell_type": "code", 35 | "execution_count": null, 36 | "metadata": { 37 | "tags": [] 38 | }, 39 | "outputs": [], 40 | "source": [ 41 | "from azureml.core.compute import AmlCompute, ComputeTarget\r\n", 42 | "\r\n", 43 | "compute_name = \"cpu-cluster\"\r\n", 44 | "\r\n", 45 | "if compute_name in ws.compute_targets:\r\n", 46 | " compute_target = ws.compute_targets[compute_name]\r\n", 47 | " if compute_target and type(compute_target) is AmlCompute:\r\n", 48 | " print('Compute target exists, will reuse it.')\r\n", 49 | "else:\r\n", 50 | " print('Creating a new compute target...')\r\n", 51 | " config = AmlCompute.provisioning_configuration(vm_size = 'STANDARD_DS3_V2',\r\n", 52 | " min_nodes = 0, \r\n", 53 | " max_nodes = 1)\r\n", 54 | " compute_target = ComputeTarget.create(ws, compute_name, config)\r\n", 55 | " compute_target.wait_for_completion(show_output=True)\r\n", 56 | " print(compute_target.get_status().serialize())" 57 | ] 58 | }, 59 | { 60 | "cell_type": "markdown", 61 | "metadata": {}, 62 | "source": [ 63 | "# Generate experiment and run it in Azure Machine Learning" 64 | ] 65 | }, 66 | { 67 | "cell_type": "markdown", 68 | "metadata": {}, 69 | "source": [ 70 | "Create experiment:" 71 | ] 72 | }, 73 | { 74 | "cell_type": "code", 75 | "execution_count": null, 76 | "metadata": {}, 77 | "outputs": [], 78 | "source": [ 79 | "from azureml.core import Experiment\r\n", 80 | "\r\n", 81 | "experiment_name = 'german_credit_data_remote'\r\n", 82 | "experiment = Experiment(ws, experiment_name)" 83 | ] 84 | }, 85 | { 86 | "cell_type": "markdown", 87 | "metadata": {}, 88 | "source": [ 89 | "Define runtime environmnent:" 90 | ] 91 | }, 92 | { 93 | "cell_type": "code", 94 | "execution_count": null, 95 | "metadata": {}, 96 | "outputs": [], 97 | "source": [ 98 | "from azureml.core import Environment\r\n", 99 | "from azureml.core.runconfig import DockerConfiguration\r\n", 100 | "from azureml.core.conda_dependencies import CondaDependencies\r\n", 101 | "\r\n", 102 | "train_env = Environment(\"train-env\")\r\n", 103 | "train_env.python.conda_dependencies = CondaDependencies(\"../config/train-conda.yml\")\r\n", 104 | "\r\n", 105 | "docker_config = DockerConfiguration(use_docker=True)" 106 | ] 107 | }, 108 | { 109 | "cell_type": "markdown", 110 | "metadata": {}, 111 | "source": [ 112 | "Load our dataset and create a script run:" 113 | ] 114 | }, 115 | { 116 | "cell_type": "code", 117 | "execution_count": null, 118 | "metadata": {}, 119 | "outputs": [], 120 | "source": [ 121 | "from azureml.core import ScriptRunConfig\r\n", 122 | "\r\n", 123 | "dataset = Dataset.get_by_name(ws, name='german_credit_file')\r\n", 124 | "\r\n", 125 | "config = ScriptRunConfig(source_directory='../', \r\n", 126 | " script='train.py',\r\n", 127 | " arguments=['--data_path', dataset.as_download(), '--model_name', 'credit-prediction.pkl'],\r\n", 128 | " compute_target=compute_target, \r\n", 129 | " environment=train_env,\r\n", 130 | " docker_runtime_config=docker_config)\r\n", 131 | "\r\n", 132 | "run = experiment.submit(config=config)\r\n", 133 | "run" 134 | ] 135 | }, 136 | { 137 | "cell_type": "code", 138 | "execution_count": null, 139 | "metadata": {}, 140 | "outputs": [], 141 | "source": [ 142 | "from azureml.widgets import RunDetails\n", 143 | "RunDetails(run).show()" 144 | ] 145 | }, 146 | { 147 | "cell_type": "code", 148 | "execution_count": null, 149 | "metadata": {}, 150 | "outputs": [], 151 | "source": [ 152 | "run.get_metrics()" 153 | ] 154 | }, 155 | { 156 | "cell_type": "markdown", 157 | "metadata": {}, 158 | "source": [ 159 | "## Register model to workspace" 160 | ] 161 | }, 162 | { 163 | "cell_type": "code", 164 | "execution_count": null, 165 | "metadata": {}, 166 | "outputs": [], 167 | "source": [ 168 | "model = run.register_model(model_name='german-credit-basic-model',\n", 169 | " model_path='outputs/credit-prediction.pkl',\n", 170 | " datasets=[['training-dataset', dataset]],\n", 171 | " tags={'source': 'amlcompute_training_demo'})" 172 | ] 173 | } 174 | ], 175 | "metadata": { 176 | "kernelspec": { 177 | "display_name": "Python 3.7.9 64-bit ('azureml': conda)", 178 | "name": "python379jvsc74a57bd054b76a1167e0a2b6a6b8c7f2df323eb2ecfae9d2bbefe58fb0609bf9141d6860" 179 | }, 180 | "language_info": { 181 | "name": "python", 182 | "version": "" 183 | } 184 | }, 185 | "nbformat": 4, 186 | "nbformat_minor": 4 187 | } -------------------------------------------------------------------------------- /models/german-credit-basic/notebooks/german-credit-local.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "## Connect to AML workspace & access data" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": null, 13 | "metadata": {}, 14 | "outputs": [], 15 | "source": [ 16 | "from azureml.core import Workspace, Dataset\n", 17 | "\n", 18 | "ws = Workspace.from_config()" 19 | ] 20 | }, 21 | { 22 | "cell_type": "code", 23 | "execution_count": null, 24 | "metadata": {}, 25 | "outputs": [], 26 | "source": [ 27 | "print(ws.name, ws.resource_group, ws.location, ws.subscription_id, sep = '\\n')" 28 | ] 29 | }, 30 | { 31 | "cell_type": "code", 32 | "execution_count": null, 33 | "metadata": {}, 34 | "outputs": [], 35 | "source": [ 36 | "dataset = Dataset.get_by_name(ws, name='german_credit_dataset')\r\n", 37 | "df = dataset.to_pandas_dataframe()" 38 | ] 39 | }, 40 | { 41 | "cell_type": "code", 42 | "execution_count": null, 43 | "metadata": {}, 44 | "outputs": [], 45 | "source": [ 46 | "df.head()" 47 | ] 48 | }, 49 | { 50 | "cell_type": "code", 51 | "execution_count": null, 52 | "metadata": {}, 53 | "outputs": [], 54 | "source": [ 55 | "import matplotlib.pyplot as plt\r\n", 56 | "\r\n", 57 | "df.plot(kind='scatter', x='Age', y='Credit amount')\r\n", 58 | "plt.show()" 59 | ] 60 | }, 61 | { 62 | "cell_type": "markdown", 63 | "metadata": {}, 64 | "source": [ 65 | "## Train model" 66 | ] 67 | }, 68 | { 69 | "cell_type": "code", 70 | "execution_count": null, 71 | "metadata": {}, 72 | "outputs": [], 73 | "source": [ 74 | "from sklearn.compose import ColumnTransformer\r\n", 75 | "from sklearn.impute import SimpleImputer\r\n", 76 | "from sklearn.linear_model import LogisticRegression\r\n", 77 | "from sklearn.model_selection import train_test_split\r\n", 78 | "from sklearn.pipeline import Pipeline\r\n", 79 | "from sklearn.preprocessing import LabelEncoder, OneHotEncoder, StandardScaler\r\n", 80 | "\r\n", 81 | "df.drop(\"Sno\", axis=1, inplace=True)\r\n", 82 | "\r\n", 83 | "y_raw = df['Risk']\r\n", 84 | "X_raw = df.drop('Risk', axis=1)\r\n", 85 | "\r\n", 86 | "categorical_features = X_raw.select_dtypes(include=['object']).columns\r\n", 87 | "numeric_features = X_raw.select_dtypes(include=['int64', 'float']).columns\r\n", 88 | "\r\n", 89 | "categorical_transformer = Pipeline(steps=[\r\n", 90 | " ('imputer', SimpleImputer(strategy='constant', fill_value=\"missing\")),\r\n", 91 | " ('onehotencoder', OneHotEncoder(categories='auto', sparse=False))])\r\n", 92 | "\r\n", 93 | "numeric_transformer = Pipeline(steps=[\r\n", 94 | " ('scaler', StandardScaler())])\r\n", 95 | "\r\n", 96 | "feature_engineering_pipeline = ColumnTransformer(\r\n", 97 | " transformers=[\r\n", 98 | " ('numeric', numeric_transformer, numeric_features),\r\n", 99 | " ('categorical', categorical_transformer, categorical_features)\r\n", 100 | " ], remainder=\"drop\")\r\n", 101 | "\r\n", 102 | "# Encode Labels\r\n", 103 | "le = LabelEncoder()\r\n", 104 | "encoded_y = le.fit_transform(y_raw)" 105 | ] 106 | }, 107 | { 108 | "cell_type": "code", 109 | "execution_count": null, 110 | "metadata": {}, 111 | "outputs": [], 112 | "source": [ 113 | "# Train test split\n", 114 | "X_train, X_test, y_train, y_test = train_test_split(X_raw, encoded_y, test_size=0.20, stratify=encoded_y, random_state=42)\n", 115 | "\n", 116 | "# Create sklearn pipeline\n", 117 | "lr_clf = Pipeline(steps=[('preprocessor', feature_engineering_pipeline),\n", 118 | " ('classifier', LogisticRegression(solver=\"lbfgs\"))])" 119 | ] 120 | }, 121 | { 122 | "cell_type": "code", 123 | "execution_count": null, 124 | "metadata": {}, 125 | "outputs": [], 126 | "source": [ 127 | "from azureml.core import Experiment\r\n", 128 | "\r\n", 129 | "experiment_name = 'german_credit_data_local'\r\n", 130 | "experiment = Experiment(ws, experiment_name)\r\n", 131 | "\r\n", 132 | "run = experiment.start_logging()\r\n", 133 | "\r\n", 134 | "# Train the model\r\n", 135 | "lr_clf.fit(X_train, y_train)\r\n", 136 | "\r\n", 137 | "# Capture metrics\r\n", 138 | "train_acc = lr_clf.score(X_train, y_train)\r\n", 139 | "test_acc = lr_clf.score(X_test, y_test)\r\n", 140 | "print(\"Train accuracy: %.3f\" % train_acc)\r\n", 141 | "print(\"Test accuracy: %.3f\" % test_acc)\r\n", 142 | "\r\n", 143 | "# Log to Azure ML\r\n", 144 | "run.log('Train accuracy', train_acc)\r\n", 145 | "run.log('Test accuracy', test_acc)\r\n", 146 | " \r\n", 147 | "run.complete()" 148 | ] 149 | }, 150 | { 151 | "cell_type": "code", 152 | "execution_count": null, 153 | "metadata": {}, 154 | "outputs": [], 155 | "source": [ 156 | "from sklearn.externals import joblib\n", 157 | "\n", 158 | "joblib.dump(value=lr_clf, filename='model.pkl')\n", 159 | "\n", 160 | "# Upload our model to our experiment\n", 161 | "run.upload_file(name = 'outputs/model.pkl', path_or_stream = './model.pkl')" 162 | ] 163 | }, 164 | { 165 | "cell_type": "markdown", 166 | "metadata": {}, 167 | "source": [ 168 | "Finally, we can register the model in the Model Registry, optionally also with no-code deployment settings" 169 | ] 170 | }, 171 | { 172 | "cell_type": "code", 173 | "execution_count": null, 174 | "metadata": {}, 175 | "outputs": [], 176 | "source": [ 177 | "from azureml.core import Model\r\n", 178 | "from azureml.core.resource_configuration import ResourceConfiguration\r\n", 179 | "\r\n", 180 | "model = run.register_model(model_name='german-credit-local-model',\r\n", 181 | " model_path='outputs/model.pkl',\r\n", 182 | " datasets=[['training-dataset', dataset]],\r\n", 183 | " tags={\"source\": \"local_training_demo\"},\r\n", 184 | " # optional for no-code deployments\r\n", 185 | " model_framework=Model.Framework.SCIKITLEARN,\r\n", 186 | " model_framework_version='0.20.3',\r\n", 187 | " resource_configuration=ResourceConfiguration(cpu=0.5, memory_in_gb=0.25))" 188 | ] 189 | }, 190 | { 191 | "cell_type": "markdown", 192 | "metadata": {}, 193 | "source": [ 194 | "## Calculate Model Explaination" 195 | ] 196 | }, 197 | { 198 | "cell_type": "code", 199 | "execution_count": null, 200 | "metadata": {}, 201 | "outputs": [], 202 | "source": [ 203 | "# Explain model\n", 204 | "from azureml.core.run import Run\n", 205 | "from azureml.interpret import ExplanationClient\n", 206 | "from interpret.ext.blackbox import TabularExplainer\n", 207 | "\n", 208 | "client = ExplanationClient.from_run(run)\n", 209 | "\n", 210 | "explainer = TabularExplainer(lr_clf.steps[-1][1], \n", 211 | " initialization_examples=X_train, \n", 212 | " features=X_raw.columns, \n", 213 | " classes=[\"Good\", \"Bad\"], \n", 214 | " transformations=feature_engineering_pipeline)\n", 215 | "\n", 216 | "# explain overall model predictions (global explanation)\n", 217 | "global_explanation = explainer.explain_global(X_test)\n", 218 | "\n", 219 | "# Sorted SHAP values\n", 220 | "print('ranked global importance values: {}'.format(global_explanation.get_ranked_global_values()))\n", 221 | "# Corresponding feature names\n", 222 | "print('ranked global importance names: {}'.format(global_explanation.get_ranked_global_names()))\n", 223 | "# Feature ranks (based on original order of features)\n", 224 | "print('global importance rank: {}'.format(global_explanation.global_importance_rank))\n", 225 | "\n", 226 | "client = ExplanationClient.from_run(run)\n", 227 | "client.upload_model_explanation(global_explanation, comment='Global Explanation: All Features')\n" 228 | ] 229 | }, 230 | { 231 | "cell_type": "code", 232 | "execution_count": null, 233 | "metadata": {}, 234 | "outputs": [], 235 | "source": [] 236 | } 237 | ], 238 | "metadata": { 239 | "kernelspec": { 240 | "display_name": "Python 3.7.9 64-bit ('azureml': conda)", 241 | "name": "python379jvsc74a57bd054b76a1167e0a2b6a6b8c7f2df323eb2ecfae9d2bbefe58fb0609bf9141d6860" 242 | }, 243 | "language_info": { 244 | "name": "python", 245 | "version": "" 246 | } 247 | }, 248 | "nbformat": 4, 249 | "nbformat_minor": 4 250 | } -------------------------------------------------------------------------------- /models/german-credit-basic/notebooks/german-credit-training-pipeline.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Connect to workspace" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": null, 13 | "metadata": { 14 | "tags": [] 15 | }, 16 | "outputs": [], 17 | "source": [ 18 | "import os\r\n", 19 | "import azureml.core\r\n", 20 | "from azureml.core import Workspace, Experiment, Dataset, RunConfiguration\r\n", 21 | "from azureml.pipeline.core import Pipeline, PipelineData\r\n", 22 | "from azureml.pipeline.steps import PythonScriptStep\r\n", 23 | "from azureml.data.dataset_consumption_config import DatasetConsumptionConfig\r\n", 24 | "\r\n", 25 | "print(\"Azure ML SDK version:\", azureml.core.VERSION)\r\n", 26 | "\r\n", 27 | "ws = Workspace.from_config()\r\n", 28 | "print(ws)" 29 | ] 30 | }, 31 | { 32 | "cell_type": "markdown", 33 | "metadata": {}, 34 | "source": [ 35 | "# Create Pipeline" 36 | ] 37 | }, 38 | { 39 | "cell_type": "code", 40 | "execution_count": null, 41 | "metadata": { 42 | "tags": [] 43 | }, 44 | "outputs": [], 45 | "source": [ 46 | "dataset = Dataset.get_by_name(ws, \"german_credit_file\")\r\n", 47 | "dataset_consumption = DatasetConsumptionConfig(\"training_dataset\", dataset).as_download()" 48 | ] 49 | }, 50 | { 51 | "cell_type": "code", 52 | "execution_count": null, 53 | "metadata": {}, 54 | "outputs": [], 55 | "source": [ 56 | "from azureml.core import Environment\r\n", 57 | "from azureml.core.conda_dependencies import CondaDependencies\r\n", 58 | "\r\n", 59 | "runconfig = RunConfiguration()\r\n", 60 | "runconfig.target = 'cpu-cluster'\r\n", 61 | "\r\n", 62 | "train_env = Environment(\"train-env\")\r\n", 63 | "train_env.docker.enabled = True\r\n", 64 | "train_env.python.conda_dependencies = CondaDependencies(\"../config/train-conda.yml\")\r\n", 65 | "\r\n", 66 | "runconfig.environment = train_env\r\n", 67 | "\r\n", 68 | "\r\n", 69 | "train_step = PythonScriptStep(name=\"train-step\",\r\n", 70 | " source_directory=\"../\",\r\n", 71 | " script_name=\"train.py\",\r\n", 72 | " arguments=['--data_path', dataset_consumption, '--model_name', 'credit-prediction.pkl'],\r\n", 73 | " inputs=[dataset_consumption],\r\n", 74 | " runconfig=runconfig,\r\n", 75 | " allow_reuse=False)\r\n", 76 | "\r\n", 77 | "steps = [train_step]\r\n", 78 | "\r\n", 79 | "pipeline = Pipeline(workspace=ws, steps=steps)\r\n", 80 | "pipeline.validate()" 81 | ] 82 | }, 83 | { 84 | "cell_type": "code", 85 | "execution_count": null, 86 | "metadata": {}, 87 | "outputs": [], 88 | "source": [ 89 | "pipeline_run = Experiment(ws, 'german-credit-training').submit(pipeline)\r\n", 90 | "pipeline_run.wait_for_completion()" 91 | ] 92 | }, 93 | { 94 | "cell_type": "markdown", 95 | "metadata": {}, 96 | "source": [ 97 | "Publish pipelines to AML Workspace for reuse:" 98 | ] 99 | }, 100 | { 101 | "cell_type": "code", 102 | "execution_count": null, 103 | "metadata": {}, 104 | "outputs": [], 105 | "source": [ 106 | "published_pipeline = pipeline.publish('german-credit-training')\r\n", 107 | "published_pipeline\r\n" 108 | ] 109 | } 110 | ], 111 | "metadata": { 112 | "kernelspec": { 113 | "display_name": "Python 3.7.9 64-bit ('azureml': conda)", 114 | "metadata": { 115 | "interpreter": { 116 | "hash": "54b76a1167e0a2b6a6b8c7f2df323eb2ecfae9d2bbefe58fb0609bf9141d6860" 117 | } 118 | }, 119 | "name": "python3" 120 | }, 121 | "language_info": { 122 | "codemirror_mode": { 123 | "name": "ipython", 124 | "version": 3 125 | }, 126 | "file_extension": ".py", 127 | "mimetype": "text/x-python", 128 | "name": "python", 129 | "nbconvert_exporter": "python", 130 | "pygments_lexer": "ipython3", 131 | "version": "3.7.9" 132 | } 133 | }, 134 | "nbformat": 4, 135 | "nbformat_minor": 4 136 | } -------------------------------------------------------------------------------- /models/german-credit-basic/notebooks/test_webservices.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "cells": [ 3 | { 4 | "cell_type": "markdown", 5 | "metadata": {}, 6 | "source": [ 7 | "# Test AKS webservice" 8 | ] 9 | }, 10 | { 11 | "cell_type": "code", 12 | "execution_count": null, 13 | "metadata": {}, 14 | "outputs": [], 15 | "source": [ 16 | "import requests\r\n", 17 | "import json\r\n", 18 | "\r\n", 19 | "url = 'https://:443/api/v1/service/german-credit-prod-1/score'\r\n", 20 | "key = ''\r\n", 21 | "\r\n", 22 | "test_data = {\r\n", 23 | " 'data': [{\r\n", 24 | " \"Age\": 20,\r\n", 25 | " \"Sex\": \"male\",\r\n", 26 | " \"Job\": 0,\r\n", 27 | " \"Housing\": \"own\",\r\n", 28 | " \"Saving accounts\": \"little\",\r\n", 29 | " \"Checking account\": \"little\",\r\n", 30 | " \"Credit amount\": 100,\r\n", 31 | " \"Duration\": 48,\r\n", 32 | " \"Purpose\": \"radio/TV\"\r\n", 33 | " }]\r\n", 34 | "}\r\n", 35 | "\r\n", 36 | "headers = {'Content-Type': 'application/json', 'Authorization': 'Bearer ' + key}\r\n", 37 | "resp = requests.post(url, json=test_data, headers=headers)\r\n", 38 | "\r\n", 39 | "print(\"Prediction (good, bad):\", resp.text)" 40 | ] 41 | }, 42 | { 43 | "cell_type": "markdown", 44 | "metadata": {}, 45 | "source": [ 46 | "# Continuously create load on the webservice" 47 | ] 48 | }, 49 | { 50 | "cell_type": "code", 51 | "execution_count": null, 52 | "metadata": {}, 53 | "outputs": [], 54 | "source": [ 55 | "import requests\r\n", 56 | "import json\r\n", 57 | "import random\r\n", 58 | "import time\r\n", 59 | "\r\n", 60 | "url = 'https://:443/api/v1/service/german-credit-prod-1/score'\r\n", 61 | "key = ''\r\n", 62 | "\r\n", 63 | "for i in range(1, 1000):\r\n", 64 | "\r\n", 65 | " test_data = {\r\n", 66 | " 'data': [{\r\n", 67 | " \"Age\": random.randint(18, 80),\r\n", 68 | " \"Sex\": random.choice([\"male\", \"female\"]),\r\n", 69 | " \"Job\": random.randint(0, 3),\r\n", 70 | " \"Housing\": random.choice([\"own\", \"rent\"]),\r\n", 71 | " \"Saving accounts\": random.choice([\"little\", \"moderate\", \"rich\"]),\r\n", 72 | " \"Checking account\": random.choice([\"little\", \"moderate\", \"rich\"]),\r\n", 73 | " \"Credit amount\": random.randint(100, 100000),\r\n", 74 | " \"Duration\": random.choice([6, 12, 24, 36, 48]),\r\n", 75 | " \"Purpose\": random.choice([\"radio/TV\", \"car\", \"business\"])\r\n", 76 | " }]\r\n", 77 | " }\r\n", 78 | "\r\n", 79 | " headers = {'Content-Type': 'application/json', 'Authorization': 'Bearer ' + key}\r\n", 80 | " resp = requests.post(url, json=test_data, headers=headers)\r\n", 81 | "\r\n", 82 | " print(\"Prediction (good, bad):\", resp.text)\r\n", 83 | " time.sleep(1)" 84 | ] 85 | } 86 | ], 87 | "metadata": { 88 | "kernelspec": { 89 | "display_name": "Python 3.6 - AzureML", 90 | "language": "python", 91 | "name": "python3-azureml" 92 | }, 93 | "language_info": { 94 | "codemirror_mode": { 95 | "name": "ipython", 96 | "version": 3 97 | }, 98 | "file_extension": ".py", 99 | "mimetype": "text/x-python", 100 | "name": "python", 101 | "nbconvert_exporter": "python", 102 | "pygments_lexer": "ipython3", 103 | "version": "3.6.9" 104 | } 105 | }, 106 | "nbformat": 4, 107 | "nbformat_minor": 4 108 | } -------------------------------------------------------------------------------- /models/german-credit-basic/pipelines/register.py: -------------------------------------------------------------------------------- 1 | import os 2 | import argparse 3 | from azureml.core import Run 4 | from azureml.pipeline.core import PipelineRun 5 | 6 | parser = argparse.ArgumentParser() 7 | parser.add_argument('--model_name', type=str, help='Name under which model will be registered') 8 | parser.add_argument('--model_path', type=str, help='Model directory') 9 | parser.add_argument('--repo', type=str, help='Model git repo') 10 | parser.add_argument('--branch', type=str, help='Model git branch') 11 | parser.add_argument('--commit', type=str, help='Model git commit') 12 | parser.add_argument('--build_id', type=str, help='DevOps build id') 13 | args, _ = parser.parse_known_args() 14 | 15 | print(f'Arguments: {args}') 16 | 17 | # current run is the registration step 18 | current_run = Run.get_context() 19 | ws = current_run.experiment.workspace 20 | 21 | # parent run is the overall pipeline 22 | parent_run = current_run.parent 23 | print(f'Parent run id: {parent_run.id}') 24 | 25 | pipeline_run = PipelineRun(parent_run.experiment, parent_run.id) 26 | 27 | training_run = pipeline_run.find_step_run('train-step')[0] 28 | print(f'Training run: {training_run}') 29 | 30 | # Retrieve input training datasets 31 | datasets = [] 32 | details = training_run.get_details() 33 | for d in details['inputDatasets']: 34 | print(f'Dataset info: {d}') 35 | name = d['consumptionDetails']['inputName'] 36 | dataset = d['dataset'] 37 | datasets.append([name, dataset]) 38 | 39 | # Register model 40 | model = training_run.register_model(model_name=args.model_name, 41 | model_path=args.model_path, 42 | datasets=datasets, 43 | properties={'repo': args.repo, 44 | 'branch': args.branch, 45 | 'commit': args.commit, 46 | 'build_id': args.build_id 47 | }) -------------------------------------------------------------------------------- /models/german-credit-basic/pipelines/training.py: -------------------------------------------------------------------------------- 1 | import os 2 | import azureml.core 3 | from azureml.core import Workspace, Dataset, RunConfiguration, Environment 4 | from azureml.pipeline.core import Pipeline, PipelineDraft, PipelineParameter 5 | from azureml.pipeline.steps import PythonScriptStep 6 | from azureml.data.dataset_consumption_config import DatasetConsumptionConfig 7 | 8 | print("Azure ML SDK version:", azureml.core.VERSION) 9 | 10 | print(f"git repo: {os.getenv('BUILD_REPOSITORY_URI')}") 11 | print(f"git branch: {os.getenv('BUILD_SOURCEBRANCH')}") 12 | print(f"git commit: {os.getenv('BUILD_SOURCEVERSION')}") 13 | print(f"DevOps build number: {os.getenv('BUILD_BUILDNUMBER')}") 14 | 15 | print('Connecting to workspace') 16 | ws = Workspace.from_config() 17 | print(f'WS name: {ws.name}\nRegion: {ws.location}\nSubscription id: {ws.subscription_id}\nResource group: {ws.resource_group}') 18 | 19 | print('Loading dataset') 20 | training_dataset = Dataset.get_by_name(ws, name='german_credit_file', version=1) 21 | 22 | # Parametrize dataset input to the pipeline 23 | training_dataset_parameter = PipelineParameter(name="training_dataset", default_value=training_dataset) 24 | training_dataset_consumption = DatasetConsumptionConfig("training_dataset", training_dataset_parameter).as_download() 25 | 26 | runconfig = RunConfiguration() 27 | runconfig.environment = Environment.from_conda_specification('training-pipeline-env', 'config/train-conda.yml') 28 | 29 | train_step = PythonScriptStep(name="train-step", 30 | runconfig=runconfig, 31 | source_directory='./', 32 | script_name='train.py', 33 | arguments=['--data_path', training_dataset_consumption, '--model_name', 'credit-prediction.pkl'], 34 | inputs=[training_dataset_consumption], 35 | compute_target='cpu-cluster', 36 | allow_reuse=False) 37 | 38 | register_step = PythonScriptStep(name="register-step", 39 | runconfig=runconfig, 40 | source_directory='pipelines/', 41 | script_name='register.py', 42 | arguments=['--model_name', 'credit-model-ci', '--model_path', 'outputs/credit-prediction.pkl', 43 | '--repo', os.getenv('BUILD_REPOSITORY_URI'), '--branch', os.getenv('BUILD_SOURCEBRANCH'), 44 | '--commit', os.getenv('BUILD_SOURCEVERSION'), '--build_id', os.getenv("BUILD_BUILDNUMBER")], 45 | compute_target='cpu-cluster', 46 | allow_reuse=False) 47 | 48 | register_step.run_after(train_step) 49 | steps = [train_step, register_step] 50 | 51 | print('Creating, validating and publishing pipeline') 52 | pipeline = Pipeline(workspace=ws, steps=steps) 53 | pipeline.validate() 54 | published_pipeline = pipeline.publish(name='credit-training-pipeline', 55 | description=f'Published pipeline from build id {os.getenv("BUILD_BUILDNUMBER")}') 56 | 57 | # Output pipeline_id in specified format which will convert it to a variable in Azure DevOps 58 | print(f'##vso[task.setvariable variable=pipeline_id]{published_pipeline.id}') -------------------------------------------------------------------------------- /models/german-credit-basic/score.py: -------------------------------------------------------------------------------- 1 | import json 2 | import os 3 | import numpy as np 4 | import pandas as pd 5 | import joblib 6 | from inference_schema.schema_decorators import input_schema, output_schema 7 | from inference_schema.parameter_types.standard_py_parameter_type import StandardPythonParameterType 8 | from azureml.monitoring import ModelDataCollector 9 | 10 | # Automatically generate the swagger interface by providing an data example 11 | input_sample = [{ 12 | "Age": 20, 13 | "Sex": "male", 14 | "Job": 0, 15 | "Housing": "own", 16 | "Saving accounts": "little", 17 | "Checking account": "little", 18 | "Credit amount": 100, 19 | "Duration": 48, 20 | "Purpose": "radio/TV" 21 | }] 22 | output_sample = [[0.7, 0.3]] 23 | 24 | def init(): 25 | # Load model 26 | global model 27 | model_dir = os.getenv('AZUREML_MODEL_DIR') 28 | model_path = os.path.join(model_dir, 'credit-prediction.pkl') 29 | model = joblib.load(model_path) 30 | 31 | # Setup Data Collection 32 | global inputs_dc 33 | global predictions_dc 34 | inputs_dc = ModelDataCollector("best_model", designation="inputs", feature_names=["Age", "Sex", "Job", "Housing", "Saving accounts", "Checking account", "Credit amount", "Duration", "Purpose"]) 35 | predictions_dc = ModelDataCollector("best_model", designation="predictions", feature_names=["good", "bad"]) 36 | 37 | @input_schema('data', StandardPythonParameterType(input_sample)) 38 | @output_schema(StandardPythonParameterType(output_sample)) 39 | def run(data): 40 | try: 41 | # Predict 42 | df = pd.DataFrame(data) 43 | proba = model.predict_proba(df) 44 | result = {"predict_proba": proba.tolist()} 45 | 46 | # Collect data 47 | correlations = inputs_dc.collect(df) 48 | predictions_data = predictions_dc.add_correlations(proba, correlations) 49 | predictions_dc.collect(predictions_data) 50 | 51 | return result 52 | except Exception as e: 53 | error = str(e) 54 | return error 55 | -------------------------------------------------------------------------------- /models/german-credit-basic/train.py: -------------------------------------------------------------------------------- 1 | import os 2 | import argparse 3 | import joblib 4 | import pandas as pd 5 | from azureml.core import Run, Dataset 6 | from sklearn.compose import ColumnTransformer 7 | from sklearn.impute import SimpleImputer 8 | from sklearn.linear_model import LogisticRegression 9 | from sklearn.model_selection import train_test_split 10 | from sklearn.pipeline import Pipeline 11 | from sklearn.preprocessing import LabelEncoder, OneHotEncoder, StandardScaler 12 | 13 | from interpret.ext.blackbox import TabularExplainer 14 | from azureml.interpret import ExplanationClient 15 | 16 | def get_runtime_args(): 17 | parser = argparse.ArgumentParser() 18 | parser.add_argument('--model_name', type=str) 19 | parser.add_argument('--data_path', type=str) 20 | args = parser.parse_args() 21 | return args 22 | 23 | def main(): 24 | args = get_runtime_args() 25 | 26 | df = pd.read_csv(os.path.join(args.data_path, 'german_credit_data.csv')) 27 | clf = model_train(df) 28 | 29 | #copying model to "outputs" directory, this will automatically upload it to Azure ML 30 | output_dir = './outputs/' 31 | os.makedirs(output_dir, exist_ok=True) 32 | joblib.dump(value=clf, filename=os.path.join(output_dir, args.model_name)) 33 | 34 | def model_train(df): 35 | run = Run.get_context() 36 | 37 | df.drop("Sno", axis=1, inplace=True) 38 | 39 | y_raw = df['Risk'] 40 | X_raw = df.drop('Risk', axis=1) 41 | 42 | categorical_features = X_raw.select_dtypes(include=['object']).columns 43 | numeric_features = X_raw.select_dtypes(include=['int64', 'float']).columns 44 | 45 | categorical_transformer = Pipeline(steps=[ 46 | ('imputer', SimpleImputer(strategy='constant', fill_value="missing")), 47 | ('onehotencoder', OneHotEncoder(categories='auto', sparse=False))]) 48 | 49 | numeric_transformer = Pipeline(steps=[ 50 | ('scaler', StandardScaler())]) 51 | 52 | feature_engineering_pipeline = ColumnTransformer( 53 | transformers=[ 54 | ('numeric', numeric_transformer, numeric_features), 55 | ('categorical', categorical_transformer, categorical_features) 56 | ], remainder="drop") 57 | 58 | # Encode Labels 59 | le = LabelEncoder() 60 | encoded_y = le.fit_transform(y_raw) 61 | 62 | # Train test split 63 | X_train, X_test, y_train, y_test = train_test_split(X_raw, encoded_y, test_size=0.20, stratify=encoded_y, random_state=42) 64 | 65 | # Create sklearn pipeline 66 | lr_clf = Pipeline(steps=[('preprocessor', feature_engineering_pipeline), 67 | ('classifier', LogisticRegression(solver="lbfgs"))]) 68 | # Train the model 69 | lr_clf.fit(X_train, y_train) 70 | 71 | # Capture metrics 72 | train_acc = lr_clf.score(X_train, y_train) 73 | test_acc = lr_clf.score(X_test, y_test) 74 | print("Training accuracy: %.3f" % train_acc) 75 | print("Testing accuracy: %.3f" % test_acc) 76 | 77 | # Log to Azure ML 78 | run.log('Train accuracy', train_acc) 79 | run.log('Test accuracy', test_acc) 80 | 81 | # Explain model 82 | explainer = TabularExplainer(lr_clf.steps[-1][1], 83 | initialization_examples=X_train, 84 | features=X_raw.columns, 85 | classes=["Good", "Bad"], 86 | transformations=feature_engineering_pipeline) 87 | 88 | # explain overall model predictions (global explanation) 89 | global_explanation = explainer.explain_global(X_test) 90 | 91 | # Sorted SHAP values 92 | print('ranked global importance values: {}'.format(global_explanation.get_ranked_global_values())) 93 | # Corresponding feature names 94 | print('ranked global importance names: {}'.format(global_explanation.get_ranked_global_names())) 95 | # Feature ranks (based on original order of features) 96 | print('global importance rank: {}'.format(global_explanation.global_importance_rank)) 97 | 98 | client = ExplanationClient.from_run(run) 99 | client.upload_model_explanation(global_explanation, comment='Global Explanation: All Features') 100 | 101 | return lr_clf 102 | 103 | if __name__ == "__main__": 104 | main() 105 | -------------------------------------------------------------------------------- /pipelines-approval/german-credit-config.yml: -------------------------------------------------------------------------------- 1 | variables: 2 | # Name of your Azure DevOps service connection to your AML workspace 3 | ml_workspace_connection: 'aml-workspace-dev' 4 | 5 | # Name of your AML workspace and its resource group 6 | ml_workspace_name: 'aml-fsi-demo-dev' 7 | ml_workspace_rg: 'aml-fsi-demo-dev' 8 | 9 | # Name of your Compute Cluster (needs to exist) 10 | ml_compute_name: 'cpu-cluster' 11 | 12 | # Name of your experiment for training tracking 13 | ml_experiment_name: 'german-credit-basic-ci' 14 | 15 | # Model name for registration and its path in the repo 16 | ml_model_name: 'german-credit-basic-model' 17 | ml_model_path: 'models/german-credit-basic' 18 | 19 | # Desired model output name 20 | ml_model_filename: 'credit-prediction.pkl' 21 | 22 | # AKS cluster to deploy to (needs to exist) 23 | ml_aks_cluster: 'aks-dev' 24 | 25 | # Model deployment name 26 | ml_deployment_name: 'german-credit-deployment' -------------------------------------------------------------------------------- /pipelines-approval/german-credit-deploy-model.yml: -------------------------------------------------------------------------------- 1 | variables: 2 | - template: german-credit-config.yml 3 | 4 | trigger: none 5 | 6 | pool: 7 | vmImage: ubuntu-20.04 8 | 9 | stages: 10 | - stage: DeployModel 11 | displayName: Deploy model 12 | jobs: 13 | - deployment: DeployModel 14 | displayName: Deploy Model to AKS 15 | environment: model-production 16 | variables: 17 | ml_model_name: credit-model-ci 18 | ml_model_version: 4 19 | strategy: 20 | runOnce: 21 | deploy: 22 | steps: 23 | - checkout: self 24 | # TODO: Checkout commit from which registered model was trained on 25 | - template: ../templates/install-aml-cli.yml 26 | - template: ../templates/connect-to-workspace.yml 27 | - template: ../templates/deploy-model-by-name.yml -------------------------------------------------------------------------------- /pipelines-approval/german-credit-full-approval-flow.yml: -------------------------------------------------------------------------------- 1 | variables: 2 | - template: german-credit-config.yml 3 | 4 | trigger: none 5 | 6 | pool: 7 | vmImage: ubuntu-20.04 8 | 9 | stages: 10 | - stage: TrainModel 11 | displayName: Train & register model 12 | jobs: 13 | - job: TrainModel 14 | steps: 15 | - template: ../templates/install-aml-cli.yml 16 | - template: ../templates/connect-to-workspace.yml 17 | - template: ../templates/train-model.yml 18 | - template: ../templates/register-model.yml 19 | - template: ../templates/publish-artifacts.yml 20 | 21 | - stage: DeployModel 22 | displayName: Deploy model 23 | jobs: 24 | - deployment: DeployModel 25 | displayName: Deploy Model to AKS 26 | environment: model-production 27 | variables: 28 | ml_model_path: $(Pipeline.Workspace)/model_drop 29 | strategy: 30 | runOnce: 31 | deploy: 32 | steps: 33 | - template: ../templates/download-artifacts.yml 34 | - template: ../templates/install-aml-cli.yml 35 | - template: ../templates/connect-to-workspace.yml 36 | - template: ../templates/deploy-model.yml -------------------------------------------------------------------------------- /pipelines-approval/german-credit-publish-pipeline.yml: -------------------------------------------------------------------------------- 1 | variables: 2 | - template: german-credit-config.yml 3 | 4 | trigger: none 5 | 6 | pool: 7 | vmImage: ubuntu-20.04 8 | 9 | stages: 10 | - stage: DeployTrainingPipeline 11 | displayName: Deploy model training pipeline 12 | jobs: 13 | - job: TrainModel 14 | steps: 15 | - template: ../templates/install-aml-cli.yml 16 | - template: ../templates/install-aml-sdk.yml 17 | - template: ../templates/connect-to-workspace.yml 18 | - template: ../templates/deploy-pipeline.yml -------------------------------------------------------------------------------- /pipelines-staged/README.md: -------------------------------------------------------------------------------- 1 | # Multi-stage, multi-enviroment Pipelines 2 | 3 | This folder contains a typical DEV-to-PROD release pipeline in [`german-credit-rollout.yml`](german-credit-rollout.yml). 4 | 5 | This enables the following flow, where a model is trained, tested and deployed in a development environment. Once that has been completed successfully, the same process is repeated in the production environment: 6 | 7 | ![alt text](../media/stages.png "Staged deployment") 8 | 9 | * Dev Stage 10 | * Train and register model (in Dev environment) 11 | * Deploy model (in Dev environment) 12 | * Prod Stage (after Dev Stage completed successfully) 13 | * Train and register model (in Prod environment) 14 | * Deploy model (in Prod environment) 15 | 16 | 17 | -------------------------------------------------------------------------------- /pipelines-staged/german-credit-config-dev.yml: -------------------------------------------------------------------------------- 1 | variables: 2 | # Name of your Azure DevOps service connection to your AML workspace 3 | ml_workspace_connection: 'aml-workspace-dev' 4 | 5 | # Name of your AML workspace and its resource group 6 | ml_workspace_name: 'aml-fsi-demo-dev' 7 | ml_workspace_rg: 'aml-fsi-demo-dev' 8 | 9 | # Name of your Compute Cluster (needs to exist) 10 | ml_compute_name: 'cpu-cluster' 11 | 12 | # Name of your experiment for training tracking 13 | ml_experiment_name: 'german-credit-basic-dev-ci' 14 | 15 | # Model name for registration and its path in the repo 16 | ml_model_name: 'german-credit-basic-model-dev' 17 | ml_model_path: 'models/german-credit-basic' 18 | 19 | # Desired model output name 20 | ml_model_filename: 'credit-prediction.pkl' 21 | 22 | # AKS cluster to deploy to (needs to exist) 23 | ml_aks_cluster: 'aks-dev' 24 | 25 | # Model deployment name 26 | ml_deployment_name: 'german-credit-deployment-dev' -------------------------------------------------------------------------------- /pipelines-staged/german-credit-config-prod.yml: -------------------------------------------------------------------------------- 1 | variables: 2 | # Name of your Azure DevOps service connection to your AML workspace 3 | ml_workspace_connection: 'aml-workspace-prod' 4 | 5 | # Name of your AML workspace and its resource group 6 | ml_workspace_name: 'aml-fsi-demo-prod' 7 | ml_workspace_rg: 'aml-fsi-demo-prod' 8 | 9 | # Name of your Compute Cluster (needs to exist) 10 | ml_compute_name: 'cpu-cluster' 11 | 12 | # Name of your experiment for training tracking 13 | ml_experiment_name: 'german-credit-basic-prod-ci' 14 | 15 | # Model name for registration and its path in the repo 16 | ml_model_name: 'german-credit-basic-model-prod' 17 | ml_model_path: 'models/german-credit-basic' 18 | 19 | # Desired model output name 20 | ml_model_filename: 'credit-prediction.pkl' 21 | 22 | # AKS cluster to deploy to (needs to exist) 23 | ml_aks_cluster: 'aks-prod' 24 | 25 | # Model deployment name 26 | ml_deployment_name: 'german-credit-deployment-prod' -------------------------------------------------------------------------------- /pipelines-staged/german-credit-rollout.yml: -------------------------------------------------------------------------------- 1 | trigger: none 2 | 3 | pool: 4 | vmImage: ubuntu-20.04 5 | 6 | stages: 7 | - stage: TrainDev 8 | displayName: Train (Dev Stage) 9 | jobs: 10 | - template: template-train-and-register.yml 11 | parameters: 12 | variablesFile: german-credit-config-dev.yml 13 | 14 | - stage: DeployDev 15 | displayName: Deploy (Dev Stage) 16 | jobs: 17 | - template: template-deploy.yml 18 | parameters: 19 | variablesFile: german-credit-config-dev.yml 20 | 21 | - stage: TrainProd 22 | displayName: Train (Prod Stage) 23 | jobs: 24 | - template: template-train-and-register.yml 25 | parameters: 26 | variablesFile: german-credit-config-prod.yml 27 | 28 | - stage: DeployProd 29 | displayName: Deploy (Prod Stage) 30 | jobs: 31 | - template: template-deploy.yml 32 | parameters: 33 | variablesFile: german-credit-config-prod.yml -------------------------------------------------------------------------------- /pipelines-staged/template-deploy.yml: -------------------------------------------------------------------------------- 1 | parameters: 2 | variablesFile: '' 3 | 4 | jobs: 5 | - job: DeployModel 6 | displayName: Deploy Model 7 | variables: 8 | - template: ${{ parameters.variablesFile }} 9 | steps: 10 | - task: AzureCLI@2 11 | displayName: 'Install AML CLI' 12 | inputs: 13 | azureSubscription: ${{ variables.ml_workspace_connection }} 14 | scriptLocation: inlineScript 15 | scriptType: bash 16 | inlineScript: | 17 | az extension add -n azure-cli-ml 18 | 19 | - task: AzureCLI@2 20 | displayName: 'Attach to workspace' 21 | inputs: 22 | azureSubscription: ${{ variables.ml_workspace_connection }} 23 | scriptLocation: inlineScript 24 | scriptType: bash 25 | workingDirectory: ${{ variables.ml_model_path }} 26 | inlineScript: | 27 | az ml folder attach -g ${{ variables.ml_workspace_rg }} -w ${{ variables.ml_workspace_name }} 28 | 29 | - task: AzureCLI@2 30 | displayName: 'Deploy model to AKS for Production' 31 | inputs: 32 | azureSubscription: ${{ variables.ml_workspace_connection }} 33 | scriptLocation: inlineScript 34 | scriptType: bash 35 | workingDirectory: ${{ variables.ml_model_path }} 36 | inlineScript: | 37 | LATEST_VERSION=`az ml model list -n ${{ variables.ml_model_name }} --query '[0].version'` 38 | az ml model deploy -n ${{ variables.ml_deployment_name }} \ 39 | --compute-target ${{ variables.ml_aks_cluster }} \ 40 | --model ${{ variables.ml_model_name }}:$LATEST_VERSION \ 41 | --inference-config-file config/inference-config.yml \ 42 | --deploy-config-file config/deployment-config-aks.yml \ 43 | --overwrite 44 | -------------------------------------------------------------------------------- /pipelines-staged/template-train-and-register.yml: -------------------------------------------------------------------------------- 1 | parameters: 2 | variablesFile: '' 3 | 4 | jobs: 5 | - job: TrainAndRegister 6 | displayName: Train and Register Model 7 | variables: 8 | - template: ${{ parameters.variablesFile }} 9 | steps: 10 | - task: AzureCLI@2 11 | displayName: 'Install AML CLI' 12 | inputs: 13 | azureSubscription: ${{ variables.ml_workspace_connection }} 14 | scriptType: bash 15 | scriptLocation: inlineScript 16 | inlineScript: | 17 | az extension add -n azure-cli-ml 18 | 19 | - task: AzureCLI@2 20 | displayName: 'Attach to workspace' 21 | inputs: 22 | azureSubscription: ${{ variables.ml_workspace_connection }} 23 | scriptType: bash 24 | scriptLocation: inlineScript 25 | workingDirectory: ${{ variables.ml_model_path }} 26 | inlineScript: | 27 | az ml folder attach -g ${{ variables.ml_workspace_rg }} -w ${{ variables.ml_workspace_name }} 28 | 29 | - task: AzureCLI@2 30 | displayName: 'Train and explain model' 31 | inputs: 32 | azureSubscription: ${{ variables.ml_workspace_connection }} 33 | scriptType: bash 34 | scriptLocation: inlineScript 35 | workingDirectory: ${{ variables.ml_model_path }} 36 | inlineScript: | 37 | az ml run submit-script \ 38 | --run-configuration-name config/train-amlcompute \ 39 | --target ${{ variables.ml_compute_name }} \ 40 | --experiment-name ${{ variables.ml_experiment_name }} \ 41 | --output-metadata-file $(Build.ArtifactStagingDirectory)/run.json 42 | 43 | - task: AzureCLI@2 44 | displayName: 'Register model' 45 | inputs: 46 | azureSubscription: ${{ variables.ml_workspace_connection }} 47 | scriptType: bash 48 | scriptLocation: inlineScript 49 | workingDirectory: ${{ variables.ml_model_path }} 50 | inlineScript: | 51 | az ml model register \ 52 | --name ${{ variables.ml_model_name }} \ 53 | --asset-path outputs/${{ variables.ml_model_filename }} \ 54 | --run-metadata-file $(Build.ArtifactStagingDirectory)/run.json \ 55 | --output-metadata-file $(Build.ArtifactStagingDirectory)/model.json \ 56 | --property git-repo-url=$(Build.Repository.Uri) \ 57 | --property git-commit-id=$(Build.SourceVersion) 58 | -------------------------------------------------------------------------------- /pipelines/german-credit-config.yml: -------------------------------------------------------------------------------- 1 | variables: 2 | # Name of your Azure DevOps service connection to your AML workspace 3 | ml_workspace_connection: 'aml-workspace-dev' 4 | 5 | # Name of your AML workspace and its resource group 6 | ml_workspace_name: 'aml-fsi-demo-dev' 7 | ml_workspace_rg: 'aml-fsi-demo-dev' 8 | 9 | # Name of your Compute Cluster (needs to exist) 10 | ml_compute_name: 'cpu-cluster' 11 | 12 | # Name of your experiment for training tracking 13 | ml_experiment_name: 'german-credit-basic-ci' 14 | 15 | # Model name for registration and its path in the repo 16 | ml_model_name: 'german-credit-basic-model' 17 | ml_model_path: 'models/german-credit-basic' 18 | 19 | # Desired model output name 20 | ml_model_filename: 'credit-prediction.pkl' 21 | 22 | # AKS cluster to deploy to (needs to exist) 23 | ml_aks_cluster: 'aks-dev' 24 | 25 | # Model deployment name 26 | ml_deployment_name: 'german-credit-deployment' -------------------------------------------------------------------------------- /pipelines/german-credit-deploy.yml: -------------------------------------------------------------------------------- 1 | variables: 2 | - template: german-credit-config.yml 3 | 4 | trigger: none 5 | 6 | pool: 7 | vmImage: ubuntu-20.04 8 | 9 | steps: 10 | - task: AzureCLI@2 11 | displayName: 'Install AML CLI' 12 | inputs: 13 | azureSubscription: $(ml_workspace_connection) 14 | scriptType: bash 15 | scriptLocation: inlineScript 16 | inlineScript: | 17 | az extension add -n azure-cli-ml 18 | 19 | - task: AzureCLI@2 20 | displayName: 'Attach to workspace' 21 | inputs: 22 | azureSubscription: $(ml_workspace_connection) 23 | workingDirectory: $(ml_model_path) 24 | scriptType: bash 25 | scriptLocation: inlineScript 26 | inlineScript: | 27 | az ml folder attach -g $(ml_workspace_rg) -w $(ml_workspace_name) 28 | 29 | - task: AzureCLI@2 30 | displayName: 'Deploy model to AKS' 31 | inputs: 32 | azureSubscription: $(ml_workspace_connection) 33 | workingDirectory: $(ml_model_path) 34 | scriptLocation: inlineScript 35 | scriptType: bash 36 | inlineScript: | 37 | LATEST_VERSION=`az ml model list -n $(ml_model_name) --query '[0].version'` 38 | az ml model deploy -n $(ml_deployment_name) \ 39 | --compute-target $(ml_aks_cluster) \ 40 | --model $(ml_model_name):$LATEST_VERSION \ 41 | --inference-config-file config/inference-config.yml \ 42 | --deploy-config-file config/deployment-config-aks.yml \ 43 | --overwrite 44 | -------------------------------------------------------------------------------- /pipelines/german-credit-train-and-register.yml: -------------------------------------------------------------------------------- 1 | variables: 2 | - template: german-credit-config.yml 3 | 4 | trigger: none 5 | 6 | pool: 7 | vmImage: ubuntu-20.04 8 | 9 | steps: 10 | - task: AzureCLI@2 11 | displayName: 'Install AML CLI' 12 | inputs: 13 | azureSubscription: $(ml_workspace_connection) 14 | scriptType: bash 15 | scriptLocation: inlineScript 16 | inlineScript: | 17 | az extension add -n azure-cli-ml 18 | 19 | - task: AzureCLI@2 20 | displayName: 'Attach to workspace' 21 | inputs: 22 | azureSubscription: $(ml_workspace_connection) 23 | scriptType: bash 24 | scriptLocation: inlineScript 25 | workingDirectory: $(ml_model_path) 26 | inlineScript: | 27 | az ml folder attach -g $(ml_workspace_rg) -w $(ml_workspace_name) 28 | 29 | - task: AzureCLI@2 30 | displayName: 'Train and explain model' 31 | inputs: 32 | azureSubscription: $(ml_workspace_connection) 33 | scriptType: bash 34 | scriptLocation: inlineScript 35 | workingDirectory: $(ml_model_path) 36 | inlineScript: | 37 | az ml run submit-script \ 38 | --run-configuration-name config/train-amlcompute \ 39 | --target $(ml_compute_name) \ 40 | --experiment-name $(ml_experiment_name) \ 41 | --output-metadata-file $(Build.ArtifactStagingDirectory)/run.json 42 | 43 | - task: AzureCLI@2 44 | displayName: 'Register model' 45 | inputs: 46 | azureSubscription: $(ml_workspace_connection) 47 | scriptType: bash 48 | scriptLocation: inlineScript 49 | workingDirectory: $(ml_model_path) 50 | inlineScript: | 51 | az ml model register \ 52 | --name $(ml_model_name) \ 53 | --asset-path outputs/$(ml_model_filename) \ 54 | --run-metadata-file $(Build.ArtifactStagingDirectory)/run.json \ 55 | --output-metadata-file $(Build.ArtifactStagingDirectory)/model.json \ 56 | --property git-repo-url=$(Build.Repository.Uri) \ 57 | --property git-commit-id=$(Build.SourceVersion) 58 | -------------------------------------------------------------------------------- /templates/connect-to-workspace.yml: -------------------------------------------------------------------------------- 1 | steps: 2 | - task: AzureCLI@2 3 | displayName: 'Connect to workspace' 4 | inputs: 5 | azureSubscription: $(ml_workspace_connection) 6 | scriptType: bash 7 | scriptLocation: inlineScript 8 | workingDirectory: $(ml_model_path) 9 | inlineScript: | 10 | az ml folder attach -g $(ml_workspace_rg) -w $(ml_workspace_name) -------------------------------------------------------------------------------- /templates/deploy-model-by-name.yml: -------------------------------------------------------------------------------- 1 | steps: 2 | - task: AzureCLI@2 3 | displayName: 'Deploy model to AKS' 4 | inputs: 5 | azureSubscription: $(ml_workspace_connection) 6 | workingDirectory: $(ml_model_path) 7 | scriptLocation: inlineScript 8 | scriptType: bash 9 | inlineScript: | 10 | ls -al 11 | az ml model deploy -n $(ml_deployment_name) \ 12 | --model $(ml_model_name):$(ml_model_version) \ 13 | --compute-target $(ml_aks_cluster) \ 14 | --inference-config-file config/inference-config.yml \ 15 | --deploy-config-file config/deployment-config-aks.yml \ 16 | --overwrite -------------------------------------------------------------------------------- /templates/deploy-model.yml: -------------------------------------------------------------------------------- 1 | steps: 2 | - task: AzureCLI@2 3 | displayName: 'Deploy model to AKS' 4 | inputs: 5 | azureSubscription: $(ml_workspace_connection) 6 | workingDirectory: $(ml_model_path) 7 | scriptLocation: inlineScript 8 | scriptType: bash 9 | inlineScript: | 10 | ls -al 11 | az ml model deploy -n $(ml_deployment_name) \ 12 | --model-metadata-file model.json \ 13 | --compute-target $(ml_aks_cluster) \ 14 | --inference-config-file config/inference-config.yml \ 15 | --deploy-config-file config/deployment-config-aks.yml \ 16 | --overwrite -------------------------------------------------------------------------------- /templates/deploy-pipeline.yml: -------------------------------------------------------------------------------- 1 | steps: 2 | - task: AzureCLI@2 3 | displayName: 'Deploy AML pipeline' 4 | inputs: 5 | azureSubscription: $(ml_workspace_connection) 6 | scriptType: bash 7 | scriptLocation: inlineScript 8 | workingDirectory: $(ml_model_path) 9 | inlineScript: | 10 | set -e 11 | python pipelines/training.py -------------------------------------------------------------------------------- /templates/download-artifacts.yml: -------------------------------------------------------------------------------- 1 | steps: 2 | - checkout: none 3 | - download: current 4 | artifact: model_drop -------------------------------------------------------------------------------- /templates/install-aml-cli.yml: -------------------------------------------------------------------------------- 1 | steps: 2 | - task: AzureCLI@2 3 | displayName: 'Install AML CLI' 4 | inputs: 5 | azureSubscription: $(ml_workspace_connection) 6 | scriptType: bash 7 | scriptLocation: inlineScript 8 | inlineScript: | 9 | az extension add -n azure-cli-ml -------------------------------------------------------------------------------- /templates/install-aml-sdk.yml: -------------------------------------------------------------------------------- 1 | 2 | steps: 3 | - task: UsePythonVersion@0 4 | displayName: 'Setting Python version' 5 | inputs: 6 | versionSpec: '3.6' 7 | - task: AzureCLI@2 8 | displayName: 'Install AML SDK' 9 | inputs: 10 | azureSubscription: $(ml_workspace_connection) 11 | scriptType: bash 12 | scriptLocation: inlineScript 13 | inlineScript: | 14 | python --version 15 | pip install --upgrade azureml-sdk 16 | pip install azure-cli -------------------------------------------------------------------------------- /templates/publish-artifacts.yml: -------------------------------------------------------------------------------- 1 | steps: 2 | - task: CopyFiles@2 3 | displayName: Copy model source & metadata 4 | inputs: 5 | SourceFolder: $(ml_model_path) 6 | Contents: | 7 | **/* 8 | !.azureml/**/* 9 | TargetFolder: $(Build.ArtifactStagingDirectory) 10 | - publish: $(Build.ArtifactStagingDirectory) 11 | displayName: Publish model source & metadata 12 | artifact: model_drop -------------------------------------------------------------------------------- /templates/register-model.yml: -------------------------------------------------------------------------------- 1 | steps: 2 | - task: AzureCLI@2 3 | displayName: 'Register model' 4 | inputs: 5 | azureSubscription: $(ml_workspace_connection) 6 | scriptType: bash 7 | scriptLocation: inlineScript 8 | workingDirectory: $(ml_model_path) 9 | inlineScript: | 10 | az ml model register \ 11 | --name $(ml_model_name) \ 12 | --asset-path outputs/$(ml_model_filename) \ 13 | --run-metadata-file run.json \ 14 | --output-metadata-file model.json \ 15 | --property git-repo-url=$(Build.Repository.Uri) \ 16 | --property git-commit-id=$(Build.SourceVersion) -------------------------------------------------------------------------------- /templates/train-model.yml: -------------------------------------------------------------------------------- 1 | steps: 2 | - task: AzureCLI@2 3 | displayName: 'Train and explain model' 4 | inputs: 5 | azureSubscription: $(ml_workspace_connection) 6 | scriptType: bash 7 | scriptLocation: inlineScript 8 | workingDirectory: $(ml_model_path) 9 | inlineScript: | 10 | az ml run submit-script \ 11 | --run-configuration-name config/train-amlcompute \ 12 | --target $(ml_compute_name) \ 13 | --experiment-name $(ml_experiment_name) \ 14 | --output-metadata-file run.json --------------------------------------------------------------------------------