├── LICENSE.md ├── README.md ├── SECURITY.md ├── img ├── Azure_Solution_Scale_Set.png ├── Deployment_Complete_Select_Storage.png └── Resource_Group_Delete.png └── releases ├── R2019a_and_older ├── README.md ├── azuredeploy-R2018a.json ├── azuredeploy-R2018b.json ├── azuredeploy-R2019a.json ├── azuredeploy-existing-vnet-R2018a.json ├── azuredeploy-existing-vnet-R2018b.json ├── azuredeploy-existing-vnet-R2019a.json ├── azuredeploy-test-R2018a.json ├── azuredeploy-test-R2018b.json ├── azuredeploy-test-R2019a.json ├── azuredeploy-with-license-manager-R2018a.json ├── azuredeploy-with-license-manager-R2018b.json ├── azuredeploy-with-license-manager-R2019a.json ├── license-manager-instructions.md ├── online-licensing-instructions.md ├── parameters-existing-vnet.json ├── parameters-with-license-manager.json ├── parameters.json └── scripts │ ├── createFileShare.ps1 │ ├── listFileShares.ps1 │ └── startmjs.ps1 ├── R2019b ├── README.md ├── azuredeploy-R2019b.json ├── azuredeploy-existing-vnet-R2019b.json ├── azuredeploy-existing-vnet-test-R2019b.json └── azuredeploy-test-R2019b.json ├── R2020a ├── README.md ├── azuredeploy-R2020a-test.json ├── azuredeploy-R2020a.json ├── azuredeploy-existing-vnet-R2020a-test.json └── azuredeploy-existing-vnet-R2020a.json ├── R2020b ├── README.md ├── azuredeploy-R2020b-test.json ├── azuredeploy-R2020b.json ├── azuredeploy-existing-vnet-R2020b-test.json └── azuredeploy-existing-vnet-R2020b.json ├── R2021a ├── README.md ├── azuredeploy-R2021a-test.json ├── azuredeploy-R2021a.json ├── azuredeploy-existing-vnet-R2021a-test.json └── azuredeploy-existing-vnet-R2021a.json ├── R2021b ├── README.md ├── azuredeploy-R2021b-test.json ├── azuredeploy-R2021b.json ├── azuredeploy-existing-vnet-R2021b-test.json └── azuredeploy-existing-vnet-R2021b.json ├── R2022a ├── README.md ├── azuredeploy-R2022a-test.json ├── azuredeploy-R2022a.json ├── azuredeploy-existing-vnet-R2022a-test.json └── azuredeploy-existing-vnet-R2022a.json ├── R2022b ├── README.md ├── azuredeploy-R2022b-test.json ├── azuredeploy-R2022b.json ├── azuredeploy-existing-vnet-R2022b-test.json └── azuredeploy-existing-vnet-R2022b.json ├── R2023a ├── README.md ├── azuredeploy-R2023a-test.json ├── azuredeploy-R2023a.json ├── azuredeploy-existing-vnet-R2023a-test.json └── azuredeploy-existing-vnet-R2023a.json ├── R2023b ├── README.md ├── azuredeploy-R2023b-test.json ├── azuredeploy-R2023b.json ├── azuredeploy-existing-vnet-R2023b-test.json └── azuredeploy-existing-vnet-R2023b.json ├── R2024a ├── README.md ├── azuredeploy-R2024a-test.json ├── azuredeploy-R2024a.json ├── azuredeploy-existing-vnet-R2024a-test.json └── azuredeploy-existing-vnet-R2024a.json ├── R2024b ├── README.md ├── azuredeploy-R2024b-test.json ├── azuredeploy-R2024b.json ├── azuredeploy-existing-vnet-R2024b-test.json └── azuredeploy-existing-vnet-R2024b.json └── R2025a ├── README.md ├── azuredeploy-R2025a.json └── azuredeploy-existing-vnet-R2025a.json /LICENSE.md: -------------------------------------------------------------------------------- 1 | MATHWORKS CLOUD REFERENCE ARCHITECTURE LICENSE 2 | 3 | The files in this GitHub repository refer to commercial software products and services, virtual machine images, and related materials of The MathWorks, Inc. (“MathWorks Programs”). MathWorks Programs are separately licensed under the MathWorks Software License Agreement, available in the desktop installation of the MathWorks Programs or in the virtual machine image. The files in this GitHub repository may also refer to third-party software licensed under separate terms provided by such third parties. 4 | 5 | The following license terms apply only to the files in this GitHub repository, including files in this folder and its subfolders, and do not apply to MathWorks Programs. References to “software” and “code” in the following license terms refer to the files in this GitHub repository. 6 | 7 | Copyright (c) 2025, The MathWorks, Inc. 8 | 9 | All rights reserved. 10 | 11 | Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 12 | 13 | 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 14 | 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. 15 | 3. In all cases, the software is, and all modifications and derivatives of the software shall be, licensed to you solely for use in conjunction with MathWorks products and service offerings. 16 | 17 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # MATLAB Parallel Server on Microsoft Azure 2 | 3 | This repository helps you automate the process of deploying MATLAB® Parallel Server™ and MATLAB Job Scheduler using your Azure® account. 4 | 5 | Use this repository to deploy a compute cluster using compute, storage, and network resources hosted by Azure. The cloud resources are created using Azure Resource Manager (ARM) templates. For information about the architecture of this solution, see [Learn About Cluster Architecture](#learn-about-cluster-architecture). 6 | 7 | # Requirements 8 | 9 | Before starting, you need the following: 10 | 11 | * A MATLAB Parallel Server license. You can use either: 12 | * A MATLAB Parallel Server license configured to use online licensing for MATLAB. For information on how to configure your license for cloud use, see [Configure MATLAB Parallel Server Licensing for Cloud Platforms](https://www.mathworks.com/help/matlab-parallel-server/configure-matlab-parallel-server-licensing-for-cloud-platforms.html). 13 | * A network license manager for MATLAB hosting sufficient MATLAB Parallel Server licenses for your cluster. MathWorks® provides a reference architecture to deploy a suitable [Network License Manager for MATLAB on Azure](https://github.com/mathworks-ref-arch/license-manager-for-matlab-on-azure) or you can use an existing license manager. 14 | * MATLAB and Parallel Computing Toolbox™ on your desktop. 15 | * An Azure account. To configure your account to enable autoscaling (since R2022b), see [Configure Azure Account](#configure-azure-account). 16 | 17 | # Costs 18 | You are responsible for the cost of the Azure services you use when you create cloud resources using this repository. Resource settings, such as virtual machine (VM) type, affect the cost of deployment. For cost estimates, see the pricing pages for each Azure service you use. Prices are subject to change. 19 | 20 | # Deployment Steps 21 | 22 | To view instructions for deploying the MATLAB Parallel Server reference architecture, select a MATLAB release. 23 | 24 | | Linux | Windows | 25 | | ----- | ------- | 26 | | [R2025a](https://github.com/mathworks-ref-arch/matlab-parallel-server-on-azure-lin/tree/master/releases/R2025a/README.md) | [R2025a](releases/R2025a/README.md) | 27 | | [R2024b](https://github.com/mathworks-ref-arch/matlab-parallel-server-on-azure-lin/tree/master/releases/R2024b/README.md) | [R2024b](releases/R2024b/README.md) | 28 | | [R2024a](https://github.com/mathworks-ref-arch/matlab-parallel-server-on-azure-lin/tree/master/releases/R2024a/README.md) | [R2024a](releases/R2024a/README.md) | 29 | | [R2023b](https://github.com/mathworks-ref-arch/matlab-parallel-server-on-azure-lin/tree/master/releases/R2023b/README.md) | [R2023b](releases/R2023b/README.md) | 30 | | [R2023a](https://github.com/mathworks-ref-arch/matlab-parallel-server-on-azure-lin/tree/master/releases/R2023a/README.md) | [R2023a](releases/R2023a/README.md) | 31 | | [R2022b](https://github.com/mathworks-ref-arch/matlab-parallel-server-on-azure-lin/tree/master/releases/R2022b/README.md) | [R2022b](releases/R2022b/README.md) | 32 | | [R2022a](https://github.com/mathworks-ref-arch/matlab-parallel-server-on-azure-lin/tree/master/releases/R2022a/README.md) | [R2022a](releases/R2022a/README.md) | 33 | | | [R2021b](releases/R2021b/README.md) | 34 | | | [R2021a](releases/R2021a/README.md) | 35 | | | [R2020b](releases/R2020b/README.md) | 36 | | | [R2020a](releases/R2020a/README.md) | 37 | | | [R2019b](releases/R2019b/README.md) | 38 | | | [R2019a\_and\_older](releases/R2019a_and_older/README.md) | 39 | 40 | 41 | # Learn About Cluster Architecture 42 | 43 | This diagram illustrates the cluster architecture created by the template. When you use the Azure Resource Manager template in this repository, it automatically creates the [MATLAB Job Scheduler](#what-is-matlab-job-scheduler) and necessary resources. For more information about each resource, see the [Azure template reference.](https://learn.microsoft.com/azure/templates/) 44 | 45 | ![Cluster Architecture](img/Azure_Solution_Scale_Set.png?raw=true) 46 | 47 | *Figure 1: Cluster Architecture* 48 | 49 | The following resources are created. 50 | 51 | ### Networking Resources 52 | * Virtual network (Microsoft.Network/virtualNetworks) The virtual network includes the following components: 53 | * Subnet (Microsoft.Network/virtualNetworks/subnets) 54 | * Network security group (Microsoft.Network/networkSecurityGroups) 55 | * Each VM deployed to the virtual network creates the following: 56 | * Network interface (Microsoft.Network/networkInterfaces) 57 | * Public IP address (Microsoft.Network/publicIPAddresses) 58 | 59 | ### Compute Resources 60 | * Head node VM (Microsoft.Compute/virtualMachines): A compute VM for the cluster head node. The MATLAB install is part of the VM image, and the job database is stored either locally on the root volume or on a separate data disk. The head node communicates with the clients using a secure SSL connection. 61 | * Database volume (optional): An optional separate data disk to store the MATLAB Job Scheduler job database. 62 | * Worker scaling set (Microsoft.Compute/virtualMachineScaleSets): A scale set for worker VMs to be deployed into. Clients and workers communicate using a secure SSL connection. 63 | 64 | ### Storage Resources 65 | * Storage account (Microsoft.Storage/storageAccounts): A standard locally redundant storage (LRS) account that hosts the file share used to distribute files amongst cluster VMs. 66 | * File share created inside storage account: A file share inside the storage account. 67 | * Contains the shared secret created by the head node for all worker VMs. The worker VMs require the shared secret to register and establish a secure connection with the job scheduler. 68 | * Contains the cluster profile. The client machine requires the cluster profile to authenticate and establish a connection with the head node. 69 | 70 | ## Configure Azure Account 71 | 72 | To enable autoscaling or auto-termination for your cluster, you must have the following permissions: 73 | 74 | * `Microsoft.Authorization/roleDefinitions/write` 75 | * `Microsoft.Authorization/roleAssignments/write` 76 | 77 | To check if you have these permissions, see [Check access for a user to Azure resources](https://learn.microsoft.com/azure/role-based-access-control/check-access). 78 | 79 | If you do not have these permissions, the administrator or [owners of the subscription](https://learn.microsoft.com/azure/role-based-access-control/role-assignments-list-portal#list-owners-of-a-subscription) can either: 80 | 81 | 1. Assign you the built-in Azure role [User Access Administrator](https://learn.microsoft.com/azure/role-based-access-control/built-in-roles#user-access-administrator) in addition to your existing role. see [Assign Azure roles using the Azure portal](https://learn.microsoft.com/azure/role-based-access-control/role-assignments-portal). 82 | 2. Create a custom role containing these permissions and attach it along with your existing role. see [Create or update Azure custom roles using the Azure portal](https://learn.microsoft.com/azure/role-based-access-control/custom-roles-portal). 83 | 84 | ## FAQ 85 | 86 | ### What can I do with MATLAB Parallel Server? 87 | 88 | Parallel Computing Toolbox and MATLAB Parallel Server let you solve computationally and data-intensive programs using MATLAB and Simulink on computer clusters, clouds, and grids. Parallel processing constructs such as parallel for-loops and code blocks, distributed arrays, parallel numerical algorithms, and message-passing functions let you implement task-parallel and data-parallel algorithms in MATLAB. For more information, see the [Parallel Computing Toolbox](https://www.mathworks.com/help/parallel-computing/) and [MATLAB Parallel Server](https://www.mathworks.com/help/matlab-parallel-server) documentation. 89 | 90 | ### What is MATLAB Job Scheduler? 91 | 92 | MATLAB Job Scheduler is a scheduler that ships with MATLAB Parallel Server. The scheduler coordinates the execution of jobs and distributes the tasks for evaluation to the server’s individual MATLAB sessions called workers. For more details, see [How Parallel Computing Toolbox Runs a Job](https://www.mathworks.com/help/parallel-computing/how-parallel-computing-products-run-a-job.html). The MATLAB Job Scheduler and the resources required by it are created using [Azure Resource Manager templates](https://learn.microsoft.com/azure/azure-resource-manager/management/overview). 93 | 94 | ### What is Microsoft Azure? 95 | 96 | Microsoft Azure is a set of cloud services that allow you to build, deploy, and manage applications hosted in Microsoft’s global network of data centers. For more information about the range of services offered by Microsoft Azure, see [Azure Services](https://azure.microsoft.com/products/). Services deployed in Azure can be created, managed, and deleted using the Azure Portal UI. For more information about the Azure Portal, see [Azure Portal](https://azure.microsoft.com/get-started/azure-portal/). 97 | 98 | ### What skills or specializations do I need to use this reference architecture? 99 | 100 | No programming or cloud experience required. 101 | 102 | ### How long does it take to deploy the reference architecture? 103 | 104 | If you already have an Azure account set up and ready to use, you can start a MATLAB Parallel Server reference architecture cluster in less than 15 minutes. Startup time varies depending on the size of your cluster. 105 | 106 | ### How do I manage limits for Azure services? 107 | 108 | For information about setting quotas, see [Azure subscription and service limits](https://learn.microsoft.com/azure/azure-resource-manager/management/azure-subscription-service-limits). 109 | 110 | ### What is an Azure Spot Virtual Machine, and what factors should I consider before enabling it? 111 | 112 | Spot Virtual Machines allow you to use Azure VMs at a reduced cost. Azure uses Spot Virtual Machines to sell unused instances within their data centers. However, Azure can reclaim these virtual machines at any time. For more details, see the Azure documentation on [Spot Virtual Machines](https://azure.microsoft.com/products/virtual-machines/spot). 113 | 114 | Before enabling Spot Virtual Machines, consider these aspects: 115 | 116 | * Pricing: Spot Virtual Machines offer discounts compared to on-demand virtual machines. The actual discount depends on the available unused capacity of the virtual machines within the availability zone (AZ). For more details, see the [Azure documentation](https://azure.microsoft.com/pricing/details/virtual-machines/linux/). 117 | 118 | * Behavior of your cluster when Azure reclaims a Spot Virtual Machine: Spot Virtual Machines are used only for the worker nodes, whereas the head node always uses an on-demand virtual machine. This behavior protects your user job and task information from being lost when a virtual machine is reclaimed by Azure. Jobs in the queue are run when a new worker instance is available. For details on how jobs are run, refer to [How Parallel Computing Toolbox Runs a Job](https://www.mathworks.com/help/parallel-computing/how-parallel-computing-products-run-a-job.html). If a Spot VM for a worker is interrupted when it is running a task, the task is marked as failed. You can set the maximum number of times to rerun a failed task using the `MaximumRetries` property. For more details, see [Access task properties and behaviors](https://www.mathworks.com/help/parallel-computing/parallel.task.html). 119 | 120 | * The following VM sizes are not supported for Azure Spot Virtual Machines: 121 | * B-series 122 | * Promo versions of any size (such as the Dv2, NV, NC, and H promo sizes) 123 | 124 | For more information about the limitations of using Spot Virtual Machines, see [Azure Spot Virtual Machines for Virtual Machine Scale Sets](https://learn.microsoft.com/azure/virtual-machine-scale-sets/use-spot). 125 | 126 | # Technical Support 127 | If you need help or have a request for additional features or capabilities, contact [MathWorks Technical Support](https://www.mathworks.com/support/contact_us.html). 128 | 129 | ---- 130 | 131 | Copyright 2018-2025 The MathWorks, Inc. 132 | 133 | ---- -------------------------------------------------------------------------------- /SECURITY.md: -------------------------------------------------------------------------------- 1 | Reporting Security Vulnerabilities 2 | ================================== 3 | If you believe you have discovered a security vulnerability, please report it to 4 | security@mathworks.com instead of GitHub. Please see 5 | [MathWorks Vulnerability Disclosure Policy for Security Researchers](https://www.mathworks.com/company/aboutus/policies_statements/vulnerability-disclosure-policy.html) 6 | for additional information. -------------------------------------------------------------------------------- /img/Azure_Solution_Scale_Set.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mathworks-ref-arch/matlab-parallel-server-on-azure/64a2e658b1cc5363c4599af0e288adeaee8450e8/img/Azure_Solution_Scale_Set.png -------------------------------------------------------------------------------- /img/Deployment_Complete_Select_Storage.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mathworks-ref-arch/matlab-parallel-server-on-azure/64a2e658b1cc5363c4599af0e288adeaee8450e8/img/Deployment_Complete_Select_Storage.png -------------------------------------------------------------------------------- /img/Resource_Group_Delete.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/mathworks-ref-arch/matlab-parallel-server-on-azure/64a2e658b1cc5363c4599af0e288adeaee8450e8/img/Resource_Group_Delete.png -------------------------------------------------------------------------------- /releases/R2019a_and_older/README.md: -------------------------------------------------------------------------------- 1 | # MATLAB Parallel Server on Microsoft Azure 2 | 3 | # Requirements 4 | 5 | Before starting, you will need the following: 6 | 7 | - MATLAB Parallel Server™ license. For more information on how to configure your license for cloud use, see [MATLAB Parallel Server on the Cloud](https://www.mathworks.com/help/licensingoncloud/matlab-parallel-server-on-the-cloud.html) 8 | 9 | - MATLAB® and Parallel Computing Toolbox™ on your desktop. 10 | 11 | - An Azure™ account. 12 | 13 | # Costs 14 | You are responsible for the cost of the Azure services used when you create cloud resources using this guide. Resource settings, such as instance type, will affect the cost of deployment. For cost estimates, see the pricing pages for each Azure service you will be using. Prices are subject to change. 15 | 16 | # Introduction 17 | The following guide will help you automate the process of launching MATLAB Parallel Server and MATLAB Job Scheduler on Azure using your Azure account. The cloud resources are created using Azure Resource Manager (ARM) templates. For information about the architecture of this solution, see [Learn About Cluster Architecture](#learn-about-cluster-architecture). 18 | 19 | # Choose a Deployment Option 20 | The MATLAB Parallel Server cloud reference architecture for Azure supports two license configurations: online licensing and a network license manager. For more information on how to configure your license for cloud use, see MATLAB Parallel Server on the Cloud. 21 | 22 | * [Deploy MATLAB Parallel Server on Azure using Online Licensing](online-licensing-instructions.md) 23 | * [Deploy MATLAB Parallel Server on Azure using a Network License Manager](license-manager-instructions.md) 24 | 25 | # Learn About Cluster Architecture 26 | 27 | Parallel Computing Toolbox and MATLAB Parallel Server software let you solve computationally and data-intensive programs using MATLAB and Simulink on computer clusters, clouds, and grids. Parallel processing constructs such as parallel-for loops and code blocks, distributed arrays, parallel numerical algorithms, and message-passing functions let you implement task-parallel and data-parallel algorithms at a high level in MATLAB. To learn more see the documentation: [Parallel Computing Toolbox](https://www.mathworks.com/help/parallel-computing/) and [MATLAB Parallel Server](https://www.mathworks.com/help/matlab-parallel-server). 28 | 29 | The MATLAB Job Scheduler is a built-in scheduler that ships with MATLAB Parallel Server. The scheduler coordinates the execution of jobs, and distributes the tasks for evaluation to the server’s individual MATLAB sessions called workers. 30 | 31 | Microsoft Azure is a set of cloud services which allow you to build, deploy, and manage applications hosted in Microsoft’s global network of data centres. This document will help you launch a compute cluster using compute, storage, and network services hosted by Azure. For more information about the range of services offered by Microsoft Azure, see [Azure Services](https://azure.microsoft.com/en-gb/services/). Services launched in Azure can be created, managed, and deleted using the Azure Portal UI. For more information about the Azure Portal, see [Azure Portal](https://azure.microsoft.com/en-gb/features/azure-portal/). 32 | 33 | The MATLAB Job Scheduler and the resources required by it are created using [Azure Resource Manager templates](https://docs.microsoft.com/en-gb/azure/azure-resource-manager/resource-group-overview). The architecture of the cluster resources created by the template is illustrated in Figure 2. For more information about each resource, see the [Azure template reference.](https://docs.microsoft.com/en-us/azure/templates/) 34 | 35 | ![Cluster Architecture](../../img/Azure_Solution_Scale_Set.png?raw=true) 36 | 37 | *Figure 2: Cluster Architecture* 38 | 39 | The following resources are created. 40 | 41 | ### Networking resources 42 | * Virtual Network (Microsoft.Network/virtualNetworks) The Virtual Network includes the following components: 43 | * Subnet (Microsoft.Network/virtualNetworks/subnets) 44 | * Network Security Group (Microsoft.Network/networkSecurityGroups) : Ingress rules from client IP address: 45 | * Allow 3389: Required for Remote Desktop Protocol to the cluster nodes. 46 | * Allow 27350 – 27357 + (4 * number of workers): Open 8 ports, plus 4 additional ports for each worker service on the Virtual Machine (VM). Required for communication from clients to the job scheduler and worker processes. 47 | * Allow all internal traffic: Open access to network traffic between all cluster nodes internally. 48 | * Each instance deployed to the Virtual Network will create the following: 49 | * Network interface (Microsoft.Network/networkInterfaces) 50 | * Public IP Address (Microsoft.Network/publicIPAddresses) 51 | 52 | ### Instances 53 | * Headnode instance (Microsoft.Compute/virtualMachines): A Compute instance for the cluster headnode. The MATLAB install is part of the VM image and the job database is stored either locally on the root volume, or optionally, a separate data disk can be used. Communication between clients and the headnode is secured using SSL. 54 | * Database Volume (optional) (Microsoft.Compute/disks): A separate data disk to store the MJS job database. This is optional, and if not chosen the root volume will be used for the job database. 55 | * Custom Script Extension (Microsoft.Compute/virtualMachines/extensions): An extension which configures this instance at deployment time as the headnode of the cluster. 56 | * Worker Scaling Set (Microsoft.Compute/virtualMachineScaleSets): A scale set for worker instances to be launched into. The scaling features are not currently used. The scale set is configured to attach an extension to each instance which configures the instance at deployment time as a worker node of the cluster. Communication between clients and workers is secured using SSL. 57 | 58 | ### Storage 59 | * Storage Account (Microsoft.Storage/storageAccounts): A standard geographically redundant storage (GRS) Storage Account which hosts the File Share used to distribute files amongst cluster instances. 60 | * File Share created inside Storage Account. Created by the head node and mounted by all instances to K: drive. 61 | * Used to distribute the Shared Secret created by head node to all worker VMs. The Shared Secret is required for worker instances to register and establish a secure connection with the job scheduler. 62 | * Used to distribute the Cluster Profile to clients. The Cluster Profile is required to authenticate that a user has permission to connect to the cluster. 63 | * Files uploaded to this File Share will be available to all workers using the K: drive. 64 | -------------------------------------------------------------------------------- /releases/R2019a_and_older/license-manager-instructions.md: -------------------------------------------------------------------------------- 1 | # MATLAB Parallel Server on Azure using a Network License Manager 2 | 3 | # Requirements 4 | 5 | Before starting, you will need the following: 6 | 7 | - A network license manager for MATLAB hosting sufficient MATLAB Parallel Server licenses for your cluster. MathWorks provide a reference architecture to deploy a suitable [Network License Manager for MATLAB on Azure](https://github.com/mathworks-ref-arch/license-manager-for-matlab-on-azure). 8 | 9 | ## Step 1. Launch the Template 10 | 11 | Click the **Deploy to Azure** button below to deploy the cloud resources on Azure. This will open the Azure Portal in your web browser. 12 | 13 | 14 | 15 | 16 | 17 | > Cluster Platform: Windows Server 2016 18 | 19 | > MATLAB Release: R2019a 20 | 21 | For other releases, see [How do I launch a template that uses a previous MATLAB release?](#how-do-i-launch-a-template-that-uses-a-previous-matlab-release) 22 | 23 | ## Step 2. Configure the Cloud Resources 24 | Clicking the Deploy to Azure button opens the "Custom deployment" page in your browser. You can configure the parameters on this page. It is easier to complete the steps if you position these instructions and the Azure Portal window side by side. 25 | 26 | 1. Specify and check the defaults for these resource parameters: 27 | 28 | | Parameter label | Description 29 | | ----------------------------------------------- | ----------- 30 | | **Subscription** | Choose an Azure subscription to use for purchasing resources. 31 | | **Resource group** | Choose a name for the resource group that will hold the resources. It is recommended to create a new resource group for each deployment. This allows all resources to be deleted simultaneously. 32 | | **Location** | Choose the region to start resources in. Ensure that you select a location which supports your requested instance types. To check which services are supported in each location, see [Products available by region](https://azure.microsoft.com/en-gb/regions/services/). 33 | | **Cluster Name** | Choose a name to use for the cluster. This name will be shown in MATLAB when connected to the cluster. 34 | | **Num Worker Nodes** | Choose the number of Azure instances to start for the workers. 35 | | **Num Workers Per Node** | Choose the number of MATLAB workers to start on each node. Specify 1 worker for every physical core. The number of vCPUs per physical core depends on the instance family type. See the [Azure documentation](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/acu) for details on vCPUs for each instance type. 36 | | **Database Volume Size** | The size of the data disk to use for the database. All job and task information, including input and output data will be stored on this volume and it should therefore have enough capacity to store the expected amount of data. If this parameter is set to 0 no volume will be created and the root volume of the instance will be used for the database. 37 | | **Headnode Instance Type** | Choose the Azure instance type to use for the head node, which will run the job manager. No workers will be started on this node, so this can be a smaller instance type than the worker nodes. The selected [virtual machine size](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes) must support premium storage. 38 | | **Worker Instance Type** | Choose the Azure instance type to use for the workers. The selected [virtual machine size](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes) must support premium storage. 39 | | **Client IP Address** | This is the IP address range that will be allowed to access the resources. The format for this field is IP Address/Mask.

Example:

10.0.0.1/32

40 | | **License Manager Host Name** | The hostname or ip address of the virtual machine hosting your network license manager. If deployed in a different virtual network, ensure that the hostname or address can be correctly resolved. 41 | | **License Manager Port** | The port to use when connecting to your network license manager. 42 | | **Admin Username** | Choose the admin username for all instances. This username is required when logging into any instance using remote desktop protocol. 43 | | **Admin Password** | Choose the admin password for all instances. This password is required when logging into any instance using remote desktop protocol. For the deployment to succeed, your password must meet [Azure's password requirements](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/faq#what-are-the-password-requirements-when-creating-a-vm). 44 | | **Virtual Network Resource ID** | The Resource ID of an existing virtual network to deploy your cluster into. You can find this under the Properties of your virtual network. 45 | | **Subnet Name** | The name of an existing subnet within your virtual network to deploy your cluster into. 46 | 47 | **NOTE**: The port and hostname of the network license manager must be reachable from the worker MATLAB VMs. It is therefore recommended that you deploy your cluster into a subnet within the same virtual network as the network license manager. 48 | 49 | 2. Tick the box to accept the Azure Marketplace terms and conditions. 50 | 51 | 3. Click the **Purchase** button. 52 | 53 | When you click the **Purchase** button, the resources are created using Azure template deployments. Template deployment can take several minutes. 54 | 55 | # Step 3: Connect to Your Cluster From MATLAB 56 | 57 | 1. After clicking **Purchase** you will be taken to the Azure Portal Dashboard. To montior the progress of your deployment, select your resource group from the Resource Groups panel. Wait for the all **Deployments** to reach **Succeeded**. 58 | 2. Select the Storage Account ending with **storage**. The screen should look like the one in Figure 1. 59 | 60 | ![Resource Group On Completion](../../img/Deployment_Complete_Select_Storage.png) 61 | 62 | *Figure 1: Resource Group On Completion* 63 | 64 | 3. Select the Files container type. 65 | 4. Select the File Share named "shared". 66 | 5. Download the file, `cluster/.settings`, where NAME is the name of your MATLAB job scheduler. 67 | 6. Open MATLAB. 68 | 7. In the Parallel drop-down menu in the MATLAB toolstrip select **Create and Manage Clusters...**. 69 | 8. Click **Import**. 70 | 9. Select the downloaded profile and click open. 71 | 10. Click **Set as Default**. 72 | 11. (Optional) Validate your cluster by clicking the **Validate** button. 73 | 74 | After setting the cloud cluster as default, the next time you run a parallel language command (such as `parfor`, `spmd`, `parfeval` or `batch`) MATLAB connects to the cluster. The first time you run a task on a worker it will take several minutes for the worker MATLAB to start. This delay is due to provisioning the instance disk. This is a one-time operation, and subsequent tasks begin much faster. 75 | 76 | Your cluster is now ready to use. It will remain running after you close MATLAB. 77 | 78 | **NOTE**: Use the profile and client IP address range to control access to your cloud resources. Anyone with this file can connect to your resources from a machine within the specified IP address range and run jobs on it. 79 | 80 | # Additional Information 81 | ## Delete Your Cloud Resources 82 | You can remove the Resource Group and all associated resources when you are done with them. Note that you cannot recover resources once they are deleted. After you delete the cloud resources you cannot use the downloaded profile again. 83 | 1. Login to the Azure Portal. 84 | 2. Select the Resource Group containing your resources. 85 | 3. Select the "Delete resource group" icon to destroy all resources deplyoyed in this group. 86 | 4. You will be prompted to enter the name of the resource group to confirm the deletion. 87 | 88 | ![Resource Group Delete](../../img/Resource_Group_Delete.png) 89 | 90 | ## Troubleshooting 91 | If your resource group fails to deploy, check the Deployments section of the Resource Group. It will indicate which resource deployments failed and allow you to navigate to the causing error message. 92 | 93 | If the resource group deployed successfully but you are unable to validate the cluster, you may need to view the logs on the instances to diagnose the error. The deployment logs are output to C:/Windows/Temp/MDCSLog*.txt on the instance nodes. The cluster logs are output to C:/Windows/Temp/MDCE/Log. 94 | 95 | ## How do I launch a template that uses a previous MATLAB release? 96 | You can launch the reference architecture within previous releases of MATLAB using the following templates. 97 | 98 | | Release | Windows Server 2016 VM | 99 | |---------------|-----------------| 100 | | MATLAB R2018b | | 101 | | MATLAB R2018a | | 102 | -------------------------------------------------------------------------------- /releases/R2019a_and_older/online-licensing-instructions.md: -------------------------------------------------------------------------------- 1 | # MATLAB Parallel Server on Azure using Online Licensing 2 | 3 | # Requirements 4 | 5 | Before starting, you will need the following: 6 | 7 | - MATLAB Parallel Server license configured to use online licensing for MATLAB. For more information on how to configure your license for cloud use, see [MATLAB Parallel Server on the Cloud](https://www.mathworks.com/help/licensingoncloud/matlab-parallel-server-on-the-cloud.html). 8 | 9 | ## Step 1. Launch the Template 10 | 11 | Click the **Deploy to Azure** button below to deploy the cloud resources on Azure. This will open the Azure Portal in your web browser. 12 | 13 | 14 | 15 | 16 | 17 | > Cluster Platform: Windows Server 2016 18 | 19 | > MATLAB Release: R2019a 20 | 21 | For other releases, see [How do I launch a template that uses a previous MATLAB release?](#how-do-i-launch-a-template-that-uses-a-previous-matlab-release) 22 | 23 | ## Step 2. Configure the Cloud Resources 24 | Clicking the Deploy to Azure button opens the "Custom deployment" page in your browser. You can configure the parameters on this page. It is easier to complete the steps if you position these instructions and the Azure Portal window side by side. 25 | 26 | 1. Specify and check the defaults for these resource parameters: 27 | 28 | | Parameter label | Description 29 | | ----------------------------------------------- | ----------- 30 | | **Subscription** | Choose an Azure subscription to use for purchasing resources. 31 | | **Resource group** | Choose a name for the resource group that will hold the resources. It is recommended to create a new resource group for each deployment. This allows all resources to be deleted simultaneously. 32 | | **Location** | Choose the region to start resources in. Ensure that you select a location which supports your requested instance types. To check which services are supported in each location, see [Products available by region](https://azure.microsoft.com/en-gb/regions/services/). 33 | | **Cluster Name** | Choose a name to use for the cluster. This name will be shown in MATLAB when connected to the cluster. 34 | | **Num Worker Nodes** | Choose the number of Azure instances to start for the workers. 35 | | **Num Workers Per Node** | Choose the number of MATLAB workers to start on each node. Specify 1 worker for every physical core. The number of vCPUs per physical core depends on the instance family type. See the [Azure documentation](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/acu) for details on vCPUs for each instance type. 36 | | **Database Volume Size** | The size of the data disk to use for the database. All job and task information, including input and output data will be stored on this volume and it should therefore have enough capacity to store the expected amount of data. If this parameter is set to 0 no volume will be created and the root volume of the instance will be used for the database. 37 | | **Headnode Instance Type** | Choose the Azure instance type to use for the head node, which will run the job manager. No workers will be started on this node, so this can be a smaller instance type than the worker nodes. The selected [virtual machine size](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes) must support premium storage. 38 | | **Worker Instance Type** | Choose the Azure instance type to use for the workers. The selected [virtual machine size](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes) must support premium storage. 39 | | **Client IP Address** | This is the IP address range that will be allowed to access the resources. The format for this field is IP Address/Mask.

Example:

10.0.0.1/32

40 | | **Admin Username** | Choose the admin username for all instances. This username is required when logging into any instance using remote desktop protocol. 41 | | **Admin Password** | Choose the admin password for all instances. This password is required when logging into any instance using remote desktop protocol. For the deployment to succeed, your password must meet [Azure's password requirements](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/faq#what-are-the-password-requirements-when-creating-a-vm). 42 | 43 | 2. Tick the box to accept the Azure Marketplace terms and conditions. 44 | 45 | 3. Click the **Purchase** button. 46 | 47 | When you click the **Purchase** button, the resources are created using Azure template deployments. Template deployment can take several minutes. 48 | 49 | # Step 3: Connect to Your Cluster From MATLAB 50 | 51 | 1. After clicking **Purchase** you will be taken to the Azure Portal Dashboard. To montior the progress of your deployment, select your resource group from the Resource Groups panel. Wait for the all **Deployments** to reach **Succeeded**. 52 | 2. Select the Storage Account ending with **storage**. The screen should look like the one in Figure 1. 53 | 54 | ![Resource Group On Completion](../../img/Deployment_Complete_Select_Storage.png) 55 | 56 | *Figure 1: Resource Group On Completion* 57 | 58 | 3. Select the Files container type. 59 | 4. Select the File Share named "shared". 60 | 5. Download the file, `cluster/.settings`, where NAME is the name of your MATLAB job scheduler. 61 | 6. Open MATLAB. 62 | 7. In the Parallel drop-down menu in the MATLAB toolstrip select **Create and Manage Clusters...**. 63 | 8. Click **Import**. 64 | 9. Select the downloaded profile and click open. 65 | 10. Click **Set as Default**. 66 | 11. (Optional) Validate your cluster by clicking the **Validate** button. 67 | 68 | After setting the cloud cluster as default, the next time you run a parallel language command (such as `parfor`, `spmd`, `parfeval` or `batch`) MATLAB connects to the cluster. The first time you connect, you will be prompted for your MathWorks account login. The first time you run a task on a worker it will take several minutes for the worker MATLAB to start. This delay is due to provisioning the instance disk. This is a one-time operation, and subsequent tasks begin much faster. 69 | 70 | Your cluster is now ready to use. It will remain running after you close MATLAB. 71 | 72 | **NOTE**: Use the profile and client IP address range to control access to your cloud resources. Anyone with this file can connect to your resources from a machine within the specified IP address range and run jobs on it. 73 | 74 | # Additional Information 75 | ## Delete Your Cloud Resources 76 | You can remove the Resource Group and all associated resources when you are done with them. Note that you cannot recover resources once they are deleted. After you delete the cloud resources you cannot use the downloaded profile again. 77 | 1. Login to the Azure Portal. 78 | 2. Select the Resource Group containing your resources. 79 | 3. Select the "Delete resource group" icon to destroy all resources deplyoyed in this group. 80 | 4. You will be prompted to enter the name of the resource group to confirm the deletion. 81 | 82 | ![Resource Group Delete](../../img/Resource_Group_Delete.png) 83 | 84 | ## Troubleshooting 85 | If your resource group fails to deploy, check the Deployments section of the Resource Group. It will indicate which resource deployments failed and allow you to navigate to the causing error message. 86 | 87 | If the resource group deployed successfully but you are unable to validate the cluster, you may need to view the logs on the instances to diagnose the error. The deployment logs are output to C:/Windows/Temp/MDCSLog*.txt on the instance nodes. The cluster logs are output to C:/Windows/Temp/MDCE/Log. 88 | 89 | ## Use Existing Virtual Network 90 | You can launch the reference architecture within an existing virtual network and subnet using the following template. 91 | 92 | 93 | 94 | 95 | 96 | > Cluster Platform: Windows Server 2016 97 | 98 | > MATLAB Release: R2019a 99 | 100 | This template requires the following two additional parameters: 101 | 102 | | Parameter label | Description 103 | | ------------------------------------------- | ----------- 104 | | **Virtual Network Resource ID** | The Resource ID of an existing virtual network to deploy your cluster into. You can find this under the Properties of your virtual network. 105 | | **Subnet Name** | The name of an existing subnet within your virtual network to deploy your cluster into. 106 | 107 | ## How do I launch a template that uses a previous MATLAB release? 108 | You can launch the reference architecture within previous releases of MATLAB using the following templates. 109 | 110 | | Release | Windows Server 2016 VM | Windows Server 2016 VM using existing Virtual Network | 111 | |---------------|------------------------|-----------------| 112 | | MATLAB R2018b | | | 113 | | MATLAB R2018a | | | 114 | -------------------------------------------------------------------------------- /releases/R2019a_and_older/parameters-existing-vnet.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#", 3 | "contentVersion": "1.0.0.0", 4 | "parameters": { 5 | "clusterName": { 6 | "value": "myCluster" 7 | }, 8 | "numWorkerNodes": { 9 | "value": 2 10 | }, 11 | "numWorkersPerNode": { 12 | "value": 2 13 | }, 14 | "headnodeInstanceType": { 15 | "value": "Standard_D2s_v3" 16 | }, 17 | "workerInstanceType": { 18 | "value": "Standard_D2s_v3" 19 | }, 20 | "databaseVolumeSize": { 21 | "value": 100 22 | }, 23 | "clientIPAddress": { 24 | "value": null 25 | }, 26 | "adminUsername": { 27 | "value": "clouduser" 28 | }, 29 | "adminPassword": { 30 | "value": null 31 | }, 32 | "virtualNetworkResourceID": { 33 | "value": null 34 | }, 35 | "subnetName": { 36 | "value": null 37 | } 38 | } 39 | } -------------------------------------------------------------------------------- /releases/R2019a_and_older/parameters-with-license-manager.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#", 3 | "contentVersion": "1.0.0.0", 4 | "parameters": { 5 | "clusterName": { 6 | "value": "myCluster" 7 | }, 8 | "numWorkerNodes": { 9 | "value": 2 10 | }, 11 | "numWorkersPerNode": { 12 | "value": 2 13 | }, 14 | "headnodeInstanceType": { 15 | "value": "Standard_D2s_v3" 16 | }, 17 | "workerInstanceType": { 18 | "value": "Standard_D2s_v3" 19 | }, 20 | "databaseVolumeSize": { 21 | "value": 100 22 | }, 23 | "clientIPAddress": { 24 | "value": null 25 | }, 26 | "licenseManagerHostName": { 27 | "value": "netlm-server" 28 | }, 29 | "licenseManagerPort": { 30 | "value": 27000 31 | }, 32 | "adminUsername": { 33 | "value": "clouduser" 34 | }, 35 | "adminPassword": { 36 | "value": null 37 | }, 38 | "virtualNetworkResourceID": { 39 | "value": null 40 | }, 41 | "subnetName": { 42 | "value": null 43 | } 44 | } 45 | } -------------------------------------------------------------------------------- /releases/R2019a_and_older/parameters.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#", 3 | "contentVersion": "1.0.0.0", 4 | "parameters": { 5 | "clusterName": { 6 | "value": "myCluster" 7 | }, 8 | "numWorkerNodes": { 9 | "value": 2 10 | }, 11 | "numWorkersPerNode": { 12 | "value": 2 13 | }, 14 | "headnodeInstanceType": { 15 | "value": "Standard_D2s_v3" 16 | }, 17 | "workerInstanceType": { 18 | "value": "Standard_D2s_v3" 19 | }, 20 | "databaseVolumeSize": { 21 | "value": 100 22 | }, 23 | "clientIPAddress": { 24 | "value": null 25 | }, 26 | "adminUsername": { 27 | "value": "clouduser" 28 | }, 29 | "adminPassword": { 30 | "value": null 31 | } 32 | } 33 | } -------------------------------------------------------------------------------- /releases/R2019a_and_older/scripts/createFileShare.ps1: -------------------------------------------------------------------------------- 1 | <# 2 | .SYNOPSIS 3 | createFileShare: Create a file share in an existing storage account 4 | 5 | # Copyright 2018 The MathWorks, Inc. 6 | 7 | .DESCRIPTION 8 | Usage: createFileShare [-StorageAccountName storage_account_name] 9 | [-StorageAccountKey storage_account_key] 10 | [-ShareName share_name] 11 | [-ShareQuota share_quota] 12 | 13 | -StorageAccountName Name of the storage account in which to create the file share. 14 | 15 | -StorageAccountKey Key to the storage account in which to create the file share. 16 | 17 | -ShareName Name of the file share to create. 18 | 19 | -ShareQuota Optional. Specifies the maximum size of the share in GB. Must be greater than 0, 20 | and less than or equal to 5TB(5120). 21 | 22 | .EXAMPLE 23 | createFileShare -StorageAccountName myAccount -StorageAccountKey exampleKey== -ShareName myShare -ShareQuota 100 24 | #> 25 | 26 | Param ( 27 | [Parameter(Mandatory=$True)][ValidateNotNullOrEmpty()][string]$StorageAccountName, 28 | [Parameter(Mandatory=$True)][ValidateNotNullOrEmpty()][string]$StorageAccountKey, 29 | [Parameter(Mandatory=$True)][ValidateNotNullOrEmpty()][string]$ShareName, 30 | [Parameter(Mandatory=$False)][ValidateRange(0, 5120)][string]$ShareQuota 31 | ) 32 | 33 | $Version = "2015-12-11" 34 | 35 | $Url = "https://$StorageAccountName.file.core.windows.net/${ShareName}?restype=share"; 36 | 37 | $Date = (Get-Date).ToUniversalTime() 38 | $Datestr = $Date.ToString("R"); 39 | 40 | $StrToSign = "PUT`n`n`n`n`n`n`n`n`n`n`n`nx-ms-date:$Datestr" 41 | If ($PSBoundParameters.ContainsKey('ShareQuota')) { 42 | $StrToSign = $StrToSign + "`nx-ms-share-quota:$ShareQuota" 43 | } 44 | $StrToSign = $StrToSign + "`nx-ms-version:$Version" 45 | $StrToSign = $StrToSign + "`n/${StorageAccountName}/${ShareName}" 46 | $StrToSign = $StrToSign + "`nrestype:share" 47 | 48 | [byte[]]$DataBytes = ([System.Text.Encoding]::UTF8).GetBytes($StrToSign) 49 | $Hmacsha256 = New-Object System.Security.Cryptography.HMACSHA256 50 | $Hmacsha256.Key = [Convert]::FromBase64String($StorageAccountKey) 51 | $Sig = [Convert]::ToBase64String($Hmacsha256.ComputeHash($DataBytes)) 52 | $AuthHdr = "SharedKey ${StorageAccountName}:$Sig" 53 | 54 | $RequestHeader = New-Object "System.Collections.Generic.Dictionary[[String],[String]]" 55 | 56 | $RequestHeader.Add("Authorization", $AuthHdr) 57 | $RequestHeader.Add("x-ms-date", $Datestr) 58 | $RequestHeader.Add("x-ms-version", $Version) 59 | If ($PSBoundParameters.ContainsKey('ShareQuota')) { 60 | $RequestHeader.Add("x-ms-share-quota", $ShareQuota) 61 | } 62 | $RequestHeader.Add("Content-Length", "0") 63 | 64 | $Response = New-Object PSObject 65 | $Response = (Invoke-RestMethod -Uri $Url -Method put -Headers $RequestHeader) -------------------------------------------------------------------------------- /releases/R2019a_and_older/scripts/listFileShares.ps1: -------------------------------------------------------------------------------- 1 | <# 2 | .SYNOPSIS 3 | listFileShares: List all file shares in an existing storage account 4 | 5 | # Copyright 2018 The MathWorks, Inc. 6 | 7 | .DESCRIPTION 8 | Usage: listFileShares [-StorageAccountName storage_account_name] 9 | [-StorageAccountKey storage_account_key] 10 | 11 | -StorageAccountName Name of the storage account. 12 | 13 | -StorageAccountKey Key to the storage account. 14 | 15 | .EXAMPLE 16 | listFileShares -StorageAccountName myAccount -StorageAccountKey exampleKey== 17 | #> 18 | 19 | Param ( 20 | [Parameter(Mandatory=$True)][ValidateNotNullOrEmpty()][string]$StorageAccountName, 21 | [Parameter(Mandatory=$True)][ValidateNotNullOrEmpty()][string]$StorageAccountKey 22 | ) 23 | 24 | $Version = "2017-04-17" 25 | 26 | $Url = "https://$StorageAccountName.file.core.windows.net/?comp=list"; 27 | 28 | $Date = (Get-Date).ToUniversalTime() 29 | $Datestr = $Date.ToString("R"); 30 | 31 | $StrToSign = "GET`n`n`n`n`n`n`n`n`n`n`n`nx-ms-date:$Datestr" 32 | $StrToSign = $StrToSign + "`nx-ms-version:$Version" 33 | $StrToSign = $StrToSign + "`n/${StorageAccountName}/${ShareName}" 34 | $StrToSign = $StrToSign + "`ncomp:list" 35 | 36 | [byte[]]$DataBytes = ([System.Text.Encoding]::UTF8).GetBytes($StrToSign) 37 | $Hmacsha256 = New-Object System.Security.Cryptography.HMACSHA256 38 | $Hmacsha256.Key = [Convert]::FromBase64String($StorageAccountKey) 39 | $Sig = [Convert]::ToBase64String($Hmacsha256.ComputeHash($DataBytes)) 40 | $AuthHdr = "SharedKey ${StorageAccountName}:$Sig" 41 | 42 | $RequestHeader = New-Object "System.Collections.Generic.Dictionary[[String],[String]]" 43 | 44 | $RequestHeader.Add("Authorization", $AuthHdr) 45 | $RequestHeader.Add("x-ms-date", $Datestr) 46 | $RequestHeader.Add("x-ms-version", $Version) 47 | 48 | $Response = (Invoke-RestMethod -Uri $Url -Method get -Headers $RequestHeader) 49 | [xml]$ResponseXml = $Response.Substring($Response.IndexOf("<")) 50 | If($ResponseXml) { 51 | $Shares = $ResponseXml.EnumerationResults.Shares.Share 52 | Return $Shares 53 | } 54 | Return @() -------------------------------------------------------------------------------- /releases/R2019a_and_older/scripts/startmjs.ps1: -------------------------------------------------------------------------------- 1 | <# 2 | .SYNOPSIS 3 | startmjs: Configure the current VM as part of an MJS cluster 4 | 5 | # Copyright 2018 The MathWorks, Inc. 6 | 7 | .DESCRIPTION 8 | Usage: startmjs [-Role vm_role] 9 | [-NumWorkers number_of_workers] 10 | [-ClusterName mjs_name] 11 | [-StorageAccountName storage_account_name] 12 | [-StorageAccountKey storage_account_key] 13 | [-HeadnodeInternalHostname headnode_internal_hostname] 14 | [-HeadnodeExternalHostname headnode_external_hostname] 15 | [-HeadnodeInternalIPAddress headnode_internal_ip_address] 16 | [-ReleaseDate release_date] 17 | [-LicenseServer license_server] 18 | 19 | -Role Role of VM in MJS cluster. Allowed values are "headnode" or "worker". 20 | Only a single headnode VM is allowed. 21 | 22 | -NumWorkers Number of worker processes to start on VM. To select as many workers as 23 | cores on the VM select -1. 24 | 25 | -ClusterName Name of the MJS scheduler service. If the role is "headnode" a job manager 26 | service with this name will be started. Any worker processes started will 27 | attempt to connect to a job manager service with this name. 28 | 29 | -StorageAccountName Name of Storage Account to create File Share. 30 | 31 | -StorageAccountKey Key to Storage Account to create File Share. 32 | 33 | -HeadnodeInternalHostname The private internal hostname of the headnode VM. 34 | 35 | -HeadnodeExternalHostname The publicly visible external hostname of the headnode VM. 36 | 37 | -HeadnodeInternalIPAddress The private internal ip address of the headnode VM. 38 | 39 | -ReleaseDate Release identifier. 40 | 41 | -LicenseServer License server identifier of the form port@hostname. For example, to use a 42 | Network License Manager running at hostname "licenseHost" on port 27000, use 43 | "27000@licenseHost". 44 | If LicenseServer is empty, or the value is not of the expected form port@hostname, 45 | Online Licensing will be used instead. 46 | 47 | .EXAMPLE 48 | startmjs -Role worker -NumWorkers 2 -ClusterName myCluster 49 | -StorageAccountName store -StorageAccountKey exampleabcdef123456== 50 | -HeadnodeInternalHostname headnode -HeadnodeExternalHostname headnode.example.com 51 | -HeadnodeInternalIPAddress 10.0.0.4 -ReleaseDate development 52 | #> 53 | 54 | Param ( 55 | [Parameter(Mandatory=$True)][ValidateSet('headnode','worker')][string]$Role, 56 | [Parameter(Mandatory=$True)][ValidateRange(-1, [int]::MaxValue)][string]$NumWorkers, 57 | [Parameter(Mandatory=$True)][string]$ClusterName, 58 | [Parameter(Mandatory=$True)][string]$StorageAccountName, 59 | [Parameter(Mandatory=$True)][string]$StorageAccountKey, 60 | [Parameter(Mandatory=$True)][string]$HeadnodeInternalHostname, 61 | [Parameter(Mandatory=$True)][string]$HeadnodeExternalHostname, 62 | [Parameter(Mandatory=$True)][string]$HeadnodeInternalIPAddress, 63 | [Parameter(Mandatory=$False)][string]$ReleaseDate="development", 64 | [Parameter(Mandatory=$False)][string]$LicenseServer="mhlm" 65 | ) 66 | 67 | Function StartTranscript() { 68 | $Datetimestr = (Get-Date).ToString('yyyy-MM-dd-HH-mm-ss') 69 | $Script:Logfile = "$Env:Windir\Temp\MDCSLog-$Datetimestr.log" 70 | Start-Transcript -Path $Script:Logfile -Force 71 | } 72 | 73 | Function StopTranscript() { 74 | Stop-Transcript 75 | } 76 | 77 | Function FindMatlabRoot() { 78 | $Computername = $Env:Computername 79 | $MatlabKey="SOFTWARE\\MathWorks\\MATLAB" 80 | $Reg=[microsoft.win32.registrykey]::OpenRemoteBaseKey('LocalMachine',$Computername) 81 | $Regkey=$Reg.OpenSubKey($MatlabKey) 82 | If ($Regkey) { 83 | # MATLAB install found 84 | $Subkeys=$Regkey.GetSubKeyNames() 85 | $Matlabroot = "" 86 | Foreach($Key In $Subkeys){ 87 | $ThisKey=$MatlabKey + "\\" + $Key 88 | $ThisSubKey=$Reg.OpenSubKey($ThisKey) 89 | $Thisroot = $ThisSubKey.GetValue("MATLABROOT") 90 | If($Matlabroot -lt $Thisroot) { 91 | $Matlabroot = $Thisroot 92 | } 93 | } 94 | Return $Matlabroot 95 | } Else { 96 | # Searching for attached volume with name "matlab" to use as matlab root. 97 | $Matlabvolume = Get-WmiObject win32_logicaldisk | Where-Object -FilterScript {$_.VolumeName -Eq "matlab"} 98 | $Matlabroot = $Matlabvolume.DeviceID 99 | Return $Matlabroot 100 | } 101 | } 102 | 103 | Function RunCommand() { 104 | Param( 105 | [Parameter( 106 | Position=0, 107 | Mandatory=$True, 108 | ValueFromPipeline=$True) 109 | ] 110 | [String[]]$Command 111 | ) 112 | 113 | $Output = Invoke-Expression -Command "$Command" 114 | Echo $Output 115 | If($LASTEXITCODE -ne 0) { 116 | Throw "Error executing command ${Command}: $Output" 117 | } 118 | } 119 | 120 | Function GetPublicIP() { 121 | # Curl VM MetaData for public IP address 122 | $Url = "http://169.254.169.254/metadata/instance/network/interface/0/ipv4/ipAddress/0/publicIpAddress?api-version=2017-04-02&format=text"; 123 | 124 | $PollAttempts = 0 125 | While ($PollAttempts -lt 10) { 126 | $IpAddress = Invoke-RestMethod -Headers @{"Metadata"="true"} -URI $Url -Method get 127 | $PollAttempts ++ 128 | # Test that the address given is valid, otherwise try again. 129 | Try { 130 | $ResolvedIP = [ipaddress]$IpAddress 131 | Return $IpAddress 132 | } Catch { 133 | Echo "Failed to resolve public ip address, $IpAddress" 134 | } 135 | Start-Sleep -s 5 136 | } 137 | Throw "Unable to obtain a valid public ip address using Azure VM metadata" 138 | } 139 | 140 | Function PollUntilTrue() { 141 | param( 142 | [Parameter(Mandatory=$True)] 143 | [scriptblock] 144 | $ScriptBlock, 145 | [Parameter(Mandatory=$False)] 146 | [int] 147 | $TimeoutMinutes = 30 148 | ) 149 | # Poll for existence 150 | $PollPeriodSeconds = 10 151 | $MaxPollAttempts = (60 * $TimeoutMinutes)/$PollPeriodSeconds 152 | $PollAttempts = 0 153 | While ($PollAttempts -lt $MaxPollAttempts) { 154 | $Result = Invoke-Command -ScriptBlock $ScriptBlock 155 | If ($Result) { 156 | Return $True 157 | } 158 | Start-Sleep -s $PollPeriodSeconds; 159 | $PollAttempts ++ 160 | } 161 | Return $False 162 | } 163 | 164 | Function WaitForPathToExist() { 165 | param( 166 | [Parameter(Mandatory=$True)] 167 | [string] 168 | $Path, 169 | [Parameter(Mandatory=$False)] 170 | [int] 171 | $TimeoutMinutes = 30 172 | ) 173 | Echo "Waiting for $Path to exist ..." 174 | If (-Not (PollUntilTrue -TimeoutMinutes $TimeoutMinutes -ScriptBlock {Test-Path $Path})) { 175 | Throw "$Path does not exist after waiting for $TimeoutMinutes minutes. Unable to continue." 176 | } 177 | } 178 | 179 | Function DoesFileShareExist() { 180 | param( 181 | [Parameter(Mandatory=$True)] 182 | [string] 183 | $StorageAccountName, 184 | [Parameter(Mandatory=$True)] 185 | [string] 186 | $StorageAccountKey, 187 | [Parameter(Mandatory=$True)] 188 | [string] 189 | $FileShareName 190 | ) 191 | $listFileSharesScript = Join-Path $PSScriptRoot "listFileShares.ps1" 192 | $Shares = . $listFileSharesScript -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey 193 | ForEach ($Share in $Shares) { 194 | If ($Share.Name -eq $FileShareName) { 195 | Return $True 196 | } 197 | } 198 | Return $False 199 | } 200 | 201 | Function WaitForFileShareToExist() { 202 | param( 203 | [Parameter(Mandatory=$True)] 204 | [string] 205 | $StorageAccountName, 206 | [Parameter(Mandatory=$True)] 207 | [string] 208 | $StorageAccountKey, 209 | [Parameter(Mandatory=$True)] 210 | [string] 211 | $FileShareName, 212 | [Parameter(Mandatory=$False)] 213 | [int] 214 | $TimeoutMinutes = 30 215 | ) 216 | Echo "Waiting for $FileShareName File Share to exist ..." 217 | If (-Not (PollUntilTrue -TimeoutMinutes $TimeoutMinutes -ScriptBlock {DoesFileShareExist ` 218 | -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey -FileShareName $FileShareName})) { 219 | Throw "$FileShareName File Share does not exist after waiting for $TimeoutMinutes minutes. Unable to continue." 220 | } 221 | } 222 | 223 | #****************************************************************************** 224 | # Script body 225 | # Execution begins here 226 | #****************************************************************************** 227 | 228 | # Set default behaviour to stop on error 229 | $ErrorActionPreference = "Stop" 230 | 231 | # Start logging output 232 | StartTranscript 233 | $Currentdir = $Pwd 234 | 235 | # Log current user 236 | $User = whoami 237 | Echo "Current user is: $User" 238 | # Log node type information 239 | Echo "VM role is: $Role" 240 | 241 | $MHLMContext = "MDCS_Azure" 242 | Echo "setenv MHLM_CONTEXT=$MHLMContext" 243 | [Environment]::SetEnvironmentVariable("MHLM_CONTEXT", $MHLMContext, "Machine") 244 | 245 | #****************************************************************************** 246 | # Configure network parameters 247 | #****************************************************************************** 248 | 249 | # Log host information 250 | $InternalHostname = Hostname 251 | $ExternalIP = GetPublicIP 252 | Echo "VM EXTERNAL IP is: $ExternalIP" 253 | Echo "VM INTERNAL NAME is: $InternalHostname" 254 | Echo "Headnode EXTERNAL NAME is: $HeadnodeExternalHostname" 255 | Echo "Headnode INTERNAL NAME is: $HeadnodeInternalHostname" 256 | Echo "Headnode INTERNAL IP Address is: $HeadnodeInternalIPAddress" 257 | 258 | # Open all firewall ports. Rely on external network security group to protect subnet. 259 | Echo "Configure firewall" 260 | Get-NetFirewallRule | ?{$_.Name -like "RemoteSvcAdmin*"} | Enable-NetFirewallRule 261 | New-NetFirewallRule -Name "mdcs_inbound" -DisplayName "mdcs_inbound" -Direction Inbound -Action Allow -Protocol TCP -LocalPort 0-65535 -ErrorAction SilentlyContinue 262 | New-NetFirewallRule -Name "mdcs_outbound" -DisplayName "mdcs_outbound" -Direction Outbound -Action Allow -Protocol TCP -LocalPort 0-65535 -ErrorAction SilentlyContinue 263 | 264 | # Map headnode external hostname to internal hostname 265 | $HostsFile = "$Env:Windir\System32\drivers\etc\hosts" 266 | If (-Not (Select-String -Path $HostsFile -Pattern $HeadnodeExternalHostname)) { 267 | $HostsLine = "$HeadnodeInternalIPAddress`t$HeadnodeExternalHostname" 268 | Echo "Adding $HostsLine to hosts file: $HostsFile" 269 | Add-Content $HostsFile "$HostsLine" 270 | } 271 | 272 | # Set hostname override environment variables. The forces communication between MDCS services to use the 273 | # internal addresses. 274 | Echo "Override internal and external hostnames" 275 | Echo "setenv MDCE_OVERRIDE_INTERNAL_HOSTNAME=$InternalHostname" 276 | Echo "setenv MDCE_OVERRIDE_EXTERNAL_HOSTNAME=$ExternalIP" 277 | [Environment]::SetEnvironmentVariable("MDCE_OVERRIDE_INTERNAL_HOSTNAME", $InternalHostname, "Machine") 278 | [Environment]::SetEnvironmentVariable("MDCE_OVERRIDE_EXTERNAL_HOSTNAME", $ExternalIP, "Machine") 279 | 280 | #****************************************************************************** 281 | # Configure external storage 282 | #****************************************************************************** 283 | 284 | # Check for database disk, and if so mount it to M drive. 285 | $CheckpointDir = "$Env:Windir\Temp\MDCE\Checkpoint" 286 | $LogDir = "$Env:Windir\Temp\MDCE\Log" 287 | $DBDriveLetter = "M" 288 | $Disks = Get-Disk | Where partitionstyle -eq 'raw' | sort number 289 | If ($Disks) { 290 | # Always mount the first data disk as the database. If more data disks are found, there is 291 | # no way of distinguishing which is which. 292 | $DBDisk = $Disks[0] 293 | $DBDriveLabel = "database" 294 | $DBDisk | 295 | Initialize-Disk -PartitionStyle MBR -PassThru -Confirm:$False | 296 | New-Partition -UseMaximumSize -DriveLetter $DBDriveLetter | 297 | Format-Volume -FileSystem NTFS -NewFileSystemLabel $DBDriveLabel -Confirm:$False -Force -ErrorAction SilentlyContinue 298 | } 299 | If (Test-Path "${DBDriveLetter}:\") { 300 | # Set checkpoint directory to mounted disk 301 | $CheckpointDir = "${DBDriveLetter}:\MDCE\Checkpoint" 302 | } 303 | 304 | # If the headnode, create File Share. If a worker, wait for File Share to exist 305 | $FileShareHost = "$StorageAccountName.file.core.windows.net" 306 | $FileShareName = "shared" 307 | $MountPath = "\\${FileShareHost}\$FileShareName" 308 | If (0 -eq $Role.ToLower().CompareTo("headnode")) { 309 | # Create File Share if it does not already exist 310 | $FileShareAlreadyExists = DoesFileShareExist -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey -FileShareName $FileShareName 311 | If ($FileShareAlreadyExists -eq $False) { 312 | $CreateFileShareScript = Join-Path $PSScriptRoot "createFileShare.ps1" 313 | Echo "Creating file share: $FileShareName" 314 | . $CreateFileShareScript -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey -ShareName $FileShareName 315 | Echo "Created file share: $FileShareName" 316 | } Else { 317 | Echo "Using existing file share: $FileShareName" 318 | } 319 | } Else { 320 | # Poll for existence of File Share 321 | WaitForFileShareToExist -StorageAccountName $StorageAccountName -StorageAccountKey $StorageAccountKey -FileShareName $FileShareName -TimeoutMinutes 30 322 | } 323 | 324 | # Mount File Share 325 | Echo "Mounting File Share: $FileShareName" 326 | . cmdkey /add:$FileShareHost /user:Azure\$StorageAccountName /pass:$StorageAccountKey 327 | . net use K: /delete 328 | . net use K: $MountPath 329 | [Environment]::SetEnvironmentVariable("SHARED_DRIVE", $MountPath, "Machine") 330 | Echo "Mounted File Share: $FileShareName" 331 | 332 | #****************************************************************************** 333 | # Create shared files 334 | #****************************************************************************** 335 | 336 | # Change directory to MATLAB script bin 337 | $Matlabroot = FindMatlabRoot 338 | Echo "Using matlab root $Matlabroot" 339 | $Mdcsdir = $Matlabroot + "\toolbox\distcomp\bin" 340 | Set-Location $Mdcsdir 341 | 342 | # Check if LicenseServer string is of the form port@host. For example "27000@netlm-server". 343 | If ($LicenseServer -and $LicenseServer -match '(?\d+)\@(?.+)') { 344 | $FlexPort = $Matches.Port 345 | $FlexMachineName = $Matches.Host 346 | Echo "Configuring worker MATLABs to checkout license using Network License Manager: $FlexPort@$FlexMachineName" 347 | $LicensesDir = Join-Path $Matlabroot "licenses" 348 | $LicenseFile = Join-Path $LicensesDir "network.lic" 349 | $LicenseText = "SERVER $FlexMachineName 123456789ABC $FlexPort`r`nUSE_SERVER" 350 | Set-Content -Path $LicenseFile -Value $LicenseText -NoNewline 351 | $LicenseSource = "flex" 352 | } Else { 353 | Echo "Configuring worker MATLABs to checkout license using Online Licensing" 354 | $LicenseSource = "mhlm" 355 | } 356 | 357 | $SharedFolder = "K:\cluster" 358 | If(-Not (Test-Path $SharedFolder)) { 359 | New-Item $SharedFolder -Type Directory -Force 360 | } 361 | $SharedSecretFile = Join-Path $SharedFolder "secret" 362 | $SharedCertFile = Join-Path $SharedFolder "cert" 363 | $SharedProfile = Join-Path $SharedFolder "$ClusterName.settings" 364 | $LocalCertFile = "cert" 365 | $LocalSharedSecretFile = "secret" 366 | 367 | If (0 -eq $Role.ToLower().CompareTo("headnode")) { 368 | If(-Not (Test-Path $SharedSecretFile)) { 369 | # Create shared secret for SSL connections 370 | Echo "Creating shared secret" 371 | $SharedSecretCommand = ".\createSharedSecret.bat -file $SharedSecretFile" 372 | Echo "RunCommand: $SharedSecretCommand" 373 | RunCommand $SharedSecretCommand 374 | } 375 | 376 | If(-Not (Test-Path $SharedCertFile)) { 377 | # Create client certificate 378 | Echo "Creating certificate" 379 | $GenerateCertificateCommand = ".\generateCertificate.bat -secretfile $SharedSecretFile -certfile $SharedCertFile" 380 | Echo "RunCommand: $GenerateCertificateCommand" 381 | RunCommand $GenerateCertificateCommand 382 | } 383 | 384 | If(-Not (Test-Path $SharedProfile)) { 385 | # Generate profile 386 | Echo "Creating cluster profile $SharedProfile" 387 | $CreateProfileCommand = ".\createProfile.bat -name $ClusterName -host $HeadnodeExternalHostname -outfile $SharedProfile -certfile $SharedCertFile" 388 | Echo "RunCommand: $CreateProfileCommand" 389 | RunCommand $CreateProfileCommand 390 | } 391 | } Else { 392 | # Workers simply wait for the required files in File Share 393 | WaitForPathToExist -Path $SharedSecretFile -TimeoutMinutes 10 394 | WaitForPathToExist -Path $SharedCertFile -TimeoutMinutes 10 395 | } 396 | 397 | # Copy file Share items to local disk 398 | Copy-Item $SharedSecretFile $LocalSharedSecretFile -Force 399 | Copy-Item $SharedCertFile $LocalCertFile -Force 400 | 401 | #****************************************************************************** 402 | # Start Cluster 403 | #****************************************************************************** 404 | 405 | # Uninstall any existing mdce service 406 | Echo "Uninstall any existing mdce service" 407 | $MdceUnInstallCommand = ".\mdce.bat uninstall -cleanPreserveJobs -disableelevate -checkpointbase $CheckpointDir" 408 | Echo "RunCommand: $MdceUnInstallCommand" 409 | RunCommand $MdceUnInstallCommand -ErrorAction SilentlyContinue 410 | 411 | # Install mdce service 412 | Echo "Install mdce service" 413 | $MdceInstallCommand = ".\mdce.bat install -cleanPreserveJobs -disableelevate -checkpointbase $CheckpointDir" 414 | Echo "RunCommand: $MdceInstallCommand" 415 | RunCommand $MdceInstallCommand 416 | 417 | # Start MDCE service. 418 | If(0 -eq $Role.ToLower().CompareTo("headnode")) { 419 | # For headnode use external name to be discoverable 420 | $HostnameToUse = $HeadnodeExternalHostname 421 | } Else { 422 | # For workers use internal name 423 | $HostnameToUse = $InternalHostname 424 | } 425 | 426 | $LogLevel = "2" 427 | $MdceCommand = "-cleanPreserveJobs" 428 | $MdceCommand = "-loglevel " + $LogLevel 429 | $MdceCommand = $MdceCommand + " -disableelevate" 430 | $MdceCommand = $MdceCommand + " -workerproxiespoolconnections" 431 | $MdceCommand = $MdceCommand + " -enablepeerlookup" 432 | $MdceCommand = $MdceCommand + " -hostname " + $HostnameToUse 433 | $MdceCommand = $MdceCommand + " -untrustedclients" 434 | $MdceCommand = $MdceCommand + " -usesecurecommunication" 435 | $MdceCommand = $MdceCommand + " -sharedsecretfile " + $LocalSharedSecretFile 436 | $MdceCommand = $MdceCommand + " -checkpointbase " + $CheckpointDir 437 | If ($LicenseSource -eq "mhlm") { 438 | $MdceCommand = $MdceCommand + " -usemhlm" 439 | } 440 | 441 | Echo "Start mdce service" 442 | $MdceStartCommand = ".\mdce.bat start $MdceCommand" 443 | Echo "RunCommand: $MdceStartCommand" 444 | RunCommand $MdceStartCommand 445 | 446 | # Install MJS - headnode only. 447 | If(0 -eq $Role.ToLower().CompareTo("headnode")) { 448 | # - Start MJS job manager 449 | $JobManagerCommand = "-name $ClusterName" 450 | $JobManagerCommand = $JobManagerCommand + " -certificate " + $LocalCertFile 451 | 452 | Echo "Starting job manager $ClusterName" 453 | $StartJobManagerCommand = ".\startjobmanager.bat $JobManagerCommand" 454 | Echo "RunCommand: $StartJobManagerCommand" 455 | RunCommand $StartJobManagerCommand 456 | } 457 | 458 | # Launch any workers 459 | # If NumWorkers = -1, set the number of workers to the number of cores. 460 | If($NumWorkers -eq -1) { 461 | $NumWorkers = (Get-WmiObject -Class win32_processor -Property "numberOfCores").NumberOfCores 462 | } 463 | 464 | If($NumWorkers -gt 0) { 465 | Echo "Starting workers (numworkers = $NumWorkers)" 466 | $StartWorkerCommand = ".\startworker.bat -name $InternalHostname -jobmanagerhost $HeadnodeExternalHostname -jobmanager $ClusterName -num $NumWorkers" 467 | Echo "RunCommand: $StartWorkerCommand" 468 | RunCommand $StartWorkerCommand 469 | } 470 | 471 | Set-Location $Currentdir 472 | StopTranscript 473 | Exit 474 | -------------------------------------------------------------------------------- /releases/R2019b/README.md: -------------------------------------------------------------------------------- 1 | # MATLAB Parallel Server on Azure 2 | 3 | ## Step 1. Launch the Template 4 | 5 | Click the **Deploy to Azure** button below to deploy the cloud resources on Azure. This will open the Azure Portal in your web browser. 6 | 7 | 8 | 9 | 10 | 11 | > Cluster Platform: Windows Server 2016 12 | 13 | > MATLAB Release: R2019b 14 | 15 | ## Step 2. Configure the Cloud Resources 16 | Clicking the Deploy to Azure button opens the "Custom deployment" page in your browser. You can configure the parameters on this page. It is easier to complete the steps if you position these instructions and the Azure Portal window side by side. 17 | 18 | 1. Specify and check the defaults for these resource parameters: 19 | 20 | | Parameter label | Description 21 | | ----------------------------------------------- | ----------- 22 | | **Subscription** | Choose an Azure subscription to use for purchasing resources. 23 | | **Resource group** | Choose a name for the resource group that will hold the resources. It is recommended to create a new resource group for each deployment. This allows all resources to be deleted simultaneously. 24 | | **Location** | Choose the region to start resources in. Ensure that you select a location which supports your requested instance types. To check which services are supported in each location, see [Products available by region](https://azure.microsoft.com/en-gb/regions/services/). 25 | | **Cluster Name** | Choose a name to use for the cluster. This name will be shown in MATLAB when connected to the cluster. 26 | | **Num Worker Nodes** | Choose the number of Azure instances to start for the workers. 27 | | **Num Workers Per Node** | Choose the number of MATLAB workers to start on each node. Specify one worker for every physical core. The number of vCPUs per physical core depends on the instance family type. See the [Azure documentation](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/acu) for details on vCPUs for each instance type. 28 | | **Database Volume Size** | The size of the data disk to use for the database. All job and task information, including input and output data will be stored on this volume and it should therefore have enough capacity to store the expected amount of data. If this parameter is set to 0 no volume will be created and the root volume of the instance will be used for the database. 29 | | **Headnode Instance Type** | Choose the Azure instance type to use for the head node, which will run the job manager. No workers will be started on this node, so this can be a smaller instance type than the worker nodes. The selected [virtual machine size](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes) must support premium storage. 30 | | **Worker Instance Type** | Choose the Azure instance type to use for the workers. The selected [virtual machine size](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/sizes) must support premium storage. 31 | | **Client IP Address** | This is the IP address range that will be allowed to access the resources. The format for this field is IP Address/Mask.

Example:

10.0.0.1/32
  • This is the public IP address which can be found by searching for "what is my ip address" on the web. The mask determines the number of IP addresses to include.
  • A mask of 32 is a single IP address.
  • Use a [CIDR calculator](https://www.ipaddressguide.com/cidr) if you need a range of more than one IP addresses.
  • You may need to contact your IT administrator to determine which address is appropriate.

32 | | **Admin Username** | Choose the admin username for all instances. This username is required when logging into any instance using remote desktop protocol. 33 | | **Admin Password** | Choose the admin password for all instances. This password is required when logging into any instance using remote desktop protocol. For the deployment to succeed, your password must meet [Azure's password requirements](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/faq#what-are-the-password-requirements-when-creating-a-vm-). 34 | | **License Manager for MATLAB connection string** | Optional License Manager for MATLAB string in the form \@\. If not specified, online licensing is used. If specified, the license manager must be accessible from the specified VPC and subnets. | 35 | 36 | **NOTE**: The port and hostname of the network license manager must be reachable from the worker MATLAB VMs. It is therefore recommended that you deploy your cluster into a subnet within the same virtual network as the network license manager. 37 | 38 | 2. Tick the box to accept the Azure Marketplace terms and conditions. 39 | 40 | 3. Click the **Purchase** button. 41 | 42 | When you click the **Purchase** button, the resources are created using Azure template deployments. Template deployment can take several minutes. 43 | 44 | # Step 3: Connect to Your Cluster From MATLAB 45 | 46 | 1. After clicking **Purchase** you will be taken to the Azure Portal Dashboard. To montior the progress of your deployment, select your resource group from the Resource Groups panel. Wait for the all **Deployments** to reach **Succeeded**. 47 | 2. Select the Storage Account ending with **storage**. The screen should look like the one in Figure 1. 48 | 49 | ![Resource Group On Completion](../../img/Deployment_Complete_Select_Storage.png) 50 | 51 | *Figure 1: Resource Group On Completion* 52 | 53 | 3. Select the Files container type. 54 | 4. Select the File Share named "shared". 55 | 5. Download the file, `cluster/.settings`, where NAME is the name of your MATLAB job scheduler. 56 | 6. Open MATLAB. 57 | 7. In the Parallel drop-down menu in the MATLAB toolstrip select **Create and Manage Clusters...**. 58 | 8. Click **Import**. 59 | 9. Select the downloaded profile and click open. 60 | 10. Click **Set as Default**. 61 | 11. (Optional) Validate your cluster by clicking the **Validate** button. 62 | 63 | After setting the cloud cluster as default, the next time you run a parallel language command (such as `parfor`, `spmd`, `parfeval` or `batch`) MATLAB connects to the cluster. The first time you connect, you will be prompted for your MathWorks account login. The first time you run a task on a worker it will take several minutes for the worker MATLAB to start. This delay is due to provisioning the instance disk. This is a one-time operation, and subsequent tasks begin much faster. 64 | 65 | Your cluster is now ready to use. It will remain running after you close MATLAB. 66 | 67 | **NOTE**: Use the profile and client IP address range to control access to your cloud resources. Anyone with this file can connect to your resources from a machine within the specified IP address range and run jobs on it. 68 | 69 | # Additional Information 70 | ## Delete Your Cloud Resources 71 | You can remove the Resource Group and all associated resources when you are done with them. Note that you cannot recover resources once they are deleted. After you delete the cloud resources you cannot use the downloaded profile again. 72 | 1. Login to the Azure Portal. 73 | 2. Select the Resource Group containing your resources. 74 | 3. Select the "Delete resource group" icon to destroy all resources deplyoyed in this group. 75 | 4. You will be prompted to enter the name of the resource group to confirm the deletion. 76 | 77 | ![Resource Group Delete](../../img/Resource_Group_Delete.png) 78 | 79 | ## Troubleshooting 80 | If your resource group fails to deploy, check the Deployments section of the Resource Group. It will indicate which resource deployments failed and allow you to navigate to the causing error message. 81 | 82 | If the resource group deployed successfully but you are unable to validate the cluster, you may need to view the logs on the instances to diagnose the error. The deployment logs are output to C:/Windows/Temp/MDCSLog*.txt on the instance nodes. The cluster logs are output to C:/Windows/Temp/MDCE/Log. 83 | 84 | ## Use Existing Virtual Network 85 | You can launch the reference architecture within an existing virtual network and subnet using the following template. 86 | 87 | 88 | 89 | 90 | 91 | > Cluster Platform: Windows Server 2016 92 | 93 | > MATLAB Release: R2019b 94 | 95 | This template requires the following two additional parameters: 96 | 97 | | Parameter label | Description 98 | | ------------------------------------------- | ----------- 99 | | **Virtual Network Resource ID** | The Resource ID of an existing virtual network to deploy your cluster into. You can find this under the Properties of your virtual network. 100 | | **Subnet Name** | The name of an existing subnet within your virtual network to deploy your cluster into. 101 | -------------------------------------------------------------------------------- /releases/R2020a/README.md: -------------------------------------------------------------------------------- 1 | # MATLAB Parallel Server on Azure 2 | 3 | ## Step 1. Launch the Template 4 | 5 | Click the **Deploy to Azure** button below to deploy the cloud resources on Azure. This will open the Azure Portal in your web browser. 6 | 7 | | Create Virtual Network | Use Existing Virtual Network | 8 | | --- | --- | 9 | | Use this option to deploy the resources in a new virtual network



| Use this option to deploy the resources in an existing virtual network



| 10 | 11 | > Cluster Platform: Windows Server 2016 12 | 13 | > MATLAB Release: R2020a 14 | 15 | ## Step 2. Configure the Cloud Resources 16 | Clicking the Deploy to Azure button opens the "Custom deployment" page in your browser. You can configure the parameters on this page. It is easier to complete the steps if you position these instructions and the Azure Portal window side by side. 17 | 18 | 1. Specify and check the defaults for these resource parameters: 19 | 20 | | Parameter label | Description | 21 | | --------------- | ----------- | 22 | | **Cluster Name** | A name to use for this cluster. This name will be shown in MATLAB as the cluster profile name. | 23 | | **Num Worker Nodes** | The number of Azure instances to start for the workers to run on. | 24 | | **Num Workers Per Node** | The number of MATLAB workers to start on each instance. Specify 1 worker for every 2 vCPUs, because this results in 1 worker per physical core. For example a Standard_D64s_v3 instance has 64 vCPUs, so can support 32 MATLAB workers. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes for details on vCPUs for each instance type. | 25 | | **Headnode Instance Type** | The Azure instance type to use for the head node, which will run the job manager. No workers will be started on this node, so this can be a smaller instance type than the worker nodes. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes. for a list of instance types. | 26 | | **Worker Instance Type** | The Azure instance type to use for the workers. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes for a list of instance types. | 27 | | **Database Volume Size** | The size of the volume in Gigabytes used to store the database files. If set to 0, a separate volume will not be created and the root volume will be used for the database. | 28 | | **Client IP Address** | The IP address range that can be used to access the cluster from MATLAB. This must be a valid IP CIDR range of the form x.x.x.x/x. Use the value /32 to restrict access to only your computer. | 29 | | **Admin Username** | Choose the admin username for all instances. This username is required when logging into any instance using remote desktop protocol. | 30 | | **Admin Password** | Choose the admin password for all instances. This password is required when logging into any instance using remote desktop protocol. For the deployment to succeed, your password must meet Azure's password requirements. See https://docs.microsoft.com/en-us/azure/virtual-machines/windows/faq#what-are-the-password-requirements-when-creating-a-vm- for information on the password requirements. | 31 | | **Virtual Network Resource ID** | The Resource ID of an existing virtual network to deploy your cluster into. You can find this under the Properties of your virtual network. Specify this parameter only when deploying with the Existing Virtual Network option. | 32 | | **Subnet Name** | The name of an existing subnet within your virtual network to deploy your cluster into. Specify this parameter only when deploying with the Existing Virtual Network option. | 33 | | **License Server** | Optional License Manager for MATLAB string in the form port@hostname. If not specified, online licensing is used. If specified, the license manager must be accessible from the specified virtual network and subnets. | 34 | 35 | 36 | **NOTE**: The port and hostname of the network license manager must be reachable from the worker MATLAB VMs. It is therefore recommended that you deploy your cluster into a subnet within the same virtual network as the network license manager. 37 | 38 | 2. Tick the box to accept the Azure Marketplace terms and conditions. 39 | 40 | 3. Click the **Purchase** button. 41 | 42 | When you click the **Purchase** button, the resources are created using Azure template deployments. Template deployment can take several minutes. 43 | 44 | # Step 3: Connect to Your Cluster From MATLAB 45 | 46 | 1. After clicking **Purchase** you will be taken to the Azure Portal Dashboard. To montior the progress of your deployment, select your resource group from the Resource Groups panel. Wait for the all **Deployments** to reach **Succeeded**. 47 | 2. Select the Storage Account ending with **storage**. The screen should look like the one in Figure 1. 48 | 49 | ![Resource Group On Completion](../../img/Deployment_Complete_Select_Storage.png) 50 | 51 | *Figure 1: Resource Group On Completion* 52 | 53 | 3. Select the Files container type. 54 | 4. Select the File Share named "shared". 55 | 5. Download the file, `cluster/.settings`, where NAME is the name of your MATLAB job scheduler. 56 | 6. Open MATLAB. 57 | 7. In the Parallel drop-down menu in the MATLAB toolstrip select **Create and Manage Clusters...**. 58 | 8. Click **Import**. 59 | 9. Select the downloaded profile and click open. 60 | 10. Click **Set as Default**. 61 | 11. (Optional) Validate your cluster by clicking the **Validate** button. 62 | 63 | After setting the cloud cluster as default, the next time you run a parallel language command (such as `parfor`, `spmd`, `parfeval` or `batch`) MATLAB connects to the cluster. The first time you connect, you will be prompted for your MathWorks account login. The first time you run a task on a worker it will take several minutes for the worker MATLAB to start. This delay is due to provisioning the instance disk. This is a one-time operation, and subsequent tasks begin much faster. 64 | 65 | Your cluster is now ready to use. It will remain running after you close MATLAB. 66 | 67 | **NOTE**: Use the profile and client IP address range to control access to your cloud resources. Anyone with this file can connect to your resources from a machine within the specified IP address range and run jobs on it. 68 | 69 | # Additional Information 70 | 71 | ## Port requirements for accessing MATLAB Parallel Server 72 | 73 | To access a MATLAB Parallel Server cluster from your client MATLAB, your client machine must be able to communicate on specific ports. Make sure that the network firewall allows the following outgoing connections: 74 | 75 | | Required ports | Description | 76 | | -------------- | ---------------------------------- | 77 | | TCP 27350 to 27358 + 4*N

TCP 443

TCP 3389 | Ports 27350 to 27358 + 4*N, where N is the maximum number of workers on a single node

HTTPS access to (at least) *.mathworks and *.microsoft.com

Remote Desktop Protocol to access to cluster nodes | 78 | 79 | *Table 1: Outgoing port requirements* 80 | 81 | ## Delete Your Cloud Resources 82 | You can remove the Resource Group and all associated resources when you are done with them. Note that you cannot recover resources once they are deleted. After you delete the cloud resources you cannot use the downloaded profile again. 83 | 1. Login to the Azure Portal. 84 | 2. Select the Resource Group containing your resources. 85 | 3. Select the "Delete resource group" icon to destroy all resources deplyoyed in this group. 86 | 4. You will be prompted to enter the name of the resource group to confirm the deletion. 87 | 88 | ![Resource Group Delete](../../img/Resource_Group_Delete.png) 89 | 90 | ## Troubleshooting 91 | If your resource group fails to deploy, check the Deployments section of the Resource Group. It will indicate which resource deployments failed and allow you to navigate to the causing error message. 92 | 93 | If the resource group deployed successfully but you are unable to validate the cluster, you may need to view the logs on the instances to diagnose the error. The deployment logs are output to C:/Windows/Temp/MDCSLog*.txt on the instance nodes. The cluster logs are output to C:/Windows/Temp/MDCE/Log. 94 | -------------------------------------------------------------------------------- /releases/R2020b/README.md: -------------------------------------------------------------------------------- 1 | # MATLAB Parallel Server on Azure 2 | 3 | ## Step 1. Launch the Template 4 | 5 | Click the **Deploy to Azure** button below to deploy the cloud resources on Azure. This will open the Azure Portal in your web browser. 6 | 7 | | Create Virtual Network | Use Existing Virtual Network | 8 | | --- | --- | 9 | | Use this option to deploy the resources in a new virtual network



| Use this option to deploy the resources in an existing virtual network



| 10 | 11 | > Cluster Platform: Windows Server 2016 12 | 13 | > MATLAB Release: R2020b 14 | 15 | ## Step 2. Configure the Cloud Resources 16 | Clicking the Deploy to Azure button opens the "Custom deployment" page in your browser. You can configure the parameters on this page. It is easier to complete the steps if you position these instructions and the Azure Portal window side by side. 17 | 18 | 1. Specify and check the defaults for these resource parameters: 19 | 20 | | Parameter label | Description | 21 | | --------------- | ----------- | 22 | | **Cluster Name** | A name to use for this cluster. This name will be shown in MATLAB as the cluster profile name. | 23 | | **Num Worker Nodes** | The number of Azure instances to start for the workers to run on. | 24 | | **Num Workers Per Node** | The number of MATLAB workers to start on each instance. Specify 1 worker for every 2 vCPUs, because this results in 1 worker per physical core. For example a Standard_D64s_v3 instance has 64 vCPUs, so can support 32 MATLAB workers. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes for details on vCPUs for each instance type. | 25 | | **Headnode Instance Type** | The Azure instance type to use for the head node, which will run the job manager. No workers will be started on this node, so this can be a smaller instance type than the worker nodes. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes. for a list of instance types. | 26 | | **Worker Instance Type** | The Azure instance type to use for the workers. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes for a list of instance types. | 27 | | **Database Volume Size** | The size of the volume in Gigabytes used to store the database files. If set to 0, a separate volume will not be created and the root volume will be used for the database. | 28 | | **Client IP Address** | The IP address range that can be used to access the cluster from MATLAB. This must be a valid IP CIDR range of the form x.x.x.x/x. Use the value /32 to restrict access to only your computer. | 29 | | **Admin Password** | Choose the admin password for the user "matlab".for all instances. This password is required when logging into any instance using remote desktop protocol. For the deployment to succeed, your password must meet Azure's password requirements. See https://docs.microsoft.com/en-us/azure/virtual-machines/windows/faq#what-are-the-password-requirements-when-creating-a-vm- for information on the password requirements. | 30 | | **Virtual Network Resource ID** | The Resource ID of an existing virtual network to deploy your cluster into. You can find this under the Properties of your virtual network. Specify this parameter only when deploying with the Existing Virtual Network option. | 31 | | **Subnet Name** | The name of an existing subnet within your virtual network to deploy your cluster into. Specify this parameter only when deploying with the Existing Virtual Network option. | 32 | | **License Server** | Optional License Manager for MATLAB string in the form port@hostname. If not specified, online licensing is used. If specified, the license manager must be accessible from the specified virtual network and subnets. | 33 | 34 | 35 | **NOTE**: The port and hostname of the network license manager must be reachable from the worker MATLAB VMs. It is therefore recommended that you deploy your cluster into a subnet within the same virtual network as the network license manager. 36 | 37 | 2. Tick the box to accept the Azure Marketplace terms and conditions. 38 | 39 | 3. Click the **Create** button. 40 | 41 | When you click the **Create** button, the resources are created using Azure template deployments. Template deployment can take several minutes. 42 | 43 | # Step 3: Connect to Your Cluster From MATLAB 44 | 45 | 1. After clicking **Create** you will be taken to the Azure Portal Dashboard. To montior the progress of your deployment, select your resource group from the Resource Groups panel. Wait for the all **Deployments** to reach **Succeeded**. 46 | 2. Select the Storage Account ending with **storage**. The screen should look like the one in Figure 1. 47 | 48 | ![Resource Group On Completion](../../img/Deployment_Complete_Select_Storage.png) 49 | 50 | *Figure 1: Resource Group On Completion* 51 | 52 | 3. Select the Files container type. 53 | 4. Select the File Share named "shared". 54 | 5. Download the file, `cluster/.settings`, where NAME is the name of your MATLAB job scheduler. 55 | 6. Open MATLAB. 56 | 7. In the Parallel drop-down menu in the MATLAB toolstrip select **Create and Manage Clusters...**. 57 | 8. Click **Import**. 58 | 9. Select the downloaded profile and click open. 59 | 10. Click **Set as Default**. 60 | 11. (Optional) Validate your cluster by clicking the **Validate** button. 61 | 62 | After setting the cloud cluster as default, the next time you run a parallel language command (such as `parfor`, `spmd`, `parfeval` or `batch`) MATLAB connects to the cluster. The first time you connect, you will be prompted for your MathWorks account login. The first time you run a task on a worker it will take several minutes for the worker MATLAB to start. This delay is due to provisioning the instance disk. This is a one-time operation, and subsequent tasks begin much faster. 63 | 64 | Your cluster is now ready to use. It will remain running after you close MATLAB. 65 | 66 | **NOTE**: Use the profile and client IP address range to control access to your cloud resources. Anyone with this file can connect to your resources from a machine within the specified IP address range and run jobs on it. 67 | 68 | # Additional Information 69 | 70 | ## Port requirements for accessing MATLAB Parallel Server 71 | 72 | To access a MATLAB Parallel Server cluster from your client MATLAB, your client machine must be able to communicate on specific ports. Make sure that the network firewall allows the following outgoing connections: 73 | 74 | | Required ports | Description | 75 | | -------------- | ---------------------------------- | 76 | | TCP 27350 to 27358 + 4*N

TCP 443

TCP 3389 | Ports 27350 to 27358 + 4*N, where N is the maximum number of workers on a single node

HTTPS access to (at least) *.mathworks and *.microsoft.com

Remote Desktop Protocol to access to cluster nodes | 77 | 78 | *Table 1: Outgoing port requirements* 79 | 80 | ## Delete Your Cloud Resources 81 | You can remove the Resource Group and all associated resources when you are done with them. Note that you cannot recover resources once they are deleted. After you delete the cloud resources you cannot use the downloaded profile again. 82 | 1. Login to the Azure Portal. 83 | 2. Select the Resource Group containing your resources. 84 | 3. Select the "Delete resource group" icon to destroy all resources deplyoyed in this group. 85 | 4. You will be prompted to enter the name of the resource group to confirm the deletion. 86 | 87 | ![Resource Group Delete](../../img/Resource_Group_Delete.png) 88 | 89 | ## Troubleshooting 90 | If your resource group fails to deploy, check the Deployments section of the Resource Group. It will indicate which resource deployments failed and allow you to navigate to the causing error message. 91 | 92 | If the resource group deployed successfully but you are unable to validate the cluster, you may need to view the logs on the instances to diagnose the error. The deployment logs are output to C:/Windows/Temp/MDCSLog*.txt on the instance nodes. The cluster logs are output to C:/Windows/Temp/MDCE/Log. 93 | -------------------------------------------------------------------------------- /releases/R2021a/README.md: -------------------------------------------------------------------------------- 1 | # MATLAB Parallel Server on Azure (Windows VM) 2 | 3 | ## Step 1. Launch the Template 4 | 5 | Click the **Deploy to Azure** button below to deploy the cloud resources on Azure. This will open the Azure Portal in your web browser. 6 | 7 | | Create Virtual Network | Use Existing Virtual Network | 8 | | --- | --- | 9 | | Use this option to deploy the resources in a new virtual network



| Use this option to deploy the resources in an existing virtual network



| 10 | 11 | > Cluster Platform: Windows Server 2016 12 | 13 | > MATLAB Release: R2021a 14 | 15 | ## Step 2. Configure the Cloud Resources 16 | Clicking the Deploy to Azure button opens the "Custom deployment" page in your browser. You can configure the parameters on this page. It is easier to complete the steps if you position these instructions and the Azure Portal window side by side. 17 | 18 | 1. Specify and check the defaults for these resource parameters: 19 | 20 | | Parameter label | Description | 21 | | --------------- | ----------- | 22 | | **Name** | A name to use for this cluster. This name will be shown in MATLAB as the cluster profile name. | 23 | | **Worker Nodes** | The number of Azure instances to start for the workers to run on. | 24 | | **Workers Per Node** | The number of MATLAB workers to start on each instance. Specify 1 worker for every 2 vCPUs, because this results in 1 worker per physical core. For example a Standard_D64s_v3 instance has 64 vCPUs, so can support 32 MATLAB workers. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes for details on vCPUs for each instance type. | 25 | | **Node Vm Size** | The Azure instance type to use for the head node, which will run the job manager. No workers will be started on this node, so this can be a smaller instance type than the worker nodes. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes. for a list of instance types. | 26 | | **Vm Size** | The Azure instance type to use for the workers. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes for a list of instance types. | 27 | | **Volume Size** | The size of the volume in Gigabytes used to store the database files. If set to 0, a separate volume will not be created and the root volume will be used for the database. | 28 | | **IP Address** | The IP address range that can be used to access the cluster from MATLAB. This must be a valid IP CIDR range of the form x.x.x.x/x. Use the value /32 to restrict access to only your computer. | 29 | | **Password** | Choose the admin password for the user "matlab" for all instances. This password is required when logging into any instance using remote desktop protocol. For the deployment to succeed, your password must meet Azure's password requirements. See https://docs.microsoft.com/en-us/azure/virtual-machines/windows/faq#what-are-the-password-requirements-when-creating-a-vm- for information on the password requirements. | 30 | | **Network Resource ID** | The Resource ID of an existing virtual network to deploy your cluster into. You can find this under the Properties of your virtual network. | 31 | | **Name** | The name of an existing subnet within your virtual network to deploy your cluster into. | 32 | | **Server** | Optional License Manager for MATLAB string in the form port@hostname. If not specified, online licensing is used. If specified, the license manager must be accessible from the specified virtual network and subnets. | 33 | 34 | 35 | **NOTE**: If you are using network license manager, the port and hostname of the network license manager must be reachable from the MATLAB Parallel Server VMs. It is therefore recommended that you deploy into a subnet within the same virtual network as the network license manager. 36 | 37 | 2. Tick the box to accept the Azure Marketplace terms and conditions. 38 | 39 | 3. Click the **Create** button. 40 | 41 | When you click the **Create** button, the resources are created using Azure template deployments. Template deployment can take several minutes. 42 | 43 | # Step 3: Connect to Your Cluster From MATLAB 44 | 45 | 1. After clicking **Create** you will be taken to the Azure Portal Dashboard. To montior the progress of your deployment, select your resource group from the Resource Groups panel. Wait for the all **Deployments** to reach **Succeeded**. 46 | 2. Select the Storage Account ending with **storage**. The screen should look like the one in Figure 1. 47 | 48 | ![Resource Group On Completion](../../img/Deployment_Complete_Select_Storage.png) 49 | 50 | *Figure 1: Resource Group On Completion* 51 | 52 | 3. Select the Files container type. 53 | 4. Select the File Share named "shared". 54 | 5. Download the file, `cluster/.settings`, where NAME is the name of your MATLAB job scheduler. 55 | 6. Open MATLAB. 56 | 7. In the Parallel drop-down menu in the MATLAB toolstrip select **Create and Manage Clusters...**. 57 | 8. Click **Import**. 58 | 9. Select the downloaded profile and click open. 59 | 10. Click **Set as Default**. 60 | 11. (Optional) Validate your cluster by clicking the **Validate** button. 61 | 62 | After setting the cloud cluster as default, the next time you run a parallel language command (such as `parfor`, `spmd`, `parfeval` or `batch`) MATLAB connects to the cluster. The first time you connect, you will be prompted for your MathWorks account login. The first time you run a task on a worker it will take several minutes for the worker MATLAB to start. This delay is due to provisioning the instance disk. This is a one-time operation, and subsequent tasks begin much faster. 63 | 64 | Your cluster is now ready to use. It will remain running after you close MATLAB. 65 | 66 | **NOTE**: Use the profile and client IP address range to control access to your cloud resources. Anyone with this file can connect to your resources from a machine within the specified IP address range and run jobs on it. 67 | 68 | # Additional Information 69 | 70 | ## Port requirements for accessing MATLAB Parallel Server 71 | 72 | To access a MATLAB Parallel Server cluster from your client MATLAB, your client machine must be able to communicate on specific ports. Make sure that the network firewall allows the following outgoing connections: 73 | 74 | | Required ports | Description | 75 | | -------------- | ---------------------------------- | 76 | | TCP 27350 to 27358 + 4*N

TCP 443

TCP 3389 | Ports 27350 to 27358 + 4*N, where N is the maximum number of workers on a single node

HTTPS access to (at least) *.mathworks and *.microsoft.com

Remote Desktop Protocol to access to cluster nodes | 77 | 78 | *Table 1: Outgoing port requirements* 79 | 80 | ## Delete Your Cloud Resources 81 | You can remove the Resource Group and all associated resources when you are done with them. Note that you cannot recover resources once they are deleted. After you delete the cloud resources you cannot use the downloaded profile again. 82 | 1. Login to the Azure Portal. 83 | 2. Select the Resource Group containing your resources. 84 | 3. Select the "Delete resource group" icon to destroy all resources deplyoyed in this group. 85 | 4. You will be prompted to enter the name of the resource group to confirm the deletion. 86 | 87 | ![Resource Group Delete](../../img/Resource_Group_Delete.png) 88 | 89 | ## Troubleshooting 90 | If your resource group fails to deploy, check the Deployments section of the Resource Group. It will indicate which resource deployments failed and allow you to navigate to the causing error message. 91 | 92 | If the resource group deployed successfully but you are unable to validate the cluster, you may need to view the logs on the instances to diagnose the error. The deployment logs are output to C:/Windows/Temp/MDCSLog*.txt on the instance nodes. The cluster logs are output to C:/Windows/Temp/MDCE/Log. 93 | -------------------------------------------------------------------------------- /releases/R2021b/README.md: -------------------------------------------------------------------------------- 1 | # MATLAB Parallel Server on Azure (Windows VM) 2 | 3 | ## Step 1. Launch the Template 4 | 5 | Click the **Deploy to Azure** button below to deploy the cloud resources on Azure. This will open the Azure Portal in your web browser. 6 | 7 | | Create Virtual Network | Use Existing Virtual Network | 8 | | --- | --- | 9 | | Use this option to deploy the resources in a new virtual network



| Use this option to deploy the resources in an existing virtual network



| 10 | 11 | > Cluster Platform: Windows Server 2019 12 | 13 | > MATLAB Release: R2021b 14 | 15 | ## Step 2. Configure the Cloud Resources 16 | Clicking the Deploy to Azure button opens the "Custom deployment" page in your browser. You can configure the parameters on this page. It is easier to complete the steps if you position these instructions and the Azure Portal window side by side. 17 | 18 | 1. Specify and check the defaults for these resource parameters: 19 | 20 | | Parameter label | Description | 21 | | --------------- | ----------- | 22 | | **Name** | A name to use for this cluster. This name will be shown in MATLAB as the cluster profile name. | 23 | | **Worker Nodes** | The number of Azure instances to start for the workers to run on. | 24 | | **Workers Per Node** | The number of MATLAB workers to start on each instance. Specify 1 worker for every 2 vCPUs, because this results in 1 worker per physical core. For example a Standard_D64s_v3 instance has 64 vCPUs, so can support 32 MATLAB workers. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes for details on vCPUs for each instance type. | 25 | | **Node Vm Size** | The Azure instance type to use for the head node, which will run the job manager. No workers will be started on this node, so this can be a smaller instance type than the worker nodes. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes. for a list of instance types. | 26 | | **Vm Size** | The Azure instance type to use for the workers. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes for a list of instance types. | 27 | | **Volume Size** | The size of the volume in Gigabytes used to store the database files. If set to 0, a separate volume will not be created and the root volume will be used for the database. | 28 | | **IP Address** | The IP address range that can be used to access the cluster from MATLAB. This must be a valid IP CIDR range of the form x.x.x.x/x. Use the value /32 to restrict access to only your computer. | 29 | | **Password** | Choose the admin password for the user "matlab" for all instances. This password is required when logging into any instance using remote desktop protocol. For the deployment to succeed, your password must meet Azure's password requirements. See https://docs.microsoft.com/en-us/azure/virtual-machines/windows/faq#what-are-the-password-requirements-when-creating-a-vm- for information on the password requirements. | 30 | | **Network Resource ID** | The Resource ID of an existing virtual network to deploy your cluster into. You can find this under the Properties of your virtual network. | 31 | | **Name** | The name of an existing subnet within your virtual network to deploy your cluster into. | 32 | | **Server** | Optional License Manager for MATLAB string in the form port@hostname. If not specified, online licensing is used. If specified, the license manager must be accessible from the specified virtual network and subnets. | 33 | 34 | 35 | **NOTE**: If you are using network license manager, the port and hostname of the network license manager must be reachable from the MATLAB Parallel Server VMs. It is therefore recommended that you deploy into a subnet within the same virtual network as the network license manager. 36 | 37 | 2. Tick the box to accept the Azure Marketplace terms and conditions. 38 | 39 | 3. Click the **Create** button. 40 | 41 | When you click the **Create** button, the resources are created using Azure template deployments. Template deployment can take several minutes. 42 | 43 | # Step 3: Connect to Your Cluster From MATLAB 44 | 45 | 1. After clicking **Create** you will be taken to the Azure Portal Dashboard. To montior the progress of your deployment, select your resource group from the Resource Groups panel. Wait for the all **Deployments** to reach **Succeeded**. 46 | 2. Select the Storage Account ending with **storage**. The screen should look like the one in Figure 1. 47 | 48 | ![Resource Group On Completion](../../img/Deployment_Complete_Select_Storage.png) 49 | 50 | *Figure 1: Resource Group On Completion* 51 | 52 | 3. Select the Files container type. 53 | 4. Select the File Share named "shared". 54 | 5. Download the file, `cluster/.settings`, where NAME is the name of your MATLAB job scheduler. 55 | 6. Open MATLAB. 56 | 7. In the Parallel drop-down menu in the MATLAB toolstrip select **Create and Manage Clusters...**. 57 | 8. Click **Import**. 58 | 9. Select the downloaded profile and click open. 59 | 10. Click **Set as Default**. 60 | 11. (Optional) Validate your cluster by clicking the **Validate** button. 61 | 62 | After setting the cloud cluster as default, the next time you run a parallel language command (such as `parfor`, `spmd`, `parfeval` or `batch`) MATLAB connects to the cluster. The first time you connect, you will be prompted for your MathWorks account login. The first time you run a task on a worker it will take several minutes for the worker MATLAB to start. This delay is due to provisioning the instance disk. This is a one-time operation, and subsequent tasks begin much faster. 63 | 64 | Your cluster is now ready to use. It will remain running after you close MATLAB. 65 | 66 | **NOTE**: Use the profile and client IP address range to control access to your cloud resources. Anyone with this file can connect to your resources from a machine within the specified IP address range and run jobs on it. 67 | 68 | # Additional Information 69 | 70 | ## Port requirements for accessing MATLAB Parallel Server 71 | 72 | To access a MATLAB Parallel Server cluster from your client MATLAB, your client machine must be able to communicate on specific ports. Make sure that the network firewall allows the following outgoing connections: 73 | 74 | | Required ports | Description | 75 | | -------------- | ----------- | 76 | | TCP 27350 to 27358 + 4*N | Ports 27350 to 27358 + 4*N, where N is the maximum number of workers on a single node | 77 | | TCP 443 | HTTPS access to (at least) *.mathworks and *.microsoft.com | 78 | | TCP 3389 | Remote Desktop Protocol to access to cluster nodes | 79 | 80 | *Table 1: Outgoing port requirements* 81 | 82 | ## Delete Your Cloud Resources 83 | You can remove the Resource Group and all associated resources when you are done with them. Note that you cannot recover resources once they are deleted. After you delete the cloud resources you cannot use the downloaded profile again. 84 | 1. Login to the Azure Portal. 85 | 2. Select the Resource Group containing your resources. 86 | 3. Select the "Delete resource group" icon to destroy all resources deployed in this group. 87 | 4. You will be prompted to enter the name of the resource group to confirm the deletion. 88 | 89 | ![Resource Group Delete](../../img/Resource_Group_Delete.png) 90 | 91 | ## Troubleshooting 92 | If your resource group fails to deploy, check the Deployments section of the Resource Group. It will indicate which resource deployments failed and allow you to navigate to the causing error message. 93 | 94 | If the resource group deployed successfully but you are unable to validate the cluster, you may need to view the logs on the instances to diagnose the error. The deployment logs are output to C:/Windows/Temp/MDCSLog*.txt on the instance nodes. The cluster logs are output to C:/Windows/Temp/MDCE/Log. -------------------------------------------------------------------------------- /releases/R2022a/README.md: -------------------------------------------------------------------------------- 1 | # MATLAB Parallel Server on Azure (Windows VM) 2 | 3 | ## Step 1. Launch the Template 4 | 5 | Click the **Deploy to Azure** button below to deploy the cloud resources on Azure. This will open the Azure Portal in your web browser. 6 | 7 | | Create Virtual Network | Use Existing Virtual Network | 8 | | --- | --- | 9 | | Use this option to deploy the resources in a new virtual network



| Use this option to deploy the resources in an existing virtual network



| 10 | 11 | > Cluster Platform: Windows Server 2019 12 | 13 | > MATLAB Release: R2022a 14 | 15 | ## Step 2. Configure the Cloud Resources 16 | Clicking the Deploy to Azure button opens the "Custom deployment" page in your browser. You can configure the parameters on this page. It is easier to complete the steps if you position these instructions and the Azure Portal window side by side. 17 | 18 | 1. Specify and check the defaults for these resource parameters: 19 | 20 | | Parameter label | Description | 21 | | --------------- | ----------- | 22 | | **Cluster Name** | Name to use for this cluster. This name will be shown in MATLAB as the cluster profile name. | 23 | | **Num Worker Nodes** | The number of Azure instances to start for the workers to run on. | 24 | | **Num Workers Per Node** | The number of MATLAB workers to start on each instance. Specify 1 worker for every 2 vCPUs, because this results in 1 worker per physical core. For example a Standard_D64s_v3 instance has 64 vCPUs, so can support 32 MATLAB workers. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes for details on vCPUs for each instance type. | 25 | | **Head Node Vm Size** | The Azure instance type to use for the headnode, which will run the job manager. No workers will be started on this node, so this can be a smaller instance type than the worker nodes. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes. for a list of instance types. | 26 | | **Worker Vm Size** | The Azure instance type to use for the workers. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes for a list of instance types. | 27 | | **Database Volume Size** | The size of the volume in Gigabytes used to store the database files. If set to 0, a separate volume will not be created and the root volume will be used for the database. | 28 | | **Client IP Address** | The IP address range that can be used to access the cluster from MATLAB. This must be a valid IP CIDR range of the form x.x.x.x/x. Use the value <your_public_ip_address>/32 to restrict access to only your computer. | 29 | | **Admin Password** | Choose the admin password for the user "matlab" for all instances. This password is required when logging into any instance using remote desktop protocol. For the deployment to succeed, your password must meet Azure's password requirements. See https://docs.microsoft.com/en-us/azure/virtual-machines/windows/faq#what-are-the-password-requirements-when-creating-a-vm- for information on the password requirements. | 30 | | **Virtual Network Resource ID** | The Resource ID of an existing virtual network to deploy your cluster into. You can find this under the Properties of your virtual network. Specify this parameter only when deploying with the Existing Virtual Network option. | 31 | | **Subnet Name** | The name of an existing subnet within your virtual network to deploy your cluster into. Specify this parameter only when deploying with the Existing Virtual Network option. | 32 | | **License Server** | Optional License Manager for MATLAB string in the form port@hostname. If not specified, online licensing is used. If specified, the license manager must be accessible from the specified virtual network and subnets. | 33 | 34 | 35 | **NOTE**: If you are using network license manager, the port and hostname of the network license manager must be reachable from the MATLAB Parallel Server VMs. It is therefore recommended that you deploy into a subnet within the same virtual network as the network license manager. 36 | 37 | 2. Tick the box to accept the Azure Marketplace terms and conditions. 38 | 39 | 3. Click the **Create** button. 40 | 41 | When you click the **Create** button, the resources are created using Azure template deployments. Template deployment can take several minutes. 42 | 43 | # Step 3: Connect to Your Cluster From MATLAB 44 | 45 | 1. After clicking **Create** you will be taken to the Azure Portal Dashboard. To montior the progress of your deployment, select your resource group from the Resource Groups panel. Wait for the all **Deployments** to reach **Succeeded**. 46 | 2. Select the Storage Account ending with **storage**. The screen should look like the one in Figure 1. 47 | 48 | ![Resource Group On Completion](../../img/Deployment_Complete_Select_Storage.png) 49 | 50 | *Figure 1: Resource Group On Completion* 51 | 52 | 3. Select the Files container type. 53 | 4. Select the File Share named "shared". 54 | 5. Download the file, `cluster/.settings`, where NAME is the name of your MATLAB job scheduler. 55 | 6. Open MATLAB. 56 | 7. In the Parallel drop-down menu in the MATLAB toolstrip select **Create and Manage Clusters...**. 57 | 8. Click **Import**. 58 | 9. Select the downloaded profile and click open. 59 | 10. Click **Set as Default**. 60 | 11. (Optional) Validate your cluster by clicking the **Validate** button. 61 | 62 | After setting the cloud cluster as default, the next time you run a parallel language command (such as `parfor`, `spmd`, `parfeval` or `batch`) MATLAB connects to the cluster. The first time you connect, you will be prompted for your MathWorks account login. The first time you run a task on a worker it will take several minutes for the worker MATLAB to start. This delay is due to provisioning the instance disk. This is a one-time operation, and subsequent tasks begin much faster. 63 | 64 | Your cluster is now ready to use. It will remain running after you close MATLAB. 65 | 66 | **NOTE**: Use the profile and client IP address range to control access to your cloud resources. Anyone with this file can connect to your resources from a machine within the specified IP address range and run jobs on it. 67 | 68 | # Additional Information 69 | 70 | ## Port requirements for accessing MATLAB Parallel Server 71 | 72 | To access a MATLAB Parallel Server cluster from your client MATLAB, your client machine must be able to communicate on specific ports. Make sure that the network firewall allows the following outgoing connections: 73 | 74 | | Required ports | Description | 75 | | -------------- | ----------- | 76 | | TCP 27350 to 27358 + 4*N | Ports 27350 to 27358 + 4*N, where N is the maximum number of workers on a single node | 77 | | TCP 443 | HTTPS access to (at least) *.mathworks and *.microsoft.com | 78 | | TCP 3389 | Remote Desktop Protocol to access to cluster nodes | 79 | 80 | *Table 1: Outgoing port requirements* 81 | 82 | ## Delete Your Cloud Resources 83 | You can remove the Resource Group and all associated resources when you are done with them. Note that you cannot recover resources once they are deleted. After you delete the cloud resources you cannot use the downloaded profile again. 84 | 1. Login to the Azure Portal. 85 | 2. Select the Resource Group containing your resources. 86 | 3. Select the "Delete resource group" icon to destroy all resources deployed in this group. 87 | 4. You will be prompted to enter the name of the resource group to confirm the deletion. 88 | 89 | ![Resource Group Delete](../../img/Resource_Group_Delete.png) 90 | 91 | ## Troubleshooting 92 | If your resource group fails to deploy, check the Deployments section of the Resource Group. It will indicate which resource deployments failed and allow you to navigate to the causing error message. 93 | 94 | If the resource group deployed successfully but you are unable to validate the cluster, you may need to view the logs on the instances to diagnose the error. The deployment logs are output to C:/Windows/Temp/MDCSLog*.txt on the instance nodes. The cluster logs are output to C:/Windows/Temp/MDCE/Log. -------------------------------------------------------------------------------- /releases/R2022a/azuredeploy-R2022a.json: -------------------------------------------------------------------------------- 1 | { 2 | "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#", 3 | "contentVersion": "1.0.0.0", 4 | "parameters": { 5 | "clusterName": { 6 | "type": "string", 7 | "defaultValue": "myCluster", 8 | "minLength": 1, 9 | "maxLength": 61, 10 | "metadata": { 11 | "description": "Name to use for this cluster. This name will be shown in MATLAB as the cluster profile name." 12 | } 13 | }, 14 | "numWorkerNodes": { 15 | "type": "Int", 16 | "defaultValue": 2, 17 | "minValue": 0, 18 | "maxValue": 1000, 19 | "metadata": { 20 | "description": "The number of Azure instances to start for the workers to run on." 21 | } 22 | }, 23 | "numWorkersPerNode": { 24 | "type": "Int", 25 | "defaultValue": 2, 26 | "minValue": 1, 27 | "maxValue": 1000, 28 | "metadata": { 29 | "description": "The number of MATLAB workers to start on each instance. Specify 1 worker for every 2 vCPUs, because this results in 1 worker per physical core. For example a Standard_D64s_v3 instance has 64 vCPUs, so can support 32 MATLAB workers. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes for details on vCPUs for each instance type." 30 | } 31 | }, 32 | "headNodeVmSize": { 33 | "type": "string", 34 | "defaultValue": "Standard_D4s_v3", 35 | "metadata": { 36 | "description": "The Azure instance type to use for the headnode, which will run the job manager. No workers will be started on this node, so this can be a smaller instance type than the worker nodes. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes. for a list of instance types." 37 | } 38 | }, 39 | "workerVmSize": { 40 | "type": "string", 41 | "defaultValue": "Standard_F4s_v2", 42 | "metadata": { 43 | "description": "The Azure instance type to use for the workers. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes for a list of instance types." 44 | } 45 | }, 46 | "databaseVolumeSize": { 47 | "type": "Int", 48 | "defaultValue": 100, 49 | "minValue": 0, 50 | "maxValue": 1023, 51 | "metadata": { 52 | "description": "The size of the volume in Gigabytes used to store the database files. If set to 0, a separate volume will not be created and the root volume will be used for the database." 53 | } 54 | }, 55 | "clientIPAddress": { 56 | "minLength": 1, 57 | "type": "String", 58 | "metadata": { 59 | "description": "The IP address range that can be used to access the cluster from MATLAB. This must be a valid IP CIDR range of the form x.x.x.x/x. Use the value <your_public_ip_address>/32 to restrict access to only your computer." 60 | } 61 | }, 62 | "adminPassword": { 63 | "type": "SecureString", 64 | "metadata": { 65 | "description": "Choose the admin password for the user \"matlab\" for all instances. This password is required when logging into any instance using remote desktop protocol. For the deployment to succeed, your password must meet Azure's password requirements. See https://docs.microsoft.com/en-us/azure/virtual-machines/windows/faq#what-are-the-password-requirements-when-creating-a-vm- for information on the password requirements." 66 | } 67 | }, 68 | "licenseServer": { 69 | "defaultValue": "mhlm", 70 | "type": "String", 71 | "metadata": { 72 | "description": "Optional License Manager for MATLAB string in the form port@hostname. If not specified, online licensing is used. If specified, the license manager must be accessible from the specified virtual network and subnets." 73 | } 74 | } 75 | }, 76 | "variables": { 77 | "adminUserName": "matlab", 78 | "namingPrefix": "[toLower(take(parameters('clusterName'), 15))]", 79 | "ArtifactsBase": "C:/Windows/Temp/", 80 | "addressPrefix": "10.0.0.0/16", 81 | "subnetPrefix": "10.0.0.0/24", 82 | "newOrExistingVirtualNetwork": "New", 83 | "virtualNetworkCreateName": "[concat(variables('namingPrefix'), '-vnet')]", 84 | "subnetCreateName": "[concat(variables('namingPrefix'), '-subnet')]", 85 | "virtualNetworkName": "[if(equals(variables('newOrExistingVirtualNetwork'),'Existing'),last(split(parameters('virtualNetworkResourceID'),'/')),variables('virtualNetworkCreateName'))]", 86 | "subnetName": "[if(equals(variables('newOrExistingVirtualNetwork'),'Existing'),parameters('subnetName'),variables('subnetCreateName'))]", 87 | "subnetRef": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('virtualNetworkName'), variables('subnetName'))]", 88 | "subnet": "[if(equals(variables('newOrExistingVirtualNetwork'),'New'),variables('subnetRef'),concat(parameters('virtualNetworkResourceID'), '/subnets/', parameters('subnetName')))]", 89 | "cuaTrackingNum": "pid-bcfd52e4-d2d7-4ca3-ba20-9a37f7f85c5f-partnercenter", 90 | "headnodeMachineName": "[concat(take(variables('namingPrefix'), 6), '-headnode')]", 91 | "vmssName": "[concat(take(variables('namingPrefix'), 4), '-vmss')]", 92 | "publisher": "mathworks-inc", 93 | "offer": "matlab-parallel-server-ref-arch-vm-win-offer", 94 | "sku": "matlab-parallel-server-ref-arch-r2022a-vm-win-sku", 95 | "version": "1.0.3", 96 | "imageReference": { 97 | "publisher": "[variables('publisher')]", 98 | "offer": "[variables('offer')]", 99 | "sku": "[variables('sku')]", 100 | "version": "[variables('version')]" 101 | }, 102 | "plan": { 103 | "name": "[variables('sku')]", 104 | "publisher": "[variables('publisher')]", 105 | "product": "[variables('offer')]" 106 | }, 107 | "osDisk": { 108 | "createOption": "fromImage", 109 | "managedDisk": { 110 | "storageAccountType": "Premium_LRS" 111 | } 112 | }, 113 | "storageProfileWithoutDataDisk": { 114 | "imageReference": "[variables('imageReference')]", 115 | "osDisk": "[variables('osDisk')]" 116 | }, 117 | "storageProfileWithDataDisk": { 118 | "imageReference": "[variables('imageReference')]", 119 | "osDisk": "[variables('osDisk')]", 120 | "dataDisks": [ 121 | { 122 | "name": "[concat(variables('headnodeMachineName'), '-database')]", 123 | "diskSizeGB": "[parameters('databaseVolumeSize')]", 124 | "lun": 0, 125 | "createOption": "Empty", 126 | "caching": "None" 127 | } 128 | ] 129 | }, 130 | "publicIPPostfix": "pub", 131 | "headnodeIPName": "[concat(variables('namingPrefix'), variables('publicIPPostfix'))]", 132 | "dnsLabelPrefix": "[concat('azure-', uniqueString(resourceGroup().id))]", 133 | "publicIpTimeout": "20", 134 | "networkInterfacePostfix": "nic", 135 | "publicIpConfigPostfix": "ipConfig", 136 | "networkSecurityGroupName": "[concat(variables('namingPrefix'), '-nsg')]", 137 | "totalPorts": "[add(7, mul(4, parameters('numWorkersPerNode')))]", 138 | "startPort": 27350, 139 | "finalPort": "[add(variables('startPort'), variables('totalPorts'))]", 140 | "storageAccountName": "[concat(take(concat(variables('namingPrefix'), uniquestring(resourceGroup().id)), 17), 'storage')]", 141 | "storageAccountType": "Standard_LRS", 142 | "scriptsUrl": "C:/Windows/Temp/", 143 | "startScript": "startmjs.ps1", 144 | "fileUris": [ 145 | "[concat(variables('scriptsUrl'), variables('startScript'))]" 146 | ], 147 | "scriptExe": "[concat(variables('ArtifactsBase'), variables('startScript'))]", 148 | "installCommand": "[concat('powershell \"Start-Sleep -Seconds 60\"; powershell -ExecutionPolicy Unrestricted -File ', variables('scriptExe'), ' -clusterName ', parameters('clusterName'), ' -HeadnodeInternalHostname ', variables('headnodeMachineName'), ' -licenseServer ', parameters('licenseServer'))]", 149 | "installHeadnodeCommand": "[concat(variables('installCommand'), ' -Role headnode -NumWorkers 0')]", 150 | "installWorkerCommand": "[concat(variables('installCommand'), ' -Role worker -NumWorkers ', parameters('numWorkersPerNode'))]", 151 | "privateIPAddressForNodes": "[concat(variables('headnodeMachineName'), variables('networkInterfacePostfix'))]", 152 | "tagsForAll": { 153 | "provider": "[variables('cuaTrackingNum')]" 154 | } 155 | }, 156 | "resources": [ 157 | { 158 | "condition": "[equals(variables('newOrExistingVirtualNetwork'),'New')]", 159 | "type": "Microsoft.Network/virtualNetworks", 160 | "apiVersion": "2020-05-01", 161 | "name": "[variables('virtualNetworkName')]", 162 | "location": "[resourceGroup().location]", 163 | "tags": "[variables('tagsForAll')]", 164 | "properties": { 165 | "addressSpace": { 166 | "addressPrefixes": [ 167 | "[variables('addressPrefix')]" 168 | ] 169 | }, 170 | "subnets": [ 171 | { 172 | "name": "[variables('subnetCreateName')]", 173 | "properties": { 174 | "addressPrefix": "[variables('subnetPrefix')]" 175 | } 176 | } 177 | ] 178 | } 179 | }, 180 | { 181 | "type": "Microsoft.Network/publicIPAddresses", 182 | "apiVersion": "2020-05-01", 183 | "name": "[variables('headnodeIPName')]", 184 | "location": "[resourceGroup().location]", 185 | "tags": "[variables('tagsForAll')]", 186 | "properties": { 187 | "publicIPAllocationMethod": "Static", 188 | "dnsSettings": { 189 | "domainNameLabel": "[concat(variables('headnodeMachineName'), variables('dnsLabelPrefix'))]" 190 | }, 191 | "idleTimeoutInMinutes": "[variables('publicIpTimeout')]" 192 | } 193 | }, 194 | { 195 | "type": "Microsoft.Storage/storageAccounts", 196 | "apiVersion": "2019-06-01", 197 | "name": "[variables('storageAccountName')]", 198 | "location": "[resourceGroup().location]", 199 | "tags": "[variables('tagsForAll')]", 200 | "sku": { 201 | "name": "[variables('storageAccountType')]" 202 | }, 203 | "kind": "Storage", 204 | "properties": {} 205 | }, 206 | { 207 | "type": "Microsoft.Network/networkSecurityGroups", 208 | "apiVersion": "2020-05-01", 209 | "name": "[variables('networkSecurityGroupName')]", 210 | "location": "[resourceGroup().location]", 211 | "dependsOn": [ 212 | "[resourceId('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]" 213 | ], 214 | "tags": "[variables('tagsForAll')]", 215 | "properties": { 216 | "securityRules": [ 217 | { 218 | "name": "default-allow-rdp", 219 | "properties": { 220 | "priority": 1000, 221 | "sourceAddressPrefix": "[parameters('clientIPAddress')]", 222 | "protocol": "Tcp", 223 | "destinationPortRange": "3389", 224 | "access": "Allow", 225 | "direction": "Inbound", 226 | "sourcePortRange": "*", 227 | "destinationAddressPrefix": "*" 228 | } 229 | }, 230 | { 231 | "name": "mathworks-job-scheduler-inbound", 232 | "properties": { 233 | "priority": 1010, 234 | "sourceAddressPrefix": "[parameters('clientIPAddress')]", 235 | "protocol": "Tcp", 236 | "destinationPortRange": "[concat(variables('startPort'), '-', variables('finalPort'))]", 237 | "access": "Allow", 238 | "direction": "Inbound", 239 | "sourcePortRange": "*", 240 | "destinationAddressPrefix": "*" 241 | } 242 | }, 243 | { 244 | "name": "allow-internal-traffic", 245 | "properties": { 246 | "priority": 1020, 247 | "sourceAddressPrefix": "VirtualNetwork", 248 | "protocol": "Tcp", 249 | "destinationPortRange": "*", 250 | "access": "Allow", 251 | "direction": "Inbound", 252 | "sourcePortRange": "*", 253 | "destinationAddressPrefix": "VirtualNetwork" 254 | } 255 | } 256 | ] 257 | } 258 | }, 259 | { 260 | "type": "Microsoft.Network/networkInterfaces", 261 | "apiVersion": "2020-05-01", 262 | "name": "[concat(variables('headnodeMachineName'), variables('networkInterfacePostfix'))]", 263 | "location": "[resourceGroup().location]", 264 | "dependsOn": [ 265 | "[resourceId('Microsoft.Network/publicIpAddresses', variables('headnodeIPName'))]", 266 | "[resourceId('Microsoft.Network/networkSecurityGroups', variables('networkSecurityGroupName'))]" 267 | ], 268 | "tags": "[variables('tagsForAll')]", 269 | "properties": { 270 | "ipConfigurations": [ 271 | { 272 | "name": "[concat(variables('headnodeMachineName'), variables('publicIpConfigPostfix'))]", 273 | "properties": { 274 | "subnet": { 275 | "id": "[variables('subnet')]" 276 | }, 277 | "privateIPAllocationMethod": "Dynamic", 278 | "publicIpAddress": { 279 | "id": "[resourceId('Microsoft.Network/publicIpAddresses', variables('headnodeIPName'))]" 280 | } 281 | } 282 | } 283 | ], 284 | "networkSecurityGroup": { 285 | "id": "[resourceId('Microsoft.Network/networkSecurityGroups', variables('networkSecurityGroupName'))]" 286 | } 287 | } 288 | }, 289 | { 290 | "type": "Microsoft.Compute/virtualMachines", 291 | "apiVersion": "2020-06-01", 292 | "name": "[variables('headnodeMachineName')]", 293 | "location": "[resourceGroup().location]", 294 | "dependsOn": [ 295 | "[resourceId('Microsoft.Network/networkInterfaces', concat(variables('headnodeMachineName'), variables('networkInterfacePostfix')))]" 296 | ], 297 | "tags": "[variables('tagsForAll')]", 298 | "plan": "[variables('plan')]", 299 | "properties": { 300 | "osProfile": { 301 | "computerName": "[variables('headnodeMachineName')]", 302 | "AdminUsername": "[variables('adminUsername')]", 303 | "AdminPassword": "[parameters('AdminPassword')]", 304 | "windowsConfiguration": { 305 | "provisionVmAgent": true 306 | } 307 | }, 308 | "hardwareProfile": { 309 | "vmSize": "[parameters('headNodeVmSize')]" 310 | }, 311 | "storageProfile": "[if(greater(parameters('databaseVolumeSize'), 0), variables('storageProfileWithDataDisk'), variables('storageProfileWithoutDataDisk'))]", 312 | "networkProfile": { 313 | "networkInterfaces": [ 314 | { 315 | "id": "[resourceId('Microsoft.Network/networkInterfaces', concat(variables('headnodeMachineName'), variables('networkInterfacePostfix')))]" 316 | } 317 | ] 318 | } 319 | } 320 | }, 321 | { 322 | "type": "Microsoft.Compute/virtualMachines/extensions", 323 | "apiVersion": "2020-06-01", 324 | "name": "[concat(variables('headnodeMachineName'), '/Installation')]", 325 | "location": "[resourceGroup().location]", 326 | "dependsOn": [ 327 | "[resourceId('Microsoft.Compute/virtualMachines/', variables('headnodeMachineName'))]", 328 | "[resourceId('Microsoft.Storage/storageAccounts/', variables('storageAccountName'))]" 329 | ], 330 | "tags": "[variables('tagsForAll')]", 331 | "properties": { 332 | "publisher": "Microsoft.Compute", 333 | "type": "CustomScriptExtension", 334 | "typeHandlerVersion": "1.5", 335 | "autoUpgradeMinorVersion": true, 336 | "settings": { 337 | "fileUris": "[variables('fileUris')]" 338 | }, 339 | "protectedSettings": { 340 | "commandToExecute": "[concat(variables('installHeadnodeCommand'), ' -StorageAccountName ', variables('storageAccountName'), ' -StorageAccountKey ', listKeys(variables('storageAccountName'), '2017-06-01').keys[0].value, ' -HeadnodeExternalHostname ', reference(variables('headnodeIPName')).dnsSettings.fqdn, ' -HeadnodeInternalIPAddress ', reference(variables('privateIPAddressForNodes')).ipConfigurations[0].properties.privateIPAddress)]" 341 | } 342 | } 343 | }, 344 | { 345 | "type": "Microsoft.Compute/virtualMachineScaleSets", 346 | "apiVersion": "2020-06-01", 347 | "name": "[variables('vmssName')]", 348 | "location": "[resourceGroup().location]", 349 | "dependsOn": [ 350 | "[resourceId('Microsoft.Network/publicIpAddresses/', variables('headnodeIPName'))]", 351 | "[resourceId('Microsoft.Network/networkInterfaces/', concat(variables('headnodeMachineName'), variables('networkInterfacePostfix')))]", 352 | "[resourceId('Microsoft.Storage/storageAccounts/', variables('storageAccountName'))]" 353 | ], 354 | "tags": "[variables('tagsForAll')]", 355 | "sku": { 356 | "name": "[parameters('workerVmSize')]", 357 | "tier": "Standard", 358 | "capacity": "[parameters('numWorkerNodes')]" 359 | }, 360 | "plan": "[variables('plan')]", 361 | "properties": { 362 | "overprovision": false, 363 | "upgradepolicy": { 364 | "mode": "Automatic" 365 | }, 366 | "virtualMachineProfile": { 367 | "osProfile": { 368 | "computerNamePrefix": "[variables('vmssName')]", 369 | "AdminUsername": "[variables('adminUsername')]", 370 | "AdminPassword": "[parameters('AdminPassword')]", 371 | "windowsConfiguration": { 372 | "provisionVmAgent": true 373 | } 374 | }, 375 | "storageProfile": "[variables('storageProfileWithoutDataDisk')]", 376 | "networkProfile": { 377 | "networkInterfaceConfigurations": [ 378 | { 379 | "name": "[concat(variables('vmssName'), variables('networkInterfacePostfix'))]", 380 | "properties": { 381 | "primary": true, 382 | "ipConfigurations": [ 383 | { 384 | "name": "[concat(variables('vmssName'), variables('publicIpConfigPostfix'))]", 385 | "properties": { 386 | "subnet": { 387 | "id": "[variables('subnet')]" 388 | }, 389 | "publicIpAddressConfiguration": { 390 | "name": "[variables('publicIpPostfix')]", 391 | "properties": { 392 | "dnsSettings": { 393 | "domainNameLabel": "[concat(variables('vmssName'), variables('dnsLabelPrefix'))]" 394 | }, 395 | "idleTimeoutInMinutes": "[variables('publicIpTimeout')]" 396 | } 397 | } 398 | } 399 | } 400 | ], 401 | "networkSecurityGroup": { 402 | "id": "[resourceId('Microsoft.Network/networkSecurityGroups', variables('networkSecurityGroupName'))]" 403 | } 404 | } 405 | } 406 | ] 407 | }, 408 | "extensionProfile": { 409 | "extensions": [ 410 | { 411 | "name": "[concat(variables('vmssName'), 'installation')]", 412 | "properties": { 413 | "publisher": "Microsoft.Compute", 414 | "type": "CustomScriptExtension", 415 | "typeHandlerVersion": "1.5", 416 | "autoUpgradeMinorVersion": true, 417 | "settings": { 418 | "fileUris": "[variables('fileUris')]" 419 | }, 420 | "protectedSettings": { 421 | "commandToExecute": "[concat(variables('installWorkerCommand'), ' -StorageAccountName ', variables('storageAccountName'), ' -StorageAccountKey ', listKeys(variables('storageAccountName'), '2017-06-01').keys[0].value, ' -HeadnodeExternalHostname ', reference(variables('headnodeIPName')).dnsSettings.fqdn, ' -HeadnodeInternalIPAddress ', reference(variables('privateIPAddressForNodes')).ipConfigurations[0].properties.privateIPAddress)]" 422 | } 423 | } 424 | } 425 | ] 426 | } 427 | } 428 | } 429 | }, 430 | { 431 | "type": "Microsoft.Resources/deployments", 432 | "apiVersion": "2020-06-01", 433 | "name": "pid-bcfd52e4-d2d7-4ca3-ba20-9a37f7f85c5f-partnercenter", 434 | "dependsOn": [ 435 | "[resourceId('Microsoft.Compute/virtualMachineScaleSets/', variables('vmssName'))]" 436 | ], 437 | "properties": { 438 | "mode": "Incremental", 439 | "template": { 440 | "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#", 441 | "contentVersion": "1.0.0.0", 442 | "resources": [] 443 | } 444 | } 445 | } 446 | ], 447 | "outputs": { 448 | "virtualNetworkResourceID": { 449 | "type": "String", 450 | "value": "[resourceId('Microsoft.Network/virtualNetworks', variables('virtualNetworkName'))]" 451 | } 452 | } 453 | } -------------------------------------------------------------------------------- /releases/R2022b/README.md: -------------------------------------------------------------------------------- 1 | # MATLAB Parallel Server on Azure (Windows VM) 2 | 3 | ## Step 1. Launch the Template 4 | 5 | Click the **Deploy to Azure** button below to deploy the cloud resources on Azure. This will open the Azure Portal in your web browser. 6 | 7 | | Create Virtual Network | Use Existing Virtual Network | 8 | | --- | --- | 9 | | Use this option to deploy the resources in a new virtual network



| Use this option to deploy the resources in an existing virtual network



| 10 | 11 | > Cluster Platform: Windows Server 2019 12 | 13 | > MATLAB Release: R2022b 14 | 15 | ## Step 2. Configure the Cloud Resources 16 | Clicking the Deploy to Azure button opens the "Custom deployment" page in your browser. You can configure the parameters on this page. It is easier to complete the steps if you position these instructions and the Azure Portal window side by side. 17 | 18 | 1. Specify and check the defaults for these resource parameters: 19 | 20 | | Parameter label | Description | 21 | | --------------- | ----------- | 22 | | **Cluster Name** | Name to use for this cluster. This name will be shown in MATLAB as the cluster profile name. | 23 | | **Num Worker Nodes** | The number of Azure instances to start for the workers to run on. | 24 | | **Min Worker Nodes** | Minimum number of Azure instances that can run at any time. | 25 | | **Max Worker Nodes** | Maximum number of Azure instances that can run at any time. | 26 | | **Num Workers Per Node** | The number of MATLAB workers to start on each instance. Specify 1 worker for every 2 vCPUs, because this results in 1 worker per physical core. For example a Standard_D64s_v3 instance has 64 vCPUs, so can support 32 MATLAB workers. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes for details on vCPUs for each instance type. | 27 | | **Head Node Vm Size** | The Azure instance type to use for the headnode, which will run the job manager. No workers will be started on this node, so this can be a smaller instance type than the worker nodes. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes. for a list of instance types. | 28 | | **Worker Vm Size** | The Azure instance type to use for the workers. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes for a list of instance types. | 29 | | **Database Volume Size** | The size of the volume in Gigabytes used to store the database files. If set to 0, a separate volume will not be created and the root volume will be used for the database. | 30 | | **Client IP Address** | The IP address range that can be used to access the cluster from MATLAB. This must be a valid IP CIDR range of the form x.x.x.x/x. Use the value <your_public_ip_address>/32 to restrict access to only your computer. | 31 | | **Admin Username** | Admin username for the cluster. To avoid any deployment errors, please check the list of [disallowed values](https://docs.microsoft.com/en-us/rest/api/compute/virtual-machines/create-or-update?tabs=HTTP#osprofile) for adminUsername. | 32 | | **Admin Password** | Choose the password for the admin user of the cluster. This password and the chosen admin username are required to login into any instance in the cluster using RDP. For the deployment to succeed, your password must meet Azure's password requirements. See [Password requirements when creating a VM](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/faq?WT.mc_id=Portal-fx#what-are-the-password-requirements-when-creating-a-vm-) for information on the password requirements. | 33 | | **Virtual Network Resource ID** | The Resource ID of an existing virtual network to deploy your cluster into. You can find this under the Properties of your virtual network. Specify this parameter only when deploying with the Existing Virtual Network option. | 34 | | **Subnet Name** | The name of an existing subnet within your virtual network to deploy your cluster into. Specify this parameter only when deploying with the Existing Virtual Network option. | 35 | | **License Server** | Optional License Manager for MATLAB string in the form port@hostname. If not specified, online licensing is used. If specified, the license manager must be accessible from the specified virtual network and subnets. | 36 | | **Enable Autoscaling** | Flag indicating whether instance autoscaling is enabled. For more information about autoscaling, refer to the 'Use Autoscaling' section in the deployment README. | 37 | 38 | 39 | **NOTE**: If you are using network license manager, the port and hostname of the network license manager must be reachable from the MATLAB Parallel Server VMs. It is therefore recommended that you deploy into a subnet within the same virtual network as the network license manager. 40 | 41 | 2. Tick the box to accept the Azure Marketplace terms and conditions. 42 | 43 | 3. Click the **Create** button. 44 | 45 | When you click the **Create** button, the resources are created using Azure template deployments. Template deployment can take several minutes. 46 | 47 | # Step 3: Connect to Your Cluster From MATLAB 48 | 49 | 1. After clicking **Create** you will be taken to the Azure Portal Dashboard. To monitor the progress of your deployment, select your resource group from the Resource Groups panel. Wait for the all **Deployments** to reach **Succeeded**. 50 | 2. Select the Storage Account ending with **storage**. The screen should look like the one in Figure 1. 51 | 52 | ![Resource Group On Completion](../../img/Deployment_Complete_Select_Storage.png) 53 | 54 | *Figure 1: Resource Group On Completion* 55 | 56 | 3. Select the Files container type. 57 | 4. Select the File Share named "shared". 58 | 5. Download the file, `.mlsettings`, where NAME is the name of your MATLAB job scheduler. 59 | 6. Open MATLAB. 60 | 7. In the Parallel drop-down menu in the MATLAB toolstrip select **Create and Manage Clusters...**. 61 | 8. Click **Import**. 62 | 9. Select the downloaded profile and click open. 63 | 10. Click **Set as Default**. 64 | 11. (Optional) Validate your cluster by clicking the **Validate** button. 65 | 66 | After setting the cloud cluster as default, the next time you run a parallel language command (such as `parfor`, `spmd`, `parfeval` or `batch`) MATLAB connects to the cluster. The first time you connect, you will be prompted for your MathWorks account login. The first time you run a task on a worker it will take several minutes for the worker MATLAB to start. This delay is due to provisioning the instance disk. This is a one-time operation, and subsequent tasks begin much faster. 67 | 68 | Your cluster is now ready to use. It will remain running after you close MATLAB. 69 | 70 | **NOTE**: Use the profile and client IP address range to control access to your cloud resources. Anyone with this file can connect to your resources from a machine within the specified IP address range and run jobs on it. 71 | 72 | # Additional Information 73 | 74 | ## Port requirements for accessing MATLAB Parallel Server 75 | 76 | To access a MATLAB Parallel Server cluster from your client MATLAB, your client machine must be able to communicate on specific ports. Make sure that the network firewall allows the following outgoing connections: 77 | 78 | | Required ports | Description | 79 | | -------------- | ----------- | 80 | | TCP 27350 to 27358 + 4*N | Ports 27350 to 27358 + 4*N, where N is the maximum number of workers on a single node | 81 | | TCP 443 | HTTPS access to (at least) *.mathworks and *.microsoft.com | 82 | | TCP 3389 | Remote Desktop Protocol to access to cluster nodes | 83 | 84 | *Table 1: Outgoing port requirements* 85 | 86 | ## Delete Your Cloud Resources 87 | You can remove the Resource Group and all associated resources when you are done with them. Note that you cannot recover resources once they are deleted. After you delete the cloud resources you cannot use the downloaded profile again. 88 | 1. Login to the Azure Portal. 89 | 2. Select the Resource Group containing your resources. 90 | 3. Select the "Delete resource group" icon to destroy all resources deployed in this group. 91 | 4. You will be prompted to enter the name of the resource group to confirm the deletion. 92 | 93 | ![Resource Group Delete](../../img/Resource_Group_Delete.png) 94 | 95 | ## Use Autoscaling 96 | 97 | To optimize the number of Virtual Machines running MATLAB workers, enable autoscaling by setting `Enable Autoscaling` to `Yes` when you deploy the template. Autoscaling is optional and is disabled by default. 98 | 99 | When autoscaling is disabled, the Virtual Machine Scale Set launches `Num Worker Nodes` instances. To change the number of worker nodes, use the Azure Portal. 100 | 101 | When autoscaling is enabled, the capacity of the Virtual Machine Scale Set is regulated by the number of workers needed by the cluster. The number of Virtual Machines is initially set at `Num Worker Nodes`. It then fluctuates between `Min` and `Max Worker Nodes`. To change these limits after the deployment, use the Azure Portal and modify the tags on the VMSS. To change the amount of time for which idle nodes are preserved, adjust the value of the tag `mwWorkerIdleTimeoutMinutes`. 102 | 103 | Do not use the "manual scale" option to change the instance count, as this can lead to the unsafe termination of MATLAB workers. 104 | 105 | Ensure that the `Max Worker Nodes` parameter is within your Azure subscription quotas for the specific instance type. To learn about setting quotas, see [Azure subscription and service limits, quotas, and constraints](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits). 106 | 107 | To disable autoscaling after the deployment, redeploy the template with autoscaling disabled. 108 | 109 | ## Troubleshooting 110 | If your resource group fails to deploy, check the Deployments section of the Resource Group. It will indicate which resource deployments failed and allow you to navigate to the causing error message. 111 | 112 | If the resource group deployed successfully but you are unable to validate the cluster, you may need to view the logs on the instances to diagnose the error. The deployment logs are output to C:\ProgramData\MathWorks\MDCSLog-*.log on the instance nodes. The cluster logs are output to C:\ProgramData\MJS\Log. -------------------------------------------------------------------------------- /releases/R2023a/README.md: -------------------------------------------------------------------------------- 1 | # MATLAB Parallel Server on Azure (Windows VM) 2 | 3 | ## Step 1. Launch the Template 4 | 5 | Click the **Deploy to Azure** button below to deploy the cloud resources on Azure. This will open the Azure Portal in your web browser. 6 | 7 | | Create Virtual Network | Use Existing Virtual Network | 8 | | --- | --- | 9 | | Use this option to deploy the resources in a new virtual network



| Use this option to deploy the resources in an existing virtual network



| 10 | 11 | > Cluster Platform: Windows Server 2019 12 | 13 | > MATLAB Release: R2023a 14 | 15 | ## Step 2. Configure the Cloud Resources 16 | Clicking the Deploy to Azure button opens the "Custom deployment" page in your browser. You can configure the parameters on this page. It is easier to complete the steps if you position these instructions and the Azure Portal window side by side. 17 | 18 | 1. Specify and check the defaults for these resource parameters: 19 | 20 | | Parameter label | Description | 21 | | --------------- | ----------- | 22 | | **Cluster Name** | Name to use for this cluster. This name will be shown in MATLAB as the cluster profile name. | 23 | | **Num Worker Nodes** | The number of Azure instances to start for the workers to run on. | 24 | | **Min Worker Nodes** | Minimum number of Azure instances that can run at any time. | 25 | | **Max Worker Nodes** | Maximum number of Azure instances that can run at any time. | 26 | | **Num Workers Per Node** | The number of MATLAB workers to start on each instance. Specify 1 worker for every 2 vCPUs, because this results in 1 worker per physical core. For example a Standard_D64s_v3 instance has 64 vCPUs, so can support 32 MATLAB workers. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes for details on vCPUs for each instance type. | 27 | | **Head Node Vm Size** | The Azure instance type to use for the headnode, which will run the job manager. No workers will be started on this node, so this can be a smaller instance type than the worker nodes. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes. for a list of instance types. | 28 | | **Worker Vm Size** | The Azure instance type to use for the workers. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes for a list of instance types. | 29 | | **Database Volume Size** | The size of the volume in Gigabytes used to store the database files. If set to 0, a separate volume will not be created and the root volume will be used for the database. | 30 | | **Client IP Address** | The IP address range that can be used to access the cluster from MATLAB. This must be a valid IP CIDR range of the form x.x.x.x/x. Use the value <your_public_ip_address>/32 to restrict access to only your computer. | 31 | | **Admin Username** | Admin username for the cluster. To avoid any deployment errors, please check the list of [disallowed values](https://docs.microsoft.com/en-us/rest/api/compute/virtual-machines/create-or-update?tabs=HTTP#osprofile) for adminUsername. | 32 | | **Admin Password** | Choose the password for the admin user of the cluster. This password and the chosen admin username are required to login into any instance in the cluster using RDP. For the deployment to succeed, your password must meet Azure's password requirements. See [Password requirements when creating a VM](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/faq?WT.mc_id=Portal-fx#what-are-the-password-requirements-when-creating-a-vm-) for information on the password requirements. | 33 | | **Virtual Network Resource ID** | The Resource ID of an existing virtual network to deploy your cluster into. You can find this under the Properties of your virtual network. Specify this parameter only when deploying with the Existing Virtual Network option. | 34 | | **Subnet Name** | The name of an existing subnet within your virtual network to deploy your cluster into. Specify this parameter only when deploying with the Existing Virtual Network option. | 35 | | **License Server** | Optional License Manager for MATLAB, specified as a string in the form port@hostname. If not specified, use online licensing. If specified, the license manager must be accessible from the specified virtual network and subnets. For more information, see https://github.com/mathworks-ref-arch/license-manager-for-matlab-on-azure. | 36 | | **Enable Autoscaling** | Flag indicating whether instance autoscaling is enabled. For more information about autoscaling, refer to the 'Use Autoscaling' section in the deployment README. | 37 | 38 | 39 | **NOTE**: If you are using network license manager, the port and hostname of the network license manager must be reachable from the MATLAB Parallel Server VMs. It is therefore recommended that you deploy into a subnet within the same virtual network as the network license manager. 40 | 41 | 2. Tick the box to accept the Azure Marketplace terms and conditions. 42 | 43 | 3. Click the **Create** button. 44 | 45 | When you click the **Create** button, the resources are created using Azure template deployments. Template deployment can take several minutes. 46 | 47 | # Step 3: Connect to Your Cluster From MATLAB 48 | 49 | 1. After clicking **Create** you will be taken to the Azure Portal Dashboard. To monitor the progress of your deployment, select your resource group from the Resource Groups panel. Wait for all the **Deployments** to reach **Succeeded**. 50 | 2. Select the Storage Account ending with **storage**. The screen should look like the one in Figure 1. 51 | 52 | ![Resource Group On Completion](../../img/Deployment_Complete_Select_Storage.png) 53 | 54 | *Figure 1: Resource Group On Completion* 55 | 56 | 3. Select the Files container type. 57 | 4. Select the File Share named "shared". 58 | 5. Download the file, `.mlsettings`, where NAME is the name of your MATLAB job scheduler. 59 | 6. Open MATLAB. 60 | 7. In the Parallel drop-down menu in the MATLAB toolstrip select **Create and Manage Clusters...**. 61 | 8. Click **Import**. 62 | 9. Select the downloaded profile and click open. 63 | 10. Click **Set as Default**. 64 | 11. (Optional) Validate your cluster by clicking the **Validate** button. 65 | 66 | After setting the cloud cluster as default, the next time you run a parallel language command (such as `parfor`, `spmd`, `parfeval` or `batch`) MATLAB connects to the cluster. The first time you connect, you will be prompted for your MathWorks account login. The first time you run a task on a worker it will take several minutes for the worker MATLAB to start. This delay is due to provisioning the instance disk. This is a one-time operation, and subsequent tasks begin much faster. 67 | 68 | Your cluster is now ready to use. It will remain running after you close MATLAB. 69 | 70 | **NOTE**: Use the profile and client IP address range to control access to your cloud resources. Anyone with this file can connect to your resources from a machine within the specified IP address range and run jobs on it. 71 | 72 | # Additional Information 73 | 74 | ## Port requirements for accessing MATLAB Parallel Server 75 | 76 | To access a MATLAB Parallel Server cluster from your client MATLAB, your client machine must be able to communicate on specific ports. Make sure that the network firewall allows the following outgoing connections: 77 | 78 | | Required ports | Description | 79 | | -------------- | ----------- | 80 | | TCP 27350 to 27358 + 4*N | Ports 27350 to 27358 + 4*N, where N is the maximum number of workers on a single node | 81 | | TCP 443 | HTTPS access to (at least) *.mathworks and *.microsoft.com | 82 | | TCP 3389 | Remote Desktop Protocol to access to cluster nodes | 83 | 84 | *Table 1: Outgoing port requirements* 85 | 86 | ## Delete Your Cloud Resources 87 | You can remove the Resource Group and all associated resources when you are done with them. Note that you cannot recover resources once they are deleted. After you delete the cloud resources you cannot use the downloaded profile again. 88 | 1. Login to the Azure Portal. 89 | 2. Select the Resource Group containing your resources. 90 | 3. Select the "Delete resource group" icon to destroy all resources deployed in this group. 91 | 4. You will be prompted to enter the name of the resource group to confirm the deletion. 92 | 93 | ![Resource Group Delete](../../img/Resource_Group_Delete.png) 94 | 95 | ## Use Autoscaling 96 | 97 | To optimize the number of Virtual Machines running MATLAB workers, enable autoscaling by setting `Enable Autoscaling` to `Yes` when you deploy the template. Autoscaling is optional and is disabled by default. 98 | 99 | When autoscaling is disabled, the Virtual Machine Scale Set launches `Num Worker Nodes` instances. To change the number of worker nodes, use the Azure Portal. 100 | 101 | When autoscaling is enabled, the capacity of the Virtual Machine Scale Set is regulated by the number of workers needed by the cluster. The number of Virtual Machines is initially set at `Num Worker Nodes`. It then fluctuates between `Min` and `Max Worker Nodes`. To change these limits after the deployment, use the Azure Portal and modify the tags on the VMSS. To change the amount of time for which idle nodes are preserved, adjust the value of the tag `mwWorkerIdleTimeoutMinutes`. 102 | 103 | Do not use the "manual scale" option to change the instance count, as this can lead to the unsafe termination of MATLAB workers. 104 | 105 | Ensure that the `Max Worker Nodes` parameter is within your Azure subscription quotas for the specific instance type. To learn about setting quotas, see [Azure subscription and service limits, quotas, and constraints](https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits). 106 | 107 | To disable autoscaling after the deployment, redeploy the template with autoscaling disabled. 108 | 109 | ## Troubleshooting 110 | If your resource group fails to deploy, check the Deployments section of the Resource Group. It will indicate which resource deployments failed and allow you to navigate to the causing error message. 111 | 112 | If the resource group deployed successfully but you are unable to validate the cluster, you may need to view the logs on the instances to diagnose the error. The deployment logs are output to C:\ProgramData\MathWorks\MDCSLog-*.log on the instance nodes. The cluster logs are output to C:\ProgramData\MJS\Log. -------------------------------------------------------------------------------- /releases/R2023b/README.md: -------------------------------------------------------------------------------- 1 | # MATLAB Parallel Server on Azure (Windows VM) 2 | 3 | ## Step 1. Launch the Template 4 | 5 | Click the **Deploy to Azure** button below to deploy the cloud resources on Azure®. This will open the Azure Portal in your web browser. 6 | 7 | | Create Virtual Network | Use Existing Virtual Network | 8 | | --- | --- | 9 | | Use this option to deploy the resources in a new virtual network



| Use this option to deploy the resources in an existing virtual network



| 10 | 11 | > Cluster Platform: Windows Server 2019 12 | 13 | > MATLAB® Release: R2023b 14 | 15 | ## Step 2. Configure the Cloud Resources 16 | Clicking the Deploy to Azure button opens the "Custom deployment" page in your browser. You can configure the parameters on this page. It is easier to complete the steps if you position these instructions and the Azure Portal window side by side. 17 | 18 | 1. Specify and check the defaults for these resource parameters: 19 | 20 | | Parameter label | Description | 21 | | --------------- | ----------- | 22 | | **Cluster Name** | Name to use for this cluster. This name will be shown in MATLAB as the cluster profile name. | 23 | | **Num Worker Nodes** | The number of Azure instances to start for the workers to run on. | 24 | | **Min Worker Nodes** | Minimum number of Azure instances that can run at any time. | 25 | | **Max Worker Nodes** | Maximum number of Azure instances that can run at any time. | 26 | | **Num Workers Per Node** | The number of MATLAB workers to start on each instance. Specify 1 worker for every 2 vCPUs, because this results in 1 worker per physical core. For example, a Standard_D64s_v3 instance has 64 vCPUs, so can support 32 MATLAB workers. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes for details on vCPUs for each instance type. | 27 | | **Head Node Vm Size** | The Azure instance type to use for the headnode, which will run the job manager. No workers will be started on this node, so this can be a smaller instance type than the worker nodes. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes. for a list of instance types. | 28 | | **Worker Vm Size** | The Azure instance type to use for the workers. See https://docs.microsoft.com/en-us/azure/virtual-machines/sizes for a list of instance types. | 29 | | **Database Volume Size** | The size of the volume in Gigabytes used to store the database files. If set to 0, a separate volume will not be created and the root volume will be used for the database. | 30 | | **Client IP Address** | The IP address range that can be used to access the cluster from MATLAB. This must be a valid IP CIDR range of the form x.x.x.x/x. Use the value <your_public_ip_address>/32 to restrict access to only your computer. | 31 | | **Admin Username** | Admin username for the cluster. To avoid any deployment errors, please check the list of [disallowed values](https://docs.microsoft.com/en-us/rest/api/compute/virtual-machines/create-or-update?tabs=HTTP#osprofile) for adminUsername. | 32 | | **Admin Password** | Choose the password for the admin user of the cluster. This password and the chosen admin username are required to login into any instance in the cluster using RDP. For the deployment to succeed, your password must meet Azure's password requirements. See [Password requirements when creating a VM](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/faq?WT.mc_id=Portal-fx#what-are-the-password-requirements-when-creating-a-vm-) for information on the password requirements. | 33 | | **Virtual Network Resource ID** | The Resource ID of an existing virtual network to deploy your cluster into. You can find this under the Properties of your virtual network. Specify this parameter only when deploying with the Existing Virtual Network option. | 34 | | **Subnet Name** | The name of an existing subnet within your virtual network to deploy your cluster into. Specify this parameter only when deploying with the Existing Virtual Network option. | 35 | | **License Server** | Optional License Manager for MATLAB, specified as a string in the form port@hostname. If not specified, use online licensing. If specified, the license manager must be accessible from the specified virtual network and subnets. For more information, see https://github.com/mathworks-ref-arch/license-manager-for-matlab-on-azure. | 36 | | **Enable Autoscaling** | Flag indicating whether instance autoscaling is enabled. For more information about autoscaling, refer to the 'Use Autoscaling' section in the deployment README. | 37 | 38 | 39 | **NOTE**: If you are using network license manager, the port and hostname of the network license manager must be reachable from the MATLAB Parallel Server VMs. It is therefore recommended that you deploy into a subnet within the same virtual network as the network license manager. 40 | 41 | 2. Tick the box to accept the Azure Marketplace terms and conditions. 42 | 43 | 3. Click the **Create** button. 44 | 45 | When you click the **Create** button, the resources are created using Azure template deployments. Template deployment can take several minutes. 46 | 47 | # Step 3: Connect to Your Cluster From MATLAB 48 | 49 | 1. After clicking **Create** you will be taken to the Azure Portal Dashboard. To monitor the progress of your deployment, select your resource group from the Resource Groups panel. Wait for all the **Deployments** to reach **Succeeded**. 50 | 2. Select the Storage Account ending with **storage**. The screen should look like the one in Figure 1. 51 | 52 | ![Resource Group On Completion](../../img/Deployment_Complete_Select_Storage.png) 53 | 54 | *Figure 1: Resource Group On Completion* 55 | 56 | 3. Select the Files container type. 57 | 4. Select the File Share named "shared". 58 | 5. Download the file, `.mlsettings`, where NAME is the name of your MATLAB job scheduler. 59 | 6. Open MATLAB. 60 | 7. In the Parallel drop-down menu in the MATLAB toolstrip select **Create and Manage Clusters...**. 61 | 8. Click **Import**. 62 | 9. Select the downloaded profile and click open. 63 | 10. Click **Set as Default**. 64 | 11. (Optional) Validate your cluster by clicking the **Validate** button. 65 | 66 | After setting the cloud cluster as default, the next time you run a parallel language command (such as `parfor`, `spmd`, `parfeval` or `batch`) MATLAB connects to the cluster. The first time you connect, you will be prompted for your MathWorks® account login. The first time you run a task on a worker it will take several minutes for the worker MATLAB to start. This delay is due to provisioning the instance disk. This is a one-time operation, and subsequent tasks begin much faster. 67 | 68 | Your cluster is now ready to use. It will remain running after you close MATLAB. 69 | 70 | **NOTE**: Use the profile and client IP address range to control access to your cloud resources. Anyone with this file can connect to your resources from a machine within the specified IP address range and run jobs on it. 71 | 72 | # Additional Information 73 | 74 | ## Port requirements for accessing MATLAB Parallel Server 75 | 76 | To access a MATLAB Parallel Server cluster from your client MATLAB, your client machine must be able to communicate on specific ports. Make sure that the network firewall allows the following outgoing connections: 77 | 78 | | Required ports | Description | 79 | | -------------- | ----------- | 80 | | TCP 27350 to 27358 + 4*N | Ports 27350 to 27358 + 4*N, where N is the maximum number of workers on a single node | 81 | | TCP 443 | HTTPS access to (at least) *.mathworks and *.microsoft.com | 82 | | TCP 3389 | Remote Desktop Protocol to access to cluster nodes | 83 | 84 | *Table 1: Outgoing port requirements* 85 | 86 | ## Delete Your Cloud Resources 87 | You can remove the Resource Group and all associated resources when you are done with them. Note that you cannot recover resources once they are deleted. After you delete the cloud resources you cannot use the downloaded profile again. 88 | 1. Login to the Azure Portal. 89 | 2. Select the Resource Group containing your resources. 90 | 3. Select the "Delete resource group" icon to destroy all resources deployed in this group. 91 | 4. You will be prompted to enter the name of the resource group to confirm the deletion. 92 | 93 | ![Resource Group Delete](../../img/Resource_Group_Delete.png) 94 | 95 | ## Use Autoscaling 96 | 97 | To optimize the number of Virtual Machines running MATLAB workers, enable autoscaling by setting `Enable Autoscaling` to `Yes` when you deploy the template. Autoscaling is optional and is disabled by default. 98 | 99 | When autoscaling is disabled, the Virtual Machine Scale Set launches `Num Worker Nodes` instances. To change the number of worker nodes, use the Azure Portal. 100 | 101 | When autoscaling is enabled, the capacity of the Virtual Machine Scale Set is regulated by the number of workers needed by the cluster. The number of Virtual Machines is initially set at `Num Worker Nodes`. It then fluctuates between `Min` and `Max Worker Nodes`. To change these limits after the deployment, use the Azure Portal and modify the tags on the VMSS. To change the amount of time for which idle nodes are preserved, adjust the value of the tag `mwWorkerIdleTimeoutMinutes`. 102 | 103 | Do not use the "manual scale" option to change the instance count, as this can lead to the unsafe termination of MATLAB workers. 104 | 105 | Ensure that the `Max Worker Nodes` parameter is within your Azure subscription quotas for the specific instance type. To learn about setting quotas, see [Azure subscription and service limits, quotas, and constraints](https://learn.microsoft.com/en-us/azure/azure-resource-manager/management/azure-subscription-service-limits). 106 | 107 | To disable autoscaling after the deployment, redeploy the template with autoscaling disabled. 108 | 109 | ## Troubleshooting 110 | If your resource group fails to deploy, check the Deployments section of the Resource Group. It will indicate which resource deployments failed and allow you to navigate to the causing error message. 111 | 112 | If the resource group deployed successfully but you are unable to validate the cluster, you may need to view the logs on the instances to diagnose the error. The deployment logs are output to C:\ProgramData\MathWorks\MDCSLog-*.log on the instance nodes. The cluster logs are output to C:\ProgramData\MJS\Log. 113 | 114 | ---- 115 | 116 | Copyright 2018-2023 The MathWorks, Inc. 117 | 118 | ---- -------------------------------------------------------------------------------- /releases/R2024a/README.md: -------------------------------------------------------------------------------- 1 | # MATLAB Parallel Server on Azure (Windows VM) 2 | 3 | ## Step 1. Deploy the Template 4 | 5 | Click the **Deploy to Azure** button below to deploy the cloud resources on Azure®. This opens the Azure Portal in your web browser. 6 | 7 | | Create Virtual Network | Use Existing Virtual Network | 8 | | --- | --- | 9 | | Use this option to deploy the resources in a new virtual network



| Use this option to deploy the resources in an existing virtual network



| 10 | 11 | > Cluster Platform: Windows Server 2019 12 | 13 | > MATLAB® Release: R2024a 14 | 15 | ## Step 2. Configure the Cloud Resources 16 | Clicking the **Deploy to Azure** button opens the "Custom deployment" page in your browser. You can configure the parameters on this page. It is easier to complete the steps if you position these instructions and the Azure Portal window side by side. Create a new resource group by clicking **Create New**. Alternatively, you can select an existing resource group, but this can cause conflicts if resources are already deployed in it. 17 | 18 | 1. Specify and check the defaults for these resource parameters: 19 | 20 | | Parameter label | Description | 21 | | --------------- | ----------- | 22 | | **Cluster Name** | Name to use for this cluster. This name is shown in MATLAB as the cluster profile name. | 23 | | **Num Worker Nodes** | Number of Azure Virtual Machines to start for the workers to run on. | 24 | | **Min Worker Nodes** | Minimum number of running Azure Virtual Machines. | 25 | | **Max Worker Nodes** | Maximum number of running Azure Virtual Machines. | 26 | | **Num Workers Per Node** | Number of MATLAB worker processes to start on each Virtual Machine (VM). Specify 1 worker for every 2 vCPUs, because this results in 1 worker per physical core. For example, a Standard_D64s_v3 VM has 64 vCPUs, so can support 32 MATLAB workers. See https://learn.microsoft.com/azure/virtual-machines/sizes for details on vCPUs for each VM size. | 27 | | **Head Node Vm Size** | Azure VM size to use for the headnode, which runs the job manager. No workers are started on this node, so this can be a smaller VM type than the worker nodes. By default, the heap memory for the job manager is set between 1024 MiB and a maximum of half of the VM memory, depending on the total number of MATLAB workers. See https://learn.microsoft.com/azure/virtual-machines/sizes for a list of VMs. | 28 | | **Worker Vm Size** | Azure VM size to use for the workers. By default, the heap memory for all worker process is set between 1024 MiB and a maximum of a quarter of the VM memory, depending on the number of MATLAB workers on the VM. See https://learn.microsoft.com/azure/virtual-machines/sizes for a list of VMs. | 29 | | **Use Spot Instances For Worker Nodes** | Option indicating whether to enable Azure Spot Virtual Machines for worker nodes. For more information, refer to the FAQ section in the deployment README. | 30 | | **Database Volume Size** | Size in GB of the volume to store the database files. All job and task information, including input and output data is stored on this volume and should therefore have enough capacity to store the expected amount of data. If set to 0, no volume is created and the root volume of the VM is used for the database. | 31 | | **Client IP Address** | IP address range that can be used to access the cluster from MATLAB. This must be a valid IP CIDR range of the form x.x.x.x/x. Use the value <your_public_ip_address>/32 to restrict access to only your computer. | 32 | | **Admin Username** | Admin username for the cluster. To avoid any deployment errors, check the list of [disallowed values](https://learn.microsoft.com/rest/api/compute/virtual-machines/create-or-update?tabs=HTTP#osprofile) for adminUsername. | 33 | | **Admin Password** | Choose the password for the admin user of the cluster. This password and the chosen admin username are required to login into any VM in the cluster using RDP. For the deployment to succeed, your password must meet Azure's password requirements. See [Password requirements when creating a VM](https://learn.microsoft.com/azure/virtual-machines/windows/faq?WT.mc_id=Portal-fx#what-are-the-password-requirements-when-creating-a-vm-) for information on the password requirements. | 34 | | **Virtual Network Resource ID** | Resource ID of an existing virtual network to deploy your cluster into. You can find this under the Properties of your virtual network. Specify this parameter only when deploying with the Existing Virtual Network option. | 35 | | **Subnet Name** | Name of an existing subnet within your virtual network to deploy your cluster into. Specify this parameter only when deploying with the Existing Virtual Network option. | 36 | | **License Server** | Optional License Manager for MATLAB, specified as a string in the form port@hostname. If not specified, online licensing is used. Otherwise, license manager must be accessible from the specified virtual network and subnets. For more information, see https://github.com/mathworks-ref-arch/license-manager-for-matlab-on-azure. | 37 | | **MJS Security Level** | Security level for the cluster. Level 0: Any user can access any jobs and tasks. Level 1: Accessing other users' jobs and tasks issues a warning. However, all users can still perform all actions. Level 2: Users must enter a password to access their jobs and tasks. The job owner can grant access to other users. | 38 | | **Enable Autoscaling** | Option indicating whether VM autoscaling is enabled. For more information about autoscaling, refer to the 'Use Autoscaling' section in the deployment README. | 39 | | **MJS Scheduling Algorithm** | Scheduling algorithm for the job manager. 'standard' spreads communicating jobs across as few worker machines as possible to reduce communication overheads and fills in unused spaces on worker machines with independent jobs. Suitable for good behaviour for a wide range of uses including autoscaling. 'loadBalancing' distributes load evenly across the cluster to give as many resources as possible to running jobs and tasks when the cluster is underutilized. | 40 | | **Optional User Command** | Provide an optional inline PowerShell command to run on machine launch. For example, to set an environment variable CLOUD=AZURE, use this command excluding the angle brackets: <[System.Environment]::SetEnvironmentVariable("CLOUD","AZURE", "Machine");>. You can use either double quotes or two single quotes. To run an external script, use this command excluding the angle brackets: <Invoke-WebRequest "https://www.example.com/script.ps1" -OutFile script.ps1; .\script.ps1>. Find the logs at '$Env:ProgramData\MathWorks\startup.log'. | 41 | 42 | 43 | **NOTE**: If you are using network license manager, the port and hostname of the network license manager must be reachable from the MATLAB Parallel Server™ Virtual Machines (VMs). It is therefore recommended that you deploy into a subnet within the same virtual network as the network license manager. 44 | 45 | 2. Click the **Review + create** button to review the Azure Marketplace terms and conditions. 46 | 47 | 3. Click the **Create** button. 48 | 49 | When you click the **Create** button, the resources are created using Azure template deployments. Template deployment can take several minutes. 50 | 51 | # Step 3: Connect to Your Cluster From MATLAB 52 | 53 | 1. After clicking **Create**, you are taken to the Deployment Details page, where you can monitor the progress of your deployment. Wait for the message **Your deployment is complete**. 54 | 2. Go to your resource group, and select the Storage Account ending with **storage**. The screen should look like the one in Figure 1. 55 | 56 | ![Resource Group On Completion](../../img/Deployment_Complete_Select_Storage.png) 57 | 58 | *Figure 1: Resource Group On Completion* 59 | 60 | 3. Under **Data Storage** on the left panel, click **File shares**, and select the file share named "shared". 61 | 4. Click **Browse** on the left panel, then open the "cluster" folder. 62 | 5. Download the file, `.mlsettings`, where NAME is the name of your MATLAB job scheduler. 63 | 6. Open MATLAB. 64 | 7. In the Parallel drop-down menu in the MATLAB toolstrip select **Create and Manage Clusters**. 65 | 8. Click **Import**. 66 | 9. Select the downloaded profile and click open. 67 | 10. Click **Set as Default**. 68 | 11. (Optional) Validate your cluster by clicking the **Validate** button. 69 | 70 | After setting the cloud cluster as default, the next time you run a parallel language command (such as `parfor`, `spmd`, `parfeval` or `batch`), MATLAB connects to the cluster. The first time you connect, you are prompted for your MathWorks® account login. The first time you run a task on a worker, it takes several minutes for the worker MATLAB to start. This delay is due to provisioning the instance disk. This is a one-time operation, and subsequent tasks begin much faster. 71 | 72 | Your cluster is now ready to use. 73 | 74 | **NOTE**: Use the profile and client IP address range to control access to your cloud resources. Anyone with this file can connect to your resources from a machine within the specified IP address range and run jobs on it. 75 | 76 | Your cluster remains running after you close MATLAB. To delete your cluster, follow these instructions. 77 | 78 | ## Delete Your Cloud Resources 79 | You can remove the resource group and all associated resources when you are done with them. Note that you cannot recover resources once they are deleted. After you delete the cloud resources, you cannot use the downloaded profile again. 80 | 1. Login to the Azure Portal. 81 | 2. Select the resource group containing your resources. 82 | 3. Select the "Delete resource group" icon to destroy all resources deployed in this group. 83 | 4. You are prompted to enter the name of the resource group to confirm the deletion. 84 | 85 | ![Resource Group Delete](../../img/Resource_Group_Delete.png) 86 | 87 | # Additional Information 88 | 89 | ## Port requirements for accessing MATLAB Parallel Server 90 | 91 | To access a MATLAB Parallel Server cluster from your client MATLAB, your client machine must be able to communicate on specific ports. Make sure that the network firewall allows the following outgoing connections: 92 | 93 | | Required ports | Description | 94 | | -------------- | ----------- | 95 | | TCP 27350 to 27358 + 4*N | Ports 27350 to 27358 + 4*N, where N is the maximum number of workers on a single node | 96 | | TCP 443 | HTTPS access to (at least) *.mathworks and *.microsoft.com | 97 | | TCP 3389 | Remote Desktop Protocol to access to cluster nodes | 98 | 99 | *Table 1: Outgoing port requirements* 100 | 101 | By default, MATLAB Parallel Server is configured with the public hostname of each machine to allow the MATLAB client to access both the scheduler and workers. If you modify the Azure Resource Manager (ARM) template provided, ensure that a public hostname is provided for the headnode and the worker nodes. 102 | 103 | ## Use Autoscaling 104 | 105 | To optimize the number of Virtual Machines running MATLAB workers, enable autoscaling by setting `Enable Autoscaling` to `Yes` when you deploy the template. Autoscaling is optional and is disabled by default. 106 | 107 | When autoscaling is disabled, the Virtual Machine Scale Set (VMSS) deploys `Num Worker Nodes` instances. To change the number of worker nodes, use the Azure Portal. 108 | 109 | If you enable autoscaling, the capacity of the VMSS is regulated by the number of workers needed by the cluster. The number of Virtual Machines is initially set at `Num Worker Nodes`. It then fluctuates between `Min` and `Max Worker Nodes`. To change these limits after the deployment, use the Azure Portal and modify the tags on the VMSS. To change the amount of time for which idle nodes are preserved, adjust the value of the tag `mwWorkerIdleTimeoutMinutes`. Do not use the "manual scale" option to change the instance count, as this can lead to the unsafe termination of MATLAB workers. 110 | 111 | Ensure that the `Max Worker Nodes` parameter is within your Azure subscription quotas for the specific instance type. To learn about setting quotas, see [Azure subscription and service limits, quotas, and constraints](https://learn.microsoft.com/azure/azure-resource-manager/management/azure-subscription-service-limits). 112 | 113 | To disable autoscaling after the deployment, redeploy the template with autoscaling disabled. 114 | 115 | ## MATLAB Job Scheduler Configuration 116 | 117 | By default, MATLAB Job Scheduler (MJS) is configured to manage a wide range of cluster uses. 118 | 119 | To change the MJS configuration for advanced use cases, replace the default `mjs_def` with your own file using the template parameter `OptionalUserCommand`. This overwrites all MJS startup parameters, except for *DEFAULT_JOB_MANAGER_NAME*, *HOSTNAME*, and *SHARED_SECRET_FILE*. To learn more about the MJS startup parameters and to edit them, see [Define MATLAB Job Scheduler Startup Parameters](https://www.mathworks.com/help/matlab-parallel-server/define-startup-parameters.html). 120 | For example, to retrieve and use your edited `mjs_def` from a storage service (e.g. Azure blob storage), set the `OptionalUserCommand` to the following: 121 | ``` 122 | Invoke-WebRequest "https://.blob.core.windows.net//mjs_def.bat" -OutFile $Env:MJSDefFile 123 | ``` 124 | 125 | ## Troubleshooting 126 | If your resource group fails to deploy, check the Deployments section of the Resource Group. This section indicates which resource deployments failed and allow you to navigate to the causing error message. 127 | 128 | If the resource group deployed successfully but you are unable to validate the cluster, check the logs on the instances to diagnose the error. The deployment logs are output to C:\ProgramData\MathWorks\MDCSLog-*.log on the instance nodes. The cluster logs are output to C:\ProgramData\MJS\Log. 129 | 130 | ---- 131 | 132 | Copyright 2018 - 2024 The MathWorks, Inc. 133 | 134 | ---- -------------------------------------------------------------------------------- /releases/R2024b/README.md: -------------------------------------------------------------------------------- 1 | # MATLAB Parallel Server on Azure (Windows VM) 2 | 3 | ## Step 1. Deploy the Template 4 | 5 | Click the **Deploy to Azure** button below to deploy the cloud resources on Azure®. Doing so opens the Azure Portal in your web browser. 6 | 7 | | Create Virtual Network | Use Existing Virtual Network | 8 | | --- | --- | 9 | | Use this option to deploy the resources in a new virtual network



| Use this option to deploy the resources in an existing virtual network



| 10 | 11 | > Cluster Platform: Windows Server 2022 12 | 13 | > MATLAB® Release: R2024b 14 | 15 | ## Step 2. Configure the Cloud Resources 16 | Clicking the **Deploy to Azure** button opens the "Custom deployment" page in your browser. You can configure the parameters on this page. It is easier to complete the steps if you position these instructions and the Azure Portal window side by side. Create a new resource group by clicking **Create New**. Alternatively, you can select an existing resource group, but doing so can cause conflicts if resources are already deployed in it. 17 | 18 | 1. Specify and check the default settings for these resource parameters. 19 | 20 | | Parameter Label | Description | 21 | | --------------- | ----------- | 22 | | **Cluster Name** | Name to use for this cluster. This name is shown in MATLAB as the cluster profile name. | 23 | | **Num Worker Nodes** | Number of Azure virtual machines to start for the workers to run on. | 24 | | **Min Worker Nodes** | Minimum number of running Azure virtual machines. | 25 | | **Max Worker Nodes** | Maximum number of running Azure virtual machines. | 26 | | **Num Workers Per Node** | Number of MATLAB worker processes to start on each virtual machine (VM). Specify 1 worker for every 2 vCPUs so that each physical core has 1 worker. For example, a Standard_D64s_v3 VM has 64 vCPUs, so it can support 32 MATLAB workers. See https://learn.microsoft.com/azure/virtual-machines/sizes for details on vCPUs for each VM size. | 27 | | **Head Node Vm Size** | Azure virtual machine (VM) size to use for the head node, which runs the job manager. No workers are started on this node, so this node can be a smaller VM type than the worker nodes. By default, the heap memory for the job manager is set between 1024 MiB and a maximum of half of the VM memory, depending on the total number of MATLAB workers. See https://learn.microsoft.com/azure/virtual-machines/sizes for a list of VMs. | 28 | | **Worker Vm Size** | Azure virtual machine (VM) size to use for the workers. By default, the heap memory for all worker process is set between 1024 MiB and a maximum of a quarter of the VM memory, depending on the number of MATLAB workers on the VM. See https://learn.microsoft.com/azure/virtual-machines/sizes for a list of VMs. | 29 | | **Use Spot Instances For Worker Nodes** | Option indicating whether to enable Azure Spot Virtual Machines for worker nodes. For more information, refer to the FAQ section in the deployment README. | 30 | | **Database Volume Size** | Size in GB of the volume to store the database files. All job and task information, including input and output data, is stored on this volume and so it must have enough capacity to store the expected amount of data. If set to 0, no volume is created and the root volume of the VM is used for the database. | 31 | | **Client IP Address** | IP address range that can be used to access the cluster from MATLAB. This range must be a valid IP CIDR range of the form x.x.x.x/x. Use the value <your_public_ip_address>/32 to restrict access to only your computer. | 32 | | **Admin Username** | Admin username for the cluster. To avoid any deployment errors, check the list of [disallowed values](https://learn.microsoft.com/rest/api/compute/virtual-machines/create-or-update?tabs=HTTP#osprofile) for adminUsername. | 33 | | **Admin Password** | Choose the password for the admin user of the cluster. This password and the chosen admin username are required to log in into any VM in the cluster using RDP. For the deployment to succeed, your password must meet Azure's password requirements. See [Password requirements when creating a VM](https://learn.microsoft.com/azure/virtual-machines/windows/faq?WT.mc_id=Portal-fx#what-are-the-password-requirements-when-creating-a-vm-) for information on the password requirements. | 34 | | **Virtual Network Resource ID** | Resource ID of an existing virtual network to deploy your cluster into. You can find ID this under the properties of your virtual network. Specify this parameter only when deploying with the Existing Virtual Network option. | 35 | | **Subnet Name** | Name of an existing subnet within your virtual network to deploy your cluster into. Specify this parameter only when deploying with the Existing Virtual Network option. | 36 | | **License Server** | License manager for MATLAB, specified as a string in the form port@hostname. If not specified, online licensing is used. Otherwise, license manager must be accessible from the specified virtual network and subnets. For more information, see https://github.com/mathworks-ref-arch/license-manager-for-matlab-on-azure. | 37 | | **MJS Security Level** | Security level for the cluster. Level 0: Any user can access any jobs and tasks. Level 1: Accessing other users' jobs and tasks issues a warning. However, all users can still perform all actions. Level 2: Users must enter a password to access their jobs and tasks. The job owner can grant access to other users. | 38 | | **Enable Autoscaling** | Option indicating whether VM autoscaling is enabled. For more information about autoscaling, refer to the 'Use Autoscaling' section in the deployment README. | 39 | | **Automatically Terminate Cluster** | Option to autoterminate the cluster after a few hours or when idle. When the cluster is terminated, all worker nodes are deleted and the headnode is deallocated. Select 'Never' to disable auto-termination now but you can enable it later. Select 'Disable auto-termination' to fully disable this feature or if you do not have the permissions to create and assign roles in your subscription. For more information, see [Automatically terminate the MATLAB Parallel Server cluster](https://github.com/mathworks-ref-arch/matlab-parallel-server-on-azure-lin#automatically-terminate-the-matlab-parallel-server-cluster). | 40 | | **MJS Scheduling Algorithm** | Scheduling algorithm for the job manager. 'standard' spreads communicating jobs across as few worker machines as possible to reduce communication overheads and fills in unused spaces on worker machines with independent jobs. Suitable for good behavior for a wide range of uses including autoscaling. 'loadBalancing' distributes load evenly across the cluster to give as many resources as possible to running jobs and tasks when the cluster is underutilized. | 41 | | **Optional User Command** | Provide an optional inline PowerShell command to run on machine launch. For example, to set an environment variable CLOUD=AZURE, use this command excluding the angle brackets: <[System.Environment]::SetEnvironmentVariable("CLOUD","AZURE", "Machine");>. You can use either double quotes or two single quotes. To run an external script, use this command excluding the angle brackets: <Invoke-WebRequest "https://www.example.com/script.ps1" -OutFile script.ps1; .\script.ps1>. Find the logs at 'C:\ProgramData\MathWorks\startup.log'. | 42 | 43 | 44 | **NOTE**: If you are using the network license manager, the port and hostname of the network license manager must be reachable from the MATLAB Parallel Server™ virtual machines (VMs). Deploying into a subnet within the same virtual network as the network license manager is a best practice. 45 | 46 | 2. Click the **Review + create** button to review the Azure Marketplace terms and conditions. 47 | 48 | 3. Click the **Create** button. 49 | 50 | When you click the **Create** button, the resources are created using Azure template deployments. Template deployment can take several minutes. 51 | 52 | # Step 3: Connect to Your Cluster From MATLAB 53 | 54 | 1. Clicking **Create** opens the Deployment Details page, where you can monitor the progress of your deployment. Wait for the message **Your deployment is complete**. 55 | 2. Go to your resource group, and select the Storage Account starting with **mwstorage**. The screen must look like the one in Figure 1. 56 | 57 | ![Resource Group On Completion](../../img/Deployment_Complete_Select_Storage.png) 58 | 59 | *Figure 1: Resource Group On Completion* 60 | 61 | 3. Under **Data Storage** on the left panel, click **File shares**, and select the file share named "shared". 62 | 4. Click **Browse** on the left panel, then open the "cluster" folder. 63 | 5. Download the file, `.mlsettings`, where NAME is the name of your MATLAB job scheduler. 64 | 6. Open MATLAB. 65 | 7. In the Parallel drop-down menu in the MATLAB toolstrip select **Create and Manage Clusters**. 66 | 8. Click **Import**. 67 | 9. Select the downloaded profile and click open. 68 | 10. Click **Set as Default**. 69 | 11. (Optional) Validate your cluster by clicking the **Validate** button. 70 | 71 | After you set the cloud cluster as default, the next time you run a parallel language command (such as `parfor`, `spmd`, `parfeval` or `batch`), MATLAB connects to the cluster. The first time you connect, you are prompted for your MathWorks® account login. The first time you run a task on a worker, it takes several minutes for the worker MATLAB to start. This delay is due to provisioning the instance disk. This is a one-time operation, and subsequent tasks begin faster. 72 | 73 | Your cluster is now ready to use. 74 | 75 | **NOTE**: Use the profile and client IP address range to control access to your cloud resources. Anyone with this file can connect to your resources from a machine within the specified IP address range and run jobs on it. 76 | 77 | Your cluster remains running after you close MATLAB. 78 | 79 | ## Delete Your Cloud Resources 80 | You can remove the resource group and all associated resources when you are done with them. Note that you cannot recover resources once they are deleted. After you delete the cloud resources, you cannot use the downloaded profile again. 81 | 1. Log in to the Azure Portal. 82 | 2. Select the resource group containing your resources. 83 | 3. Select the "Delete resource group" icon to destroy all resources deployed in this group. 84 | 4. You are prompted to enter the name of the resource group to confirm the deletion. 85 | 86 | ![Resource Group Delete](../../img/Resource_Group_Delete.png) 87 | 88 | # Additional Information 89 | 90 | ## Port Requirements 91 | 92 | Before you can use your MATLAB Parallel Server cluster, you must configure certain required ports on the cluster and client firewall. These ports allow your client machine to connect to the cluster headnode and facilitate communication between the cluster nodes. 93 | 94 | ### Cluster Nodes 95 | 96 | For details about the port requirements for cluster nodes, see this information from MathWorks® Support Team on MATLAB Answers: [How do I configure MATLAB Parallel Server using the MATLAB Job Scheduler to work within a firewall?]( https://www.mathworks.com/matlabcentral/answers/94254-how-do-i-configure-matlab-parallel-server-using-the-matlab-job-scheduler-to-work-within-a-firewall). 97 | 98 | Additionally, if your client machine is outside the cluster’s network, then you must configure the network security group of your cluster to allow incoming traffic from your client machine on the following ports. For information on how to configure your network security group, see [Create a security rule](https://learn.microsoft.com/azure/virtual-network/manage-network-security-group). To troubleshoot, see this [page](https://learn.microsoft.com/troubleshoot/azure/virtual-machines/windows/troubleshoot-rdp-nsg-problem). 99 | 100 | | Required ports | Description | 101 | | -------------- | ----------- | 102 | | TCP 27350 to 27358 + 4*N | For connecting to the job manager on the cluster headnode and to the worker nodes for parallel pools. Calculate the required ports based on N, the maximum number of workers on any single node across the entire cluster. | 103 | | TCP 443 | If you are using online licensing, you must open this port for outbound communication from all cluster machines. If you’re using Network License Manager instead, then you must configure ports as listed on [Network License Manager for MATLAB on Microsoft Azure](https://github.com/mathworks-ref-arch/license-manager-for-matlab-on-azure?tab=readme-ov-file#networking-resources). | 104 | | TCP 3389 | Remote Desktop Protocol to access to cluster nodes. | 105 | 106 | *Table 1: Outgoing port requirements* 107 | 108 | By default, MATLAB Parallel Server is configured with the public hostname of each machine to allow the MATLAB client to access both the scheduler and workers. If you modify the Azure Resource Manager (ARM) template provided, make sure that you provide a public hostname for the head node and the worker nodes. 109 | 110 | ## Use Autoscaling 111 | 112 | To optimize the number of virtual machines running MATLAB workers, enable autoscaling by setting `Enable Autoscaling` to `Yes` when you deploy the template. Autoscaling is optional and is disabled by default. 113 | 114 | When autoscaling is disabled, the Virtual Machine Scale Set (VMSS) deploys `Num Worker Nodes` instances. To change the number of worker nodes, use the Azure Portal. 115 | 116 | If you enable autoscaling, the capacity of the VMSS is regulated by the number of workers needed by the cluster. The number of virtual machines is initially set to `Num Worker Nodes`. It then fluctuates between `Min` and `Max Worker Nodes`. To change these limits after deployment, use the Azure Portal and modify the tags on the VMSS. To change the amount of time for which idle nodes are preserved, adjust the value of the tag `mwWorkerIdleTimeoutMinutes`. Do not use the "manual scale" option to change the instance count, as doing so can lead to the unsafe termination of MATLAB workers. 117 | 118 | Ensure that the `Max Worker Nodes` parameter is within your Azure subscription quotas for the specific instance type. To learn about setting quotas, see [Azure subscription and service limits, quotas, and constraints](https://learn.microsoft.com/azure/azure-resource-manager/management/azure-subscription-service-limits). 119 | 120 | To disable autoscaling after the deployment, redeploy the template with autoscaling disabled. 121 | 122 | ## Automatically Terminate the MATLAB Parallel Server Cluster 123 | 124 | Use the `Automatically Terminate Cluster` parameter while deploying the resource group to manage costs efficiently. You can choose one of these options: 125 | 126 | * `Never` (default): No auto-termination but can be enabled after deployment. 127 | * `When cluster is idle`: Terminates the cluster when it is idle for about 10 minutes (30 minutes at startup). 128 | * `After x hours`: Terminates the cluster after 'x' hours (where `x` is between 1 and 24). 129 | * `Disable auto-termination`: No auto-termination. Use this option to fully disable this feature or if you do not have the permissions to create and assign roles in your subscription. 130 | 131 | When the cluster is auto-terminated, the head node virtual machine is deallocated and all worker virtual machines are deleted. To use the cluster again, restart the head node. 132 | 133 | To modify the termination policy after deploying the resource group, edit the value of the tag `mw-autoshutdown` that is attached to the head node. Set the value of the tag to either `never`, `on_idle`, or `After x hours`, where x must be an integer between 1 and 24. 134 | 135 | ## MATLAB Job Scheduler Configuration 136 | 137 | By default, MATLAB Job Scheduler (MJS) is configured to manage a wide range of cluster uses. 138 | 139 | To change the MJS configuration for advanced use cases, replace the default `mjs_def` file with your own file using the template parameter `OptionalUserCommand`. Doing so overwrites all MJS startup parameters, except for *DEFAULT_JOB_MANAGER_NAME*, *HOSTNAME*, and *SHARED_SECRET_FILE*. For more information about the MJS startup parameters and to edit them, see [Define MATLAB Job Scheduler Startup Parameters](https://www.mathworks.com/help/matlab-parallel-server/define-startup-parameters.html). 140 | For example, to retrieve and use your edited `mjs_def` from a storage service (such as Azure blob storage), set the `OptionalUserCommand` to the following: 141 | ``` 142 | Invoke-WebRequest "https://.blob.core.windows.net//mjs_def.bat" -OutFile $Env:MJSDefFile 143 | ``` 144 | 145 | ## Troubleshooting 146 | If your resource group fails to deploy, check the Deployments section of the Resource Group. This section indicates which resource deployments failed and allow you to navigate to the causing error message. 147 | 148 | If the resource group deployed successfully but you are unable to validate the cluster, check the logs on the instances to diagnose the error. The deployment logs are output to C:\ProgramData\MathWorks\MDCSLog-*.log on the instance nodes. The cluster logs are output to C:\ProgramData\MJS\Log. The system startup logs are output to C:\ProgramData\MathWorks\startup.log on every instance of the cluster. 149 | 150 | ---- 151 | 152 | Copyright 2018-2024 The MathWorks, Inc. 153 | 154 | ---- -------------------------------------------------------------------------------- /releases/R2025a/README.md: -------------------------------------------------------------------------------- 1 | # MATLAB Parallel Server on Azure (Windows VM) 2 | 3 | ## Step 1. Deploy the Template 4 | 5 | Click the **Deploy to Azure** button below to deploy the cloud resources on Azure®. Doing so opens the Azure Portal in your web browser. 6 | 7 | | Create Virtual Network | Use Existing Virtual Network | 8 | | --- | --- | 9 | | Use this option to deploy the resources in a new virtual network



| Use this option to deploy the resources in an existing virtual network



| 10 | 11 | > Cluster Platform: Windows Server 2022 12 | 13 | > MATLAB® Release: R2025a 14 | 15 | ## Step 2. Configure the Cloud Resources 16 | Clicking the **Deploy to Azure** button opens the "Custom deployment" page in your browser. You can configure the parameters on this page. It is easier to complete the steps if you position these instructions and the Azure Portal window side by side. Create a new resource group by clicking **Create New**. Alternatively, you can select an existing resource group, but doing so can cause conflicts if resources are already deployed in it. 17 | 18 | 1. Specify and check the default settings for these resource parameters. 19 | 20 | | Parameter Label | Description | 21 | | --------------- | ----------- | 22 | | **Cluster Name** | Name to use for this cluster. This name is shown in MATLAB as the cluster profile name. | 23 | | **Num Worker Nodes** | Number of Azure virtual machines to start for the workers to run on. | 24 | | **Min Worker Nodes** | Minimum number of running Azure virtual machines. | 25 | | **Max Worker Nodes** | Maximum number of running Azure virtual machines. | 26 | | **Num Workers Per Node** | Number of MATLAB worker processes to start on each virtual machine (VM). Specify 1 worker for every 2 vCPUs so that each physical core has 1 worker. For example, a Standard_D64s_v3 VM has 64 vCPUs, so it can support 32 MATLAB workers. See https://learn.microsoft.com/azure/virtual-machines/sizes for details on vCPUs for each VM size. | 27 | | **Head Node Vm Size** | Azure virtual machine (VM) size to use for the head node, which runs the job manager. No workers are started on this node, so this node can be a smaller VM type than the worker nodes. By default, the heap memory for the job manager is set between 1024 MiB and a maximum of half of the VM memory, depending on the total number of MATLAB workers. See https://learn.microsoft.com/azure/virtual-machines/sizes for a list of VMs. | 28 | | **Worker Vm Size** | Azure virtual machine (VM) size to use for the workers. By default, the heap memory for all worker process is set between 1024 MiB and a maximum of a quarter of the VM memory, depending on the number of MATLAB workers on the VM. See https://learn.microsoft.com/azure/virtual-machines/sizes for a list of VMs. | 29 | | **Use Spot Instances For Worker Nodes** | Option indicating whether to enable Azure Spot Virtual Machines for worker nodes. For more information, refer to the FAQ section in the deployment README. | 30 | | **Database Volume Size** | Size in GB of the volume to store the database files. All job and task information, including input and output data, is stored on this volume and so it must have enough capacity to store the expected amount of data. If set to 0, no volume is created and the root volume of the VM is used for the database. | 31 | | **Client IP Address** | IP address range that can be used to access the cluster from MATLAB. This range must be a valid IP CIDR range of the form x.x.x.x/x. Use the value <your_public_ip_address>/32 to restrict access to only your computer. | 32 | | **Admin Username** | Admin username for the cluster. To avoid deployment errors, conform to the list of [Disallowed Values (Microsoft)](https://learn.microsoft.com/rest/api/compute/virtual-machines/create-or-update?tabs=HTTP#osprofile) for adminUsername. | 33 | | **Admin Password** | Choose the password for the admin user of the cluster. This password and the chosen admin username are required to log in into any VM in the cluster using RDP or SSH. For the deployment to succeed, your password must meet Azure's password requirements. See [Password requirements when creating a VM](https://learn.microsoft.com/azure/virtual-machines/windows/faq?WT.mc_id=Portal-fx#what-are-the-password-requirements-when-creating-a-vm-) for information on the password requirements. | 34 | | **Virtual Network Resource ID** | Resource ID of an existing virtual network to deploy your cluster into. You can find ID this under the properties of your virtual network. Specify this parameter only when deploying with the Existing Virtual Network option. | 35 | | **Subnet Name** | Name of an existing subnet within your virtual network to deploy your cluster into. Specify this parameter only when deploying with the Existing Virtual Network option. | 36 | | **License Server** | License manager for MATLAB, specified as a string in the form port@hostname. If not specified, online licensing is used. Otherwise, license manager must be accessible from the specified virtual network and subnets. For more information, see https://github.com/mathworks-ref-arch/license-manager-for-matlab-on-azure. | 37 | | **MJS Security Level** | Security level for the cluster. Level 0: Any user can access any jobs and tasks. Level 1: Accessing other users' jobs and tasks issues a warning. However, all users can still perform all actions. Level 2: Users must enter a password to access their jobs and tasks. The job owner can grant access to other users. | 38 | | **Enable Autoscaling** | Option indicating whether VM autoscaling is enabled. For more information about autoscaling, refer to the 'Use Autoscaling' section in the deployment README. | 39 | | **Automatically Terminate Cluster** | Option to autoterminate the cluster after a few hours or when idle. When the cluster is terminated, all worker nodes are deleted and the headnode is deallocated. Select 'Never' to disable auto-termination now but you can enable it later. Select 'Disable auto-termination' to fully disable this feature or if you do not have the permissions to create and assign roles in your subscription. For more information, see [Automatically terminate the MATLAB Parallel Server cluster](https://github.com/mathworks-ref-arch/matlab-parallel-server-on-azure-lin#automatically-terminate-the-matlab-parallel-server-cluster). | 40 | | **MJS Scheduling Algorithm** | Scheduling algorithm for the job manager. 'standard' spreads communicating jobs across as few worker machines as possible to reduce communication overheads and fills in unused spaces on worker machines with independent jobs. Suitable for good behavior for a wide range of uses including autoscaling. 'loadBalancing' distributes load evenly across the cluster to give as many resources as possible to running jobs and tasks when the cluster is underutilized. | 41 | | **Optional User Command** | Provide an optional inline PowerShell command to run on machine launch. For example, to set an environment variable CLOUD=AZURE, use this command excluding the angle brackets: <[System.Environment]::SetEnvironmentVariable("CLOUD","AZURE", "Machine");>. You can use either double quotes or two single quotes. To run an external script, use this command excluding the angle brackets: <Invoke-WebRequest "https://www.example.com/script.ps1" -OutFile script.ps1; .\script.ps1>. Find the logs at 'C:\ProgramData\MathWorks\startup.log'. | 42 | | **Image ID** | Optional Resource ID of a custom managed image in the target region. To use a prebuilt MathWorks image instead, leave this field empty. If you customize the build, for example by removing or modifying the included scripts, this can make the image incompatible with the provided ARM template. To ensure compatibility, modify the ARM template or image accordingly. | 43 | 44 | 45 | **NOTE**: If you are using the network license manager, the port and hostname of the network license manager must be reachable from the MATLAB Parallel Server™ virtual machines (VMs). Deploying into a subnet within the same virtual network as the network license manager is a best practice. 46 | 47 | 2. Click the **Review + create** button to review the Azure Marketplace terms and conditions. 48 | 49 | 3. Click the **Create** button. 50 | 51 | When you click the **Create** button, the resources are created using Azure template deployments. Template deployment can take several minutes. 52 | 53 | # Step 3: Connect to Your Cluster From MATLAB 54 | 55 | 1. Clicking **Create** opens the Deployment Details page, where you can monitor the progress of your deployment. Wait for the message **Your deployment is complete**. 56 | 2. Go to your resource group, and select the Storage Account starting with **mwstorage**. The screen must look like the one in Figure 1. 57 | 58 | ![Resource Group On Completion](../../img/Deployment_Complete_Select_Storage.png) 59 | 60 | *Figure 1: Resource Group On Completion* 61 | 62 | 3. Under **Data Storage** on the left panel, click **File shares**, and select the file share named "shared". 63 | 4. Click **Browse** on the left panel, then open the "cluster" folder. 64 | 5. Download the file, `.mlsettings`, where NAME is the name of your MATLAB job scheduler. 65 | 6. Open MATLAB. 66 | 7. In the Parallel drop-down menu in the MATLAB toolstrip select **Create and Manage Clusters**. 67 | 8. Click **Import**. 68 | 9. Select the downloaded profile and click open. 69 | 10. Click **Set as Default**. 70 | 11. (Optional) Validate your cluster by clicking the **Validate** button. 71 | 72 | After you set the cloud cluster as default, the next time you run a parallel language command (such as `parfor`, `spmd`, `parfeval` or `batch`), MATLAB connects to the cluster. The first time you connect, you are prompted for your MathWorks® account login. The first time you run a task on a worker, it takes several minutes for the worker MATLAB to start. This delay is due to provisioning the instance disk. This is a one-time operation, and subsequent tasks begin faster. 73 | 74 | Your cluster is now ready to use. 75 | 76 | **NOTE**: Use the profile and client IP address range to control access to your cloud resources. Anyone with this file can connect to your resources from a machine within the specified IP address range and run jobs on it. 77 | 78 | Your cluster remains running after you close MATLAB. 79 | 80 | ## Delete Your Cloud Resources 81 | You can remove the resource group and all associated resources when you are done with them. Note that you cannot recover resources once they are deleted. After you delete the cloud resources, you cannot use the downloaded profile again. 82 | 1. Log in to the Azure Portal. 83 | 2. Select the resource group containing your resources. 84 | 3. Select the "Delete resource group" icon to destroy all resources deployed in this group. 85 | 4. You are prompted to enter the name of the resource group to confirm the deletion. 86 | 87 | ![Resource Group Delete](../../img/Resource_Group_Delete.png) 88 | 89 | # Additional Information 90 | 91 | ## Port Requirements 92 | 93 | Before you can use your MATLAB Parallel Server cluster, you must configure certain required ports on the cluster and client firewall. These ports allow your client machine to connect to the cluster headnode and facilitate communication between the cluster nodes. 94 | 95 | ### Cluster Nodes 96 | 97 | For details about the port requirements for cluster nodes, see this information from MathWorks® Support Team on MATLAB Answers: [How do I configure MATLAB Parallel Server using the MATLAB Job Scheduler to work within a firewall?]( https://www.mathworks.com/matlabcentral/answers/94254-how-do-i-configure-matlab-parallel-server-using-the-matlab-job-scheduler-to-work-within-a-firewall). 98 | 99 | Additionally, if your client machine is outside the cluster’s network, then you must configure the network security group of your cluster to allow incoming traffic from your client machine on the following ports. For information on how to configure your network security group, see [Create a security rule](https://learn.microsoft.com/azure/virtual-network/manage-network-security-group). To troubleshoot, see this [page](https://learn.microsoft.com/troubleshoot/azure/virtual-machines/windows/troubleshoot-rdp-nsg-problem). 100 | 101 | | Required ports | Description | 102 | | -------------- | ----------- | 103 | | TCP 27350 to 27358 + 4*N | For connecting to the job manager on the cluster headnode and to the worker nodes for parallel pools. Calculate the required ports based on N, the maximum number of workers on any single node across the entire cluster. | 104 | | TCP 443 | If you are using online licensing, you must open this port for outbound communication from all cluster machines. If you’re using Network License Manager instead, then you must configure ports as listed on [Network License Manager for MATLAB on Microsoft Azure](https://github.com/mathworks-ref-arch/license-manager-for-matlab-on-azure?tab=readme-ov-file#networking-resources). | 105 | | TCP 3389 | Remote Desktop Protocol to access to cluster nodes. | 106 | | TCP 22 | SSH access to cluster nodes. | 107 | 108 | *Table 1: Outgoing port requirements* 109 | 110 | By default, MATLAB Parallel Server is configured with the public hostname of each machine to allow the MATLAB client to access both the scheduler and workers. If you modify the Azure Resource Manager (ARM) template provided, make sure that you provide a public hostname for the head node and the worker nodes. 111 | 112 | ## Use Autoscaling 113 | 114 | To optimize the number of virtual machines running MATLAB workers, enable autoscaling by setting `Enable Autoscaling` to `Yes` when you deploy the template. Autoscaling is optional and is disabled by default. 115 | 116 | When autoscaling is disabled, the Virtual Machine Scale Set (VMSS) deploys `Num Worker Nodes` instances. To change the number of worker nodes, use the Azure Portal. 117 | 118 | If you enable autoscaling, the capacity of the VMSS is regulated by the number of workers needed by the cluster. The number of virtual machines is initially set to `Num Worker Nodes`. It then fluctuates between `Min` and `Max Worker Nodes`. To change these limits after deployment, use the Azure Portal and modify the tags on the VMSS. To change the amount of time for which idle nodes are preserved, adjust the value of the tag `mwWorkerIdleTimeoutMinutes`. Do not use the "manual scale" option to change the instance count, as doing so can lead to the unsafe termination of MATLAB workers. 119 | 120 | Ensure that the `Max Worker Nodes` parameter is within your Azure subscription quotas for the specific instance type. To learn about setting quotas, see [Azure subscription and service limits, quotas, and constraints](https://learn.microsoft.com/azure/azure-resource-manager/management/azure-subscription-service-limits). 121 | 122 | To disable autoscaling after the deployment, redeploy the template with autoscaling disabled. 123 | 124 | ## Automatically Terminate the MATLAB Parallel Server Cluster 125 | 126 | Use the `Automatically Terminate Cluster` parameter while deploying the resource group to manage costs efficiently. You can choose one of these options: 127 | 128 | * `Never` (default): No auto-termination but can be enabled after deployment. 129 | * `When cluster is idle`: Terminates the cluster when it is idle for about 10 minutes (30 minutes at startup). 130 | * `After x hours`: Terminates the cluster after 'x' hours (where `x` is between 1 and 24). 131 | * `Disable auto-termination`: No auto-termination. Use this option to fully disable this feature or if you do not have the permissions to create and assign roles in your subscription. 132 | 133 | When the cluster is auto-terminated, the head node virtual machine is deallocated and all worker virtual machines are deleted. To use the cluster again, restart the head node. 134 | 135 | To modify the termination policy after deploying the resource group, edit the value of the tag `mw-autoshutdown` that is attached to the head node. Set the value of the tag to either `never`, `on_idle`, or `After x hours`, where x must be an integer between 1 and 24. 136 | 137 | ## MATLAB Job Scheduler Configuration 138 | 139 | By default, MATLAB Job Scheduler (MJS) is configured to manage a wide range of cluster uses. 140 | 141 | To change the MJS configuration for advanced use cases, replace the default `mjs_def` file with your own file using the template parameter `OptionalUserCommand`. Doing so overwrites all MJS startup parameters, except for *DEFAULT_JOB_MANAGER_NAME*, *HOSTNAME*, and *SHARED_SECRET_FILE*. For more information about the MJS startup parameters and to edit them, see [Define MATLAB Job Scheduler Startup Parameters](https://www.mathworks.com/help/matlab-parallel-server/define-startup-parameters.html). 142 | For example, to retrieve and use your edited `mjs_def` from a storage service (such as Azure blob storage), set the `OptionalUserCommand` to the following: 143 | ``` 144 | Invoke-WebRequest "https://.blob.core.windows.net//mjs_def.bat" -OutFile $Env:MJSDefFile 145 | ``` 146 | 147 | ## Troubleshooting 148 | If your resource group fails to deploy, check the Deployments section of the Resource Group. This section indicates which resource deployments failed and allow you to navigate to the causing error message. 149 | 150 | If the resource group deployed successfully but you are unable to validate the cluster, check the logs on the instances to diagnose the error. The deployment logs are output to C:\ProgramData\MathWorks\MDCSLog-*.log on the instance nodes. The cluster logs are output to C:\ProgramData\MJS\Log. The system startup logs are output to C:\ProgramData\MathWorks\startup.log on every instance of the cluster. 151 | 152 | ---- 153 | 154 | Copyright 2018-2025 The MathWorks, Inc. 155 | 156 | ---- --------------------------------------------------------------------------------