├── Best Practices.md ├── LICENSE ├── Marketplace Stack Schema.md ├── Marketplace and Quick Start.md ├── README.md ├── actions ├── README.md ├── build-orm-zip │ ├── Dockerfile │ └── entrypoint.sh ├── create-image │ ├── Dockerfile │ ├── action.yml │ ├── entrypoint.sh │ └── marketplace_image.json ├── terraform-apply │ ├── Dockerfile │ ├── apply_test.go │ └── entrypoint.sh ├── update-listing │ ├── Dockerfile │ ├── __init__.py │ ├── entrypoint.sh │ ├── mpapihelper.py │ └── mpctl.py └── update-stack-vars │ ├── Dockerfile │ └── entrypoint.sh └── scripts ├── README.md └── mkpl_setup.sh /Best Practices.md: -------------------------------------------------------------------------------- 1 | # Best Practices 2 | 3 | ## What is a Quick Start? 4 | A Quick Start allows a customer to deploy some ISV's software product (e.g. Couchbase, Cloudera, Datastax). It deploys an appropriate architecture and installs the software in their tenancy. A Quick Start has defaults that deploy a simple example of the software. It requires as few parameters as possible and does not depend on any existing resources. It is parameterized in such a way that the same Quickstart can be used for more complicated deployments. 5 | 6 | ## Core Principle - Usability First 7 | Quick Start deployments are intended to give OCI users a quick start running some piece of software. As a result we prioritize usability above all else. A user who knows little to nothing about OCI should be able to stumble in, hit deploy and get a running thing in 5-10 minutes. Some might argue the system isn't production grade. That's ok. It can be improved. Additional Terraform for more complex systems can be provided. But, the base version needs to be braindead simple. 8 | 9 | ## Packages exist for a reason. Use them. 10 | Yes, there may be a tgz or zip. Don't use it! Use the package. Somebody who knows about the application at hand built a package. It includes lots of logic you don't want to rewrite. Follow the three virtues and be lazy! 11 | 12 | ## We use cloud-init. We do use not remote-exec or local-exec. 13 | Yes, those are nifty features of Terraform. That doesn't mean you have to use them. Before Terraform there was cloud-init. It's more scalable than SSH'ing into each node. It's more robust to connectivity issues (like closing your laptop before deploy is done) and it runs asynchronously. Beyond that, it's the model every other cloud uses. We use it wherever we can. If there's some reason we can, then fine, drop down to SSH'ing to a node. 14 | 15 | ## Reference oci-quickstart-prerequisites or oke-quickstart-prerequisites 16 | Don't create individual env-vars files. Don't provide an OCI how to in each repo. Otherwise we're going to end up maintaining many copies of the same thing. 17 | 18 | ## Repo Structure 19 | * A Quick Start will always be some collection of TF and shell. 20 | * At a minimum a Quick Start will create a VCN and some set of instances. 21 | * A style guide or a generic example template(s) should be developed. 22 | * Since loops and conditionals in TF aren't fully developed (and there are open features/issues around this) we should have standard examples for common constructs. 23 | * For MVP Quick Starts they should deploy to one region and instances requiring HA should be mapped to FD's. 24 | * TF allows at least 3 ways to execute shell on an instance we should standardize on one. The best combination of flexibility and readability may be using user_data resolving TF vars as shell vars inline. Use of remote-exec precludes the use of ORM. ORM can use remote-exec, but that comes with the same issues as running it locally, plus needing to deal with keys differently than locally. 25 | * Gathering information in the shell is in general preferred over passing in a parameter. 26 | * Since referencing a generic module (e.g. a VCN template) in multiple Quick Starts requires CI/CD of the module, at first all Quick Starts should contain all required resources. 27 | * We should conditionalize the use of either NVME or block storage based on shape. 28 | 29 | ## Documentation 30 | Documentation of a Quick Start should be standardized and contain the same sections: 31 | 32 | * Introduction on what the software is and what it's used for. 33 | * Architecture diagram and description of default deployment and any required parameters. 34 | * Detailed description of all parameters. 35 | * Examples of more complicated deployments with arch diagrams if necessary. 36 | * Any necessary post-deploy actions. Note, these should be minimized and ideally there are none. 37 | 38 | ## Security 39 | * Passwords are set to a non-default value. 40 | * Only required ports are opened. 41 | * Connections allowed only from the user's CIDR block optional. 42 | * No HTTP allowed for management consoles, HTTP ports may be open if they redirect to HTTPS. 43 | * For software that manages the install/config private keys should not be held by OCI. They should be generated on the fly and the public key passed via object storage to instances that need it. 44 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2018-2020 Oracle and/or its affiliates. All rights reserved. 2 | 3 | The Universal Permissive License (UPL), Version 1.0 4 | 5 | Subject to the condition set forth below, permission is hereby granted to any person obtaining a copy of this 6 | software, associated documentation and/or data (collectively the "Software"), free of charge and under any and 7 | all copyright rights in the Software, and any and all patent rights owned or freely licensable by each licensor 8 | hereunder covering either (i) the unmodified Software as contributed to or provided by such licensor, or 9 | (ii) the Larger Works (as defined below), to deal in both 10 | 11 | (a) the Software, and 12 | (b) any piece of software and/or hardware listed in the lrgrwrks.txt file if one is included with the Software 13 | (each a “Larger Work” to which the Software is contributed by such licensors), 14 | 15 | without restriction, including without limitation the rights to copy, create derivative works of, display, 16 | perform, and distribute the Software and make, use, sell, offer for sale, import, export, have made, and have 17 | sold the Software and the Larger Work(s), and to sublicense the foregoing rights on either these or other terms. 18 | 19 | This license is subject to the following condition: 20 | The above copyright notice and either this complete permission notice or at a minimum a reference to the UPL must 21 | be included in all copies or substantial portions of the Software. 22 | 23 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO 24 | THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 25 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF 26 | CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS 27 | IN THE SOFTWARE. 28 | -------------------------------------------------------------------------------- /Marketplace Stack Schema.md: -------------------------------------------------------------------------------- 1 | # Marketplace Stack Schema 2 | Oracle Cloud Infrastructure Marketplace is an online store that offers applications specifically for customers of Oracle Cloud Infrastructure. In the Oracle Cloud Infrastructure Marketplace catalog, customers can find and launch Images and Stacks from Oracle or trusted partners. These Marketplace Images or Stacks are deployed into the customer's tenancy. 3 | 4 | Marketplace Images are templates of virtual hard drives that determine the operating system and software to run on an instance. Publishers will publish a [Custom Image](https://docs.cloud.oracle.com/iaas/Content/Compute/Tasks/managingcustomimages.htm) as an Artifact in the [Marketplace Partner Portal](https://partner.cloudmarketplace.oracle.com/partner/index.html). 5 | 6 | [Marketplace Stacks](https://docs.oracle.com/en/cloud/marketplace/partner-portal/partp/how-do-i-publish-oracle-cloud-infrastructure-stack.html) are composed by Terraform configuration files (templates) and a YAML descriptor [file](https://docs.oracle.com/en/cloud/marketplace/partner-portal/partp/schema-stack-input-variables.html) containing the definition of Terraform Input variables, packaged all together in a zip file. A Stack run as a job in the OCI [Resource Manager](https://docs.cloud.oracle.com/iaas/Content/ResourceManager/Concepts/resourcemanager.htm) service. 7 | 8 | A configuration is a set of one or more Terraform configuration (.tf) file written in HCL, that specify the Oracle Cloud Infrastructure resources, including resource metadata, and other important information, like data source definitions and variables declarations. 9 | 10 | Terraform variables are defined as environment variables, command line arguments or placed into TFVars file when they are launched from Terraform CLI. Running a standalone ORM Stack (not Marketplace related) is possible to set variables at the time a Stack is created via OCI CLI, by specifying a json file. 11 | 12 | In order to enable any user - technical or non-technical to launch a Marketplace Stack directly from the OCI console without depending on any CLI, Marketplace/ORM provided a unique feature to enable publishers to create a custom UI for their Stacks where they can create a Form based on the same look and feel of OCI console and also can link the OCI created resources with the Marketplace listing that originated it. This UI is defined in the YAML descriptor file packaged along with the Terraform templates. 13 | 14 | ## Marketplace Stack Requirements 15 | 16 | | # | Description | 17 | | :--- | :---------- | 18 | | 1 | Schema file is based on [YAML](https://yaml.org/) | 19 | | 2 | All variables declared in the schema file MUST exist in the Terraform configuration. | 20 | | 3 | The `tenancy_ocid` and `region` variables must be declared as part of the schema, but placed in a hidden `variableGroups` section as their values are set automatically by ORM based on customer selections in the OCI console. | 21 | | 4 | Variables not declared in the schema but declared in the Terraform root module will be displayed in the bottom of the Variables configuration page in ORM. If you don't want to display these variables in ORM, declare them within the `variableGroups` section as part of a group that is hidden from users. | 22 | | 5 | You must create different sections within the `variableGroups` definition for grouping similar OCI Services, e.g. Network, Storage, Compute. That will make the user experience similar to standard OCI UI. For example, create a section to group Network resources and put all variables that manage network (VCN, Subnets, etc) into that group. | 23 | 24 | ## YAML Schema Building Blocks 25 | 26 | ### 1. Marketplace Application Information Tab 27 | 28 | This block contains general information related to the listing that is displayed in the Application Information tab in the OCI Console, e.g. application description, logo/icon URL, listing-id, content-language and some boilerplate code related to the schema version. 29 | 30 | #### Fields 31 | | Name | Description | 32 | | :--- | :---------- | 33 | | title | Title of the listing displayed in the OCI console - Application Information tab | 34 | | description | Sub Title shown in Application Information tab. | 35 | | schemaVersion | YAML Schema version. Fixed Value = `1.1.0` | 36 | | version | Marketplace API Version. Fixed Value = `20190404` | 37 | | logoUrl (optional) | URL of Logo Icon used on Application Information tab. You can copy the `contentId` from the Marketplace listing logo URL in the Marketplace Partner portal. | 38 | | source (optional)| This field is used by ORM to display the "View Instructions" section of the Marketplace listing.
`type: marketplace` is a fixed value
`reference:` this is the listing idenfier and you can copy this information from the preview URL of the Marketplace listing `application_id=12345` URL parameter. | 39 | | locale | Listing Locale, e.g. `en` for English | 40 | 41 | #### Sample Code 42 | ```yaml 43 | title: "My Super Application" 44 | description: "This is the best application in the Marketplace" 45 | schemaVersion: 1.1.0 46 | version: "20190304" 47 | 48 | logoUrl: "https://cloudmarketplace.oracle.com/marketplace/content?contentId=58352039" 49 | 50 | source: 51 | type: marketplace 52 | reference: 47726045 53 | 54 | locale: "en" 55 | ``` 56 | 57 | ### 2. variableGroups 58 | 59 | #### Fields 60 | 61 | | Name | Description | 62 | | :--- | :---------- | 63 | | variableGroups | Top level identifier of `variableGroups` section | 64 | | title | Title of the Group | 65 | | variables | list of variables that are presented/hidden in this group | 66 | | visible (optional) | set the visibility of variableGroups block - use a boolean or an expression. Visibility can be defined at variable level.| 67 | 68 | #### Sample Code 69 | 70 | ```yaml 71 | variableGroups: 72 | - title: "Hidden Variable Group" 73 | visible: false 74 | variables: 75 | #variables used by Terraform but not necesarrily exposed exposed to end user 76 | - tenancy_ocid 77 | - region 78 | - mp_listing_id 79 | - mp_listing_resource_id 80 | - mp_listing_resource_version 81 | - availability_domain_number 82 | 83 | - title: "Compute Configuration" 84 | variables: 85 | - compute_compartment_ocid 86 | - vm_display_name 87 | - hostname_label 88 | - vm_compute_shape 89 | - availability_domain_name 90 | - ssh_public_key 91 | 92 | - title: "Virtual Cloud Network" 93 | variables: 94 | - network_compartment_ocid 95 | - network_strategy 96 | - network_configuration_strategy 97 | - vcn_id 98 | - vcn_display_name 99 | - vcn_dns_label 100 | - vcn_cidr_block 101 | 102 | #Management subnet group contains logic to toggle the visibility of the group based on variables defined by customer during Stack configuration in the OCI console. 103 | - title: "Management Subnet" 104 | visible: #($network_strategy == ""Use Existing VCN and Subnet"") OR (network_configuration_strategy == "Customize Network Configuration") 105 | or: 106 | - eq: 107 | - network_strategy 108 | - "Use Existing VCN and Subnet" 109 | - eq: 110 | - network_configuration_strategy 111 | - "Customize Network Configuration" 112 | variables: 113 | - mgmt_subnet_type 114 | - mgmt_subnet_id 115 | - mgmt_subnet_display_name 116 | - mgmt_subnet_cidr_block 117 | - mgmt_subnet_dns_label 118 | - mgmt_nsg_configuration_strategy 119 | 120 | ``` 121 | 122 | ### 3. Terraform Variables 123 | 124 | Use this section to declare all Terraform Input variables that are presented in your Template. All variables in the Terraform Root Module should be declared. 125 | 126 | #### Variables 127 | 128 | | Name | Description | 129 | | :--- | :---------- | 130 | | variables | Top level identifier of `variables` section | 131 | | `` | The name of the input variable declared in the Terraform template. | 132 | | type | Definition of the variable type supported by the schema. This is not a Terraform variable type as the OCI Marketplace schema support dynamic types that will automatically lookup for OCI resource types. See Dynamic types.| 133 | | minLength | Minimum Length of the variable | 134 | | maxLength | Maximum Length of the variable | 135 | | pattern | Regex pattern | 136 | | title | Label of the variable. In case a label is not specified, ORM will use the variable name as the title. | 137 | | description | Tooltip with the description of the variable. ORM will look at the value provided within the variables definition of the Terraform template in case a description is not provided in the schema | 138 | | default | Variable default value | 139 | | required | set the variable requirement - use a boolean or an expression | 140 | | visible | set the visibility of a variable - use a boolean or an expression | 141 | 142 | #### Variable Types 143 | 144 | Variable Types in the Schema definition are statically or dynamically resolved based on customer's tenancy and OCI Console navigation. Default value defined in the terraform configuration is overwritten by the value specified in ORM. 145 | 146 | | Static Types | Description | 147 | | :----------- | :---------- | 148 | | array | List of values | 149 | | boolean | `true` or `false` | 150 | | enum | List of static sorted values | 151 | | integer | Integer | 152 | | number | Number | 153 | | string | Text | 154 | | password | Hidden Text* | 155 | | datetime | Current Date Time | 156 | 157 | ***Note**: variable is visible as plaintext in the terraform state file 158 | 159 | | Dynamic Types | Description | Depends on | Filter | 160 | | :------------ | :---------- | :--------- | :----- | 161 | | oci:identity:region:name | OCI Region | | | 162 | | oci:identity:compartment:id | Compartment Id | | | 163 | | oci:core:image:id | List of values | | | 164 | | oci:core:instanceshape:name | List of Compute Shapes | compartmentId | imageId | 165 | | oci:core:vcn:id | List of VCNs in the existing region | compartmentId | | 166 | | oci:core:subnet:id | List of Subnets | vcnId compartmentId | hidePublicSubnet hidePrivateSubnet hideRegionalSubnet hideAdSubnet | 167 | | oci:identity:availabilitydomain:name | Region Availability Domain | compartmentId | | 168 | | oci:identity:faultdomain:name | AD Fault Domains | compartmentId availabilityDomainName | | 169 | | oci:database:dbsystem:id | Oracle DB Systems (Exadata, Bare Metal and VM) | | | 170 | | oci:database:dbhome:id | Oracle DB Systems Home Folder | compartmentId dbSystemId | | 171 | | oci:database:database:id | Oracle DB ID | | | 172 | | oci:database:autonomousdatabase:id | Oracle Autonomous Database ID | | | 173 | 174 | #### Sample Code 175 | 176 | ```yaml 177 | variables: 178 | # HIDDEN variables 179 | tenancy_ocid: 180 | type: string 181 | title: Tenancy ID 182 | description: The Oracle Cloud Identifier (OCID) for your tenancy 183 | required: true 184 | 185 | region: 186 | type: oci:identity:region:name 187 | title: Region 188 | description: The region in which to create all resources 189 | required: true 190 | 191 | # MARKETPLACE VARIABLES 192 | mp_listing_id: 193 | type: string 194 | required: true 195 | description: Marketplace Listing ID 196 | 197 | mp_listing_resource_id: 198 | type: oci:core:image:id 199 | required: true 200 | description: Marketplace Image OCID 201 | dependsOn: 202 | compartmentId: compute_compartment_ocid 203 | 204 | mp_listing_resource_version: 205 | type: string 206 | required: true 207 | description: Marketplace Listing package version 208 | 209 | availability_domain_number: 210 | type: string 211 | required: false 212 | description: Availability Domain Number 213 | 214 | # COMPUTE VARIABLES 215 | compute_compartment_ocid: 216 | type: oci:identity:compartment:id 217 | required: true 218 | title: Compute Compartment 219 | description: The compartment in which to create all Compute resources 220 | default: compartment_ocid 221 | 222 | availability_domain_name: 223 | type: oci:identity:availabilitydomain:name 224 | dependsOn: 225 | compartmentId: compute_compartment_ocid 226 | required: true 227 | default: 1 228 | title: Availability Domain 229 | description: Availability Domain 230 | 231 | ssh_public_key: 232 | type: string 233 | required: true 234 | title: Public SSH Key string 235 | description: Public SSH Key to access VM via SSH 236 | 237 | vm_display_name: 238 | type: string 239 | required: true 240 | title: Instance Name 241 | description: The name of the Instance 242 | 243 | vm_compute_shape: 244 | type: oci:core:instanceshape:name 245 | default: VM.Standard2.4 246 | title: Compute Shape 247 | required: true 248 | dependsOn: 249 | compartmentId: compute_compartment_ocid 250 | imageId: mp_listing_resource_id 251 | 252 | hostname_label: 253 | type: string 254 | required: false 255 | title: DNS Hostname Label 256 | 257 | 258 | # NETWORK CONFIGURATION VARIABLES 259 | network_compartment_ocid: 260 | type: oci:identity:compartment:id 261 | required: true 262 | title: Network Compartment 263 | description: The compartment in which to create all Network resources 264 | default: compartment_ocid 265 | 266 | network_strategy: 267 | type: enum 268 | title: Network Strategy 269 | description: Create or use existing Network Stack (VCN and Subnet) 270 | enum: 271 | - "Create New VCN and Subnet" 272 | - "Use Existing VCN and Subnet" 273 | required: true 274 | default: "Create New VCN and Subnet" 275 | 276 | network_configuration_strategy: 277 | visible: #($network_strategy == ""Create New VCN and Subnet"") 278 | eq: 279 | - network_strategy 280 | - "Create New VCN and Subnet" 281 | type: enum 282 | title: Configuration Strategy 283 | description: Use recommended configuration or customize it 284 | enum: 285 | - "Use Recommended Configuration" 286 | - "Customize Network Configuration" 287 | required: true 288 | default: "Use Recommended Configuration" 289 | 290 | # VCN VARIABLES 291 | vcn_display_name: 292 | visible: #($network_strategy == ""Create New VCN and Subnet"") AND (network_configuration_strategy == "Customize Network Configuration") 293 | and: 294 | - eq: 295 | - network_strategy 296 | - "Create New VCN and Subnet" 297 | - eq: 298 | - network_configuration_strategy 299 | - "Customize Network Configuration" 300 | type: string 301 | required: true 302 | title: Name 303 | description: The name of the new Virtual Cloud Network (VCN) 304 | 305 | vcn_id: 306 | visible: #($network_strategy == "Use Existing VCN and Subnet") 307 | eq: 308 | - network_strategy 309 | - "Use Existing VCN and Subnet" 310 | type: oci:core:vcn:id 311 | dependsOn: 312 | compartmentId: network_compartment_ocid 313 | required: true 314 | title: Existing VCN 315 | description: An existing Virtual Cloud Network (VCN) in which to create the compute instances, network resources, and load balancers. If not specified, a new VCN is created. 316 | 317 | vcn_cidr_block: 318 | visible: #($network_strategy == ""Create New VCN and Subnet"") AND (network_configuration_strategy == "Customize Network Configuration") 319 | and: 320 | - eq: 321 | - network_strategy 322 | - "Create New VCN and Subnet" 323 | - eq: 324 | - network_configuration_strategy 325 | - "Customize Network Configuration" 326 | type: string 327 | required: true 328 | pattern: "^(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9]).(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9]).(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9]).(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\\/(3[0-2]|[1-2]?[0-9])$" 329 | title: CIDR Block 330 | description: The CIDR of the new Virtual Cloud Network (VCN). If you plan to peer this VCN with another VCN, the VCNs must not have overlapping CIDRs. 331 | 332 | vcn_dns_label: 333 | visible: #($network_strategy == ""Create New VCN and Subnet"") AND (network_configuration_strategy == "Customize Network Configuration") 334 | and: 335 | - eq: 336 | - network_strategy 337 | - "Create New VCN and Subnet" 338 | - eq: 339 | - network_configuration_strategy 340 | - "Customize Network Configuration" 341 | type: string 342 | required: true 343 | title: DNS Label 344 | maxLenght: 15 345 | description: VCN DNS Label. Only letters and numbers, starting with a letter. 15 characters max. 346 | 347 | # MANAGEMENT SUBNET VARIABLES 348 | mgmt_subnet_type: 349 | visible: #($network_strategy == ""Create New VCN and Subnet"") 350 | eq: 351 | - network_strategy 352 | - "Create New VCN and Subnet" 353 | type: enum 354 | title: Subnet Type 355 | description: Choose between private and public subnets 356 | enum: 357 | - "Private Subnet" 358 | - "Public Subnet" 359 | required: true 360 | default: "Public Subnet" 361 | 362 | mgmt_subnet_display_name: 363 | visible: #($network_strategy == ""Create New VCN and Subnet"") 364 | eq: 365 | - network_strategy 366 | - "Create New VCN and Subnet" 367 | type: string 368 | required: true 369 | title: Name 370 | description: The name of the new Management Subnet 371 | 372 | mgmt_subnet_cidr_block: 373 | visible: #($network_strategy == ""Create New VCN and Subnet"") 374 | eq: 375 | - network_strategy 376 | - "Create New VCN and Subnet" 377 | type: string 378 | pattern: "^(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9]).(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9]).(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9]).(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9]?[0-9])\\/(3[0-2]|[1-2]?[0-9])$" 379 | required: true 380 | title: CIDR Block 381 | description: The CIDR of the new Subnet. The new subnet's CIDR should not overlap with any other subnet CIDRs. 382 | 383 | mgmt_subnet_id: 384 | visible: #($network_strategy == "Use Existing VCN and Subnet") 385 | eq: 386 | - network_strategy 387 | - "Use Existing VCN and Subnet" 388 | type: oci:core:subnet:id 389 | dependsOn: 390 | vcnId: vcn_id 391 | compartmentId: network_compartment_ocid 392 | hidePublicSubnet: false 393 | hidePrivateSubnet: false 394 | hideRegionalSubnet: false 395 | hideAdSubnet: true 396 | default: '' 397 | required: true 398 | title: Existing Subnet 399 | description: An existing Management subnet. This subnet must already be present in the chosen VCN. 400 | 401 | mgmt_subnet_dns_label: 402 | visible: #($network_strategy == ""Create New VCN and Subnet"") 403 | eq: 404 | - network_strategy 405 | - "Create New VCN and Subnet" 406 | type: string 407 | required: true 408 | title: DNS Label 409 | maxLenght: 15 410 | description: Subnet DNS Label. Only letters and numbers, starting with a letter. 15 characters max. 411 | 412 | ``` 413 | 414 | ### 4. Terraform Outputs 415 | 416 | #### Fields 417 | 418 | | Name | Description | 419 | | :--- | :---------- | 420 | | outputs | Top level identifier of Terraform `outputs` section | 421 | | `` | The name of the output variable declared in the Terraform template.| 422 | | type | Output Variable type: `link`, `csv`, `ocid` | 423 | | title | Label of the Output variable | 424 | | displayText | Tooltip with the description of the variable | 425 | | visible | set the visibility of a variable - use a boolean or an expression | 426 | 427 | #### Sample Code 428 | 429 | ```yaml 430 | outputs: 431 | instance_mgmt_https_url: 432 | type: link 433 | title: Open Management URL 434 | visible: false 435 | 436 | instance_mgmt_public_ip: 437 | type: link 438 | title: Management Public IP 439 | visible: #($mgmt_subnet_type == "Public Subnet") 440 | eq: 441 | - mgmt_subnet_type 442 | - "Public Subnet" 443 | 444 | instance_mgmt_private_ip: 445 | type: link 446 | title: Management Private IP 447 | visible: true 448 | 449 | public_ips_csv: 450 | type: csv 451 | title: Public IPs 452 | 453 | load_balancer_ocid: 454 | type: ocid 455 | title: Load Balancer 456 | 457 | ``` 458 | 459 | ### 5. Primary Output Button 460 | 461 | #### Fields 462 | 463 | | Name | Description | 464 | | :--- | :----------| 465 | | primaryOutputButton | Output Variable linked to the Marketplace Action Button in the Instance Tab (Compute UI) | 466 | 467 | #### Sample Code 468 | 469 | ```yaml 470 | primaryOutputButton: instance_mgmt_https_url 471 | ``` 472 | 473 | ### 6.Output Groups 474 | 475 | Use this section to declare all Terraform Output variables that are presented in your Template. All variables in the Terraform Root Module should be declared. 476 | 477 | 478 | #### Fields 479 | 480 | | Name | Description | 481 | | :--- | :---------- | 482 | | outputGroups | Top level identifier of `outputGroups` section | 483 | | title | Title of the Output Variables Group | 484 | | outputs | Terraform output variables | 485 | | `` | Name of the Output Variable | 486 | 487 | #### Sample Code 488 | 489 | ```yaml 490 | outputGroups: 491 | - title: Schema Registry 492 | outputs: 493 | - schemaRegistryUrl 494 | - schemaRegistryPublicIps 495 | - schemaRegistryInstances 496 | - schemaRegistryLoadBalancer 497 | 498 | 499 | - title: Broker / Connect 500 | outputs: 501 | - brokerPublicIps 502 | - brokerInstances 503 | - connectUrl 504 | - connectPublicIps 505 | - restUrl 506 | ``` 507 | -------------------------------------------------------------------------------- /Marketplace and Quick Start.md: -------------------------------------------------------------------------------- 1 | # Marketplace and Quick Start 2 | This is a guide for ISVs to promote their OCI Quick Start to an OCI Marketplace Stack listing. We're viewing the Quick Start as an incubator for Terraform based OCI Marketplace work. We hope you choose to publish your Quick Start in the Marketplace as well. 3 | 4 | ## Contractual Steps 5 | 1. [Join Oracle Partner Network (OPN)](https://www.oracle.com/partners/en/partner-with-oracle/get-started/join-opn/index.html) for $500. 6 | 2. You'll need to complete the publisher agreement [here](https://go.oracle.com/LP=83217). 7 | 8 | ## Technical - Register as a Publisher 9 | Sign up to "Become a Publisher" in the upper right of the [Partner Portal](https://cloudmarketplace.oracle.com/marketplace/en_US/partnerLandingPage). You will need your OPN number. 10 | 11 | ## Technical - Get an OCI Tenancy 12 | You'll need to sign up for an OCI account to build and test the work. We're working on a system to provide credits to ISVs we're partnering with to offset the cost here. Unfortunately, that program is not yet available. 13 | 14 | 1. Sign up for a trial [here](https://cloud.oracle.com/tryit). 15 | 2. Convert the trial to a paid account. 16 | 3. Request increases to service limits. For instance, you might want to request access to GPU instances. 17 | 18 | ## Next Steps 19 | With all that done, it's probably time for technical onboarding. We suggest arranging a joint call to walk through that process and work together on publishing. If you already have a Quick Start, the process will be very lightweight. 20 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # oci-quickstart 2 | The [Oracle Cloud Infrastructure (OCI) Quick Start](https://github.com/oracle-quickstart) is a collection of examples that allow OCI users to get a quick start deploying advanced infrastructure on OCI. 3 | 4 | ## Have an example you want to contribute? 5 | If you are either an Oracle employee or a partner and have something you think might be a good fit for the Quick Start, please reach out to Collin Poczatek (joseph.poczatek@oracle.com). Resources for partners interested in building Quick Starts include: 6 | * [Best Practices](Best%20Practices.md) 7 | * [Marketplace and Quick Start](Marketplace%20and%20Quick%20Start.md) 8 | * [Marketplace Stack Schema](Marketplace%20Stack%20Schema.md) 9 | -------------------------------------------------------------------------------- /actions/README.md: -------------------------------------------------------------------------------- 1 | # Actions 2 | 3 | This collection of Github Actions helps partners test their terraform code against a OCI tenancy, create custom images that are ready for OCI Marketplace validation and also update their listings on the Marketplace. 4 | 5 | The actions can be combined into a workflow that offers an automated way of updating a published marketplace listings. 6 | 7 | ## Requirements 8 | 9 | A partner that wants to leverage the Quick Start Actions must have the following: 10 | 11 | - a Quick Start repo 12 | - the Terraform code must me placed under a 'terraform' directory in the repo root 13 | - The following OCI credentials must be saved as Github secrets in the repo. This requires admin privileges on the repo. The terraform code will be tested against this tenancy, so resources will be created and destroyed in this account. Also, with each run the "create-image" actions will create custom images in this tenancy, make sure to clean these up. 14 | - TF_VAR_user_ocid 15 | - TF_VAR_fingerprint 16 | - TF_VAR_private_key 17 | - TF_VAR_tenancy_ocid 18 | - TF_VAR_compartment_ocid 19 | - API_creds -- for Marketplace login 20 | - a Workflow file that must be stored in the repo under .github/workflows 21 | 22 | A good example of such a repo is https://github.com/oracle-quickstart/oci-h2o 23 | 24 | ## terraform-apply 25 | This action will test any Terraform code that is under the 'terraform' directory in the repo root. The action will create and then destroy all resources. The action uses [Terratest](https://github.com/gruntwork-io/terratest). No other assertions are being performed currently. 26 | 27 | ## build-orm-zip 28 | This action packages the terraform code from the partner repo into an archive that can be later submitted to the marketplace. 29 | 30 | ## create-image 31 | This action creates a marketplace ready image that is based on an existing OCI platform image. 32 | 33 | ## update-listing 34 | This actions takes as input a OCI custom image OCID and a stack archive (terraform provisioning code) and used the API_creds secret to update an existing Marketplace listing with the new code or software that has been updated in the repo. 35 | 36 | ## Example Workflow 37 | 38 | ``` 39 | on: 40 | push: 41 | branches: 42 | - 'master' 43 | name: Marketplace 44 | jobs: 45 | update-stack-listing: 46 | name: Update Stack Listing 47 | runs-on: ubuntu-latest 48 | env: 49 | TF_VAR_compartment_ocid: ${{ secrets.TF_VAR_compartment_ocid }} 50 | TF_VAR_fingerprint: ${{ secrets.TF_VAR_fingerprint }} 51 | TF_VAR_private_key: ${{ secrets.TF_VAR_private_key }} 52 | TF_VAR_private_key_path: $GITHUB_WORKSPACE/oci.pem 53 | TF_VAR_tenancy_ocid: ${{ secrets.TF_VAR_tenancy_ocid }} 54 | TF_VAR_user_ocid: ${{ secrets.TF_VAR_user_ocid }} 55 | API_CREDS: ${{ secrets.API_CREDS }} 56 | steps: 57 | - name: Checkout 58 | uses: actions/checkout@master 59 | - name: Terraform Apply 60 | uses: "oci-quickstart/oci-quickstart/actions/terraform-apply@master" 61 | - name: Build ORM Zip 62 | uses: "oci-quickstart/oci-quickstart/actions/build-orm-zip@master" 63 | if: success() 64 | - name: Update Listing 65 | uses: "oci-quickstart/oci-quickstart/actions/update-listing@master" 66 | ``` 67 | -------------------------------------------------------------------------------- /actions/build-orm-zip/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu 2 | 3 | LABEL "name"="Build ORM Zip" 4 | LABEL "version"="1.0" 5 | 6 | COPY entrypoint.sh / 7 | ENTRYPOINT ["/entrypoint.sh"] 8 | -------------------------------------------------------------------------------- /actions/build-orm-zip/entrypoint.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | apt-get update 4 | apt install -y zip git 5 | 6 | cd ${GITHUB_WORKSPACE}/marketplace 7 | ./build.sh 8 | -------------------------------------------------------------------------------- /actions/create-image/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM golang 2 | 3 | LABEL "name"="Create Image" 4 | LABEL "version"="1.0" 5 | 6 | COPY marketplace_image.json / 7 | COPY entrypoint.sh / 8 | ENTRYPOINT ["/entrypoint.sh"] 9 | -------------------------------------------------------------------------------- /actions/create-image/action.yml: -------------------------------------------------------------------------------- 1 | name: 'Partner Marketplace-ready Image' 2 | description: 'From a partner custom image creates a OCI image that is ready to be submitted to Marketplace' 3 | inputs: 4 | base-image: 5 | description: 'This is the OCID of the custom image that serves as a baseline.' 6 | required: true 7 | default: 'ocid1.image.oc1.iad.aaaaaaaavxqdkuyamlnrdo3q7qa7q3tsd6vnyrxjy3nmdbpv7fs7um53zh5q' 8 | outputs: 9 | image-ocid: 10 | description: 'The OCID of the output image' 11 | runs: 12 | using: 'docker' 13 | image: 'Dockerfile' 14 | args: 15 | - ${{ inputs.base-image }} 16 | -------------------------------------------------------------------------------- /actions/create-image/entrypoint.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | apt-get update 5 | apt install -y jq unzip 6 | 7 | #Installing Packer 8 | export VER="1.4.4" 9 | wget -q https://releases.hashicorp.com/packer/${VER}/packer_${VER}_linux_amd64.zip 10 | unzip packer_${VER}_linux_amd64.zip -d /usr/bin 11 | 12 | #Set up environment 13 | echo "${TF_VAR_private_key}" > ${GITHUB_WORKSPACE}/oci.pem 14 | export TF_VAR_private_key_path=${GITHUB_WORKSPACE}/oci.pem 15 | 16 | #Replace values with credentials for tenancy where image is being built 17 | echo "Use Packer to create OCI image with cleanup-script" 18 | jq '.builders[].user_ocid |= "'$TF_VAR_user_ocid'" | 19 | .builders[].tenancy_ocid |= "'$TF_VAR_tenancy_ocid'" | 20 | .builders[].fingerprint |= "'$TF_VAR_fingerprint'" | 21 | .builders[].key_file |= "'$TF_VAR_private_key_path'" | 22 | .builders[].compartment_ocid |= "'$TF_VAR_compartment_ocid'" | 23 | .builders[].base_image_ocid |= "'$1'" | 24 | .builders[].image_name |= "'$(echo $GITHUB_REPOSITORY |cut -d'/' -f 2).$(date +"%Y%m%d_%H%M%S")'"' /marketplace_image.json > dummy_image.json 25 | packer build dummy_image.json 26 | cat manifest.json | jq -r .builds[0].artifact_id | cut -d':' -f2 > ${GITHUB_WORKSPACE}/ocid.txt 27 | echo ::set-output name=image-ocid::$(cat manifest.json | jq -r .builds[0].artifact_id | cut -d':' -f2) 28 | -------------------------------------------------------------------------------- /actions/create-image/marketplace_image.json: -------------------------------------------------------------------------------- 1 | { 2 | "variables": {}, 3 | "builders": [{ 4 | "user_ocid": "user_ocid_placeholder", 5 | "tenancy_ocid": "tenancy_ocid_placeholder", 6 | "fingerprint": "fingerprint_placeholder", 7 | "key_file": "key_file_placeholder", 8 | "compartment_ocid": "compartment_ocid_placeholder", 9 | "availability_domain": "IYfK:US-ASHBURN-AD-1", 10 | "base_image_ocid": "ocid1.image.oc1.iad.aaaaaaaavxqdkuyamlnrdo3q7qa7q3tsd6vnyrxjy3nmdbpv7fs7um53zh5q", 11 | "image_name": "image_name_placeholder", 12 | "shape": "VM.Standard2.1", 13 | "ssh_username": "opc", 14 | "region": "us-ashburn-1", 15 | "subnet_ocid": "ocid1.subnet.oc1.iad.aaaaaaaavycjgiklg2gtmvdxb5post7kxtkwqso3ou5ia225ziq7iskarr3q", 16 | "type": "oracle-oci" 17 | }], 18 | "provisioners": [{ 19 | "type": "shell", 20 | "inline": [ 21 | "#!/usr/bin/env bash", 22 | "sudo /usr/libexec/oci-image-cleanup -f" 23 | ] 24 | 25 | }], 26 | "post-processors": [ 27 | [{ 28 | "output": "manifest.json", 29 | "strip_path": true, 30 | "type": "manifest" 31 | }] 32 | ] 33 | } 34 | -------------------------------------------------------------------------------- /actions/terraform-apply/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM golang 2 | 3 | LABEL "name"="Terraform Apply" 4 | LABEL "version"="1.0" 5 | 6 | COPY apply_test.go / 7 | COPY entrypoint.sh / 8 | ENTRYPOINT ["/entrypoint.sh"] 9 | -------------------------------------------------------------------------------- /actions/terraform-apply/apply_test.go: -------------------------------------------------------------------------------- 1 | package test 2 | 3 | import ( 4 | "testing" 5 | "os" 6 | "github.com/gruntwork-io/terratest/modules/terraform" 7 | ) 8 | 9 | func TestTerraform(t *testing.T) { 10 | t.Parallel() 11 | terraformOptions := &terraform.Options{ 12 | TerraformDir: os.Getenv("TF_ACTION_WORKING_DIR"), 13 | Vars: map[string]interface{}{ 14 | "region": "us-ashburn-1", 15 | "ssh_public_key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKNF77nMzrf1+wUGJmPe3ZLDD0/xXe4v3QJT0SAeZzlgOEwJFyc7O2a2Fe4pq+g0JIZkNL/ta2KV5YaT6hmbSZRqpjqdld8B6flm7xt7J2MRMPOAADy4eClJNBklnPzhStGzQmV/o0McxIJZbMPUCDK8R6e4yAMva1AX40Ub4+qX2mu48x7229mmSvKM8rzCGYZcu02RC1w7iGg37TVLKn0c0ds18bXkN8zlHhNBMfbSFJ/dZ8lHtPqwjCbL/UFH832tMrUA8D9BvYlfo6/qe2VvsnMxBS+JDu372NbubNh6Caeo7/I/6n3jL0TuJlOEd+TUX0vc39H6+KHaNm3WrX", 16 | "shape": "VM.Standard2.1", 17 | }, 18 | NoColor: true, 19 | } 20 | 21 | defer terraform.Destroy(t, terraformOptions) 22 | terraform.InitAndApply(t, terraformOptions) 23 | } 24 | -------------------------------------------------------------------------------- /actions/terraform-apply/entrypoint.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | apt-get update 5 | apt install -y build-essential unzip go-dep 6 | 7 | #Installing terraform and upgrading code to Terraform 12 8 | wget -q https://releases.hashicorp.com/terraform/0.12.10/terraform_0.12.10_linux_amd64.zip 9 | unzip terraform_0.12.10_linux_amd64.zip -d /usr/bin 10 | terraform init 11 | 12 | #Set up environment for running terratest in go 13 | mkdir -p $HOME/go/src/terratest/test 14 | export GOROOT=/usr/local/go 15 | export GOPATH=$HOME/go 16 | export PATH=$GOROOT/bin:$GOPATH/bin:/usr/bin:$PATH 17 | mv /apply_test.go $HOME/go/src/terratest/test 18 | cd $HOME/go/src/terratest/test 19 | 20 | #Install terratest and dependencies 21 | cat << EOF > Gopkg.toml 22 | [[constraint]] 23 | name = "github.com/gruntwork-io/terratest" 24 | version = "0.19.1" 25 | EOF 26 | 27 | dep ensure 28 | 29 | #Set up environment to run the terraform code 30 | echo "${TF_VAR_private_key}" > ${GITHUB_WORKSPACE}/oci.pem 31 | export TF_VAR_private_key_path=${GITHUB_WORKSPACE}/oci.pem 32 | export TF_ACTION_WORKING_DIR=${GITHUB_WORKSPACE}/${LISTING_DIR} 33 | 34 | go test -v $HOME/go/src/terratest/test/apply_test.go -timeout 20m 35 | -------------------------------------------------------------------------------- /actions/update-listing/Dockerfile: -------------------------------------------------------------------------------- 1 | 2 | FROM python 3 | 4 | LABEL "name"="Update Listing" 5 | LABEL "version"="1.0" 6 | 7 | COPY entrypoint.sh / 8 | COPY entrypoint.sh / 9 | COPY mpctl.py / 10 | COPY mpapihelper.py / 11 | COPY __init__.py / 12 | ENTRYPOINT ["/entrypoint.sh"] 13 | -------------------------------------------------------------------------------- /actions/update-listing/__init__.py: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/oracle-quickstart/oci-quickstart/5201f6e1309966e6cab6a2f244f86fc3cc58201e/actions/update-listing/__init__.py -------------------------------------------------------------------------------- /actions/update-listing/entrypoint.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | pip install requests 4 | pip install pyyaml 5 | 6 | echo "${API_CREDS}" > ${GITHUB_WORKSPACE}/api_creds.yaml 7 | 8 | export LISTING_ID=$(grep -e listingId "${GITHUB_WORKSPACE}/${LISTING_DIR}/marketplace/metadata.yaml" 2> /dev/null | grep -oe [0-9]*) 9 | export ZIP_FILE=$(ls ${GITHUB_WORKSPACE}/upload 2> /dev/null) 10 | export ZIP_PATH=${GITHUB_WORKSPACE}/upload 11 | export OCID=$(cat ${GITHUB_WORKSPACE}/ocid.txt 2> /dev/null) 12 | 13 | cp "${GITHUB_WORKSPACE}/${LISTING_DIR}/marketplace/metadata.yaml" ${GITHUB_WORKSPACE}/metadata.yaml 14 | cp "${GITHUB_WORKSPACE}/${LISTING_DIR}/marketplace/icon.png" ${GITHUB_WORKSPACE}/icon.png 15 | 16 | pushd ${GITHUB_WORKSPACE} 17 | export COMMIT_HASH=$(git rev-parse HEAD | cut -c1-6) 18 | popd 19 | 20 | if [ -z "$LISTING_ID" ] || [ "$LISTING_ID" = "0" ] 21 | then 22 | if [ -z "$OCID" ] 23 | then 24 | echo "python /mpctl.py -credsFile ${GITHUB_WORKSPACE}/api_creds.yaml -action create_listing -fileName $ZIP_PATH/$ZIP_FILE -commitHash $COMMIT_HASH" 25 | python /mpctl.py -credsFile ${GITHUB_WORKSPACE}/api_creds.yaml -action create_listing -fileName $ZIP_PATH/$ZIP_FILE -commitHash $COMMIT_HASH 26 | else 27 | echo "python /mpctl.py -credsFile ${GITHUB_WORKSPACE}/api_creds.yaml -action create_listing -imageOcid $OCID" 28 | python /mpctl.py -credsFile ${GITHUB_WORKSPACE}/api_creds.yaml -action create_listing -imageOcid $OCID 29 | fi 30 | else 31 | if [ -z "$OCID" ] 32 | then 33 | echo "python /mpctl.py -credsFile ${GITHUB_WORKSPACE}/api_creds.yaml -action update_listing -fileName $ZIP_PATH/$ZIP_FILE -commitHash $COMMIT_HASH" 34 | python /mpctl.py -credsFile ${GITHUB_WORKSPACE}/api_creds.yaml -action update_listing -fileName $ZIP_PATH/$ZIP_FILE -commitHash $COMMIT_HASH 35 | else 36 | echo "python /mpctl.py -credsFile ${GITHUB_WORKSPACE}/api_creds.yaml -action update_listing -imageOcid $OCID" 37 | python /mpctl.py -credsFile ${GITHUB_WORKSPACE}/api_creds.yaml -action update_listing -imageOcid $OCID 38 | fi 39 | fi 40 | -------------------------------------------------------------------------------- /actions/update-listing/mpapihelper.py: -------------------------------------------------------------------------------- 1 | from time import gmtime, strftime 2 | import datetime 3 | import requests 4 | import base64 5 | import yaml 6 | import json 7 | import os.path 8 | import re 9 | import pprint 10 | 11 | api_url = 'https://ocm-apis-cloud.oracle.com/' 12 | picCompartmentOcid = 'ocid1.compartment.oc1..aaaaaaaaxrcshrhpq6exsqibhdzseghk4yjgrwxn3uaev6poaek2ooz4n7eq' 13 | picTenancyId = '59030347' 14 | 15 | 16 | class Config: 17 | class __Config: 18 | kwargs = {} 19 | 20 | def __init__(self, **kwargs): 21 | self.kwargs = kwargs 22 | 23 | def __str__(self): 24 | return repr(self) 25 | 26 | instance = None 27 | 28 | def __init__(self, **kwargs): 29 | if not Config.instance: 30 | Config.instance = Config.__Config(**kwargs) 31 | else: 32 | Config.instance.kwargs = {**Config.instance.kwargs, **kwargs} 33 | 34 | def set(self, key, value): 35 | Config.instance.kwargs[key] = value 36 | 37 | def get(self, key): 38 | if key not in Config.instance.kwargs: return None 39 | return Config.instance.kwargs[key] 40 | 41 | 42 | class Request: 43 | class __Request: 44 | kwargs = {} 45 | 46 | def __init__(self, **kwargs): 47 | self.kwargs = kwargs 48 | 49 | def __str__(self): 50 | return repr(self) 51 | 52 | instance = None 53 | 54 | def __init__(self, **kwargs): 55 | config = Config() 56 | if not Request.instance: 57 | Request.instance = Request.__Request(**kwargs) 58 | Request.set_access_token(self, config.get('creds_file')) 59 | else: 60 | Request.instance.kwargs = {**Request.instance.kwargs, **kwargs} 61 | Request.bind_action_dic(self) 62 | Request.instance.kwargs['api_call'] = Request.instance.kwargs['action_api_uri_dic'][config.get('action')] 63 | Request.instance.kwargs['uri'] = api_url + Request.instance.kwargs['api_call'] 64 | 65 | def bind_action_dic(self): 66 | config = Config() 67 | Request.instance.kwargs['action_api_uri_dic'] = { 68 | 'get_listingVersions': 'appstore/publisher/v1/listings', 69 | 'get_listingVersion': f'appstore/publisher/v1/applications/{config.get("listingVersionId")}', 70 | 'get_artifacts': 'appstore/publisher/v1/artifacts', 71 | 'get_artifact': f'appstore/publisher/v1/artifacts/{config.get("artifactId")}', 72 | 'get_applications': 'appstore/publisher/v1/applications', 73 | 'get_application': f'appstore/publisher/v1/applications/{config.get("listingVersionId")}', 74 | 'get_listing_packages': f'appstore/publisher/v2/applications/{config.get("listingVersionId")}' 75 | f'/packages', 76 | 'get_application_packages': f'appstore/publisher/v2/applications/{config.get("listingVersionId")}' 77 | f'/packages', 78 | 'get_application_package': f'appstore/publisher/v2/applications/{config.get("listingVersionId")}' 79 | f'/packages/{config.get("packageVersionId")}', 80 | 'get_terms': 'appstore/publisher/v1/terms', 81 | 'get_terms_version': f'appstore/publisher/v1/terms/{config.get("termsId")}/version/' 82 | f'{config.get("termsVersionId")}', 83 | 'create_listing': f'appstore/publisher/v1/applications', 84 | 'create_new_version': f'appstore/publisher/v1/applications/{config.get("listingVersionId")}/version', 85 | 'new_package_version': f'appstore/publisher/v2/applications/{config.get("listingVersionId")}' 86 | f'/packages/{config.get("packageVersionId")}/version', 87 | 'upload_icon': f'appstore/publisher/v1/applications/{config.get("listingVersionId")}/icon', 88 | } 89 | 90 | def set_access_token(self, creds_file): 91 | 92 | with open(creds_file, 'r') as stream: 93 | creds = yaml.safe_load(stream) 94 | 95 | auth_string = creds['client_id'] 96 | auth_string += ':' 97 | auth_string += creds['secret_key'] 98 | 99 | encoded = base64.b64encode(auth_string.encode('ascii')) 100 | encoded_string = encoded.decode('ascii') 101 | 102 | token_url = 'https://login.us2.oraclecloud.com:443/oam/oauth2/tokens?grant_type=client_credentials' 103 | 104 | auth_headers = {} 105 | auth_headers['Content-Type'] = 'application/x-www-form-urlencoded' 106 | auth_headers['charset'] = 'UTF-8' 107 | auth_headers['X-USER-IDENTITY-DOMAIN-NAME'] = 'usoracle30650' 108 | auth_headers['Authorization'] = f'Basic {encoded_string}' 109 | 110 | r = requests.post(token_url, headers=auth_headers) 111 | 112 | Request.instance.kwargs['access_token'] = json.loads(r.text).get('access_token') 113 | api_headers = {} 114 | api_headers['charset'] = 'UTF-8' 115 | api_headers['X-Oracle-UserId'] = creds['user_email'] 116 | api_headers['Authorization'] = f'Bearer {Request.instance.kwargs["access_token"]}' 117 | 118 | Request.instance.kwargs['api_headers'] = api_headers 119 | 120 | def get(self, qsp = None): 121 | uri = Request.instance.kwargs['uri'] + (f'?{qsp}' if qsp is not None else '') 122 | config = Config() 123 | if config.get('debug'): 124 | print(f'get: {uri}') 125 | r = requests.get(uri, headers=Request.instance.kwargs['api_headers']) 126 | if r.status_code > 299: 127 | print(r.text) 128 | return json.loads(r.text) 129 | 130 | def post(self, files=None, data=None): 131 | config = Config() 132 | pp = pprint.PrettyPrinter(indent=4) 133 | if config.get('debug'): 134 | print(f'post: {Request.instance.kwargs["uri"]}') 135 | if data is not None: 136 | print('data:') 137 | pp.pprint(data) 138 | if files is not None: 139 | print('files:') 140 | pp.pprint(files) 141 | 142 | # neither files nor data 143 | if files is None and data is None: 144 | r = requests.post(Request.instance.kwargs['uri'], headers=Request.instance.kwargs['api_headers']) 145 | 146 | # data but no files 147 | elif data is not None and files is None: 148 | Request.instance.kwargs['api_headers']['Content-Type'] = 'application/json' 149 | r = requests.post(Request.instance.kwargs['uri'], headers=Request.instance.kwargs['api_headers'], data=data) 150 | del Request.instance.kwargs['api_headers']['Content-Type'] 151 | 152 | # files and data 153 | elif data is not None and files is not None: 154 | r = requests.post(Request.instance.kwargs['uri'], headers=Request.instance.kwargs['api_headers'], 155 | files=files, data=data) 156 | 157 | # only files 158 | else: 159 | r = requests.post(Request.instance.kwargs['uri'], headers=Request.instance.kwargs['api_headers'], 160 | files=files) 161 | 162 | if r.status_code > 299: 163 | print(r.text) 164 | return json.loads(r.text) 165 | 166 | def patch(self, data=None, is_json=False): 167 | config = Config() 168 | pp = pprint.PrettyPrinter(indent=4) 169 | if config.get('debug'): 170 | print(f'patch: {Request.instance.kwargs["uri"]}') 171 | if data is not None: 172 | print('data:') 173 | pp.pprint(data) 174 | if data is None: 175 | r = requests.patch(Request.instance.kwargs['uri'], headers=Request.instance.kwargs['api_headers']) 176 | else: 177 | if is_json: 178 | Request.instance.kwargs['api_headers']['Content-Type'] = 'application/json' 179 | r = requests.patch(Request.instance.kwargs['uri'], headers=Request.instance.kwargs['api_headers'], data=data) 180 | del Request.instance.kwargs['api_headers']['Content-Type'] 181 | 182 | if r.status_code > 299: 183 | print(r.text) 184 | return json.loads(r.text) 185 | 186 | def put(self, data): 187 | config = Config() 188 | pp = pprint.PrettyPrinter(indent=4) 189 | if config.get('debug'): 190 | print(f'put: {Request.instance.kwargs["uri"]}') 191 | if data is not None: 192 | print('data:') 193 | pp.pprint(data) 194 | r = requests.put(Request.instance.kwargs['uri'], headers=Request.instance.kwargs['api_headers'], files=data) 195 | if r.status_code > 299: 196 | print(r.text) 197 | return json.loads(r.text) 198 | 199 | 200 | def sanitize_name(name): 201 | return re.sub('[^a-zA-Z0-9_\.\-\+ ]+', '', name) 202 | 203 | 204 | def find_file(file_name): 205 | # github actions put files in a chroot jail, so we need to look in root 206 | file_name = file_name if os.path.isfile(file_name) \ 207 | else '/{}'.format(file_name) if os.path.isfile('/{}'.format(file_name)) \ 208 | else 'marketplace/{}'.format(file_name) if os.path.isfile('marketplace/{}'.format(file_name)) \ 209 | else '/marketplace/{}'.format(file_name) if os.path.isfile('/marketplace/{}'.format(file_name)) \ 210 | else file_name # return the original file name if not found so the open can throw the exception 211 | return file_name 212 | 213 | 214 | def get_time_stamp(): 215 | return strftime("%Y%m%d%H%M", gmtime()) 216 | 217 | 218 | def do_get_action(qsp = None): 219 | request = Request() 220 | result = request.get(qsp) 221 | return result 222 | 223 | 224 | def get_new_version_id(): 225 | config = Config() 226 | config.set('action','create_new_version') 227 | request = Request() 228 | result = request.post() 229 | return result['entityId'] if 'entityId' in result else result 230 | 231 | 232 | def update_version_metadata(newVersionId): 233 | config = Config() 234 | config.set('action','get_listingVersion') 235 | config.set('listingVersionId', newVersionId) 236 | file_name = find_file('metadata.yaml') 237 | if not os.path.isfile(file_name): 238 | return f'metadata file metadata.yaml not found. skipping metadata update.' 239 | with open(file_name, 'r') as stream: 240 | metadata = yaml.safe_load(stream) 241 | 242 | updateable_items = ['longDescription', 'name', 'shortDescription', 'systemRequirements', 'tagLine', 'tags', 243 | 'usageInformation'] 244 | 245 | for k in list(metadata.keys()): 246 | if k not in updateable_items: 247 | del metadata[k] 248 | 249 | body = json.dumps(metadata) 250 | 251 | request = Request() 252 | result = request.patch(body, True) 253 | if 'message' in result: 254 | return result['message'] 255 | else: 256 | return result.text if 'text' in result else result 257 | 258 | 259 | def get_package_id(new_version_id): 260 | config = Config() 261 | config.set('action','get_application_packages') 262 | config.set('listingVersionId', new_version_id) 263 | r = do_get_action() 264 | return r['items'][0]['Package']['id'] 265 | 266 | 267 | def get_new_package_version_id(new_version_id, package_id): 268 | config = Config() 269 | config.set('action','new_package_version') 270 | config.set('listingVersionId', new_version_id) 271 | config.set('packageVersionId', package_id) 272 | request = Request() 273 | result = request.patch() 274 | 275 | return result['entityId'] if 'entityId' in result else result 276 | 277 | 278 | def update_versioned_package_version(new_package_version_id): 279 | config = Config() 280 | config.set('action','get_application_package') 281 | config.set('packageVersionId', new_package_version_id) 282 | 283 | if config.get('listing_type') == 'stack': 284 | service_type = 'OCIOrchestration' 285 | else: 286 | service_type = 'OCI' 287 | body = {} 288 | body['version'] = sanitize_name(config.get('versionString')) + ' ' + get_time_stamp() 289 | body['serviceType'] = service_type 290 | payload = {'json': (None, json.dumps(body))} 291 | 292 | request = Request() 293 | result = request.put(payload) 294 | if 'message' in result: 295 | return result['message'] 296 | else: 297 | return result.text if 'text' in result else result 298 | 299 | 300 | def set_package_version_as_default(new_version_id, new_package_version_id): 301 | config = Config() 302 | config.set('action','get_application_package') 303 | config.set('listingVersionId', new_version_id) 304 | config.set('packageVersionId', new_package_version_id) 305 | body = {} 306 | body['action'] = 'default' 307 | request = Request() 308 | result = request.patch(json.dumps(body), True) 309 | if 'message' in result: 310 | return result['message'] 311 | else: 312 | return result.text if 'text' in result else result 313 | 314 | 315 | def create_new_stack_artifact(file_name): 316 | config = Config() 317 | config.set('action','get_artifacts') 318 | body = {} 319 | body['name'] = config.get('commitHash') if config.get('commitHash') is not None \ 320 | else sanitize_name(config.get('versionString')) + ' ' + get_time_stamp() 321 | body['artifactType'] = 'TERRAFORM_TEMPLATE' 322 | payload = {'json': (None, json.dumps(body))} 323 | index = file_name.rfind('/') 324 | name = file_name[index + 1:] 325 | files = {'file': (name, open(file_name, 'rb'))} 326 | request = Request() 327 | result = request.post(files, payload) 328 | return result['entityId'] if 'entityId' in result else result 329 | 330 | 331 | def create_new_image_artifact(old_listing_artifact_version): 332 | config = Config() 333 | config.set('action','get_artifacts') 334 | if old_listing_artifact_version is not None: 335 | new_version = {key: old_listing_artifact_version[key] for key in 336 | ['name', 'artifactType', 'source', 'artifactProperties']} 337 | new_version['name'] = sanitize_name(config.get('versionString')) + ' ' + get_time_stamp() 338 | new_version['source']['uniqueIdentifier'] = config.get('imageOcid') 339 | new_version['artifactType'] = 'OCI_COMPUTE_IMAGE' 340 | else: 341 | new_version = {} 342 | new_version['name'] = sanitize_name(config.get('versionString')) + ' ' + get_time_stamp() 343 | new_version['artifactType'] = 'OCI_COMPUTE_IMAGE' 344 | new_version['source'] = {} 345 | new_version['source']['regionCode'] = 'us-ashburn-1' 346 | new_version['source']['uniqueIdentifier'] = config.get('imageOcid') 347 | new_version['artifactProperties'] = [{}, {}] 348 | new_version['artifactProperties'][0]['artifactTypePropertyName'] = 'compartmentOCID' 349 | new_version['artifactProperties'][0]['value'] = picCompartmentOcid 350 | new_version['artifactProperties'][1]['artifactTypePropertyName'] = 'ociTenancyID' 351 | new_version['artifactProperties'][1]['value'] = picTenancyId 352 | request = Request() 353 | data = json.dumps(new_version) 354 | result = request.post(None, data) 355 | return result['entityId'] if 'entityId' in result else result 356 | 357 | 358 | def associate_artifact_with_package(artifact_id, new_package_version_id): 359 | config = Config() 360 | body = {} 361 | body['resources'] = [{}] 362 | body['resources'][0]['serviceType'] = 'OCIOrchestration' if config.get('listing_type') == 'stack' else 'OCI' 363 | body['resources'][0]['type'] = 'terraform' if config.get('listing_type') == 'stack' else 'ocimachineimage' 364 | body['resources'][0]['properties'] = [{}] 365 | body['resources'][0]['properties'][0]['name'] = 'artifact' 366 | body['resources'][0]['properties'][0]['value'] = artifact_id 367 | body['resources'][0]['properties'][0]['valueProperties'] = [{}] 368 | body['resources'][0]['properties'][0]['valueProperties'][0]['name'] = 'name' 369 | body['resources'][0]['properties'][0]['valueProperties'][0]['value'] = sanitize_name(config.get('versionString')) 370 | 371 | payload = {'json': (None, json.dumps(body))} 372 | config.set('action','get_application_package') 373 | config.set('packageVersionId', new_package_version_id) 374 | request = Request() 375 | result = request.put(payload) 376 | if 'message' in result: 377 | return result['message'] 378 | else: 379 | return result.text if 'text' in result else result 380 | 381 | 382 | def submit_listing(): 383 | config = Config() 384 | auto_approve = 'true' 385 | while True: 386 | config.set('action','get_listingVersion') 387 | body = {} 388 | body['action'] = 'submit' 389 | body['note'] = 'submitting new version' 390 | body['autoApprove'] = auto_approve 391 | data = json.dumps(body) 392 | request = Request() 393 | result = request.patch(data, True) 394 | if 'message' in result: 395 | return result['message'] 396 | if auto_approve == 'false': 397 | return 'this partner has not yet been approved for auto approval. please contact MP admin.' 398 | else: 399 | auto_approve = 'false' 400 | 401 | 402 | def publish_listing(): 403 | config = Config() 404 | config.set('action','get_listingVersion') 405 | body = {} 406 | body['action'] = 'publish' 407 | data = json.dumps(body) 408 | request = Request() 409 | result = request.patch(data, True) 410 | if 'message' in result: 411 | return result['message'] 412 | else: 413 | return 'Failed to auto-publish, please contact MP admin to maunaully approve listing.' 414 | 415 | 416 | def create_new_listing(): 417 | config = Config() 418 | config.set('action','get_applications') 419 | file_name = find_file('metadata.yaml') 420 | with open(file_name, 'r') as stream: 421 | new_listing_metadata = yaml.safe_load(stream) 422 | del new_listing_metadata['listingId'] 423 | if 'versionDetails' in new_listing_metadata: 424 | vd = new_listing_metadata['versionDetails'] 425 | config.set('versionString', vd['versionNumber']) 426 | new_listing_metadata['versionDetails']['releaseDate'] = datetime.datetime.now().strftime( 427 | "%Y-%m-%dT%H:%M:%S.000Z") 428 | body = json.dumps(new_listing_metadata) 429 | request = Request() 430 | result = request.post(None, body) 431 | return result['entityId'] if 'entityId' in result else result 432 | 433 | 434 | def create_new_package(artifact_id): 435 | config = Config() 436 | body = {} 437 | body['version'] = sanitize_name(config.get('versionString')) 438 | body['description'] = config.get('versionString') 439 | body['serviceType'] = 'OCIOrchestration' 440 | body['resources'] = [{}] 441 | body['resources'][0]['serviceType'] = 'OCIOrchestration' 442 | body['resources'][0]['type'] = 'terraform' 443 | body['resources'][0]['properties'] = [{}] 444 | body['resources'][0]['properties'][0]['name'] = 'artifact' 445 | body['resources'][0]['properties'][0]['value'] = artifact_id 446 | 447 | config.set('action','get_application_packages') 448 | payload = {'json': (None, json.dumps(body))} 449 | request = Request() 450 | result = request.post(payload) 451 | if 'message' in result: 452 | return result['message'] 453 | else: 454 | return result.text if 'text' in result else result 455 | 456 | 457 | def upload_icon(): 458 | config = Config() 459 | config.set('action','upload_icon') 460 | file_name = find_file('icon.png') 461 | files = {'image': open(file_name, 'rb')} 462 | request = Request() 463 | result = request.post(files) 464 | return result['entityId'] if 'entityId' in result else result 465 | 466 | def validate_package(listing_version_id, package_version_id): 467 | config = Config() 468 | config.set('action', 'get_application_package') 469 | config.set('listingVersionId', listing_version_id) 470 | config.set('packageVersionId', package_version_id) 471 | body = {} 472 | body['action'] = 'validate' 473 | data = json.dumps(body) 474 | request = Request() 475 | result = request.patch(data, True) 476 | return result['entityId'] if 'entityId' in result else result 477 | -------------------------------------------------------------------------------- /actions/update-listing/mpctl.py: -------------------------------------------------------------------------------- 1 | import sys 2 | import argparse 3 | import json 4 | import os.path 5 | from time import sleep 6 | from mpapihelper import * 7 | 8 | ####################################################################################################################### 9 | # 10 | # usage: 11 | # 12 | # update listing with new terraform template 13 | # python3 mpctl.py -credsFile -action update_listing -fileName 14 | # 15 | # update listing with new image 16 | # python3 mpctl.py -credsFile -action update_listing -imageOcid 17 | # 18 | # create new terraform listing 19 | # python3 mpctl.py -credsFile -action create_listing -fileName 20 | # 21 | # create new image listing 22 | # python3 mpctl.py -credsFile -action create_listing -imageOcid 23 | # 24 | # get one listing 25 | # python3 mpctl.py -credsFile -action get_listingVersion -listingVersionId 26 | # 27 | # 28 | # get the published version of one listing 29 | # python3 mpctl.py -credsFile -action get_listingVersion -listingId 30 | # 31 | # get all listings 32 | # python3 mpctl.py -credsFile -action get_listingVersions 33 | # 34 | # build one listing tree 35 | # python3 mpctl.py -credsFile -action build_listings -listingVersionId 36 | # 37 | # 38 | # build one listing tree of the published version 39 | # python3 mpctl.py -credsFile -action build_listings -listingId 40 | # 41 | # build all listings tree for partner 42 | # python3 mpctl.py -credsFile -action build_listings [-includeUnpublished] 43 | # 44 | # dump metadata file for a listing 45 | # python3 mpctl.py -credsFile -action dump_metadata -listingVersionId 46 | # 47 | # dump metadata file for a listing's published version 48 | # python3 mpctl.py -credsFile -action dump_metadata -listingId 49 | ####################################################################################################################### 50 | 51 | 52 | class ListingMetadata: 53 | git_metadata = {} 54 | api_metadata = {} 55 | 56 | def __init__(self, file_name, lv): 57 | 58 | self.git_metadata = {} 59 | self.api_metadata = {} 60 | 61 | if file_name is not None and os.path.isfile(file_name): 62 | with open(file_name, 'r') as stream: 63 | self.git_metadata = yaml.safe_load(stream) 64 | 65 | if lv is not None: 66 | lvd = lv.listing_version_details 67 | self.api_metadata['versionDetails'] = {} 68 | self.api_metadata['listingId'] = args.listingId 69 | self.api_metadata['versionDetails']['versionNumber'] = lvd['versionDetails']['versionNumber'] \ 70 | if 'versionDetails' in lvd and 'versionNumber' in lvd['versionDetails'] else '' 71 | self.api_metadata['name'] = lvd['name'] if 'name' in lvd else '' 72 | self.api_metadata['shortDescription'] = lvd['shortDescription'] if 'shortDescription' in lvd else '' 73 | self.api_metadata['longDescription'] = lvd['longDescription'] if 'longDescription' in lvd else '' 74 | self.api_metadata['usageInformation'] = lvd['usageInformation'] if 'usageInformation' in lvd else '' 75 | self.api_metadata['tags'] = lvd['tags'] if 'tags' in lvd else '' 76 | self.api_metadata['tagLine'] = lvd['tagLine'] if 'tagLine' in lvd else '' 77 | self.api_metadata['systemRequirements'] = lvd['systemRequirements'] if 'systemRequirements' in lvd else '' 78 | 79 | def write_metadata(self, file_name): 80 | 81 | if file_name is None: 82 | file_name = f'metadata.yaml' 83 | 84 | with open(file_name, 'w+') as stream: 85 | yaml.safe_dump(self.api_metadata, stream) 86 | 87 | 88 | class ArtifactVersion: 89 | details = [] 90 | 91 | def __init__(self, details): 92 | self.details = details 93 | 94 | def __str__(self): 95 | ppstring = '' 96 | ppstring += '\n' 97 | ppstring += json.dumps(self.details, indent=4, sort_keys=False) 98 | return ppstring 99 | 100 | 101 | class Artifact: 102 | versions = [] 103 | resource = [] 104 | 105 | def __init__(self, resource): 106 | config = Config() 107 | self.resource = resource 108 | self.versions = [] 109 | for resource_property in resource['properties']: 110 | config.set('action','get_artifact') 111 | config.set('artifactId', int(resource_property['value'])) 112 | r = do_get_action() 113 | av = ArtifactVersion(r) 114 | self.versions.append(av) 115 | 116 | def __str__(self): 117 | ppstring = '' 118 | ppstring += '\n' 119 | ppstring += json.dumps(self.resource, indent=4, sort_keys=False) 120 | for version in self.versions: 121 | ppstring += '\n' 122 | ppstring += str(version) 123 | return ppstring 124 | 125 | 126 | class Package: 127 | package = [] 128 | artifacts = [] 129 | 130 | def __init__(self, package): 131 | self.artifacts = [] 132 | self.package = package['Package'] 133 | for resource in self.package['resources']: 134 | a = Artifact(resource) 135 | self.artifacts.append(a) 136 | 137 | def __str__(self): 138 | ppstring = '' 139 | ppstring += '\n' 140 | ppstring += json.dumps(self.package, indent=4, sort_keys=False) 141 | for artifact in self.artifacts: 142 | ppstring += '\n' 143 | ppstring += str(artifact) 144 | return ppstring 145 | 146 | 147 | class ListingVersion: 148 | package_versions = [] 149 | listing_version = '' 150 | listing_version_details = '' 151 | packages = [] 152 | listing_metadata = {} 153 | 154 | def __init__(self, listing_version): 155 | self.packages = [] 156 | self.listing_version = listing_version 157 | 158 | if 'packageVersions' in self.listing_version: 159 | self.package_versions = self.listing_version['packageVersions'] 160 | 161 | config = Config() 162 | config.set('action','get_listingVersion') 163 | config.set('listingVersionId', self.listing_version['listingVersionId']) 164 | self.listing_version_details = do_get_action() 165 | self.listing_metadata = ListingMetadata(find_file('metadata.yaml'), self) 166 | config.set('action','get_application_packages') 167 | packages = do_get_action() 168 | 169 | for package in packages['items']: 170 | if not args.includeUnpublished and package['Package']['status']['code'] == 'unpublished': 171 | continue 172 | p = Package(package) 173 | self.packages.append(p) 174 | 175 | def __str__(self): 176 | ppstring = '' 177 | ppstring += '\n' 178 | ppstring += json.dumps(self.listing_version, indent=4, sort_keys=False) 179 | ppstring += '\n' 180 | ppstring += json.dumps(self.listing_version_details, indent=4, sort_keys=False) 181 | ppstring += '\n' 182 | ppstring += json.dumps(self.package_versions, indent=4, sort_keys=False) 183 | for package in self.packages: 184 | ppstring += str(package) 185 | pass 186 | return ppstring 187 | 188 | 189 | class Listing: 190 | listing_versions = [] 191 | listingId = 0 192 | 193 | def __init__(self, listing_version): 194 | self.listingId = listing_version['listingId'] 195 | self.listing_versions = [ListingVersion(listing_version)] 196 | 197 | def __str__(self): 198 | ppstring = '' 199 | for listingVersion in self.listing_versions: 200 | ppstring += '\n' 201 | ppstring += str(listingVersion) 202 | return ppstring 203 | 204 | 205 | class TermVersion(): 206 | term_version = [] 207 | 208 | def __init__(self, terms_id, term_version): 209 | config = Config() 210 | config.set('action','get_terms_version') 211 | config.set('termsId', terms_id) 212 | config.set('termsVersionId', term_version['termsVersionId']) 213 | tv = do_get_action() 214 | self.term_version = tv 215 | 216 | def __str__(self): 217 | return json.dumps(self.term_version, indent=4, sort_keys=False) 218 | 219 | 220 | class Terms(): 221 | terms = [] 222 | termVersions = [] 223 | 224 | def __init__(self, terms): 225 | self.terms = terms 226 | self.termVersions = [] 227 | 228 | for termVersion in terms['termVersions']: 229 | tv = TermVersion(terms['termsId'], termVersion) 230 | self.termVersions.append(tv) 231 | 232 | def __str__(self): 233 | ppstring = '' 234 | ppstring += str(self.terms) 235 | for termVersion in self.termVersions: 236 | ppstring += str(termVersion) 237 | return ppstring 238 | 239 | 240 | class Partner: 241 | listings = [] 242 | terms = [] 243 | 244 | def __init__(self): 245 | config = Config() 246 | 247 | if args.all or config.get('listingVersionId') is None or config.get('listingVersionId') == '0': 248 | config.set('action','get_listingVersions') 249 | else: 250 | config.set('action','get_listingVersion') 251 | listing_versions = do_get_action() 252 | 253 | if 'items' in listing_versions: 254 | for item in listing_versions['items']: 255 | if not args.includeUnpublished and item['GenericListing']['status']['code'] == 'UNPUBLISHED': 256 | continue 257 | found = False 258 | for listing in self.listings: 259 | if listing.listingId == item['GenericListing']['listingId']: 260 | listing.listing_versions.append(ListingVersion(item['GenericListing'])) 261 | found = True 262 | break 263 | 264 | if not found: 265 | listing = Listing(item['GenericListing']) 266 | self.listings.append(listing) 267 | 268 | else: 269 | listing = Listing(listing_versions) 270 | self.listings.append(listing) 271 | 272 | config.set('action','get_terms') 273 | terms = do_get_action() 274 | if 'items' in terms: 275 | for item in terms['items']: 276 | t = Terms(item['terms']) 277 | self.terms.append(t) 278 | else: 279 | t = Terms(terms) 280 | self.terms.append(t) 281 | 282 | def __str__(self): 283 | ppstring = '' 284 | ppstring += (f'The parnter has {len(self.listings)} listing(s)') 285 | ppstring += '\n' 286 | for listing in self.listings: 287 | ppstring += str(listing) 288 | ppstring += '\n' 289 | ppstring += '\n' 290 | for terms in self.terms: 291 | ppstring += str(terms) 292 | return ppstring 293 | 294 | 295 | def do_create(): 296 | config = Config() 297 | config.set('listingVersionId', create_new_listing()) 298 | 299 | upload_icon_message = upload_icon() 300 | 301 | # TODO: refactor common code for image artifact status polling 302 | if config.get('listing_type') == 'stack': 303 | # create new artifact for stack listing 304 | artifact_id = create_new_stack_artifact(args.fileName) 305 | else: 306 | # create new artifact for iamge listing 307 | done = False 308 | retry_count_remaining = 6 309 | while not done and retry_count_remaining > 0: 310 | artifactId = create_new_image_artifact(None) 311 | sleep(10) # give api a moment 312 | config.set('action', 'get_artifact') 313 | config.set('artifactId', artifactId) 314 | artifact_status = do_get_action()['status'] 315 | if artifact_status == 'Available': 316 | done = True 317 | else: 318 | print(f'artifact {artifactId} in status {artifact_status}, sleeping for 10 minutes.') 319 | sleep(600) 320 | retry_count_remaining = retry_count_remaining - 1 321 | if retry_count_remaining <= 0: 322 | raise Exception( 323 | f'artifact {artifactId} in status {artifact_status} after one hour. Please contact MP admin') 324 | 325 | newPackageId = create_new_package(artifact_id) 326 | 327 | # submit the new version of the listing for approval 328 | message = submit_listing() 329 | 330 | # TODO: possible also publish new listings 331 | # message = publish_listing() 332 | 333 | return message 334 | 335 | 336 | def do_update_listing(): 337 | partner = Partner() 338 | 339 | if config.get('listing_type') == 'image': 340 | old_listing_artifact_version = \ 341 | partner.listings[0].listing_versions[0].packages[0].artifacts[0].versions[0].details 342 | 343 | if config.get('debug'): 344 | print('getting new artifact id') 345 | 346 | if config.get('listing_type') == 'stack': 347 | # create new artifact for stack listing 348 | 349 | artifactId = create_new_stack_artifact(args.fileName) 350 | 351 | if config.get('debug'): 352 | print(f'got artifact id {artifactId}') 353 | else: 354 | # create new artifact for image listing 355 | done = False 356 | retry_count_remaining = 6 357 | while not done and retry_count_remaining > 0: 358 | artifactId = create_new_image_artifact(old_listing_artifact_version) 359 | sleep(10) # give api a moment 360 | config.set('action', 'get_artifact') 361 | config.set('artifactId', artifactId) 362 | artifact_status = do_get_action()['status'] 363 | if artifact_status == 'Available': 364 | done = True 365 | else: 366 | print(f'artifact {artifactId} in status {artifact_status}, sleeping for 10 minutes.') 367 | sleep(600) 368 | retry_count_remaining = retry_count_remaining - 1 369 | if retry_count_remaining <= 0: 370 | raise Exception(f'artifact {artifactId} in status {artifact_status} after one hour. Please contact MP admin') 371 | if config.get('debug'): 372 | print(f'got artifact id {artifactId}') 373 | 374 | # create a new version for the application listing 375 | if config.get('debug'): 376 | print('getting new listing version id') 377 | new_version_id = get_new_version_id() 378 | if config.get('debug'): 379 | print(f'got new version id {new_version_id}') 380 | 381 | if config.get('debug'): 382 | print('update version metadata') 383 | updated_metadata_message = update_version_metadata(new_version_id) 384 | if config.get('debug'): 385 | print(f'update result: {updated_metadata_message}') 386 | 387 | # get the package version id needed for package version creation 388 | if config.get('debug'): 389 | print('getting existing package id') 390 | package_id = get_package_id(new_version_id) 391 | if config.get('debug'): 392 | print(f'got package id {package_id}') 393 | 394 | # create a package version from existing package 395 | if config.get('debug'): 396 | print('getting new package version id') 397 | new_package_version_id = get_new_package_version_id(new_version_id, package_id) 398 | if config.get('debug'): 399 | print(f'got new package version id {new_package_version_id}') 400 | 401 | # update versioned package details 402 | if config.get('debug'): 403 | print('update versioned package details') 404 | message = update_versioned_package_version(new_package_version_id) 405 | if config.get('debug'): 406 | print(f'response: {message}') 407 | 408 | # update versioned package details - associate newly created artifact 409 | if config.get('debug'): 410 | print('update versioned package details - associate newly created artifact') 411 | message = associate_artifact_with_package(artifactId, new_package_version_id) 412 | if config.get('debug'): 413 | print(f'response: {message}') 414 | 415 | # set new package version as default 416 | if config.get('debug'): 417 | print('set new package version as default') 418 | message = set_package_version_as_default(new_version_id, new_package_version_id) 419 | if config.get('debug'): 420 | print(f'response: {message}') 421 | 422 | # submit the new version of the listing for approval 423 | if config.get('debug'): 424 | print('submit the new version of the listing for approval') 425 | message = submit_listing() 426 | if config.get('debug'): 427 | print(f'response: {message}') 428 | 429 | # attempt to publish the listing (succeeds if partner is whitelisted) 430 | if config.get('debug'): 431 | print('attempt to publish the listing (succeeds if partner is whitelisted)') 432 | message = publish_listing() 433 | if config.get('debug'): 434 | print(f'response: {message}') 435 | 436 | return message 437 | 438 | 439 | def lookup_listing_version_id_from_listing_id(listing_id): 440 | config = Config() 441 | config.set('action','get_listingVersions') 442 | listing_versions = do_get_action('status=PUBLISHED') 443 | for item in listing_versions['items']: 444 | if item['GenericListing']['listingId'] == listing_id and item['GenericListing']['status']['code'] == 'PUBLISHED': 445 | return item['GenericListing']['listingVersionId'] 446 | return '0' 447 | 448 | 449 | def find_listing_version_id(): #TODO: this shouldn't have the side effect of setting the version string 450 | 451 | listing_version_id = None 452 | metadata_file_name = find_file('metadata.yaml') 453 | config = Config() 454 | if not os.path.isfile(metadata_file_name): 455 | config.set('versionString', 'No Version') 456 | if args.listingVersionId is None: 457 | listing_version_id = 0 458 | if args.listingId is not None: 459 | listing_version_id = lookup_listing_version_id_from_listing_id(args.listingId) 460 | else: 461 | with open(metadata_file_name, 'r') as stream: 462 | metadata = yaml.safe_load(stream) 463 | config.set('versionString', metadata['versionDetails']['versionNumber']) 464 | if args.listingVersionId is None: 465 | if args.listingId is None: 466 | listing_version_id = lookup_listing_version_id_from_listing_id(metadata['listingId']) 467 | else: 468 | listing_version_id = lookup_listing_version_id_from_listing_id(args.listingId) 469 | return listing_version_id 470 | 471 | 472 | if __name__ == '__main__': 473 | 474 | usage_text = '''usage: 475 | 476 | update listing with new terraform template 477 | python3 mpctl.py -credsFile -action update_listing -fileName 478 | 479 | update listing with new image 480 | python3 mpctl.py -credsFile -action update_listing -imageOcid 481 | 482 | create new terraform listing 483 | python3 mpctl.py -credsFile -action create_listing -fileName 484 | 485 | create new image listing 486 | python3 mpctl.py -credsFile -action create_listing -imageOcid 487 | 488 | get one listing 489 | python3 mpctl.py -credsFile -action get_listingVersion -listingVersionId 490 | 491 | get the published version of one listing 492 | python3 mpctl.py -credsFile -action get_listingVersion -listingId 493 | 494 | get all listings 495 | python3 mpctl.py -credsFile -action get_listingVersions 496 | 497 | build one listing tree 498 | python3 mpctl.py -credsFile -action build_listings -listingVersionId 499 | 500 | build one listing tree of the published version 501 | python3 mpctl.py -credsFile -action build_listings -listingId 502 | 503 | build all listings tree for partner 504 | python3 mpctl.py -credsFile -action build_listings [-includeUnpublished] 505 | 506 | dump metadata file for a listing 507 | python3 mpctl.py -credsFile -action dump_metadata -listingVersionId 508 | 509 | dump metadata file for a listing's published version 510 | python3 mpctl.py -credsFile -action dump_metadata -listingId 511 | 512 | ''' 513 | 514 | parser = argparse.ArgumentParser(prog='mpctl', 515 | description='publisher api tool', 516 | epilog=usage_text, 517 | formatter_class=argparse.RawDescriptionHelpFormatter) 518 | parser.add_argument('-action', 519 | help='the action to perform') 520 | parser.add_argument('-includeUnpublished', action='store_true', 521 | help='include unpublished versions when building tree') 522 | parser.add_argument('-listingId', type=int, 523 | help='the listing to act on') 524 | parser.add_argument('-listingVersionId', type=int, 525 | help='the listing version to act on [optional]. Can be inferred from listingId') 526 | parser.add_argument('-packageVersionId', type=int, 527 | help='the package version to act on') 528 | parser.add_argument('-termsId', type=int, 529 | help='the terms to act on') 530 | parser.add_argument('-termsVersionId', type=int, 531 | help='the terms version to act on') 532 | parser.add_argument('-artifactId', type=int, 533 | help='the artifact to act on') 534 | parser.add_argument('-fileName', 535 | help='the name of the TF file') 536 | parser.add_argument('-imageOcid', 537 | help='the ocid of the update image') 538 | parser.add_argument('-credsFile', 539 | help='the path to the creds file') 540 | parser.add_argument('-all', action='store_true', 541 | help='get all listings even if listing id is known') 542 | parser.add_argument('-commitHash', 543 | help='a string to append to package version') 544 | parser.add_argument('-debug', action='store_true', 545 | help='print api call payloads') 546 | 547 | if len(sys.argv) == 1: 548 | parser.print_help(sys.stderr) 549 | sys.exit(1) 550 | 551 | args = parser.parse_args() 552 | 553 | config = Config(creds_file = args.credsFile) 554 | if args.debug: 555 | config.set('debug', True) 556 | if args.artifactId is not None: 557 | config.set('artifactId', args.artifactId) 558 | if args.action is not None: 559 | config.set('action', args.action) 560 | if args.listingVersionId is not None: 561 | config.set('listingVersionId', args.listingVersionId) 562 | if args.packageVersionId is not None: 563 | config.set('packageVersionId', args.packageVersionId) 564 | if args.termsId is not None: 565 | config.set('termsId', args.termsId) 566 | if args.termsVersionId is not None: 567 | config.set('termsVersionId', args.termsVersionId) 568 | if args.imageOcid is not None: 569 | config.set('imageOcid', args.imageOcid) 570 | config.set('listing_type', 'image') 571 | else: 572 | config.set('listing_type', 'stack') 573 | if args.commitHash is not None: 574 | config.set('commitHash', args.commitHash) 575 | 576 | if args.listingVersionId is None: 577 | config.set('listingVersionId', find_listing_version_id()) 578 | else: 579 | config.set('listingVersionId', args.listingVersionId) 580 | 581 | if 'get' in args.action: 582 | r_json = do_get_action() 583 | print(json.dumps(r_json, indent=4, sort_keys=True)) 584 | 585 | if 'create' in args.action: 586 | print(do_create()) 587 | 588 | if 'build' in args.action: 589 | partner = Partner() 590 | print(partner) 591 | 592 | if 'update_listing' in args.action: 593 | print(do_update_listing()) 594 | 595 | if 'dump_metadata' in args.action: 596 | partner = Partner() 597 | lmd = ListingMetadata(None, partner.listings[0].listing_versions[0]) 598 | lmd.write_metadata(f'metadata.yaml') 599 | -------------------------------------------------------------------------------- /actions/update-stack-vars/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM rackspacedot/python37:latest 2 | 3 | LABEL "name"="update stack listing vars" 4 | LABEL "version"="1.0" 5 | 6 | COPY entrypoint.sh /entrypoint.sh 7 | ENTRYPOINT ["/entrypoint.sh"] 8 | -------------------------------------------------------------------------------- /actions/update-stack-vars/entrypoint.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | export OCID=$(cat ${GITHUB_WORKSPACE}/ocid.txt 2> /dev/null) 5 | 6 | sed -i 's/ocid1\.image\.oc1\.[a-z]*\.[a-z0-9]*/'"$OCID"'/' "${GITHUB_WORKSPACE}/${STACK_VARS_FILE}" 7 | 8 | git config --local user.name "Automated Publisher" 9 | git config --local user.email "actions@users.noreply.github.com" 10 | git add "${GITHUB_WORKSPACE}/${STACK_VARS_FILE}" 11 | git commit -m "automated publish from update in ${GITHUB_WORKSPACE}/${LISTING_DIR}" 12 | git push "https://cmp-deploy:${PAT}@github.com/${GITHUB_REPOSITORY}.git" HEAD:${BRANCH} 13 | -------------------------------------------------------------------------------- /scripts/README.md: -------------------------------------------------------------------------------- 1 | # scripts 2 | This directory contains utility scripts intended to be run on their own. 3 | 4 | ## `mkpl_setup.sh` 5 | 6 | Used to setup the compartment and policy in a new publisher's tenancy in cloud-shell 7 | in the OCI console. **Note: these commands must be run in the home region of your tenancy.** 8 | 9 | Can be run with: 10 | 11 | ``` 12 | curl -s https://raw.githubusercontent.com/oracle-quickstart/oci-quickstart/master/scripts/mkpl_setup.sh | bash 13 | ``` 14 | 15 | Running the script will create in the root compartment: 16 | - a compartment called `marketplace` 17 | 18 | - a policy called `marketplace` 19 | 20 | - id values are printed at the end to be used in tenancy setup in the partner portal 21 | 22 | ### Options/Info: 23 | 24 | - If you add `export TESTING='true';` before the curl it will create resources with 25 | random names then delete as a test. 26 | 27 | - Run `unset ` to remove either override. 28 | 29 | - Command may be run twice with no effect. If the policy exists it is deleted and recreated under the same name. A `409 ResourceExists` error returned which can be ignored. 30 | 31 | - Note, the command may be run locally if desired, provided the `oci` CLI is installed 32 | and configured. 33 | -------------------------------------------------------------------------------- /scripts/mkpl_setup.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | # To test run: export TESTING="true" 4 | # To clear: unset TESTING 5 | 6 | # Colors 7 | RED='\033[0;31m' 8 | GREEN='\033[0;32m' 9 | BLUE='\033[0;34m' 10 | CYAN='\033[0;36m' 11 | NC='\033[0m' 12 | 13 | # Get tenancy id via CLI 14 | tenancy_id=$(oci iam availability-domain list | jq -r ' .data[0]."compartment-id"') 15 | # Alternatives that work now, but may not in the future 16 | # tenancy_id=$(grep tenancy $OCI_CLI_CONFIG_FILE | head -n 1 | cut -d'=' -f2) 17 | # oci iam compartment list | jq -cr '[.data[] | select(."compartment-id" | contains("tenancy"))] | .[0]."compartment-id"' 18 | 19 | # Alternative without jq is tenancy_id=$(oci iam availability-domain list --query 'data[0]."compartment-id"' --raw-output) 20 | 21 | if [ -z "$tenancy_id" ] 22 | then 23 | echo "Error: Unable to discover tenancy ocid, exiting" 24 | exit 1 25 | else 26 | echo -e "${GREEN}SUCCESS: teancy ocid discovered: $tenancy_id${NC}" 27 | fi 28 | 29 | 30 | comp_name="marketplace" 31 | policy_name="marketplace" 32 | 33 | # change to new policy 34 | policy='[ 35 | "ALLOW SERVICE marketplace to manage App-catalog-publisher-listing IN TENANCY", 36 | "ALLOW SERVICE marketplace to read tenant IN TENANCY", 37 | "ALLOW SERVICE marketplace to read compartments IN TENANCY", 38 | "ALLOW SERVICE marketplace to read instance-images IN TENANCY", 39 | "ALLOW SERVICE marketplace to inspect instances IN TENANCY", 40 | "ALLOW SERVICE marketplace to read orm-stacks IN TENANCY", 41 | "ALLOW SERVICE marketplace to read orm-jobs IN TENANCY" 42 | ]' 43 | 44 | echo $policy > tmp_mkpl_policy.json 45 | 46 | # testing override 47 | if [ -n "$TESTING" ] 48 | then 49 | rand=$RANDOM 50 | comp_name="clitest$rand" 51 | policy_name="clitest$rand" 52 | echo -e "${CYAN}INFO: in test mode.${NC}" 53 | fi 54 | 55 | echo -e "${CYAN}INFO: will create compartment and policy named $comp_name and $policy_name${NC}" 56 | 57 | # Check if policy exists 58 | echo -e "${CYAN}INFO: Checking for compartment...${NC}" 59 | comp_json=$(oci iam compartment list \ 60 | --compartment-id $tenancy_id \ 61 | --name $comp_name) 62 | 63 | if [ -z "$comp_json" ] 64 | then 65 | # Create compartment under root compartment 66 | echo -e "${CYAN}INFO: Comp does not exist, creating...${NC}" 67 | comp_json="$(oci iam compartment create \ 68 | --compartment-id $tenancy_id \ 69 | --description "To contain custom images + stacks read by the marketplace service" \ 70 | --name $comp_name)" 71 | comp_return=$? 72 | comp_id=$(echo $comp_json | jq -r '.data.id') 73 | echo $comp_json | jq -M . 74 | else 75 | echo -e "${CYAN}INFO: Compartment exists.${NC}" 76 | comp_id=$(echo $comp_json | jq -r '.data[0].id') 77 | fi 78 | 79 | if [[ $comp_return -eq 0 ]] 80 | then 81 | echo -e "${GREEN}SUCCESS: compartment $comp_name created/exists.${NC}" 82 | else 83 | echo -e "${RED}ERROR: compartment not created.${NC}" 84 | fi 85 | 86 | # Check if policy exists 87 | echo -e "${CYAN}INFO: Checking for policy...${NC}" 88 | policy_json=$(oci iam policy list \ 89 | --compartment-id $tenancy_id \ 90 | --name $policy_name) 91 | 92 | if [ -z "$policy_json" ] 93 | then 94 | echo -e "${CYAN}INFO: Policy does not exist, continuing...${NC}" 95 | else 96 | policy_id=$(echo $policy_json | jq -r '.data[0].id') 97 | echo -e "${CYAN}INFO: Policy exists, deleting $policy_id ${NC}" 98 | # no subshell 99 | oci iam policy delete --force --policy-id $policy_id 100 | fi 101 | 102 | # Create policy under root compartment 103 | echo -e "${CYAN}INFO: Creating policy...${NC}" 104 | policy_json=$(oci iam policy create \ 105 | --compartment-id $tenancy_id \ 106 | --description "Allow marketplace service to read images + stacks" \ 107 | --name $policy_name \ 108 | --statements file://./tmp_mkpl_policy.json) 109 | policy_return=$? 110 | echo $policy_json | jq -M . 111 | 112 | if [[ $policy_return -eq 0 ]] 113 | then 114 | echo -e "${GREEN}SUCCESS: policy $policy_name created.${NC}" 115 | else 116 | echo -e "${RED}ERROR: policy not created.${NC}" 117 | fi 118 | 119 | echo -e "${CYAN}INFO: cleaning policy tmp file...${NC}" 120 | rm -f tmp_mkpl_policy.json 121 | 122 | echo -e "${CYAN}INFO: script is idempotent, 409 errors are ignorable.${NC}" 123 | 124 | echo -e "\n\n\n" 125 | echo -e "${CYAN}INFO: values to setup tenancy in partner portal${NC}" 126 | echo -e "${CYAN}tenancy_id: $tenancy_id${NC}" 127 | echo -e "${CYAN}compartment_id: $comp_id${NC}" 128 | echo -e "" 129 | 130 | # testing override 131 | if [ -n "$TESTING" ] 132 | then 133 | echo -e "${CYAN}INFO: deleting testing resources...${NC}" 134 | id=$(echo $comp_json | jq -r '.data.id') 135 | echo -e "${CYAN}INFO: deleting $id ...${NC}" 136 | oci iam compartment delete --force --compartment-id $id 137 | id=$(echo $policy_json | jq -r '.data.id') 138 | echo -e "${CYAN}INFO: deleting $id ...${NC}" 139 | oci iam policy delete --force --policy-id $id 140 | fi 141 | --------------------------------------------------------------------------------