├── .github ├── ISSUE_TEMPLATE │ ├── community-meeting.md │ └── free-form.md └── workflows │ └── issues.yml ├── .gitignore ├── LICENSE ├── MAINTAINERS.md ├── OWNERS ├── README.md ├── ROADMAP.md ├── assets └── icons │ ├── horizontal │ ├── black │ │ ├── shipwright-horizontal-black.png │ │ └── shipwright-horizontal-black.svg │ ├── color │ │ ├── shipwright-horizontal-color.png │ │ └── shipwright-horizontal-color.svg │ └── white │ │ ├── shipwright-horizontal-white.png │ │ └── shipwright-horizontal-white.svg │ ├── icon │ ├── black │ │ ├── shipwright-icon-black.png │ │ └── shipwright-icon-black.svg │ ├── color │ │ ├── shipwright-icon-color.png │ │ └── shipwright-icon-color.svg │ └── white │ │ ├── shipwright-icon-white.png │ │ └── shipwright-icon-white.svg │ └── stacked │ ├── black │ ├── shipwright-stacked-black.png │ └── shipwright-stacked-black.svg │ ├── color │ ├── shipwright-stacked-color.png │ └── shipwright-stacked-color.svg │ └── white │ ├── shipwright-stacked-white.png │ └── shipwright-stacked-white.svg └── ships ├── 0001-previous-proposals.md ├── 0002-build-execution-using-build-run.md ├── 0003-buildstrategy.md ├── 0004-buildstrategy-steps-resources.md ├── 0005-runtime-image.md ├── 0006-local-registry-image-specs.md ├── 0007-shipwright-website.md ├── 0008-remote-artifacts.md ├── 0009-cli.md ├── 0010-buildstrategy-annotation-propagation.md ├── 0011-dedicated-shipwright-operator.md ├── 0012-webhook-validation.md ├── 0013-modifying-output-image.md ├── 0014-parameterize-strategies.md ├── 0015-removal-tekton-resources.md ├── 0016-enable-local-source-code-support.md ├── 0017-deprecate-runtime.md ├── 0018-build-env-vars.md ├── 0019-package-manager-releases.md ├── 0020-shipwright-image-api.md ├── 0021-local-source-upload.md ├── 0022-build-strategy-volumes.md ├── 0023-surface-results-in-buildruns.md ├── 0024-surfacing-errors-to-buildrun.md ├── 0025-strategy-parameters-enhancements.md ├── 0026-shipwright-managed-push.md ├── 0027-image-abstraction-layer.md ├── 0028-buildrun-cleanup.md ├── 0029-one-off-builds.md ├── 0031-shipwright-trigger.md ├── 0032-image-read-mirror-from-namespace.md ├── 0033-build-output-vulnerability-scanning.md ├── 0035-beta-api-changes.md ├── 0036-runAs-for-supporting-steps.md ├── 0037-build-output-timestamp.md ├── 0038-release-branching-backports.md ├── 0039-build-scheduler-opts.md ├── README.md ├── assets ├── 0005-source-upload-sequence-diagram.drawio.png ├── 0006-hub-and-spoke-model.png ├── 0031-shipwright-trigger-components.drawio.png └── 0031-shipwright-trigger-webhook.drawio.png └── guidelines └── ship-template.md /.github/ISSUE_TEMPLATE/community-meeting.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Community Meeting 3 | about: Create a a community meeting 4 | title: "MONTH DAY, YEAR Community Meeting" 5 | labels: community,meeting 6 | assignees: '' 7 | 8 | --- 9 | 10 | - Please add a topic in this thread and add a link to the GitHub issue associated with the topic. 11 | - Please make sure you give folks enough time to review/discuss the topic offline on GitHub before coming into the meeting 12 | - (optional) Paste the image of an animal 😸 13 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/free-form.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Free Form 3 | about: Create an unstructured issue 4 | title: '' 5 | labels: '' 6 | assignees: '' 7 | 8 | --- 9 | -------------------------------------------------------------------------------- /.github/workflows/issues.yml: -------------------------------------------------------------------------------- 1 | name: Add issue or pull request to Project 2 | 3 | on: 4 | issues: 5 | types: 6 | - opened 7 | pull_request_target: 8 | types: 9 | - opened 10 | - reopened 11 | 12 | jobs: 13 | add-to-project: 14 | runs-on: ubuntu-latest 15 | steps: 16 | - name: Add issue to project 17 | uses: actions/add-to-project@v0.5.0 18 | with: 19 | project-url: https://github.com/orgs/shipwright-io/projects/6 20 | github-token: ${{ secrets.ADD_TO_PROJECT_TOKEN }} 21 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Binaries for programs and plugins 2 | *.exe 3 | *.exe~ 4 | *.dll 5 | *.so 6 | *.dylib 7 | 8 | # Test binary, built with `go test -c` 9 | *.test 10 | 11 | # Output of the go coverage tool, specifically when used with LiteIDE 12 | *.out 13 | 14 | # Dependency directories (remove the comment below to include it) 15 | # vendor/ 16 | -------------------------------------------------------------------------------- /MAINTAINERS.md: -------------------------------------------------------------------------------- 1 | # Maintainers 2 | 3 | The current Maintainers Group for the Shipwright Project consists of: 4 | 5 | | Name | Employer | GitHub ID | Responsibilities | 6 | | ----------------- | -------- | ----------------------------------------------------- | -------------------------- | 7 | | Sascha Schwarze | IBM | [SaschaSchwarze0](https://github.com/SaschaSchwarze0) | Administrator, Maintainer | 8 | | Adam Kaplan | Red Hat | [adambkaplan](https://github.com/adambkaplan) | Administrator, Maintainer | 9 | | Enrique Encalada | IBM | [qu1queee](https://github.com/qu1queee) | Administrator, Maintainer | 10 | 11 | 12 | This list must be kept in sync with the [CNCF Project Maintainers list](https://github.com/cncf/foundation/blob/master/project-maintainers.csv). 13 | 14 | The Project Governance and roles definition is under development. This document will be updated in the future with a link to the governance file. 15 | -------------------------------------------------------------------------------- /OWNERS: -------------------------------------------------------------------------------- 1 | # The OWNERS file lists individuals who are allowed to approve pull requests by prow 2 | 3 | approvers: 4 | - adambkaplan 5 | - qu1queee 6 | - SaschaSchwarze0 7 | 8 | reviewers: 9 | - qu1queee 10 | - SaschaSchwarze0 11 | - adambkaplan 12 | - HeavyWombat 13 | 14 | emeritus_approvers: 15 | - sbose78 16 | 17 | emeritus_reviewers: 18 | - sbose78 19 | - gabemontero 20 | - ImJasonH 21 | - otaviof # 2024-05-24 -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Welcome to Project Shipwright! 2 | 3 | This is the central hub for community process documentation, as well as assets owned by the project. 4 | 5 | ## What is Shipwright? 6 | 7 | Shipwright is a framework for building container images on Kubernetes. 8 | The core consists of the following code repositories: 9 | 10 | * [build](https://github.com/shipwright-io/build) - the Build APIs and associated controller to run builds. 11 | * [cli](https://github.com/shipwright-io/cli) - the `shp` command line for Shipwright builds 12 | * [operator](https://github.com/shipwright-io/operator) - an operator to install Shipwright components on Kubernetes via OLM. 13 | 14 | Documentation is spread across the code repositories, and is consolidated in the [website](https://github.com/shipwright-io/website) repository. 15 | 16 | ## How can I get involved? 17 | 18 | * Review our [Contributing Guide](https://github.com/shipwright-io/.github/blob/main/CONTRIBUTING.md). 19 | * Join our mailing lists for [users](https://lists.shipwright.io/archives/list/shipwright-users@lists.shipwright.io/) or [contributors](https://lists.shipwright.io/archives/list/shipwright-dev@lists.shipwright.io/) 20 | * Talk with us on the Kubernetes Slack - [#shipwright](https://kubernetes.slack.com/messages/shipwright) channel 21 | * Attend one of our community meetings - see our [public calendar](https://zoom-lfx.platform.linuxfoundation.org/meetings/shipwright) for up to date information. 22 | * Weekly Community Update - every Monday, 9AM Eastern Time 23 | * Bi-weekly Backlog Refinement - every other Thursday, 10AM Eastern Time 24 | 25 | ### External Integrations 26 | 27 | Do you want to automatically track what Shipwright is working on? Email the 28 | [Shipwright Admins](mailto:shipwright-admins@lists.cncf.io) to request access to Shipwright's webhook feed. 29 | 30 | ## SHIPs 31 | 32 | Enhancements to Shipwright are discussed through the Shipwright Improvement Proposal (SHIP) process. 33 | If you have an idea that you want to submit to Shipwright, please read the proposal process [guidelines](/ships/README.md). 34 | 35 | ## Roadmap 36 | 37 | Want to know what we are working on? Start by taking a look at the project [roadmap](/ROADMAP.md). 38 | 39 | ## Icons and logos 40 | 41 | Graphic assets for the project, such as official logos, are located under the `assets` directory. 42 | 43 | ## License 44 | 45 | Unless otherwise noted, all Project Shipwright code and assets are copyrighted under the Apache 2.0 license. 46 | 47 | Copyright The Shipwright Contributors 48 | 49 | SPDX-License-Identifier: Apache-2.0 50 | -------------------------------------------------------------------------------- /ROADMAP.md: -------------------------------------------------------------------------------- 1 | # Community-Driven Roadmap 2 | 3 | Shipwright's detailed roadmap can be found on the 4 | [Shipwright Overview GitHub Project](https://github.com/orgs/shipwright-io/projects/6). All 5 | repositories are currently tied to the release cycle of the core 6 | [build](https://github.com/shipwright-io/build) sub-project. Shipwright aims to issue minor 7 | releases quarterly, using the same major and minor semantic version across repositories. 8 | 9 | ## 2025 Release Schedule 10 | 11 | Below is the tentative release schedule for 2025. Actual release dates may vary based on community 12 | availability and release stability. 13 | 14 | - v0.15.0: week of 2025-02-28 15 | - v0.16.0: week of 2025-05-30 16 | - v0.17.0: week of 2025-08-29 17 | - v0.18.0: week of 2025-11-28 18 | -------------------------------------------------------------------------------- /assets/icons/horizontal/black/shipwright-horizontal-black.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shipwright-io/community/592c9226d08d4d0a331eb0d4b0b2540f4dad5188/assets/icons/horizontal/black/shipwright-horizontal-black.png -------------------------------------------------------------------------------- /assets/icons/horizontal/black/shipwright-horizontal-black.svg: -------------------------------------------------------------------------------- 1 | 2 | 10 | 12 | 15 | 17 | 22 | 26 | 30 | 33 | 36 | 39 | 42 | 45 | 48 | 51 | 54 | 57 | 60 | 61 | 62 | 63 | 64 | 65 | -------------------------------------------------------------------------------- /assets/icons/horizontal/color/shipwright-horizontal-color.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shipwright-io/community/592c9226d08d4d0a331eb0d4b0b2540f4dad5188/assets/icons/horizontal/color/shipwright-horizontal-color.png -------------------------------------------------------------------------------- /assets/icons/horizontal/white/shipwright-horizontal-white.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shipwright-io/community/592c9226d08d4d0a331eb0d4b0b2540f4dad5188/assets/icons/horizontal/white/shipwright-horizontal-white.png -------------------------------------------------------------------------------- /assets/icons/horizontal/white/shipwright-horizontal-white.svg: -------------------------------------------------------------------------------- 1 | 2 | 10 | 12 | 15 | 18 | 23 | 27 | 32 | 35 | 38 | 41 | 44 | 47 | 50 | 53 | 56 | 59 | 62 | 63 | 64 | 65 | 66 | 67 | -------------------------------------------------------------------------------- /assets/icons/icon/black/shipwright-icon-black.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shipwright-io/community/592c9226d08d4d0a331eb0d4b0b2540f4dad5188/assets/icons/icon/black/shipwright-icon-black.png -------------------------------------------------------------------------------- /assets/icons/icon/black/shipwright-icon-black.svg: -------------------------------------------------------------------------------- 1 | 2 | 10 | 12 | 15 | 17 | 22 | 23 | 24 | 25 | -------------------------------------------------------------------------------- /assets/icons/icon/color/shipwright-icon-color.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shipwright-io/community/592c9226d08d4d0a331eb0d4b0b2540f4dad5188/assets/icons/icon/color/shipwright-icon-color.png -------------------------------------------------------------------------------- /assets/icons/icon/color/shipwright-icon-color.svg: -------------------------------------------------------------------------------- 1 | 2 | 10 | 12 | 15 | 18 | 24 | 27 | 38 | 41 | 44 | 47 | 48 | 51 | 62 | 73 | 84 | 85 | 86 | 87 | 88 | -------------------------------------------------------------------------------- /assets/icons/icon/white/shipwright-icon-white.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shipwright-io/community/592c9226d08d4d0a331eb0d4b0b2540f4dad5188/assets/icons/icon/white/shipwright-icon-white.png -------------------------------------------------------------------------------- /assets/icons/icon/white/shipwright-icon-white.svg: -------------------------------------------------------------------------------- 1 | 2 | 10 | 12 | 15 | 18 | 23 | 24 | 25 | 26 | -------------------------------------------------------------------------------- /assets/icons/stacked/black/shipwright-stacked-black.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shipwright-io/community/592c9226d08d4d0a331eb0d4b0b2540f4dad5188/assets/icons/stacked/black/shipwright-stacked-black.png -------------------------------------------------------------------------------- /assets/icons/stacked/black/shipwright-stacked-black.svg: -------------------------------------------------------------------------------- 1 | 2 | 10 | 12 | 15 | 19 | 25 | 30 | 36 | 40 | 44 | 48 | 52 | 56 | 60 | 64 | 68 | 72 | 76 | 77 | 78 | 79 | 80 | 81 | -------------------------------------------------------------------------------- /assets/icons/stacked/color/shipwright-stacked-color.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shipwright-io/community/592c9226d08d4d0a331eb0d4b0b2540f4dad5188/assets/icons/stacked/color/shipwright-stacked-color.png -------------------------------------------------------------------------------- /assets/icons/stacked/color/shipwright-stacked-color.svg: -------------------------------------------------------------------------------- 1 | 2 | 10 | 12 | 15 | 17 | 20 | 26 | 39 | 52 | 53 | 58 | 63 | 66 | 69 | 72 | 75 | 78 | 81 | 84 | 87 | 90 | 93 | 94 | 95 | 96 | 97 | 98 | -------------------------------------------------------------------------------- /assets/icons/stacked/white/shipwright-stacked-white.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shipwright-io/community/592c9226d08d4d0a331eb0d4b0b2540f4dad5188/assets/icons/stacked/white/shipwright-stacked-white.png -------------------------------------------------------------------------------- /assets/icons/stacked/white/shipwright-stacked-white.svg: -------------------------------------------------------------------------------- 1 | 2 | 10 | 12 | 15 | 18 | 24 | 28 | 33 | 36 | 39 | 42 | 45 | 48 | 51 | 54 | 57 | 60 | 63 | 64 | 65 | 66 | 67 | 68 | -------------------------------------------------------------------------------- /ships/0001-previous-proposals.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | --- 8 | title: previous-proposals 9 | authors: 10 | - "@adambkaplan" 11 | reviewers: 12 | - "@ImJasonH" 13 | - "@alicedoe" 14 | approvers: 15 | - "@qu1queee" 16 | - "@sbose78" 17 | creation-date: 2021-05-24 18 | last-updated: 2021-06-03 19 | status: implemented 20 | see-also: 21 | - https://github.com/shipwright-io/build/tree/main/docs/proposals 22 | replaces: [] 23 | superseded-by: [] 24 | --- 25 | 26 | # Previous Proposals 27 | 28 | ## Release Signoff Checklist 29 | 30 | - [ ] Enhancement is `implementable` 31 | - [ ] Design details are appropriately documented from clear requirements 32 | - [ ] Test plan is defined 33 | - [ ] Graduation criteria for dev preview, tech preview, GA 34 | - [ ] User-facing documentation is created in [docs](/docs/) 35 | 36 | ## Summary 37 | 38 | This is a summary of the merged proposals that pre-dated the SHIP process. 39 | Full details of these proposals can be found in [shipwright-io/build/docs/proposals](https://github.com/shipwright-io/build/tree/main/docs/proposals). 40 | 41 | ## Motivation 42 | 43 | As of May 24, 2021, the community repository will serve as the official record of all new Shipwright enhancement proposals. 44 | Previously discussed and merged proposals will not be migrated here - they will remain in shipwright-io/build for posterity. 45 | Proposals actively under discussion at this time will also remain in shipwright-io/build until the respective pull requests are merged or closed. 46 | 47 | ### Goals 48 | 49 | * Document enhancement proposals approved prior to the SHIP process. 50 | * Provide links back to prior proposals for reference. 51 | 52 | ### Non-Goals 53 | 54 | * Convert the older enhancement proposals to SHIPs. 55 | 56 | ## Previous Proposals 57 | 58 | | Title | Status | Last Updated | Link | 59 | | ------ | ----- | ------------ | ---- | 60 | | Build execution using the BuildRun API | design | 2020-05-25 | [link](https://github.com/shipwright-io/build/blob/main/docs/proposals/build-execution-using-build-run.md) | 61 | | The BuildStrategy API | design |2020-05-28 | [link](https://github.com/shipwright-io/build/blob/main/docs/proposals/buildstrategy.md) | 62 | | Runtime Image Support | implementable | 2020-06-14| [link](https://github.com/shipwright-io/build/blob/main/docs/proposals/runtime-image.md) | 63 | | Build Steps Resource Limitations | implementable | 2020-06-15 | [link](https://github.com/shipwright-io/build/blob/main/docs/proposals/buildstrategy-steps-resources.md) | 64 | | Local Registry Image Specs | provisional | 2020-08-10 | [link](https://github.com/shipwright-io/build/blob/main/docs/proposals/local-registry-image-specs.md) | 65 | | Shipwright Website | implementable | 2020-09-14 | [link](https://github.com/shipwright-io/build/blob/main/docs/proposals/shipwright-website.md) | 66 | | Remote Artifacts | implementable | 2020-11-12 | [link](https://github.com/shipwright-io/build/blob/main/docs/proposals/remote-artifacts.md) | 67 | | CLI | provisional | 2020-12-02 | [link](https://github.com/shipwright-io/build/blob/main/docs/proposals/cli.md) | 68 | | Build Strategy Annotations | implementable | 2021-01-21 | [link](https://github.com/shipwright-io/build/blob/main/docs/proposals/buildstrategy-annotation-propagation.md) | 69 | | Dedicated Shipwright Operator | implementable | 2021-02-09 | [link](https://github.com/shipwright-io/build/blob/main/docs/proposals/dedicated-shipwright-operator.md) | 70 | | Webhook Validation | provisional | 2021-03-19 | [link](https://github.com/shipwright-io/build/blob/main/docs/proposals/webhook-validation.md) | 71 | | Modifying Output Image | provisional | 2021-04-14 | [link](https://github.com/shipwright-io/build/blob/main/docs/proposals/modifying-output-image.md) | 72 | | Parameterize Build Strategies | implementable | 2021-04-20 | [link](https://github.com/shipwright-io/build/blob/main/docs/proposals/parameterize-strategies.md) | 73 | | Removing Tekton resource usages | implementable | 2021-04-20 | [link](https://github.com/shipwright-io/build/blob/main/docs/proposals/removal-tekton-resources.md) | 74 | | Enable Local Source Code Support | implementable | 2021-05-03 | [link](https://github.com/shipwright-io/build/blob/main/docs/proposals/enable-local-source-code-support.md) | 75 | 76 | 77 | ### Implementation Notes 78 | 79 | Not applicable. 80 | 81 | ### Test Plan 82 | 83 | Not applicable. 84 | 85 | ### Release Criteria 86 | 87 | Not applicable. 88 | 89 | ### Risks and Mitigations 90 | 91 | Not applicable. 92 | 93 | ## Drawbacks 94 | 95 | Not applicable. 96 | 97 | ## Alternatives 98 | 99 | Not applicable. 100 | 101 | ## Implementation History 102 | 103 | 2021-06-03: Implemented as a reference. 104 | -------------------------------------------------------------------------------- /ships/0002-build-execution-using-build-run.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | --- 8 | 9 | title: build-execution-using-build-run 10 | authors: 11 | 12 | - "@sbose78" 13 | - "@zhangtbj" 14 | 15 | status: design 16 | last-updated: 2021-13-09 17 | replaces: 18 | - [build-execution-using-build-run](https://github.com/shipwright-io/build/blob/main/docs/proposals/build-execution-using-build-run.md) 19 | --- 20 | 21 | # Build execution using the BuildRun API 22 | 23 | ## About 24 | 25 | A `BuildRun` is an immutable CR/object that represents a single execution of the `Build`. 26 | 27 | The simplest `BuildRun` looks like this: 28 | 29 | ```yaml 30 | apiVersion: build.dev/v1alpha1 31 | kind: BuildRun 32 | metadata: 33 | generateName: account-service-build- 34 | spec: 35 | buildRef: 36 | name: kaniko-golang-build 37 | ``` 38 | 39 | The `BuildRun` API also provides a way to override/specify execution time information. 40 | 41 | As an example, a `Build` configuration should not have to accurately specify the resource requirements - the information on resource requirements is valuable at build execution time which varies from environment to environment. 42 | 43 | ## Defining a BuildRun 44 | 45 | The `BuildRun`, apart from triggering a build execution, also provides the user an opportunity to specify/override a subset of Build configuration which are likely to change at build execution time. 46 | 47 | ### What is deterministic 48 | 49 | #### What are we building 50 | 51 | - The source code that's being built into an image. 52 | - The application binary that's being packaged into an image. 53 | 54 | The above information wraps the integrity of the code 55 | that we are building into an image. 56 | 57 | #### How are we building 58 | 59 | - The `BuildStrategy` being used to build the image. 60 | - Inputs associated with the `BuildStrategy` : Dockerfile, builder image, etc. 61 | - The runtime base image being used to build a lean image. 62 | 63 | The above information wraps the know-how that drives the process of converting source code into an image. 64 | 65 | ### What is non-deterministic 66 | 67 | - The Image being pushed to 68 | - Repository 69 | - Tag 70 | - The Service Account used to execute the build. 71 | - Execution-time resource requiements. 72 | 73 | As an example, the source code [nodejs-rest-http-crud](https://github.com/nodeshift-starters/nodejs-rest-http-crud) might be pushed to different image repositories depending on who is executing the build. 74 | 75 | Note: 76 | In some scenarios, the source code 'revision' may not necessarily be deterministic as well, especially in case of builds triggered from PRs/forks. 77 | 78 | ### Deciding what should be overridden 79 | 80 | - Could the value of attribute X be reasonably determined at the time of `Build` configuration specification ? 81 | - Could the modification of attribute X reasonably compromise the integrity of the build ? 82 | - Could the value of attribute X reasonably differ in the context of a clusters or a namespace ? 83 | 84 | The above is a non-exhaustive list of questions that should help us modify the BuildRun API in future. 85 | 86 | ### Next steps / Consequences 87 | 88 | Here's what it means to adopt this proposal: 89 | 90 | 1. Make `spec.output` optional in `Build` API's `spec` . 91 | 2. Introduce `spec.output` in the `BuildRun` API's `spec`. 92 | 3. Make the `BuildRun` immutable. 93 | -------------------------------------------------------------------------------- /ships/0003-buildstrategy.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | --- 8 | title: buildstrategy 9 | authors: 10 | - "@sbose78" 11 | - "@qu1queee" 12 | status: design 13 | last-updated: 2021-13-09 14 | replaces: 15 | - [buildstrategy](https://github.com/shipwright-io/build/blob/main/docs/proposals/buildstrategy.md) 16 | --- 17 | 18 | # The BuildStrategy API 19 | 20 | ## Goals 21 | 22 | ### User-defined build strategies 23 | 24 | Users and enterprises have strong opinions on how to build container images from source code. 25 | This project aims to enable admins to define build strategies for building container images in a Kubernetes cluster. 26 | 27 | ### Accomplish more by specifying less! 28 | 29 | A slim BuildStrategy is one where the BuildStrategy author gets to accomplish more by specifying less. Without enabling authors to do so, 30 | we would be inadvertently making it hard to define BuildStrategies, thereby undermining the very premise of simplicity that this project aims to provide with the BuildStrategy CRD. 31 | 32 | For example, the author of the build strategy should not have to specify how to push images to remote registries. 33 | 34 | ### Simplicity 35 | 36 | This project uses the `TaskRun` API under-the-hood to execute an image build without leaking abstraction. The complexity of defining 37 | a BuildStrategy must be less than that of defining a Tekton `Task`. 38 | 39 | ## Defining a BuildStrategy 40 | 41 | A BuildStrategy CR is defined as a list of [corev1.Container](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/#container-v1-core) 42 | execution steps. 43 | 44 | ### What must be specified 45 | 46 | The author of the BuildStrategy CR may thereby specify the command(s) that need to be executed inside a container to convert source code 47 | into a container image. 48 | 49 | As an example, the build author might wish to express the following commands as container steps to do a [buildpacks-v3](https://github.com/buildpacks/lifecycle) image build: 50 | 51 | * detector - chooses buildpacks (via /bin/detect) 52 | * analyzer - restores launch layer metadata from the previous build 53 | * builder - executes buildpacks (via /bin/build) 54 | 55 | ### What need not be specified 56 | 57 | The Build controller should take care of executing build steps common to all strategies without making it mandatory 58 | for users to specify 'well-known' steps. 59 | 60 | The BuildStrategy author need not necessarily specify: 61 | 62 | * How the image is to be pushed to a registry. 63 | * Where the root CA certificates are to be picked up for communicating with secure registries. 64 | * How to generate lean runtime images from the built image. 65 | * Or, anything that can be classified as common or popular activity in an image build flow. 66 | 67 | From an implementation perspective, well-known `BuildStrategy` steps are `Tekton` `Task` steps that are dynamically generated on-the-fly. 68 | 69 | ### Optional overrides 70 | 71 | If the BuildStrategy author wishes to be explicit about the "how" of pushing an image to registry, the author should be able 72 | express that. 73 | 74 | The BuildStrategy author could convey the same to the controller by annotating the `BuildStrategy` with: 75 | 76 | ```sh 77 | buildstrategy.build.dev/contains-image-push: true 78 | buildstrategy.build.dev/contains-runtime-image: true 79 | ``` 80 | 81 | ### Parameterization 82 | 83 | Attributes from the `Build` CR could be used as parameters while defining a `BuildStrategy`. 84 | 85 | Examples: 86 | 87 | * `$(build.output.image)` implies that the `Build` CR's `spec.output.image` value be used. 88 | 89 | * `$(build.parameters.skip_ssl_verify)` implies that the value corresponding to key `skip_ssl_verify` in `Build` CR's `spec.parameters` be used. 90 | 91 | ### Consequences 92 | 93 | At the time of writing this proposal, we've already had `BuildStrategies` defined in `/samples` where the `BuildStrategy` author has expressed [corev1.Container](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.11/) steps to build images and push them to a registry. 94 | 95 | Here's what it means to adopt this proposal: 96 | 97 | 1. Explore the possibility of removing the image push step in all `BuildStrategies`. 98 | 99 | 2. Explore the possibility of generating lean runtime images if the user provides the runtime base image information in the `Build` CR. 100 | 101 | 3. If the above experiments succeed, ensure that **Optional overrides** as stated in the previous section, are supported. 102 | -------------------------------------------------------------------------------- /ships/0005-runtime-image.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | --- 8 | title: runtime-image-support 9 | authors: 10 | - "@sbose78" 11 | - "@otaviof" 12 | status: withdrawn 13 | last-updated: 2021-13-09 14 | replaces: 15 | - [runtime-image](https://github.com/shipwright-io/build/blob/main/docs/proposals/runtime-image.md) 16 | --- 17 | 18 | Runtime-Image Support 19 | --------------------- 20 | 21 | # Summary 22 | 23 | This enhancement proposal describes the support of runtime-image in build-v2 operator. A runtime-image is a composition based on the image just created by a third-party builder, used as input for the runtime-image. 24 | 25 | # Motivation 26 | 27 | With the runtime-image feature in place, it will be employed for the creation of _lean-images_, which takes the ability of picking only certain parts of the original image, creating a runtime-image with the reduced data-set. 28 | 29 | Additionally, creating images with a different base-image, and other advanced use-cases are also supported by this feature. 30 | 31 | # Proposal 32 | 33 | The sections below are describing the changes necessary to introduce the runtime-image. 34 | 35 | ## API Changes 36 | 37 | A new attribute will be added on Build resource, as the following example: 38 | 39 | 40 | ```yml 41 | --- 42 | apiVersion: build.dev/v1alpha1 43 | kind: Build 44 | # ... 45 | spec: 46 | runtime: 47 | base: 48 | image: runtime-base-image:latest 49 | workDir: /workspace/source 50 | env: 51 | JAVA_HOME: /path/to/java-home 52 | labels: 53 | custom-image-label: value 54 | paths: 55 | - /path/to/file 56 | - /path/to/directory 57 | - /path/to/another/directory:/target/location 58 | user: 59 | name: username 60 | group: 1001 61 | run: 62 | - groupadd -g 1001 group 63 | - useradd -u 1001 -g group username 64 | entrypoint: 65 | - java 66 | - -jar 67 | - ... 68 | ``` 69 | 70 | The new attributes added are: 71 | 72 | - `spec.runtime.base`: specifies the runtime base-image to be used, using [Image](https://github.com/shipwright-io/build/blob/97012ab56417ce1691a70896d90e490ea6a4d23c/pkg/apis/build/v1alpha1/build_types.go#L58) as type; 73 | - `spec.runtime.workDir`: path to `WORKDIR` in runtime-image; 74 | - `spec.runtime.env`: runtime-image additional environment variables, key-value; 75 | - `spec.runtime.labels`: runtime-image additional labels, key-value; 76 | - `spec.runtime.paths`: list of directories/files paths to be copied to runtime-image, those can be defined as `:` split by colon (`:`); 77 | - `spec.runtime.user.name`: username employed on `USER` directive, and also to change ownership of files copied to the runtime-image; 78 | - `spec.runtime.user.group`: group name (or GID), employed to change ownership and on `USER` directive; 79 | - `spec.runtime.run`: arbitrary commands to be executed as `RUN` blocks, before `COPY`; 80 | - `spec.runtime.entrypoint`: entrypoint command, specified as a list; 81 | 82 | ## Runtime-Image 83 | 84 | Given the API changes described above, we will now address how the runtime-image will be created. This "runtime" is always based on an image built by the `BuildStrategy` (or `ClusterBuildStrategy`) steps, and therefore, we can introduce new tasks to add the elements needed for the runtime-image creation. 85 | 86 | ### Runtime Dockerfile 87 | 88 | The first step to be appended on the given `BuildStrategy` instance is the `Dockerfile` for the runtime-image, commonly named `Dockerfile.runtime`. 89 | 90 | This Dockerfile would like the following example: 91 | 92 | ``` 93 | FROM output-image:tag as builder 94 | 95 | FROM runtime-image:tag 96 | ENV VARIABLE=value 97 | LABEL label=value 98 | RUN groupadd -g 1001 group 99 | RUN useradd -u 1001 -g group username 100 | COPY --chown="username:1001" --from=builder "/path/to/source" "/path/to/destination" 101 | USER username:1001 102 | WORKDIR "/path/to/destination" 103 | ENTRYPOINT [ "command", "args" ] 104 | ``` 105 | 106 | The example shows a [multi-stage image build](https://docs.docker.com/develop/develop-images/multistage-build/) using the outcomes of the original `BuildStrategy` steps as `builder` image, and afterwards, rendering all the elements supported by the [new `runtime` section](#API-Changes). To create this file we may employ [`text/template`](https://golang.org/pkg/text/template/) to render its contents. 107 | 108 | #### `COPY` and `USER` 109 | 110 | When copying data to the runtime-image, it will automatically set the `--chown` flag in order to change ownership during copy of paths. When `.spec.runtime.user` is not informed, this argument won't take place. The same concept is applied on `USER` directive, it will only be set when `.spec.runtime.user.name` is informed. 111 | 112 | As for `.spec.runtime.user.group`, it will take part of `USER` and `COPY --chown` when informed. 113 | 114 | ### Tekton Tasks 115 | 116 | During the generation of [Tekton's `TaskRun`](https://github.com/shipwright-io/build/blob/6cad175fca9a0443c669ecf84ce526764e0260c1/pkg/reconciler/buildrun/resources/taskrun.go#L58), we have the opportunity to add extra steps. The first one to be added is the rendering of the `Dockerfile.runtime`, for instance: 117 | 118 | ```yml 119 | --- 120 | apiVersion: tekton.dev/v1beta1 121 | kind: TaskRun 122 | metadata: 123 | # ... 124 | spec: 125 | taskSpec: 126 | steps: 127 | - name: runtime-dockerfile 128 | image: $(build.builder.image) 129 | securityContext: 130 | runAsUser: 0 131 | workingDir: /workspace/source 132 | command: 133 | - /bin/bash 134 | args: 135 | - -x 136 | - -c 137 | - > 138 | echo '' >Dockerfile.runtime 139 | ``` 140 | 141 | During the implementation phase `DOCKERFILE_CONTENT` will become the actual [runtime Dockerfile](#Runtime-Dockerfile). 142 | 143 | And, the last step to be added is the container-image build, for this position we can take either [`buildah`](https://github.com/shipwright-io/build/blob/97012ab56417ce1691a70896d90e490ea6a4d23c/samples/build/build_buildah_cr.yaml) or [`kaniko`](https://github.com/shipwright-io/build/blob/97012ab56417ce1691a70896d90e490ea6a4d23c/samples/build/build_kaniko_cr.yaml) strategies for guidance. 144 | 145 | ## Experiments 146 | 147 | During the development phase we've evaluated two different application ecosystems. A common [Node.js](https://gist.github.com/otaviof/eccf5abe879a8218cf5b807f520367f4) application, and [Pet-Clinic](https://gist.github.com/otaviof/53aad504ccc59681fe3875dbf3150c55), a Java based application. 148 | 149 | Since the controller generates a Dockerfile on the fly, those use cases worked as expected, producing another container image, reusing container image URL and tag. 150 | -------------------------------------------------------------------------------- /ships/0006-local-registry-image-specs.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | --- 8 | title: local-registries 9 | authors: 10 | - "@adambkaplan" 11 | reviewers: 12 | - "@coreydaley" 13 | - "@otaviof" 14 | approvers: 15 | - "@qu1queee" 16 | - "@sbose78" 17 | creation-date: 2020-07-24 18 | last-updated: 2021-13-09 19 | status: provisional 20 | see-also: [] 21 | replaces: 22 | - [local-registry-image-specs](https://github.com/shipwright-io/build/blob/main/docs/proposals/local-registry-image-specs.md) 23 | superseded-by: [] 24 | --- 25 | 26 | # Local Registry Image Specs 27 | 28 | ## Release Signoff Checklist 29 | 30 | - [ ] Enhancement is `implementable` 31 | - [ ] Design details are appropriately documented from clear requirements 32 | - [ ] Test plan is defined 33 | - [ ] Graduation criteria for dev preview, tech preview, GA 34 | - [ ] User-facing documentation is created in [docs](/docs/) 35 | 36 | ## Summary 37 | 38 | [KEP 1755](https://github.com/kubernetes/enhancements/tree/master/keps/sig-cluster-lifecycle/generic/1755-communicating-a-local-registry) 39 | provides an official convention for clusters to declare the presence of an internal image registry. 40 | This proposal aims to take advantage of declared local image registries, and let application 41 | developers simplify how they declare image pull and push specs. This may also help `Build` objects 42 | be portable across clusters. 43 | 44 | ## Motivation 45 | 46 | The `Build` API in its present form requires that users provide a full image pull spec for image 47 | references. This can be cumbersome when referencing an image on an internal registry, such as those 48 | present in [KIND](https://kind.sigs.k8s.io/docs/user/local-registry/), 49 | [Rancher](https://github.com/rancher/k3d/blob/main/docs/usage/guides/registries.md#using-a-local-registry), 50 | [microk8s](https://microk8s.io/docs/registry-built-in), and 51 | [OpenShift](https://docs.openshift.com/container-platform/4.5/registry/architecture-component-imageregistry.html). 52 | This proposal will provide a means of declaring that an image spec references a local registry, 53 | thereby allowing host information to be removed from the image reference. 54 | 55 | ### Goals 56 | 57 | - Simplify how images on local image registries can be referenced in `Build` objects. 58 | - Provide a means of declaring that an image references a cluster-local image registry. 59 | 60 | ### Non-Goals 61 | 62 | - Automate how pull/push secrets can be added to a `Build` object. 63 | - Support secure image pull/push if a local registry uses self-signed certificates. 64 | - Configure image registry mirrors within builds. 65 | - Alter the default image registry for image pull/push if no hostname is specified. 66 | 67 | ## Proposal 68 | 69 | Application developers will be able to specify `local: true|false` if an image reference points 70 | to a cluster-local image registry. 71 | 72 | Per [KEP 1755](https://github.com/kubernetes/enhancements/tree/master/keps/sig-cluster-lifecycle/generic/1755-communicating-a-local-registry), 73 | a cluster has a local registry if a `ConfigMap` named `local-registry-hosting` is present in the 74 | `kube-public` namespace. The data in this `ConfigMap` contains hosname information for the image 75 | registry installed on the cluster. The build controller/operator will read this `ConfigMap` to check if 76 | a local image registry is declared. If declared, it will infer the local registry host information 77 | from what is provided in the `ConfigMap`. 78 | 79 | When a `BuildRun` is instantiated, the image specs that declare `local: true` are mutated to 80 | prepend the cluster's local image registry hostname. 81 | 82 | ### User Stories [optional] 83 | 84 | As an application developer 85 | I would like to declare that images in builds reference a local image registry 86 | So that I do not have to provide host information in the pull spec 87 | 88 | ### Implementation Details/Notes/Constraints [optional] 89 | 90 | #### Obtaining local registry information 91 | 92 | Clusters which deploy a local image registry must provide the releavant host info by 93 | publishing it in the `local-registry-hosting` 94 | [ConfigMap](https://github.com/kubernetes/enhancements/tree/master/keps/sig-cluster-lifecycle/generic/1755-communicating-a-local-registry#the-local-registry-hosting-configmap). 95 | The build operator/controller must have RBAC which allows it to read the `ConfigMap` and obtain the 96 | registry host information. For `Build` and `BuildRun` objects, the `HostFromClusterNetwork` value 97 | should be used as the hostname. 98 | 99 | If the `local-registry-hosting` ConfigMap is created, updated, or deleted, the build controller 100 | and/or operator need to react accordingly. 101 | 102 | If the `local-registry-hosting` ConfigMap is not present, the build operator/controller must 103 | be able to function normally. Absence of this information is not considered an error. 104 | 105 | #### Declaring an image as local 106 | 107 | `Image` references in the API will be updated to have the `local` field. An example using the 108 | `output` spec: 109 | 110 | ```yaml 111 | spec: 112 | output: 113 | image: mynamespace/myapp:latest 114 | local: true 115 | ``` 116 | 117 | When an image is declared as `local`, the build controller prepends the local image registry 118 | hostname to the provided image reference. 119 | 120 | If the `local-registry-hosting` ConfigMap specified in KEP-1755 is not present, or does not contain 121 | sufficient information to inject the registry hostname, `Build` and `BuildRun` objects should 122 | present appropriate error messages in there statuses: 123 | 124 | - `Build` objects will set the `Registered` status to `false`. 125 | - `BuildRun` objects will present an error message in their status conditions. Note that this 126 | standard status conditions are not present in the `BuildRun` API at present. 127 | 128 | ### Risks and Mitigations 129 | 130 | **Risk**: App developers use `local: true` on clusters that don't have a local registry defined, 131 | In this event build could inadvertently pull or push images to `docker.io` or an image registry in 132 | the container runtime's unqualified search path. 133 | 134 | *Mitigation*: if a `Build` references a local registry with no registry defined, it's `Registered` 135 | status should be set to `false`. For `BuildRun`, the run should fail with a clear error message in 136 | its status conditions. 137 | 138 | **Risk**: Local image registry configurations can be altered while a build is running. 139 | 140 | *Mitigation*: Image spec resolution should only happen when the `BuildRun` is transformed into an 141 | appropriate Tekton `TaskRun`. The `BuildRun` will likely fail, but that can be addressed by re- 142 | running the same build via a new `BuildRun` object. 143 | 144 | The build controller itself will need to be restarted by a separate operator and reload new 145 | configuration via a deployment rollout. This is a separate matter that is identified in 146 | [#310](https://github.com/redhat-developer/build/issues/310). 147 | 148 | ## Design Details 149 | 150 | ### Test Plan 151 | 152 | 1. Deploy the build operator/controller on a cluster that populates the `local-registry-hosting` 153 | ConfigMap OR as cluster-admin, manually populate the `local-registry-hosting` ConfigMap. 154 | 2. Create a `Build` and/or `BuildRun` which references a local image in the following mannter: 155 | 1. As output for a build. 156 | 2. As the `builder` image - ex. for use in the source-to-image `BuildStrategy`. 157 | 3. As the `runtime` base image for builds that create lean runtime images. 158 | 3. Ensure that a build can push an image referencing the local image spec. 159 | 160 | ### Graduation Criteria 161 | 162 | #### Dev Preview 163 | 164 | As an initial implementation, we can define an environment variable to set the local registry 165 | hostname for the build operator. This can be changed by alterting the deployment for the build 166 | operator, which would force a rollout with the new value. 167 | 168 | #### Tech Preview and GA 169 | 170 | The operator must fully read the `local-registry-hosting` ConfigMap, with the ability to update 171 | the build controller if this ConfigMap changes. 172 | 173 | #### Examples 174 | 175 | ### Upgrade / Downgrade Strategy 176 | 177 | Upgrades add a new field to the API. On downgrade the `local` attribute should be ignored. 178 | 179 | ### Version Skew Strategy 180 | 181 | N/A 182 | 183 | ## Implementation History 184 | 185 | 2020-07-24: Initial proposal 186 | 187 | ## Drawbacks 188 | 189 | 1. The KEP is relatively new, and not many *KS providers will have a need to implement it. 190 | 2. In production environments, may users push images to registries that reside outside of the 191 | cluster. Therefore this feature may not prove valuable. 192 | 3. `Build` and `BuildRun` definitions will not be portable across namespaces with this 193 | implementation - they will only be portable across clusters. 194 | 195 | ## Alternatives 196 | 197 | ### okd ImageStreams 198 | 199 | OpenShift/okd [Imagestreams](https://docs.okd.io/latest/openshift_images/images-understand.html#images-imagestream-use_images-understand) 200 | are a primary motivation for this enhancement proposal. Shortened image pull specs which resolve to 201 | a local image registry is one of many features provided by this API. 202 | 203 | Upstreaming Imagestreams would require significant effort and may be beyond the scope of this 204 | project. 205 | 206 | ### APIs for Default Search Paths 207 | 208 | Most container image build tools inject `docker.io` as the host if an image spec does not have a 209 | domain. This can often be overrode to reference another "default" registry: 210 | 211 | - Kaniko - the `--registry-mirror` option lets you override `docker.io` as the default. [1] 212 | - Buildah - uses the file `/etc/containers/registries.conf` to configure unqualified search 213 | paths. [2] 214 | - Buildpacks - no known means of changing `docker.io` as the default path for pulling images. 215 | 216 | To reference a local image registry, the following would be needed: 217 | 218 | 1. An API needs to be exposed so that the default image registry can be changed to a local registry 219 | on the cluster. This would not need to be published via the mechanisms in KEP-1755. 220 | 2. Builds would need to expose this configuration setting to every build (ex: a `ConfigMap` volume 221 | mount). 222 | 3. Build strategies would need to provide the appropriate configuration option to the commands that 223 | execute the build. 224 | 225 | The main downside of this approach is that every build strategy would need to opt into this 226 | feature. Using `docker.io` as the default image registry is also assumed for the general k8s 227 | ecosystem - alterting this has proved to be a source of bugs, unexpected behavior, and difficult 228 | debug situations. 229 | 230 | Changing the default image registry is also a blunt configuration - it is applied to all image pull 231 | and push actions for the duration of the build. You cannot use the local registry for some image 232 | references, and `docker.io` for others. 233 | 234 | [1] https://github.com/GoogleContainerTools/kaniko#--registry-mirror 235 | [2] https://www.mankier.com/5/containers-registries.conf#Description-Global_Settings 236 | 237 | 238 | ## Infrastructure Needed [optional] 239 | 240 | - A cluster which populates the `local-registry-hosting` ConfigMap 241 | 242 | ## Open Questions [optional] 243 | 244 | 1. How can we make `Build` objects with local image pull specs portable across namespaces? 245 | -------------------------------------------------------------------------------- /ships/0007-shipwright-website.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | --- 8 | title: shipwright-website 9 | authors: 10 | - "@adambkaplan" 11 | reviewers: 12 | - "@otaviof" 13 | - "@SaschaSchwarze0" 14 | approvers: 15 | - "@qu1queee" 16 | - "@sbose78" 17 | creation-date: 2020-09-14 18 | last-updated: 2021-13-09 19 | status: implemented 20 | replaces: 21 | - [shipwright-website](https://github.com/shipwright-io/build/blob/main/docs/proposals/shipwright-website.md) 22 | --- 23 | 24 | # Shipwright Website 25 | 26 | ## Release Signoff Checklist 27 | 28 | - [x] Enhancement is `implementable` 29 | - [x] Design details are appropriately documented from clear requirements 30 | - [ ] Test plan is defined 31 | - [ ] Graduation criteria for dev preview, tech preview, GA 32 | - [ ] User-facing documentation is created in [docs](/docs/) 33 | 34 | ## Summary 35 | 36 | Create a website to host documentation, release notes, news, and blog posts. 37 | 38 | ## Motivation 39 | 40 | The `build` repository is not sufficient to evangelize the project. Running a separate website with 41 | documentation, release notes, and how-tos is a preferable format for users to consume information. 42 | 43 | ### Goals 44 | 45 | - Lay out a framework for managing website content. 46 | - Establish a process for individuals to contribute content. 47 | 48 | ### Non-Goals 49 | 50 | - Create exhaustive documentation 51 | - Author blog posts/tutorials 52 | 53 | ## Proposal 54 | 55 | Shipwright can take advantage of [GitHub Pages](https://docs.github.com/en/github/working-with-github-pages/getting-started-with-github-pages) 56 | and the [Hugo templating engine](https://gohugo.io/) to generate a static website. Hugo is a 57 | popular framework to manage content, and is used by several communities (including upstream 58 | [Kubernetes](https://github.com/kubernetes/website)). The site will consist of two GitHub 59 | repositories: 60 | 61 | 1. A `website` repo with the Hugo assets, template, and content in Markdown format 62 | 2. The `shipwright-io.github.io` repository, our org's GitHub Pages repo. 63 | 64 | The `website` repository will contain the theme and the GitHub Pages repositories as submodules. 65 | A deployment script in `website` will update the site content on pull request merges. 66 | This can be automated via Travis CI. 67 | 68 | [Docsy](https://www.docsy.dev/) will be used as the baseline theme, which is optimized for software 69 | projects. This is the base theme used by upstream Kubernetes. 70 | 71 | ### Implementation Details/Notes/Constraints 72 | 73 | GitHub pages has support for [custom domains](https://docs.github.com/en/github/working-with-github-pages/configuring-a-custom-domain-for-your-github-pages-site). 74 | With appropriate `CNAME` and DNS records, the GitHub Pages site can be the host for `shipwright.io` 75 | content. GitHub Pages can also [enforce HTTPS](https://docs.github.com/en/github/working-with-github-pages/securing-your-github-pages-site-with-https). 76 | 77 | Since the website is content (not software), the 78 | [Creative Commons Attribution 4.0 International](https://creativecommons.org/licenses/by/4.0/legalcode) 79 | license is appropriate for the `website` repo and generated content on `shipwright.io`. The latter 80 | can identify licensing via a custom footer. 81 | 82 | The Docsy theme supports internationalization with appropriate configuration in place. We should 83 | use this to support content translations in the future. 84 | 85 | ### Risks and Mitigations 86 | 87 | **Risk**: Unsanctioned content is published. 88 | 89 | _Mitigation_: The main GitHub Pages site will enforce branch protection on the default branch. Only 90 | admins and sanctioned robot accounts will be allowed to merge to the default branch. 91 | 92 | ## Design Details 93 | 94 | The following sections are proposed for content: 95 | 96 | 1. Documentation (`/docs`) 97 | 2. Blog (`/blog`) 98 | 1. News (`/news`) 99 | 2. Release Notes (`/releases`) 100 | 101 | ## Implementation History 102 | 103 | - 2020-09-14: Proposal 104 | 105 | ## Drawbacks 106 | 107 | - Docs are separated from the repositories where code is written. 108 | - Release note content may be duplicated. 109 | 110 | ## Alternatives 111 | 112 | ### Host docs and blogs alongside code 113 | 114 | Only one repository has active development at present (`build`). Documentation can be hosted as 115 | Markdown in the existing `docs` folder. Release notes can be hosted directly on GitHub without an 116 | organization page. 117 | 118 | Blogs and tutorials can be hosted as Markdown in `docs`, or can be published on sites run by 119 | project maintainers. 120 | 121 | ## Infrastructure Needed 122 | 123 | 1. GitHub repositories (`website` and `shipwright-io.github.io`) 124 | 2. GitHub robot account to push changes from `website` onto `shipwright-io.github.io`. 125 | 3. Appropriate branch protection on `website` and `shipwright-io.github.io` 126 | 127 | ## Open Questions [optional] 128 | -------------------------------------------------------------------------------- /ships/0008-remote-artifacts.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | --- 8 | title: remote-artifacts 9 | authors: 10 | - @otaviof 11 | 12 | reviewers: 13 | - @SaschaSchwarze0 14 | - @qu1queee 15 | - @adambkaplan 16 | - @zhangtbj 17 | - @coreydaley 18 | 19 | approvers: 20 | - TBD 21 | creation-date: 2020-10-02 22 | last-updated: 2021-13-09 23 | status: provisional 24 | replaces: 25 | - [remote-artifacts](https://github.com/shipwright-io/build/blob/main/docs/proposals/remote-artifacts.md) 26 | --- 27 | 28 | Shipwright Remote Artifacts 29 | --------------------------- 30 | 31 | # Summary 32 | 33 | Remote artifacts are dependencies of the software building process, they represent binaries or 34 | other data stored outside of the version control system (Git). Hence, they are required when 35 | dealing with container images as well, and this enhancement proposal focuses on adding a remote 36 | artifacts support to Shipwright's Build API. 37 | 38 | # Motivation 39 | 40 | Give Shipwright's operator broader build use-case support by enhancing its capabilities to include 41 | the concept of Artifacts. In other words, remote entities that will be available for the image 42 | build process. 43 | 44 | End users will be able to include remote artifacts as a build runtime dependency, artifacts 45 | controlled by remote systems will be downloaded before the build process starts. Those artifacts may 46 | be a pre-compile binary, `jar` files, `war` files, etc. 47 | 48 | ## Goals 49 | 50 | * Provide means to declare remote artifacts, dependencies that can be employed on builds; 51 | * Create the mechanism to download and prepare remote artifacts for builds; 52 | 53 | ## Non-Goals 54 | 55 | * Automate the upload of local artifacts into the cluster; 56 | * Manage remote artifacts; 57 | * Amend container images created by the given `BuildStrategy`; 58 | * Walking a remote directory tree, similarly to a `git checkout`; 59 | 60 | # Proposal 61 | 62 | The enhancement proposal is centered around the idea of declaring external dependencies, here 63 | called "artifacts", and being able to use them during the container image building process. 64 | 65 | ## Build Artifacts 66 | 67 | The remote artifacts will be directly expressed in the body of a `Build` resource, as the 68 | following example: 69 | 70 | ```yml 71 | --- 72 | apiVersion: build.dev/v1alpha1 73 | kind: Build 74 | metadata: 75 | name: license-file 76 | spec: 77 | source: 78 | # ... 79 | sources: 80 | - name: license-file 81 | type: http 82 | url: https://licenses.company.com/customer-id/license.tar 83 | http: 84 | path: $(workspace)/licenses/license.tar 85 | ``` 86 | 87 | We will add `sources` section, side by side with current `source`. The idea is to accommodate both 88 | constructions in `v1alpha1`, and save API breaking changes for upcoming `v1beta1`. 89 | 90 | The new `sources` will contain the following attributes: 91 | 92 | - `name`: source name (required); 93 | - `type`: input source type, `git` or `http` (required); 94 | - `sourceRef`: use a external resource, the name in this field is the resource name (optional); 95 | - `url`: the URL of the repository or remote artifact (optional); 96 | - `credentials`: the credentials to interact with the artifact (optional); 97 | - `http`: settings for `http`, containing inner attribute `.spec.http.path` (optional); 98 | - `git`: settings for `git`, as exists today; 99 | 100 | The resource `UID`/`GID` will be defined by the same user running the Tekton task, later on, we 101 | can extend the API to support arbitrary configuration. 102 | 103 | ### Single vs. Multiple Git Repositories 104 | 105 | For the initial implementation, we will only support one git repository that is specified in 106 | `.spec.source`. Remote artifacts will be specified n `.spec.sources` only. 107 | 108 | We will address supporting multiple Git repositories in a future enhancement proposal. This will 109 | involve re-defining `/workspace/source` location and the `$workspace` placeholder. 110 | 111 | ## Steps and Helper 112 | 113 | Artifacts are requirements for the build process, thus downloading and preparing artifacts must 114 | happen before the build process starts. To achieve the objective, the operator will generate a new 115 | task step to download the artifacts. 116 | 117 | The download may happen using existing open-source software, more specifically `wget`. The 118 | container image which contains this software will be specified via environment variable, and when 119 | not informed will use a default value. 120 | 121 | Therefore, the `buildrun` controller needs to generate a `TaskRun` step, having a `wget` commands 122 | plus arguments, to download the specified artifacts. 123 | 124 | The step generated by the operator will look like the following: 125 | 126 | ```yml 127 | --- 128 | apiVersion: tekton.dev/v1beta1 129 | kind: TaskRun 130 | metadata: 131 | name: artifact-download 132 | spec: 133 | steps: 134 | - name: artifact-download 135 | image: busybox:latest 136 | command: 137 | - /bin/bash 138 | args: 139 | - -c 140 | - > 141 | wget --output="/workspace/source/classpath/main.jar" https://nexus.company.com/app/main.jar && \ 142 | chown 1000:1001 /workspace/source/classpath/main.jar && \ 143 | chmod 0644 /workspace/source/classpath/main.jar 144 | ``` 145 | 146 | ## Example Use-Case 147 | 148 | Using [Shipwright's proposed logos](https://github.com/shipwright-io/build/issues/325) as example, 149 | let's assume we are building a [TypeScript application](https://github.com/otaviof/typescript-ex) 150 | which will use the project logo, and we would like to create two different builds, one with the 151 | default project logo, and another with the alternative. 152 | 153 | By using remote artifacts, we can keep the separation of project source code and assets, and we 154 | can describe those resources as: 155 | 156 | ```yml 157 | --- 158 | apiVersion: build.dev/v1alpha1 159 | kind: BuildSource 160 | metadata: 161 | name: ship-logo 162 | spec: 163 | sources: 164 | - name: project-logo 165 | type: http 166 | url: https://user-images.githubusercontent.com/2587818/92114986-69bfb600-edfa-11ea-820e-96cdb1014f58.png 167 | http: 168 | path: $(workspace)/assets/images/shipwright-logo.png 169 | ``` 170 | 171 | And, the alternative logo: 172 | 173 | ```yml 174 | --- 175 | apiVersion: build.dev/v1alpha1 176 | kind: BuildSource 177 | metadata: 178 | name: axes-logo 179 | spec: 180 | sources: 181 | - name: project-logo 182 | type: http 183 | url: https://user-images.githubusercontent.com/2587818/92100668-c1ebbd80-ede4-11ea-9e8a-7379c3875ea0.png 184 | http: 185 | path: $(workspace)/assets/images/shipwright-logo.png 186 | ``` 187 | 188 | Then, we can create the `Build` resource, as per: 189 | 190 | ```yml 191 | --- 192 | apiVersion: build.dev/v1alpha1 193 | kind: Build 194 | metadata: 195 | name: typescript-ex 196 | spec: 197 | strategy: 198 | name: buildpacks-v3 199 | kind: ClusterBuildStrategy 200 | source: 201 | url: https://github.com/example/project.git 202 | sources: 203 | - name: source 204 | type: git 205 | url: https://github.com/otaviof/typescript-ex.git 206 | - name: project-logo 207 | sourceRef: ship-logo 208 | output: 209 | image: quay.io/otaviof/typescript-ex:latest 210 | ``` 211 | 212 | Now, we can create two `BuildRun` resources. The first only runs the build with original settings: 213 | 214 | 215 | ```yml 216 | --- 217 | apiVersion: build.dev/v1alpha1 218 | kind: BuildRun 219 | metadata: 220 | name: typescript-ex 221 | spec: 222 | buildRef: 223 | name: typescript-ex 224 | ``` 225 | 226 | And later, we can create yet another `BuildRun`, but this time use the alternative logo. Here we are 227 | overwriting the `project-logo` source name, with an alternative resource, i.e.: 228 | 229 | ```yml 230 | --- 231 | apiVersion: build.dev/v1alpha1 232 | kind: BuildRun 233 | metadata: 234 | name: typescript-ex-alternative-logo 235 | spec: 236 | buildRef: 237 | name: typescript-ex 238 | output: 239 | image: quay.io/otaviof/typescript-ex:alternative 240 | sources: 241 | - name: project-logo 242 | sourceRef: axes-logo 243 | ``` 244 | 245 | When the build processes are done, the following images will be available: 246 | * `quay.io/otaviof/typescript-ex:latest` 247 | * `quay.io/otaviof/typescript-ex:alternative` 248 | 249 | A number of real world use-cases can be derived from this example, the `BuildSource` is the 250 | foundation. 251 | 252 | ## Test Plan 253 | 254 | 1. Deploy the Shipwright Build operator in a cluster; 255 | 2. Create a `BuildSource` resource instance, point to a remote binary; 256 | 3. Create `Build` and `BuildRun` resources, using `BuildSource`; 257 | 4. Make sure the build process happens successfully, being able to use remote artifact; 258 | 259 | ## Alternatives 260 | 261 | ### Standalone CRD 262 | 263 | Alternatively, we may define the artifacts as a standalone CRD, that is a `BuildSource` resource. 264 | The advantage of this design is being able to exchange, and reuse, artifacts on several builds. 265 | For instance, if two projects are sharing a common logo image, both `Builds` will refer to the 266 | same `BuildSource`. 267 | 268 | Splitting up resources will imply on modifying `Build` and `BuildRun` resources, and include 269 | `.spec.sourceRef`. The new attribute will make possible linking the build with a `BuildSource` 270 | resource. 271 | 272 | The following snippet shows how an Artifact, i.e.: `BuildSource`, will be represented. 273 | 274 | ```yml 275 | --- 276 | apiVersion: build.dev/v1alpha1 277 | kind: BuildSource 278 | metadata: 279 | name: license-file 280 | spec: 281 | sources: 282 | - name: license.tar 283 | type: http 284 | url: https://licenses.company.com/customer-id/license.tar 285 | http: 286 | path: $(workspace)/licenses/license.tar 287 | ``` 288 | 289 | #### Usage 290 | 291 | The usage of this feature is based on declaring a slice of `.spec.sources` and later on overwriting 292 | entries using `BuildRun` resource. For instance: 293 | 294 | ```yml 295 | --- 296 | apiVersion: build.dev/v1alpha1 297 | kind: Build 298 | metadata: 299 | name: example 300 | spec: 301 | source: 302 | # ... 303 | sources: 304 | - name: license-file 305 | ``` 306 | 307 | Could have its `license-file` overwritten in a `BuildRun` with: 308 | 309 | ```yml 310 | --- 311 | apiVersion: build.dev/v1alpha1 312 | kind: BuildRun 313 | metadata: 314 | name: license-file 315 | spec: 316 | buildRef: 317 | name: example 318 | sources: 319 | - name: license-file 320 | sourceRef: alternative-file 321 | ``` 322 | -------------------------------------------------------------------------------- /ships/0009-cli.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | --- 8 | title: CLI 9 | authors: 10 | - "@otaviof" 11 | 12 | reviewers: 13 | - "@gabemontero" 14 | - "@HeavyWombat" 15 | - "@coreydaley" 16 | 17 | approvers: 18 | - TBD 19 | creation-date: 2020-10-07 20 | last-updated: 2021-13-09 21 | status: implemented 22 | replaces: 23 | - [cli](https://github.com/shipwright-io/build/blob/main/docs/proposals/cli.md) 24 | --- 25 | 26 | CLI 27 | --- 28 | 29 | # Summary 30 | 31 | This enhancement proposal describes a command-line client for Shipwright Build operator, covering 32 | from the command line structure, the usage as a kubectl plugin, and as a standalone binary, as 33 | well as testing this new project. 34 | 35 | # Motivation 36 | 37 | Command-line interface is a fundamental component for developer experience and modern automation. 38 | This enhancement proposal describes a command-line interface, i.e. "CLI", to enhance the developer 39 | experience and provide easier means to interact and automate Shipwright's Builds. 40 | 41 | ## Goals 42 | 43 | - Improve the user experience by providing a specialized client 44 | - Make possible extended automation with shell scripting 45 | 46 | ## Non-Goals 47 | 48 | - Re-implement operator logic 49 | - Become the mandatory API client 50 | 51 | # Proposal 52 | 53 | Create a command-line client for Shipwright Build to improve the user experience and also improve 54 | the ability to automate workflows. 55 | 56 | This new project will concentrate our effects on the API client side of the Shipwright project, 57 | and by maintaining the expected separation of `cmd` and `pkg`, developers can embed and extend the 58 | possibilities of consuming Shipwright Operator API. 59 | 60 | A [proof-of-concept command-line interface project](https://github.com/otaviof/shp) has been 61 | written to scaffold the features, and evaluate the changes proposed in this document. The project 62 | documentation (`README.md`) is focused on developers and contributors, while the features it will 63 | provide are contained in this document. 64 | 65 | ## Usage and Installation 66 | 67 | This command-line application will be consumed as a standalone binary, or as a plugin in `kubectl`. 68 | This implies a few more extra settings, and making sure the application binary complies with the 69 | [expected convention](https://krew.sigs.k8s.io/docs/developer-guide/develop/best-practices/). 70 | 71 | ### Standalone 72 | 73 | The standalone binary will be available via the usual means, final users will be able to simply 74 | install with `go get…`, or pick a pre-compiled binary from GitHub release pages. Consider the 75 | [release section](#Releasing) for more. 76 | 77 | ### Plugin 78 | 79 | As a [krew plugin](https://krew.sigs.k8s.io/docs/developer-guide/), the expected usage is: 80 | 81 | ```sh 82 | $ kubectl shp ... 83 | ``` 84 | 85 | And the following global flags are expected, to keep consistency with `kubectl`: 86 | 87 | - `-h/--help` 88 | - `-n/--namespace` 89 | - `-A/--all-namespaces` 90 | 91 | Those three flags are mandatory to keep compatibility. However, the 92 | [upstream kubectl package](https://pkg.go.dev/k8s.io/kubectl@v0.17.6/pkg/cmd/util) provides 93 | [more options](https://github.com/otaviof/shp/blob/55ce2e8d58435b0264e3db0bef5cf439abfeca18/vendor/k8s.io/cli-runtime/pkg/genericclioptions/config_flags.go#L317) 94 | out of the box. 95 | 96 | ## Command 97 | 98 | The project will use [cobra](https://github.com/spf13/cobra)/[viper](https://github.com/spf13/viper) 99 | for scaffolding the command-line structure, which in terms of organization and workflow, follows 100 | `kubectl` convention. Thus, the following pattern: 101 | 102 | ```sh 103 | $ shp [options] 104 | ``` 105 | 106 | For instance: 107 | 108 | ```sh 109 | $ shp create build nodejs-ex 110 | $ shp run build nodejs-ex 111 | ``` 112 | 113 | Alternatively, we may have a more conventional pattern, as in: 114 | 115 | ```sh 116 | $ shp [verb] [options] 117 | ``` 118 | 119 | For example: 120 | 121 | ```sh 122 | $ shp build create nodejs-ex 123 | $ shp build run nodejs-ex 124 | $ shp logs nodejs-ex 125 | ``` 126 | 127 | During the review process of this enhancement proposal, we must decide which pattern suits better, 128 | which one we prefer to go ahead with. 129 | 130 | ## Features 131 | 132 | For the initial release, the aim is to be able to define a `Build`, run it and display logs. 133 | 134 | ### Defining a Build 135 | 136 | ```sh 137 | $ shp build [options] 138 | ``` 139 | 140 | Will be responsible for managing Build objects, using the verbs: `create`, `update`, list and 141 | `delete`. Later on, end users should have the ability to bootstrap a local repository clone using a 142 | single command. 143 | 144 | ### Running a Build 145 | 146 | ```sh 147 | $ shp run build [options] 148 | ``` 149 | 150 | This sub-command will instantiate a new BuildRun resource for a Build, effectively starting the 151 | build process. And, will print follow up commands that may be issued to see the BuildRun status 152 | and inspect logs. 153 | 154 | ### Logs 155 | 156 | ```sh 157 | $ shp logs [options] 158 | ``` 159 | 160 | Retrieve all logs related to the informed `BuildRun` name. It will also be able to follow 161 | (`--follow`) container log output as they are executed. The log lines displayed are organised by 162 | the sequence of steps, and easy to read the whole build process output in a single go. 163 | 164 | Additionally, it must provide easy syntax to reach a specific `BuildRun` instance, or the most 165 | recent `BuildRun` for a `Build`. 166 | 167 | ## More Features 168 | 169 | The command-line interface is planned to host more features, like for instance managing local and 170 | remove artifacts, as well as help end-users to upload data into the cluster. 171 | 172 | Thus, the initial design of the CLI must allow more subcommands to be added, accommodating more 173 | features which will depend in a client. 174 | 175 | ## Testing 176 | 177 | The strategy to test the command-line application will be based on `go test` for the unit testing, 178 | and for end-to-end testing we should adopt [bats](https://github.com/sstephenson/bats). Bats 179 | framework gives us a structured way to run shell script commands and what we expect them to 180 | return, the command-line client shp is no more than a shell command. 181 | 182 | The testing structure will be composed of: 183 | 184 | - **Unit**: written in Golang and using [Gomega](https://onsi.github.io/gomega/) for assertion; 185 | - **End-to-End**: written in Golang and located at the traditional `test/e2e` directory; 186 | - **System**: written using Bats framework (Bash), or similar approach; 187 | 188 | Bats will also be helpful for a future `shp` container-image, we are able to mount the Bats files 189 | in the container-image produced, and run our system testing against it. Therefore, Bats covers 190 | testing of a local command-line binary, as well as it does a container-image. Another tool may take 191 | Bats' place, covering the same test-use cases accordingly. 192 | 193 | ## Releasing 194 | 195 | The command-line release will initially be available on GitHub Pages process, on which we can also 196 | index on Shipwright website. Later on we can evaluate the need for a container-image carrying on 197 | the binary, or an RPM. 198 | -------------------------------------------------------------------------------- /ships/0010-buildstrategy-annotation-propagation.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | --- 8 | title: propagating-annotations-from-the-build-strategy-to-the-pod 9 | 10 | authors: 11 | - "@SaschaSchwarze0" 12 | 13 | reviewers: 14 | - "@zhangtbj" 15 | 16 | approvers: 17 | - "@adambkaplan" 18 | - "@qu1queee" 19 | 20 | creation-date: 2020-12-16 21 | last-updated: 2021-13-09 22 | status: implemented 23 | replaces: 24 | - [buildstrategy-annotation-propagation](https://github.com/shipwright-io/build/blob/main/docs/proposals/buildstrategy-annotation-propagation.md) 25 | --- 26 | 27 | # Propagating annotations from the build strategy to the pod 28 | 29 | ## Release Signoff Checklist 30 | 31 | - [X] Enhancement is `implementable` 32 | - [ ] Design details are appropriately documented from clear requirements 33 | - [X] Test plan is defined 34 | - [ ] Graduation criteria for dev preview, tech preview, GA 35 | - [ ] User-facing documentation is created in [docs](/docs/) 36 | 37 | ## Open Questions [optional] 38 | 39 | None. 40 | 41 | ## Summary 42 | 43 | A build strategy administrator MUST only define very few details of how the `BuildRun`'s `Pod` looks at the end. This is intended and follows the principle from [The BuildStrategy API](buildstrategy.md): 44 | 45 | > A slim BuildStrategy is one where the BuildStrategy author gets to accomplish more by specifying less. 46 | 47 | But, beside the steps that are to be performed, the build strategy administrator already CAN define certain runtime behavior aspects like container resources and security context. 48 | 49 | By enabling the administrator to also control the annotations of the `BuildRun` `Pod` IF necessary, more scenarios CAN be supported. 50 | 51 | ## Motivation 52 | 53 | The runtime behavior of a `Pod` is not only specified in the containers. There are use cases where behavior of `Pod`s is described through annotations. This is especially the case for alpha and beta features of Kubernetes. Examples: 54 | 55 | - The Kubernetes [Network Traffic Shaping](https://kubernetes.io/docs/concepts/extend-kubernetes/compute-storage-net/network-plugins/#support-traffic-shaping) feature looks for the `kubernetes.io/ingress-bandwidth` and `kubernetes.io/egress-bandwidth` annotations to limit the network bandwidth the `Pod` is allowed to use. 56 | - The [SecComp profile selection](https://kubernetes.io/docs/tutorials/clusters/seccomp/#create-a-pod-with-a-seccomp-profile-for-syscall-auditing) used to be done through the `seccomp.security.alpha.kubernetes.io/pod` annotation until Kubernetes 1.18 - 1.19 makes it a first-class property in the `Pod`'s security context. 57 | - The [AppArmor profile of a container](https://kubernetes.io/docs/tutorials/clusters/apparmor/) is defined using the `container.apparmor.security.beta.kubernetes.io/` annotation. 58 | 59 | To use those features for `BuildRun` `Pod`s, control over the `Pod`'s annotations is necessary. As those features are clearly something that administrators should define rather then the users that define `Build`s and `BuildRun`s, it makes sense to restrict this feature to the `BuildStrategy` and `ClusterBuildStrategy` only. 60 | 61 | ### Goals 62 | 63 | - Enable the build strategy administrator to define annotations that are copied to the `BuildRun`'s `Pod`. 64 | 65 | ### Non-Goals 66 | 67 | - Enable users to define annotations on `Build` and `BuildRun` that are copied to the `BuildRun`'s `Pod`. 68 | - Enable anybody to define labels on one of our custom resources and make them appear on the `BuildRun`'s `Pod`. This should be covered separately. 69 | 70 | ## Proposal 71 | 72 | In the `BuildStrategy` and `ClusterBuildStrategy`, the build strategy administrator can define annotations in the metadata. This is possible in the same way as for all Kubernetes objects, see the [Annotations](https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/) topic in the Kubernetes documentation. Annotations are key/value pairs. The key consists of up to two parts: an optional prefix and a name. If both are defined, the `/` is used as separators. The prefix must be a DNS subdomain. Kubernetes reserves the `kubernetes.io` and `k8s.io` prefixes for its own core components. In build we already use annotations in the build controller using the `build.build.dev` prefix for two use cases: 73 | 74 | - The `build.build.dev/build-run-deletion` annotation on builds controls whether its `BuildRun`s are deleted when the build is deleted. 75 | - The `build.build.dev/referenced.secret` annotation tells the build controller that a secret is related to builds. 76 | 77 | Based on that naming, the other three prefixes reserved by our controllers are: `buildstrategy.build.dev`, `clusterbuildstrategy.build.dev` and `buildrun.build.dev`. 78 | 79 | When generating a Tekton `TaskRun`, the idea is to look at the annotations of the `BuildStrategy` or `ClusterBuildStrategy` and copy all annotations over to the `TaskRun`, except those that use one of our own prefixes because we today have no feature anyway where we look for one of our annotations on a `TaskRun` or `Pod`, and except the `kubectl.kubernetes.io/last-applied-configuration` annotation. 80 | 81 | Tekton automatically copies all `TaskRun` annotations to the `Pod`, see [pod.go](https://github.com/tektoncd/pipeline/blob/v0.21.0/pkg/pod/pod.go#L258). 82 | 83 | For example, this metadata of a cluster build strategy: 84 | 85 | ```yaml 86 | apiVersion: build.dev/v1alpha1 87 | kind: ClusterBuildStrategy 88 | metadata: 89 | annotations: 90 | kubernetes.io/egress-bandwidth: 100M 91 | clusterbuildstrategy.build.dev/dummy: aValue 92 | ``` 93 | 94 | will lead to the following metadata on the `TaskRun` (and `Pod`): 95 | 96 | ```yaml 97 | apiVersion: tekton.dev/v1beta1 98 | kind: TaskRun 99 | metadata: 100 | annotations: 101 | kubernetes.io/egress-bandwidth: 100M 102 | ``` 103 | 104 | ### Implementation Details/Notes/Constraints [optional] 105 | 106 | The implementation requires the [BuilderStrategy interface](../../pkg/apis/build/v1alpha1/buildstrategy.go) to be extended with a `GetAnnotations` functions that is implemented in the [BuildStrategy](../../pkg/apis/build/v1alpha1/buildstrategy_types.go) and [ClusterBuildStrategy](../../pkg/apis/build/v1alpha1/clusterbuildstrategy_types.go) types by returning the object's annotations. 107 | 108 | The assignment of the `TaskRun` annotations needs to be done in the [generate_taskrun.go](../../pkg/reconciler/buildrun/resources/taskrun.go) file in the `GenerateTaskRun` function. The annotations from the build strategy need to be copied to the `TaskRun` except those using one of the four Shipwright Build owned prefixes mentioned under [Proposal](#proposal), and except the `kubectl.kubernetes.io/last-applied-configuration` annotation. 109 | 110 | ### Risks and Mitigations 111 | 112 | A risk is that the build strategy administrator starts to use an annotation-controlled feature that the Kubernetes administrator does not want to be used. Third-party policy engines like [Open Policy Agent](https://www.openpolicyagent.org/) can be used by the Kubernetes administrator to prevent this without requiring anything from our operator - an [EP in Tekton](https://github.com/tektoncd/community/blob/main/teps/0035-document-tekton-position-around-policy-authentication-authorization.md#proposal) is suggesting the same. If this is considered not enough, then option (2) from the [alternatives](#alternatives) might be required. 113 | 114 | ## Design Details 115 | 116 | ### Test Plan 117 | 118 | - The unit testing for the `TaskRun` generation must be extended. 119 | - An integration test must be added to verify that annotations are copied over selectively from the `BuildStrategy` and `ClusterBuildStrategy` to the `TaskRun`. 120 | 121 | ### Upgrade / Downgrade Strategy 122 | 123 | There is a behavior change that annotations on the `BuildStrategy` or `ClusterBuildStrategy` that a build strategy administrator has defined for whatever reason are now copied over to the `TaskRun` and `Pod`. These are either annotations without a behavioral change to the `Pod`, or annotations that the user already expected to be copied over which makes this proposal a fix for his scenario. 124 | 125 | ### Version Skew Strategy 126 | 127 | N/A 128 | 129 | ## Implementation History 130 | 131 | N/A 132 | 133 | ## Drawbacks 134 | 135 | None 136 | 137 | ## Alternatives 138 | 139 | (1) Instead of copying over the annotations from the `BuildStrategy` or `ClusterBuildStrategy` metadata, one could follow a similar pattern as the Kubernetes deployment with its PodTemplate where the annotations of the deployment are separated from the designated annotations for the `Pod`s created through the deployment (example: [here](https://github.com/kubernetes/kubernetes/issues/37666#issuecomment-283109237)). Translated into our use case, this would mean that the annotations for the `TaskRun` and `Pod` are then explicitly listed in the spec of the `BuildStrategy` or `ClusterBuildStrategy` rather than in the metadata: 140 | 141 | ```yaml 142 | apiVersion: build.dev/v1alpha1 143 | kind: ClusterBuildStrategy 144 | metadata: 145 | name: a-cbs 146 | annotations: 147 | clusterbuildstrategy.build.dev/dummy: aValue 148 | spec: 149 | podAnnotations: 150 | kubernetes.io/egress-bandwidth: 100M 151 | ``` 152 | 153 | This idea was not considered because it is unnecessary. The described filtering already eliminates annotations that are not of interest for the `Pod`. 154 | 155 | (2) Instead of filtering out a hard-coded list of annotations by prefix (our Shipwright prefixes) or by full key (`kubectl.kubernetes.io/last-applied-configuration`), one could have introduced an extension to our [configuration](../../pkg/config/config.go) to allow the administrator of the build operator to configure which annotations are copied from the `BuildStrategy` and `ClusterBuildStrategy` to the `TaskRun`, using white- or black-listing. 156 | 157 | This idea was not considered because we were not seeing a relevant use case for it. 158 | -------------------------------------------------------------------------------- /ships/0012-webhook-validation.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | --- 8 | title: webhook-validation 9 | authors: 10 | - "@ImJasonH" 11 | reviewers: 12 | - "@gmontero" 13 | - "@zhangtbj" 14 | approvers: 15 | - "@qu1queee" 16 | - "@adamkaplan" 17 | creation-date: 2020-03-19 18 | last-updated: 2021-13-09 19 | status: provisional 20 | replaces: 21 | - [webhook-validation](https://github.com/shipwright-io/build/blob/main/docs/proposals/webhook-validation.md) 22 | --- 23 | 24 | # Webhook Validation 25 | 26 | ## Release Signoff Checklist 27 | 28 | - [ ] Enhancement is `implementable` 29 | - [ ] Design details are appropriately documented from clear requirements 30 | - [ ] Test plan is defined 31 | - [ ] Graduation criteria for dev preview, tech preview, GA 32 | - [ ] User-facing documentation is created in [docs](/docs/) 33 | 34 | ## Open Questions [optional] 35 | 36 | Do we want to require that operators run another job to synchronously validate client requests? 37 | 38 | Do we want to rely on Knative's webhook packages, or write our own code to manage certs ourselves? 39 | Is there anything already out there in controller-gen we should be using? 40 | 41 | ## Summary 42 | 43 | Kubernetes provides facilities to inject synchronous validation of CRUD operations from clients for specified types, such as Shipwright's CRD types. 44 | 45 | Webhooks can also mutate incoming requests, including setting default values and converting between versions of CRDs. These features should be considered later, and are not described in this proposal. 46 | 47 | We should take advantage of these features and include a webhook validation deployment in the Shipwright installation to provide this functionality. 48 | 49 | ## Motivation 50 | 51 | Today, when a user creates a resource (e.g., a Build, BuildRun, etc.), they are immediately stored in the cluster's etcd storage, and the Shipwright controller begins reconciling the object. 52 | 53 | If the controller finds that the resource is invalid, it updates the resource's `.status` to indicate this state and stops reconciling it. 54 | 55 | Examples of invalid resources include: 56 | 57 | - a BuildRun referencing a Build that doesn't exist 58 | - a BuildRun referencing a ServiceAccount that doesn't exist 59 | - a Build referencing a (Cluster)BuildStrategy that doesn't exist 60 | - a BuildStrategy containing zero steps 61 | 62 | (This is a partial list, and new validation can be assumed to be added over time as the Shipwright API evolves.) 63 | 64 | By allowing invalid resources to be accepted and stored, we require clients to watch or poll all resources to ensure their request was valid, and we add to the API surface of resources that otherwise would not need to report a status, like a Build. 65 | 66 | Always accepting and storing resources, even invalid ones, can also lead to additional storage of known-invalid objects, which clutter the user experience and can lead to confusion, and in extreme cases, cluster resource exhaustion. 67 | 68 | Mixing reconciliation and validation code in the controller also complicates our codebase, leading to PRs like https://github.com/shipwright-io/build/pull/641 which add new validation-centric statuses that the user must watch for. 69 | As we add new validation scenarios, we currently add new `Reason` strings that clients must be aware of, which is cumbersome and won't scale indefinitely. 70 | 71 | ### Goals 72 | 73 | Enforce synchronous validation via a new Deployment and a [ValidatingAdmissionWebhook](https://kubernetes.io/docs/reference/access-authn-authz/admission-controllers/#validatingadmissionwebhook), and remove validation-centric logic from the reconciler. 74 | 75 | The reconciler should be able to assume in many cases that the resource it is reconciling has already been validated. 76 | 77 | ### Non-Goals 78 | 79 | - Include a mutating or defaulting or conversion webhook. 80 | If we decide these are useful, we can add them in future proposals. 81 | 82 | ## Proposal 83 | 84 | Rely on [kubebuilder's webhook support](https://book.kubebuilder.io/cronjob-tutorial/webhook-implementation.html) to add a new webhook validation controller, and include it in Shipwright's default installation as a K8s Deployment. 85 | 86 | 87 | ### Implementation Details/Notes/Constraints [optional] 88 | 89 | #### Certificates 90 | 91 | The webhook deployment will also need to manage certificates since K8s webhook requests are sent over HTTPS. 92 | Setting up these certificates securely can be cumbersome. 93 | 94 | The [Knative](https://knative.dev) project has [some libraries to make this management easier](https://github.com/knative/pkg/tree/main/webhook) by taking care of cert generation, registration and refreshing, but mixing Knative-style components and existing `controller-gen`/KubeBuilder-style components might make complicate development. 95 | 96 | #### Webhook Availability 97 | 98 | Once registered, if the K8s API server can't reach the defined webhook service, _all_ requests will fail for registered CRD types. 99 | The webhook deployment is stateless, so it can be easily replicated and scheduled across multiple nodes to ensure availability in the case of node failure. 100 | 101 | ### Risks and Mitigations 102 | 103 | Clients that expect to have requests accepted and asynchronously validated might be surprised to find the API synchronously rejecting their invalid requests instead. 104 | We should communicate this change to users and operators in release notes and in mailing list announcements, and be available to answer any questions. 105 | 106 | ## Design Details 107 | 108 | ### Test Plan 109 | 110 | Our current tests to validate resources should be mostly reusable; validation logic itself isn't going anywhere, just moving. 111 | 112 | Any tests that we have that assume an invalid resource will be accepted and eventually reconciled should be updated to expect immediate failure. 113 | 114 | ### Version Skew Strategy 115 | 116 | A user who has previously installed a version of Shipwright that did not synchronously validate might have invalid resources in their etcd storage, and our controller may be asked to reconcile them. 117 | Basic validation logic should remain in the reconciler, but either update a status to indicate invalidity, or merely log and ignore invalid resources. 118 | 119 | During installation or upgrade of Shipwright components, the webhook job and the reconciling controller job might briefly run two versions of validation code, so we should be careful to consider this as we add (or remove, or refactor) validation support over time. 120 | In practice, Tekton hasn't experienced significant issues with this, but I wanted to call it out because it's possible. 121 | 122 | ## Implementation History 123 | 124 | Major milestones in the life cycle of a proposal should be tracked in `Implementation History`. 125 | 126 | ## Drawbacks 127 | 128 | This requires us to build a new component in the Shipwright installation, and for the operator component to operate it. 129 | 130 | ## Alternatives 131 | 132 | Continue to validate asynchronously, which probably requires investing more in scaling the addition and documentation of validation failures. 133 | -------------------------------------------------------------------------------- /ships/0013-modifying-output-image.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | --- 8 | title: modifying-output-image 9 | authors: 10 | - "@ImJasonH" 11 | reviewers: 12 | - TBD 13 | approvers: 14 | - TBD 15 | creation-date: 2021-04-14 16 | last-updated: 2021-13-09 17 | status: provisional 18 | replaces: 19 | - [modifying-output-image.md](https://github.com/shipwright-io/build/blob/main/docs/proposals/modifying-output-image.md) 20 | --- 21 | 22 | # Modifying Output Image 23 | 24 | ## Release Signoff Checklist 25 | 26 | - [ ] Enhancement is `implementable` 27 | - [X] Design details are appropriately documented from clear requirements 28 | - [X] Test plan is defined 29 | - [X] Graduation criteria for dev preview, tech preview, GA 30 | - [ ] User-facing documentation is created in [docs](/docs/) 31 | 32 | ## Open Questions 33 | 34 | 1. Should this feature also allow users to _unset_ existing labels/annotations inherited from base images? 35 | 36 | ## Summary 37 | 38 | Allow Build authors to describe modifications they'd like to make to output images. 39 | For this proposal, we'll scope this down to staticly-defined image [labels](https://docs.docker.com/engine/reference/builder/#label) and [OCI annotations](https://github.com/opencontainers/image-spec/blob/master/annotations.md). 40 | 41 | Future proposals might allow Build authors to configure image tags, and/or to allow authors to configure dynamic values based on input values. 42 | These are non-goals of this proposal. 43 | 44 | ## Motivation 45 | 46 | ### Goals 47 | 48 | - Begin to enable simple modifications of the output image by Build authors. 49 | 50 | ### Non-Goals 51 | 52 | - Enable dynamic values or placeholders/templating (e.g., git commit SHA) 53 | - Automatically inject labels or annotations without user input 54 | - Enable modification of image layer contents (see [runtime image](https://github.com/shipwright-io/build/blob/master/docs/proposals/runtime-image.md)) 55 | 56 | ## Proposal 57 | 58 | Add new fields to describe output image labels and annotations: 59 | 60 | ```yaml= 61 | apiVersion: shipwright.io/v1alpha1 62 | kind: Build 63 | metadata: 64 | name: my-build-name 65 | spec: 66 | output: 67 | image: quay.io/my/image 68 | credentials: 69 | name: my-push-creds 70 | # New fields 71 | labels: 72 | maintainer: team@my-company.com 73 | description: "This is my cool image" 74 | annotations: 75 | org.opencontainers.image.url: https://my-company.com/images 76 | org.opencontainers.image.source: https://github.com/org/repo 77 | ``` 78 | 79 | This will be enabled by embedding the existing `Image` struct in a new struct that 80 | 81 | ```go= 82 | type BuildSpec struct { 83 | ... 84 | Output OutputImage `json:"output"` 85 | } 86 | 87 | type OutputImage struct { 88 | Image // embedding existing type 89 | 90 | // New fields 91 | Labels map[string]string `json:"labels,omitempty"` 92 | Annotations map[string]string `json:"annotations,omitempty"` 93 | } 94 | ``` 95 | 96 | ### Implementation Details/Notes/Constraints 97 | 98 | When a BuildRun executes for a Build that specifies `labels` or `annotations`, the BuildRun controller will append a new step to the end of the Tekton TaskRun that augments the image with the specified labels and annotations, and pushes to the same tag. 99 | 100 | This could be accomplished using a containerized CLI tool like [`crane`](https://github.com/google/go-containerregistry/blob/main/cmd/crane), or a purpose-built binary that Shipwright provides and packages in our releases. 101 | 102 | In either case, the TaskRun produced by the BuildRun controller would include a step along the lines of: 103 | 104 | ```yaml 105 | - name: modify-output-image 106 | image: gcr.io/go-containerregistry/crane # TODO: pin this to a digest, configurable by Shipwright operators 107 | script: | 108 | crane mutate $(params.output-image) \ 109 | --add-label maintaner="team@my-company.com" \ 110 | --add-label description="This is my cool image" \ 111 | --add-annotation org.opencontainers.image.url=https://my-company.com/images \ 112 | --add-annotation org.opencontainers.image.source=https://github.com/org/repo \ 113 | ``` 114 | 115 | _(This is intended as an illustration)_ 116 | 117 | ### Risks and Mitigations 118 | 119 | **Risk:** Automation that watches for image changes by tag will see two pushes as a result of the implementation above; one for the original push done by the specified BuildStrategy, and another immediately after when the next step modifies the image to set labels/annotations. 120 | 121 | **Mitigation:** We can document this behavior as expected. 122 | 123 | **Future Improvements:** 124 | As a future improvement, we can make the Build's specified labels and annotations available as parameters to BuildStrategy authors, so they can pass labels/annotations to their build process directly. 125 | In this case the appended step will perform a no-op push, which will not trigger a subsequent image change notification. 126 | 127 | Another possible future improvement would be to support and document a local filesystem path that build strategies can write image data to instead of pushing it to the remote registry. 128 | If the BuildRun sees image data at that location, it could perform some modifications before pushing the image itself. 129 | This would require support from build tooling to support writing to a local file path instead of pushing, and for build strategy authors to take advantage of this support. 130 | 131 | ## Design Details 132 | 133 | ### Test Plan 134 | 135 | And e2e test case will check that a Build that specifies labels and annotations, when run, produces an image with those values set. 136 | 137 | ## Drawbacks 138 | 139 | It's not ideal that the implementation produces two image pushes. 140 | 141 | ## Alternatives 142 | 143 | We could jump directly to the first **Future Improvement** listed above, and pass a Build's specified labels and annotations to the BuildStrategy as parameters. 144 | In this case we could also just expose image labels and annotations as optional BuildStrategy parameters only for those BuildStrategies that support them. 145 | This depends on mature support for BuildStrategy parameterization (https://github.com/shipwright-io/build/pull/697). 146 | 147 | ## Infrastructure Needed 148 | 149 | None. 150 | -------------------------------------------------------------------------------- /ships/0017-deprecate-runtime.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | --- 8 | title: deprecate-runtime 9 | authors: 10 | - "@imjasonh" 11 | reviewers: 12 | - "@adambkaplan" 13 | - "@qu1queee" 14 | approvers: 15 | - "@adambkaplan" 16 | - "@qu1queee" 17 | creation-date: 2021-05-26 18 | last-updated: 2021-05-26 19 | status: implementable 20 | replaces: 21 | - "https://github.com/shipwright-io/build/blob/main/docs/proposals/runtime-image.md" 22 | --- 23 | 24 | # Deprecate `runtime` 25 | 26 | ## Release Signoff Checklist 27 | 28 | - [x] Enhancement is `implementable` 29 | - [x] Design details are appropriately documented from clear requirements 30 | - [x] Test plan is defined 31 | - [x] Graduation criteria for dev preview, tech preview, GA 32 | - [ ] User-facing documentation is created in [docs](/docs/) 33 | 34 | ## Open Questions [optional] 35 | 36 | What features of `runtime` do we expect future users to want, and how should we deliver them in a manner that's consistent with the future direction of the API? 37 | 38 | ## Summary 39 | 40 | ### Background 41 | 42 | The `.spec.runtime` field allows users to describe operations that Shipwright should take on images, after they're built by the defined `.spec.strategy`. 43 | 44 | These include: 45 | 46 | - swapping out the base image layers 47 | - changing the image's working directory, environment variables, labels, user, entrypoint 48 | - injecting file paths 49 | - running arbitrary commands in the container and reflecting any modifications in the output image 50 | 51 | These operations are implemented by adding a step after the strategy's steps that executes a generated Dockerfile based on the built image, with `RUN`, `USER`, `ENTRYPOINT`, etc., commands expressed. 52 | This Dockerfile is executed with `kaniko`, which also pushes the image, replacing the originally built image in the registry. 53 | 54 | ### Proposal 55 | 56 | This proposal suggests marking this functionality as deprecated, and removing the feature in a future release. 57 | 58 | ## Motivation 59 | 60 | The original motivation for `runtime` seems to have been to simulate [multi-stage docker builds](https://docs.docker.com/develop/develop-images/multistage-build/) to produce leaner output images. 61 | Using this approach, a build process requiring build tools (e.g., a JDK, `go`) can be modified so that development tools are stripped from the resulting image, leaving only the built, runnable artifact. 62 | 63 | This is a noble goal, but since multi-stage docker builds have been available since Docker 17.05, and image build tools like Buildpacks already produce lean images by default, this feature is non-standard. 64 | 65 | Furthermore, allowing Build authors to modify the image, and inject files and commands in the container environment, using a generated ephemeral Dockerfile that's not persisted after the build execution, presents an opportunity for confusing behavior at best, and supply chain attacks at worst. 66 | 67 | Since the feature is implemented by pushing a new image with the same tag as the image built and pushed by the build strategy steps, clients observing registry changes will be able to observe two tag pushes -- this might lead to downstream automated systems deploying or attempting to deploy the first, unmodified image, leading to confusing or insecure runtime behavior. 68 | 69 | ### Goals 70 | 71 | Deter new users from adopting `runtime`, until we are comfortable removing the feature entirely. 72 | 73 | ### Non-Goals 74 | 75 | Replace every aspect of the feature with new Shipwright-level features. 76 | 77 | - Some features should be the responsibility of the build tool (e.g., multi-stage docker builds, Buildpacks), and Shipwright should encourage and document them. 78 | - Some features might end up becoming other more focused Shipwright-level features (e.g., rebasing an image) 79 | 80 | In any case, these additions are out of scope for this proposal. 81 | 82 | ## Proposal 83 | 84 | Announce deprecation and set a timeline for removing the feature in a future release. 85 | As an initial proposal, I would suggest two releases from now -- at time of writing, that would be v0.7.0, scheduled to be released in ~3 months. 86 | 87 | ### Implementation Notes 88 | 89 | - Document that the feature is Deprecated in [user-facing documentation](https://github.com/shipwright-io/build/blob/main/docs/build.md#Runtime-Image). 90 | - Mark the `.spec.runtime` field as `// Deprecated` in Go source. 91 | - Notify `shipwright-users@` of the deprecation plan and timeline. 92 | 93 | If we [adopt admission controllers](https://github.com/shipwright-io/build/blob/main/docs/proposals/webhook-validation.md), we can [set a warning in the response](https://kubernetes.io/blog/2020/09/03/warnings/#admission-webhooks) that would be shown to `kubectl` users and potentially other clients, when a `Build` request specifies the `.spec.runtime` field. 94 | I'm not sure if admission controllers will land in time to be useful in this case. 95 | 96 | ### Test Plan 97 | 98 | While deprecated, existing tests should continue to guard against regressions. 99 | 100 | When the feature is removed, tests covering it can also be removed, and existing tests should ensure no other feature regressions. 101 | 102 | ### Release Criteria 103 | 104 | #### Removing a deprecated feature 105 | 106 | - Announce deprecation and support policy of the existing feature 107 | - Deprecate the feature 108 | 109 | ### Risks and Mitigations 110 | 111 | At this time we don't believe the feature has significant usage, but due to the nature of installations, there's no way to be sure. 112 | While the feature is marked as deprecated, we might find a user significantly depends on it, and we should be open to considering un-deprecating it, or modifying our removal timeline/process as necessary. 113 | 114 | ## Drawbacks 115 | 116 | We are deprecating and removing a feature without a direct replacement. 117 | 118 | ## Alternatives 119 | 120 | We could modify how the feature is implemented to close potential security holes, and better document and warn against any holes that remain. 121 | Ultimately, we believe the feature isn't widely used, and the functionality is better served by existing build tool features, so this investment (and opportunity cost) would need to be well motivated. 122 | 123 | ## Infrastructure Needed [optional] 124 | 125 | None 126 | -------------------------------------------------------------------------------- /ships/0018-build-env-vars.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | --- 8 | title: build-env-vars 9 | authors: 10 | - "@adambkaplan" 11 | reviewers: 12 | - "@gabemontero" 13 | - "@ImJasonH" 14 | approvers: 15 | - "qu1queee" 16 | - "@sbose78" 17 | creation-date: 2021-06-03 18 | last-updated: 2021-06-29 19 | status: implementable 20 | see-also: [] 21 | replaces: 22 | - https://github.com/shipwright-io/build/pull/726 23 | superseded-by: [] 24 | --- 25 | 26 | # Build Environment Variables 27 | 28 | ## Release Signoff Checklist 29 | 30 | - [x] Enhancement is `implementable` 31 | - [x] Design details are appropriately documented from clear requirements 32 | - [ ] Test plan is defined 33 | - [ ] Graduation criteria for dev preview, tech preview, GA 34 | - [ ] User-facing documentation is created in [docs](/docs/) 35 | 36 | ## Open Questions [optional] 37 | 38 | 1. How can we validate against the environment variables present in the base image of the build? 39 | We're considering this out of scope - this would be very difficult to implement in practice. 40 | 41 | ## Summary 42 | 43 | This proposal will give developers the abilty to add environment variables to build strategy steps. 44 | Conversely, this proposal will also give build strategy authors the ability to fix the values of environment variables in build steps or provide non-empty default values. 45 | 46 | ## Motivation 47 | 48 | Developers using Shipwright may need to alter the behavior of a build by overriding or augmenting the environment variables. 49 | These may not be known ahead of time by the build strategy author, and hence not exposed via build parameters. 50 | For example, source-to-image and Cloud Native Buildpacks rely on environment variables to tune the behavior of their supported language/framework runtimes. 51 | The set of environment variables to support is effectively unbounded, as each underlying runtime has its own tooling (`npm`, `mvn`, etc.) that can read dozens of environment variables at build time. 52 | 53 | At the same time, build stategy authors may want to control which environment variables can be altered by the end developer. 54 | Many tools use environment variables to override basic security settings, like TLS verification, that are otherwise enabled by default. 55 | A build strategy author may not want developers to change these settings, or would perfer to provide default values that adhere to best practices. 56 | 57 | ### Goals 58 | 59 | * Allow developers to set environment variables used in a build step. 60 | * Provide a means for build strategy authors to keep fixed values for environment variables. 61 | 62 | ### Non-Goals 63 | 64 | * Allow developers to alter or override other aspects of the build step, such as commands. 65 | * Set environment variables in the output image of the build. 66 | * Validate environment variables against those in the base image of a build step. 67 | * Provide direct support for Dockerfile [build arguments](https://docs.docker.com/engine/reference/commandline/build/#set-build-time-variables---build-arg). 68 | 69 | ## Proposal 70 | 71 | ### User Stories 72 | 73 | #### Story: Alter build behavior via environment variables 74 | 75 | As a nodejs developer using buildpacks 76 | I want to set the `NPM_INCLUDE` environment variable in my Build and BuildRuns 77 | So that I can include development and debug resources in my container image 78 | 79 | #### Story: Prevent insecure behavior in a build strategy 80 | 81 | As a build strategy author 82 | I want to keep fixed values for some environment variables in my build strategy 83 | So that I don't allow end users to enable insecure or potentially harmful features. 84 | 85 | #### Story: Provide credentials via environment variables 86 | 87 | As a Java developer using maven to build my application (via buildpacks or source-to-image) 88 | I want to use environment variables to pass usernames and passwords to values defined in my `pom.xml` file 89 | So that I don't store sensitive information in source control. 90 | 91 | ### Implementation Notes 92 | 93 | #### Build Strategy Behavior 94 | 95 | Build strategies which declare an environment variable in a build strategy step will by default be fixed. 96 | A developer will not be able to change this value directly. 97 | 98 | If a build strategy author wants to allow an environment variable to be tuned, they have two options: 99 | 100 | 1. Omit the environment variable - this works if an empty/zero default value is acceptable in the build step. 101 | 2. Declare the environment variable, but use a build strategy parameter to set the value. 102 | This is useful if the build strategy should use a nonempty/nonzero default value in the build step. 103 | 104 | For example, a BuildKit build strategy author could use the following to ensure that TLS is always verified when pushing the resulting image: 105 | 106 | ```yaml 107 | spec: 108 | steps: 109 | - name: build 110 | ... 111 | env: 112 | - name: DOCKER_TLS_VERIFY 113 | value: "true" 114 | ``` 115 | 116 | Or alternatively, the build strategy author can make this configurable with a build parameter: 117 | 118 | ```yaml 119 | spec: 120 | parameters: 121 | - name: TLS_VERIFY 122 | description: "Verify TLS when pushing the resulting container image." 123 | default: "true" 124 | steps: 125 | - name: build 126 | ... 127 | env: 128 | - name: DOCKER_TLS_VERIFY 129 | value: $(params.TLS_VERIFY) 130 | ``` 131 | 132 | #### Build/BuildRun Environment Variables 133 | 134 | The `Build` and `BuildRun` APIs will be extended with a `spec.env` field. 135 | This is where a developer declares that they are appending environment variables to a build step. 136 | `env` will have an array of Kubernetes envrionment variables that will be appended to every step of the build strategy. 137 | 138 | Referring to the nodejs buildpacks story above, a developer who wants their build to install `dev` dependencies would do the following in their `Build` or `BuildRun`: 139 | 140 | ```yaml 141 | spec: 142 | sources: 143 | - name: git 144 | type: Git 145 | git: 146 | uri: https://github.com/sclorg/nodejs-ex.git 147 | strategy: 148 | kind: BuildStrategy 149 | name: buildpacks-v3 150 | env: 151 | - name: NPM_INCLUDE 152 | value: dev 153 | ``` 154 | 155 | Environment variables will always be appended to the environment variables declared in the build strategy's steps. 156 | If a `BuildRun` is started and either the referenced `Build` or the `BuildRun` has an environment variable name that collides with any step in the build strategy, the `BuildRun` should fail. 157 | A `Build` could reference environment variable names that collide with the build strategy, as it would be hard to guarantee that a `Build` does not have bad enviroment variable names over time. 158 | 159 | The environment variable API will take advantage of Kubernete's core `EnvVar` interface, which allows environment variable data to be sourced from a Secret or ConfigMap. 160 | 161 | A `BuildRun` can override the environment variables specified in a `Build` object. 162 | 163 | #### CLI Enhancements 164 | 165 | The `shp build run` command should be extended to allow environment variables to be passed in using the `-e` and `--env` flag. 166 | This flag can be specified more than once, and the value should use the `=` syntax. 167 | For example, to set the `NPM_INCLUDE` environment variable when invoking a run of the `buildpacks-nodejs` build, use the following: 168 | 169 | ```shell 170 | $ shp build run buildpacks-nodejs -e NPM_INCLUDE=dev 171 | ``` 172 | 173 | ### Test Plan 174 | 175 | **Note:** *Section not required until targeted at a release.* 176 | 177 | TBD 178 | 179 | ### Release Criteria 180 | 181 | **Note:** *Section not required until targeted at a release.* 182 | 183 | TBD 184 | 185 | #### Upgrade Strategy [if necessary] 186 | 187 | TBD 188 | 189 | ### Risks and Mitigations 190 | 191 | There is a risk that developers will want an environment variable to be set in one build step, but unset or restored to a default value in a different build step. 192 | This could be addressed in a future enhancemenent via separate API field (ex: `excludeFromSteps`). 193 | Users who are comfortable modifying build strategies can work around this issue by writing their own strategies with the desired environment variables fixed or exposed as a parameter. 194 | 195 | ## Drawbacks 196 | 197 | The primary drawback is that build strategy authors need to provide fixed or default values for environment variables they want to control. 198 | A strategy author cannot simply declare that an environment variable is "off limits" - they must declare when a controlled env var is used and what value it should have. 199 | This is a fair compromise to establish a clear contract between a build strategy author and the developer augmenting the build strategy's behavior. 200 | 201 | This proposal won't allow developers to set environment variables in some build strategy steps, but not others. 202 | This capability implies that the developer knows some information about the build strategy, which the community consideres an advanced skill. 203 | The Shipwright community does not expect developers to know the names of the steps declared in build strategies. 204 | Appending environment variables to all build strategy steps is simpler behavior that is easier to comprehend. 205 | This capabilty could be added in the future, but we do not consider this a requirement at present. 206 | 207 | ## Alternatives 208 | 209 | ### Allow/deny lists for environment variables 210 | 211 | A previous [build enhancement proposal](https://github.com/shipwright-io/build/pull/726) took a different approach to supporting environment variable overrides. 212 | In that proposal, it was incumbent on the build strategy authors to declare which environment variables could be overrode, and which environment variables could not be set. 213 | This was accomplished through allow/block lists, as well as a separate list of environment variables that could be declared with default values (and which were not declared in build steps). 214 | The downside of this proposal was that it severed the relationship of an environment variable from the place where the environment variable is used. 215 | Its complexity also led to competing paths to implement similar capabilites. 216 | 217 | This proposal avoids some of these problems by tying environment variables directly to build steps. 218 | Environment variables declared in build steps are assumed to be fixed unless they are exposed via a build parameter. 219 | 220 | ### Apply env vars to individual build steps 221 | 222 | A draft of this proposal applied environment variables to individual build steps, rather than all build steps. 223 | This required developers to discover the step names in the referenced build strategies to fully utilize this feature. 224 | These names are not immediately obvious to a developer using Shipwright. 225 | 226 | Mitigating this concern proved difficult, requiring one or more of the following: 227 | 228 | - Establishing a convention where `build` is the main step in any build strategy. 229 | Most build strategies only have one step - this convention will allow developers to use `env.build` to provide overrides in a natural fashion. 230 | - Enhancing the CLI to print the steps of a build strategy. 231 | 232 | We could not easily find a "kubernetes-native" means of printing the step names for each build strategy. 233 | Kubebuilder allows CRDs to have [additional printer columns](https://book.kubebuilder.io/reference/generating-crd.html#additional-printer-columns), however it does not support aribitrary-length arrays well. 234 | One could not instruct Kubernetes to print the `name` field of all items in a JSONPath list as comma-separated values. 235 | Instead, Kubernetes printed the specified value of the first item in the array. 236 | 237 | ## Implementation History 238 | 239 | - 2021-06-03: Initial draft proposal. 240 | - 2021-06-07: Move away from "overrides". 241 | - 2021-06-28: Implementable 242 | -------------------------------------------------------------------------------- /ships/0019-package-manager-releases.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | --- 8 | title: Release Shipwright CLI in well known package managers 9 | 10 | authors: 11 | - "@heavywombat 12 | 13 | reviewers: 14 | - "@otaviof" 15 | - "@alicerum" 16 | - "@adambkaplan" 17 | - "@qu1queee 18 | 19 | approvers: 20 | - "@qu1queee" 21 | - "@sbose78 22 | 23 | creation-date: 2021-07-07 24 | 25 | last-updated: 2021-07-09 26 | 27 | status: implementable 28 | 29 | see-also: n/a 30 | 31 | replaces: n/a 32 | 33 | superseded-by: n/a 34 | --- 35 | 36 | # Release Shipwright CLI in well known package managers 37 | 38 | ## Release Signoff Checklist 39 | 40 | - [X] Enhancement is `implementable` 41 | - [X] Design details are appropriately documented from clear requirements 42 | - [X] Test plan is defined 43 | - [ ] Graduation criteria for dev preview, tech preview, GA 44 | - [ ] User-facing documentation is created in [docs](/docs/) 45 | 46 | ## Open Questions [optional] 47 | 48 | _n/a_ 49 | 50 | ## Summary 51 | 52 | Users of CLI tools are accustomed to the usage of package managers to install a tool and to keep it up-to-date. The `shp` CLI tool should have an option to be installed with such a package manager. 53 | 54 | ## Motivation 55 | 56 | Make `shp` installable on the common platforms as easy, reliable and trustworthy as possible. 57 | 58 | ### Goals 59 | 60 | - Setup a Homebrew Tap for Shipwright tools and publish `shp` in it. 61 | - Create a `snap` to be available in [snapcraft.io](https://snapcraft.io/) store. 62 | 63 | ### Non-Goals 64 | 65 | Make us available in package managers that require a lengthy approval process, e.g. `apt`, or `yum`. 66 | 67 | ## Proposal 68 | 69 | Create a new repository https://github.com/shipwright-io/tap to serve as a Homebrew Tap. Register `shp` as a new `snap` in the [snapcraft.io](https://snapcraft.io/) store. Introduce GoReleaser configuration that builds and publishes to the repository and store on each release. 70 | 71 | ### Implementation Notes 72 | 73 | The setup for Homebrew and `snap` are pretty straightforward. For Homebrew, we only need a repository in GitHub with a well-known name, ie. `tap` or `homebrew-tap`. The [snapcraft.io](https://snapcraft.io/) is also relatively easy, it requires some metadata and a logo to be registered. The rest is handled by GoReleaser as it comes with publishing capabilities for these targets. As a side effect, this will also release a GitHub release with a change log and binaries. The [`dyff` GoReleaser configuration](https://github.com/homeport/dyff/blob/main/.goreleaser.yml) contains the respective configuration snippets for reference. 74 | 75 | ### Test Plan 76 | 77 | - On systems that support `brew`, such as macOS and Linux, run a test installation with `brew install shipwright-io/tap/shp` and verify the CLI version. 78 | - The `snap` system is also supported on a series of Linux systems, the test installation should be `snap install shp`. 79 | 80 | ### Release Criteria 81 | 82 | **Note:** *Section not required until targeted at a release.* 83 | 84 | ### Risks and Mitigations 85 | 86 | With the release process fully automated, there should be no risks of additional maintenance. For this to work, access credentials need to be configured to publish the binaries as a GitHub release and to commit to the Homebrew Tap, however, this can be done completely within GitHub and only accessed through GitHub Actions. The risk of credential exposure is therefore at the same level as the other credentials that are in place and relies heavily on the integrity of GitHub itself. 87 | 88 | ## Drawbacks 89 | 90 | None. 91 | 92 | ## Alternatives 93 | 94 | Present "install from source" and "curl to pipe" style install options. 95 | 96 | ## Infrastructure Needed 97 | 98 | - GitHub 99 | - GitHub Actions 100 | 101 | ## Implementation History 102 | 103 | _n/a_ 104 | -------------------------------------------------------------------------------- /ships/0020-shipwright-image-api.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | --- 8 | title: shipwright-image-api 9 | authors: 10 | - "@imjasonh" 11 | reviewers: 12 | - "@shbose78" 13 | - "@adambkaplan" 14 | - "@SaschaSchwarze0" 15 | - "@ricardomaraschini" 16 | approvers: 17 | - "@adambkaplan" 18 | - "@SaschaSchwarze0" 19 | creation-date: 2021-08-16 20 | last-updated: 2021-08-16 21 | status: provisional 22 | see-also: 23 | - https://github.com/shipwright-io/community/issues/8 24 | --- 25 | 26 | # Shipwright Image API 27 | 28 | ## Release Signoff Checklist 29 | 30 | - [ ] Enhancement is `implementable` 31 | - [ ] Design details are appropriately documented from clear requirements 32 | - [ ] Test plan is defined 33 | - [ ] Graduation criteria for dev preview, tech preview, GA 34 | - [ ] User-facing documentation is created in [docs](/docs/) 35 | 36 | ## Open Questions 37 | 38 | 1. Is this a worthwhile and well-aligned addition to the existing Shipwright project scope? 39 | 40 | 1. Should Shipwright Image subproject release at the same cadence as the Shipwright Build subproject? (don't need to answer soon) 41 | 42 | ## Summary 43 | 44 | Add a subproject to the Shipwright project that focuses on describing a set of general-purpose `Image` APIs. 45 | This may include a registry-protocol proxy for in-cluster or out-of-cluster image registries, that updates `Image` resources when images change, and can use Kubernetes RBAC rules on `Image` resources to guard pushes/pulls/etc of proxied image resources. 46 | 47 | This was previously briefly discussed at https://github.com/shipwright-io/community/issues/8 48 | 49 | ## Motivation 50 | 51 | Shipwright's existing scope has been mainly about describing and executing `Build`s, which produce container image artifacts. 52 | This has produced a set of APIs similar to OpenShift's Build API, among others, and has been generally well received. 53 | 54 | While the Build API helps developers and operators reason about and manage executions of builds, it lacks a concept for the container image artifact itself. 55 | This means that, though operators can use existing APIs to learn about images that were built by Shipwright, they can't do much to learn about other images that might exist in their platform. 56 | With existing APIs they can also only reason about images in the context of their _builds_, and not as first-class resources. 57 | 58 | OpenShift has had a similar concept for years, in [ImageStreams](https://docs.openshift.com/container-platform/4.8/openshift_images/images-understand.html#images-imagestream-use_images-understand) -- this proposal suggests a spiritual successor to this concept, separated from OpenShift, as a Shipwright subproject. 59 | 60 | An OpenShift developer, @ricardomaraschini, has prototyped https://github.com/ricardomaraschini/tagger, which implements most of what's being proposed, and more. 61 | He's interested in identifying a subset of this project that meets the Shiwpright community's needs and contributing that to a prospective Shipwright Image API subproject. 62 | 63 | ### Goals 64 | 65 | Create a Shipwright subproject focused on designing and implementing Image-focused CRD APIs that work well with the Build APIs. 66 | 67 | It should be possible to use Build APIs without Image APIs, and to use Image APIs without Build APIs, but to have the two work well together if both are being used. 68 | 69 | 70 | ### Non-Goals 71 | 72 | This proposal doesn't suggest fully specifying the Image API from the outset, only to create organization repos to serve as a testbed for further exploration of these APIs. 73 | 74 | The proposal should not include a full registry API implementation with storage; there are many in-cluster and out-of-cluster implementations in existence, and it's not a good use of our time to produce another one. 75 | 76 | ## Proposal 77 | 78 | Spawn a subproject under the Shipwright umbrella to develop these new APIs, and ensure they work well with existing Shipwright Build APIs, CLI, and Operator efforts. 79 | 80 | ### User Stories 81 | 82 | #### Story 1 83 | 84 | As an operator, I want to be able to configure a Kubernetes `Deployment` to be updated automatically when an `Image` resource in my cluster changes, whether that's as the result of a `Build`, or some external event such as an external registry push. 85 | 86 | #### Story 2 87 | 88 | As an operator, I want to have a Shipwright API that can tell me the history of an image tag, and let me use this information to perform a rollback to a previous version of my software. 89 | 90 | 91 | ### Implementation Notes 92 | 93 | None so far. 94 | 95 | ### Test Plan 96 | 97 | As with the Build APIs today, the Image subproject should include thorough and high-quality unit tests, integration tests, and end-to-end tests. 98 | 99 | ### Release Criteria 100 | 101 | Not currently targetting a release. 102 | 103 | ### Risks and Mitigations 104 | 105 | Widening the scope of the Shipwright project beyond what its contributors or users find useful is a potential risk. 106 | This can be mitigated by identifying new contributors who have an interest in maintaining this new featureset, and attracing users with a broader set of useful APIs that work well together. 107 | 108 | ## Drawbacks 109 | 110 | Expanding the scope of any software project is inherently risky. 111 | Existing contributors will have wider responsibilities, and current and prospective Shipwright users will have more APIs and features to learn and use (or to take time to learn why they don't care to use). 112 | 113 | ## Alternatives 114 | 115 | We could simply not embark on this effort, and remain focused on the existing Build APIs, CLI, operator, etc. 116 | 117 | ## Infrastructure Needed 118 | 119 | GitHub repository for Image APIs, teams to organize its collaborators, release processes to release any artifacts the subproject produces. 120 | 121 | ## Implementation History 122 | 123 | None so far. 124 | -------------------------------------------------------------------------------- /ships/0028-buildrun-cleanup.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | --- 8 | title: buildrun-cleanup 9 | authors: 10 | - @SaschaSchwarze0 11 | reviewers: 12 | - @GabeMontero 13 | - @qu1queee 14 | - @raghavbhatnagar96 15 | approvers: 16 | - @GabeMontero 17 | - @qu1queee 18 | creation-date: 2022-02-14 19 | last-updated: 2022-03-02 20 | status: implementable 21 | --- 22 | 23 | # BuildStrategy Parameter Enhancements 24 | 25 | ## Release Signoff Checklist 26 | 27 | - [x] Enhancement is `implementable` 28 | - [x] Design details are appropriately documented from clear requirements 29 | - [x] Test plan is defined 30 | - [x] Graduation criteria for dev preview, tech preview, GA 31 | - [ ] User-facing documentation is created in [docs](/docs/) 32 | 33 | ## Open Questions [optional] 34 | 35 | None 36 | 37 | ## Summary 38 | 39 | This ship proposes means for Build users to define that their completed BuildRuns should be automatically cleaned up after some time or when a certain number of BuildRuns is reached. 40 | 41 | ## Motivation 42 | 43 | In Shipwright, we separated the definition of a build and the run of a build into two artifacts, the Build and BuildRun. The BuildRun by its nature is a run-to-completion workload. Once it is completed, its only purpose is to view its status. Retaining the BuildRun for some time makes sense. In case of a succeeded BuildRun, one finds there the digest of the image. For a failed BuildRun, one finds the error details and a pointer to the pod logs. The lifecycle of the BuildRun is coupled with the TaskRun and Pod using [owner references](https://kubernetes.io/docs/concepts/overview/working-with-objects/owners-dependents/). 44 | 45 | Over times, the number of BuildRuns in the system grows. This consumes resources in Kubernetes' database (usually etcd) and on the nodes (the pod logs). 46 | 47 | Eventually, users must manually clean up old BuildRuns which is not nice. Providing them means to define that BuildRuns get automatically cleaned up simplifies their lives. 48 | 49 | ### Goals 50 | 51 | - Allow Build user to define how many completed BuildRuns should be kept. 52 | - Allow Build user to define how long a completed BuildRun should be kept. 53 | - Allow Build user to separate succeeded and failed BuildRuns in their cleanup handling. 54 | 55 | ### Non-Goals 56 | 57 | - Allowing BuildRuns to be archived to an external database. 58 | 59 | ## Proposal 60 | 61 | ### API Changes 62 | 63 | We will extend the Build resource with an additional optional `retention` section`: 64 | 65 | ```yaml 66 | spec: 67 | retention: 68 | ttlAfterFailed: duration 69 | ttlAfterSucceeded: duration 70 | succeededLimit: uint 71 | failedLimit: uint 72 | ``` 73 | 74 | All fields inside the `retention` section are optional. 75 | 76 | The `ttlAfterFailed` and `ttlAfterSucceeded` fields holds a duration. They define the duration on how long a BuildRun should live after it failed, or succeeded. 77 | 78 | A succeeded BuildRun is a BuildRun where the `type=Succeeded` condition is set to `"true"`. A failed BuildRun has the `type=Succeeded` condition is set to `"false"`, and contains therefore for example canceled BuildRuns as well. 79 | 80 | The two TTL fields behaves the same as the [`ttlSecondsAfterFinished` field for Jobs](https://kubernetes.io/docs/concepts/workloads/controllers/ttlafterfinished/). I decided against naming consistency with that feature, in favor of consistency inside our resource. Builds and BuildRuns already have a `timeout` field where we also use a duration while Kubernetes Jobs use an integer to store just seconds. Durations are easier to specify for users because they can simply write something like `6h` and don't need to calculate the amount of seconds. 81 | 82 | The `succeededLimit` and `failedLimit` fields hold a positive integer number. They define how many succeeded and failed BuildRuns for a Build can exist. 83 | 84 | If both TTL and limit are defined, then the BuildRun will get deleted once the first criteria is met. 85 | 86 | The BuildRun resource will be extended with a `retention` section with only the `ttlAfterFailed`, and `ttlAfterSucceeded` fields. They override the values of the Build. This is done for consistency (the `timeout` can also be overwritten in the BuildRun) and in support of standalone BuildRuns that we may have in the future. 87 | 88 | Updates to TTL or limit fields in the Build after a BuildRun was created/completed will have the following behavior: 89 | 90 | - If a TTL field is updated, then no changes are made to BuildRuns. A TTL therefore applies to an individual BuildRun. This is consistent with timeouts that are also only considered at BuildRun creation. 91 | - If a limit field is updated, then this new value applies. The values in the BuildRun's `status.buildSpec.retention.*Limit` are therefore obsolete. A limit is applicable to all BuildRuns of a Build in a namespace. Handling this differently is not possible as there can only be a single limit for them. 92 | 93 | This means that TTL and limit behave slightly different which needs to be explained in the documentation. 94 | 95 | ### Backend 96 | 97 | We will introduce new controllers for the BuildRun cleanup inside the same manager. The reasons for this are that our BuildRun controller is already highly complex - both in its predicate functions as well as in its `Reconcile` function - as it must handle TaskRuns and BuildRuns in various possible state combinations. There is one simple scenario where it simply does nothing: when the BuildRun is already completed. Instead of adding additional complexity for completed BuildRuns, the new controllers will focus exclusively on those aspects. 98 | 99 | We will introduce a new `buildrun-ttl-cleanup-controller` which will reconcile completed BuildRuns. It will watch BuildRuns with those predicate functions: 100 | 101 | - Create: if the BuildRun is completed (`status` of condition `type=Succeeded` set to `"true"` or `"false"`), and if a applicable TTL is set. This is handled for managers becoming the leader. 102 | - Update: if the BuildRun completed (`status` of condition `type=Succeeded` is changed from `"unknown"` to `"true"` or `"false"`), and if an applicable TTL is set. 103 | 104 | In the `Reconcile` function, we will check the BuildRun: When the `ttlAfterFailed` field is set for a BuildRun that failed, or when the `ttlAfterSucceeded` field is set for a BuildRun that succeeded, then we will calculate `.status.conditions[type=Succeeded].lastTransitionTime + TTL - now`. If the value is smaller than or equal zero, then we will delete the BuildRun. Otherwise, we will return a reconciliation result where `Requeue` is set to `true` and `RequeueAfter` is set to the calculated duration. 105 | 106 | We will introduce a new `build-limit-cleanup-controller` which will reconcile Builds. It will watch Builds with those predicate functions: 107 | 108 | - Create: if the Build has limits configured. This is handled for managers becoming the leader. 109 | - Update: if the Build has limits introduced, or changed to lower values. 110 | 111 | It will watch BuildRuns with those predicate functions: 112 | 113 | - Update: if the BuildRun completed (`status` of condition `type=Succeeded` is changed from `"unknown"` to `"true"` or `"false"`), and if it is related to a Build (standalone BuildRuns when introduced with SHIP 0029 will be ignored). 114 | 115 | In the `Reconcile` function, we will check the Build: if it has any limit set, then 116 | 117 | - We will run a list for BuildRuns in the namespace with the label filter `build.shipwright.io/name: `. 118 | - All BuildRuns will be put into three buckets: succeeded (`status` of condition `type=Succeeded` set to `"true"`), failed (`status` of condition `type=Succeeded` set to `"false"`) and the rest. 119 | - If the number of succeeded BuildRuns is larger than `succeededLimit`, then the oldest succeeded BuildRun(s) sorted by `.metadata.creationTimestamp` will be deleted. 120 | - If the number of failed BuildRuns is larger than `failedLimit`, then the oldest failed BuildRun(s) sorted by `.metadata.creationTimestamp` will be deleted. 121 | 122 | ### Staging 123 | 124 | The implementation can be done in stages. In particular, the TTL-based cleanup and the limit-based cleanup are very much independent. 125 | 126 | ### Documentation updates 127 | 128 | We will need to update the documentation about builds to introduce this new feature. 129 | 130 | ### User Stories 131 | 132 | #### As a Build user, I want to define how many finished BuildRuns I want to keep for my Build so that the amount of finished BuildRuns does not grow endlessly 133 | 134 | #### As a Build user, I want to define how long a finished BuildRun should stay so that I do not have obsolete and very old BuildRuns in the system 135 | 136 | ### Test Plan 137 | 138 | It should include: 139 | 140 | - Integration tests 141 | - Unit tests 142 | - e2e tests 143 | 144 | ### Release Criteria 145 | 146 | Should be available before release v1.0.0 147 | 148 | ### Risks and Mitigations 149 | 150 | We will have more controllers and reconcilers. Consumers may have to increase memory limits in the deployment. 151 | 152 | For the support of limits, especially the required `list` calls can become a problem if `succeededLimit`, or `failedLimit` are set to high values and we have to retrieve a large amount of BuildRuns from the API server and have them in memory. A mitigation would be that we define maximum values for those values (maybe 100). 153 | 154 | ## Drawbacks 155 | 156 | None. 157 | 158 | ## Alternatives 159 | 160 | For the use case: 161 | 162 | - We can help users who manually cleanup BuildRuns by providing CLI commands, for example to delete all BuildRuns that are older than some provided time. 163 | - We can provide samples to periodically cleanup BuildRuns, for example using [Kubernetes CronJobs](https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/). 164 | 165 | For the implementation: 166 | 167 | - Instead of adding new controllers, we could have extended the existing one. 168 | 169 | Extensions: 170 | 171 | - User may want to mark a BuildRun as not applicable for cleanup. For example through an annotation. The scenario is that there is a run one wants to keep for debugging purpose. We can consider this once somebody asks for it. 172 | 173 | ## Implementation History 174 | 175 | None 176 | -------------------------------------------------------------------------------- /ships/0032-image-read-mirror-from-namespace.md: -------------------------------------------------------------------------------- 1 | --- 2 | title: read-mirror-information-from-target-namespace 3 | authors: 4 | - "@ricardomaraschini" 5 | reviewers: 6 | - "@otaviof" 7 | - "@adambkaplan" 8 | - "@SaschaSchwarze0" 9 | - "@ImJasonH" 10 | - "@GabeMontero" 11 | approvers: 12 | - "@otaviof" 13 | - "@adambkaplan" 14 | - "@SaschaSchwarze0" 15 | - "@ImJasonH" 16 | - "@GabeMontero" 17 | creation-date: 2022-04-06 18 | last-updated: 2022-04-06 19 | status: implementable 20 | --- 21 | 22 | # Allow mirror registry configuration per namespace 23 | 24 | Shipwright Image operator reads mirror registry credentials from a Secret in the namespace where 25 | it is running therefore it is not possible for users to provide different mirrors in the same 26 | cluster. This enhancement proposal suggests a way of allowing users to provide authentications for 27 | different mirrors in a per namespace basis (each namespace holds the mirror registry access data). 28 | 29 | ## Release Signoff Checklist 30 | 31 | - [X] Enhancement is `implementable` 32 | - [ ] Design details are appropriately documented from clear requirements 33 | - [ ] Test plan is defined 34 | - [ ] Graduation criteria for dev preview, tech preview, GA 35 | - [ ] User-facing documentation is created in [docs](/docs/) 36 | 37 | ## Open Questions [optional] 38 | 39 | ## Summary 40 | 41 | This enhancement proposal suggests the introduction of a per namespace mirror registry config, 42 | without loosing the feature of having a global mirror configuration. The global mirror registry 43 | configuration becomes optional and is used if no credentials were found in the target namespace. 44 | 45 | ## Motivation 46 | 47 | When dealing with a single mirror registry the authentication provided to the operator must be 48 | able to write to different repositories inside the same registry. For example: when mirroring 49 | images in the `application` namespace images are mirrored into `registry.io/application` repo 50 | while images in the `development` namespace are mirrored into `registry.io/development`. Moreover, 51 | with a single mirror registry multi tenancy becomes more complex to be implemented if possible. 52 | 53 | ### Goals 54 | 55 | 1. Allow users to specify a per namespace registry mirror configuration 56 | 2. Allow users to specify a global mirror registry configuration as a fallback 57 | 58 | ### Non-Goals 59 | 60 | ## Proposal 61 | 62 | The Secret will be named `mirror-registry-config`, it can exist in any namespace and it can 63 | contain the following properties: 64 | 65 | | Name | Description | 66 | | -----------| ----------------------------------------------------------------------------------- | 67 | | address | The mirror registry URL | 68 | | repository | The repository, inside the registry, used to store images for a namespace (optional)| 69 | | username | Username Shipwright Images should use when accessing the mirror registry | 70 | | password | The password to be used by Shipwright Images | 71 | | token | The auth token to be used by Shipwright Images (optional) | 72 | | insecure | Allows Shipwright Images to access insecure registry if set to "true" (string) | 73 | 74 | If such a Secret exists in a given namespace users can mirror images inside it. If the Secret does 75 | not exist the operator should attempt to read it from the operator namespace allowing the current 76 | setup to keep working. In other words: the operator has a global mirror configuration living in 77 | its own namespace but uses a per namespace secret to overwrite the global setup. If neither local 78 | and global configurations are present the operator should properly report this situation in the 79 | ImageImport object status. 80 | 81 | The `repository` configuration is used in conjunction with the `address` to compose an unique 82 | repository address. For example: if `address` points to `my.registry.io` and `repository` is set 83 | to `myapp` all images inside the namespace will be mirrored into `my.registry.io/myapp`. If the 84 | `repository` configuration is empty the `namespace` name is used instead, for example an image 85 | living in the `my-namespace` namespace will be mirrored into `registry.address.io/my-namespace` 86 | repository. 87 | 88 | ### User Stories [optional] 89 | 90 | #### Story 1 91 | 92 | As an OpenShift administrator 93 | I would like to be able to specify a different mirror per namespace 94 | So I can easily isolate different users/use cases 95 | 96 | ### Implementation Notes 97 | 98 | This should be relatively easy to implement as the logic to parse the proposed secret already 99 | exists inside the operator. 100 | 101 | ### Test Plan 102 | 103 | We improve the existing `kuttl` end to end tests and deploy multiple mirror registries. Tests 104 | for global (operator namespace) and local secrets (target namespace) should be implemented. 105 | 106 | ## Implementation History 107 | 108 | None so far. 109 | -------------------------------------------------------------------------------- /ships/0035-beta-api-changes.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | --- 8 | title: Shipwright BETA API Release 9 | authors: 10 | - "@qu1queee" 11 | reviewers: 12 | - "@SaschaSchwarze0" 13 | - "@adambkaplan" 14 | approvers: 15 | - "@SaschaSchwarze0" 16 | - "@adambkaplan" 17 | creation-date: 2022-07-09 18 | last-updated: 2023-08-21 19 | status: implemented 20 | --- 21 | 22 | # Shipwright BETA API Release 23 | 24 | ## Release Signoff Checklist 25 | 26 | - [x] Enhancement is `implementable` 27 | - [x] Design details are appropriately documented from clear requirements 28 | - [x] Test plan is defined 29 | - [x] Graduation criteria for dev preview, tech preview, GA 30 | - [ ] User-facing documentation is created in [docs](/docs/) 31 | 32 | ## Open Questions [optional] 33 | 34 | ## Summary 35 | 36 | Propose a path to release Shipwright Beta API. Achieving a Beta API will provide users a more stable version they can depend on, with the certainty of not dropping features. In this document we provide the modifications, deprecations or additions to the Beta API. 37 | 38 | ## Motivation 39 | 40 | 1. The majority of the Shipwright Alpha API did not change over time. Part of the current Alpha API is considered mature enough for graduating to Beta. 41 | 42 | 2. The Shipwright API got many novel ideas over time, supporting different use cases. These features were implemented as part of the Alpha API, but the desired state was not reached. Planning for Beta API puts into triage these features and their potential deprecation. 43 | 44 | 3. The Shipwright API was created with some assumptions in mind, over time new features force us to rethink on better states of the API. 45 | 46 | ### Goals 47 | 48 | 1. Layout the changes for Beta API. 49 | 50 | 2. Document the deprecation policy for Beta API. 51 | 52 | 3. Define supported use cases for Beta API. 53 | 54 | 4. Document migration strategies of API conventions or object definitions from Alpha to Beta. 55 | 56 | ## Proposal 57 | 58 | This proposal introduces a Beta API for Shipwright, with version `v1beta1` following the Kubernetes API [versioning scheme](https://kubernetes.io/docs/reference/using-api/#api-versioning). The current API (`v1alpha1`) will be deprecated and eventually removed. 59 | 60 | We will define the `v1alpha1` API version as the stored version in the next releases(_v0.12.0_). One release after(_v0.14.0_) we will switch the stored version to `v1beta1`. 61 | 62 | 63 | ### Deprecation Policy 64 | 65 | In the spirit of the Kubernetes Deprecation [policies](https://kubernetes.io/docs/reference/using-api/deprecation-policy/), the beta API will come with the following support guarantees: 66 | 67 | 1. _No breaking changes, such as removal of fields, will be introduced without introducing a new beta API version (ex - `v1beta2`)._ 68 | 2. _Beta API versions may be deprecated, however they will continue to be served and supported for a period of 9 months after the deprecation is announced._ 69 | 3. _The beta API will never be deprecated in favor of an `alpha` API (ex - `v1alpha2`)._ 70 | 4. _Minimum API lifetime is determined by the API stability level_ 71 | 72 | Related to 4), in this document we propose that for Beta API versions, we provide coverage after deprecation for **9** months. We do not encourage to be based on releases quantity, per the faster cadence we have compared to Kubernetes. 73 | 74 | ### API Changes 75 | 76 | 1. In all resources, introduce a convention for objects with an array, where each element contains the `name` key. An existing example is `.spec.volumes`. 77 | 2. In the `Build` resource, allow multiple types within `.spec.source`. The types will be `.spec.source.git`, `.spec.source.local` and `.spec.source.ociartifact`. 78 | 3. In the `Build` resource, replace `.spec.source.credentials` in favor of a simple field(_key_) per Build type that highlights the need for such secret. For example, for the type `Git`, the field key name will be `.spec.source.git.cloneSecret`. 79 | 4. In the `Build` resource, replace `.spec.output.credentials` with a single field(_key_). This will become `.spec.output.pushSecret` and will reference a secret name in the current namespace. 80 | 5. In the `BuildStrategy` resource(_namespace and cluster scope_), replace `.spec.buildSteps` in favor of `steps`. The same applies to the `BuildStep` Go type. 81 | 6. In the `BuildStrategy` resource(_namespace and cluster scope_), replace the `.spec.buildSteps[]` items with a custom type, instead of the whole `corev1.Container` Go type. 82 | 7. In the `Build` resource, replace the `build.shipwright.io/build-run-deletion` annotation in favor of `.spec.retention.atBuildDeletion`. 83 | 8. In the `BuildRun` resource, rename `.status.latestTaskRunRef` to `.status.taskRunName`. 84 | 9. In the `BuildRun` resource, consolidate `.spec.buildRef.name` and `.spec.buildSpec` into `.spec.build.name` and `.spec.build.spec`. 85 | 10. Use `type discriminators` if an API object has some form of implied polymorphism. For example, source types or volume sources. 86 | 87 | ### Deprecation 88 | 89 | The deprecation of the API fields in the below list applies to `v1alpha1`. Deprecated fields are removed only in `v1beta1`. 90 | 91 | 1. In the `Build` resource, deprecate `.spec.sources`. We only allow `.spec.source`. 92 | 2. In the `Build` resource, deprecate `.spec.dockerfile`. 93 | 3. In the `Build` resource, deprecate `.spec.builder`. 94 | 4. In the `Build` resource, deprecate `.spec.volumes[].description`. 95 | 5. In the `Build`, `BuildStrategy` and `ClusterBuildStrategy` resource, remove the `status` subresource. 96 | 6. In the `BuildRun` resource, remove the support for auto-generating a service account. 97 | 98 | ### Supported Use Cases 99 | 100 | #### Story 1 101 | 102 | As a user, I can create a Shipwright _v1beta1_ custom resource. Shipwright should be able to convert this into a _v1alpha1_ custom resource and stored that version. 103 | 104 | #### Story 2 105 | 106 | As an external Kubernetes controller, I should be able to operate on Shipwright _v1beta1_ or _v1alpha1_ custom resources, independently of the API stored version. 107 | 108 | ### Implementation Details/Notes/Constraints [optional] 109 | 110 | We rely on the controller-runtime conversion model known as "hub and spoke". In this model, we will mark one version as the "hub", and all other versions just define conversion to and from the hub. In the scenario of needing to convert between two non-hub versions, we first convert to the hub version, and from there to the desired version: 111 | 112 | 113 | ![Controller-runtime hub and spoke model](assets/0006-hub-and-spoke-model.png) 114 | 115 | For requests to a particular version that is not the API stored version in ETCD, we will require a webhook. The webhook will be called by the API server to do the conversion. For the webhook, we will rely on the [webhook](https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/webhook/conversion) package from controller-runtime. The [conversion](https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/conversion) package will provide us the interface definitions that an API type must implement to be supported by the generic conversion webhook from controller-runtime. 116 | 117 | From the above, the implementation can be categorize in two areas: 118 | 119 | 120 | #### Webhook Implementation 121 | 122 | 1. We need to implement a webhook for conversion. To achieve this, we will introduce a new deployment, which will run the webhook code and expose a `/convert` endpoint, in order for CRD's to use. CRD's then will use the webhook service as documented in [Configure CustomResourceDefinition to use conversion webhooks](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning/#configure-customresourcedefinition-to-use-conversion-webhooks) to be able to convert their Custom Resources. This webhook besides invoking the conversion functions for the different CR types, will handle a `ConversionReview` request and will return conversion results wrapped in `ConversionResponse`. 123 | 124 | #### API types and conversion functions 125 | 126 | 1. Decide on the storage version. Here we propose to make _v1alpha1_ the storage version. This makes the _v1alpha1_ the "hub", meaning we convert to and from it. 127 | 2. Add the _v1beta1_ API types, under [apis/build](https://github.com/shipwright-io/build/tree/main/pkg/apis/build) 128 | 3. Introduce `_conversion.go` files next to the types, here we will define functions for conversion. This applies to all API versions. 129 | 4. Introduce a `webhook` pkg, which will be hosting the `Conversion` interface and the implementation behind the `/convert` endpoint. 130 | 5. For both _v1alpha1_ and _v1beta1_, the CR types should implement the `Conversion` interface, by adding a `ConvertFrom()` and `ConvertTo()` function. 131 | 6. Populate the conversion functions for _v1beta1_ and _v1alpha1_ types, to adhere to the API changes and deprecations mentioned above. 132 | 7. Regenerate all artifacts, from generated code to CRDs based on the new API types. Ensure the CRDs are updated with the `spec.conversion` block using the `/convert` endpoint and the proper webhook service. Ensure the CRDs explicitly state which API version must be stored and which not. 133 | 8. Ensure that helper scripts for autogenerating are aware of the _v1beta1_ type. That a CABundle is autogenerated for the CRD's, etc. 134 | 135 | 136 | Note: We do not need to modify shipwright business logic or test cases, as they currently use the _v1alpha1_ API version. However, once the _hub_ API version is modified, we will need to modify this accordingly. 137 | 138 | 139 | ### Risks and Mitigations 140 | 141 | None. Users should be able to use both API versions without any inconvenience. 142 | 143 | ## Design Details 144 | 145 | ### Test Plan 146 | 147 | - Not required. We do not modify for the time being any test case. 148 | - We need implement unit-tests to the `*_conversion.go`. 149 | 150 | ### Graduation Criteria 151 | 152 | This will be part of the Shipwright/Build v0.12.0 release- 153 | 154 | ### Upgrade / Downgrade Strategy 155 | 156 | Not applicable. 157 | 158 | ## Implementation History 159 | 160 | Not applicable. 161 | 162 | -------------------------------------------------------------------------------- /ships/0037-build-output-timestamp.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | --- 8 | title: build-output-timestamp 9 | authors: 10 | - "@HeavyWombat" 11 | reviewers: 12 | - "@adambkaplan" 13 | - "@qu1queee" 14 | - "@SaschaSchwarze0" 15 | approvers: 16 | - "@adambkaplan" 17 | - "@qu1queee" 18 | creation-date: 2023-04-24 19 | last-updated: 2023-12-11 20 | status: implemented 21 | --- 22 | 23 | # Build Output Image Timestamp 24 | 25 | ## Release Signoff Checklist 26 | 27 | - [x] Enhancement is `implementable` 28 | - [x] Design details are appropriately documented from clear requirements 29 | - [x] Test plan is defined 30 | - [ ] Graduation criteria for dev preview, tech preview, DA 31 | - [ ] User-facing documentation is created in [docs](/docs/) 32 | 33 | ## Summary 34 | 35 | This _ship_ proposes an extension to the `spec.output` of a Build (or BuildRun through the build specification) to explicitly set the result image timestamp. 36 | 37 | ## Motivation 38 | 39 | With Shipwright, we can help users to achieve [reproducible builds](https://reproducible-builds.org/docs/timestamps/) depending on the tools that are used in the strategy. One factor of reproducible builds is avoiding the usage of build system specific timestamps in the final image. Ideally, when using a `Dockerfile` based build, running the build twice with the exact same source input should yield the same image. 40 | 41 | ### Goals 42 | 43 | - Allow a Build user to specify that a neutral timestamp is used for the creation time of the container image, i.e. unix timestamp zero. 44 | - Allow a Build user to specify that -- if possible -- Shipwright is suppose to come up with a reasonable deterministic timestamp of the build, e.g. the Git commit timestamp for Git source based builds similar to how GoReleaser does it. 45 | - Allow a Build user to specify that Shipwright is not suppose to do anything with regards to the container image creation timestamp. 46 | 47 | ### Non-Goals 48 | 49 | - Modification of timestamps of the source files. 50 | - No post-processing of container images that used strategy managed push, i.e. Paketo Buildpacks. 51 | - No magic build strategy manipulation of steps other than the `image-processing` step. 52 | 53 | ### User Stories 54 | 55 | #### Story 1 56 | 57 | As a Shipwright user, I want to use a `Dockerfile` based build where the resulting container image creation timestamp is set to unix timestamp zero. 58 | 59 | #### Story 2 60 | 61 | As a Shipwright user, I want to use build a container image that has the image creation timestamp that matches the one of the Git commit that was used for the build. 62 | 63 | #### Story 3 64 | 65 | As a Shipwright user, I want write a build that results in a build run that does not touch the image timestamp at all. 66 | 67 | ## Proposal 68 | 69 | ### API Changes 70 | 71 | We will extend the Build resource by an additional (Unix epoch) timestamp entry in `spec.output`: 72 | 73 | ```yaml 74 | spec: 75 | output: 76 | timestamp: Zero | SourceTimestamp | BuildTimestamp | | (null) 77 | ``` 78 | 79 | The `timestamp` entry is an optional string value field with the default being `null`. It can have multiple values that lead to different behavior: 80 | 81 | - `Zero` refers to the Unix epoch start date of January, 1st 1970. 82 | - `SourceTimestamp` would lead to taking the provided source code meta data to define the timestamp. In case the source is a Git repository, the Git commit that is used for the build defines the source timestamp. In any other case, the newest creation time stamp of the source files is used. Note, the timestamp needs to be taken from the user provided source files since the build process might introduce new files as part of the build itself. The source step needs to be extended to introduce a reliable point and single spot in which the source timestamp is being obtained. 83 | - `BuildTimestamp` defines that the image timestamp is to be set to the current actual execution timestamp of the Build Run. 84 | - `` must be a parsable Unix epoch timestamp that lies in the past and will be then used as-is for the creation timestamp of the image, i.e. custom neutral timestamp. An empty string is considered not parsable and therefore results in an error. 85 | - Null (unset) results in no timestamp usage at all, which means that the respective build strategy timestamp is used. 86 | 87 | The same fields will be made available in a BuildRun's `spec.output` and `spec.buildSpec.output`. 88 | 89 | The BuildRun status will be extended to include the timestamp in the output field. 90 | 91 | ### Backend 92 | 93 | The timestamp processing will be performed as part of the [image-processing step that is introduced with Shipwright-managed push](0026-shipwright-managed-push.md#backend). 94 | 95 | It can only work with Shipwright managed push strategies and it will rely on the source step providing the necessary timestamp piece of information. If not provided with a timestamp, it will fail with condition: 96 | ```yaml 97 | conditions: 98 | - type: Succeeded 99 | reason: Failed 100 | status: "False" 101 | lastTransitionTime: "2023-10-10T10:10:10Z" 102 | message: 'Cannot change image timestamp in the context of a build strategy that pushes the image itself as part of its build process.' 103 | ``` 104 | 105 | The respective source step needs to be extended with a flag or feature so that it produces a result file for the commit timestamp, similar to the result files for author or commit SHA. This would only be done and subsequently used in case the `SourceTimestamp` is configured in the build. 106 | 107 | - `Git` build source will use the timestamp of the last commit to produce the timestamp file. 108 | - `Local` build source will produce a timestamp file with the timestamp of the newest file in the uploaded set of files. 109 | - `OCI` build source will produce a timestamp file with the timestamp of the newest file in the uploaded bundle container. 110 | 111 | The `image-processing` step and code needs to be updated to take the configured timestamp and mutates the image to set the respective timestamp prior to pushing the image. 112 | 113 | ### Configuration 114 | 115 | n/a since no deployment configuration is required 116 | 117 | ### CLI 118 | 119 | The new output settings will be added as flags to the appropriate CLI commands. 120 | 121 | ### Sample updates 122 | 123 | Extend build samples to show all possible options at least once. 124 | 125 | ### Documentation updates 126 | 127 | The Build and BuildRun documentation are updated to explain the new fields and their behavior. 128 | 129 | ### Test Plan 130 | 131 | The implementation has to be tested on a `unit`, and `e2e` level to ensure correctness. 132 | 133 | ## Release Criteria 134 | 135 | n/a since this can be added at any point since the (new) default behavior is the current behavior 136 | 137 | ## Risks and Mitigations 138 | 139 | Having a `SourceTimestamp` convenience setting is great for the main use case of using a Git repository, it could be confusing for a user when they accidentally try to use it with a non-Git source build and their Build or BuildRun fails to be registered. A clear and easy to understand error message is required in this case. 140 | 141 | ## Drawbacks 142 | 143 | n/a since this does not has to be used by a Build user 144 | 145 | ## Alternatives 146 | 147 | ### Generic post-processing step 148 | 149 | Instead of using the `image-processing` step, we could inject a common post-processing step for all strategies which could modify the timestamp of the image after it got pushed. This would work for all strategies that push images, independently whether it is strategy managed push or Shipwright managed push. 150 | 151 | ## Implementation History 152 | 153 | Nothing so far. 154 | -------------------------------------------------------------------------------- /ships/README.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | # Shipwright Improvement Proposals (SHIPs) 8 | 9 | A Shipwright Improvement Proposal (SHIP) is a way to propose, communicate, and coordinate on new efforts for Shipwright. 10 | 11 | ## Quick start 12 | 13 | 1. Socialize an idea with others. Make sure others think the work is worth doing, and are willing to review design and code changes required. 14 | 2. Draft a proposal by copying the [proposal template](/ships/guidelines/ship-template.md) into this directory. 15 | Name your SHIP with the file name pattern `ship-NNNN-tile-name-here.md`, incrementing to the next number in the sequenced of merged proposals. 16 | 3. Submit a pull request with your proposal. 17 | 18 | ## SHIP Lifecycle 19 | 20 | * **Draft**: SHIPs start their lifecycle as a pull request, copying the proposal template. 21 | They graduate to the _provisional_ or _implementable_ phase when the pull request is merged. 22 | * **Provisional**: SHIPs that reach this phase focus on the _why_. 23 | The objective is to agree that an idea addresses a need for the community. 24 | The summary, motivation, and goals should be clearly articulated. 25 | Drawbacks and alternatives may not need to be identified at this phase, though the community may ask that these be included during review. 26 | Merging a SHIP as provisional does not imply that the community will implement a feature within any reasonable timeframe. 27 | * **Implementable**: SHIPs that reach this phase focus on the _how_. 28 | To graduate to this phase, the proposal needs to provide details on how the feature will be implemented and any new APIs that are introduced. 29 | The proposal must also highlight the risks, drawbacks, and alternatives considered for this feature. 30 | Test and release plans do not need to be fully articulated, though the community may suggest ways to test or release the feature at this stage. 31 | Implementable proposals indicate that the community agrees to a feature and related code can be merged. 32 | * **Implemented**: SHIPs that reach this phase focus on _what_ and _when_. 33 | At this point the feature may have related code pull requests that are under review or merged. 34 | Implemented proposals should have test and release plans identified. 35 | At this point the propsosal is considered "done." 36 | 37 | Other phases in the proposal lifecycle can be: 38 | 39 | * **Rejected**: Used if a provisional or implementable proposal is later rejected by the community. 40 | * **Withdrawn**: Used if the author of a provisional or implementable proposal later withdraws it from consideration. 41 | * **Replaced**: Used if a subsequent proposal is used to implement the same set of functionality. 42 | 43 | ## FAQs 44 | 45 | **Do I have to use the process?** 46 | 47 | If the enhancement has broad scope, yes. 48 | As a general rule, features that cross multiple components or introduce new APIs should have an enhancement prposal. 49 | It helps everyone track why, when, how, and by whom work is done. 50 | 51 | **Why would I want to use the process?** 52 | 53 | Provide a mechanism to communicate design and implementation strategies across the Shipwright community. 54 | 55 | **Do I need to fill out the entire template?** 56 | 57 | You do not have to complete every section of the proposal template - different sections serve different purposes in the SHIP lifecycle. 58 | 59 | **My FAQ isn't answered here!** 60 | 61 | Open an issue and ask or even better open a PR with a question and proposed answer. 62 | -------------------------------------------------------------------------------- /ships/assets/0005-source-upload-sequence-diagram.drawio.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shipwright-io/community/592c9226d08d4d0a331eb0d4b0b2540f4dad5188/ships/assets/0005-source-upload-sequence-diagram.drawio.png -------------------------------------------------------------------------------- /ships/assets/0006-hub-and-spoke-model.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shipwright-io/community/592c9226d08d4d0a331eb0d4b0b2540f4dad5188/ships/assets/0006-hub-and-spoke-model.png -------------------------------------------------------------------------------- /ships/assets/0031-shipwright-trigger-components.drawio.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shipwright-io/community/592c9226d08d4d0a331eb0d4b0b2540f4dad5188/ships/assets/0031-shipwright-trigger-components.drawio.png -------------------------------------------------------------------------------- /ships/assets/0031-shipwright-trigger-webhook.drawio.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/shipwright-io/community/592c9226d08d4d0a331eb0d4b0b2540f4dad5188/ships/assets/0031-shipwright-trigger-webhook.drawio.png -------------------------------------------------------------------------------- /ships/guidelines/ship-template.md: -------------------------------------------------------------------------------- 1 | 6 | 7 | --- 8 | title: neat-enhancement-idea 9 | authors: 10 | - "@janedoe" 11 | reviewers: 12 | - TBD 13 | - "@alicedoe" 14 | approvers: 15 | - TBD 16 | - "@oscardoe" 17 | creation-date: yyyy-mm-dd 18 | last-updated: yyyy-mm-dd 19 | status: provisional|implementable|implemented|rejected|withdrawn|replaced 20 | see-also: 21 | - "/docs/proposals/this-other-neat-thing.md" 22 | replaces: 23 | - "/docs/proposals/that-less-than-great-idea.md" 24 | superseded-by: 25 | - "/docs/proposals/our-past-effort.md" 26 | --- 27 | 28 | # Neat Enhancement Idea 29 | 30 | This is the title of the enhancement. Keep it simple and descriptive. A good title can help 31 | communicate what the enhancement is and should be considered as part of any review. 32 | 33 | The YAML `title` should be lowercased and spaces/punctuation should be replaced with `-`. 34 | 35 | To get started with this template: 36 | 37 | 1. **Make a copy of this template.** Copy this template into the main 38 | `proposals` directory, with a filename like `NNNN-neat-enhancement-idea.md` 39 | where `NNNN` is an incrementing number associated with this SHIP. 40 | 2. **Fill out the "overview" sections.** This includes the Summary and Motivation sections. These 41 | should be easy and explain why the community should desire this enhancement. 42 | 3. **Create a PR.** Assign it to folks with expertise in that domain to help 43 | sponsor the process. The PR title should be like "SHIP-NNNN: Neat 44 | Enhancement Idea", where "NNNN" is the number associated with this SHIP. 45 | 4. **Merge at each milestone.** Merge when the design is able to transition to a new status 46 | (provisional, implementable, implemented, etc.). View anything marked as `provisional` as an idea 47 | worth exploring in the future, but not accepted as ready to execute. Anything marked as 48 | `implementable` should clearly communicate how an enhancement is coded up and delivered. Aim for 49 | single topic PRs to keep discussions focused. If you disagree with what is already in a document, 50 | open a new PR with suggested changes. 51 | 52 | The `Metadata` section above is intended to support the creation of tooling around the enhancement 53 | process. 54 | 55 | ## Release Signoff Checklist 56 | 57 | - [ ] Enhancement is `implementable` 58 | - [ ] Design details are appropriately documented from clear requirements 59 | - [ ] Test plan is defined 60 | - [ ] Graduation criteria for dev preview, tech preview, GA 61 | - [ ] User-facing documentation is created in [docs](/docs/) 62 | 63 | ## Open Questions [optional] 64 | 65 | This is where to call out areas of the design that require closure before deciding to implement the 66 | design. For instance: 67 | 68 | > 1. This locks a build strategy to run privileged pods. Should we do this? 69 | 70 | ## Summary 71 | 72 | The `Summary` section is incredibly important for producing high quality user-focused documentation 73 | such as release notes or a development roadmap. It should be possible to collect this information 74 | before implementation begins in order to avoid requiring implementors to split their attention 75 | between writing release notes and implementing the feature itself. 76 | 77 | A good summary is probably at least a paragraph in length. 78 | 79 | ## Motivation 80 | 81 | This section is for explicitly listing the motivation, goals and non-goals of this proposal. 82 | Describe why the change is important and the benefits to users. 83 | 84 | ### Goals 85 | 86 | List the specific goals of the proposal. How will we know that this has succeeded? 87 | 88 | ### Non-Goals 89 | 90 | What is out of scope for this proposal? Listing non-goals helps to focus discussion and make 91 | progress. 92 | 93 | ## Proposal 94 | 95 | This is where we get down to the nitty gritty of what the proposal actually is. 96 | 97 | ### User Stories [optional] 98 | 99 | Detail the things that people will be able to do if this is implemented. Include as much detail as 100 | possible so that people can understand the "how" of the system. The goal here is to make this feel 101 | real for users without getting bogged down. 102 | 103 | #### Story 1 104 | 105 | #### Story 2 106 | 107 | ### Implementation Notes 108 | 109 | **Note:** *Section not required until feature is ready to be marked 'implementable'.* 110 | 111 | Describe in detail what you propose to change. Be specific as to how you intend to implement this 112 | feature. If you plan to introduce a new API field, provide examples of how the new API will fit in 113 | the broader context and how end users could invoke the new behavior. 114 | 115 | ### Test Plan 116 | 117 | **Note:** *Section not required until targeted at a release.* 118 | 119 | Consider the following in developing a test plan for this enhancement: 120 | 121 | - Will there be e2e and integration tests, in addition to unit tests? 122 | - How will it be tested in isolation vs with other components? 123 | 124 | No need to outline all of the test cases, just the general strategy. Anything that would count as 125 | tricky in the implementation and anything particularly challenging to test should be called out. 126 | 127 | All code is expected to have adequate tests (eventually with coverage expectations). 128 | 129 | ### Release Criteria 130 | 131 | **Note:** *Section not required until targeted at a release.* 132 | 133 | #### Removing a deprecated feature [if necessary] 134 | 135 | - Announce deprecation and support policy of the existing feature 136 | - Deprecate the feature 137 | 138 | #### Upgrade Strategy [if necessary] 139 | 140 | If applicable, how will the component be upgraded? Make sure this is in the test 141 | plan. 142 | 143 | Consider the following in developing an upgrade strategy for this enhancement: 144 | 145 | - What changes (in invocations, configurations, API use, etc.) is an existing cluster required to 146 | make on upgrade in order to keep previous behavior? 147 | - What changes (in invocations, configurations, API use, etc.) is an existing cluster required to 148 | make on upgrade in order to make use of the enhancement? 149 | 150 | ### Risks and Mitigations 151 | 152 | What are the risks of this proposal and how do we mitigate? Think broadly. For example, consider 153 | both security and how this will impact the larger Shipwright ecosystem. 154 | 155 | How will security be reviewed and by whom? How will UX be reviewed and by whom? 156 | 157 | ## Drawbacks 158 | 159 | The idea is to find the best form of an argument why this enhancement should _not_ be implemented. 160 | 161 | ## Alternatives 162 | 163 | Similar to the `Drawbacks` section the `Alternatives` section is used to highlight and record other 164 | possible approaches to delivering the value proposed by an enhancement. 165 | 166 | ## Infrastructure Needed [optional] 167 | 168 | Use this section if you need things from the project. Examples include a new subproject, repos 169 | requested, github details, and/or testing infrastructure. 170 | 171 | Listing these here allows the community to get the process for these resources started right away. 172 | 173 | ## Implementation History 174 | 175 | Major milestones in the life cycle of a proposal should be tracked in `Implementation History`. 176 | 177 | 178 | --------------------------------------------------------------------------------