├── .gitignore ├── diagrams ├── Salt-Release-Timeline.png ├── diagrams_new-branch-strategy.png ├── diagrams_old-branch-strategy.png └── Working-group-timelines.svg ├── .github ├── CODEOWNERS └── config.yml ├── abandoned ├── README.md ├── 0027_Create_Community_Advisory_Board └── 0005-worker-cache.md ├── accepted ├── 0012-update-engines-without-minion-restart.md ├── 0001-changelog-format.md ├── 0010-pr-merge-requirements.md ├── 0015-black.md ├── 0017-job-ack-event.md ├── 33-readd-point-zero.md ├── 0003-more-sep-statuses.md ├── 0008-python3-ssh-template.md ├── 0018-release-label-improvements.md ├── package-salt-with-tiamat.md ├── 0005-retire-py2-support.md ├── 0022-old-releases.md ├── 0006-remote_client.md ├── 0024-sub-state-returns.md ├── 0019-master-non-root.md ├── 0009-salt-working-groups.md ├── 0031-windows-install-anywhere.md ├── 0014-dev-overhaul.md └── 0020-py-version-support.md ├── 0029-pkg-beacon-updates.md ├── 0000-template.md ├── 0040-lts-releases.md ├── pull ├── 0000-readd-point-zero.md └── 0027_Create_Community_Advisory_Board ├── inclusive-language.md ├── 0037-pluggable-transports.md └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | *.sw[a-p] 2 | .idea/* 3 | -------------------------------------------------------------------------------- /diagrams/Salt-Release-Timeline.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/saltstack/salt-enhancement-proposals/HEAD/diagrams/Salt-Release-Timeline.png -------------------------------------------------------------------------------- /diagrams/diagrams_new-branch-strategy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/saltstack/salt-enhancement-proposals/HEAD/diagrams/diagrams_new-branch-strategy.png -------------------------------------------------------------------------------- /diagrams/diagrams_old-branch-strategy.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/saltstack/salt-enhancement-proposals/HEAD/diagrams/diagrams_old-branch-strategy.png -------------------------------------------------------------------------------- /.github/CODEOWNERS: -------------------------------------------------------------------------------- 1 | # SALTSTACK CODE OWNERS 2 | 3 | # See https://help.github.com/articles/about-codeowners/ 4 | # for more info about the CODEOWNERS file 5 | 6 | # Lines starting with '#' are comments. 7 | # Each line is a file pattern followed by one or more owners. 8 | 9 | # This file uses an fnmatch-style matching pattern. 10 | 11 | # Team Core 12 | * @saltstack/team-core 13 | -------------------------------------------------------------------------------- /abandoned/README.md: -------------------------------------------------------------------------------- 1 | # Abandoned SEPs 2 | 3 | SEPs here have been abandoned, or rejected without prejudice. They may be 4 | fundamentally sound, or need major reworking, or the author may have had 5 | other priorities in their life. 6 | 7 | In any case, they are up for grabs - if you like an idea here and you 8 | believe in it, you can adopt the SEP. Take it home with you, look at the 9 | comments on the original PR, figure out what needs to happen to get this SEP 10 | into shape, and create a new PR that references the original PR. 11 | -------------------------------------------------------------------------------- /accepted/0012-update-engines-without-minion-restart.md: -------------------------------------------------------------------------------- 1 | - Feature Name: update-engines-without-minion-restart 2 | - Start Date: 2019-07-16 3 | - SEP Status: Accepted 4 | - SEP PR: https://github.com/saltstack/salt-enhancement-proposals/pull/16 5 | - Salt Issue: 6 | 7 | # Summary 8 | [summary]: #summary 9 | 10 | Being able to restart minion engines without having to restart the minion itself. 11 | 12 | # Motivation 13 | [motivation]: #motivation 14 | 15 | As a user I need to run a new engine on a minion, but restarting the minion needs to be done carefully and may not happen when I need it. 16 | This is already possible with beacons. 17 | 18 | # Design 19 | [design]: #detailed-design 20 | 21 | The engine configuration could be either in the configuration or in the pillar. 22 | When refreshing the pillar, loop over the engines configuration and start or stop the engines according to the new configuration. 23 | 24 | ## Alternatives 25 | [alternatives]: #alternatives 26 | 27 | The alternative is to manually restart the salt minion, with all the problems this can bring. 28 | 29 | ## Unresolved questions 30 | [unresolved]: #unresolved-questions 31 | 32 | If this is too problematic to run on every pillar refresh, then an additional function could be provided to only refresh the engines. 33 | 34 | # Drawbacks 35 | [drawbacks]: #drawbacks 36 | 37 | The engines documentation would need to add this new possibility to configure the engines using pillar. 38 | -------------------------------------------------------------------------------- /0029-pkg-beacon-updates.md: -------------------------------------------------------------------------------- 1 | - Feature Name: Changes to PKG Beacon 2 | - Start Date: 2020-12-18 3 | - SEP Status: Draft 4 | - SEP PR: (leave this empty) 5 | - Salt Issue: (leave this empty) 6 | 7 | # Summary 8 | [summary]: #summary 9 | 10 | Changes to the PKG beacon to fire events when software packages are removed and installed, in addition to when upgrades are available. 11 | Because of the way beacons work, this change will also require changes to all PKG modules to ensure that there is the option in the form of a keyword argument to pull an updated list of installed packages and bypass the ___context dunder. By default the list_pkgs functions will still continue to pull a list of installed packages from the __context__ dunder dictionary. 12 | 13 | # Motivation 14 | [motivation]: #motivation 15 | 16 | This change would be an enhancement to the PKG beacon, bringing additional functionality. 17 | 18 | # Design 19 | [design]: #detailed-design 20 | 21 | Currently the PKG beacon will monitor the list of installed packages and fire an event when an upgrade is available. 22 | 23 | This change would update that functionality to fire an event the status of a package changes from installed to uninstalled and vice versa, in addition to when an updated version is available. 24 | 25 | # Drawbacks 26 | [drawbacks]: #drawbacks 27 | 28 | Why should we *not* do this? Please consider: 29 | 30 | - This would change the format and the contents returned from the PKG beacon to the event bus. This change will need to be communicated out to anyone relying on the existing format. 31 | -------------------------------------------------------------------------------- /0000-template.md: -------------------------------------------------------------------------------- 1 | - Feature Name: (fill with a unique identifier, ex: my-awesome-feature) 2 | - Start Date: (fill with today's date, YYYY-MM-DD) 3 | - SEP Status: Draft 4 | - SEP PR: (leave this empty) 5 | - Salt Issue: (leave this empty) 6 | 7 | # Summary 8 | [summary]: #summary 9 | 10 | Brief explanation of the feature. 11 | 12 | # Motivation 13 | [motivation]: #motivation 14 | 15 | Why are we doing this? What use cases does it support? What is the expected outcome? 16 | 17 | If this SEP is not accepted, the motivation could be used to develop alternative solutions. Enumerate the constraints you are trying to solve without coupling them too closely to the solution you have in mind. 18 | 19 | # Design 20 | [design]: #detailed-design 21 | 22 | This is the bulk of the SEP. Explain the design in enough detail for somebody familiar 23 | with the product to understand, and for somebody familiar with the internals to implement. It should include: 24 | 25 | - Definition of any new terminology 26 | - Examples of how the feature is used. 27 | - Corner-cases 28 | - A basic code example in case the proposal involves a new or changed API 29 | - Outline of a test plan for this feature. How do you plan to test it? Can it be automated? 30 | 31 | ## Alternatives 32 | [alternatives]: #alternatives 33 | 34 | What other designs have been considered? What is the impact of not doing this? 35 | 36 | ## Unresolved questions 37 | [unresolved]: #unresolved-questions 38 | 39 | What parts of the design are still TBD? 40 | 41 | # Drawbacks 42 | [drawbacks]: #drawbacks 43 | 44 | Why should we *not* do this? Please consider: 45 | 46 | - Implementation cost, both in term of code size and complexity 47 | - Integration of this feature with other existing and planned features 48 | - Cost of migrating existing Salt setups (is it a breaking change?) 49 | - Documentation (would Salt documentation need to be re-organized or altered?) 50 | 51 | 52 | There are tradeoffs to choosing any path. Attempt to identify them here. 53 | -------------------------------------------------------------------------------- /accepted/0001-changelog-format.md: -------------------------------------------------------------------------------- 1 | - Feature Name: Standarize on the format of changelogs 2 | - Start Date: 2019-02-15 3 | - SEP Status: Approved 4 | - SEP PR: http://github.com/saltstack/salt-enhancement-proposals/pull/2 5 | 6 | # Summary 7 | [summary]: #summary 8 | 9 | This RFC proposes a standard changelog format for software released by the SaltStack project. 10 | 11 | # Motivation 12 | [motivation]: #motivation 13 | 14 | The current format for changelog entries at Salt is unstructured and can be difficult to read. 15 | 16 | Many users of Salt have complained that they weren't aware of a breaking change which may 17 | be the result of a poorly structured and inconsistent changelog between releases. 18 | 19 | # Design 20 | [design]: #detailed-design 21 | 22 | This RFC standardizes Salt on the 1.0.0 spec of project changelogs outlined at 23 | https://keepachangelog.com/en/1.0.0/ 24 | 25 | Specifically, it provides for the following principles: 26 | 27 | - Changelogs are for humans, not machines. 28 | - There should be an entry for every single version. 29 | - The same types of changes should be grouped. 30 | - Versions and sections should be linkable. 31 | - The latest version comes first. 32 | - The release date of each version is displayed. 33 | - Mention whether you follow Semantic Versioning. 34 | 35 | Types of changes 36 | ---------------- 37 | - Added for new features. 38 | - Changed for changes in existing functionality. 39 | - Deprecated for soon-to-be removed features. 40 | - Removed for now removed features. 41 | - Fixed for any bug fixes. 42 | - Security in case of vulnerabilities. 43 | 44 | ## Alternatives 45 | [alternatives]: #alternatives 46 | 47 | Current alternative is what we have now. ;) 48 | 49 | ## Unresolved questions 50 | [unresolved]: #unresolved-questions 51 | 52 | Perhaps a sample changelog written for a previous version might 53 | be appropriate to put together and attach to this RFC? 54 | 55 | # Drawbacks 56 | [drawbacks]: #drawbacks 57 | 58 | People could be used to the current version and depend on it. 59 | This would be more work for contributors. It would take more attention from the core 60 | team to attend to. 61 | 62 | -------------------------------------------------------------------------------- /0040-lts-releases.md: -------------------------------------------------------------------------------- 1 | - Start Date: 2022-04-20 2 | - SEP Status: Final Comment 3 | - SEP PR: (leave this empty) 4 | - Salt Issue: (leave this empty) 5 | 6 | # Summary 7 | [summary]: #summary 8 | 9 | The core team would like to move to an LTS release strategy 10 | 11 | # Motivation 12 | [motivation]: #motivation 13 | 14 | Our current release strategy causes us to provide security and bugfixes for as 15 | many as five versions. Our documented [life cycle](https://docs.saltproject.io/salt/install-guide/en/latest/topics/salt-version-support-lifecycle.html#salt-version-support-lifecycle) has become less 16 | sustainable as we've increased the frequency of releasing new versions of Salt. 17 | 18 | # Design 19 | [design]: #detailed-design 20 | 21 | We would like to move to maintaining at most two branches/versions of Salt. We 22 | can accomplish this by designating a single version as Long Term Support (LTS). 23 | A LTS version would be supported with bugfixes and security fixes for a longer 24 | period of time than our current life cycle allows. I propose a 2 year LTS window. 25 | 26 | ![Release Timing](diagrams/Salt-Release-Timeline.png) 27 | 28 | 29 | ## Alternatives 30 | [alternatives]: #alternatives 31 | 32 | Change the current life cycle to provide support for less versions and/or for a shorter duration per version. 33 | 34 | ## Unresolved questions 35 | [unresolved]: #unresolved-questions 36 | 37 | - One big question is how long does an LTS version remain supported? I propose a 38 | 2 year support cycle but I could see a case made for 3 years. 39 | 40 | - At what point do we stop supporting OSes that have reached end-of-life status for the LTS version? 41 | 42 | @dmurphy18: I would propose that we stop support as now for OS's which have EOL'd on the LTS Release, 43 | - reasoning being if the OS is not providing FREE security fixes, we should not 44 | either. PAID support for security fixes is another matter and between 45 | customer and VMware, not the Salt Project. 46 | - non-LTS branch - does it get security fixes or do they come in the next 47 | release ?, if we could get releases done quicker than what we have now, I 48 | would go with next release. If we cannot improve release cycle time, then 49 | guess we would have to provide them for non-LTS branch with a minor revision 50 | bump 51 | 52 | # Drawbacks 53 | [drawbacks]: #drawbacks 54 | 55 | I think the core team all agrees this is the best path forward. I'm looking 56 | forward to hearing what drawbacks the broader community can come up with. 57 | -------------------------------------------------------------------------------- /.github/config.yml: -------------------------------------------------------------------------------- 1 | # Configuration for welcome - https://github.com/behaviorbot/welcome 2 | # Configuration for new-issue-welcome - https://github.com/behaviorbot/new-issue-welcome 3 | # Comment to be posted to on first time issues 4 | newIssueWelcomeComment: > 5 | Hello! Thank you for submitting a Salt Enhancement Proposal! Our process is 6 | detailed in the [README.md](https://github.com/saltstack/salt-enhancement-proposals/blob/master/README.md) 7 | and more about the [SEP Life-cycle](https://github.com/saltstack/salt-enhancement-proposals/blob/master/README.md#the-sep-life-cycle). 8 | An Open Core Team member will be assigned to follow up and help guide this 9 | SEP soon and you will find the this in the Community Slack channel #sep. 10 | 11 | Please be sure to review our [Code of Conduct](https://github.com/saltstack/salt/blob/master/CODE_OF_CONDUCT.md). 12 | 13 | You can also check out some of our community 14 | resources: 15 | - [Community Wiki](https://github.com/saltstack/community/wiki) 16 | - [Salt’s Contributor Guide](https://docs.saltstack.com/en/latest/topics/development/contributing.html) 17 | - [Join our Community Slack](https://saltstackcommunity.herokuapp.com/) 18 | - [IRC on LiberaChat](https://web.libera.chat/#salt) 19 | - [SaltStack YouTube channel](https://www.youtube.com/user/SaltStack) 20 | - [SaltStackInc Twitch channel](https://www.twitch.tv/saltstackinc) 21 | 22 | # Comment to be posted to on PRs from first time contributors in your repository 23 | newPRWelcomeComment: > 24 | Hello! Thank you for submitting a Salt Enhancement Proposal! Our process is 25 | detailed in the [README.md](https://github.com/saltstack/salt-enhancement-proposals/blob/master/README.md) 26 | and more about the [SEP Life-cycle](https://github.com/saltstack/salt-enhancement-proposals/blob/master/README.md#the-sep-life-cycle). 27 | An Open Core Team member will be assigned to follow up and help guide this 28 | SEP soon and you will find the this in the Community Slack channel #sep. 29 | 30 | Please be sure to review our [Code of Conduct](https://github.com/saltstack/salt/blob/master/CODE_OF_CONDUCT.md). 31 | 32 | You can also check out some of our community 33 | resources: 34 | - [Community Wiki](https://github.com/saltstack/community/wiki) 35 | - [Salt’s Contributor Guide](https://docs.saltstack.com/en/latest/topics/development/contributing.html) 36 | - [Join our Community Slack](https://saltstackcommunity.herokuapp.com/) 37 | - [IRC on LiberaChat](https://web.libera.chat/#salt) 38 | - [SaltStack YouTube channel](https://www.youtube.com/user/SaltStack) 39 | - [SaltStackInc Twitch channel](https://www.twitch.tv/saltstackinc) 40 | 41 | # Comment to be posted to on pull requests merged by a first time user 42 | firstPRMergeComment: > 43 | Congratulations on your first PR being merged! :tada: 44 | -------------------------------------------------------------------------------- /accepted/0010-pr-merge-requirements.md: -------------------------------------------------------------------------------- 1 | - Feature Name: PR Merge Requirements 2 | - Start Date: 2019-05-21 3 | - SEP Status: Accepted 4 | - SEP PR: https://github.com/saltstack/salt-enhancement-proposals/pull/13 5 | - Salt Issue: (leave this empty) 6 | 7 | # Summary 8 | [summary]: #summary 9 | 10 | Lay out the requirements to merge a PR 11 | 12 | # Motivation 13 | [motivation]: #motivation 14 | 15 | As we begin our working groups, to make sure that our community is all on the same page, we want to 16 | clearly define our requirements to merge a PR. This should benefit developers, reviewers and those 17 | merging the PRs. 18 | 19 | # Design 20 | [design]: #detailed-design 21 | 22 | All PR requirements 23 | - Approval Reviews: 1 approval review from core team member OR 24 | 1 approval review from captain of working group 25 | - All tests pass. 26 | - Cannot merge your own PR until 1 reviewer approves from defined list above that is not the author. 27 | 28 | Bug Fix PR requirements: 29 | - Test Coverage: regression test written to cover bug fix. 30 | - Point to the issue the PR is resolving. If there is not an issue one will need to be created. 31 | 32 | Feature PR requirements: 33 | - Test Coverage: tests written to cover new feature. 34 | - Release Notes: Add note in release notes of new feature for relative release. 35 | - Add `.. versionadded:: ` to module's documentation 36 | 37 | Exceptions: 38 | - Changes that do not require writing tests: documentation, cosmetic changes (ex. log.debug -> log.trace), 39 | fixing tests, pylint, changes outside of the salt directory. 40 | - If an exception needs to be made around the requirement for tests passing or writing a test alongside 41 | a bug or feature it requires 3 approvals before it can be merged. 42 | - If a test is not included in a bug fix or feature and the author cannot write a test, keep the PR open 43 | for 3 months with the “Needs Testcase” label. 44 | 45 | Clarifications: 46 | - Contributers only need to write test coverage for their specific changes. 47 | 48 | Requirements for Merge Access: 49 | - Two factor authentication is required for any user with merge access. 50 | 51 | ## Unresolved questions 52 | [unresolved]: #unresolved-questions 53 | 54 | - Will need to document this if approved and merged. Documentation needs to include everything specified in 55 | this SEP and the following details: 56 | * Document current test infrastructure and how contributors can mirror the test runs to help troubleshoot 57 | test failures 58 | * Clarify where the release notes are located and how to determine the version to include in versionadded. 59 | If it is still not clear the contributor can include TBD and the reviewer would clarify the version. 60 | * Document when integration tests are more applicable over unit tests. 61 | * Document what test coverage still needs to be added. 62 | 63 | # Drawbacks 64 | [drawbacks]: #drawbacks 65 | 66 | Requiring tests and ensuring all tests pass will slow down the PR merge process, but will provide more 67 | stability in salt. 68 | -------------------------------------------------------------------------------- /accepted/0015-black.md: -------------------------------------------------------------------------------- 1 | - Feature Name: blackened-seasoning 2 | - Start Date: 2019-10-04 3 | - SEP Status: Draft 4 | - SEP PR: https://github.com/saltstack/salt-enhancement-proposals/pull/21 5 | - Salt Issue: (leave this empty) 6 | 7 | # Summary 8 | [summary]: #summary 9 | 10 | Adopt `pre-commit`, `black`, and `isort` for Salt. 11 | 12 | # Motivation 13 | [motivation]: #motivation 14 | 15 | There have been a *number* of PRs to Salt that have had minor style changes 16 | requested, either to bring the coding style in to line with the general Salt 17 | project or within the specific module, when it deviates from Salt's typical 18 | standards. 19 | 20 | Additionally there are a number of commits that are produced just to satisfy 21 | the linter. If we automate the formatting of our code, this goes away. We also 22 | achieve consistency throughout the Salt codebase. 23 | 24 | 25 | # Design 26 | [design]: #detailed-design 27 | 28 | ### Black, Pre-commit, and You 29 | 30 | Salt will adopt [Black](https://github.com/psf/black) as our code formatter of 31 | choice. By adopting an opinionated formatter we are taking the approach that 32 | there is only “one way to do it”. Accepting double quotes was extremely 33 | difficult for Tom, but if he can do it, anyone can. 34 | 35 | Salt will also adopt [pre-commit](https://pre-commit.com) in order to automate 36 | the use of Black and lint. As `pre-commit` runs `black`, `isort` (using `force_single_line=True`) and lint 37 | checks, it will allow us all to focus on the logic and design of the PRs 38 | which will increase the stability and quality of Salt. 39 | 40 | #### Lint Names 41 | 42 | Additionally, on the rare occasion that we need to disable lint rules, we will 43 | disable based on name, instead of number. The existing disabled lint numbers 44 | will be changed to the corresponding names as part of the PR that implements 45 | this SEP. 46 | 47 | ### Git Blame? 48 | 49 | Because the introduction of Black will absolutely touch every file in the 50 | codebase (except perhaps a `__init__.py` file or two), a member of the Salt 51 | core team will create that commit using the `--author` flag set to `Salt 52 | Refactor ` so it’s 53 | obvious in `git blame` that those changes are due to `black`ening the codebase. 54 | 55 | This commit hash will also be added to a `.git-blame-ignore-revs` file, so it 56 | can easily be used in the `git blame --ignore-revs-file=.git-blame-ignore-revs` 57 | command, as well as automatically be used with the `git hyper-blame` extension. 58 | 59 | ### Updated Docs 60 | 61 | Adopting these tools for automation will require us to update our documentation 62 | to make it simple for Salt contributors. 63 | 64 | ## Alternatives 65 | [alternatives]: #alternatives 66 | 67 | We could use `sq-black`, a fork of Black that allows single quotes. This 68 | would require maintenance of the fork. 69 | 70 | ## Unresolved questions 71 | [unresolved]: #unresolved-questions 72 | 73 | None? 74 | 75 | # Drawbacks 76 | [drawbacks]: #drawbacks 77 | 78 | We could *not* do this, because it will add some overhead when researching 79 | issues via `git blame`. Especially formatting changes. It will require updating 80 | the documentation. There will also require rebasing of any work done before the 81 | formatting change. 82 | -------------------------------------------------------------------------------- /pull/0000-readd-point-zero.md: -------------------------------------------------------------------------------- 1 | - Feature Name: Add point zero to major releases for Salt 2 | - Start Date: 2021-06-01 3 | - SEP Status: Draft 4 | - SEP PR: (leave this empty) 5 | - Salt Issue: (leave this empty) 6 | 7 | # Summary 8 | [summary]: #summary 9 | 10 | Originally, Salt version numbers were date based as in YYYY.MM.DD and from SEP 14 we moved away from dates and general numbers, and removed the point zero from major release version numbers. This is to add that back to the major versions because it is problematic and inconsistent. 11 | 12 | 13 | # Motivation 14 | [motivation]: #motivation 15 | 16 | Why are we doing this? What use cases does it support? What is the expected outcome? 17 | 18 | When we changed from date-based version numbers to numbered releases with `3000` ideally there isn't a need for point releases, however; it isn't realistic in regards to where the stability of Salt releases are, today, it causes inconsistency in versions for example if there is a Security Release, or critical bug to be fixed, and it causes problems with packaging. The expectation is eventually only have point releases to major versions if there is a Security Release, but to also continue to use the version `3000.0` for major releases as a indication of the major release and to ensure we are not causing issues for the packaging of Salt. 19 | 20 | # Design 21 | [design]: #detailed-design 22 | 23 | Revert changes made to not use the point version on major releases and henceforth and for the foreseable future use points in all releases. 24 | 25 | ## Alternatives 26 | [alternatives]: #alternatives 27 | 28 | What other designs have been considered? This is a reversal so we have tried the alternatives and know it didn't work. 29 | 30 | What is the impact of not doing this? The impact of not making this change is the current pain and suffering we now see from not moving towards an iterative or phased change implementation. We did not forsee the problems with packaging in [SEP 14](https://github.com/saltstack/salt-enhancement-proposals/blob/master/accepted/0014-dev-overhaul.md) implementation, but do now and wish to be kinder to our future selves. 31 | 32 | ## Unresolved questions 33 | [unresolved]: #unresolved-questions 34 | 35 | What parts of the design are still TBD? 36 | Do we need to do this with supported major versions of Salt? If so, how and when? 37 | 38 | # Drawbacks 39 | [drawbacks]: #drawbacks 40 | 41 | Why should we *not* do this? Please consider: 42 | 43 | - Implementation cost, both in term of code size and complexity. This change will revert complexities in the current code base and return it to a previous state, and will cost time and effort to revert the changes, plus change any previous versions, if decided. 44 | - Integration of this feature with other existing and planned features - this shouldn't be an problem, but is likely a risk. 45 | - Cost of migrating existing Salt setups (is it a breaking change?). No this should be a correction to breaking packaging with SEP 14 changes in the first release of Salt with version `3000`. 46 | - Documentation (would Salt documentation need to be re-organized or altered?) Yes, any reference to major release as `3000` instead of `300x.0` such as `3001` `3002` and `3003` this likely will need to be corrected in the .rst files but can be left as-is on any non-code or linked documenation such a blog posts, social media, and the like. 47 | 48 | 49 | There are tradeoffs to choosing any path. Attempt to identify them here. 50 | -------------------------------------------------------------------------------- /pull/0027_Create_Community_Advisory_Board: -------------------------------------------------------------------------------- 1 | # SEP 27: Create a Community Advisory Board 2 | - Feature Name: Create a Community Advisory Board 3 | - Start Date: 2020-09-22 4 | - SEP Status: Abandoned 5 | - SEP PR: https://github.com/saltstack/salt-enhancement-proposals/pull/36 6 | - Salt Issue: (leave this empty) 7 | 8 | ## Summary 9 | To help give the community a voice for sharing its opinions and driving the direction of Salt, we’re establishing a Community Advisory Board. The board will represent the Salt contributor community, meet regularly with the Salt Open team, and influence changes to Salt and its community. 10 | 11 | ## Motivation 12 | The Salt community wants to be more involved in decisions that impact the Salt open project. Currently, this process is ad hoc based on who is in attendance at Open Hours or is vocal in Slack, IRC, or the comments of issues and PRs. Community members often miss change announcements or don’t realize that decisions that impact them are being made. They want a say in Salt Enhancement Proposals (SEPs), Working Groups, technical decisions, and best practices. 13 | Creating an Advisory Board will bring structure and process to influencing the Salt project. It will give community members a voice to discuss, agree, and disagree with proposed changes and decisions. Since the board will meet on a regular cadence, board members and those they represent will be able to plan, prepare for, and prioritize proposed changes. 14 | The Advisory Board’s main goal will be to grow and improve community collaboration and participation. Its members will work with users, contributors, and the Salt Core team. It will not be a replacement for current contributor involvement opportunities, but a supplement to those opportunities. 15 | 16 | ## Design 17 | 18 | ### Board Logistics 19 | - The board will be made up of 5 members from diverse technical areas and geographic locations of the Salt contributor community. 20 | - The board will meet via video call with the SaltStack Core team every month. 21 | - Meetings will be moderated by the Community Manager. 22 | - Meeting agendas will be set and shared prior to meetings. 23 | - Board meetings will be recorded and require someone to create meeting minutes that will be shared with the greater Salt community. 24 | 25 | ### Membership 26 | - Membership will be determined by the combined opinions of the community and the Salt Core team. 27 | - All active contributors are eligible to join the Advisory Board 28 | - Anyone who is interested in joining the board can nominate themselves. They will need to get 3 references from other contributors. The Salt Core team will weigh this feedback when determining who to join the board. 29 | 30 | ### Expectations 31 | - All Advisory Board members must adhere to the Contributor Code of Conduct. 32 | - Members will discuss and influence the following areas: SEPs, Working Groups, technical decisions, and best practices. 33 | - The anticipated time commitment for board membership is 3-5 hours per month. 34 | 35 | ## Alternatives 36 | - Maintain the community-core team interactions as they are. 37 | - Create another formal venue for contributors to influence the project. 38 | 39 | ## Unresolved questions 40 | - How frequently should the board meet? 41 | - Who are candidates to join the board? 42 | - How will the board keep its projects prioritized and organized? 43 | - How long is a board member’s tenure? 44 | - What is the timeline for sharing agendas before and minutes after the meetings? 45 | - What will be the main communication platform for board members? 46 | - How will the board decide and prioritize actitivties and projects? 47 | 48 | ## Drawbacks 49 | - Managing an Advisory Board will be time-consuming, especially in its early stages. 50 | - The board may not accurately represent contributors or their wishes. 51 | - The board may not execute on its commitments. 52 | -------------------------------------------------------------------------------- /abandoned/0027_Create_Community_Advisory_Board: -------------------------------------------------------------------------------- 1 | # SEP 27: Create a Community Advisory Board 2 | - Feature Name: Create a Community Advisory Board 3 | - Start Date: 2020-09-22 4 | - SEP Status: Abandoned 5 | - SEP PR: https://github.com/saltstack/salt-enhancement-proposals/pull/36 6 | - Salt Issue: (leave this empty) 7 | 8 | ## Summary 9 | To help give the community a voice for sharing its opinions and driving the direction of Salt, we’re establishing a Community Advisory Board. The board will represent the Salt contributor community, meet regularly with the Salt Open team, and influence changes to Salt and its community. 10 | 11 | ## Motivation 12 | The Salt community wants to be more involved in decisions that impact the Salt open project. Currently, this process is ad hoc based on who is in attendance at Open Hours or is vocal in Slack, IRC, or the comments of issues and PRs. Community members often miss change announcements or don’t realize that decisions that impact them are being made. They want a say in Salt Enhancement Proposals (SEPs), Working Groups, technical decisions, and best practices. 13 | Creating an Advisory Board will bring structure and process to influencing the Salt project. It will give community members a voice to discuss, agree, and disagree with proposed changes and decisions. Since the board will meet on a regular cadence, board members and those they represent will be able to plan, prepare for, and prioritize proposed changes. 14 | The Advisory Board’s main goal will be to grow and improve community collaboration and participation. Its members will work with users, contributors, and the Salt Core team. It will not be a replacement for current contributor involvement opportunities, but a supplement to those opportunities. 15 | 16 | ## Design 17 | 18 | ### Board Logistics 19 | - The board will be made up of 5 members from diverse technical areas and geographic locations of the Salt contributor community. 20 | - The board will meet via video call with the SaltStack Core team every month. 21 | - Meetings will be moderated by the Community Manager. 22 | - Meeting agendas will be set and shared prior to meetings. 23 | - Board meetings will be recorded and require someone to create meeting minutes that will be shared with the greater Salt community. 24 | 25 | ### Membership 26 | - Membership will be determined by the combined opinions of the community and the Salt Core team. 27 | - All active contributors are eligible to join the Advisory Board 28 | - Anyone who is interested in joining the board can nominate themselves. They will need to get 3 references from other contributors. The Salt Core team will weigh this feedback when determining who to join the board. 29 | 30 | ### Expectations 31 | - All Advisory Board members must adhere to the Contributor Code of Conduct. 32 | - Members will discuss and influence the following areas: SEPs, Working Groups, technical decisions, and best practices. 33 | - The anticipated time commitment for board membership is 3-5 hours per month. 34 | 35 | ## Alternatives 36 | - Maintain the community-core team interactions as they are. 37 | - Create another formal venue for contributors to influence the project. 38 | 39 | ## Unresolved questions 40 | - How frequently should the board meet? 41 | - Who are candidates to join the board? 42 | - How will the board keep its projects prioritized and organized? 43 | - How long is a board member’s tenure? 44 | - What is the timeline for sharing agendas before and minutes after the meetings? 45 | - What will be the main communication platform for board members? 46 | - How will the board decide and prioritize actitivties and projects? 47 | 48 | ## Drawbacks 49 | - Managing an Advisory Board will be time-consuming, especially in its early stages. 50 | - The board may not accurately represent contributors or their wishes. 51 | - The board may not execute on its commitments. 52 | -------------------------------------------------------------------------------- /accepted/0017-job-ack-event.md: -------------------------------------------------------------------------------- 1 | - Feature Name: job-ack-event 2 | - Start Date: 2019-10-24 3 | - SEP Status: Accepted 4 | - SEP PR: #23 5 | - Salt Issue: (leave this empty) 6 | 7 | # Summary 8 | [summary]: #summary 9 | 10 | Trigger job acknowledge events from minion, before it starts executing the job. 11 | 12 | # Motivation 13 | [motivation]: #motivation 14 | 15 | Currently, there seems to be no possibilty to detect if Minion successfully obtained job event. Although there is `saltutil.running`, which could be used to determine it, it creates the same problem as it's another job execution. 16 | 17 | The main focus is for integrations to determine whether the job has been started on minion, or if it went stale (e.g. minion is not executing the job although it should), so it can be safely retried. 18 | 19 | There are existing ideas how to achieve job retries (such as [RFC 0003 - Job retry](https://github.com/saltstack/salt/blob/develop/rfcs/0003-job-retry.md)), but the approach is different. 20 | 21 | 22 | # Design 23 | [design]: #detailed-design 24 | 25 | Overall idea is to introduce new job event type `salt/job//ack/`. The event would be fired by Salt Minion immediatelly after it receives new job. 26 | This event could be intercepted by Salt engines / returners, so integrated services would know whether the job was really received and is being executed. 27 | 28 | The aim of this SEP is to introduce this new event type, not to implement job retries as such, as the retry scope relies on all minions firing this event. 29 | 30 | Because it might be undesirable for minions to fire more events than necessary, minion-sided config option could be introduced to enable/disable the feature. 31 | Due to the design of this SEP, the implementation should be agnostic to any underlying transport layer. 32 | 33 | 34 | Overall implementation could be as simple as 35 | 36 | ```diff 37 | class Minion(MinionBase): 38 | @classmethod 39 | def _thread_return(cls, minion_instance, opts, data): 40 | ''' 41 | This method should be used as a threading target, start the actual 42 | minion side execution. 43 | ''' 44 | minion_instance.gen_modules() 45 | fn_ = os.path.join(minion_instance.proc_dir, data['jid']) 46 | 47 | salt.utils.process.appendproctitle('{0}._thread_return {1}'.format(cls.__name__, data['jid'])) 48 | 49 | sdata = {'pid': os.getpid()} 50 | sdata.update(data) 51 | log.info('Starting a new job %s with PID %s', data['jid'], sdata['pid']) 52 | 53 | + # send ack 54 | + acktag = tagify([data['jid'], 'ack', opts['id']], 'job') 55 | + minion_instance._fire_master(tag=acktag) 56 | ``` 57 | 58 | With similar adjustments to metaproxy minion if necessary. The final event could look like this 59 | 60 | ``` 61 | salt/job/20191010142457031142/ack/test-minion.tld { 62 | "_stamp": "2019-10-10T14:24:57.414999", 63 | "cmd": "_minion_event", 64 | "data": {}, 65 | "id": "test-minion.tld", 66 | "pretag": null, 67 | "tag": "salt/job/20191010142457031142/ack/test-minion.tld" 68 | } 69 | ``` 70 | 71 | ## Alternatives 72 | [alternatives]: #alternatives 73 | 74 | Another option to achieve this is to implement detection into transport protocol itself, relying on whether we can safely determine if the event was received by minion. However we would be only able to tell if event was received, not that job is starting and for various reasons, this might be impossible to implement. 75 | 76 | ## Unresolved questions 77 | [unresolved]: #unresolved-questions 78 | 79 | Should the triggered event be part of `_return` cmd, some new one, or is `_minion_event` enough? 80 | 81 | # Drawbacks 82 | [drawbacks]: #drawbacks 83 | 84 | Main drawback might be pushing another event to Salt Master even if it's not necessary for many users. For this reason the firing could be optional with specific config option. 85 | -------------------------------------------------------------------------------- /accepted/33-readd-point-zero.md: -------------------------------------------------------------------------------- 1 | - Feature Name: Add point zero to major releases for Salt 2 | - Start Date: 2021-06-01 3 | - SEP Status: Accepted 4 | - SEP PR: https://github.com/saltstack/salt-enhancement-proposals/pull/49 5 | - Salt Issue: (leave this empty) 6 | 7 | # Summary 8 | [summary]: #summary 9 | 10 | Originally, Salt version numbers were date based as in YYYY.MM.DD and from SEP 14 we moved away from dates and general numbers, and removed the point zero from major release version numbers. This is to add the standard format back to the major versions because it is problematic and inconsistent. 11 | 12 | 13 | # Motivation 14 | [motivation]: #motivation 15 | 16 | Why are we doing this? What use cases does it support? What is the expected outcome? 17 | 18 | When we changed from date-based version numbers to numbered releases with `3000` ideally there isn't a need for point releases, however; it isn't realistic in regards to where the stability of Salt releases are, today, it causes inconsistency in versions for example if there is a Security Release, or critical bug to be fixed, and it causes problems with packaging. The expectation is eventually only have point releases to major versions if there is a Security Release, but to also continue to use the version `3000.0` for major releases as a indication of the major release and to ensure we are not causing issues for the packaging of Salt. 19 | 20 | # Design 21 | [design]: #detailed-design 22 | 23 | Revert changes made to not use the point version on major releases and henceforth and for the foreseable future use points in all releases. 24 | 25 | ## Alternatives 26 | [alternatives]: #alternatives 27 | 28 | What other designs have been considered? This is a reversal so we have tried the alternatives and know it didn't work. 29 | 30 | What is the impact of not doing this? The impact of not making this change is the current pain and suffering we now see from not moving towards an iterative or phased change implementation. We did not forsee the problems with packaging in [SEP 14](https://github.com/saltstack/salt-enhancement-proposals/blob/master/accepted/0014-dev-overhaul.md) implementation, but do now and wish to be kinder to our future selves. 31 | 32 | ## Unresolved questions 33 | [unresolved]: #unresolved-questions 34 | 35 | What parts of the design are still TBD? 36 | Do we need to do this with supported major versions of Salt? If so, how and when? Updated: 2021-JUN-22 Answer: No, it is not the intent of this SEP to revisit the past releases, but to set for the standard operating procedure and the process going forward. If warranted to re-release a major version of Salt originally released as `####` without the point zero, likely it would need separate analysis and discussion. Today, and for this SEP it is to adjust the processs for future major releases, only. 37 | 38 | # Drawbacks 39 | [drawbacks]: #drawbacks 40 | 41 | Why should we *not* do this? Please consider: 42 | 43 | - Implementation cost, both in term of code size and complexity, exist. This change will revert complexities in the current code base and return it to a previous state, and will cost time and effort to revert the changes. 44 | - Integration of this feature with other existing and planned features - this shouldn't be an problem, but is likely a risk. 45 | - Cost of migrating existing Salt setups (is it a breaking change?). No this should be a correction to breaking packaging with SEP 14 changes in the first release of Salt with version `3000` but isn't an attempt to re-release. 46 | - Documentation (would Salt documentation need to be re-organized or altered?) Yes, any reference to major release as `3000` instead of `300x.0` such as `3001` `3002` and `3003` this likely will need to be corrected in the .rst files but can be left as-is on any non-code or linked documenation such a blog posts, social media, and the like. 47 | - Inconsisencies will exist and this SEP is one attempt at mitigation, likely there are other ways to improve and we can cite this SEP in all documentation and likely write a blog post or other content, as well. 48 | - Unknown risks are common and likely and can be added once identified. 49 | 50 | 51 | There are tradeoffs to choosing any path. Attempt to identify them here. 52 | -------------------------------------------------------------------------------- /accepted/0003-more-sep-statuses.md: -------------------------------------------------------------------------------- 1 | - Feature Name: More SEP Statuses 2 | - Start Date: 2019-03-18 3 | - SEP Status: Accepted 4 | - SEP PR: https://github.com/saltstack/salt-enhancement-proposals/pull/6 5 | 6 | # Summary 7 | [summary]: #summary 8 | 9 | Enhancing SEP statuses 10 | 11 | # Motivation 12 | [motivation]: #motivation 13 | 14 | In discussions about legacy RFCs and existing SEPs, it's become evident that we 15 | need a few more statuses for our SEPs: 16 | 17 | - **Abandoned** 18 | - **Up for Grabs** 19 | 20 | The primary motivator is that we have some SEPs (and legacy RFCs) that are generally 21 | accepted as fundamentally good ideas, but they haven't built up enough momentum 22 | to get to a point where we're happy with the SEP. Rather than leaving the PRs 23 | just cluttering up the space, we'd like to close them without prejudice. The 24 | `Abandonded` status would indicate that: 25 | 26 | 1. They need more work 27 | 2. Are open for anyone who wants to champion the idea 28 | 29 | As the original champion either ran out of interest, motivation, or perhaps 30 | free time. 31 | 32 | For the `Up for Grabs` status, this would indicate that while the SEP has been 33 | accepted, there is no current implementation champion. As the SEP process 34 | states: 35 | 36 | > Note that acceptance does not mean that the feature will be immediately 37 | > implemented, or that it will be implemented at all; It merely means that the 38 | > core development team has agreed to it in principle. 39 | 40 | When a SEP is accepted, and the original champion indicates they won't 41 | be able to implement it either explicitly through comments or messages via some 42 | other medium, or simply through inaction, the SEP will be marked as "Up for 43 | Grabs", meaning that whoever would like to actually implement the SEP is 44 | welcome to do it without stepping on anyone's toes, or repeating work. 45 | 46 | # Design 47 | [design]: #detailed-design 48 | 49 | ## Abandoned SEPs 50 | 51 | Abandoned SEPs, if they are renewed by the original author, will re-enter at 52 | the same place in the workflow where they left off. If they were eligible to 53 | enter the **Final Comment** status, they will still be eligible. If they were 54 | abandoned during the **Final Comment** phase, they will still be in **Final 55 | Comment** phase when they are renewed. 56 | 57 | If they are adopted by a new author after being abandoned, they will restart 58 | the process, requiring at least two (2) weeks initial comment period. The 59 | exception to this rule is if the original author hands off to another champion 60 | in the comments on the SEP. Something to the effect of "So and so can take over 61 | on this for me." For instance where there are more than one champion and there 62 | are foreseeable circumstances, this should minimize the disruption in the SEP 63 | process. 64 | 65 | ## Up for Grabs 66 | 67 | As previously mentioned, approval of an SEP is no guarantee of implementation. 68 | For SEPs where the original author also has a desire and ability to see the 69 | feature through to completion, it seems natural that they will do so. There may 70 | be cases where the author lacks either the desire or the ability to finish a 71 | feature. In most cases, the author should indicate this at some point before 72 | the SEP is merged, at which point it will be **Accepted** with the additional 73 | **Up for Grabs** status. 74 | 75 | In some cases, the original champion may not be able to implement the SEP, and 76 | may become unresponsive. After an inactive period of two weeks (14 calendar 77 | days), if the SEP author fails to communicate on a SEP, or push code on a SEP 78 | related branch, the SEP will be considered **Up for Grabs** and anyone who 79 | desires to step up and champion the development will be more than welcome. 80 | 81 | 82 | ## Alternatives 83 | [alternatives]: #alternatives 84 | 85 | We could leave stale SEPs in open status, or reject them. As far as has been 86 | discussed, neither approach is ideal. Rejecting them could cause confusion, 87 | since that's an indication that we don't actually want that feature. Leaving 88 | them open is also probably not ideal, as there's constantly a mental step 89 | (however small) of reviewing them when looking through the SEPs. 90 | 91 | ## Unresolved questions 92 | [unresolved]: #unresolved-questions 93 | 94 | Is two weeks for **Up for Grabs** too long? Too short? 95 | 96 | How long without comment before we mark an in-progress SEP as abandoned? 97 | 98 | # Drawbacks 99 | [drawbacks]: #drawbacks 100 | 101 | Maybe not any? 102 | -------------------------------------------------------------------------------- /accepted/0008-python3-ssh-template.md: -------------------------------------------------------------------------------- 1 | - Feature Name: Python 3 Only Salt-SSH 2 | - Start Date: 2019-05-01 3 | - SEP Status: Draft 4 | - SEP PR: https://github.com/saltstack/salt-enhancement-proposals/pull/11 5 | - Salt Issue: (leave this empty) 6 | 7 | # Summary 8 | [summary]: #summary 9 | 10 | Since Python 2 will be deprecated in 2020 we need to figure out a solution for Salt-SSH 11 | to run on machines regardless of the Python version on the targeted system. 12 | 13 | # Motivation 14 | [motivation]: #motivation 15 | 16 | Python 2 will be deprecated in 2020 and Salt plans on supporting only Python 3 in the 17 | Sodium release or later. Due to this deprecation and migration to Python 3 we will 18 | need to find a specific solution for Salt-SSH to continue to work against hosts 19 | that still have only Python 2 installed, since by default Salt-SSH only works against 20 | hosts that have the same python version. 21 | 22 | # Design 23 | [design]: #detailed-design 24 | 25 | The design proposal to allow Salt-SSH to work with Python 3, includes multiple options: 26 | 1. By default, if python versions match we will simply use the system python and Salt-SSH 27 | will run as it always has. 28 | 2. Add a Salt-SSH configuration (for example: pre_flight) to run raw commands before the 29 | Salt-SSH command. This will allow a user to create a custom script they can run to install 30 | Python 3. It will only run these pre_flight commands if the tarball is not copied over. 31 | 3. Update the current ssh_ext_alternatives feature in Salt-SSH to automatically attempt to 32 | find the necessary python libraries that Salt-SSH uses to copy over and check to see if 33 | they are importable. If a user needs additional libraries other than the default they will 34 | need to add those manually to the configuration. 35 | 4. Maintain and build a Python 3 binary that will be copied over with the tarball to be included 36 | in the salt-call run. This will also be gated by a config option that a user will need to enable 37 | if they always want to use the binary option. 38 | 39 | If the python versions do not match when Salt-SSH is run a message will be printed out asking the user 40 | if they want to copy over the binary, and informing them of the other alternative options pre_flight and 41 | ssh_ext_alternatives. 42 | 43 | 44 | How are we going to build the Python 3 binary? 45 | We will build the binary statically for both x86_64 and ARM architecture. We will include the x86_64 46 | binary by default in the Salt-SSH package. If the ARM architecture is detected we will include a warning 47 | to the user to download the binary. To make this download or upgrade of the binary easier, a switch/arg 48 | will be added to the salt-ssh cli tool that will handle this logic for both binaries. We will include the 49 | required libraries to run salt-call and include the pip binary. If users want to include other dependencies 50 | on top of this binary they can use the ssh_ext_alternatives feature to include the additional dependencies. 51 | 52 | Python Binary Security Releases: 53 | We will need to monitor and make sure our Python 3 binary and other built libraries are kept up to date 54 | with any security releases. Since the Python 3 binary will be managed outside of the salt repo we can do 55 | releases outside of the Salt release cycle to include these new patches and updates quickly. We will add 56 | a tool to help monitor if there are vulnerabiltiies in our dependencies. Either [pyup](https://pyup.io) or 57 | [github security alerts](https://help.github.com/en/articles/about-security-alerts-for-vulnerable-dependencies) 58 | 59 | Python 3 Binary Test Plan: 60 | We will need to make sure this is thoroughly tested. We will need to ensure that changes to Salt work with 61 | the Salt vendored Python 3. We will also need to ensure that changes to the Salt vendored Python 3 work with 62 | the existing Salt branches. We will do this by building automated tests against the Salt vendored Python 3 63 | for each Salt branch and supported OS. 64 | 65 | 66 | ## Alternatives 67 | [alternatives]: #alternatives 68 | 69 | Alternatives for Python 3 static binary: 70 | - Use an already maintained minimal version of Python 3, but we will not have as much control over what is added into the binary. 71 | - Use a dynamic build, but this will require building out a package per distro. 72 | 73 | ## Unresolved questions 74 | [unresolved]: #unresolved-questions 75 | 76 | When we build the static python build we will need to review the licenses of all the libraries included in the build. 77 | 78 | 79 | # Drawbacks 80 | [drawbacks]: #drawbacks 81 | 82 | Why should we *not* do this? Please consider: 83 | 84 | - Adding anything additional to the tarball can impact performance of Salt-SSH. We will need to make sure that 85 | the addition of this Python 3 binary is as small as it possibly can be for the current Salt-SSH run. We should 86 | also be using similar hashing comparisons so that the Python 3 binary is only copied over once. 87 | -------------------------------------------------------------------------------- /accepted/0018-release-label-improvements.md: -------------------------------------------------------------------------------- 1 | - Feature Name: release-label-improvements 2 | - Start Date: 2020-01-13 3 | - SEP Status: Accepted 4 | - SEP PR: https://github.com/saltstack/salt-enhancement-proposals/pull/24 5 | 6 | # Summary 7 | [summary]: #summary 8 | 9 | The current GitHub labels are complicated and not very intuitive. This SEP is 10 | designed to simplify the way we use labels to track what should make it into a 11 | release, and clarify the high-level purpose of specific labels. 12 | 13 | # Motivation 14 | [motivation]: #motivation 15 | 16 | In community and internal discussions regarding labels, it's become clear that 17 | we need a more streamlined approach to labels and their release priority. 18 | Complicated, unclear labeling is confusing for contributors and consumers of 19 | the Salt project. 20 | 21 | We combine labels such as P1 (everyone will see this issue) with Medium 22 | Severity (it’s just cosmetic). Or we see Critical (data loss) with P4 (not many 23 | people will see it). These pairings are confusing, even if they’re technically 24 | correct. 25 | 26 | # What This Is Not 27 | 28 | This SEP is not concerned with labels regarding functional areas, functional 29 | groups, test status, or other labels. It is also not concerned with being an 30 | exhaustive list of criteria for labels. It also does not attempt to address the 31 | entire development and triage process. 32 | 33 | # Design 34 | [design]: #detailed-design 35 | 36 | Based on the discussions we've had, there are three main perspectives that are 37 | confused about our current labels in regards to the release process. 38 | 39 | As bug reporter (or someone with the same issue), I just want to know: **when 40 | will my bug be fixed?** In other words, what major or point release will 41 | solve my problem? 42 | 43 | As a contributor, I want to know: **what should I be working on next**? 44 | 45 | From a release manager point of view, I want to know: **is there anything that 46 | still needs to be done before the next release**? 47 | 48 | This SEP will address the labels that can answer these questions. 49 | 50 | ## Description 51 | 52 | Going forward, there will be four (4) priority labels that will make it easier 53 | to know when an issue can be expected in a release. 54 | 55 | The highest priority contains **only issues that are severe enough to block a 56 | release**. These issues affect most or all users within a functional area, 57 | cause data loss, prevent Salt from starting, cause salt to crash, or have 58 | similarly negative effects. These issues will block a release until corrected 59 | or determined to be a lower priority. This label will be **Critical**. 60 | 61 | The next highest priority is for issues that are serious issues, but **not 62 | serious enough to be considered a release blocker**. These affect a many or 63 | most users in the functional area, may cause data loss, crashes or hangs, 64 | and/or makes the system unresponsive. This label will be **High** 65 | 66 | The next highest priority is for issues that affect about half the users in a 67 | functional area, producing incorrect functionality, bad functionality, a 68 | confusing user experience, or a confusing error message. This label will be 69 | **Medium**. 70 | 71 | The final label applies to issues that affect few users within a functional 72 | area, are limited to corner/edge cases, especially when a workaround exists, 73 | and also includes cosmetic fixes such as spelling, formatting, and colors. This 74 | label will be **Low** 75 | 76 | To recap, the new labels will be: 77 | 78 | - **Critical** - release blockers, serious issues. 79 | - **High** - serious issues, but not release blockers. 80 | - **Medium** - painful but not serious issues, especially ones that lack a 81 | reasonable workaround. 82 | - **Low** - cosmetic issues, or issues that have a simple/reasonable 83 | workaround. 84 | 85 | https://github.com/saltstack/salt/blob/master/doc/topics/development/labels.rst 86 | will be updated to reflect these new labels. 87 | 88 | These labels will replace the following labels: 89 | 90 | - Blocker 91 | - Critical 92 | - High Severity 93 | - Medium Severity 94 | - P1 95 | - P2 96 | - P3 97 | - P4 98 | 99 | ## How Does This Fix the Problem(s)? 100 | 101 | Moving to these labels will simplify the number of labels required. It will 102 | also make it easy to see what issues are certain to make it into the next 103 | release, and which issues are *not* the focus for the core team. 104 | 105 | ## Alternatives 106 | [alternatives]: #alternatives 107 | 108 | ### Keep It Up 109 | 110 | We could keep doing what we’ve been doing, and simply do better at publicizing 111 | / documenting the changes. 112 | 113 | ### Different Colors of Bikeshed 114 | 115 | We could use other label names, such as “Blocker”, “Major”, “Minor”, 116 | “Cosmetic”. 117 | 118 | # Drawbacks 119 | [drawbacks]: #drawbacks 120 | 121 | We have to update documentation and re-train ourselves. But even with current 122 | documentation, it seems like there are enough complications that we *still* 123 | would need to train ourselves. By simplifying things it should be easier. We 124 | may need to go through and re-label existing issues. 125 | -------------------------------------------------------------------------------- /accepted/package-salt-with-tiamat.md: -------------------------------------------------------------------------------- 1 | - Feature Name: Use Tiamat built packages for distribution of salt 2 | - Start Date: 2020-06-12 3 | - SEP Status: Accepted 4 | - SEP PR: https://github.com/saltstack/salt-enhancement-proposals/pull/34 5 | - Salt Issue: (leave this empty) 6 | 7 | # Summary 8 | [summary]: #summary 9 | 10 | Instead of packaging salt with dependencies on the system’s python and python library versions, we use 11 | [tiamat](https://gitlab.com/saltstack/pop/tiamat) to build the packages. 12 | 13 | 14 | # Motivation 15 | [motivation]: #motivation 16 | 17 | Tiamat bundles all libraries with the package. This allows us to package for a much larger number of distros, switch 18 | vendors, oses, etc. without worrying about the large differences in library versions between them. 19 | 20 | It also means that testing is much easier as there are fewer combinations of libraries, oses, and distros that need to 21 | be tested. There is more of a guarantee that we test the same thing that gets deployed to end users. Essentially, 22 | rather than testing a combination of installed libraries and their particular versions on an OS, you just have to test 23 | on the OS. 24 | 25 | Another benefit is that because we don’t have to worry about building the dependencies for all of the oses, it makes it 26 | easier to build the packages in a pipeline. The way package building is currently done is hard to automate because 27 | every time a new dependency is added or adjusted, we have to then build that dependency if the os doesn’t have it 28 | already. Because there are no dependencies on the system’s python libraries, it would now be more feasible to have 29 | salt packages be built on every PR. 30 | 31 | It also means that for salt-ssh, we will no longer rely on the end system having python installed as we can include 32 | python, and all relevant libraries with the tarball that salt-ssh deploys. 33 | 34 | It also means that we can have a single binary as an option (similar to a go binary) that can be used for use cases 35 | that it would be helpful in. 36 | 37 | 38 | # Design 39 | [design]: #detailed-design 40 | 41 | [Tiamat](https://gitlab.com/saltstack/pop/tiamat) is a wrapper around [pyinstaller](https://www.pyinstaller.org/) which 42 | allows building a python environment that includes python, all needed python libraries, and all c libraries. This 43 | means that we can control all dependencies without interfering with any other software on the system. 44 | 45 | We would not use Tiamat for Windows or Mac since they already have a python environment bundled with their installers. 46 | It doesn't make sense to disrupt people already familiar with how those packages work when they are already fast to 47 | build, and we have the same control over the python environment that we would with Tiamat packages. 48 | 49 | We would build packages for each OS in the same packaging method we always have (exe, msi, rpm, deb, pkg). For the 50 | Linux distros we support, we would build them on containers matching the os version we intend to support. See 51 | https://gitlab.com/saltstack/open/cicd/containers for those. This solves problems with glibc compatibility. 52 | 53 | Because we have no dependencies, we can build whenever we want without requiring manual effort to add packages for 54 | dependencies. The effort required for maintenance of the packaging would be reduced tremendously. Because of this, we 55 | would be able to do nightly builds via ci. See https://gitlab.com/saltstack/open/salt-pkg for more on that. This also 56 | means that could build on every PR and have an artifact of the pipeline be packages that could be installed. We don’t 57 | intend on signing them on PR. 58 | 59 | The resulting packages from nightly builds would go on artifactory in the 60 | [open-debian-staging](https://artifactory.saltstack.net/ui/repos/tree/General/open-debian-staging) and 61 | [open-rpm-staging](https://artifactory.saltstack.net/ui/repos/tree/General/open-rpm-staging) repos. This would allow 62 | anyone to install from the nightly builds in order to test actual packages during the release cycle. 63 | 64 | The combination of building on every PR and nightly builds would dramatically increase the ability of the community to 65 | test salt on their own machines during the release cycle. 66 | 67 | Tiamat would be tested that it can build at a minimum, salt, before we release new versions of Tiamat. We would 68 | ideally test that other applications can be built as well before release. 69 | 70 | ## Alternatives 71 | [alternatives]: #alternatives 72 | 73 | - We keep packaging the way we are which means we have added slowness in our release cycle and have a harder time 74 | testing the right versions of libraries. 75 | 76 | ## Unresolved questions 77 | [unresolved]: #unresolved-questions 78 | 79 | - Will we package arm64 versions now that packages are architecture specific? 80 | 81 | 82 | # Drawbacks 83 | [drawbacks]: #drawbacks 84 | 85 | - It would be a learning curve for people to understand how to add python libraries to the new environment. 86 | - Adding python libraries to the new environment might be harder in an air gapped network. 87 | - People would have to re-install their python packages potentially if we updated the python in the tiamat environment 88 | on upgrades. 89 | -------------------------------------------------------------------------------- /inclusive-language.md: -------------------------------------------------------------------------------- 1 | - Feature Name: SEP34: Inclusive Language 2 | - Start Date: 2021-07-06 3 | - SEP Status: Draft 4 | - SEP PR: https://github.com/saltstack/salt-enhancement-proposals/pull/54 5 | - Salt Issue: (leave this empty) 6 | 7 | # Summary 8 | [summary]: #summary 9 | 10 | The Salt Project has always sought to be inclusive and not offensive. Original designs and names used were done so 11 | with the intent of being inclusive. Over the last decade this landscape has evolved and certain changes need to be made. 12 | 13 | This SEP proposes how we can make a solid design that can be sustained as this landscape continues to evolve. 14 | 15 | This SEP is intended to supersede SEP #28 16 | 17 | # Motivation 18 | [motivation]: #motivation 19 | 20 | The use of inclusive language needs to be a continual discussion, but the project also needs to document 21 | why certain choices were made and the liabilities that come from specific changes to language. 22 | 23 | Understanding the intent derived from original language choices, as well as formalizing what have previously 24 | been informal policies will allow us to maintain a clean situation moving forward. 25 | 26 | # Design 27 | [design]: #detailed-design 28 | 29 | In 2020 a very valid surge of concern around inclusive language surfaced and has touched many aspects 30 | of society. It has also highlighted that our collective understating of inclusive language is 31 | evolving. The Salt Project has no intention of perpetuating language, practices, culture, or code 32 | that is offensive or non-inclusive. 33 | 34 | The evolving nature of the problem issues a concern on how to best manage this situation in the 35 | long term. The Salt Project is not staffed internally to adequately address these concerns. We 36 | also feel that the creation of a single static document to track this effort is not adequate. 37 | 38 | Fortunately, the primary sponsor of The Salt Project - VMware - maintains a significant effort to 39 | maintain inclusion and diversity. This SEP proposes that we strive to adhere to the guidelines 40 | presented by VMware for inclusive language. 41 | 42 | Details on the Diversity, Equity, and Inclusion efforts of VMware can be found here: 43 | 44 | https://www.vmware.com/company/diversity.html 45 | 46 | Fortunately, the Salt Project already adheres to the vast majority of rules set forth by 47 | the VMware Diversity, Equity, and Inclusion policies. Only a few things need to be addressed. 48 | 49 | Please note that these topics have been discussed at length and are actively being addressed. 50 | 51 | While we share a strong desire to adhere to inclusive language, we should not break user 52 | installs. We feel that if this initiative were to create added and onerous work, confusion, and/or 53 | frustration for our users then it could backfire in the overall intent to create a more understanding 54 | and peaceful world. This means that standard deprecation paths will be adhered to for changes and 55 | the impact of a change will need to be carefully considered. 56 | 57 | ## Whitelist/Blacklist 58 | 59 | The primary change that needs to be made, and has already started to be executed, is to 60 | remove references to Whitelist and Blacklist. 61 | 62 | The usage of whitelist will be replaced with allowlist 63 | The usage of blacklist will be replaced with blocklist 64 | 65 | ## Usage of the term "Master" 66 | Salt uses the term "Master" extensively to refer to the system which issues commands out to Salt Minions. 67 | I (Thomas Hatch) chose the term Master/Minion to refer to this process with the original intent of making 68 | the choice inclusive. The Salt Project team has extensively evaluated the usage of the term "Master" and has 69 | concluded that any attempt to change the term in its entirety would be deeply detrimental to our users. 70 | Due to the deep and extensive usage of the term "Master" we feel that an outright change would not be possible. 71 | Changing the term "Master" would break nearly every install of Salt worldwide and would therefore be detrimental 72 | to the project and, I feel, to the intent of the movement. 73 | 74 | We also feel that the term Master/Minion is non-derogatory due to the fact that the word Master is not paired 75 | with a derogatory term for a subservient entity. This was the original intent for the choice made in our verbiage. 76 | 77 | With this said, we feel that forcing the ongoing usage of the term "Master" in presentation 78 | materials will potentially be an issue. Therefore we propose adding a new startup script allowing the Salt Master 79 | to be started up via an alternative name. This alternative name will also be available for use in presentations. 80 | 81 | ## Master Git Branch 82 | 83 | The "master" branch used in git will be changed to "main". 84 | 85 | ## Interacting With Existing Systems 86 | 87 | Many systems currently exist which do not use inclusive language. In instances where the interaction or 88 | management of underlying systems does not use inclusive language, Salt will use the language of the underlying, 89 | managed system. 90 | 91 | ## Alternatives 92 | [alternatives]: #alternatives 93 | 94 | Many alternatives have been discussed previously and have been deemed untenable. These include maintaining 95 | a project level inclusiveness program as well as complete renames of the Salt Master. We feel strongly that 96 | these alternatives cannot be reliably executed given our available resources and goals. 97 | 98 | ## Unresolved questions 99 | [unresolved]: #unresolved-questions 100 | 101 | The main unresolved question of the alternative name that will be allowed for the Salt Master. 102 | 103 | # Drawbacks 104 | [drawbacks]: #drawbacks 105 | 106 | Drawbacks have been listed in this document for consideration. 107 | -------------------------------------------------------------------------------- /accepted/0005-retire-py2-support.md: -------------------------------------------------------------------------------- 1 | - Feature Name: Retiring Python 2 Support 2 | - Start Date: 2019-03-25 3 | - SEP Status: Accepted 4 | - SEP PR: https://github.com/saltstack/salt-enhancement-proposals/pull/8 5 | - Salt Issue: (leave this empty) 6 | 7 | 8 | 9 | # Summary 10 | [summary]: #summary 11 | 12 | In light of Python 2.7 reaching its end of life (EOL) on Jan 1st 2020, Python 2 13 | will be unsupported by SaltStack no earlier then the Sodium release, that is 14 | either the Sodium release or a later release. 15 | 16 | 17 | # Motivation 18 | [motivation]: #motivation 19 | 20 | Python 2.7 will reach its official end of life (EOL) on [January 1st, 21 | 2020](https://devguide.python.org/#branchstatus) and post that, it will no 22 | longer be supported by its maintainers. Consequently, SaltStack team will 23 | retire Python 2.7 support in SaltStack. 24 | 25 | The end of Python 2.7 support will enable SaltStack to focus its effort on 26 | better supporting Python 3.4+. 27 | 28 | [Python 2.x is legacy, Python 3.x is the present and future of the 29 | language](https://wiki.python.org/moin/Python2orPython3) 30 | 31 | 32 | # Design 33 | [design]: #detailed-design 34 | 35 | SaltStack will drop Python 2 support no earlier than the Sodium release 36 | (tentative release date is Feb 2020), that is either the Sodium release or a 37 | later release. Keeping in mind SaltStack's policy of announcing 'end of 38 | support' at least 2 major releases before the intended changes takes place, 39 | Sodium or a later release has been chosen as a suitable deadline to drop Python 40 | 2 support. 41 | 42 | **Migration:** After carefully evaluating the list of Python 3 versions 43 | supported by different operating systems and by also considering the EOL for 44 | all these operating systems [1]* (as denoted in below grid) and [EOL of Python 45 | branches](https://devguide.python.org/#branchstatus)[2]*, SaltStack team 46 | proposes to support Python 3.4 from Sodium or a later release. Versions of 47 | SaltStack released before the cutoff release will continue to support python 48 | 2.7 until they reach end of life. 49 | 50 | [![Grid.png](https://i.postimg.cc/V62Wh9nY/Grid.png)](https://postimg.cc/xJymJz67) 51 | 52 | ### Note 1 53 | 54 | SaltStack currently supports master and minions running on different 55 | versions of Python such as salt master running on Python 2, minion running on 56 | Python 3 and vice versa. This support will not change in this transition as a 57 | change in Python interpreter does not mandate a change in network protocol. 58 | 59 | ### Note 2 60 | 61 | The version of OpenSSL that SaltStack will support will be determined 62 | by the Python version supported by SaltStack. Check the [Python SSL 63 | docs](https://docs.python.org/3/library/ssl.html) for the versions of OpenSSL 64 | supported by each Python version, and check the [pyOpenSSL release 65 | page](https://pypi.org/project/pyOpenSSL/#history) to identify the which 66 | versions of Python are supported by pyOpenSSL. 67 | 68 | ### Note 3 69 | 70 | A future SEP will define a schedule for ongoing depreciation of Python versions 71 | as they also reach their end-of-life. 72 | 73 | ### Note 4 74 | 75 | A future SEP will define the salt-ssh Python deprecation schedule. 76 | 77 | ## Alternatives 78 | [alternatives]: #alternatives 79 | 80 | - Support Python2 forever. Given that many of Salt's dependencies have pledged [to drop Python2 support](https://python3statement.org/) this would mean also supporting and vendoring our dependencies. 81 | - Drop support for Python 2.7 in **Neon** (tentative release date is August 2019). 82 | 83 | ## Packaging Considerations 84 | [Packaging]: #Packaging-Considerations 85 | 86 | Which Python packaging solutions to use - EPEL (Extra Packages for Enterprise 87 | Linux)/SC (Software Collections) repository or monolithic packaging? 88 | 89 | While the exact nature of packages delivered by SaltStack is not being laid out 90 | in this SEP, SaltStack team guarantees that package updates will be made 91 | available for smoother transition. 92 | 93 | 94 | # Actions 95 | [Actions]: #Actions 96 | 97 | SaltStack users will see the Python 2 deprecation implemented in the next few 98 | releases as follows: 99 | 100 | 1. Future releases of SaltStack will have a Python 2 deprecation warning. 101 | 2. SaltStack will create packages that will allow for a smooth transition for users. 102 | 3. The six library will be removed from SaltStack slowly time as well as all 103 | compat library use. 104 | 5. Existing monolithic installers will move solely to Python 3 only for the 105 | cutoff release, such as the Windows installer. 106 | 107 | 108 | # References 109 | [References]: #References 110 | 111 | [1]* Below is the list of all the sources from which EOL for the below 112 | Operating system versions were retrieved 113 | 114 | |Operating Systems |Sources | 115 | |-------------------------------|-----------------------------| 116 | |RHEL 6 |https://access.redhat.com/support/policy/updates/errata 'End of Maintenance Support 2 (Product retirement)' | 117 | |RHEL 7 | https://access.redhat.com/support/policy/updates/errata 'End of Maintenance Support 2 (Product retirement)' | 118 | |Amazon Linux 2 | https://aws.amazon.com/amazon-linux-2/faqs/ | 119 | |SLES 11 | https://www.suse.com/lifecycle/ | 120 | |SLES 12 | https://www.suse.com/lifecycle/ | 121 | |SLES 15 | https://www.suse.com/lifecycle/ | 122 | |OpenSUSE 15 | https://en.opensuse.org/Lifetime | 123 | |Debian 9 (Stretch) | https://wiki.debian.org/LTS/ | 124 | |Debian 8 (Jesse) | https://wiki.debian.org/LTS/ | 125 | |Ubuntu 18.04 | https://wiki.ubuntu.com/Releases | 126 | |Ubuntu 16.04 | https://wiki.ubuntu.com/Releases | 127 | |Ubuntu 14.04 | https://wiki.ubuntu.com/Releases | 128 | |FreeBSD 11 | https://lists.freebsd.org/pipermail/freebsd-announce/2018-September/001842.html | 129 | 130 | 131 | [2]* Excerpt from EOL of Python branches 132 | 133 | [![States-of-python-branches.png](https://i.postimg.cc/sXrxxvYV/States-of-python-branches.png)](https://postimg.cc/Mc9qrZ34) 134 | -------------------------------------------------------------------------------- /accepted/0022-old-releases.md: -------------------------------------------------------------------------------- 1 | - Feature Name: move non-supported releases to a separate domain name 2 | - Start Date: 2020-05-20 3 | - SEP Status: Accepted 4 | - SEP PR: https://github.com/saltstack/salt-enhancement-proposals/pull/28 5 | 6 | # Summary 7 | [summary]: #summary 8 | With the recent CVE events, we have observed and have been alerted to 9 | situations where clients and users were accessing and downloading older 10 | unsupported releases that contain the vulnerability. The ease of access, 11 | coupled with the potential for error that we observed, mean that we need to 12 | make a decision on what to do with older, unsupported releases. 13 | 14 | We have three potential paths: release updated packages for older versions, 15 | remove older versions all together, or move older versions to an archive 16 | location that is separated from currently supported versions. From a time and 17 | effort standpoint, the option to update is simply not viable and would set the 18 | project back considerably. The option to completely remove the older packages 19 | would leave many stranded while trying to upgrade to later versions and is 20 | therefore not viable either. This leaves us with the approach of moving the 21 | older, unsupported versions to an archive location. In examining this, we 22 | assessed other open projects and their approach to older, unsupported releases. 23 | We found many following the same proposed approach that we have in this SEP. 24 | Examples include Ubuntu and Debian, which make this distinction easy by using 25 | two separate domain names: old-releases.ubuntu.com and archive.ubuntu.com for 26 | Ubuntu, and archive.debian.org and deb.debian.org for Debian. We should emulate 27 | their behavior by moving our old unsupported releases to a separate URL. 28 | 29 | On the same note, unsupported PyPi releases should also be moved to a static 30 | HTML page hosted by SaltStack and taken down from PyPi. If possible, the 31 | release pages on PyPi should be updated to direct users to the new place where 32 | old releases are hosted. If not possible, the next major release of Salt, and 33 | following, should include this information around the top. 34 | 35 | # Motivation 36 | [motivation]: #motivation 37 | We are doing this so people have to make a deliberate decision to use an 38 | unsupported, and possibly insecure, release. It will become obvious when a 39 | release is no longer supported. It also means people syncing our repo won’t 40 | have to sync as much to get all supported releases. This reduces bandwidth 41 | costs for them and SaltStack. 42 | 43 | # Design 44 | [design]: #detailed-design 45 | Whenever a major release goes out of CVE support, we will move the contents to 46 | https://archive.repo.saltstack.com, for example: 47 | 48 | ``` 49 | https://repo.saltstack.com/yum/redhat/7/x86_64/2018.3/ 50 | ``` 51 | would move to 52 | 53 | ``` 54 | https://archive.repo.saltstack.com/yum/redhat/7/x86_64/2018.3/ 55 | ``` 56 | and all of the point releases under 57 | 58 | ``` 59 | https://repo.saltstack.com/yum/redhat/7/x86_64/archive/ 60 | ``` 61 | that are part of that major release would move to 62 | 63 | ``` 64 | https://archive.repo.saltstack.com/yum/redhat/7/x86_64/archive/ 65 | ``` 66 | 67 | If a point release is affected by a CVE that is part of a major supported 68 | branch, that point release will also be moved to 69 | https://archive.repo.saltstack.com. For example, 70 | 71 | ``` 72 | https://repo.saltstack.com/yum/redhat/7/x86_64/archive/3000/ 73 | ``` 74 | was affected by the recent CVE so it would be moved to 75 | 76 | ``` 77 | https://archive.repo.saltstack.com/yum/redhat/7/x86_64/archive/3000/ 78 | ``` 79 | 80 | This would mean someone using the bootstrap script would have to specify `-R 81 | archive.repo.saltstack.com` if they wanted to install an unsupported/insecure 82 | release. Other methods of installation would also require updating the URL 83 | with the new domain name. 84 | 85 | https://archive.repo.saltstack.com will be S3 syncable just like the main repo. 86 | 87 | We will add a notice to the main page of repo.saltstack.com explaining this 88 | behavior. 89 | 90 | With regards to the source tarballs hosted on PyPi, we will host them on a web 91 | server and they will be made available on a static HTML page. The reason to do 92 | this instead of hosting our own PyPi server is to make it explicit, and not 93 | easy, to install older versions. 94 | 95 | ### Communication Plan and Timeline 96 | We are sharing this SEP via Slack, IRC, and the Salt-Users and Salt-Announce 97 | Mailing Lists. 98 | 99 | In light of the CVE and potential for exploiting unsupported Salt releases, 100 | this SEP is urgent and running on an expedited timeline. The SEP will be open 101 | for comments from May 20-26, 2020. The detailed timeline is below: 102 | - May 20, 2020: SEP Announcement 103 | - May 21, 2020: SEP discussion during the Community Open Hour 104 | - May 29, 2020: SEP closed to comments 105 | - June 1, 2020: SEP closes 106 | - June 8, 2020 or later: Unsupported releases moved to separate URL 107 | 108 | ## Alternatives 109 | [alternatives]: #alternatives 110 | - We remove unsupported releases from being publicly available at all. However, 111 | this would force users to upgrade to the latest version. 112 | - We could leave vulnerable versions where they are. This would likely lead to 113 | inadvertent installation of vulnerable versions, probably compromising those 114 | servers. 115 | 116 | ## Unresolved questions 117 | [unresolved]: #unsresolved-questions 118 | - None so far 119 | 120 | # Drawbacks 121 | [drawbacks]: #drawbacks 122 | - We will not update rpms that setup the repo, such as 123 | https://repo.saltstack.com/yum/redhat/salt-repo-2015.5-1.el5.noarch.rpm which 124 | may inconvenience people who would like to use them. 125 | - We would have to update documentation describing the new behavior. 126 | - It will take effort for someone to move unsupported versions to the archive 127 | domain name. 128 | - People pinning to minor releases on repo.saltstack.com may have unexpected 129 | 404 errors when we move the insecure release to archive.repo.saltstack.com 130 | depending on the timeline of transition and whether they are aware of the CVE 131 | release. We recommend pinning to the major or latest release to avoid that 132 | issue. 133 | -------------------------------------------------------------------------------- /accepted/0006-remote_client.md: -------------------------------------------------------------------------------- 1 | - Feature Name: Remote Client 2 | - Start Date: 2018-12-24 3 | - SEP Status: Accepted 4 | - SEP PR: https://github.com/saltstack/salt-enhancement-proposals/pull/9 5 | - Salt Issue: (leave this empty) 6 | 7 | # Summary 8 | [summary]: #summary 9 | 10 | This feature allows for people using the `salt` CLI tool to connect remotely to 11 | any Salt master and to issue commands as though they were on the master itself. 12 | 13 | It removes the limitation for the CLI tool to have access to a stored key on the 14 | master and replaces that method with a public-key authentication system. 15 | 16 | # Motivation 17 | [motivation]: #motivation 18 | 19 | At present, if a person wishes to publish commands to a minion, they most often 20 | achieve this by logging into a master directly and then using the provided 21 | `salt` CLI tool. Altneratively, they may issue commands to a master by using 22 | `salt-api` which provides a series of REST endpoints which may be accessed 23 | natively, or by using an alternative CLI designed specifically for use 24 | with `salt-api`, which is known as `pepper`. 25 | 26 | However, even between these option, there remain several serious drawbacks: 27 | 28 | * A user cannot use either the `salt` tool or the `pepper` tool to send 29 | commands to a mix of API- and non-API- endpoints. 30 | 31 | * The only way to use Salt outputters is via the `salt` CLI tool, but 32 | as mentioned, this is limited to a single, local master. This is 33 | a noteable competitive disadvantage when comparing Salt to other 34 | tools which allow for remote infrastructure control. 35 | 36 | * Salt could have a richer ecosystem of third-party client tool and 37 | libraries to interact with Salt masters if this limitation were removed. 38 | These could include smartphone clients, as well as a much easier 39 | path for Salt libraries which could interact with a Salt master 40 | in a direct manner. 41 | 42 | # Design 43 | [design]: #detailed-design 44 | 45 | At present, the Salt LocalClient communicates with the master by sending 46 | commands to the RequestServer which listens on TCP/4506. The LocalClient 47 | proves that it is local to the host by accessing a special token which 48 | is generated by the mater on start and is stored by default in 49 | `/var/cache/salt/master/.root_key`. 50 | 51 | The goal of this RFC is to remove the dependency on access to the 52 | key in question and to replace or augment it with public-key 53 | authentication. 54 | 55 | Luckily, such a system is _already_ built into Salt and is used for the 56 | authentication process for minions which connect to a master in order to 57 | have a shared AES key securely transmitted to them. 58 | 59 | As such, this proposal suggests that the existing public-key authentication 60 | system simply be extended to accomodate clients. 61 | 62 | Therefore, in addition to directories already present in `/etc/salt/master/pki` 63 | which are presently prefixed with `minions_`, such as 64 | `/etc/salt/master/pki/minions` for accepted minion public keys or 65 | `/etc/salt/master/pki/`, we recommend the addition of directories prefixed by 66 | the word `clients`, such as `/etc/salt/master/clients`. 67 | 68 | The authentication system would be modified to accept a second type of 69 | authentication which would be of the type `client`. Clients presenting 70 | a key which is present in the master's PKI store would be allowed access. 71 | 72 | This allows, in turn, for the use of the `salt-key` tooling to control client 73 | access. 74 | 75 | The LocalClient would then have the option of either presenting a stored 76 | `root_key` to preserve existing behavior, or of negotiating public-key 77 | authentication with the master. 78 | 79 | Obvious modifications would also need to be made to the CLI parser to 80 | support the addition of a `-m` flag to indicate a master or list of masters 81 | to publish to. 82 | 83 | As such, a sample command to access a remote master via the Salt CLI might 84 | be: 85 | 86 | `salt -m my_remote_master '*' test.ping` 87 | 88 | ## Alternatives 89 | 90 | There are, of course, other ways we might accomplish this. These include: 91 | 92 | * Enhancing support for a master to be controlled via events on the master event 93 | bus. This might make integration with a reactive architecture substantially 94 | more robust. 95 | 96 | * Simply creating a special kind of minion that has rights on the existing 97 | minion-publish system. However, in the author's view, the overhead of this 98 | in terms of performance and start-up speed for a client would be too severe. 99 | 100 | * Create a unified client which can publish commands seamlessly between salt-api 101 | instances and local salt masters. This could potentially be extetended to 102 | `salt-ssh` as well. Though, the goals of this are not necessarily orthoginal 103 | to this RFC. 104 | 105 | ## Unresolved questions 106 | 107 | This breaks the existing assumption that a client will be dealing with a single 108 | master and as such, the names of minions which return are expected to be 109 | unique. However, this assumption has long been broken by the introduction of 110 | syndics. This might well make that problem worse though and the handling of 111 | "duplicate" returns would need to be addressed more thoroughly. 112 | 113 | If adopted, we would need to determine whether to remove the `root_key` 114 | system or whether to allow it to continue as an option. 115 | 116 | # Drawbacks 117 | [drawbacks]: #drawbacks 118 | 119 | First and most obviously, an administrator might now wish to allow remote 120 | access. As such, it is recommended that a `client_remote_access` option be 121 | added to the master configuration option allowing this feature to be easily 122 | disabled and to return to the existing behavior of requiring a `root_key` 123 | to be presented. However, in cases where this functionality is being provided 124 | by salt-api using tokens, there are a diminished set of reasons as to why 125 | one approach would be more secure than another, though in certain cases 126 | they may exist and be important to consider. 127 | 128 | Additionally, this system might be slower or even _substantially_ slower 129 | than the existing system, which would degrade client performance. 130 | 131 | 132 | -------------------------------------------------------------------------------- /accepted/0024-sub-state-returns.md: -------------------------------------------------------------------------------- 1 | - Feature Name: Sub State Runs 2 | - Start Date: 2020-08-12 3 | - SEP Status: Draft 4 | - SEP PR: github.com/saltstack/salt/pull/57993 5 | - Salt Issue: (leave this empty) 6 | 7 | # Summary 8 | [summary]: #summary 9 | 10 | Add the ability for states to have sub state runs. 11 | For example, if you are running a state that runs several external states under a different engine, 12 | you can now add those external state runs individually to the "sub_state_run" key of the state return. 13 | They will be parsed and printed alongside all the Salt state returns. 14 | 15 | # Motivation 16 | [motivation]: #motivation 17 | 18 | Salt has the ability to run Ansible playbooks, Idem SLSs, and Chef cookbooks. 19 | These "external state engines" run multiple "states" in their own way and Salt currently 20 | only reports whether or not external state runs as a whole were successful and doesn't report individual changes. 21 | I.E Return status of True if an entire Ansible playbook was executed successfully in an ansiblegate state 22 | 23 | We want to add the ability for Salt state returns to have a "sub_state_run" key, which gives 24 | these external state engines the ability to report the status of each of the "sub states" they 25 | ran independently, rather than reporting on the status of the entire group of sub states. 26 | 27 | The result is that state output will be very clear and verbose. 28 | Users will be able to see exactly which `idem state` passed/failed in an `idem SLS`, 29 | exactly which recipes passed/failed in a Chef cookbook, 30 | and exactly which ansible plays passed/failed in a playbook 31 | 32 | 33 | # Design 34 | [design]: #detailed-design 35 | 36 | Definitions 37 | ----------- 38 | - External state engine: A Salt module that has the ability to execute a group of configurations (I.E Ansible, Chef, Idem) 39 | - Sub state run: A new key in the return of Salt states that allows definitions of external state engine run details 40 | 41 | 42 | How it is used 43 | -------------- 44 | Some states can return multiple state runs from an external engine. 45 | State modules that extend tools like Ansible, Chef, and Idem can run multiple external 46 | "states" and then return their results individually in the "sub_state_run" portion of their return 47 | as long as their individual state runs are formatted like Salt states with low and high data. 48 | 49 | For example, the idem state module can execute multiple idem states 50 | via it's runtime and report the status of all those runs by attaching them to "sub_state_run" in it's state return. 51 | These sub_state_runs will be formatted and printed alongside other Salt states. 52 | 53 | Example 54 | ------- 55 | 56 | ```python 57 | state_return = { 58 | "name": None, # The parent state name 59 | "result": None, # The overall status of the external state engine run 60 | "comment": None, # Comments on the overall external state engine run 61 | "changes": {}, # An empty dictionary, each sub state run has it's own changes to report 62 | "sub_state_run": [ 63 | { 64 | "changes": {}, # A dictionary describing the changes made in the external state run 65 | "result": None, # The external state run name 66 | "comment": None, # Comment on the external state run 67 | "duration": None, # Optional, the duration in seconds of the external state run 68 | "start_time": None, # Optional, the timestamp of the external state run's start time 69 | "low": { 70 | "name": None, # The name of the state from the external state run 71 | "state": None, # Name of the external state run 72 | "__id__": None, # ID of the external state run 73 | "fun": None, # The Function name from the external state run 74 | }, 75 | } 76 | ], 77 | } 78 | ``` 79 | 80 | Code Changes 81 | ------------ 82 | 83 | salt/state.py would just need to be modified to look for the `sub_state_run` key in 84 | a state return and then process that data the same way it would process a regular state 85 | run, assigning a run_num (and incrementing the global counter), and formatting the low data. 86 | 87 | ```python 88 | for sub_state_data in running[tag].pop("sub_state_run", ()): 89 | self.__run_num += 1 90 | sub_tag = _gen_tag(sub_state_data["low"]) 91 | running[sub_tag] = { 92 | "name": sub_state_data["low"]["name"], 93 | "changes": sub_state_data["changes"], 94 | "result": sub_state_data["result"], 95 | "duration": sub_state_data.get("duration"), 96 | "start_time": sub_state_data.get("start_time"), 97 | "comment": sub_state_data.get("comment"), 98 | "__state_ran__": True, 99 | "__run_num__": self.__run_num, 100 | "__sls__": low["__sls__"], 101 | } 102 | ``` 103 | 104 | State Decorators would need to check for errors in these sub state runs. 105 | ```python 106 | sub_state_run = None 107 | if isinstance(result, dict): 108 | sub_state_run = result.get("sub_state_run", ()) 109 | result = self._run_policies(result) 110 | if sub_state_run: 111 | result["sub_state_run"] = [ 112 | self._run_policies(sub_state_data) 113 | for sub_state_data in sub_state_run 114 | ] 115 | ``` 116 | 117 | The state content checker would need to be modified to verify sub states. 118 | ```python 119 | for sub_state in result.get("sub_state_run", ()): 120 | self.content_check(sub_state) 121 | ``` 122 | 123 | The tests for this feature would be an exact copy of the existing state tests, 124 | only tweaked to verify `sub_state_run` returns rather than traditional state returns. 125 | ```python 126 | def test_sub_state_output_check_changes_is_dict(self): 127 | """ 128 | Test that changes key contains a dictionary. 129 | """ 130 | data = {"changes": {}, "sub_state_run": [{"changes": []}]} 131 | out = statedecorators.OutputUnifier("content_check")(lambda: data)() 132 | assert "'Changes' should be a dictionary" in out["sub_state_run"][0]["comment"] 133 | assert not out["sub_state_run"][0]["result"] 134 | ``` 135 | 136 | ## Alternatives 137 | [alternatives]: #alternatives 138 | 139 | The impact of NOT doing this is that when calling Idem SLS, Ansible playbooks, and Chef cookbooks, 140 | The output of state runs will not be verbose and it will not be clear what individual changes 141 | were made after executing a playbook/sls/cookbook. 142 | 143 | ## Unresolved questions 144 | [unresolved]: #unresolved-questions 145 | 146 | None, the proof of concept is functional, we just to double check with everyone before 147 | making a change to something as critical to Salt as the state engine. 148 | 149 | # Drawbacks 150 | [drawbacks]: #drawbacks 151 | 152 | Doing this modifies the state return code, a critical part of Salt. 153 | If it is done poorly then it could cause unintended grief down the road. 154 | -------------------------------------------------------------------------------- /accepted/0019-master-non-root.md: -------------------------------------------------------------------------------- 1 | - Feature Name: [security] Run salt master services as non root user by default 2 | - Start Date: 2021-08-03 3 | - SEP Status: Draft 4 | - SEP PR: (leave this empty) 5 | - Salt Issue: (leave this empty) 6 | 7 | # Summary 8 | [summary]: #summary 9 | 10 | Salt has historically assumed that all salt components run as root. 11 | Although support has been added for running the master and minion as non root users, this is not simple to set up and the documentation discourages it. 12 | This has had a further impact that the salt cli tools have to be run as root also. 13 | 14 | It has long been a standing best-practice to run serving daemons under regular user privileges, opening up additional capabilities as necessary. 15 | The `salt-master` and `salt-api` processes do not require root privileges for normal operation. 16 | 17 | The `salt-master` and `salt-api` both handle connections from potentially untrusted clients. By reducing the privilge of these processes we 18 | can reduce the impact of remotely exploitable vulnerabilities in those services. 19 | 20 | The `salt-minion` process generally requires root privileges in order to effectively manage the systems it runs on. It may be possible to modify the 21 | `salt-minion` process to run with lower privilege and gain additional privileges as required, but that is out of scope for this SEP due to the 22 | additional complexity that would require and because the `salt-minion` process is not require to accept incoming connections from potentially 23 | untrusted clients. 24 | 25 | An earlier version of this SEP also covered changes to config file locations, but that has been removed for simplicity and can be dealt with as 26 | a seperate SEP if required. 27 | 28 | 29 | # Motivation 30 | [motivation]: #motivation 31 | 32 | * Initially filed as SEP as suggested by Ch3LL in [#55886](https://github.com/saltstack/salt/issues/55886) 33 | * Reduce the impact of future vulnerabilities by having salt master components run as unprivileged users 34 | * Remove requirement for users of salt to have root privileges to run salt cli tools 35 | * Salt master services should be configured to use minimum privilege by default and not require additional configuration by users to do so 36 | * Salt cli tools should be usable by non root users, with good default groups and ACLs 37 | 38 | # Design 39 | [design]: #detailed-design 40 | 41 | ## Salt master services 42 | 43 | * `salt-master` and `salt-api` 44 | * run as dedicated user, eg `salt-master` 45 | * run `salt-api` as seperate dedicated user if possible 46 | * dedicated users are members of `salt` group 47 | * read only access to configs in `/etc/salt` through membership of `salt` group 48 | * `/etc/salt/master` 49 | * `/etc/salt/master.d/` 50 | * `/etc/pki/master/` 51 | * read/write access to specific directories under `/etc/salt` through `salt-master` user: 52 | * `/etc/salt/master.d/_generated.d` 53 | * `/etc/salt/pki/master/*/` 54 | * read/write access via `salt` group to: 55 | * `/var/cache/salt/master` 56 | * `/var/run/salt/master` 57 | * `/var/log/salt` 58 | 59 | 60 | ## Salt cli tools 61 | 62 | * `salt`, `salt-ssh`, `salt-cp`, `salt-key`, `salt-run` and `salt-ssh` 63 | * Users requiring use of cli tools can be added to `salt` group and additional publisher_acl group 64 | * read only access to configs in `/etc/salt` through membership of `salt` group 65 | * `/etc/salt/master` 66 | * `/etc/salt/master.d/` 67 | * `/etc/pki/master/` 68 | * read/write access via `salt` group to: 69 | * `/var/cache/salt/master` 70 | * `/var/run/salt/master` 71 | * `/var/log/salt` 72 | * Additional groups defined `salt-users` and `salt-admins` with suitable publisher_acl configs (eg `salt-admins` allowed to run any module/function, `salt-users` just functions on minions) 73 | 74 | 75 | ## OS package changes 76 | 77 | * add system user/group `salt` 78 | * add `salt-master` user 79 | * add `salt-master` (or `salt-api` user if using) to `shadow` group 80 | * change init-scripts to invoke services as `salt-master` 81 | * create `/etc/salt` with ownership `root:salt` and permissions `0750` 82 | * create `/etc/salt/master` with ownership `root:salt` and permissions `0640` 83 | * create `/etc/salt/master.d/` with ownership `root:salt` and permissions `0750` 84 | * create `/etc/salt/pki/master/` with ownership `root:salt` and permissions `0750` 85 | * create `/etc/salt/pki/master/*` with ownership `salt-master:salt` and permissions `0750` 86 | * create `/etc/salt/master.d/_generated.d/` with ownership `salt-master:salt` and permissions `0750` 87 | * create `/var/cache/salt/master/` with ownership `root:salt` and permissions `0770` 88 | * create `/var/run/salt/master/` with ownership `root:salt` and permissions `0770` 89 | * create `/var/log/salt/` with ownership `root:salt` and permissions `0770` 90 | * add `salt-users` and `salt-admins` groups 91 | * create initial master config with publisher_acl defined 92 | 93 | 94 | ## PAM 95 | 96 | [The documentation](https://docs.saltproject.io/en/master/topics/eauth/index.html) says that "eAuth using the PAM external auth system requires salt-master to be run as root as this system needs root access to check authenticationn and this is also documented in [#7762](https://github.com/saltstack/salt/issues/7762). Many Linux distributions have `/etc/shadow` and `/etc/gshadow` readable by the `shadow` group. By adding the `salt-master` user to this group we should be able to work round this, but this requires some testing. 97 | 98 | 99 | ## Migration paths 100 | 101 | * On upgrade, if existing config doesn't define non root user/group in config, install as previously 102 | * Add helper scripts to update config and directories of existing installs 103 | * Add warnings when salt cli tools run as root on systems configured with non-root user/group 104 | 105 | # Unresolved questions 106 | [unresolved]: #unresolved-questions 107 | 108 | * Ensuring files/directories written by `salt-master` user or other users maintain correct group ownership and permissions. May be solvable with setgid directories or changes to salt cli tools. 109 | * Minimum set of files/directories that need to be readable or writable by `salt-master` user and users in `salt` group. 110 | * Everything for `root` goes also for Windows' `SYSTEM`? 111 | * grains; if salt-master is configured to load these, how many of those need `root` and how graceful do they fail 112 | * Does publish_acl support `@wheel` and `@runner` syntax 113 | * Which supported distributions support the `shadow` group 114 | 115 | # Drawbacks 116 | [drawbacks]: #drawbacks 117 | 118 | * Large change in initial behaviour of installed salt master will require updates to documentation and education of users 119 | * Migration for existing setups may be complicated 120 | 121 | 122 | # Alternatives 123 | 124 | * Lock down `salt-master` and `salt-api` processes using AppArmour, SELinux or similar tools 125 | * PAM as non-root - socket auth 126 | 127 | # References 128 | 129 | * [Running the Salt Master/Minion as an Unprivileged User](https://docs.saltstack.com/en/latest/ref/configuration/nonroot.html) 130 | * [Running salt as normal user tutorial](https://docs.saltstack.com/en/latest/topics/tutorials/rooted.html) 131 | * [django-pam](https://pypi.org/project/django-pam/) 132 | * [pamela](https://pypi.org/project/pamela/) 133 | * [radicale-auth-PAM](https://pypi.org/project/radicale-auth-PAM/) 134 | -------------------------------------------------------------------------------- /abandoned/0005-worker-cache.md: -------------------------------------------------------------------------------- 1 | - Feature Name: `worker_cache` (MWorker Cache / If-Modified-Since Cache IMSCache) 2 | - Start Date: 2019-02-13 3 | - SEP status: Abandoned 4 | - SEP PR: https://github.com/saltstack/salt-enhancement-proposals/pull/3 5 | - Salt Issue: (leave this empty) 6 | 7 | # Summary 8 | [summary]: #summary 9 | 10 | Exposing a cache instance to modules running inside a MWorker context is 11 | desirable. A module author should be able to create/invalidate objects in a 12 | cache at whichever granularity they desire. The cache configuration, backend, 13 | and APIs used should be included with salt so future module authors can build 14 | more cool things using supported APIs. 15 | 16 | 17 | # Motivation 18 | [motivation]: #motivation 19 | 20 | Currently, salt has a PillarCache and plugable (salt.cache) Cache. Both caches 21 | are TTL based. Neither is exposed to modules, e.g. pillar, running 22 | inside MWorker context. 23 | 24 | For example, salt provides an interface to provide external pillar data but 25 | does not provide a mechanism to cache the data in memory. It's expected that 26 | external pillars produce computationally expensive and/or sizable 27 | results. Properly caching the result inside the MWorker process reduce load 28 | both: 29 | a) inside the MWorker and 30 | b) at the external data source 31 | 32 | The existing cache API solves only the latter. Using an in-memory cache instead 33 | avoids the de/serialization (a). 34 | 35 | A TTL Cache solves (b) but does so in such a way that it's very difficult 36 | (borderline impossible) to ensure objects are consistent with respect to one 37 | another. 38 | 39 | Exposing an in-memory cache that solves both (a) and (b), an IMSCache would 40 | greatly improve caching usefulness with regards to (a) and (b). 41 | 42 | 43 | # Design 44 | [design]: #detailed-design 45 | 46 | This is the bulk of the RFC. Explain the design in enough detail for somebody familiar 47 | with the product to understand, and for somebody familiar with the internals to implement. It should include: 48 | 49 | - Definition of any new terminology 50 | - Examples of how the feature is used. 51 | - Corner-cases 52 | - A basic code example in case the proposal involves a new or changed API 53 | - Outline of a test plan for this feature. How do you plan to test it? Can it be automated? 54 | 55 | ## Configuration 56 | 57 | Relevant configuration sections: 58 | 59 | * `worker_cache`: The cache class IMSCache, TieredIMSCache 60 | * `worker_cache_policy`: cache eviction policy 61 | * `worker_cache_tiered`: salt.cache driver to use in a tiered cache 62 | 63 | ## Exposing Cache 64 | 65 | The MWorker creates a `worker_cache` object. The `cache` is exposed where 66 | required, e.g. pass into `ext_pillar` function as kwarg. 67 | 68 | ## Cache API 69 | 70 | The `worker_cache` will implement the salt.cache interface. It's possible to use 71 | an subclass the implementation for re-use as the main salt `cache`, i.e. minion 72 | data etc, but is out of scope for this feature. 73 | 74 | The default `worker_cache` is an in-memory cache supporting IMS semantics. In 75 | order to support IMS the `worker_cache` expects the load function to return 76 | a CacheItem. When a simple object is returned instead a CacheItem will be 77 | created and stored in the cache. 78 | 79 | A CacheItem is a container object and should not have any functionality 80 | ``` 81 | class CacheItem(object): 82 | """Structure to hold metadata about cached object.""" 83 | def __init__(self, mtime, atime, ttl, data, **extra): 84 | self.__version = 0 # The CacheItem version should the structure change 85 | self.mtime = mtime 86 | self.atime = atime 87 | self.ttl = ttl 88 | self.data = data 89 | self.extra = extra 90 | 91 | @classmethod 92 | def make(cls, mtime=0, atime=0, ttl=0, data=None, **extra) 93 | return cls(mtime, atime, ttl, data, **extra) 94 | ``` 95 | 96 | A CacheItem must be serializble. The caller is responsible for ensuring any 97 | `data` or `extra` parameters can be serialized. 98 | 99 | Not all CacheItem properties are used but are included to support different 100 | cache policies. 101 | 102 | IMS validation is implemented in the user-supplied load `fun` argument to 103 | `cache`. When the key is expired the expired CacheItem is passed to the `fun` 104 | argument; When the key is not present an empty CacheItem is passed to the `fun`. 105 | The return value of `fun` is the updated CacheItem to store in the cache. 106 | 107 | A special `fun` return value `None` means the key should be deleted. 108 | 109 | ### IMSCache 110 | 111 | The reference implementation of `worker_cache` is a class IMSCache implementing 112 | the salt.cache API. The underlying data is stored in memory. Rather than 113 | implementing from scratch the MIT-Licensed package `cachetools` is used. 114 | 115 | The `worker_cache_policy` provides the user some control over how objects should 116 | be evicted. The policy should express what type of cache should be used 117 | (LRU/LFU) and the size of the cache. IMSCache uses the policy to construct the 118 | underlying cache store. 119 | 120 | The `expire` argument to `cache` is supported to override the `ttl` set on the 121 | CacheItem. 122 | 123 | 124 | ### TieredIMSCache 125 | 126 | The TieredIMSCache stacks an IMSCache with another backing salt.cache.Cache driver. 127 | The simple example is: 128 | 129 | ``` 130 | def cache(...): 131 | ... 132 | imscache.cache(fun=sharedcache.cache(...), ...) 133 | 134 | def flush(...): 135 | sharedcache.flush(...) 136 | imscache.flush(...) 137 | 138 | ``` 139 | 140 | ## Deprecations 141 | 142 | The `worker_cache` does not support `loop_fun`. Attempting to pass a `loop_fun` 143 | will raise an exception. 144 | 145 | ## Example Usage 146 | 147 | A user can use the `worker_cache` inside `ext_pillar` 148 | 149 | ``` 150 | def ext_pillar(..., extra_minion_data): 151 | cache = extra_minion_data['worker_cache'] 152 | def _get_thing(): 153 | return "A thing" 154 | 155 | return cache.cache(__name__, "thing", fun=_get_thing) 156 | ``` 157 | 158 | ## Testing plan 159 | 160 | The existing salt.cache test suite will be expanded to include IMSCache 161 | TieredIMSCache. 162 | 163 | ## Alternatives 164 | [alternatives]: #alternatives 165 | 166 | What other designs have been considered? What is the impact of not doing this? 167 | 168 | ### Do nothing 169 | 170 | Exposing a cache in MWorker is a trivial change. There is no compelling reason 171 | not to do this. 172 | 173 | ### External cache 174 | 175 | The behavior described can be implemented using an external python module that 176 | exposes an IMSCache Singleton. There is no strong reason not to do this other 177 | than it would not benefit salt community/codebase as a whole. 178 | 179 | ### Completely redesign caching 180 | 181 | Iterative improvement to existing systems should always be preferred unless the 182 | existing design is fundamentally flawed. 183 | 184 | 185 | ## Unresolved questions 186 | [unresolved]: #unresolved-questions 187 | 188 | What parts of the design are still TBD? 189 | 190 | The cache should have some strategy to prevent thundering herds (random offset 191 | in TTL, intent locking+serve stale, etc). 192 | 193 | Synchronization should be implemented where required. 194 | 195 | # Drawbacks 196 | [drawbacks]: #drawbacks 197 | 198 | `worker_cache` introduces some new classes that are compatible with existing 199 | interfaces. Introducing new code should be considered future debt. At some point 200 | in the future existing and new code should be refactored to a single use. Until 201 | we have a chance to exercise the functionality we don't know how to refactor. 202 | 203 | The code described really doesn't have a lot of logic. The logic is implemented 204 | externally (cachetools) or reuses existing conventions. 205 | 206 | The code might be too simple. One `ext_pillar` might evict another pillar's 207 | cache keys. This could be addressed by making the cache factory return the 208 | singleton used for that particular `ext_pillar`. 209 | 210 | There are no strong reasons not to expose a cache to ext pillar or to add 211 | IMSCache. 212 | -------------------------------------------------------------------------------- /accepted/0009-salt-working-groups.md: -------------------------------------------------------------------------------- 1 | - Feature Name: Working Group 2 | - Start Date: 2019-05-16 3 | - SEP Status: Accepted 4 | - SEP PR: https://github.com/saltstack/salt-enhancement-proposals/pull/12 5 | - Salt Issue: (leave this empty) 6 | 7 | # Summary 8 | [summary]: #summary 9 | 10 | Engage with the Salt community to get feedback on working groups 11 | 12 | # Motivation 13 | [motivation]: #motivation 14 | 15 | Salt's first working groups will convene by mid-June 2019. In order to benefit the participants of current and future working groups as well as those administering them, all working group details will be added as a README document in the repo `saltstack/community`. 16 | 17 | # Working Groups 18 | 19 | A Working Group is a small group of individuals who come together with a common goal and work towards achieving that goal within a predetermined amount of time. It’s an opportunity for the Salt community to lead Salt projects and be part of the process in its entirety. 20 | 21 | ## Eligibility to participate 22 | 23 | Salt community members are eligible to be part of a working group if they have exhibited interest in working on the Salt project by raising either 1 or more PR(s), issue(s) or SEP(s) (Salt Enhancement Proposal). 24 | 25 | In addition, a working group applicant is a Subject Matter Expert (SME) in areas such as Windows, Cloud, Networking or SSH 26 | 27 | If you want to lead or be part of a different working group then the ones mentioned above, Salt would be happy to include that as part of initial working groups, provided Salt receives enough interest from others to participate. 28 | 29 | ## Application Process 30 | 31 | Salt community members indicate their interest to be part of one or more working group(s) by filling out an application consisting of simple 4-5 question survey; Applicants indicate their preference for the type of working group(s) they want to join and the role they want to play: Captain or facilitator. If an application is received past the due date then the applicant will automatically be considered for the next term of the working groups. 32 | 33 | ## Election 34 | 35 | Salt will conduct an election to choose the working group members in the following scenarios: 36 | - When the number of applicants interested in joining the working groups exceeds the number that is ideal for a working group (3 to 9 members) 37 | - 2 or more community members indicate their interest in being captain 38 | - If Salt doesn’t receive any eligible applicants for captain role, a member of the Salt open/core will be elected to be captain of that working group. 39 | 40 | The rules for the election process are: 41 | - Election for the first working group captain will be internal and voting will be done by the Salt core team. 42 | - Elections for all subsequent working group captains and members will be voted on by the outgoing members of the current working group and Salt core team. 43 | - In the event of a tie, [Thomas Hatch](https://github.com/thatch45), Salt project creator, will make the final decision. 44 | 45 | ## Timelines 46 | 47 | Based on the applications submitted, Salt chooses the final working group members 48 | 49 | Below is the approximate timeline for how long each working group term will last. The actual timeline may differ. For example, the Testing and Release Working Group might decide to convene immediately after a major release and dissolve after the next major release. 50 | 51 | ![Working Group Timelines](../diagrams/Working-group-timelines.svg) 52 | 53 | 54 | 55 | *T0 is feature freeze date of current major release 56 | T1 is feature freeze date of next major release 57 | 58 | 59 | ## Collaboration 60 | Working Groups will meet not less than once a month for 60 mins. The monthly meetings which are set by SaltStack will be recorded and uploaded on Youtube. Meetings are not just restricted to the Working group members and can be attended by any Salt community members. Details of the meeting will be published in the individual working group github page; For example, for Testing working group meeting [check here](https://github.com/saltstack/community/tree/master/working_groups/wg-Testing) 61 | 62 | Below are some guidelines for working group participants, although each of these groups may operate a little differently depending on their needs and workflow. 63 | 64 | ## Working Group Structure 65 | 66 | Each member of a working group will fill one of 3 roles: captain, participant or facilitator. 67 | 68 | ### Captain 69 | 70 | Each working group captain will help lead their respective working group. 71 | 72 | #### Role of Captains 73 | - The captain helps shepherd a subset of features/critical fixes for a given release 74 | - The captain will help coordinate other contributors' work in their area 75 | - The captain will attend monthly working group meetings for their respective working group. SaltStack will set these up, but SaltStack will hold no restriction on any additional meetings the captain may deem necessary 76 | 77 | #### Captain Guidelines 78 | 79 | - The captain will have merge rights to Salt repository 80 | - A captain of one working group cannot be captain of any other Salt working groups. But the captain can be a participant of other Salt working groups 81 | - The captain can nominate the captain and/or participants of next term working group 82 | - Captains are limited to 3 successive terms. Captains must take a break after their 3rd successive term. Following the break, they may come back to fill the role of captain again. 83 | - Starting from 2nd term working groups, the captains chosen need to have been a participant in at least one working group 84 | - Once a captain’s term ends and they are no longer captaining the next term, their merge rights will be rescinded and transferred to the new captain of the working group 85 | - Single review completed by the captain on the PR will be considered as criteria met to merge the PR (No additional reviews required) 86 | - The captain must follow the [Merge Guidelines](https://github.com/saltstack/salt-enhancement-proposals/); This is to ensure the Salt project grows responsibly. 87 | - In rare occasions and in consultation with the Salt Core team, the captain can nominate a vice-captain for the working group, to help assist during a leave of absence or to share responsibilities. The captain can also give merge access to said vice-captain of their working group while ensuring they follow the Merging Guidelines. 88 | 89 | 90 | ### Participant 91 | 92 | Participants form the core of each of the working groups. 93 | 94 | #### Role of Participant 95 | - Participants will attend monthly working group meetings for their respective working group 96 | - Participants will get to work on exciting Salt projects: enhancements or issues that impacts thousands of Salt users 97 | 98 | #### Participant Guidelines 99 | - No restriction on the number of working groups a participant can be part of 100 | - Once their term ends, a participant can move to a different working group or continue in the same working group 101 | - No term limits on how many times a Salt community member can be a participant of a working group 102 | - Participants don’t have merge rights. However in rare circumstances, the captain can recommend that a participant be granted merge rights. 103 | 104 | 105 | ### Facilitator 106 | 107 | Until the working groups mature, Saltstack will facilitate the working group meetings and there will be at least 1 SaltStack facilitator in every working group meeting 108 | 109 | #### Role of Facilitator 110 | - The facilitator will work with the Salt core team to identify the list of participants, communicate/invite them to the working groups 111 | - The facilitator will ensure everyone gets a chance to voice their opinions 112 | - The facilitator will ensure members adhere to the goals of the meeting and that the meeting doesn't devolve into tangential conversations 113 | - The facilitator sets up the monthly working group meetings and invites the appropriate participants 114 | - The facilitator makes the agenda clear before every meeting 115 | - The facilitator takes notes and shares the action items after every meeting 116 | - The facilitator ensures the working group members needs/interests are met 117 | 118 | 119 | ## Benefits of Working Group 120 | 121 | - Active voice and participation in Salt projects 122 | - SaltStack will publish the efforts of working groups to the broader Salt community 123 | - SaltStack will recognize working groups members in various public platforms. 124 | - Participants and Captains will receive dedicated working group swag 125 | 126 | -------------------------------------------------------------------------------- /0037-pluggable-transports.md: -------------------------------------------------------------------------------- 1 | - Feature Name: Pluggable Transports 2 | - Start Date: 2021-09-01 3 | - SEP Status: Draft 4 | - SEP PR: 5 | - Salt Issue: 6 | 7 | # Summary 8 | [summary]: #summary 9 | 10 | Most subsystems in Salt are pluggable. For example, adding new External Pillar modules, Proxy Minion types, and SDB backends are all well documented and leveraged extensively. However, while some work has been done, Salt transports are not currently pluggable. At the moment Salt effectively has two transports: TCP, and ZeroMQ. There is great value in exploring other transport types including things like [NATS](https://nats.io), [Kafka](https://kafka.apache.org), [RabbitMQ](https://www.rabbitmq.com). 11 | 12 | Note there is a closed and unmerged PR that did some refactoring and added an http transport: [saltstack/salt#57210](https://github.com/saltstack/salt/pull/57210) 13 | 14 | # Motivation 15 | [motivation]: #motivation 16 | 17 | ZeroMQ has served the Salt Project well. In 8 years of the project, very few failures of Salt can be tied to ZeroMQ directly. Most transport related problems are due to things like misconfiguration, network issues, and filesystem corruption. ZeroMQ is very fast and basically "just works". 18 | 19 | As Salt continues to expand into larger and larger environments, ZeroMQ presents problems that are harder to solve. For example: 20 | 21 | 1. Salt's High Availability and Disaster Recovery scenarios are not as robust as they could be. Hot-hot masters can be an effective HA/DR tool but introduce their own issues. Failover masters don't always failover the way you anticipate. 22 | 2. We often talk about ZeroMQ as being a "brokerless" transport but that's not entirely true in a Salt environment. If the master goes away for some reason, the event bus also goes away. 23 | 3. While we used to recommend syndics as a solution to complex Salt topologies in recent years we are deprecating this approach and suggest users apply the patterns offered by the Salt Enterprise product (now known as vRA SaltStack Config post VMware acquisition). This doesn't solve some of the latency, deployment, and scaling issues that customers encounter. One example involves customers that want to bridge several on-prem datacenters with presences in the cloud. This turns out to generate many complexity and reliability challenges. 24 | 4. The ZeroMQ project is lightweight by design and adding observability hooks or durable queues have been out of scope, preferring instead to spend effort on the "just works" part of the library and/or recommending that library users layer their own observability on top. But Salt customers continue asking questions like "is Salt dropping events? Is my bus 'overloaded'? How do I know when I should add another master and split my minions between them? How can I predict my event bus traffic so I can know that my Salt infrastructure will perform in a crisis?" In general all we have had to offer them is rules of thumb and estimates. 25 | 26 | In some cases we can solve these problems by adding additional features inside the Salt codebase. But other scenarios are impossible without direct transport support. 27 | 28 | Rather than trying to fix our codebase for every scenario, it would be wiser to adapt Salt to make it easier to add transports, and then leverage the features of those transports where appropriate. This also means that users that do not need other transport features are not saddled with the additional code/configuration burden. ZeroMQ will remain the primary transport in Salt, making it easy for users to get started with Salt and scale it significantly before needing to be concerned with other options. 29 | 30 | Now it makes little sense to do the work to make transports pluggable without a good proof of concept. This SEP also proposes that RabbitMQ would be the first additional pluggable transport. 31 | 32 | # Design 33 | [design]: #detailed-design 34 | 35 | ## Deployment/Configuration 36 | Salt already has a concept of configurable transport (e.g. "zeromq" or "tcp") and it is set in master/minion config files. The proposal is to introduce a new transport of type "rabbitmq" and also include any additional metadata such as broker address, vhost, credentials, etc. 37 | RMQ broker may be deployed by Salt or in some cases it may be a shared enterprise resource that Salt can be pointed to. 38 | 39 | ## Abstractions 40 | Salt transport is already mostly abstracted as a client/server interface with a factory pattern. A factory pattern is used to instantiate a specific instance of client/server. 41 | The proposal is to implement the interfaces for RMQ. See https://github.com/saltstack/salt/tree/master/salt/transport. 42 | Note that as part of this effort we have an opportunity to clean up the interfaces (e.g. decouple auth from data channel) and we shall do so opportunistically. 43 | 44 | ## RabbitMQ topology 45 | Management and use of RabbitMQ objects (vhosts, users, exchanges, queues, topics, etc.): 46 | 47 | * VHOST/tenant, user, permission configuration is performed by the tenant admin persona or equivalent; master/minion read pertinent configuration from config files or equivalent. A master/minion belong to a single tenant/vhost. 48 | * Minions running in customer's environment should be considered potential malicious actors (e.g. initiators of message flood). Consider the blast radius if/when this occurs. The impact should not transcend the org/tenant that minion belongs to. 49 | * Use permissions to limit minion's access to queues and exchanges. 50 | * Configure maximum queue size (several thousand messages) 51 | 52 | * Create 1 x Fanout exchange/queue used for message publishing, one per master cluster. 53 | * Create 1 x Fanout exchange/queue used to send requests/commands to minions, one per master cluster. 54 | * [Optional] Create 1 x exchange/queue for initial master/minion authentication. Once authenticated, the minion can switch to another channel for communication. 55 | * Set "auto-ack" to True for consuming messages. This gives us parity with ZeroMQ. There are potentially areas where bringing the ack up the stack could be useful, but we do not know yet where. 56 | * Use reply queues (with auto_delete=True) and correlation ids to correlate request/response. See https://www.rabbitmq.com/tutorials/tutorial-six-dotnet.html). 57 | * Note that we need to scale to 50K minions per vhost, so need to be careful about the number of queues we create. 58 | 59 | * Associate a "dead-letter" exchange for each queue. Undelivered messages will end up there. This can be useful for troubleshooting. (See https://www.cloudamqp.com/blog/when-and-how-to-use-the-rabbitmq-dead-letter-exchange.html) 60 | * Use durable quorum queues for consistency 61 | * When it comes to trade-offs, we favour consistency over performance. 62 | * Some features are not yet available for quorum queues, such as message TTL, message priority (see https://www.rabbitmq.com/quorum-queues.html) 63 | * Configure message expiry (TTL) to be no longer than 72 hours. Expired messages will be delivered to the dead-letter exchange. Note that TTL can be configure per-message or per-queue. 64 | * Artifact cleanup (queues, exchanges) 65 | * When object is declared with auto-delete=True, it will be deleted when last consumer dies (e.g. when all connections are closed). This is gives us parity with ZeroMQ and simplifies implementation. 66 | 67 | 68 | ## Third-party libraries 69 | Python "pika" library (BSD license) is the industry standard for interacting with AMQP and RabbitMQ. It supports both sync and async patterns and also supports Tornado IO loops. 70 | The proposal is to use the "pika" library (https://pypi.org/project/pika/) to interact with RabbitMQ from Salt and use non-blocking/async-style connections as much as possible. 71 | 72 | ## Testing 73 | Existing parameterized functional tests will be updated to cover rabbit as a new transport. These tests already iterate over collection of transports ("tcp", "zeromq") and encryption types ("clear, "aes"). 74 | New functional tests will be added to cover some edge cases, e.g. connection recovery in cases when RMQ broker restarts. 75 | A scale test will be performed to confirm the new transport can support 50K minions and message throughput of 1 job per minute x 5 messages per job x 1 minion. 76 | 77 | ## Alternatives 78 | [alternatives]: #alternatives 79 | 80 | As part of early PoC efforts we also created Salt engines that bridge the ZeroMQ transport to other event buses. This can be an effective approach, and some of these bridges will persist as parts of the product so customers can run some parts of their infrastructure with ZeroMQ and others with other transports. Like any engineering solution these have tradeoffs (performance, duplication of network traffic, potential for misconfiguration). 81 | 82 | ## Unresolved questions 83 | [unresolved]: #unresolved-questions 84 | 85 | ### Minion and Master Keys 86 | 87 | One potential future scenario after pluggable transports are enabled is the ability for minions to "float" freely between a set of masters. Since multiple masters could theoretically join the same event bus and send commands to all attached minions, we need to address how all those masters will keep their sets of minion keys synchronized as well as how minions will distinguish the different masters. Currently hot-hot and failover masters are required to share the same public/private key pair and HA master scenario best practices mandate shared filesystems for minion keys. In an on-prem+cloud+multi availability zone scenario a shared filesystem for minion keys is often infeasible and a security risk. Changes to key management are not a necessary condition for this SEP to move forward. 88 | 89 | # Drawbacks 90 | [drawbacks]: #drawbacks 91 | 92 | Some reasons to NOT add pluggable transports mostly center around adding code complexity, introducing bugs during the refactor of various chunks of code, and figuring out how best to add tests with proper separation of concerns. 93 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Salt Enhancement Proposals 2 | 3 | ## Deprecation of SEP Process 4 | In 2019 we decided to create a community process to discuss and review 5 | Salt Enhancements. Unfortunately I feel that this process has not proven 6 | to be an effective way to solve the core issues around Salt Enhancements. 7 | Overall the Salt enhancement process has proven itself to be more of a 8 | burden than an accelerant to Salt stability, security, and progress. 9 | As such, I feel that the current optimal course of action is to shut 10 | the process down. 11 | 12 | Instead of the Salt Enhancement Proposal process we will add a time 13 | in the Open Hour for people to present ideas and concepts to better 14 | understand if they are worth their effort to develop. Extensive 15 | documentation around more intrusive or involved enhancements should 16 | be included in PRs. 17 | 18 | By migrating the conversation into the PR process we ensure that 19 | we are only reviewing viable proposals instead of being burdened 20 | with requests that the core team is expected to fulfill. 21 | 22 | Conversations about enhancements can also be held in the discussions 23 | tab in GitHub. 24 | 25 | Effective immediately (01/24/2024) we will be archiving and freezing 26 | the SEP repo. 27 | 28 | -Thomas S Hatch, creator and founder of Salt Project 29 | 30 | # README Retaind for Archive 31 | Many changes, including bug fixes and documentation improvements, can be 32 | implemented and reviewed via the normal GitHub pull request workflow. 33 | 34 | Some changes though are "substantial", and we ask that these be put through a 35 | bit of a design process and produce a consensus among the Salt core team. 36 | 37 | The "SEP" (Salt Enhancement Proposal) process is intended to provide a 38 | consistent and controlled path for new features to enter the project. 39 | 40 | This process is being **actively developed**, and it will still change as more 41 | features are implemented and the community settles on specific approaches to 42 | feature development. 43 | 44 | ## When to follow this process 45 | 46 | You should consider using this process if you intend to make "substantial" 47 | changes to Salt or its documentation. Some examples that would benefit from an 48 | SEP are: 49 | 50 | - A new feature that creates new API surface area 51 | - The removal of features that already shipped 52 | - The introduction of new idiomatic usage or conventions 53 | 54 | The SEP process is a great opportunity to get more eyeballs on your proposal 55 | before it becomes a part of a released version of Salt. Quite often, even 56 | proposals that seem "obvious" can be significantly improved once a wider group 57 | of interested people have a chance to weigh in. 58 | 59 | The SEP process can also be helpful to encourage discussions about a proposed 60 | feature as it is being designed, and incorporate important constraints into the 61 | design while it's easier to change, before the design has been fully 62 | implemented. 63 | 64 | Changes that do **NOT** require an SEP: 65 | 66 | - Rephrasing, reorganizing or refactoring 67 | - Bug fixes 68 | - Addition or removal of warnings 69 | - Additions only likely to be _noticed by_ other implementors-of-Salt, 70 | invisible to users-of-Salt. 71 | 72 | ## What the process is 73 | 74 | In short, to get a major feature added to Salt, one must submit the SEP via 75 | pull-request. After a comment period, the SEP will be either **Accepted** or 76 | **Rejected**. An **Accepted** SEP may then be implemented with the goal of 77 | eventual inclusion into Salt. 78 | 79 | ## The SEP life-cycle 80 | The following is a more detailed explanation of the process. 81 | 82 | ### 1. Proposal 83 | The SEP is proposed by submitting a pull request to this repo, by copying 84 | `0000-template.md` and modifying it. If the SEP pertains to any open issues, 85 | reference them in the **Salt Issue(s)** entry. 86 | 87 | When copying the file, do _not_ assign it a number. Simply name the file with a 88 | short description of the SEP (e.g. `subspace-transport.md`). Once the pull 89 | request has been opened, a SaltStack core engineer will assign the SEP a number 90 | and the SEP will enter the initial **Draft** status. At this time, the 91 | following changes can be made to the SEP file: 92 | 93 | 1. Add the pull request number to the **SEP PR** entry at the top of the file 94 | 2. Rename the file to include the assigned SEP number, then commit and push to 95 | update the pull request 96 | ```bash 97 | $ git mv subspace-transport.md 0123-subspace-transport.md 98 | $ git commit -am 'Assigned SEP number' 99 | $ git push origin branchname 100 | ``` 101 | 102 | ### 2. Discussion 103 | The pull request will remain open and serve as the comment thread for the SEP. 104 | The initial comment period will last no fewer than two (2) weeks, and may be 105 | extended as deemed necessary based on comment activity. 106 | 107 | Once the initial comment period has ended, the SEP will enter **Final Comment** 108 | status. One (1) week will be allowed for any further comments. At the end of 109 | the **Final Comment** period, a decision will be made on whether or not to 110 | accept the SEP. Acceptance requires approval from five (5) members of the core 111 | development team. As project creator, [Thomas 112 | Hatch](https://github.com/thatch45) will have a final veto on any SEP. 113 | 114 | Additionally, as 115 | [discussed](https://github.com/saltstack/salt-enhancement-proposals/pull/1#issuecomment-468823572), SaltStack will be publishing a SEP to form a Community Advisory Board (CAB). 116 | 117 | ### 3. Acceptance / Rejection 118 | At the end of the **Final Comment** stage, the SEP will be either **Accepted** 119 | or **Rejected**. Either way, the pull request will be merged. 120 | 121 | Note that acceptance does not mean that the feature will be immediately 122 | implemented, or that it will be implemented at all; It merely means that the 123 | core development team has agreed to it in principle. Additionally, the fact 124 | that an SEP pull request has been merged does not necessarily mean that the SEP 125 | has been accepted; pull requests for rejected SEPs are merged so that they are 126 | visible to others who might otherwise open an SEP for a previously-rejected 127 | topic. 128 | 129 | ### 4. Implementation 130 | An **Accepted** SEP may proceed to be implemented. If no issues on the Salt 131 | issue tracker are listed under **Salt Issue(s)**, then create one and open a 132 | pull request to update the SEP with the issue number. All SEPs which have 133 | reached the implementation step must have at least one associated issue. 134 | 135 | We should strive to write each SEP in a way that it will reflect the final 136 | design of the feature; However, if during implementation things change, the SEP 137 | document should be updated accordingly. 138 | 139 | The SEP author (like any other developer) is welcome to post an implementation 140 | for review after the SEP has been accepted. However, the author of an SEP is 141 | not obligated to implement it. If the author has no plans to implement the SEP, 142 | they can indicate on their SEP that the implementation should be considered 143 | **Up for Grabs**. Alternatively, if no apparent activity has occurred on the 144 | SEP for two (2) weeks, it will be considered **Up for Grabs**. 145 | 146 | If you are interested in working on the implementation for an accepted SEP, but 147 | cannot determine if someone else is already working on it, feel free to ask 148 | (e.g. by leaving a comment on the associated issue). 149 | 150 | ## Summary of SEP Statuses 151 | The below statuses were discussed above: 152 | - **Draft**: The initial status, from submission through the initial discussion 153 | period 154 | - **Final Comment**: A one-week period after the initial comment period has 155 | ended 156 | - **Accepted**: The SEP has been approved for future implementation 157 | - **Rejected**: The SEP has been rejected during discussion phase 158 | - **Abandoned**: The SEP is fundamentally good idea, but discussion and 159 | progress have stagnated for any number of reasons. Progress may be resumed by 160 | the original author, or the SEP re-started by a new SEP champion. 161 | 162 | In additon, SEPs can be assigned the following statuses: 163 | - **Withdrawn**: The SEP has been voluntarily withdrawn from consideration. 164 | stagnated, 165 | - **Up for Grabs**: The SEP has been accepted, but nobody is actively working 166 | towards its implmentation. 167 | - **Implemented**: The accepted SEP has been implemented. 168 | - **Obsolete**: The accepted SEP is no longer relevant due to other changes in 169 | Salt, but should be considered for re-evaluation. The re-evaluation will be 170 | done in a separate SEP. Once the new SEP is opened, the **Obsolete** SEP will 171 | be considered **Replaced**. 172 | - **Replaced**: The SEP has been superseded by another SEP. 173 | 174 | The SEP's status can be viewed in two ways: 175 | 176 | 1. In the **Status** entry at the top of the SEP file 177 | 2. Via GitHub labels applied to the SEP's pull request 178 | 179 | The following diagram may help with the potential paths a SEP may take. 180 | 181 | ![SEP status diagram](diagrams/SEP%20Statuses.svg) 182 | 183 | ## Reviewing SEPs 184 | SaltStack staff will post information about open SEPs to the 185 | **[#sep](https://saltstackcommunity.slack.com/messages/CGQ5Q6W5A/convo/C7K04SEJC-1552938425.137500/)** channel 186 | in the community Slack, as well as our community IRC and mailing list on a 187 | regular basis to encourage discussion. 188 | 189 | **This SEP process owes its inspiration to the [React RFC process], [Yarn RFC 190 | process], [Rust RFC process], and [Ember RFC process]**. 191 | 192 | [React RFC process]: https://github.com/reactjs/rfcs 193 | [Yarn RFC process]: https://github.com/yarnpkg/rfcs 194 | [Rust RFC process]: https://github.com/rust-lang/rfcs 195 | [Ember RFC process]: https://github.com/emberjs/rfcs 196 | -------------------------------------------------------------------------------- /accepted/0031-windows-install-anywhere.md: -------------------------------------------------------------------------------- 1 | - Feature Name: Windows Install Anywhere 2 | - Start Date: 2021-01-19 3 | - SEP Status: Accepted 4 | - SEP PR: https://github.com/saltstack/salt-enhancement-proposals/pull/41 5 | - Implementation PR: https://github.com/saltstack/salt/pull/60267 6 | 7 | # Summary 8 | [summary]: #summary 9 | 10 | Allow the user to select the install location (``install_dir``) for Salt in the 11 | Windows Installer. The default install location will be 12 | ``%ProgramFiles%\Salt Project\Salt`` for the binary data and 13 | ``%ProgramData%\Salt Project\Salt`` for the Root Directory (``root_dir``). Add a 14 | command line switch to allow the user to set the ``install_dir`` via the command 15 | line. 16 | 17 | # Motivation 18 | [motivation]: #motivation 19 | 20 | Windows is not Linux 21 | 22 | The Windows directory structure is such that installed software should reside in 23 | the ``%ProgramFiles%`` directory on the system drive. Software should not be 24 | installed on the root of the system drive. Salt is currently installed on the 25 | root of the ``C:`` drive. This seems to be a paradigm brought over from Linux. 26 | 27 | In Windows: 28 | 29 | - Non-changing binaries should be in ``%ProgramFiles%`` 30 | - Volatile data should be in ``%ProgramData%`` 31 | - Configuration should be in the registry, though we make an exception for Salt 32 | due to its reliance on a config file 33 | 34 | We want the Salt installer to be consistent with other enterprise installers 35 | that follow the Windows paradigm for software installation. 36 | 37 | # Design 38 | [design]: #detailed-design 39 | 40 | ## Directory Structure 41 | [directory-structure]: #directory-structure 42 | 43 | In order to follow the Windows paradigm for software installation we will 44 | separate out binary and volatile/config data. It is proposed that this be broken 45 | out as follows: 46 | 47 | __Binary Data__: 48 | 49 | ``` 50 | c:\salt\*.files -> C:\Program Files\Salt Project\Salt\*.files 51 | c:\salt\bin -> C:\Program Files\Salt Project\Salt\bin 52 | ``` 53 | 54 | __Config/PKI Data__: 55 | 56 | ``` 57 | c:\salt\conf -> C:\ProgramData\Salt Project\Salt\conf 58 | ``` 59 | 60 | __Volatile Data__: 61 | 62 | ``` 63 | c:\salt\srv -> C:\ProgramData\Salt Project\Salt\srv 64 | c:\salt\var -> C:\ProgramData\Salt Project\Salt\var 65 | ``` 66 | 67 | ## Registry 68 | 69 | [registry]: #registry 70 | 71 | In a Salt installation there is a root directory (``root_dir``) and an 72 | installation directory (``install_dir``). The ``root_dir`` is where all Salt 73 | non-binary data is stored. Other directories such as the config directory 74 | (``conf_dir``) are based on the ``root_dir``. The ``install_dir`` indicates the 75 | location of the Salt binaries. 76 | 77 | In the previous method of installation the default locations were ``C:\salt`` 78 | for the ``root_dir`` and ``C:\salt\bin`` for the ``install_dir``. The new 79 | installation method will split these out and default to standard Windows 80 | locations in ``%ProgramData%`` for ``root_dir`` and ``%ProgramFiles%`` for 81 | ``install_dir``. It will also allow the user to change the ``install_dir`` in 82 | the GUI or via a command line switch. The settings will be stored in the 83 | registry at the following locations: 84 | 85 | __install_dir__ 86 | 87 | - Path: ``HKLM\\SOFTWARE\\Salt Project\\Salt`` 88 | - Key Name: ``install_dir`` 89 | - Default: ``C:\\Program Files\\Salt Project\\Salt`` 90 | 91 | __root_dir__ 92 | 93 | - Path: ``HKLM\\SOFTWARE\\Salt Project\\Salt`` 94 | - Key Name: ``root_dir`` 95 | - Value: ``C:\\ProgramData\\Salt Project\\Salt`` 96 | 97 | These registry settings will be created by the installer after the user selects 98 | the location for the Salt installation. If it is an existing old method 99 | installation, the registry entries will be created using the old locations as 100 | their values unless the user chooses to move them in the GUI or via command- 101 | line switch. 102 | 103 | ## Program Files / ProgramData 104 | 105 | [program_files_programdata]: #program-files-programdata 106 | 107 | There are scenarios where the system drive contains only the system files with 108 | all other software installed on a separate drive, ``D:\Program Files`` for 109 | example. This is done by setting the environment variable ``%ProgramFiles%``. 110 | The default location for program data can also be set using ``%ProgramData%``. 111 | It is proposed that Salt use these environment variables to determine the 112 | default location for the ``%Program Files%`` and ``%ProgramData%`` directories. 113 | 114 | ## Directory Permissions 115 | 116 | [directory-permissions]: #directory-permissions 117 | 118 | Salt will be separated into binary data and config/volatile data. Binary data 119 | will be located in the ``%ProgramFiles%\Salt Project\Salt`` directory by 120 | default. Config/volatile data will be located in the 121 | ``%ProgramData%\Salt Project\Salt`` directory. 122 | 123 | The Salt directory in ``%ProgramData%`` will have the same permissions as those 124 | applied to ``C:\salt`` with the additional restrictions applied to the ``pki`` 125 | directory. Only the process that is running Salt should have access to those 126 | directories. It should be locked down to standard users. 127 | 128 | Binary data installed in ``%ProgramFiles%`` will inherit the permissions applied 129 | to the ``%ProgramFiles%`` directory. If Salt is installed in any other location 130 | then it will be incumbent upon the user to set permissions to that directory. 131 | 132 | ## Installer Flags 133 | 134 | [installer-flags]: #installer-flags 135 | 136 | __install-location__ 137 | 138 | A new ``/install-location`` installer flag will be added. This option will allow 139 | the user to silently set the install location for Salt. The default will be 140 | ``%ProgramFiles%\Salt Project\Salt``. This would apply to new installations only 141 | and will be ignored for all other scenarios. 142 | 143 | __move-config__ 144 | 145 | A ``/move-config`` installer flag will be added. This option will move an 146 | existing config found at ``C:\salt`` to the new ``%ProgramData%`` directory. The 147 | ``root_dir`` registry entry will use the new location. This will only apply when 148 | there is an existing config found at ``C:\salt``. Otherwise it is ignored. 149 | 150 | # Installation Scenarios 151 | 152 | [installation-scenarios]: #installation-scenarios 153 | 154 | There are 3 scenarios the installer will need to handle while installing Salt: 155 | 156 | 1. Existing New Method Installation 157 | 2. Existing Old Method Installation 158 | 3. New Installation 159 | 160 | In either case of an existing installation, old or new, we want to be able to 161 | upgrade in place. This means the ``install_dir`` and ``root_dir`` would remain 162 | unchanged. If it's an existing new method installation, the installer would use 163 | the locations defined in the registry. For old method installations the 164 | ``install_dir`` and ``root_dir`` would both be ``C:\salt``. In both cases the 165 | GUI would display the current install location, but it would be greyed out. 166 | There should be an option to move an existing config at ``C:\Salt`` to the new 167 | location at ``%ProgramData%``. This would be accomplished via a checkbox in the 168 | GUI or a command line switch (``move-config``). 169 | 170 | The New Installation and Existing Old Method Installations both allow the user 171 | to select an install location. The GUI will display a dialog box with a 172 | directory picker that will allow the user to browse to a directory. The default 173 | will be the directory defined in the ``%ProgramFiles%`` environment variable. 174 | 175 | The ``root_dir`` is not user configurable, other than the option to move an 176 | existing ``root_dir`` in ``C:\salt`` to ``%ProgramData%``. 177 | 178 | In the future we may add the ability to install Salt on a per-user basis. In 179 | that case the registry data would be found in HKCU, and the ``root_dir`` would 180 | default to ``%LOCALAPPDATA%``. 181 | 182 | We don't want the installer to change the locations of ``install_dir`` and 183 | ``root_dir`` for an existing installation without the user's knowledge. 184 | Therefore, this must be explicitely specified in the GUI by checking a check 185 | box or on the command-line by passing a swtich. 186 | 187 | Below is the logic to handle each scenario: 188 | 189 | ### 1. Check for Existing New Method Installation 190 | 191 | Check for an existing new method installation by checking the registry for the 192 | presence of ``HKLM\SOFTWARE\Salt Project\Salt``. This indicates that Salt was 193 | installed using the new method. If the registry key exists, do the following: 194 | 195 | - The ``root_dir`` is as defined in the registry 196 | - The ``install_dir`` is as defined in the registry 197 | - The install location ``install_dir`` is displayed in the GUI 198 | - The install location is greyed out in the GUI 199 | - Upgrade Salt in place 200 | - DONE 201 | 202 | ### 2. Check for Existing Old Method Installation 203 | 204 | If a new method installation is not detected, check for an existing old method 205 | installation by looking for the existence of `python.exe` in ``C:\salt\bin``. 206 | This would indicate that Salt was installed using the old method where 207 | everything is in ``C:\salt``. If the binary exists, do the following: 208 | 209 | - The ``root_dir`` is ``C:\salt`` 210 | - The ``install_dir`` is ``%ProgramFiles%\Salt Project\Salt`` 211 | - The install location ``install_dir`` is displayed in the GUI 212 | - The user can change the install location 213 | - A checkbox allows the user to move the ``root_dir`` to ``%ProgramData%`` 214 | - This moves ``conf``, ``var``, and ``srv`` to 215 | ``%ProgramData%\Salt Project\Salt`` 216 | - ONLY IF ``%ProgramData%\Salt Project\Salt`` does not exist or is empty. 217 | Otherwise, Abort 218 | - The ``root_dir`` and ``install_dir`` are saved in the registry 219 | - Upgrade Salt (use new install location) 220 | - DONE 221 | 222 | ### 3. New Installation 223 | 224 | If neither installation type has been detected, then this must be a new 225 | installation. The installer will do the following: 226 | 227 | - The ``root_dir`` is ``%ProgramData%\Salt Project\Salt`` 228 | - The ``install_dir`` defaults to ``%ProgramFiles%\Salt Project\Salt`` 229 | - The install location ``install_dir`` is displayed in the GUI 230 | - The user can change the install location 231 | - The ``root_dir`` and ``install_dir`` are saved in the registry 232 | - Install Salt 233 | - DONE 234 | -------------------------------------------------------------------------------- /accepted/0014-dev-overhaul.md: -------------------------------------------------------------------------------- 1 | - Feature Name: Improved Salt Testing and Release Strategy 2 | - Start Date: 2019-10-01 3 | - SEP Status: Approved 4 | - SEP PR: https://github.com/saltstack/salt-enhancement-proposals/pull/20 5 | 6 | # Summary 7 | [summary]: #summary 8 | 9 | The purpose of this SEP is to define an overhaul of the Salt test, development, and release process. Also, read [the FAQ](https://docs.google.com/document/d/1DfOGVmsQaqr3rZrFqo-gf9wm1mYM-nwa5a1zROatCXQ/edit#heading=h.8yucbt63lrhh) after reading this SEP. 10 | 11 | # Motivation 12 | [motivation]: #motivation 13 | 14 | Over the course of time, Salt releases have gotten increasingly complex and release timelines have become unpredictable and unstable. As the Salt core team spent a tremendous effort to stabilize the test suite for the 2019.2.1 release, a large number of problems became apparent and were identified as release issues. 15 | 16 | - Missed releases. 17 | - No clear indication to Salt users about which features/fixes are part of which releases. 18 | - Overhead of maintaining 7 branches (untenable/not a good long-term strategy). 19 | - Reduced release quality. 20 | 21 | We have missed release deadlines due to the complexity of the Salt release process. Prior to the 2019.2.1 release we were maintaining **seven** branches: 22 | 23 | - `develop` 24 | - `neon` 25 | - `2019.2.1` 26 | - `2019.2` 27 | - `2018.3` 28 | - `2017.7` 29 | - `2017.7.9` 30 | 31 | ![Existing strategy](../diagrams/diagrams_old-branch-strategy.png) 32 | 33 | The combination of bugfixes, merge forwards, and backports has been difficult to try and keep track of. There have been a number of Salt releases the community has met with confusion - why is this bugfix missing? Why was that feature added in a point release? 34 | 35 | It’s also been difficult for the community to contribute, raising questions such as: 36 | 37 | - If I want my changes in the next Salt release, what branch should I make my PR against? 38 | - How should I be testing? 39 | 40 | In some cases contributors submitted their PRs only against the branch they cared about, and it was up to the Core team to ensure that these fixes or features made it to the other releases. This constantly shifting target produced a number of bugs. With a desire to merge PRs quickly to get fixes and features in, PRs were merged without tests, or with apparently unrelated test failures. When it came time for release, a tremendous amount of effort was required to fix failing tests and stabilize each branch. It was also difficult trying to ensure that all of the desired fixes, merge forwards, and backports were made before release. 41 | 42 | During the 2019.2.1 release process it became apparent that our current approach is unsustainable and unacceptable. Our unique release process causes difficulty for contributors, is painful for users, and is impossible for maintainers. 43 | 44 | **There must be a better way.** 45 | 46 | The Salt core team has spent a tremendous amount of time, thought, and effort working to identify the issues that face our community and coming up with a plan for reducing the overhead and complexity to make it easier for Salt contributors to produce and ship high-quality software on time and to meet expectations. This proposal is the result of that process. 47 | 48 | These changes are intended to: 49 | 50 | - Improve core Salt stability. 51 | - Improve testing quality. 52 | - Reduce maintenance and testing overhead. 53 | - Be able to release Salt on demand. 54 | - Be able to announce Salt release dates with a high degree of certainty. 55 | - Maintain a **high-frequency, reliable release cadence of stable software**. 56 | 57 | # Design 58 | [design]: #detailed-design 59 | 60 | ## Single Branch Release Strategy 61 | 62 | To eliminate the biggest source of confusion and distraction, Salt will adopt the industry standard of a **single branch release** strategy. Salt will have a **master** branch that reflects production-ready code. This will avoid the confusion and overhead of which branch needs to be merged where. This will also allow Salt contributors to spend more time focusing on bug fixes and new features that are important to Salt users. 63 | 64 | ### New master branch 65 | 66 | A new branch, called **master** will be created from **2019.2.1**. Salt core team has spent close to 5 months stabilizing the 2019.2.1 branch. Because 2019.2.1 is stable and tests are already green, rather than repeating the entire process for **develop** or some other branch, PRs from `develop` and `neon` branches will be merged in a controlled manner to ensure **master** (formerly 2019.2.1) continues to stay stable and green. 67 | 68 | ![A new, more stable approach](../diagrams/diagrams_new-branch-strategy.png) 69 | 70 | **Focusing on a single release branch will help us to release better software more often**. In order to maintain this focus, and to meet our desired release cadence **there will be no more point releases on 2017.7.x and 2018.3.x branches and 2017.7.8 and 2018.3.4 will be capstone releases**. 71 | 72 | ### Tests Must Be Green 73 | 74 | The Salt core team has already been requesting regression tests for bug fixes, and appropriate quality test suites to accompany new features. The entire test suite runs for each PR and **MUST** pass before merge, as [outlined in SEP 10](https://github.com/saltstack/salt-enhancement-proposals/blob/master/accepted/0010-pr-merge-requirements.md). Not only must new tests be passing, but if a failure in existing tests is exposed, the Salt core team will fix the test suite before any new PRs are merged. This will produce a higher degree of certainty that the code that we release is stable, and that changes to Salt code do not introduce regressions. 75 | 76 | ### PR Migration plan 77 | 78 | Any changes previously added to the 2017.7 and 2018.3 branches will be merged forward into the new master branch. Also there are currently about 1,000 PRs that have been merged into the 2019.2, neon, and develop branches which will get merged into **master**, but it’s going to take a lot of work. The Salt core team has prioritized these PRs in terms of criticality to the Salt community, and ease of migration. The Salt core team is carefully working to port these PRs into the new **master**, and has reached out to many of the top contributors with a large number of PRs to explain and review our plans. 79 | 80 | While the Salt core team has committed to migrate the PRs, because of the huge size of the task before us, we would love your help! If you have a PR that you’d like to get merged more quickly, we’d love your help migrating PRs from other branches to **master**. 81 | 82 | ## Release Cadence 83 | 84 | The Salt team has a goal to be able to release when necessary. Our plan is to have a typical release cadence of 3 months. We expect to have an intermediate release cadence of 4 months. To be able release on demand, the Salt open team needs to: 85 | 86 | - Always maintain green tests for release branch. 87 | - Ensure stability of release branch all the time. 88 | - Adopt a 100% automated CI/CD release model. 89 | 90 | The Salt core team has a goal to be on a 4 month release cadence for the next few releases, with the ultimate goal of being able to release whenever needed. While we plan to stabilize on a 3-month typical release cadence, the fact of software development is that it’s done by people, and people are imperfect and will make mistakes. To be able to minimize the impact of these mistakes it’s important to be able to quickly (but carefully) test and release bugfix versions to Salt, likely resulting in more frequent, smaller releases. 91 | 92 | The Salt core team will be opening and merging PRs from develop and neon to **master** (based off 2019.2.1) as soon as this SEP is published to demo how the core team will handle the PR migration process. 93 | 94 | ## Hotfix / Patch Release 95 | 96 | Despite our best efforts, we will likely still encounter bugs after release during this transition. To reduce the impact of these bugs **master** will stay in a feature freeze for 2 weeks after release, in order to fix bugs that were not encountered until after release. 97 | 98 | The Salt core team will focus on these bugs, but quality bug fixes submitted by the community that are properly tested and documented may also be accepted during this period. 99 | 100 | After **master** is able to have new features merged, if new, severe bugs without a workaround are reported, a short-lived hotfix branch may be created. This would be an issue that: 101 | 102 | - Affects core Salt functionality. 103 | - Has no workaround (i.e. requires a code change). 104 | - Has a reasonably quick turnaround (the fix doesn’t require a serious refactoring). 105 | 106 | This branch will go through our release process - testing in the CI/CD pipeline, manual tests, RC, and release. This branch will then be merged into **master**. 107 | 108 | 109 | ## Automation 110 | 111 | To support the CI/CD approach, the Salt core team will be improving the automation of the deployment pipeline, with the goal of automatically producing nightly (or weekly) builds that flow through the entire build pipeline, including automated tests. 112 | 113 | Other automation steps will be introduced to reduce friction for Salt community contributors. Documentation will be rewritten, published, and shared as necessary to remove as many blockers to contribution as possible. 114 | 115 | ## Documentation 116 | 117 | One major challenge for the community is understanding changes coming in the next release. Focusing efforts on a single branch will help with this, but additionally Salt will create a changelog file to be updated by contributors, that will be used to document changes in a human-readable way, as specified in [SEP 1](https://github.com/saltstack/salt-enhancement-proposals/blob/master/accepted/0001-changelog-format.md). Updating this documentation will be required as part of the merge process - either from the contributor or by a Salt core team member. 118 | 119 | Salt [documentation](https://docs.saltstack.com) will be updated to point to the most current release (2019.2.1 at the time of this writing). For users who have not yet upgraded, or those who are testing the unreleased version of Salt, Salt take an approach similar to the official Python documentation, and have a drop-down or some other way to easily select documentation for other Salt versions. 120 | 121 | We will also upgrade our communication processes to keep the Salt community more aware of the release timelines. We will share progress with the [#salt channel on IRC on LiberaChat](https://web.libera.chat/#salt), the #release channel in the [SaltStack Community Slack](https://saltstackcommunity.herokuapp.com/), and the [Salt Users mailing list](https://groups.google.com/forum/#!forum/salt-users). 122 | 123 | ## Versioning (Naming) 124 | 125 | With the `neon` release, to indicate the new change in release process, Salt will change to a new, non-date based version schema beginning at 3000. The version will be MAJOR.PATCH. For a planned release containing features and/or bug fixes the MAJOR version will be incremented. 126 | 127 | With the increased release cadence in order to allow proper planning for feature deprecation, Salt will introduce a minimum 1-year, expected 3 major release cycle. If `neon` ships in January 2020, any new deprecations will not be removed until January 2021 at the earliest. 128 | 129 | Note that this only applies to deprecations within Salt’s control. If a 3rd party changes, Salt may release updates earlier. 130 | 131 | ## Support matrix 132 | 133 | To be able to focus on the stability and innovation in the Salt platform, we will be adopting the industry-standard approach of no longer supporting older releases. Salt will provide select support for serious bugs and CVEs for the most recent release. Minor bug fixes will be targeted in the next scheduled Salt release. 134 | 135 | 136 | ## Alternatives 137 | [alternatives]: #alternatives 138 | 139 | The primary alternative is to keep doing the things that Salt has been doing, and try to make incremental changes. The costs (both opportunity and real) are no longer acceptable. 140 | 141 | Long and unpredictable release times, unstable branches, missing (or extra) fixes and features, and regressions are all problems with our existing workflow. Trying to fix these problems here and there will not make the huge positive impact for our community that we are trying to make. 142 | 143 | ## Unresolved questions 144 | [unresolved]: #unresolved-questions 145 | 146 | Hopefully all the questions have been answered in this SEP. Upgrading Salt should continue to work like it always has - our changes are focused on the external development process of Salt. If you feel like you have unanswered questions, please come ask them at the Salt Office Hours on October 1st, 2019, or find us in [#salt on IRC on LiberaChat](https://web.libera.chat/#salt), the [SaltStack Community Slack](https://saltstackcommunity.herokuapp.com/), or the [Salt Users mailing list](https://groups.google.com/forum/#!forum/salt-users). 147 | 148 | # Drawbacks 149 | [drawbacks]: #drawbacks 150 | 151 | The Salt team recognizes that we have users and customers on older versions of Salt [outside their support windows](https://www.saltstack.com/product-support-lifecycle/). The Salt core team is working with support teams to carve out a plan to make the transition to supported Salt versions smoother. 152 | 153 | There are also a *lot* of PRs that we need to migrate. However, this will probably not be much more difficult than the merge forward and backport process that we’ve been used to, though with the benefit that when we finish with this set of PRs we will never have to do it again. Fixes and features will be added to one single branch, and be available in the next scheduled release. 154 | -------------------------------------------------------------------------------- /accepted/0020-py-version-support.md: -------------------------------------------------------------------------------- 1 | - Feature Name: Python Version Support 2 | - Start Date: 2020-03-24 3 | - SEP Status: Accepted 4 | - SEP PR: https://github.com/saltstack/salt-enhancement-proposals/pull/26 5 | - Salt Issue: N/A 6 | 7 | # Summary 8 | [summary]: #summary 9 | 10 | This SEP is being created to add a policy around Python versions Salt will support and 11 | detail how we will support Python versions going forward. This policy change is to align Salt Releases 12 | with Python's End of Life schedule. 13 | 14 | # Motivation 15 | [motivation]: #motivation 16 | 17 | Documentation and Communication 18 | ------------------------------- 19 | One of the motivating factors around this policy change is to clarify our Python version support 20 | for each Salt release. If it is clearly defined, then packagers, community members, and Salt users will 21 | know when a Python version will be dropped for a given Salt release. In the past we had never clearly 22 | defined how our Python version support lined up with Python's End of Life schedule. We previously made a 23 | best effort to support the Python versions that lined up with the operating systems Python versions, but never clearly 24 | defined this and did not always follow this in every case. For example, when we dropped Python 2.6 support 25 | we still supported Red Hat 6, which had Python 2.6 installed by default. To continue supporting Red Hat 6 while also 26 | removing support for the End of Life Python 2.6 version, we included Python 2.7 in our packages 27 | for that operating system. 28 | 29 | Including Python in Packages 30 | ---------------------------- 31 | Another motivating factor to make this policy change is our shift to including Python in our 32 | packages. For the Sodium release we will be using [Tiamat](https://gitlab.com/saltstack/pop/tiamat/) (previously known as [pop-build](https://gitlab.com/saltstack/pop/pop-build/)) to 33 | package for our [Fully Supported Platforms](http://get.saltstack.com/rs/304-PHQ-615/images/SaltStack-Supported-Operating-Systems.pdf). 34 | This will allow us to easily continue supporting operating systems that include End of Life Python 35 | versions by default by providing the Python binary in the Salt package included in a Salt release. 36 | 37 | Security and Maintenance Concerns 38 | --------------------------------- 39 | The last motivating factor surrounds security and maintenance concerns. Continuing to support an End 40 | of Life Python version adds additional maintenance around testing and development for each version. 41 | Supporting these versions also puts users at a security risk, since no more security fixes will be 42 | submitted and released with that Python version if the operating system does not choose to update 43 | it themselves. 44 | 45 | # Design 46 | [design]: #detailed-design 47 | 48 | The design goal of this SEP is to define a clear policy around Salt's Python version support. 49 | This new policy defines only supporting major Python versions until they reach End of Life. You can see 50 | the current status of supported Python branches here https://devguide.python.org/#status-of-Python-branches 51 | with the associated End Of Life date for each branch. You can also see details about Python versions that 52 | have already become End Of Life here: https://devguide.python.org/devcycle/#end-of-life-branches. 53 | 54 | What if a Python version drops support relatively close to a Salt Release? 55 | -------------------------------------------------------------------------- 56 | We may drop support for a Python version if the End of Life date lands relatively close to a Salt release. 57 | For example if a Python version is End of Life a couple weeks before a Salt release, support for that 58 | Python version may drop in that Salt version. We will need to communicate and decide when a Python version 59 | is dropped at the beginning of a development cycle. 60 | 61 | What about new Python versions? 62 | ------------------------------- 63 | At the beginning of a development cycle for each feature release, not only do we need to align 64 | with the Python versions that will become End of Life, but also new Python versions that will need 65 | to be added. Tests running against the new Python version will need to be added and run on the 66 | master branch. Any issues found will need to have issues created and assigned out for the 67 | scheduled release. 68 | 69 | How will this impact the Sodium release? 70 | ---------------------------------------- 71 | Because Python 3.4 became End Of Life on 2019-03-18, we will only support Python versions >= 3.5 72 | 73 | How does this impact all supported Operating Systems? 74 | ----------------------------------------------------- 75 | The details below will include the impact for our supported operating systems included here 76 | and some future operating systems we will be supporting: 77 | http://get.saltstack.com/rs/304-PHQ-615/images/SaltStack-Supported-Operating-Systems.pdf 78 | 79 | 80 | AIX/Solaris 81 | ----------- 82 | AIX/Solaris are Enterprise packages and will use salt-bin for their packaging mechanism, 83 | so they will include the correct Python version. 84 | 85 | Windows 86 | ------- 87 | Python does not come pre-installed on Windows operating systems. We have always included 88 | the Python binary in the package and will continue to do that going forward. If someone 89 | wants to pip install Salt onto windows they will need to install a supported Python version 90 | first, just as they always have. In the case for the Sodium release this will mean they 91 | will need to install Python >=3.5 92 | 93 | macOS 94 | ----- 95 | Apple does not clearly state when their operating systems become End of Life, but they do 96 | always support the three latest versions of macOS. We can estimate these End of Life on some of the 97 | operating systems based on when the latest one is released. The dates included in the table 98 | below are not confirmed by Apple and are estimates to help determine how this policy will 99 | affect macOS. 100 | 101 | | OS | End of Life | Default Python | 102 | |------------|---------------------|----------------| 103 | | 10.13 | September 2020 | 2.7 | 104 | | 10.14 | September 2021 | 2.7 | 105 | | 10.15 | TBD | 2.7 | 106 | | 10.16 | TBD | no default | 107 | 108 | We have always included the Python binary with the mac packages, so that will not change for 109 | users that use the mac Python packages. Starting with the Sodium release we will no longer 110 | support Python 2.7, so if users install Salt on macOS using pip or any other means they will 111 | need to install Python >= 3.5. 112 | 113 | Fedora 114 | ------ 115 | Fedora releases a new version every 6 months and supports these releases for approximately 13 months. 116 | Our current support mirrors that of the upstream and they are currently supporting Fedora 30, 31 and 117 | 32. They do not have current EOL dates for these releases yet here: 118 | https://endoflife.software/operating-systems/linux/fedora 119 | 120 | | OS | End of Life | Default Python | 121 | |------------|-----------------------------|-----------------| 122 | | 30 | TBD (possibly June 2020) | 3.7 | 123 | | 31 | TBD (possibly Oct 2020) | 3.7 | 124 | | 32 | TBD (possibly May 2021) | 3.8 | 125 | 126 | Based off of these dates and Python's End of Life dates, Fedora will not 127 | be of any concern with this change in policy. Fedora releases fairly regularly 128 | and includes the latest Python version. Both packages and pip installs 129 | will work against supported Fedora versions going forward. 130 | 131 | SLES/openSUSE 132 | ------------- 133 | The General End of Life dates do conflict with the current Python Version 134 | End of Life dates. Since SUSE does the packaging here they have been informed 135 | of this policy change and will include a resolution around this with their packages 136 | when these version conflicts occur. 137 | 138 | Other Linuxes 139 | ------------- 140 | 141 | | OS | End of Life | Default Python | 142 | |------------|---------------------|----------------| 143 | | RHEL 6 | 30 Nov 2020 | 2.6 | 144 | | RHEL 7 | 30 Jun 2024 | 2.7 | 145 | | RHEL 8 | 05 May 2029 | None | 146 | | Ubuntu 16.04 | April 2021 | 2.7/3.5 | 147 | | Ubuntu 18.04 | April 2023 | 3.6 | 148 | | Ubuntu 20.04 | April 2025 | 3.8 | 149 | | Debian 8 | 30 June 2020 | 2.7, 3.4 | 150 | | Debian 9 | June 2022 | 2.7, 3.5 | 151 | | Debian 10 | June 2024 | 2.7, 3.7 | 152 | 153 | Based on the chart above all of these Operating Systems will or already have conflicted with 154 | Python's End of Life Support dates. We will need to include Python in our packages for these 155 | Operating Systems especially at any given time when their default Python version is considered 156 | End of Life. This does impact users who would want to pip install Salt. They will need to install 157 | a newer version of Python before they can install Salt. 158 | 159 | Other Reasonable-Effort Supported Operating Systems 160 | --------------------------------------------------- 161 | 162 | Arch 163 | ---- 164 | Arch is a rolling release and is consistently updating their Python version, so this is not 165 | of any concern for this policy. 166 | 167 | FreeBSD 168 | ------- 169 | According to this https://wiki.freebsd.org/Python/PortsPolicy FreeBSD currently already aligns 170 | with this policy. They only support Python versions until they are End of Life so there are 171 | no concerns with this operating system. 172 | 173 | SmartOS 174 | ------- 175 | SmartOS includes the latest Python version so will not be impacted by this change. 176 | 177 | 178 | What about dependencies? 179 | ------------------------ 180 | Before dropping an End of Life Python version we should have an idea beforehand if Salt's dependencies 181 | do not support a particular Python version with our testing suite. Currently our tests use the dependencies 182 | in the requirements directory in the main Salt project. This does of course mean that if we are not testing 183 | a module we would not have this information beforehand. We are now requiring tests to be added 184 | for PRs so this will help ensure this is caught for those modules, but there are still some modules that do 185 | not have tests currently. 186 | 187 | For the Sodium Release, I installed the libraries from anything defined with `:depends:` in the documentation 188 | and the requirements defined in our requirements directory. Once these were installed I ran 189 | this tool https://pypi.org/project/checkmyreqs/ against all installed Python libraries. This brought to light 190 | the following libraries that will not support Python 3, which will impact some Salt components: 191 | 192 | - pycassa (salt/modules/cassandra.py, salt/returners/cassandra_return.py) 193 | - vbox (salt/modules/vbox_guest.py, salt/modules/vbox_guest.py, 194 | salt/modules/vboxmanage.py, salt/utils/virtualbox.py, 195 | salt/cloud/clouds/virtualbox.py) 196 | 197 | Both of these projects have not had commits added to the project in years. This will need to be documented 198 | for the Sodium release to make it clear that these modules will not work unless the upstream projects add 199 | Python 3 support. 200 | 201 | Going forward adding more test coverage will help to uncover these projects before we drop a Python version, 202 | but if we do not have enough test coverage, this exercise should be done any time a Python Version is dropped. 203 | There might be a conflict between a Salt dependency and its Python version support that will impact many users. 204 | If this conflict does occur we may need to delay dropping support for the End of Life Python version until that 205 | dependency is updated to the appropriate Python version. 206 | 207 | ## Who is Affected 208 | 209 | Salt Contributors 210 | ----------------- 211 | When a Python Version is dropped the contributor can now use new features in the lowest supported 212 | Python version. For example, f-strings were added in Python 3.6 so when our Python version support is updated 213 | to using >= Python 3.6 a contributor will be able to use f-strings now. 214 | 215 | A contributor will also need to be aware of what is deprecated and removed in all supported Python releases when 216 | making new contributions. For example, in the Python 3.8 release the macpath module was removed, 217 | so a contributor would need to ensure they do not write any new contributions using this module. 218 | 219 | A contributor can look at Python's Release Notes for details around what is added into a release or removed. 220 | For example for Python 3.6 221 | https://docs.python.org/3/whatsnew/3.6.html outlines some of the changes and removals added to this version. 222 | 223 | Salt User 224 | --------- 225 | If a Salt user is using the packages from https://repo.saltstack.com they will not be affected 226 | as the Python binary will be included in the package. If they are pip installing the package and 227 | their operating system Python version is an End of Life Python version that does not align with the 228 | installed Salt version, they will need to upgrade their Python version before being able to pip install Salt. 229 | 230 | Users creating their own packages 231 | --------------------------------- 232 | If a user creates their own packages they will only be impacted if they are packaging for an operating 233 | system that includes an End of Life Python Version. They would need to either create their packages using 234 | salt-bin or include a supported version of Python in their packages. 235 | 236 | When will this happen? 237 | ---------------------- 238 | This Python Version Support change will go into effect as soon as it is approved, but will first impact 239 | the Sodium Release. The Sodium Release is due to be released in June and will support Python versions >= 3.5. 240 | 241 | ## Alternatives 242 | [alternatives]: #alternatives 243 | 244 | - Alternative is to keep supporting End of Life Python versions that line up with supported operating systems 245 | 246 | ## Unresolved questions 247 | [unresolved]: #unresolved-questions 248 | 249 | - Will need to document our Python version support. This documentation should include which versions of Salt 250 | support which versions of Python. This document will need to be updated each release. 251 | 252 | # Drawbacks 253 | [drawbacks]: #drawbacks 254 | 255 | Why should we *not* do this? Please consider: 256 | 257 | - Would impact operating systems we do not package for, if they only have an older Python version available. 258 | -------------------------------------------------------------------------------- /diagrams/Working-group-timelines.svg: -------------------------------------------------------------------------------- 1 | --------------------------------------------------------------------------------