├── .github └── workflows │ ├── when_project_closed.yaml │ └── when_project_created.yml ├── .gitignore ├── .ruby-version ├── 0000-rfc-template.md ├── CODEOWNERS ├── README.md ├── RFCs ├── .gitkeep ├── 0000-make-pdk-modular.md ├── 0001-add-puppet-dev-flag.md ├── 0002-add-pdk-config.md ├── 0003-add-pdk-release.md ├── 0004-add-pdk-new-transport.md ├── 0005-add-pdk-publish.md ├── 0006-add-pdk-console.md ├── 0007-add-pdk-context.md └── 0008-expand-new-task.md ├── appveyor.yml └── tools ├── Gemfile ├── Gemfile.lock ├── README.md ├── SyncProjects.ps1 ├── env-for-act ├── fixtures ├── gh_existing_release_project_created.json ├── gh_new_release_project_created.json ├── gh_nonrelease_project_created.json └── gh_release_project_closed.json ├── when_project_closed.rb └── when_project_created.rb /.github/workflows/when_project_closed.yaml: -------------------------------------------------------------------------------- 1 | name: Project Closed 2 | 3 | on: 4 | project: 5 | types: [closed] 6 | 7 | jobs: 8 | run_ruby_project_closed: 9 | name: Run when_project_closed.rb 10 | runs-on: ubuntu-latest 11 | steps: 12 | - uses: actions/checkout@v2 13 | - uses: ruby/setup-ruby@v1.59.1 14 | with: 15 | ruby-version: 2.6 16 | - name: when_project_closed.rb 17 | env: 18 | GITHUB_TOKEN: ${{ secrets.PDKBOT_GITHUB_TOKEN }} 19 | run: | 20 | [ ! -d /home/runner/.rubies ] && mkdir -p /home/runner && ln -s /github/home/.rubies /home/runner/ 21 | gem install bundler --version 2.0.2 --no-document 22 | cd tools 23 | bundle install --without development --jobs 4 --retry 3 24 | bundle exec when_project_closed.rb 25 | -------------------------------------------------------------------------------- /.github/workflows/when_project_created.yml: -------------------------------------------------------------------------------- 1 | name: New Project Created 2 | 3 | on: 4 | project: 5 | types: [created] 6 | 7 | jobs: 8 | run_ruby_project_created: 9 | name: Run when_project_created.rb 10 | runs-on: ubuntu-latest 11 | steps: 12 | - uses: actions/checkout@v2 13 | - uses: ruby/setup-ruby@v1.59.1 14 | with: 15 | ruby-version: 2.6 16 | - name: when_project_created.rb 17 | env: 18 | GITHUB_TOKEN: ${{ secrets.PDKBOT_GITHUB_TOKEN }} 19 | run: | 20 | [ ! -d /home/runner/.rubies ] && mkdir -p /home/runner && ln -s /github/home/.rubies /home/runner/ 21 | gem install bundler --version 2.0.2 --no-document 22 | cd tools 23 | bundle install --without development --jobs 4 --retry 3 24 | bundle exec when_project_created.rb 25 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | /tools/.bundle 2 | -------------------------------------------------------------------------------- /.ruby-version: -------------------------------------------------------------------------------- 1 | 2.6.3 2 | -------------------------------------------------------------------------------- /0000-rfc-template.md: -------------------------------------------------------------------------------- 1 | # (RFC title goes here) 2 | 3 | ## Summary 4 | 5 | > One paragraph explanation of the feature. 6 | 7 | ## Background & Assumptions 8 | 9 | > What is the context that someone needs to understand the goals of this proposal? This information should be objective 10 | > facts (i.e. not subjective opinions) about the current state of things. 11 | 12 | ## Motivation & Goals 13 | 14 | > Why should we make this change? What problems are we trying to solve? The goals listed here should be not be overly 15 | > entangled with the proposed implementation so that they can be used to compare alternate proposals. 16 | 17 | ## Proposed Implementation Design 18 | 19 | > This is the bulk of the RFC. 20 | 21 | > Explain the design in enough detail for somebody familiar with the project to understand and for a third-party to be 22 | > able to write an implementation. This should get into specifics and consider corner-cases. 23 | 24 | > How to describe a CLI command: 25 | 26 | ``` 27 | $ pdk subcommand action [] [--optional-flag] 28 | ``` 29 | 30 | > Include examples of how the feature would be used! An example example: 31 | 32 | ``` 33 | # Example of using a specific launch pad 34 | $ pdk launch rocket falcon9 --launch-pad=40 35 | pdk (INFO): Initialized launch of 'falcon9' rocket from launch pad '40' 36 | pdk (INFO): Final countdown... 37 | pdk (INFO): Lift off! 38 | Successfully launched 'falcon9' rocket from launch pad '40'! 39 | ``` 40 | 41 | ## Unresolved Questions 42 | 43 | > Optional, but suggested for first drafts. What parts of the design are still TBD? 44 | 45 | ## Future Considerations 46 | 47 | > How might this feature be expanded in the future? Could new proposed actions apply to additional types of content? Is 48 | > the user interaction model extensible? 49 | 50 | ## Drawbacks 51 | 52 | > Why should we *not* do this? There are tradeoffs to choosing any path, please attempt to identify them here. 53 | 54 | ## Alternatives 55 | 56 | > What other designs have been considered? What is the impact of doing nothing? 57 | 58 | > This section could also include prior art, that is, how other projects have solved this problem differently. 59 | 60 | -------------------------------------------------------------------------------- /CODEOWNERS: -------------------------------------------------------------------------------- 1 | # Default owners. 2 | * @puppetlabs/pdk 3 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | [![Build status](https://ci.appveyor.com/api/projects/status/v7w84n21w9yjspq7/branch/main?svg=true)](https://ci.appveyor.com/project/puppetlabs/pdk-planning/branch/main) 2 | 3 | --- 4 | > # PDK Planning Deprecation 5 | > 6 | > This repository has been deprecated in favour of [Discussions in the PDK](https://github.com/puppetlabs/pdk/discussions) repo itself. 7 | > We have ported over any issues raised in this repo to Discussions within the [`pdk`](https://github.com/puppetlabs/pdk) repository. 8 | > The [remaining RFCs](https://github.com/puppetlabs/pdk-planning/tree/main/RFCs) will be converted to [Feature Request Discussions](https://github.com/puppetlabs/pdk/discussions/categories/feature-request) in the [`pdk`](https://github.com/puppetlabs/pdk) repository. 9 | > 10 | > Please take a look at the [Welcome to PDK Discussions](https://github.com/puppetlabs/pdk/discussions/969) for more information. 11 | --- 12 | -------------------------------------------------------------------------------- /RFCs/.gitkeep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/puppetlabs-toy-chest/pdk-planning/a4ac0af751ba0f6e925c83a8524a64b38b97e519/RFCs/.gitkeep -------------------------------------------------------------------------------- /RFCs/0000-make-pdk-modular.md: -------------------------------------------------------------------------------- 1 | # Make PDK Modular 2 | 3 | ## Summary 4 | 5 | One of the greatest features a product can have is the ability for the customer to integrate their own workflows or automations into the product without oversight. The PDK should have a modular system and allow the user to create the plugins of their dreams. 6 | 7 | ## Background & Assumptions 8 | 9 | The PDK would be turned into a modular system thereby allowing users to create features 10 | for private or public use. Without this type of modular system Puppet, Inc is responsbile 11 | for 100% of the changes going into PDK. 12 | 13 | * This puts strain on Puppet 14 | 15 | * Users are frustrated because their ideas cannot be implemented or added quick enough 16 | 17 | * The amount of time it takes an idea to make it to PDK core can often take months due to release cycles 18 | 19 | ## Motivation & Goals 20 | 21 | * Allow companies to create custom private workflows and other automations. 22 | * Allow users to create public plugins for everyone to use and share. 23 | * Reduces strain on Puppet as the community can create plugins more quickly 24 | for certain tasks. 25 | 26 | ## Proposed Implementation Design 27 | Below are some of the key requirements to create a modular system for the PDK 28 | 29 | * A plugin_loader class will search installed gems and internal pdk folders for files 30 | contained in lib/pdk/plugins or for gems that start with "pdk-" 31 | * A plugin base class will define what the plugin must implement to work correctly 32 | * PDK will call a `plugin.execute` method of the plugin to kick off chain of events 33 | * PDK will provide a context object to the plugin which gives the plugin certain actions to call instead of allowing entire access to the PDK. 34 | * limits scope of private code 35 | * makes it easier to see what code can/should be used 36 | * All subcommands should be a plugin, even help 37 | * All plugins should be lazy loaded 38 | * Any commands of a plugin, including the subcommands should be lazy loaded 39 | * All plugins should be classified as two types 40 | 1. internal 41 | 2. external 42 | * Internal plugins cannot be redefined by external plugins 43 | * External plugins can requrie external dependencies 44 | * External plugins are distrubted as gems 45 | * External plugins must be named pdk-* so they can be easily idientified. (* is the name) 46 | * External plugins can bind to a paticular version of PDK gem if required 47 | * A plugin must contain a version (gemspec) 48 | * A plugin must contain a description (gemspec) 49 | * PDK must never require an external plugin 50 | * PDK can package external plugins if deemed worthy 51 | * All plugins will be auto discovered 52 | * Plugins can require other plugins 53 | * Plugins should not implement another CLI system 54 | * Plugins should specifiy a category in which they belong in which is used for plugin info 55 | * workflow 56 | * generator 57 | * test 58 | * validation 59 | * ci 60 | * helper 61 | * release 62 | 63 | ### Templates 64 | * When a plugin requires a template, it shall provide a default template inside the gem 65 | * A plugin will also search the pdk-templates repo first before defaulting to its own internal template. 66 | 67 | 68 | ### Plugin command plugin 69 | * A new plugin command will assist with installing/removing/searching external plugins. 70 | * pass through and filter for `gem` command. 71 | * the gem comamnd can also be used for those in the know 72 | * the plugin command is actually a plugin too :) 73 | 74 | 75 | ``` 76 | pdk plugin install pdk-module-workflows 77 | pdk plugin info pdk-module-workflows 78 | pdk plugin remove pdk-module-workflows 79 | ``` 80 | 81 | ### PDK provides a plugin test harness 82 | The PDK will contain a test harness that will make umit testing plugins simple. This test harness 83 | will test that the plugin abides by the rules set above, and will also exercise mandatory methods that are to be implemented by the plugin. 84 | * Test for existance of template 85 | * Test for existance of correct naming scheme 86 | * Test for existance of correct versioning scheme 87 | * Test for proper template handling if one is available in pdk-templates 88 | * Tests all class methods that are required to be implemented (TBD) 89 | * execute 90 | * description 91 | * version 92 | * Plugin must meet a certain benchmark qualification? (execution time) 93 | * Plugin must not access the internet or network? 94 | 95 | 96 | ### Real world modular examples 97 | There are several puppet related tools that exist that provide a modular system that we can take ideas from. 98 | 99 | * puppet-lint 100 | * puppet-debugger 101 | * vmpooler 102 | * test kitchen 103 | * vagrant 104 | * retrospec 105 | 106 | 107 | ### Pluginator Example 108 | There is even a gem called `pluginator` that facilates a modular system. 109 | This adds a new dependency but also prevents us from having to roll our own plugin finder system. 110 | Find all the plugins in the lib/plugins/pdk directory across all gems. 111 | 112 | ``` 113 | require 'pluginator' 114 | plugins = Pluginator.find('pdk') 115 | ``` 116 | 117 | ### Custom plugin finder example 118 | We can create our own plugin finder system. We should improve upon the vmpooler example but also note that it has some specific use cases as well. 119 | 120 | See the [vmpooler gem](https://github.com/puppetlabs/vmpooler) 121 | 122 | Have a look at the [providers](https://github.com/puppetlabs/vmpooler/blob/master/lib/vmpooler/providers.rb) file and [PROVIDER_API](https://github.com/puppetlabs/vmpooler/blob/master/PROVIDER_API.md) file. 123 | 124 | 125 | ## Unresolved Questions 126 | 127 | * Does CRI allow us to create a modular system easily? 128 | * Should the plugin be able to access the internet or network? 129 | * Not sure if dependency issues will arise with puppet-module-posix* gems 130 | 131 | ## Future Considerations 132 | 133 | * Feature good PDK plugins on an official Puppet site 134 | * Create a bounty system to pay people to create cool plugins 135 | * Turn module rake tasks into plugins (ie. blacksmith) 136 | * Rake loads everything at once. (So slow!!!) 137 | * Some plugins may become so useful they should be added as an internal plugin 138 | 139 | ## Drawbacks 140 | 141 | * Depending on the implementation, the pdk startup time could be impacted. 142 | 143 | ## Alternatives 144 | N/A -------------------------------------------------------------------------------- /RFCs/0001-add-puppet-dev-flag.md: -------------------------------------------------------------------------------- 1 | # Add `--puppet-dev` flag to validate and test unit 2 | 3 | ## Summary 4 | 5 | Introduces a new flag to the `validate` and `test unit` commands that provides a user, with public internet access, to validate and test against the latest puppet source code on GitHub. 6 | 7 | ## Background & Assumptions 8 | 9 | Since the release of v1.5.0, PDK has been able to validate and test against various versions of the Puppet gem via the `--puppet-version` and `--pe-version` flags, however users anticipating upcoming Puppet releases may want to test and validate their modules against the unreleased [Puppet source](https://github.com/puppetlabs/puppet). This feature is geared towards giving module developers or consumers a head start on preparing their modules or installations for the new version of Puppet before it releases. 10 | 11 | ## Motivation & Goals 12 | 13 | - Allow PDK Users to update their modules in preparation for upcoming releases of Puppet. 14 | - Minimizes the risk of upgrading established Puppet installations to new versions of Puppet. 15 | - Promotes more updated modules on the Forge immediately upon each Puppet version release. 16 | 17 | ## Proposed Implementation Design 18 | 19 | Implement a new flag, `--puppet-dev`, to the `pdk test unit` and `pdk validate` subcommands. This flag will only be available to users with access to public internet and is able to access [GitHub](https://github.com). PDK installations in air-gapped or firewalled networks attempting to use this flag will result in a connectivity error, resulting in a non-zero exit code. To get this behavior with environment variables, you can also set `PDK_PUPPET_DEV=true` instead of calling validate and unit test commands with the `--puppet-dev` flag. PDK will result in an error if `PDK_PUPPET_DEV` is set to true in combination with `PDK_PUPPET_VERSION` or `PDK_PE_VERSION` variables being set as well. 20 | 21 | - `pdk validate --puppet-dev` 22 | 23 | Validates metadata and puppet code against the Puppet source on the `main` branch on GitHub. 24 | 25 | If `--puppet-version` or `--pe-version` is specified in addition to `--puppet-dev` this will result in an invalid options error, and return a non-zero exit code. 26 | 27 | The `--puppet-dev` flag will automatically result in running the validation using the latest version of Ruby packaged with PDK. 28 | 29 | - `pdk test unit --puppet-dev` 30 | 31 | Runs unit tests against the Puppet source on the `main` branch on GitHub. 32 | 33 | If `--puppet-version` or `--pe-version` is specified in addition to `--puppet-dev` this will result in an invalid options error, and return a non-zero exit code. 34 | 35 | The `--puppet-dev` flag will automatically result in running the validation using the latest version of Ruby packaged with PDK. 36 | 37 | ## Future Considerations 38 | 39 | In the future, we may add optional arguments to the `--puppet-dev` flag, which will allow the user to specify which version branch to retrieve from GitHub. 40 | 41 | Examples: 42 | 43 | - `pdk validate --puppet-dev` 44 | 45 | This will result in the default behavior defined above. 46 | 47 | - `pdk validate --puppet-dev=5.5.x` 48 | 49 | This will reset the cloned Puppet repo to the `5.5.x` branch, and run validation against that. 50 | 51 | PDK will match the major and minor version from this branch with the built-in versions map and decide which Ruby to use to run validation or tests against, branches that fail matching will default to the latest Ruby packaged with PDK. E.g. `5.5.x` will run against Ruby 2.4.4 and `4.10.x` will run against Ruby 2.1.9. 52 | 53 | - `pdk validate --puppet-dev=5` 54 | 55 | The option will require an explicit branch name. Unmatched branches will result in an error and a non-zero exit code. 56 | 57 | ## Drawbacks 58 | 59 | A drawback with this new feature is the requirement for public network access. Air-gapped users will be unable to use this feature. However, this feature should not change or impact the behavior of the basic validation and test unit functions previously defined. 60 | 61 | ## Alternatives 62 | 63 | Alternatives discussed include potentially using packaged puppet-agent nightly builds, either installing them or requiring the user to install them so that PDK can access the puppet commands. We decided this option was more complicated to develop, in addition to potentially more complicated for the user. 64 | -------------------------------------------------------------------------------- /RFCs/0002-add-pdk-config.md: -------------------------------------------------------------------------------- 1 | # Add `pdk config` subcommand 2 | 3 | ## Summary 4 | 5 | Introduce a PDK and project configuration subsystem with a new `pdk config` subcommand providing the user interface 6 | to read and write configuration keys and values. 7 | 8 | ## Background & Assumptions 9 | 10 | Prior to the introduction of the Puppet Development Kit, module development involved the coordination of a set of 11 | distinct tools, rake tasks, etc. which each carried their own implicit and explicit mechanisms for customization and 12 | configuration. 13 | 14 | While the PDK has made a lot of strides in providing a unified invocation interface for these tools, thus far we have 15 | not addressed configuration side of the way module developers interact with these tools. 16 | 17 | To illustrate this, here is a survey of the current file-based configuration landscape for a single module: 18 | 19 | - metadata.json 20 | - .sync.yml 21 | - .fixtures.yml 22 | - .nodeset.yml 23 | - .pdkignore 24 | - .puppet-lint.rc 25 | - .rspec 26 | - .rubocop[\_todo].yml 27 | - .yardopts 28 | - Rakefile 29 | 30 | (Plus the inherent configuration and customization contained within the puppetlabs\_spec\_helper gem and it’s provided 31 | rake tasks, some of which the PDK invokes.) 32 | 33 | ## Motivation & Goals 34 | 35 | - Standardize the way existing module development workflows are configured and customized for PDK users. 36 | 37 | - Reduce the number of different ways that module development workflows with PDK are configured and customized. 38 | 39 | - Resolve some inconsistencies and design challenges presented by existing PDK features. (See template-url/template-ref 40 | design discussion and some issues we have had with invalid values cached in answers.json) 41 | 42 | - Improve the discoverability of module development configuration and customization. **Currently many of the avenues for 43 | configuration are based around hidden files and their significance is not obvious to someone who is new to module 44 | development.** 45 | 46 | ## Proposed Implementation Design 47 | 48 | Implement a new pdk subcommand, `pdk config` which will offer the following actions: 49 | 50 | Ref: (Verb-Noun Reference)[https://docs.google.com/document/d/1zX0FJBAvAIK3d3L3QemD2FGQo3EsgMOD0Ha1BYHVYJE] 51 | 52 | - `pdk get config [--format=]` 53 | 54 | Lists the complete, currently resolved configuration, merging all available layers of config and presenting the 55 | formatted results. 56 | 57 | If run from within a project, will show system, user and project level config. 58 | 59 | If run outside of a project, will only show system and user level config with an indication to the user that the command was not invoked from within a project. 60 | 61 | 62 | - `pdk get config [--format=]` 63 | 64 | Retrieves a specific value from the resolved configuration. 65 | 66 | If `` is a leaf node of the configuration graph, this command will return the raw value: 67 | 68 | ``` 69 | $ pdk get config user.default_template.url 70 | https://github.com/puppetlabs/pdk-templates.git 71 | ``` 72 | 73 | If `` is a non-leaf node of the configuration graph, this command will return a structure containing the values 74 | of all sub-keys of the given key: 75 | 76 | ``` 77 | $ pdk get config user.default_template 78 | user.default_template.url = https://github.com/puppetlabs/pdk-templates.git 79 | user.default_template.ref = main 80 | ``` 81 | 82 | You can also supply the `--format` option to control how keys and values are presented: 83 | 84 | ``` 85 | $ pdk get config user.default_template.url --format=json 86 | { "user.default_template.url": "https://github.com/.../pdk-templates.git" } 87 | ``` 88 | 89 | ``` 90 | $ pdk get config user.default_template --format=json 91 | { "user.default_template.url": "https://github.com/.../pdk-templates.git", "user.default_template.ref": "main" } 92 | ``` 93 | 94 | - `pdk set config [--type|--as ] [--force] []` 95 | 96 | Sets, updates, or adds to the value(s) in the given configuration key and outputs the new value(s). 97 | 98 | Note that while `` is optional, it can only be omitted in special cases. For normal usage `` is required. 99 | 100 | If `` is not a leaf node, create an equivalent hash structure. 101 | 102 | If `` already has a value set, issues a notice before replacing existing value. 103 | 104 | If `` is designed to store multiple values (e.g. a list or an array) and `--force` is not used: 105 | 106 | - If the current value list is empty, set the first value in the list to the provided ``. 107 | 108 | - If the current value list is not empty and `` is not already present in the list, add `` to the 109 | list. 110 | 111 | - If `` is already present in the value list, issues a notice that value was already on list, outputs the 112 | current values of ``, and exits with a success status. 113 | 114 | If the `--force` flag is set then the `` will be literally set to ``. That is, list modifications like above will not be used. 115 | This flag is useful when resetting configuration to a known good state, or when automating the PDK to configure projects on first use. 116 | 117 | If `` is a configuration key where we cannot infer what type of value it should be, the user can include the `--type` (aliased with `--as`) flag 118 | to indicate what type of value it should be. The included types are those common to JSON and YAML documents (String, Number, Boolean, Array, Hash and Null) 119 | 120 | - `--type number`. Treats the value as a number. For example `1`, `1.0`, `-1.0` 121 | 122 | - `--type boolean`. Treats the value as a boolean (True/False). For example `true`, `TRUE`, `False`, `yes`, `no`, `0` (False), `-1` (True) 123 | 124 | - `--type array`. Treats the value as a array element. For example `value` would become an array with a single string element of value, that is `["value"]` 125 | 126 | - `--type array`. If given no value, treats the value as an empty array, that is `[]` 127 | 128 | - `--type empty|nil|null|nul`. See [Future Considerations](#future-considerations) 129 | 130 | - `--type string`. The default type for any value. 131 | 132 | Note that `--type hash` does not exist, as this is implied using the ``, for example a key of `setting.a.b.c` will create a Hash like 133 | 134 | ``` json 135 | "setting": { 136 | "a": { 137 | "b": { 138 | "c": "" 139 | } 140 | } 141 | } 142 | ``` 143 | 144 | Basic usage examples: 145 | 146 | - Known single value key, initial value: 147 | 148 | ``` 149 | $ pdk set config user.default_template.url "https://github.com/scotje/pdk-templates.git" 150 | pdk (INFO): Set initial value of user.default_template.url to "..." 151 | https://github.com/scotje/pdk-templates.git 152 | ``` 153 | 154 | - Known single value key, update value: 155 | 156 | ``` 157 | $ pdk set config user.default_template.url "https://github.com/example/repo.git" 158 | pdk (INFO): Changed existing value of user.default_template.url from "..." to "..." 159 | https://github.com/example/repo.git 160 | ``` 161 | 162 | - Known multi-value key, initial value: 163 | 164 | ``` 165 | $ pdk set config module.multi.example apple 166 | pdk (INFO): Added new value "apple" to module.multi.example 167 | apple 168 | ``` 169 | 170 | - Known multi-value key, additional value: 171 | 172 | ``` 173 | $ pdk set config module.multi.example banana 174 | pdk (INFO): Added new value "banana" to module.multi.example 175 | apple 176 | banana 177 | ``` 178 | 179 | - Known multi-value key, duplicate value: 180 | 181 | ``` 182 | $ pdk set config module.multi.example apple 183 | pdk (INFO): No changes made to module.multi.example as it already contains value "apple" 184 | apple 185 | banana 186 | ``` 187 | 188 | - User-defined key, initial, add, then force a value: 189 | 190 | ``` 191 | $ pdk set config --type array module.x.example strawberry 192 | pdk (INFO): Set initial value of module.x.example to array with item "strawberry" 193 | strawberry 194 | 195 | $ pdk set config module.x.example orange 196 | pdk (INFO): Added new value "orange" to module.x.example 197 | strawberry 198 | orange 199 | 200 | $ pdk set config --type boolean --force module.x.example Yes 201 | pdk (INFO): Set initial value of module.x.example to true 202 | true 203 | ``` 204 | 205 | - Invalid type conversion: 206 | 207 | ``` 208 | $ pdk set config --type boolean module.x.example strawberry 209 | pdk (ERROR): Unable to convert 'strawberry' into a boolean 210 | 211 | $ pdk set config --type number module.x.example strawberry 212 | pdk (ERROR): Unable to convert 'strawberry' into a number 213 | ``` 214 | 215 | - `pdk remove config [|--force]` 216 | 217 | Unset one more more values from the given configuration key. 218 | 219 | If `` is not a leaf node, exits non-zero with an error message. 220 | 221 | If `` currently stores no value, issues a warning but exits with a success status. 222 | 223 | If `` currently stores a single value: 224 | 225 | - If no `` is provided, unsets ``. 226 | 227 | - If provided ``, ignores `` and unsets `` 228 | 229 | - If provided `--force`, issues an info message, and unsets `` 230 | 231 | If `` currently stores multiple values: 232 | 233 | - If no `` is provided, clears all elements in `` 234 | 235 | - If provided `` is present in the list of values for ``, removes value from list. This is matched via string conversion. Unlike `pdk set` there is type conversion available. 236 | 237 | - If provided `` is not present in the list of values for ``, issues a warning but exits with a 238 | success status. 239 | 240 | - If passed `--force`, unsets `` 241 | 242 | Settings with default values create an interesting UX problem. For example, let's say we have a setting called `user.setting` with a default value of `default-string`. When I run `pdk get config` I end up with: 243 | 244 | ``` 245 | $ pdk get config 246 | ... 247 | user.setting=default-string 248 | ``` 249 | 250 | But if I try to remove the setting, and then get its value: 251 | 252 | ``` 253 | $ pdk remove config user.setting 254 | pdk (INFO): Removed 'user.setting' which had a value of 'default-string' 255 | user.setting= 256 | 257 | $ be pdk get config 258 | ... 259 | user.setting=default-string 260 | ``` 261 | 262 | This makes no sense as I just removed it, why is it back? The current UX for the configuration system does not tell the user whether a setting is defaulted or explicitly set. So the UX for the `pdk remove config` command must also take into account default values. It should log a message to the user that it attempted to remove the setting, but it is now using a default value. An example is shown below in the "Basic usage examples" 263 | 264 | Basic usage examples: 265 | 266 | - Known single value key, initial value: 267 | 268 | ``` 269 | $ pdk remove config user.setting 270 | pdk (INFO): Removed 'user.setting' which had a value of 'http://somewhere' 271 | user.setting= 272 | ``` 273 | 274 | - A key that doesn't exist 275 | 276 | ``` 277 | $ pdk remove config user.missing-setting 278 | pdk (INFO): Could not remove 'user.missing-setting' as it has not been set 279 | ``` 280 | 281 | - Removing a single value from an array 282 | 283 | ``` 284 | $ pdk remove config user.animal kangaroo 285 | pdk (INFO): Removed 'kangaroo' from 'user.animal' 286 | user.animal=["quokka"] 287 | ``` 288 | 289 | - Removing all values from an array 290 | 291 | ``` 292 | $ pdk remove config user.animal 293 | pdk (INFO): Cleared 'user.animal' which had a value of '["quokka", "kangaroo"]' 294 | user.animal=[] 295 | ``` 296 | 297 | - Removing an array 298 | 299 | ``` 300 | $ pdk remove config user.animal --force 301 | pdk (INFO): Removed 'user.animal' which had a value of '["quokka", "kangaroo"]' 302 | user.animal= 303 | ``` 304 | 305 | - Removing part of a hash 306 | 307 | ``` 308 | $ pdk get config user.setting 309 | {"foo"=>"bar", "hash"=>{"animals"=>["quokka"]}} 310 | 311 | $ pdk remove config user.setting.hash 312 | pdk (INFO): Removed 'user.setting.hash' which had a value of '{"animals"=>["quokka"]}' 313 | user.setting.hash= 314 | 315 | $pdk get config user.setting 316 | {"foo"=>"bar"} 317 | ``` 318 | 319 | - Removing a setting that has a default value: 320 | 321 | ``` 322 | $ pdk get config 323 | ... 324 | user.setting=current-value 325 | 326 | $ pdk remove config user.setting 327 | pdk (INFO): Could not remove 'user.setting' as it using a default value of 'default-string' 328 | user.setting=default-string 329 | 330 | $ pdk get config 331 | ... 332 | user.setting=default-string 333 | ``` 334 | 335 | ### Proposed Configuration Keys 336 | 337 | #### System Config 338 | 339 | The values for these config keys will be persisted in `/opt/puppetlabs/pdk/config` (`%ProgramData%\PuppetLabs\PDK` on Windows). 340 | They are intended to be system-specific settings and will apply to all developer working on the same computer. 341 | 342 | These would typically be system level defaults of the same User/Local Config settings defined below. For example: 343 | 344 | In the RGBank company, they have their own PDK template store. 345 | To make it easier for onboarding new developers this is stored in the `system.module_defaults.template.url` key. 346 | Developers can choose to override this, by setting `user.module_defaults.template.url` 347 | 348 | Setting precedence is discussed in more detail below. 349 | 350 | #### User/Local Config 351 | 352 | The values for these config keys will be persisted in `~/.config/pdk/user_config.json` (`%LOCALAPPDATA%\PDK\user_config.json` on Windows). 353 | They are intended to be developer-specific settings and will not apply to another developer working on the same codebase. 354 | 355 | | Key | Description | 356 | | --- | --- | 357 | | user.module\_defaults.template.url | Default template URL to use when generating a new module. | 358 | | user.module\_defaults.template.ref | Default template ref to use when generating a new module. (E.g. branch or tag.) | 359 | | user.module\_defaults.metadata.author | Default value for the “author” key in metadata.json for a new module. Supersedes previous ~/.pdk/cache/answers.json functionality. | 360 | | user.module\_defaults.metadata.license | Default value for the “license” key in metadata.json for a new module. Supersedes previous ~/.pdk/cache/answers.json functionality. | 361 | | user.forge.username | Primary Forge username of this developer. Note that this may be an organization rather than a personal username. Used as the default value in metadata.json when generating a new module. Supersedes previous ~/.pdk/cache/answers.json functionality. | 362 | | [user.forge.api\_key] | Placeholder for future Forge API interactions. | 363 | | [user.forge.api\_secret] | Placeholder for future Forge API interactions. | 364 | 365 | #### Project-specific Config 366 | 367 | The values for these config keys will be persisted in a new .pdk/config file within the project. 368 | Currently a project is only a Puppet Module but in the future this may extend to Control Repos or Bolt projects. 369 | The new file is intended to be checked into VCS and shared with other developers to ensure consistent application of the relevant PDK commands. 370 | 371 | For example, in a Module Project: 372 | 373 | | Key | Description | 374 | | --- | --- | 375 | | project.template.url | Template URL to use for this module. | 376 | | project.template.ref | Template ref to use for this module. Note that this will differ from what is currently stored in metadata.json, this value will be the “desired” reference (e.g. a branch name) but not the specific commit that the module was last updated to. | 377 | | project.template.custom.[...] | Filename based keys with associated config hashes to govern how the template should be applied to this specific module. Supersedes existing .sync.yml settings. | 378 | | project.test.fixtures.[...] | List of additional modules, etc. and associated config hashes needed in order to run unit tests for this module. Supersedes existing .fixtures.yml settings. | 379 | | project.x.[...] | Arbitrary keys and values to be utilized by users to persist data between module developers. Can be used as a basic shared data store for custom developer Rake tasks, etc. | 380 | 381 | ### More About `project.x` 382 | 383 | The `project.x` configuration key space is intended to be used by project developers and third-party tool developers to 384 | persist configuration data that is relevant to all developers of a project. Ultimately we want to avoid any need for 385 | users to add additional configuration files to their projects beyond what `pdk config` uses. 386 | 387 | We will define and document best practices around the usage of the `project.x` key space for the community, including 388 | things like: 389 | 390 | - Namespace your custom configuration settings under a key that matches the name of your organization or tool name, 391 | e.g. `project.x.mycompany`, `project.x.voxpupuli`, or `project.x.super_module_tool`. 392 | 393 | - If your custom configuration is tool-specific, include a key and value that you can use to version your other 394 | configuration data. That way you can significantly change the configuration format for your tool in the future 395 | without breaking older versions. E.g. `project.x.super_module_tool.config_version = 1` 396 | 397 | - Plus any other best practices we come up with. 398 | 399 | ### Setting Precedence 400 | 401 | The settings will typically be enforced using the following order: 402 | 403 | 1. Project 404 | 2. User/Local 405 | 3. System 406 | 407 | Project settings have the highest precedence whereas System settings have the lowest. 408 | That is, if a setting is defined in both Project and System levels, only the Project level will take affect. 409 | 410 | Note that **typically** only the User and System settings will overlap. 411 | It is expected that there will be few, if at all, settings that makes sense in the Project and User levels 412 | 413 | ## Unresolved Questions 414 | 415 | - Should we assign expected types to the pre-defined configuration keys so that we can validate new values and report 416 | errors if the value cannot be coerced to the expected type? 417 | 418 | - _This seems like it will help avoid a lot of pain on the consumer-side of the `pdk config` API and we already have 419 | the bits in place to validate using JSON-Schema so I recommend we implement types and type validation/coericion for 420 | all the non-arbitrary keys and values._ 421 | 422 | ## Future Considerations 423 | 424 | - We should ensure that the key/value structure we set up is sufficiently extensible for future PDK functionality such 425 | as control-repo administration, etc. 426 | 427 | - `--type empty|nil|null|nul` In the future, the PDK may distinguish between an empty or nul value versus a missing value. This could be done using the `--type` keyword. 428 | 429 | ## Drawbacks 430 | 431 | - There is a lot of existing documentation and accumulated tribal knowledge in the community around how to configure 432 | module development through things like `.sync.yml`. 433 | 434 | - Some of the issues here can be mitigated by integrating automatic migration of relevant configuration from files 435 | like `.sync.yml`, `.fixtures.yml`, etc. into `pdk config` equivalents. 436 | 437 | - Existing tooling like `pdksync`, etc. will have to be updated to use the `pdk config` subsystem. This includes any 438 | tools that PDK uses internally. 439 | 440 | 441 | ## Alternatives 442 | 443 | - Maintain status quo, with improved documentation and education about what can be configured, which files to update, 444 | etc. 445 | 446 | - Create a similar `pdk config` subcommand but have a translation layer that actually persists configuration in the 447 | existing files, rather than in new files. 448 | -------------------------------------------------------------------------------- /RFCs/0003-add-pdk-release.md: -------------------------------------------------------------------------------- 1 | # Add `pdk release` Subcommand 2 | 3 | ## Summary 4 | 5 | Add a new `pdk release` subcommand with multiple actions to allow us to separate release preparation and release 6 | packaging. 7 | 8 | ## Background & Assumptions 9 | 10 | - It is sub-optimal that the current implementation of the `pdk build` subcommand can have side-effects on the module's 11 | code before it is packaged. For example, if the module’s metadata needs to be modified to make the module Forge 12 | compatible, the contents of the tarball produced by `pdk build` may no longer match what the developer has committed 13 | (and possibly tagged) in their version control system. 14 | 15 | - This issue will only get worse if we add more “preflight” style tasks into the existing `pdk build` (e.g. changelog 16 | generation, version bump checking, etc.) 17 | 18 | ## Motivation & Goals 19 | 20 | - PDK should provide an easy way for module authors to package and distribute their modules via the Puppet Forge. 21 | 22 | - PDK should provide composable commands and actions so that module authors can automate the packaging and distribution 23 | of their modules, and integrate those steps into a comprehensive CI/CD pipeline. 24 | 25 | - PDK should encourage module authors to adopt good development and release practices, such as ensuring that distributed 26 | release artifacts are consistent with tagged content in their version control system. 27 | 28 | - PDK subcommands should be reasonably extensible without any single command or action being overloaded with multiple, 29 | distinct functions. 30 | 31 | ## Proposed Implementation Design 32 | 33 | Implement a new pdk subcommand, `pdk release` which will offer the following actions. 34 | 35 | - `pdk release prep` 36 | 37 | - Performs all the pre-release checks to ensure module is ready to be packaged, prompts user for missing info, 38 | modifies files on disk as needed, etc. 39 | 40 | - User is directed to review changes and commit the new state of the module’s code to their version control system. 41 | (The exact contents of this prompt can be predicated on whether or not there is VCS metadata present in the module’s 42 | working directory.) 43 | 44 | - Ideally the user would be able to customize the pre-release checks that were performed, including defining their own 45 | custom checks. This customization could be implemented through the [proposed `pdk config` 46 | subcommand](https://github.com/puppetlabs/pdk-planning/pull/4). 47 | 48 | - When run non-interactively (e.g. as part of a continuous delivery pipeline), it will exit non-zero if it would 49 | require any user input to complete successfully. 50 | 51 | - `pdk release build [--target-dir=]` 52 | 53 | - Replicates the existing `pdk build` functionality with the modification that it will fail and exit non-zero if any 54 | of the checks performed by `pdk release prep` would result in modification to the module’s code. 55 | 56 | - If there is VCS metadata present in the module, this action could also fail and exit non-zero if there are changes 57 | to the module’s code that have not been committed to VCS. 58 | 59 | - If there is VCS metadata present in the module, this action could also automatically add a tag to the VCS repository 60 | based on the module’s version string, or direct the user to do so. 61 | 62 | - This command has no interactive elements, it either succeeds or fails. 63 | 64 | - `pdk release push [tarball] [--force]` 65 | 66 | - _All functionality of this action is pending availability of proper Forge publish API._ 67 | 68 | - Publishes the given module tarball to the Forge. 69 | 70 | - When not passed a path to a tarball, looks in `metadata.json` for the current version and then looks for a tarball 71 | in the default build path matching that version and pushes it to Forge. Exits non-zero if it can’t find a tarball 72 | matching the current metadata.json version. 73 | 74 | - Defaults to asking for confirmation when run interactively, but can be disabled with the `--force` flag. Note that a 75 | push will still fail, regardless of the `--force` flag, if a release of the module with the same version already 76 | exists on the Forge. 77 | 78 | - When run non-interactively (e.g. as part of a continuous delivery pipeline), it will default to not asking for 79 | confirmation (`--force` behavior) and exit non-zero if it would require any user input to complete successfully. 80 | 81 | ## Unresolved Questions 82 | 83 | - It seems like we may need to change PDK's user interface from a `pdk ` pattern to a `pdk ` pattern 84 | in the near future, so maybe this command should wait until it can be introduced as `pdk module release`? 85 | 86 | - Should `pdk release prep` perhaps be called `pdk release check` instead? 87 | 88 | ## Future Considerations 89 | 90 | Once the Forge API prerequisites for `pdk release push` are completed, we could also add: 91 | 92 | - `pdk release delete [--force] ` 93 | 94 | - When given a single module release version, marks that version as “deleted” on the Forge. 95 | 96 | - When invoked with the `--all` flag, marks all released versions of the current module as “deleted” on the Forge. 97 | 98 | - Deletion reason is a required argument. 99 | 100 | - Defaults to asking for confirmation when run interactively, but confirmation prompt can be skipped with the 101 | `--force` flag. 102 | 103 | - When run non-interactively (e.g. as part of a continuous delivery pipeline), it will default to not asking for 104 | confirmation (`--force` behavior) and exit non-zero if it would require any user input to complete successfully. 105 | 106 | However this functionality might also make more sense in a new, non-module specific PDK subcommand that includes other 107 | Forge-specific actions. 108 | 109 | ## Drawbacks 110 | 111 | - We just recently introduced the `pdk build` subcommand so it's not ideal to already be deprecating it. 112 | 113 | - This is introducing additional steps and more cognitive complexity to the release process for a module. 114 | 115 | ## Alternatives 116 | 117 | - Status quo, however there will be continued tension around adding any new behaviors to `pdk build`. Also `pdk build` 118 | is a module-specific command and may be confusing when we add control-repo functions to PDK. 119 | -------------------------------------------------------------------------------- /RFCs/0004-add-pdk-new-transport.md: -------------------------------------------------------------------------------- 1 | # Add `pdk new transport` Subcommand 2 | 3 | ## Summary 4 | 5 | Add a new `pdk new transport` subcommand to create a ready-to-code Resource API transport, schema and test skeleton, similar to `pdk new provider`. 6 | 7 | ## Background & Assumptions 8 | 9 | - The Resource API (and through bundling Puppet 6.4+ and bolt 1.14+) support _transports_ as a means to manage remote resources and execute tasks for systems that do not follow the conventional UNIX/Windows model of on-system execution. 10 | 11 | - Building resources and tasks for this kind of system requires extra coding to facilitate the communication with those systems. 12 | 13 | - A transport schema (like a type) describes the information required to communicate with a remote target. 14 | 15 | - A transport (like a provider) implements the communication with the remote target. 16 | 17 | - See https://github.com/puppetlabs/puppet-specifications/blob/master/language/resource-api/README.md#transports for the full specification. 18 | 19 | ## Motivation & Goals 20 | 21 | - PDK should provide an easy way for module authors to build advanced content. 22 | 23 | - PDK should encourage module authors to adopt good development practices, such as maintaining unit test coverage of their code. 24 | 25 | ## Proposed Implementation Design 26 | 27 | Implement a new pdk subcommand, `pdk new transport ` which will render a set of templates from `object_templates/transport_*` using the provided transport name. 28 | 29 | This will build on the `pdk new provider` work for multi-file templates, and will contain the following templates: 30 | 31 | - `transport.erb`: the main file containing the implementation 32 | - `transport_spec.erb`: an example unit test covering the implementation skeleton 33 | - `transport_schema.erb`: an example schema containing default attributes recommended by us: `host`, `port`, `user`, `password`. 34 | 35 | ## Unresolved Questions 36 | 37 | - like `pdk new provider` the transport templates will require the `puppet-resource_api` gem available and `mocks_with: rspec` to be set. Are we far along the route to make those two settings the default? At least for new modules? 38 | 39 | ## Future Considerations 40 | 41 | - If we solve the issue above, we also can fix up `pdk new provider` to use the same mechanism, and remove the `[experimental]` tag. 42 | 43 | ## Drawbacks 44 | 45 | - committing to a template raises the bar for support and backwards compatibility. 46 | 47 | ## Alternatives 48 | 49 | - Expect developers to copy/paste code examples from the docs 50 | -------------------------------------------------------------------------------- /RFCs/0005-add-pdk-publish.md: -------------------------------------------------------------------------------- 1 | # Add `pdk publish` subcommand 2 | 3 | ## Summary 4 | 5 | Add a new `pdk publish` subcommand that allows publishing a given module tarball to the Forge. 6 | *Note that this RFC supersedes [RFC-0003](https://github.com/puppetlabs/pdk-planning/blob/main/RFCs/0003-add-pdk-release.md). While RFC-0003 may eventually still be implemented (likely as part of a PDK 2.x major release), this work represents a smaller scope that is more appropriate to the current PDK 1.x series.* 7 | 8 | ## Background & Assumptions 9 | 10 | - With the addition of module management endpoints to the Forge API and authentication via API keys, [authenticating](https://forgeapi.puppet.com/#section/Authentication) and [publishing](https://forgeapi.puppet.com/#operation/addRelease) to the Forge has a public and stable interface. 11 | 12 | - Publishing to the Forge is a natural progression beyond the current `pdk build` subcommand, and is an action that may be performed manually or through an automated process. 13 | 14 | ## Motivation & Goals 15 | 16 | - PDK should provide an easy way for module authors to package and distribute their modules via the Puppet Forge. 17 | 18 | - PDK should provide composable commands and actions so that module authors can automate the packaging and distribution of their modules, and integrate those steps into a comprehensive CI/CD pipeline. 19 | 20 | 21 | ## Proposed Implementation Design 22 | 23 | Implement a new pdk subcommand, `pdk publish`, with the following syntax, options, and arguments. 24 | 25 | - Basic syntax: 26 | - `pdk publish [--package-dir=] [--force] [tarball.tar.gz]` 27 | 28 | 29 | - Options and arguments: 30 | - `--package-dir=` 31 | - Use given path as path to find tarball to publish. 32 | 33 | - `--force` 34 | - Do not prompt for confirmation. 35 | 36 | - `tarball.tar.gz` 37 | - Relative or absolute path to tarball to publish. 38 | 39 | - Option combination behaviors: 40 | - `(no options/arguments)` 41 | - Looks in current module’s `metadata.json` for namespace, module name, and version and then attempts to publish `--.tar.gz` from default package path. Default package path is located at `/pkg`, where `` is the directory containing the module's `metadata.json` file. 42 | - If file matching the `--.tar.gz` pattern cannot be found in the default package path, exits with an error suggesting that user needs to use `pdk build` first. 43 | 44 | 45 | - `tarball.tar.gz` 46 | - Attempts to publish given pathed tarball. 47 | - If given tarball cannot be found, exits with an error. 48 | 49 | 50 | - `--package-dir=` 51 | - Looks in current module’s `metadata.json` for namespace, module name, and version and then attempts to publish `--.tar.gz` from given path. 52 | - If file matching the `--.tar.gz` cannot be found in the given path, exits with an error. 53 | 54 | 55 | - `--package-dir= tarball.tar.gz` 56 | - Logs a warning that `--package-dir` option is being ignored in favor of pathed tarball argument. 57 | - Attempts to publish given pathed tarball. 58 | - If given pathed tarball cannot be found, exits with an error. 59 | 60 | 61 | - `(any valid options/args) --force` 62 | - Attempts to publish without stopping to ask for confirmation. 63 | - Note that command will still fail, regardless of the `--force` flag, if a release of the module with the same version already exists on the Forge. 64 | 65 | - Defaults to asking for confirmation when run interactively, but this can be disabled. (See `--force` flag.) 66 | 67 | - When run non-interactively (e.g. as part of a continuous delivery pipeline), it will default to not asking for confirmation (`--force` behavior) and exit non-zero if it would require any user input to complete successfully. 68 | 69 | - Need to track namespace/api-key mapping in user-level config. 70 | - Look inside tarball to determine namespace 71 | - Check to see if config contains an api-key for that namespace 72 | - Prompt for api-key if not (can use existing wrappers around tty-prompt) 73 | 74 | - Need to be able to provide api-key as an environment variable with the name PDK_FORGE_KEY 75 | 76 | - As much as possible, the HTTP calls to the Forge API should be implemented in the puppet_forge gem and invoked from PDK, to allow sharing of the API interaction logic across projects. 77 | 78 | ### Examples 79 | 80 | #### Publishing a tarball matching data from metadata.json when no options / arguments are provided 81 | ``` 82 | $ pdk publish 83 | pdk (INFO): Attempting to publish puppetlabs-mysql-0.1.0.tar.gz 84 | 85 | Once published to the Forge, your module will be made available for public download. Continue? (y/N) yes 86 | pdk (INFO): Authenticating to the Forge API using key with namespace: puppetlabs 87 | Publish successful, new release available at https://forge.puppet.com/v3/releases/puppetlabs-mysql-0.1.0 88 | ``` 89 | 90 | #### Attempting to publish a release that already exists on the Forge, bypassing confirmation with `--force` 91 | ``` 92 | $ pdk publish --force 93 | pdk (INFO): Attempting to publish puppetlabs-mysql-0.1.0.tar.gz 94 | pdk (INFO): Authenticating to the Forge API using key with namespace: puppetlabs 95 | Error: 409 Conflict, a release version 0.1.0 for module puppetlabs-mysql already exists on the Forge. To create a new version of your module to publish, update the version number in metadata.json and run `pdk build`. 96 | ``` 97 | 98 | #### Prompt for API key with pathed tarball 99 | ``` 100 | $ pdk publish ../dist/puppetlabs-mysql-9.1.1.tar.gz 101 | pdk (INFO): Attempting to publish puppetlabs-mysql-0.1.1.tar.gz 102 | 103 | Once published to the Forge, your module will be made available for public download. Continue? (y/N) yes 104 | No API key was found for namespace: puppetlabs in PDK user config or as an environment variable. If you have an API key you'd like to use, enter it now: 105 | --> ************** 106 | 107 | pdk (INFO): Authenticating to the Forge API using key with namespace: puppetlabs 108 | Publish successful, new release available at https://forge.puppet.com/v3/releases/puppetlabs-mysql-0.1.1 109 | ``` 110 | 111 | ## Future Considerations 112 | 113 | - In addition to release publishing, the recent Forge API additions include endpoints for release and module deletion, and module deprecation. We may eventually want to provide PDK users access to any or all of these actions through some combination of subcommands and options. 114 | 115 | ## Drawbacks 116 | 117 | - If we choose to move forward with RFC-0003 or similar implementation at a later date, we may add to a user’s cognitive overhead in changing publish commands, and more generally, in switching from `pdk ` to `pdk `. This is a drawback that would not be limited to this particular subcommand though. 118 | 119 | ## Alternatives 120 | 121 | - As noted above, the proposed implementation supersedes RFC-0003. An alternative would be to move forward with the proposal in RFC-0003, which includes a publish action for the proposed `pdk release` subcommand. 122 | -------------------------------------------------------------------------------- /RFCs/0006-add-pdk-console.md: -------------------------------------------------------------------------------- 1 | # Add `pdk console` subcommand 2 | 3 | ## Summary 4 | 5 | This RFC is for adding a new command called `console` that would invoke the 6 | puppet debugger [REPL](https://en.wikipedia.org/wiki/Read%E2%80%93eval%E2%80%93print_loop_) when executed. This allows the puppet developer to easily test out language constructs and answer questions users might have about puppet code without jumping to external resources or running puppet apply. The console commamd would be invoked with or without a module scope and would be similar to having ruby's IRB, python's REPL, Node's REPL, ChefDK REPL, 'insert language' REPL within easy reach. 7 | 8 | 9 | ## Background & Assumptions 10 | 11 | The PDK brings in multiple versions of Ruby and it is not clear how the puppet debugger or any other gem not part of the PDK should be installed. Currently the debugger can be installed at the module level within the Gemfile or at a more "global level" so it can be invoked without the module. However, the PDK doesn't really expose a way to install global gems unless you know Ruby well. So a user would have to run `/opt/puppetlabs/pdk/private/ruby/2.4.5/bin/gem install puppet-debugger` to install the debugger at the PDK global level. 12 | 13 | Then run `/opt/puppetlabs/pdk/private/ruby/2.4.5/bin/puppet debugger` to invoke it. 14 | 15 | A user could then modify their PATH environment variable to include `/opt/puppetlabs/pdk/private/ruby/2.4.5/bin/` which would make it easier but not optimal for the user. 16 | 17 | Additionally, there is still the question of which Ruby version do we use? 18 | 19 | The other way of invoking the debugger is through the bundle exec passthrough. ie. `pdk bundle exec puppet debugger`. This means the user will need to add the puppet-debugger gem to their .sync file and run `pdk update`. Once this is complete the user will be able to run the debugger within the context of the their module. Personally, I hate typing long commands so this seems aching for a new pdk command. 20 | 21 | ## Motivation & Goals 22 | 23 | It really comes down to pure laziness. The debugger is the first thing I install after the pdk. It should be included in the PDK and to keep all other PDK users from having to install what I think is a very neccessary development tool. 24 | 25 | There also needs to be an easy way to invoke the debugger. Without the PDK, invoking the debugger command was simple. `puppet debugger`. However, if the PDK is involved this is not possible anymore. The PDK is just a broker to run other commands that are santioned. 26 | 27 | Another point here is that the debugger was not built by puppet so many users are not aware of its existance because it is not publizied by puppet. Including this tool in the PDK should ensure that a greater audience has exposure to it. 28 | 29 | I would also wager that many questions that are often asked in the forum or slack channel would be self answered 30 | via the debugger. So is it important that we put this tool in the hands of the developer with greatest ease. 31 | 32 | The motivation behind this is to make it dead simple how to start the debugger no matter which context is used. 33 | 34 | ## Proposed Implementation Design 35 | We are going to use a more generic command name called `console` that will start a puppet debugger session. This is similar to how other tooling works like rails. 36 | 37 | The PDK console command can be invoke in two contexts. 38 | 39 | 1. Without a module 40 | 2. Within a module 41 | 42 | ### Without a module 43 | The `pdk console` command is invoked outside the module and the debugger will assume any settings specified 44 | in the puppet config file or passed in as an argument. `pdk debugger --basemodulepath=~/dev/modules --log_level=debug`. 45 | Since there is no module context, the user has access to modules only specified via the modulepaths defined in the puppet settings. A user can also specify hiera settings here as well with `--hiera_config=/dev/control_repo/hiera.yaml` in order to test lookup behavior. 46 | 47 | In this example a user wants to know if 'true' is a Boolean. They are not inside a module and only want to answer this simple question without referring to documention or other users. 48 | 49 | ``` 50 | $ pdk console -e "'true' == true" --run-once 51 | Ruby Version: 2.5.1 52 | Puppet Version: 6.7.2 53 | Puppet Debugger Version: 0.12.3 54 | Created by: NWOps 55 | Type "commands" for a list of debugger commands 56 | or "help" to show the help screen. 57 | 58 | 59 | 1:>> 'true' == true 60 | => false 61 | ``` 62 | 63 | ### Within a module 64 | The other way of invoking the console will be in the context of a module. However, because the user is inside a module, it will be scoped down to only the module's dependencies inside that module. This is important because it gives the user a clean sandbox respective of their current dependencies and hiera data. 65 | 66 | Example workflow 67 | 68 | 1. `cd apache_module` 69 | 2. `pdk console` 70 | 71 | Because we can check if we are in a module via `PDK::Util.in_module_root?` we can make some assumptions and automate a few settings for the user. The settings we can assume are: 72 | 73 | 1. basemodulepath=./spec/fixtures/modules:./modules 74 | 2. environmentpath=./spec/fixtures/modules:./modules 75 | 3. modulepath=./spec/fixtures/modules:./modules 76 | 77 | *NOTE* We are setting all these paths because want the user to work in a strict sandbox. Some of these paths like environmentpath don't make sense and are only used to poison the settings found in Puppet.settings. We may decide later to allow the user to override these defaults. 78 | 79 | *NOTE* This would mean that a `rake spec_prep` would have to be run to download any modules. 80 | 81 | *NOTE* For multi-tier lookups the user would need to supply the hiera_config file location setting. By default the hiera_config setting will use what is found in Puppet.settings. 82 | 83 | The command invoke within a module is still the same. 84 | 85 | ``` 86 | $ pdk console -e "'true' == true" --run-once 87 | Ruby Version: 2.5.1 88 | Puppet Version: 6.7.2 89 | Puppet Debugger Version: 0.12.3 90 | Created by: NWOps 91 | Type "commands" for a list of debugger commands 92 | or "help" to show the help screen. 93 | 94 | 95 | 1:>> 'true' == true 96 | => false 97 | ``` 98 | 99 | ### Additional Config Options 100 | Many users have used the debugger without understanding what options can be passed in. This is because the debugger is a puppet application. It inherits all the config options that puppet accepts. So while the puppet debugger does have some options itself most are inherited and passed through to puppet (Same as puppet apply). See the following options [listed here](https://puppet.com/docs/puppet/5.5/configuration.html). 101 | 102 | With this is mind we will want to respect these options when using the `pdk console` command. We will need to forward these options to the command so the user can override anything they desire. By default, the debugger will just read the settings via the config file using `Puppet.settings`. 103 | 104 | ### Special Arguments 105 | There are two config options that are specific to PDK that make the debugger's involvement special. Without the PDK the debugger cannot switch ruby or puppet versions easily. The PDK makes this simple using the 106 | `--puppet-version=5.5.12` or `--pe-version=2018.1` arguments. These arguments will allow the user to use the debugger to see how the language changed between versions. 107 | 108 | `pdk console --puppet-version=5.5.12 -e 'true' =~ Boolean'` 109 | 110 | `pdk console --puppet-version=6.7.0 -e "'true' =~ Boolean"` 111 | 112 | This would tell the user if the code has changed behavior in the newer version. 113 | 114 | ### Add to puppet-module meta gems 115 | The puppet-debugger should also be added to the puppet-module meta gems so it can be invoke within the module. 116 | 117 | ## Unresolved Questions 118 | 119 | What is the best way to forward arguments to a subcommand if they are only to be forwarded to puppet? 120 | 121 | Example: `--basemodulepath` is not a puppet debugger argument but is instead a puppet argument. 122 | 123 | So when a user runs `pdk console --basemodulepath=~/modules` it would essentially run `/pdk_bin_path/puppet debugger --basemodulepath=~/modules` 124 | 125 | Currently PDK will fail because --basemodulepath is not a pdk console option. This can be avoided via `skip_option_parsing` method but we still want to parse the `--puppet-version`, `--pe-version`, and `--puppet-dev` arguments. 126 | 127 | There may need to be a `ignore_bad_arguments` method or something to still have the option parsing without errors. 128 | 129 | ## Future Considerations 130 | 131 | It might also be a good idea to pass in a `--debugger-version=X.X.X` option to auto install the lastest if not present. 132 | 133 | Add the [puppet-debug](https://forge.puppet.com/nwops/debug) module to the fixtures by default. See the [following article](http://logicminds.github.io/blog/2017-04-25-break-into-your-puppet-code/) for some use cases about inserting a break statement (similar to binding.pry). 134 | 135 | 136 | ## Drawbacks 137 | The debugger is a community developed gem and will have updates every so often. These updates may break the `pdk console` command or include dependencies that conflict in some way. Testing will need to be added to the PDK to account for future updates. However, this is no different than any other gem included with the pdk. 138 | 139 | ## Alternatives 140 | 141 | The alternative is to list the debugger as a dependency in the `puppet-module-posix- and puppet-module-windows-` gems and the user can invoke 142 | via `pdk bundle exec puppet debugger --basemodulepath=./spec/fixtures/modules --hiera_config=./hiera.yaml` 143 | 144 | Although this will need to be done regardless when the debugger is invoked from within a module. 145 | 146 | But as you can see you will only type this once before making an alias. -------------------------------------------------------------------------------- /RFCs/0007-add-pdk-context.md: -------------------------------------------------------------------------------- 1 | # Add the concept of a context in PDK 2 | 3 | ## Summary 4 | 5 | Currently the PDK only operates on Puppet modules. While you can "trick" the PDK into working on Control Repos and Bolt Projects, this is far from optimal. 6 | In order to facilitate adding support for Control Repositories, and in the future unknown other things, the PDK should be aware of where, or what, it's operating on. 7 | That is, the PDK should be aware of the context that is being run in. 8 | 9 | ## Background & Assumptions 10 | 11 | This is purely a private implementation detail in the PDK and for the most part PDK users will not see any change in behaviour. 12 | In the future this feature could be used to change PDK behaviour, for example: 13 | 14 | Let's say that the PDK can create new Roles and Profiles as part of a Control Repo. This does not make sense if the PDK is running in the context of a module, therefore if a user runs `pdk new --help` the commands for `pdk new role` and `pdk new profile` would not appear in a Module context. 15 | 16 | ## Motivation & Goals 17 | 18 | This change is required to allow the PDK to grow and service more needs, far beyond that of a Puppet Module. 19 | 20 | ## Proposed Implementation Design 21 | 22 | ### Use an Object Factory pattern 23 | 24 | `PDK::Context.create(context_path)` will create a `PDK::Context` object based on it's internal logic. 25 | 26 | `PDK.context` will contain a memoized context that can, and should be used, throughout a PDK session. 27 | 28 | ### `PDK::Context` object 29 | 30 | | Method | Description | 31 | |--------| ------------| 32 | | name | The name of the context e.g. `:module` or `:control_repo` | 33 | | root_path | The root path of the context. Typically in Modules this was detected where the metadata.json was | 34 | | context_path | The path of where this context is invoked from. Typically this is the current working directory | 35 | | pdk_compatible? | Whether this context is compatible with the PDK. It assumed it is compatible by default | 36 | | parent_context | A `PDK::Context` object that this context is a member of. This caters for situations like a Module in a Control Repo. A naive implementation could simply call `PDK::Context.create(File.dirname(root_path))` | 37 | 38 | The combination of `root_path` and `context_path` allows arbitrary contexts to be built for testing purposes and will probably not be used outside of that scenario. 39 | 40 | Subclasses 41 | 42 | | Context | Description | 43 | |--------| ------------| 44 | | `PDK::Context::None` | Represents a PDK Context where it's not in anything, for example a random directory before running `pdk new module` | 45 | | `PDK::Context::Module` | Represents a context of being in a Puppet Module. | 46 | | `PDK::Context::ControlRepo` (Future) | Represents a context of being in a Puppet Directory Based Control Repository | 47 | 48 | 49 | ### Modify all CLI, Validators, Generators etc. to use a context 50 | 51 | Currently a lot of the Validators etc. duplicate code etc. by calling `PDK::Util.in_module_root?` and the like. These Classes/Modules need to be modified to take a PDK::Context parameter 52 | 53 | ## Unresolved Questions 54 | 55 | Should this be using object inheritance or perhaps a plain-old Hash is enough? Personally I prefer typed objects. 56 | 57 | ## Future Considerations 58 | 59 | Future uses would include adding new contexts or making it easier to add context detection 60 | 61 | ## Drawbacks 62 | 63 | More complex however this is offset by the ability to make testing/mocking easier 64 | 65 | ## Alternatives 66 | 67 | None. 68 | -------------------------------------------------------------------------------- /RFCs/0008-expand-new-task.md: -------------------------------------------------------------------------------- 1 | # Expanded Bolt Task Authoring 2 | 3 | ## Summary 4 | 5 | Implement various enhancements to the `pdk new task` command to make it easier to use PDK to implement Bolt Tasks that conform to current best practices. 6 | 7 | ## Background & Assumptions 8 | 9 | - PDK has supported basic Bolt Task generation since version 1.2.0 which was released in October 2017. 10 | 11 | - Since that time, Bolt has continued to grow and evolve, and new helper libraries and best practices for writing Bolt Tasks have emerged. 12 | 13 | - See also: https://puppet.com/docs/bolt/latest/writing_tasks.html 14 | 15 | ## Motivation & Goals 16 | 17 | PDK should provide up-to-date and robust tools for authoring all kinds of Puppet-related content, including Bolt Tasks. The current basic implementation of `pdk new task` can be improved in the following specific ways: 18 | 19 | - Users should be able to specify what language they plan to implement a new task in and PDK should generate a language-appropriate scaffold. 20 | 21 | - Users should be able to more easily author "cross-platform" and "shared implementation" tasks. 22 | 23 | ## Proposed Implementation Design 24 | 25 | ### Add `--language` option to `pdk new task` 26 | 27 | - Extend the current Bolt Task object templates to support language-specific templates. 28 | 29 | - Add templates for Bash, PowerShell, Ruby, and Python. 30 | 31 | - Ruby and Python templates should have scaffold implementations based on those languages Bolt Task helper libraries. (https://github.com/puppetlabs/puppetlabs-ruby_task_helper and https://github.com/puppetlabs/puppetlabs-python_task_helper) 32 | 33 | - When a new task is added with `--language` set to `ruby` or `python`, PDK should ensure that the module's `metadata.json` has a dependency on the relevant Bolt Task helper library module. (`puppetlabs-ruby_task_helper` and `puppetlabs-python_task_helper`) 34 | 35 | - Ideally, users could configure a default task language as a module-specific configuration option. 36 | 37 | - The new `--language` option will require an argument which is the name of the language-specific task template to be used. 38 | 39 | - Acceptable values for the `--language` option will initially be `bash`, `powershell`, `ruby`, and `python`. 40 | 41 | - The format of future language option values should be a concise, unique, all lowercase language identifier. 42 | 43 | - If an unrecognized language option value is specified, PDK will exit with an error. 44 | 45 | - If the `--language` option is specified without an argument, PDK will exit with an error. 46 | 47 | #### Examples: 48 | 49 | ```bash 50 | $ pdk new task my_default_task 51 | pdk (INFO): Using default task language: 'bash' 52 | pdk (INFO): Creating '/Users/jesse/sandbox/pdk/testmod/tasks/my_default_task.sh' from template. 53 | pdk (INFO): Creating '/Users/jesse/sandbox/pdk/testmod/tasks/my_default_task.json' from template. 54 | ``` 55 | 56 | ```bash 57 | $ pdk new task --language=ruby my_ruby_task 58 | pdk (INFO): Creating '/Users/jesse/sandbox/pdk/testmod/tasks/my_ruby_task.rb' from template. 59 | pdk (INFO): Creating '/Users/jesse/sandbox/pdk/testmod/tasks/my_ruby_task.json' from template. 60 | ``` 61 | 62 | ```bash 63 | $ pdk new task --language=powershell my_ps_task 64 | pdk (INFO): Creating '/Users/jesse/sandbox/pdk/testmod/tasks/my_ps_task.ps1' from template. 65 | pdk (INFO): Creating '/Users/jesse/sandbox/pdk/testmod/tasks/my_ps_task.json' from template. 66 | ``` 67 | 68 | ### Add `--private` and `--remote` flags to `pdk new task` 69 | 70 | - When specified, these flags will simply pre-populate the generated Bolt Task metadata with `"private": true` and `"remote": true` key/value pairs respectively. 71 | 72 | ### Add optional `` argument to `pdk new task` 73 | 74 | - Allow `pdk new task` to accept an unbounded, space-separated list of `implementations` to be provided after the task name. 75 | 76 | - When one or more implementation files are provided, PDK will only generate a `tasks/.json` file for the new task. The generated metadata will contain an `"implementations"` key with each given implementation file listed. Each `implementation` item will have a templated `requirements` key with a placeholder value. 77 | 78 | - If the `--language` option is provided at the same time as one or more `` arguments, PDK should exit with an appropriate error message. 79 | 80 | #### Examples 81 | 82 | The following example illustrates how a user might use PDK to author a new "cross-platform" task with Windows- and Linux-specific implementations: 83 | 84 | ```bash 85 | $ pdk new task --private --language=bash sql_linux 86 | pdk (INFO): Creating '/Users/jesse/sandbox/pdk/testmod/tasks/sql_linux.sh' from template. 87 | pdk (INFO): Creating '/Users/jesse/sandbox/pdk/testmod/tasks/sql_linux.json' from template. 88 | 89 | $ pdk new task --private --language=powershell sql_windows 90 | pdk (INFO): Creating '/Users/jesse/sandbox/pdk/testmod/tasks/sql_windows.ps1' from template. 91 | pdk (INFO): Creating '/Users/jesse/sandbox/pdk/testmod/tasks/sql_windows.json' from template. 92 | 93 | $ pdk new task sql tasks/sql_linux.sh tasks/sql_windows.ps1 94 | pdk (INFO): Creating '/Users/jesse/sandbox/pdk/testmod/tasks/sql.json' from template. 95 | pdk (INFO): Added implementation 'sql_linux.sh' to '/Users/jesse/sandbox/pdk/testmod/tasks/sql.json' 96 | pdk (INFO): Added implementation 'sql_windows.rb' to '/Users/jesse/sandbox/pdk/testmod/tasks/sql.json' 97 | ``` 98 | 99 | Given the following task implementation script already exists at `tasks/init.rb`: 100 | 101 | ```ruby 102 | #!/usr/bin/env ruby 103 | require 'json' 104 | 105 | params = JSON.parse(STDIN.read) 106 | action = params['action'] || params['_task'] 107 | if ['start', 'stop'].include?(action) 108 | `systemctl #{params['_task']} #{params['service']}` 109 | end 110 | ``` 111 | 112 | The following example illustrates how a user might use PDK to author a new set of "shared-implementation" tasks 113 | 114 | ```bash 115 | $ pdk new task start tasks/init.rb 116 | pdk (INFO): Creating '/Users/jesse/sandbox/pdk/testmod/tasks/start.json' from template. 117 | pdk (INFO): Added implementation 'init.rb' to '/Users/jesse/sandbox/pdk/testmod/tasks/start.json' 118 | 119 | $ pdk new task stop tasks/init.rb 120 | pdk (INFO): Creating '/Users/jesse/sandbox/pdk/testmod/tasks/stop.json' from template. 121 | pdk (INFO): Added implementation 'init.rb' to '/Users/jesse/sandbox/pdk/testmod/tasks/stop.json' 122 | ``` 123 | 124 | ## Unresolved Questions 125 | 126 | - Should `--private` and/or `--remote` be disallowed in combination with ``? 127 | 128 | - Is there a way to infer implementation "requirements" from the given implementation arguments? E.g. can we safely infer a `shell` requirement for an implementation file ending in `.sh`? 129 | 130 | ## Future Considerations 131 | 132 | - As Bolt continues to iterate and add functionality we may need to add additional options and flags to the task generators. 133 | 134 | - We should consider ways that PDK could help authors define their task's parameters. 135 | 136 | - We should consider ways that PDK could help authors declare their task's file/module dependencies. 137 | 138 | - We should consider ways that PDK could help authors with the "Wrapping an existing script" workflow described in the Bolt "Writing Tasks" documentation[^1] 139 | 140 | ## Drawbacks 141 | 142 | - New options and flags add additional complexity to the user-interface. Some combinations of options and flags may be invalid or have unexpected outcomes that are hard to document in the limited space afforded in command-line `--help` output. 143 | 144 | - Listing task implementations as positional arguments after the task name may not be intuitive and precludes the addition of new positional arguments to `pdk new task` for other purposes in the future. 145 | 146 | ## Alternatives 147 | 148 | - Considered adding an `--implementations=` option instead of using positional arguments to list implementations. Decided that the value of shell-provided tab completion for implementation files and reduced verbosity outweighed the value provided by a more explicit option. 149 | 150 | [^1]: https://puppet.com/docs/bolt/latest/writing_tasks.html -------------------------------------------------------------------------------- /appveyor.yml: -------------------------------------------------------------------------------- 1 | version: '{build}' 2 | max_jobs: 1 3 | image: Visual Studio 2017 4 | # We only need the latest 5 | shallow_clone: true 6 | branches: 7 | only: 8 | - main 9 | 10 | environment: 11 | # Github token to commit pushed packages to repository 12 | GITHUB_USERNAME: pdk-bot 13 | GITHUB_TOKEN: 14 | secure: IUjB8ctqO2KSF954odTdyODM+hc3Rdr1gmXzaD1sy/XODBALJ7TrQe3j5UdIqE2I #https://ci.appveyor.com/tools/encrypt 15 | 16 | install: 17 | - ps: | 18 | "Build info" 19 | ' {0,-20} {1}' -f 'SCHEDULED BUILD:', ($Env:APPVEYOR_SCHEDULED_BUILD -eq 'true') 20 | ' {0,-20} {1}' -f 'FORCED BUILD:' , ($Env:APPVEYOR_FORCED_BUILD -eq 'true') 21 | ' {0,-20} {1}' -f 'RE BUILD:' , ($Env:APPVEYOR_RE_BUILD -eq 'true') 22 | ' {0,-20} {1}' -f 'GITHUB_TOKEN:' , ($Env:GITHUB_TOKEN -ne $null) 23 | 24 | build_script: 25 | - ps: | 26 | $ErrorActionPreference = 'Continue' 27 | # Only trigger if the GTIHUB_TOKEN exists 28 | # Only trigger the project sync on a forced or scheduled build, NOT per PR. 29 | if ( ($null -ne $ENV:GITHUB_TOKEN) -and (($Env:APPVEYOR_SCHEDULED_BUILD -eq 'true') -or ($Env:APPVEYOR_FORCED_BUILD -eq 'true')) ) { 30 | .\tools\SyncProjects.ps1 31 | } 32 | -------------------------------------------------------------------------------- /tools/Gemfile: -------------------------------------------------------------------------------- 1 | # frozen_string_literal: true 2 | 3 | source "https://rubygems.org" 4 | 5 | git_source(:github) {|repo_name| "https://github.com/#{repo_name}" } 6 | 7 | gem "octokit", "~> 4.0" 8 | 9 | group :development do 10 | gem "pry-byebug" 11 | end 12 | -------------------------------------------------------------------------------- /tools/Gemfile.lock: -------------------------------------------------------------------------------- 1 | GEM 2 | remote: https://rubygems.org/ 3 | specs: 4 | addressable (2.7.0) 5 | public_suffix (>= 2.0.2, < 5.0) 6 | byebug (11.1.1) 7 | coderay (1.1.2) 8 | faraday (1.0.0) 9 | multipart-post (>= 1.2, < 3) 10 | method_source (0.9.2) 11 | multipart-post (2.1.1) 12 | octokit (4.16.0) 13 | faraday (>= 0.9) 14 | sawyer (~> 0.8.0, >= 0.5.3) 15 | pry (0.12.2) 16 | coderay (~> 1.1.0) 17 | method_source (~> 0.9.0) 18 | pry-byebug (3.8.0) 19 | byebug (~> 11.0) 20 | pry (~> 0.10) 21 | public_suffix (4.0.3) 22 | sawyer (0.8.2) 23 | addressable (>= 2.3.5) 24 | faraday (> 0.8, < 2.0) 25 | 26 | PLATFORMS 27 | ruby 28 | 29 | DEPENDENCIES 30 | octokit (~> 4.0) 31 | pry-byebug 32 | 33 | BUNDLED WITH 34 | 2.0.2 35 | -------------------------------------------------------------------------------- /tools/README.md: -------------------------------------------------------------------------------- 1 | # Tools 2 | 3 | Contains useful scripts and tools for managing PDK planning 4 | 5 | ## SyncProjects.ps1 6 | 7 | This script combines the [GitHub issues and pull requests in the PDK GitHub repository](https://github.com/puppetlabs/pdk) and the [Jira issues for the PDK Project](https://tickets.puppetlabs.com/projects/PDK/issues) into a single GitHub project. This allows people to view the state of a PDK release from a single place. 8 | 9 | ### Requirements 10 | 11 | * Windows PowerShell version 5.1+ or PowerShell Core version 6.0+ 12 | 13 | * The `JiraPS` PowerShell module. This is installed automatically if not available 14 | 15 | * A GitHub account. The script uses authenticated requests to avoid tripping the API rate limiting 16 | 17 | ### Requirements for GitHub and Jira 18 | 19 | * The script only manages GitHub projects that match the following criteria: 20 | 21 | * Located in the [pdk-planning](https://github.com/puppetlabs/pdk-planning) repository 22 | 23 | * The project is named `Release x.y.z`, where x.y.z is the version. For example `Release 1.12.0` 24 | 25 | * The project must have columns called `To do`, `In progress`, and `Done`. 26 | 27 | When creating a project, use the 'Basic Kanban' template to quickly create a compliant project. 28 | 29 | * The script only detects GitHub issues in the [GitHub PDK repository](https://github.com/puppetlabs/pdk) which have a [Milestone](https://github.com/puppetlabs/pdk/milestones) that matches the project version. For example a project with name `Release 1.12.0` expects a milestone called `1.12.0` 30 | 31 | * The script only detects Jira tickets in the [Jira PDK project](https://tickets.puppetlabs.com/projects/PDK/issues) which have a [Fix Version](https://tickets.puppetlabs.com/projects/PDK?selectedItem=com.atlassian.jira.jira-projects-plugin:release-page&status=released-unreleased&contains=PDK) that matches the project version. For example a project with name `Release 1.12.0` expects a fix version called `PDK 1.12.0` 32 | 33 | ### Running the script 34 | 35 | * Set two environment variables 36 | 37 | ``` powershell 38 | PS> $ENV:GITHUB_TOKEN = 'your github token or password' 39 | 40 | PS> $ENV:GITHUB_USERNAME = 'your github username' 41 | ``` 42 | 43 | **WARNING** If you run the script with the Verbose flag (`-Verbose`) it is possible your GitHub Token could be output to the console in cleartext 44 | 45 | * Run the script 46 | 47 | ``` powershell 48 | PS> .\tools\SyncProjects.ps1 49 | ``` 50 | 51 | **WARNING** If you run the script with the Verbose flag (`-Verbose`) it is possible your GitHub Token could be output to the console in cleartext 52 | 53 | Alternately, if you are running PowerShell on macOS or Linux, you can invoke the script like this: 54 | 55 | ```bash 56 | pwsh -File tools/SyncProjects.ps1 57 | ``` 58 | 59 | which will inherit your parent shell's environment variables (such as previously exported `GITHUB_USERNAME/TOKEN`). 60 | 61 | ## when\_project\_created.rb (Github Workflow Script) 62 | 63 | This ruby script is attached to a Github workflow (see `.github/workflows/when_project_created.yml`) which responds 64 | to a new project being created in the `pdk-planning` repo and, if the new project name matches `/\ARelease /` it 65 | will create the appropriate Milestones in the `pdk`, `pdk-templates`, and `pdk-vanagon` repos. 66 | 67 | ## when\_project\_closed.rb (Github Workflow Script) 68 | 69 | This ruby script is attached to a Github workflow (see `.github/workflows/when_project_closed.yml`) which responds 70 | to a project being closed in the `pdk-planning` repo and, if the project name matches `/\ARelease /` it 71 | will close all the related Milestones in the `pdk`, `pdk-templates`, and `pdk-vanagon` repos. 72 | 73 | ## How to test the Github workflows/actions 74 | 75 | ### How to test the standalone script 76 | 77 | The script itself can be tested simply by running it with Ruby/Bundler. When run outside of the Github action context 78 | it will not actually create any new milestones: 79 | 80 | ``` 81 | ~/pdk-planning $ cd tools 82 | ~/pdk-planning/tools $ GITHUB_TOKEN= GITHUB_EVENT_PATH=fixtures/gh_new_release_project_created.json bundle exec when_project_created.rb 83 | Would have created milestone 'January TEST' on puppetlabs/pdk 84 | Would have created milestone 'January TEST' on puppetlabs/pdk-templates 85 | Would have created milestone 'January TEST' on puppetlabs/pdk-vanagon 86 | ``` 87 | 88 | There are additional fixture event payloads you can test in the `tools/fixtures` directory. Specify which payload 89 | you want to test using the `GITHUB_EVENT_PATH` environment variable as shown above. 90 | 91 | ### How to test the Github workflow 92 | 93 | You will need Docker and the [`act`](https://github.com/nektos/act#installation) tool to test the full workflow. 94 | 95 | **Warning: The Docker image that is pulled to make this workflow run locally is very large (~18GB at the time of this writing).** 96 | 97 | Run the following from the _root_ of this repo: 98 | 99 | ``` 100 | ~/pdk-planning $ act project -P ubuntu-latest=nektos/act-environments-ubuntu:18.04 -s PDKBOT_GITHUB_TOKEN=$GITHUB_TOKEN -e tools/fixtures/gh_new_release_project_created.json --env-file tools/env-for-act 101 | ``` 102 | 103 | Note that you still need a valid `GITHUB_TOKEN` set. 104 | 105 | You should see something like the following: 106 | 107 | ``` 108 | [New Project Created/Run when_project_created.rb] 🚀 Start image=nektos/act-environments-ubuntu:18.04 109 | [New Project Created/Run when_project_created.rb] 🐳 docker run image=nektos/act-environments-ubuntu:18.04 entrypoint=["/usr/bin/tail" "-f" "/dev/null"] cmd=[] 110 | [New Project Created/Run when_project_created.rb] 🐳 docker cp src=/Users/jesse/src/sdk/pdk-planning/. dst=/github/workspace 111 | [New Project Created/Run when_project_created.rb] ⭐ Run actions/checkout@v2 112 | [New Project Created/Run when_project_created.rb] ✅ Success - actions/checkout@v2 113 | [New Project Created/Run when_project_created.rb] ⭐ Run ruby/setup-ruby@v1.20.1 114 | [New Project Created/Run when_project_created.rb] ☁ git clone 'https://github.com/ruby/setup-ruby' # ref=v1.20.1 115 | [New Project Created/Run when_project_created.rb] 🐳 docker cp src=/Users/jesse/.cache/act/ruby-setup-ruby@v1.20.1 dst=/actions/ 116 | | Using 2.6.3 as input from file .ruby-version 117 | | https://github.com/ruby/ruby-builder/releases/download/builds-no-warn/ruby-2.6.3-ubuntu-18.04.tar.gz 118 | [New Project Created/Run when_project_created.rb] 💬 ::debug::Downloading https://github.com/ruby/ruby-builder/releases/download/builds-no-warn/ruby-2.6.3-ubuntu-18.04.tar.gz 119 | [New Project Created/Run when_project_created.rb] 💬 ::debug::Downloading /home/actions/temp/9a747bcf-dcae-4cb0-a0f9-eb8e2693017f 120 | [New Project Created/Run when_project_created.rb] 💬 ::debug::download complete 121 | | [command]/bin/tar xz -C /github/home/.rubies -f /home/actions/temp/9a747bcf-dcae-4cb0-a0f9-eb8e2693017f 122 | [New Project Created/Run when_project_created.rb] ⚙ ::add-path:: /github/home/.rubies/ruby-2.6.3/bin 123 | [New Project Created/Run when_project_created.rb] ⚙ ::set-output:: ruby-prefix=/github/home/.rubies/ruby-2.6.3 124 | [New Project Created/Run when_project_created.rb] ✅ Success - ruby/setup-ruby@v1.20.1 125 | [New Project Created/Run when_project_created.rb] ⭐ Run when_project_created.rb 126 | | Fetching bundler-2.0.2.gem 127 | | Successfully installed bundler-2.0.2 128 | | 1 gem installed 129 | | Don't run Bundler as root. Bundler can ask for sudo if it is needed, and 130 | | installing your bundle as root will break this application for all non-root 131 | | users on this machine. 132 | | Using public_suffix 4.0.3 133 | | Using addressable 2.7.0 134 | | Using bundler 2.0.2 135 | | Using multipart-post 2.1.1 136 | | Using faraday 1.0.0 137 | | Using sawyer 0.8.2 138 | | Using octokit 4.16.0 139 | | Bundle complete! 2 Gemfile dependencies, 7 gems now installed. 140 | | Gems in the group development were not installed. 141 | | Bundled gems are installed into `./.bundle` 142 | | Would have created milestone 'January TEST' on puppetlabs/pdk 143 | | Would have created milestone 'January TEST' on puppetlabs/pdk-templates 144 | | Would have created milestone 'January TEST' on puppetlabs/pdk-vanagon 145 | [New Project Created/Run when_project_created.rb] ✅ Success - when_project_created.rb 146 | ``` 147 | -------------------------------------------------------------------------------- /tools/SyncProjects.ps1: -------------------------------------------------------------------------------- 1 | [CmdletBinding()] 2 | param() 3 | 4 | # Needs PowerShell 5.1+ 5 | $ErrorActionPreference = 'Stop' 6 | 7 | try { 8 | # Need the JiraPS module to query Jira 9 | Import-Module JiraPS -ErrorAction Stop 10 | } 11 | catch { 12 | Install-Module JiraPS -ErrorAction Stop -RequiredVersion 2.11.1 -Force 13 | Import-Module JiraPS -ErrorAction Stop 14 | } 15 | 16 | $script:GithubToken = $ENV:GITHUB_TOKEN 17 | $script:GithubUsername = $ENV:GITHUB_USERNAME 18 | $script:JiraServer = 'https://tickets.puppetlabs.com' 19 | 20 | if ($null -eq $script:GithubToken) { Throw "This script requires the GITHUB_TOKEN environment variable to be set"; Exit 1 } 21 | if ($null -eq $script:GithubUsername) { Throw "This script requires the GITHUB_USERNAME environment variable to be set"; Exit 1 } 22 | 23 | Function Invoke-GithubAPI { 24 | [CmdletBinding()] 25 | 26 | Param( 27 | [Parameter(Mandatory = $True, ParameterSetName = 'RelativeURI')] 28 | [String]$RelativeUri, 29 | 30 | [Parameter(Mandatory = $True, ParameterSetName = 'AbsoluteURI')] 31 | [String]$AbsoluteUri, 32 | 33 | [Parameter(Mandatory = $False)] 34 | [switch]$Raw, 35 | 36 | [String]$Method = 'GET', 37 | 38 | [Object]$Body = $null 39 | ) 40 | 41 | if ($PsCmdlet.ParameterSetName -eq 'RelativeURI') { 42 | $uri = "https://api.github.com" + $RelativeUri 43 | } 44 | else { 45 | $uri = $AbsoluteUri 46 | } 47 | 48 | $result = "" 49 | 50 | $oldPreference = $ProgressPreference 51 | 52 | $auth = 'Basic ' + [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes($script:GithubUsername + ':' + $script:GithubToken)); 53 | 54 | $ProgressPreference = 'SilentlyContinue' 55 | $Headers = @{ 56 | 'Accept' = 'application/vnd.github.inertia-preview+json' # Needed for project API 57 | 'Authorization' = $auth; 58 | } 59 | $splat = @{ 60 | 'Uri' = $uri 61 | 'UseBasicParsing' = $True 62 | 'Headers' = $Headers 63 | 'Method' = $Method 64 | } 65 | if ($null -ne $Body) { $splat['Body'] = ConvertTo-Json $Body -Compress } 66 | try { 67 | $result = Invoke-WebRequest @splat -ErrorAction 'Stop' 68 | } catch { 69 | Write-Verbose "Invoke-WebRequest arguments were $($splat | ConvertTo-JSON -Depth 10)" 70 | Throw $_ 71 | } 72 | $ProgressPreference = $oldPreference 73 | 74 | if ($Raw) { 75 | Write-Output $result 76 | } 77 | else { 78 | Write-Output $result.Content | ConvertFrom-JSON 79 | } 80 | } 81 | 82 | Function Invoke-GithubAPIWithPaging($RelativeUri) { 83 | $response = Invoke-GithubAPI -RelativeUri $RelativeUri -Raw 84 | $result = $response.Content | ConvertFrom-Json 85 | if (!($result -is [Array])) { $result = @($result) } 86 | $nextLink = $response.RelationLink.next 87 | do { 88 | if ($null -ne $nextLink) { 89 | $response = Invoke-GithubAPI -AbsoluteUri $nextLink -Raw 90 | $result = $result + ($response.Content | ConvertFrom-Json) 91 | $nextLink = $response.RelationLink.next 92 | } 93 | } 94 | while ($null -ne $nextLink) 95 | 96 | Write-Output $result 97 | } 98 | 99 | Function Get-PDKProjects { 100 | Invoke-GithubAPIWithPaging -RelativeUri '/repos/puppetlabs/pdk-planning/projects?state=open' 101 | } 102 | 103 | Function Convert-GHIssueToNote { 104 | param( 105 | [Parameter(Mandatory = $true, ValueFromPipeline = $true)] 106 | [Object]$Issue, 107 | 108 | [String]$Prefix = "" 109 | ) 110 | 111 | Begin { 112 | if ($Prefix -ne "") { $Prefix = $Prefix + " " } 113 | } 114 | 115 | Process { 116 | # We only add the title, because github fills in the description as a rich-link 117 | $hash = @{ 118 | 'cardId' = -1 119 | 'note' = @" 120 | [$($Prefix)GitHub Issue $($issue.number)]($($issue.html_url)) 121 | 122 | $($issue.title) 123 | "@ 124 | 'expectedColumn' = 'To do' 125 | } 126 | 127 | # Consider all open pull requests as "in progress" 128 | if ($Issue.pull_request -ne $null) { $hash['expectedColumn'] = 'In progress' } 129 | if ($Issue.state -eq 'closed') { $hash['expectedColumn'] = 'Done' } 130 | 131 | Write-Output ([PSCustomObject]$hash) 132 | } 133 | } 134 | 135 | Function Convert-GHCardToNote { 136 | param( 137 | [Parameter(Mandatory = $true, ValueFromPipeline = $true)] 138 | [Object]$Card, 139 | 140 | [String]$ColumnName 141 | ) 142 | 143 | Process { 144 | # Write-Host ($Card | ConvertTo-JSON) -ForegroundColor Red 145 | $hash = @{ 146 | 'foundMatchingIssue' = $false 147 | 'note' = $Card.note 148 | 'id' = $Card.id 149 | 'currentColumn' = $ColumnName 150 | 'expectedColumn' = '????' 151 | } 152 | Write-Output ([PSCustomObject]$hash) 153 | } 154 | } 155 | 156 | Function Get-JiraIssues($fixVersion) { 157 | Set-JiraConfigServer $script:JiraServer 158 | 159 | $Query = "project = `"Puppet Development Kit`" and fixVersion = `"${fixVersion}`"" 160 | 161 | try { 162 | Get-JiraIssue -Query $Query -Fields "-comment" -ErrorAction 'Stop' | ForEach-Object { 163 | $Issue = $_ 164 | 165 | $hash = @{ 166 | 'cardId' = -1 167 | 'note' = @" 168 | [Jira Issue $($Issue.key)]($($issue.HttpUrl)) 169 | 170 | $($issue.Summary) 171 | "@ 172 | 'expectedColumn' = 'To do' 173 | } 174 | # There's probably more than this for in progress... 175 | if ($Issue.Status -eq 'In progress') { $hash['expectedColumn'] = 'In progress'} 176 | if ($Issue.Status -eq 'Ready for merge') { $hash['expectedColumn'] = 'In progress'} 177 | if ($Issue.Status -eq 'Ready for test') { $hash['expectedColumn'] = 'In progress'} 178 | if ($Issue.Status -eq 'Ready for ci') { $hash['expectedColumn'] = 'In progress'} 179 | if ($Issue.Status -eq 'Ready for review') { $hash['expectedColumn'] = 'In progress'} 180 | if ($Issue.Status -eq 'Closed') { $hash['expectedColumn'] = 'Done'} 181 | if ($Issue.Status -eq 'Resolved') { $hash['expectedColumn'] = 'Done'} 182 | if ($Issue.Status -eq 'Completed') { $hash['expectedColumn'] = 'Done'} 183 | Write-Output ([PSCustomObject]$hash) 184 | } 185 | } 186 | catch { 187 | Write-Warning "Error querying Jira $_" 188 | } 189 | } 190 | 191 | Function Resize-String($Value, $MaxLength) { 192 | if ($Value.Length -gt $MaxLength) { 193 | Write-Output ($Value.SubString(0, $MaxLength) + "...") 194 | } 195 | else { 196 | Write-Output $Value 197 | } 198 | } 199 | 200 | Function Invoke-ParsePDKProject($project) { 201 | if ($project.name -notmatch '^Release ([\w\. ]+)$') { 202 | Write-Verbose "Project $($project.name) is not a release" 203 | return 204 | } 205 | Write-Verbose "Parsing Project $($project.name) ..." 206 | $ProjectVersion = $Matches[1] 207 | $MilestoneName = $ProjectVersion 208 | $JiraFixVersion = "PDK $ProjectVersion" 209 | 210 | # Get all columns 211 | $ProjectColumns = Invoke-GithubAPIWithPaging -RelativeUri "/projects/$($project.id)/columns" 212 | 213 | # Get all cards, including archived 214 | Write-Verbose "Getting existing cards ..." 215 | $ProjectCards = @() 216 | $ColumnIds = @{} 217 | $ProjectColumns | ForEach-Object { 218 | $ColumnName = $_.name 219 | $ColumnIds[$_.name] = $_.id 220 | Invoke-GithubAPIWithPaging -RelativeUri "/projects/columns/$($_.Id)/cards?archived_state=all" | ForEach-Object { $ProjectCards += (Convert-GHCardToNote -Card $_ -ColumnName $ColumnName) } 221 | } 222 | 223 | # Sanity check for columns we need 224 | if ($null -eq $ColumnIds['To do']) { Write-Warning "Project $($Project.number) is missing expected column 'To do'"; Return} 225 | if ($null -eq $ColumnIds['In progress']) { Write-Warning "Project $($Project.number) is missing expected column 'In progress'"; Return} 226 | if ($null -eq $ColumnIds['Done']) { Write-Warning "Project $($Project.number) is missing expected column 'Done'"; Return} 227 | 228 | # Get all of the Github issues for the PDK version milestone 229 | $GHIssues = @() 230 | 231 | # puppetlabs/pdk 232 | Write-Verbose "Getting current issues in the PDK repo for milestone ${MilestoneName} ..." 233 | $GHIssues += ((Invoke-GithubAPIWithPaging -RelativeUri "/search/issues?q=repo:puppetlabs/pdk+milestone:`"${MilestoneName}`"").items | Convert-GHIssueToNote -Prefix 'PDK') 234 | 235 | # puppetlabs/pdk-templates 236 | Write-Verbose "Getting current issues in the PDK Templates repo for milestone ${MilestoneName} ..." 237 | $GHIssues += ((Invoke-GithubAPIWithPaging -RelativeUri "/search/issues?q=repo:puppetlabs/pdk-templates+milestone:`"${MilestoneName}`"").items | Convert-GHIssueToNote -Prefix 'PDK Template') 238 | 239 | # puppetlabs/pdk-vanagon 240 | Write-Verbose "Getting current issues in the PDK Vanagon repo for milestone ${MilestoneName} ..." 241 | $GHIssues += ((Invoke-GithubAPIWithPaging -RelativeUri "/search/issues?q=repo:puppetlabs/pdk-vanagon+milestone:`"${MilestoneName}`"").items | Convert-GHIssueToNote -Prefix 'PDK Packaging') 242 | 243 | # Get all of the Jira tickets 244 | Write-Verbose "Getting current tickets in the PDK project for fix version ${JiraFixVersion} ..." 245 | $JiraTickets = Get-JiraIssues -fixVersion $JiraFixVersion 246 | 247 | Write-Verbose "Matching Issues to Notes ..." 248 | # Match things up 249 | $ProjectCards | ForEach-Object -Process { 250 | $thisCard = $_ 251 | # Try and find a matching GHIssue 252 | $GHIssues | Where-Object { $_.cardId -eq -1} | ForEach-Object -Process { 253 | if ($_.note -eq $thisCard.note) { 254 | Write-Verbose "Found a match for card $($thisCard.Id)" 255 | $_.cardId = $thisCard.Id 256 | $thisCard.expectedColumn = $_.expectedColumn 257 | $thisCard.foundMatchingIssue = $true 258 | } 259 | } 260 | if (!$thisCard.foundMatchingIssue) { 261 | $JiraTickets | Where-Object { $_.cardId -eq -1} | ForEach-Object -Process { 262 | if ($_.note -eq $thisCard.note) { 263 | Write-Verbose "Found a match for card $($thisCard.Id)" 264 | $_.cardId = $thisCard.Id 265 | $thisCard.expectedColumn = $_.expectedColumn 266 | $thisCard.foundMatchingIssue = $true 267 | } 268 | } 269 | } 270 | } 271 | 272 | # Create new cards from GH Issues 273 | $GHIssues | Where-Object { $_.cardId -eq -1 } | ForEach-Object { 274 | $Issue = $_ 275 | $body = @{ 'note' = $Issue.note } 276 | Write-Host "Adding card for `"$(Resize-String -Value $Issue.note -MaxLength 50)`"" 277 | Invoke-GithubAPI -RelativeUri "/projects/columns/$($ColumnIds[$Issue.expectedColumn])/cards" -Method 'POST' -Body $body | Out-Null 278 | } 279 | 280 | # Create new cards from Jira Issues 281 | $JiraTickets | Where-Object { $_.cardId -eq -1 } | ForEach-Object { 282 | $Issue = $_ 283 | $body = @{ 'note' = $Issue.note } 284 | Write-Host "Adding card for `"$(Resize-String -Value $Issue.note -MaxLength 50)`"" 285 | Invoke-GithubAPI -RelativeUri "/projects/columns/$($ColumnIds[$Issue.expectedColumn])/cards" -Method 'POST' -Body $body | Out-Null 286 | } 287 | 288 | # Remove cards which can not be matched 289 | $ProjectCards | Where-Object { -not $_.foundMatchingIssue } | ForEach-Object -Process { 290 | Write-Host "Removing card `"$(Resize-String -Value $_.note -MaxLength 50)`"" 291 | Invoke-GithubAPI -RelativeUri "/projects/columns/cards/$($_.id)" -Method 'DELETE' | Out-Null 292 | } 293 | # Move cards which are in the wrong column 294 | $ProjectCards | Where-Object { $_.foundMatchingIssue -and ($_.expectedColumn -ne $_.currentColumn) } | ForEach-Object -Process { 295 | $body = @{ 'position' = 'top'; 'column_id' = $ColumnIds[$_.expectedColumn] } 296 | Write-Host "Moving card `"$(Resize-String -Value $_.note -MaxLength 50)`" to `"$($_.expectedColumn)`"" 297 | Invoke-GithubAPI -RelativeUri "/projects/columns/cards/$($_.id)/moves" -Method 'POST' -Body $Body | Out-Null 298 | } 299 | } 300 | 301 | Get-PDKProjects | ForEach-Object { Invoke-ParsePDKProject $_ } 302 | -------------------------------------------------------------------------------- /tools/env-for-act: -------------------------------------------------------------------------------- 1 | ImageOS=ubuntu18 2 | -------------------------------------------------------------------------------- /tools/fixtures/gh_existing_release_project_created.json: -------------------------------------------------------------------------------- 1 | { 2 | "action": "created", 3 | "project": { 4 | "html_url": "https://github.com/puppetlabs/pdk-planning/projects/13", 5 | "name": "Release February 2020" 6 | } 7 | } 8 | -------------------------------------------------------------------------------- /tools/fixtures/gh_new_release_project_created.json: -------------------------------------------------------------------------------- 1 | { 2 | "action": "created", 3 | "project": { 4 | "html_url": "https://github.com/puppetlabs/pdk-planning/projects/1000000", 5 | "name": "Release January TEST" 6 | } 7 | } 8 | -------------------------------------------------------------------------------- /tools/fixtures/gh_nonrelease_project_created.json: -------------------------------------------------------------------------------- 1 | { 2 | "action": "created", 3 | "project": { 4 | "html_url": "https://github.com/puppetlabs/pdk-planning/projects/120391", 5 | "name": "Not a Release Project" 6 | } 7 | } 8 | -------------------------------------------------------------------------------- /tools/fixtures/gh_release_project_closed.json: -------------------------------------------------------------------------------- 1 | { 2 | "action": "closed", 3 | "project": { 4 | "html_url": "https://github.com/puppetlabs/pdk-planning/projects/1000000", 5 | "name": "Release January TEST" 6 | } 7 | } 8 | -------------------------------------------------------------------------------- /tools/when_project_closed.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | # frozen_string_literal: true 3 | 4 | require 'json' 5 | require 'octokit' 6 | 7 | module WhenProjectClosed 8 | def self.debug? 9 | # TODO: it would be nice to have a better way to check this... 10 | ENV['GITHUB_ACTIONS'].nil? || ENV['GITHUB_ACTOR'] == 'nektos/act' 11 | end 12 | 13 | def self.gh_client 14 | @gh_client ||= Octokit::Client.new(:access_token => ENV['GITHUB_TOKEN']) 15 | end 16 | 17 | def self.event 18 | @event ||= JSON.parse(File.read(ENV['GITHUB_EVENT_PATH'])) 19 | end 20 | 21 | def self.milestone_name_from_event(event) 22 | abort("incomplete/wrong event payload received, missing 'project.name':\n\n #{event}") unless event.dig('project', 'name') 23 | 24 | event['project']['name'].gsub(/\ARelease /, '') 25 | end 26 | 27 | def self.get_open_milestone_by_name(repo, milestone_name) 28 | gh_client.list_milestones(repo, state: 'open').find { |ms| ms[:title] == milestone_name } 29 | end 30 | 31 | def self.close_milestone!(repo, milestone) 32 | if debug? 33 | puts "Would have closed milestone '#{milestone[:title]}' (##{milestone[:id]}) on '#{repo}'" 34 | return 35 | end 36 | 37 | gh_client.update_milestone(repo, milestone[:id], { :state => 'closed' }) 38 | 39 | if gh_client.last_response.status == 200 40 | puts "Closed milestone '#{milestone[:title]}' (##{milestone[:id]}) on '#{repo}'" 41 | return 42 | end 43 | 44 | error = gh_client.last_response.body 45 | abort "Failed to close milestone '#{milestone[:name]}' (##{milestone[:id]}) on '#{repo}':\n\n#{error}" 46 | end 47 | end 48 | 49 | abort("GITHUB_TOKEN must be set") unless ENV['GITHUB_TOKEN'] 50 | abort("GITHUB_EVENT_PATH must be set") unless ENV['GITHUB_EVENT_PATH'] 51 | 52 | @event = WhenProjectClosed.event 53 | 54 | if @event.dig('project', 'name') !~ /\ARelease / 55 | puts "Closed project does not match /\\ARelease /, skipping" 56 | exit(0) 57 | end 58 | 59 | @milestone_name = WhenProjectClosed.milestone_name_from_event(@event) 60 | 61 | %w[ 62 | puppetlabs/pdk 63 | puppetlabs/pdk-templates 64 | puppetlabs/pdk-vanagon 65 | ].each do |repo| 66 | milestone = WhenProjectClosed.get_open_milestone_by_name(repo, @milestone_name) 67 | 68 | if !milestone 69 | puts "Milestone '#{@milestone_name}' does not exist or is already closed on '#{repo}', skipping" 70 | next 71 | end 72 | 73 | WhenProjectClosed.close_milestone!(repo, milestone) 74 | end 75 | -------------------------------------------------------------------------------- /tools/when_project_created.rb: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env ruby 2 | # frozen_string_literal: true 3 | 4 | require 'json' 5 | require 'octokit' 6 | 7 | module WhenProjectCreated 8 | def self.debug? 9 | # TODO: it would be nice to have a better way to check this... 10 | ENV['GITHUB_ACTIONS'].nil? || ENV['GITHUB_ACTOR'] == 'nektos/act' 11 | end 12 | 13 | def self.gh_client 14 | @gh_client ||= Octokit::Client.new(:access_token => ENV['GITHUB_TOKEN']) 15 | end 16 | 17 | def self.event 18 | @event ||= JSON.parse(File.read(ENV['GITHUB_EVENT_PATH'])) 19 | end 20 | 21 | def self.milestone_from_event(event) 22 | abort("incomplete/wrong event payload received, missing 'project.name':\n\n #{event}") unless event.dig('project', 'name') 23 | abort("incomplete/wrong event payload received, missing 'project.html_url':\n\n #{event}") unless event.dig('project', 'html_url') 24 | 25 | { 26 | name: event['project']['name'].gsub(/\ARelease /, ''), 27 | link: event['project']['html_url'], 28 | } 29 | end 30 | 31 | def self.repo_has_milestone?(repo, milestone) 32 | gh_client.list_milestones(repo, state: 'all').any? { |ms| ms[:title] == milestone[:name] } 33 | end 34 | 35 | def self.create_milestone!(repo, milestone) 36 | if repo_has_milestone?(repo, milestone) 37 | puts "Milestone '#{milestone[:name]}' already exists on #{repo}, skipping" 38 | return 39 | end 40 | 41 | if debug? 42 | puts "Would have created milestone '#{milestone[:name]}' on #{repo}" 43 | return 44 | end 45 | 46 | gh_client.create_milestone(repo, milestone[:name], { 47 | description: "See #{milestone[:link]} for complete release planning", 48 | }) 49 | 50 | if gh_client.last_response.status == 201 51 | puts "Created milestone '#{milestone[:name]}' on #{repo}" 52 | return 53 | end 54 | 55 | error = gh_client.last_response.body 56 | abort "Failed to create milestone '#{milestone[:name]}' on #{repo}:\n\n#{error}" 57 | end 58 | end 59 | 60 | abort("GITHUB_TOKEN must be set") unless ENV['GITHUB_TOKEN'] 61 | abort("GITHUB_EVENT_PATH must be set") unless ENV['GITHUB_EVENT_PATH'] 62 | 63 | @event = WhenProjectCreated.event 64 | 65 | if @event.dig('project', 'name') !~ /\ARelease / 66 | puts "New project does not match /\\ARelease /, skipping" 67 | exit(0) 68 | end 69 | 70 | @milestone = WhenProjectCreated.milestone_from_event(@event) 71 | 72 | %w[ 73 | puppetlabs/pdk 74 | puppetlabs/pdk-templates 75 | puppetlabs/pdk-vanagon 76 | ].each do |repo| 77 | WhenProjectCreated.create_milestone!(repo, @milestone) 78 | end 79 | --------------------------------------------------------------------------------