├── .gitignore ├── managed_modules.yml ├── Gemfile ├── Rakefile ├── pdksync.yml ├── .travis.yml ├── Gemfile.lock └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | tar.gz 2 | -------------------------------------------------------------------------------- /managed_modules.yml: -------------------------------------------------------------------------------- 1 | --- 2 | - puppet-format 3 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | source "https://rubygems.org" 2 | gem 'pry' 3 | 4 | gem 'pdksync', github: 'puppetlabs/pdksync' 5 | 6 | -------------------------------------------------------------------------------- /Rakefile: -------------------------------------------------------------------------------- 1 | require 'pdksync/rake_tasks' 2 | 3 | desc 'validate managed modules' 4 | task :validate do 5 | require 'yaml' 6 | mods = YAML.load_file('managed_modules.yml').count 7 | mods > 0 ? exit(0) : exit(1) 8 | end 9 | -------------------------------------------------------------------------------- /pdksync.yml: -------------------------------------------------------------------------------- 1 | --- 2 | namespace: 'voxpupuli' 3 | pdk_templates_prefix: 'vp-' 4 | pdk_templates_ref: 1.17.0 5 | pdk_templates_url: 'https://github.com/voxpupuli/pdk-templates.git' 6 | module_is_authoritive: false 7 | git_base_uri: 'git@github.com' 8 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | dist: bionic 2 | language: ruby 3 | cache: bundler 4 | before_install: 5 | - gem install bundler 6 | - bundle -v 7 | - rm -f Gemfile.lock 8 | - gem update --system 9 | - gem --version 10 | - bundle -v 11 | script: 12 | - bundle exec $CHECK 13 | rvm: 14 | - 2.6.5 15 | stages: 16 | - name: test 17 | - name: deploy 18 | if: branch = master 19 | matrix: 20 | fast_finish: true 21 | include: 22 | - env: CHECK="rake validate" 23 | stage: test 24 | - env: GITHUB_TOKEN=sdfasdfa CHECK='rake pdksync:show_config' 25 | stage: test 26 | - env: CHECK='rake pdksync' 27 | if: branch = master 28 | stage: deploy 29 | branches: 30 | only: 31 | - master 32 | - "/^v\\d/" 33 | 34 | notifications: 35 | email: false 36 | irc: 37 | on_success: always 38 | on_failure: always 39 | channels: 40 | - chat.freenode.org#voxpupuli-notifications 41 | webhooks: https://voxpupu.li/incoming/travis 42 | -------------------------------------------------------------------------------- /Gemfile.lock: -------------------------------------------------------------------------------- 1 | GIT 2 | remote: https://github.com/puppetlabs/pdksync.git 3 | revision: b0f66cf78ef07ebe5ceb5f4f2890618969904c98 4 | specs: 5 | pdksync (0.5.0) 6 | colorize 7 | git (~> 1.3) 8 | gitlab 9 | jenkins_api_client 10 | octokit 11 | pdk (>= 1.14.1) 12 | rake 13 | 14 | GEM 15 | remote: https://rubygems.org/ 16 | specs: 17 | addressable (2.7.0) 18 | public_suffix (>= 2.0.2, < 5.0) 19 | childprocess (0.7.1) 20 | ffi (~> 1.0, >= 1.0.11) 21 | coderay (1.1.2) 22 | colorize (0.8.1) 23 | concurrent-ruby (1.1.6) 24 | cri (2.15.10) 25 | deep_merge (1.2.1) 26 | diff-lcs (1.3) 27 | equatable (0.6.1) 28 | facter (2.5.7) 29 | faraday (1.0.1) 30 | multipart-post (>= 1.2, < 3) 31 | fast_gettext (1.1.2) 32 | ffi (1.9.25) 33 | gettext (3.2.9) 34 | locale (>= 2.0.5) 35 | text (>= 1.3.0) 36 | gettext-setup (0.34) 37 | fast_gettext (~> 1.1.0) 38 | gettext (>= 3.0.2, < 3.3.0) 39 | locale 40 | git (1.6.0) 41 | rchardet (~> 1.8) 42 | gitlab (4.14.1) 43 | httparty (~> 0.14, >= 0.14.0) 44 | terminal-table (~> 1.5, >= 1.5.1) 45 | hitimes (1.3.0) 46 | httparty (0.18.0) 47 | mime-types (~> 3.0) 48 | multi_xml (>= 0.5.2) 49 | httpclient (2.8.3) 50 | jenkins_api_client (1.5.3) 51 | json (>= 1.0) 52 | mixlib-shellout (>= 1.1.0) 53 | nokogiri (~> 1.6) 54 | socksify (>= 1.7.0) 55 | terminal-table (>= 1.4.0) 56 | thor (>= 0.16.0) 57 | json (2.3.0) 58 | json-schema (2.8.0) 59 | addressable (>= 2.4) 60 | json_pure (2.1.0) 61 | locale (2.1.3) 62 | method_source (1.0.0) 63 | mime-types (3.3.1) 64 | mime-types-data (~> 3.2015) 65 | mime-types-data (3.2019.1009) 66 | mini_portile2 (2.8.0) 67 | minitar (0.9) 68 | mixlib-shellout (3.0.9) 69 | multi_xml (0.6.0) 70 | multipart-post (2.1.1) 71 | necromancer (0.5.1) 72 | nokogiri (1.13.3) 73 | mini_portile2 (~> 2.8.0) 74 | racc (~> 1.4) 75 | octokit (4.18.0) 76 | faraday (>= 0.9) 77 | sawyer (~> 0.8.0, >= 0.5.3) 78 | pastel (0.7.3) 79 | equatable (~> 0.6) 80 | tty-color (~> 0.5) 81 | pathspec (0.2.1) 82 | pdk (1.17.0) 83 | bundler (>= 1.15.0, < 3.0.0) 84 | childprocess (~> 0.7.1) 85 | concurrent-ruby (~> 1.1.5) 86 | cri (~> 2.10) 87 | deep_merge (~> 1.1) 88 | diff-lcs (= 1.3) 89 | facter (~> 2.5.1) 90 | ffi (~> 1.9.0) 91 | gettext-setup (~> 0.24) 92 | hitimes (= 1.3.0) 93 | httpclient (~> 2.8.3) 94 | json-schema (= 2.8.0) 95 | json_pure (~> 2.1.0) 96 | minitar (~> 0.6) 97 | pathspec (~> 0.2.1) 98 | tty-prompt (~> 0.13) 99 | tty-spinner (~> 0.5) 100 | tty-which (~> 0.3) 101 | pry (0.13.1) 102 | coderay (~> 1.1) 103 | method_source (~> 1.0) 104 | public_suffix (4.0.4) 105 | racc (1.6.0) 106 | rake (13.0.1) 107 | rchardet (1.8.0) 108 | sawyer (0.8.2) 109 | addressable (>= 2.3.5) 110 | faraday (> 0.8, < 2.0) 111 | socksify (1.7.1) 112 | terminal-table (1.8.0) 113 | unicode-display_width (~> 1.1, >= 1.1.1) 114 | text (1.3.1) 115 | thor (1.0.1) 116 | tty-color (0.5.1) 117 | tty-cursor (0.7.1) 118 | tty-prompt (0.21.0) 119 | necromancer (~> 0.5.0) 120 | pastel (~> 0.7.0) 121 | tty-reader (~> 0.7.0) 122 | tty-reader (0.7.0) 123 | tty-cursor (~> 0.7) 124 | tty-screen (~> 0.7) 125 | wisper (~> 2.0.0) 126 | tty-screen (0.7.1) 127 | tty-spinner (0.9.3) 128 | tty-cursor (~> 0.7) 129 | tty-which (0.4.2) 130 | unicode-display_width (1.7.0) 131 | wisper (2.0.1) 132 | 133 | PLATFORMS 134 | ruby 135 | 136 | DEPENDENCIES 137 | pdksync! 138 | pry 139 | 140 | BUNDLED WITH 141 | 2.1.4 142 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Pdksync 2 | 3 | Table of Contents 4 | ----------------- 5 | 6 | 1. [Overview](#overview) 7 | 2. [Requirements](#requirements) 8 | 3. [Usage](#usage) 9 | 4. [How it works](#how-it-works) 10 | 5. [Configuration](#configuration) 11 | 6. [Workflow](#workflow) 12 | 7. [Migrating from modulesync to pdksync](#migrating-from-modulesync-to-pdksync) 13 | 8. [Contributing](#contributing) 14 | 15 | ### Overview 16 | -------- 17 | 18 | Pdksync is an efficient way to run a `pdk update` command against the various Puppet module repositories that you manage — keeping them up-to-date with the changes made to PDK. It is a solution for converted modules that no longer run with modulesync. 19 | 20 | Pdksync by default expects that your Puppet module repositories live on GitHub and will behave accordingly. It also supports GitLab as an alternative Git hosting platform. 21 | 22 | ### Requirements 23 | -------- 24 | * Ruby >= 2.4.4 25 | * Bundler ~> 1.15 26 | 27 | ### Usage 28 | ---------- 29 | 30 | > Note: This tool creates a 'live' pull (merge) request against the master branch of the module it is running against — defined in `managed_modules.yml`. Before running this tool, ensure this file reflects the modules you wish it to run against. Additionally make sure that the Pdksync configuration file `$HOME/.pdksync.yml` sets the correct namespace, Git platform and Git base URI for your modules. See section [Configuration](#configuration) for details. 31 | 32 | 1. To use pdksync, clone the GitHub repo or install it as a gem. Set up the environment by exporting a GitHub token: 33 | 34 | ```shell 35 | export GITHUB_TOKEN= 36 | ``` 37 | 38 | If you use GitLab instead of GitHub export your GitLab access token: 39 | 40 | ```shell 41 | export GITLAB_TOKEN= 42 | ``` 43 | 2. Before the script will run, you need to install the gems: 44 | ```shell 45 | bundle install --path .bundle/gems/ 46 | ``` 47 | 3. Once this is complete, call the built-in rake task to run the module: 48 | ```shell 49 | bundle exec rake pdksync 50 | ``` 51 | 52 | ### How it works 53 | ------------ 54 | 55 | Pdksync is a gem that works to clone, update, and push module repositories. Create a new git repository to store your working config. You need the following files in there: 56 | 57 | Rakefile: 58 | ```ruby 59 | require 'pdksync/rake_tasks' 60 | ``` 61 | 62 | Gemfile: 63 | ```ruby 64 | # frozen_string_literal: true 65 | 66 | source "https://rubygems.org" 67 | 68 | git_source(:github) { |repo_name| "https://github.com/#{repo_name}" } 69 | 70 | gem 'pdksync', github: 'puppetlabs/pdksync', ref: 'pdksync-externalisation-fixes' 71 | gem 'rake' 72 | ``` 73 | 74 | managed_modules.yml: 75 | ```yaml 76 | --- 77 | - repo1 78 | - repo2 79 | - repo3 80 | - repo4 81 | ``` 82 | 83 | pdksync.yml: 84 | ```yaml 85 | --- 86 | namespace: 'YOUR GITHUB NAME' 87 | git_base_uri: 'git@github.com' 88 | ``` 89 | 90 | 91 | Run the following commands to check that everything is working as expected: 92 | 93 | ```shell 94 | bundle install --path .bundle/gems/ 95 | bundle exec rake git:clone_managed_modules 96 | ``` 97 | 98 | pdksync tool is extended with the feature to update the Gemfile. Puppet provides a lot of useful gems to access and manage their functionality between modules. This functionality will help user to perform gem testing prior to release. User is given new rake tasks to update SHA/Version/Branch/line in the Gemfile. Then the changes can be committed, PR can be created which will run the acceptance tests in the PR. If all the tests are executing successfully then the user can close the PRS and release the gem. Below given are the workflows for doing module gem testing with pdksync. 99 | 100 | 101 | In Workflow 1 we can clone modules, update the gem file, create the commit, push the changes and create the PR using separate rake tasks. 102 | ```shell 103 | bundle install --path .bundle/gems/ 104 | bundle exec rake git:clone_managed_modules 105 | bundle exec rake 'pdksync:gem_file_update[]' 106 | bundle exec rake 'git:create_commit[]' 107 | bundle exec rake 'git:push' 108 | bundle exec rake 'git:create_pr[]' 109 | ``` 110 | 111 | In Workflow 2 we can clone modules, update the gem file, create the commit, push the changes and create the PR using single rake task 112 | ``` 113 | Using single rake job 114 | bundle install --path .bundle/gems/ 115 | bundle exec rake 'gem_testing[]' 116 | ``` 117 | 118 | In Workflow 3 we can clone modules, update the gem file, run the tests locally for litmus modules without creating commit,pushing or creating the PR 119 | ``` 120 | Using single rake job 121 | bundle install --path .bundle/gems/ 122 | bundle exec rake git:clone_managed_modules 123 | bundle exec rake 'pdksync:gem_file_update[]' 124 | bundle exec rake 'pdksync:run_tests_locally[]' 125 | bundle exec rake 'pdksync:fetch_test_results_locally[]' 126 | ``` 127 | 128 | Once the verified gem is released we can use pdksync to update the the new version of gem released in the .sync.yaml file. 129 | 130 | pdksync tool is extended with the feature to perform multi gem testing (`puppet-module-gems`). This functionality will identify the current version and bump the version by one. Then it will build and push the gems to gemfury account. Export the GEMFURY_TOKEN to use this rake task. 131 | 132 | ```shell 133 | export GEMFURY_TOKEN= 134 | ``` 135 | 136 | Run the following commands to check that everything is working as expected: 137 | 138 | ```shell 139 | bundle install --path .bundle/gems/ 140 | bundle exec rake -T 141 | bundle exec rake 'git:clone_gem['gem_name']' 142 | ``` 143 | Below given are the workflows for doing multi gem testing with pdksync. 144 | 145 | In this workflow we can clone gems, update the version, build the gem, push the changes to gemfury and update the gem file of the required modules with the latest gem updated in the fury. Then we can create PR or run tests locally or run tests through jenkins to verify the module test results. 146 | 147 | ```shell 148 | bundle install --path .bundle/gems/ 149 | bundle exec rake 'git:clone_gem[]' 150 | bundle exec rake 'pdksync:multi_gem_testing[]' 151 | bundle exec rake 'pdksync:multigem_file_update[]' 152 | ``` 153 | 154 | The rake tasks take in a file, `managed_modules.yml`, stored within the local directory that lists all the repositories that need to be updated. It then clones them, one after another, so that a local copy exists. The `pdk update` command is ran against this local copy, with the subsequent changes being added into a commit on a unique branch. It is then pushed back to the remote master — where the local copy was originally cloned. A pull request against master is opened, and pdksync begins to clone the next repository. 155 | 156 | By default, pdksync will supply a label to a PR (default is 'maintenance'). This can be changed by creating `pdksync.yml` in the local directory and setting the `pdksync_label` key. You must ensure that the label selected exists on the modules that you are applying pdksync to. Should you wish to disable this feature, set `pdksync_label` to an empty string i.e. `''`. Similarly, when supplying a label using the `git:create_pr` rake task, the label must exist on each of the managed modules to run successfully. 157 | 158 | The following rake tasks are available with pdksync: 159 | - `pdksync:show_config` Display the current configuration of pdksync 160 | - `git:clone_managed_modules` Clone managed modules. 161 | - `git:create_commit[:branch_name, :commit_message]` Stage commits for modules, branchname and commit message eg rake 'git:create_commit[flippity, commit messagez]'. 162 | - `git:push` Push staged commits eg rake 'git:push'. 163 | - `git:create_pr[:pr_title, :label]` Create PR for modules. Label is optional eg rake 'git:create_pr[pr title goes here, optional label right here]'. 164 | - `git:clean[:branch_name]` Clean up origin branches, (branches must include pdksync in their name) eg rake 'git:clean[pdksync_origin_branch]'. 165 | - `pdksync:pdk_convert` Runs PDK convert against modules. 166 | - `pdksync:pdk_validate` Runs PDK validate against modules. 167 | - `pdksync[:additional_title]` Run full pdksync process, clone repository, pdk update, create pr. Additional information can be added to the title, which will be appended before the reference section. 168 | - `rake pdksync` PR title outputs as `pdksync - pdksync_heads/master-0-gabccfb1` 169 | - `rake 'pdksync[MODULES-8231]'` PR title outputs as `pdksync - MODULES-8231 - pdksync_heads/master-0-gabccfb1` 170 | - `pdksync:run_a_command[:command, :option]` Run a command against modules eg rake 'pdksync:run_a_command[complex command here -f -gx, 'background']'. :option is an optional parameter which states to run command in backgroud. 171 | - `pdksync:gem_file_update[[:gem_to_test, :gem_line, :gem_sha_finder, :gem_sha_replacer, :gem_version_finder, :gem_version_replacer, :gem_branch_finder, :gem_branch_replacer]]` Run gem_file_update against modules 172 | - eg rake to update gem line `pdksync:gem_file_update['puppet_litmus', "gem 'puppet_litmus'\, git: 'https://github.com/test/puppet_litmus.git'\, branch: 'testbranch'"]'` 173 | - eg rake to update sha `pdksync:gem_file_update['puppet_litmus', '', '20ee04ba1234e9e83eb2ffb5056e23d641c7a018', '20ee04ba1234e9e83eb2ffb5056e23d641c7a31']` 174 | - eg rake to update version`pdksync:gem_file_update['puppet_litmus', '', '', '', "= 0.9.0", "<= 0.10.0", '', '']` 175 | - eg rake to update branch `pdksync:gem_file_update['puppet_litmus', '', '', '', '', '', 'testbranch', 'testbranches']` 176 | - `rake 'gem_testing[:additional_title, :gem_to_test, :gem_line, :gem_sha_finder, :gem_sha_replacer, :gem_version_finder, :gem_version_replacer, :gem_branch_finder, :gem_branch_replacer]'` Run complete Gem file testing (cloning, gemfileupdate, create commit, create PR)PR title outputs as `pdksync_gemtesting - MODULES-8231 - pdksync_heads/master-0-gabccfb1` 177 | - eg rake to perform gem file testing `gem_testing['MODULES-testing', 'puppet_litmus', '', '20ee04ba1234e9e83eb2ffb5056e23d641c7a018', 'testsha']` 178 | - `pdksync:run_tests_locally[:provision_type, :puppet_collection]` Run litmus modules locally 179 | - eg rake 'pdksync:run_tests_locally["default"]' 180 | - `pdksync:fetch_test_results_locally[]` Fetch litmus modules local run results 181 | - eg rake 'pdksync:fetch_test_results_locally[]' 182 | - `pdksync:run_tests_jenkins[:jenkins_server_url, :github_branch, :test_framework, :github_user]` Run traditional modules on jenkins. For now this rake task works just for test_framework: jenkins. 183 | - eg rake 'pdksync:run_tests_jenkins[test_branch, '', test_user]' 184 | - `pdksync:test_results_jenkins[]` Fetch traditional modules jenkins run results 185 | - eg rake 'pdksync:test_results_jenkins[jenkins_server_url]' 186 | - `git:clone_gem[:gem_name]` Clone gem. 187 | - `pdksync:multi_gem_testing[:gem_name, :version_file, :build_gem, :gem_path, :gemfury_username]` Build and Push new gems built to the gemfury account for testing eg rake 'pdksync:multi_gem_testing[]' 188 | - `pdksync:multigem_file_update[:gem_name, :gemfury_username]` Update Gemfile of the modules with the new gem should be pushed to Gemfury.' 189 | 190 | ### Configuration 191 | 192 | By default pdksync will use hardcoded values for configuring itself. However, if you wish to override these values, create a `pdksync.yml` in your working directory and use the following format: 193 | ```yaml 194 | --- 195 | namespace: 'puppetlabs' 196 | pdksync_dir: 'modules_pdksync' 197 | pdksync_gem_dir: 'gems_pdksync', 198 | push_file_destination: 'origin' 199 | create_pr_against: 'master' 200 | managed_modules: 'managed_modules.yml' 201 | pdksync_label: 'maintenance' 202 | git_platform: :github 203 | git_base_uri: 'https://github.com' 204 | # Only used when git_platform is set to :gitlab 205 | gitlab_api_endpoint: 'https://gitlab.com/api/v4' 206 | ``` 207 | 208 | You may override any property. Those that are not specified in your config file will use their corresponding default value from `lib/pdksync/constants.rb`. 209 | 210 | #### Git platform support 211 | 212 | By default pdksync assumes you are hosting your Puppet modules on GitHub, and GitHub is the only platform officially supported by Puppetlabs in pdksync. 213 | 214 | Pdksync also supports the GitLab platform, but without official support by Puppetlabs. 215 | 216 | ##### GitHub 217 | Github.com and Github enterprise both work with octokit which is used in pdksync. There are some settings 218 | you will need to adjust if using the on premise github enterprise edition. 219 | 220 | 1. `api_endpoint: https://mygithub.mycompany.com/api/v3` 221 | 2. `git_base_uri: git@mygithub.mycompany.com` 222 | 3. `export GITHUB_TOKEN=k3939isdiasdf93i_` (your token goes here) 223 | 224 | To use GitHub.com you only need to export your GitHub access token as the 225 | environment variable `GITHUB_TOKEN` and configure the namespace in which your 226 | modules are hosted in `$HOME/.pdksync.yml` as described above. 227 | 228 | ##### GitLab 229 | 230 | To use GitLab at `https://gitlab.com` you need to set `git_platform: :gitlab` 231 | and configure the namespace of your modules in `$HOME/.pdksync.yml`. You also 232 | need to export your GitLab access token as the environment variable 233 | `GITLAB_TOKEN`. 234 | 235 | Your `$HOME/.pdksync.yml` then looks like this: 236 | 237 | ```yaml 238 | # ~/pdksync.yml 239 | --- 240 | namespace: 'acme' 241 | git_platform: :gitlab 242 | ``` 243 | 244 | Export your GitLab access token: 245 | 246 | ```shell 247 | export GITLAB_TOKEN= 248 | ``` 249 | 250 | If you are running your own GitLab instance on premise or use a GitLab instance 251 | other than the official one at `https://gitlab.com` you also need to configure 252 | `git_base_uri` and `api_endpoint` in `$HOME/.pdksync.yml` so that 253 | pdksync knows from where to clone your modules and where to access to GitLab 254 | API to create the live merge requests: 255 | 256 | ```yaml 257 | # ~/pdksync.yml 258 | --- 259 | namespace: 'puppetmodules' 260 | git_platform: :gitlab 261 | git_base_uri: 'https://gitlab.example.com' 262 | # alternatively use SSH: 263 | #git_base_uri: 'ssh://git@gitlab.example.com:2222' 264 | api_endpoint: 'https://gitlab.example.com/api/v4' 265 | ``` 266 | 267 | ### Setting who has the authoritive 268 | It may be desirable to allow modules to dictate which version of the pdk-templates they should sync with. 269 | There are a few settings you can tune to allow for this kind of flexability. These settings are in the pdksync.yml file. All of these settings are optional and have sane defaults. See `rake pdksync:show_config` for the settings that will be used. 270 | 271 | - pdk_templates_prefix: 'nwops-' (example only, keep as empty string) 272 | - pdk_templates_ref: 1.12.0 273 | - pdk_templates_url: https://github.com/puppetlabs/pdk-templates.git 274 | - module_is_authoritive: true 275 | 276 | The first setting is `module_is_authoritive`. When this is set to true the templates and ref specified in the metadata become the authoritive source for these settings. Even if you have pdk_templates_ref and pdk_templates_url specified in pdksync.yml the metadata settings will alwasys be used. 277 | 278 | ```json 279 | # module/metadata.json 280 | { 281 | "pdk-version": "1.11.1", 282 | "template-url": "https://github.com/puppetlabs/pdk-templates#master", 283 | "template-ref": "heads/master-0-gb096033" 284 | } 285 | 286 | ``` 287 | 288 | When `module_is_authoritive` is set to false the pdk_templates_ref and pdk_templates_url will override what is found in the modules's metadata.json file. This is very useful when you have to control pdk-template upgrades on modules. 289 | 290 | The other settings dictiate where the templates are located and which branch, tag or reference you want to use. 291 | `pdk_templates_ref: 'master'` and `pdk_templates_url: https://github.com/puppetlabs/pdk-templates.git`. These settings will only be utilized if module_is_authoritive is set to false. However, if you are performing a conversion via pdksync these settings will also be used since the metadata in the module being converted doesn't have pdk settings yet. 292 | 293 | The last setting `pdk_templates_prefix` is a special use case that allows folks with internal forks of pdk-templates to keep branches of the pdk-template tags with additional custom changes. Setting this to an empty string disables this. You will most likely need to resolve conflicts with this workflow, so it is not for everyone. If you know of a better way please submmit a pull request. 294 | 295 | This strategy works in conjunction with the pdk-template git tags and the workflow looks like: 296 | 1. git fetch upstream (github.com/puppetlabs/pdk-templates) 297 | 2. git checkout master && git rebase upstream/master 298 | 3. git checkout -b nwops-1.0.13 nwops-1.0.12 299 | 4. git rebase 1.0.13 300 | 5. git push origin nwops-1.0.13 301 | 302 | 303 | ### Supporting multiple namespaces 304 | If you have multiple namespaces that you need to support you will need to create a pdksync.yml config 305 | file for each namespace. This will allow you to create a folder structure and keep a separate 306 | managed_modules.yml for that namespace too. 307 | 308 | You can set a PDKSYNC_CONFIG_PATH environment variable that points to the specific pdksync.yml config file for use in a CI or on the CLI. This allows you to set custom names for pdksync.yml file. 309 | 310 | example: `PDKSYNC_CONFIG_PATH=pdksync_ops.yml` 311 | 312 | Or you can set a different HOME environment variable that tells pdksync where to find the pdksync.yml file. Pdksync will locate the pdksync.yml file in the HOME folder you specify. The config file name is not changable in this case. 313 | 314 | example: `HOME=ops` 315 | 316 | ### Logging output 317 | Pdksync uses a logger class to log all output. You can control how the logger works via a few environment variables. 318 | 319 | To control the level set the `LOG_LEVEL` to one of 320 | 1. info 321 | 2. debug 322 | 3. fatal 323 | 4. error 324 | 5. warn 325 | 326 | To control where the logs are sent (defaults to stdout) set the `PDKSYNC_LOG_FILENAME` to a file path. 327 | 328 | ### Workflow 329 | -------- 330 | 331 | It currently runs without additional arguments. To alter how it runs, make alterations to either `HOME/.pdksync.yml` or `managed_modules.yml`. 332 | 333 | ### Managed modules 334 | ---------- 335 | 336 | This module runs through a pre-set array of modules, with this array set within the `managed_modules.yml` file. This file makes use of a simple `yaml` style format to set out the different module names, for example: 337 | 338 | ```yaml 339 | --- 340 | - puppetlabs-motd 341 | - puppetlabs-stdlib 342 | - puppetlabs-mysql 343 | ``` 344 | 345 | To add a module, add it to the list. To remove a module, remove it from the list. If you wish to specify a custom managed modules file, use the `managed_modules` property in your configuration file to specify the path to the file. 346 | 347 | ### Migrating from modulesync to pdksync 348 | -------- 349 | 350 | If your modules are currently managed by modulesync, and you want to use PDK and keep your modules up-to-date, read the following. 351 | 352 | #### Terminology 353 | - `pdk convert` - A command to convert your module, for example, to make it compatible with the PDK. 354 | - `convert_report.txt` - A report that shows the changes PDK will make to your module when `pdk convert` is ran. 355 | - `pdk update` - A command to consume any changes that have been made to the pdk-template used to convert the module. 356 | - `update_report.txt` - A report that shows the changes PDK will make to your module when `pdk update` is ran. 357 | - `pdk validate` - A command to run basic validation checks on your module. 358 | - `pdk test unit` - A command to run all available unit tests on your module. 359 | - `.sync.yml` - A file that lists all of of your module customizations — and will require work before module conversion. 360 | 361 | ##### Prerequisites 362 | * Unit tests are in a good state — with no failures. Check by running `pdk test unit`. 363 | * The module is in good shape. Check by running `pdk validate`. 364 | 365 | When you're confident everything is in good shape, you can start converting your module to make it compatible with PDK. 366 | 367 | ##### Getting started 368 | 369 | 1) Run `pdk convert --noop`. This will output to the console a high level overview of the changes that PDK is planning to make to your files. 370 | 371 | > Note: For an in-depth diff, see the convert_report.txt that is output in the module root directory. 372 | 373 | 2) Make changes to your .sync.yml. State any configuration that the custom [pdk-templates](https://github.com/puppetlabs/pdk-templates) plan to remove. 374 | 375 | Useful commands via the .sync.yml: 376 | 377 | - Add additional gem dependencies: 378 | ```yaml 379 | Gemfile: 380 | required: 381 | ':system_tests': 382 | - gem 'octokit' 383 | platforms: ruby 384 | ``` 385 | - Make changes to your travis configuration: 386 | ```yaml 387 | .travis.yml: 388 | branches: 389 | - release 390 | ``` 391 | - Delete files that you don't want to exist in the repo: 392 | ```yaml 393 | .gitlab-ci.yml: 394 | delete: true 395 | ``` 396 | - Unmanage files that you don't want to be managed: 397 | ```yaml 398 | .gitlab-ci.yml: 399 | unmanaged: true 400 | ``` 401 | > Note: It is unlikely your module will work out of the box. 402 | 403 | 3) When you are finished customizing your .sync.yml file, run `pdk convert --noop` and confirm the changes that PDK will make when you convert. Changes can be found in the `convert_report.txt` 404 | 405 | 4) Run `pdk convert` to convert. You will be prompted to pass in Y/N — type Y and all your changes will be applied. 406 | 407 | > Note: If you have any concerns it is not too late — type N. 408 | 409 | 5) Run your unit tests to confirm that nothing has broken. If there are breakages, you might need to require a library or include a missing gem — address this issue before you continue. 410 | 411 | 6) Run `pdk validate` to ensure there are no failures. 412 | 413 | 7) Commit the changes that the `pdk convert` has made and create your pull request. 414 | 415 | 8) Remove your module from being managed via `modulesync`, and start using `pdksync` going forward — no more manually creating pull requests. 416 | 417 | For more information on keeping your module up to date with the PDK check out [Helens blog post](https://puppet.com/blog/guide-converting-module-pdk). 418 | 419 | ### Compatibility 420 | ---------- 421 | 422 | This tool has been developed and tested on OSX and Linux. **It currently does not run on Windows.** 423 | 424 | ### Contributing 425 | -------- 426 | 427 | 1. Fork the repo 428 | 2. Create your feature branch: 429 | ```shell 430 | git checkout -b my-new-feature 431 | ``` 432 | 3. Commit your changes: 433 | ```shell 434 | git commit -am 'Add some feature' 435 | ``` 436 | 4. Push to the branch: 437 | ```shell 438 | git push origin my-new-feature 439 | ``` 440 | 5. Create a new pull request 441 | --------------------------------------------------------------------------------