├── quality-metrics ├── new │ ├── .keepme │ └── qm-00-attribute-count.md ├── qm-008-cookstyle.md ├── qm-002-collaborators.md ├── qm-009-foodcritic.md ├── qm-001-published.md ├── template.md ├── qm-006-supports.md ├── qm-010-multi_platform.md ├── qm-011-contributing_doc.md ├── qm-013-testing_doc.md ├── qm-012-binaries.md ├── qm-007-integration-tests.md ├── qm-004-issues-url.md ├── qm-005-source-url.md ├── qm-003-license.md └── qm-014-version_tags.md ├── Gemfile ├── .github └── CODEOWNERS ├── Gemfile.lock ├── EDITORS.md ├── SMQM.md ├── MAINTAINERS.md ├── README.md ├── MAINTAINERS.toml ├── Rakefile └── LICENSE /quality-metrics/new/.keepme: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /Gemfile: -------------------------------------------------------------------------------- 1 | source 'https://rubygems.org' 2 | 3 | gem 'octokit' 4 | gem 'rake' 5 | gem 'tomlrb' 6 | -------------------------------------------------------------------------------- /.github/CODEOWNERS: -------------------------------------------------------------------------------- 1 | # Order is important. The last matching pattern has the most precedence. 2 | 3 | * @chef/supermarket-reviewers 4 | -------------------------------------------------------------------------------- /quality-metrics/qm-008-cookstyle.md: -------------------------------------------------------------------------------- 1 | --- 2 | SMQM: 008 3 | Author: Nathen Harvey 4 | Status: Accepted 5 | License: Apache-2.0 6 | --- 7 | 8 | # Cookbook Passes Cookstyle 9 | 10 | The cookbook should pass cookstyle lint testing. 11 | 12 | Code that follows common standards is easier to read, write, and maintain. 13 | 14 | ## Verification 15 | 16 | This is pseudocode: 17 | 18 | ```shell 19 | `cookstyle . --lint` 20 | ``` 21 | -------------------------------------------------------------------------------- /Gemfile.lock: -------------------------------------------------------------------------------- 1 | GEM 2 | remote: https://rubygems.org/ 3 | specs: 4 | addressable (2.4.0) 5 | faraday (0.9.2) 6 | multipart-post (>= 1.2, < 3) 7 | multipart-post (2.0.0) 8 | octokit (4.4.1) 9 | sawyer (~> 0.7.0, >= 0.5.3) 10 | rake (11.3.0) 11 | sawyer (0.7.0) 12 | addressable (>= 2.3.5, < 2.5) 13 | faraday (~> 0.8, < 0.10) 14 | tomlrb (1.2.3) 15 | 16 | PLATFORMS 17 | ruby 18 | 19 | DEPENDENCIES 20 | octokit 21 | rake 22 | tomlrb 23 | 24 | BUNDLED WITH 25 | 1.12.5 26 | -------------------------------------------------------------------------------- /quality-metrics/qm-002-collaborators.md: -------------------------------------------------------------------------------- 1 | --- 2 | SMQM: 002 3 | Author: Nathen Harvey 4 | Status: Implemented 5 | License: Apache-2.0 6 | --- 7 | 8 | # Cookbook has collaborators on the Supermarket 9 | 10 | The cookbook has one or more collaborators listed on the Supermarket. 11 | 12 | Having a collaborator means that there is more than one person with permission to release new versions of the cookbook. 13 | 14 | Implemented in [Supermarket 2.8.0 (2016-07-05)](https://github.com/chef/supermarket/blob/master/CHANGELOG.md#280-2016-07-05) -------------------------------------------------------------------------------- /quality-metrics/qm-009-foodcritic.md: -------------------------------------------------------------------------------- 1 | --- 2 | SMQM: 009 3 | Author: Nathen Harvey 4 | Status: Implemented 5 | License: Apache-2.0 6 | --- 7 | 8 | # Cookbook Passes Foodcritic 9 | 10 | The cookbook should pass Foodcritic correctness and metadata testing. 11 | 12 | Code that follows common standards is easier to read, write, and maintain. 13 | 14 | Implemented in [Supermarket 2.8.0 (2016-07-05)](https://github.com/chef/supermarket/blob/master/CHANGELOG.md#280-2016-07-05) 15 | 16 | This is pseudocode: 17 | 18 | `foodcritic . --tags correctness,metadata` 19 | -------------------------------------------------------------------------------- /quality-metrics/qm-001-published.md: -------------------------------------------------------------------------------- 1 | --- 2 | SMQM: 001 3 | Author: Nathen Harvey 4 | Status: In Progress - Admin Only 5 | License: Apache-2.0 6 | --- 7 | 8 | # Published to the Supermarket 9 | 10 | * The cookbook is published to the Supermarket 11 | * The cookbook is not deprecated 12 | * The cookbook is not up for adoption 13 | 14 | Deprecated or adoptable cookbooks should be used with extreme caution. 15 | 16 | ### Verification 17 | 18 | This is pseudocode 19 | 20 | it 'is published to the supermarket' do 21 | expect((cookbook.deprecated? && cookbook.up_for_adoption?)).to be false 22 | end 23 | -------------------------------------------------------------------------------- /quality-metrics/template.md: -------------------------------------------------------------------------------- 1 | --- 2 | SMQM: UNASSIGNED 3 | Author: Melissa Jones 4 | Status: Draft, Accepted, In Progress - Admin Only, Implemented, Closed 5 | License: Apache-2.0 6 | --- 7 | 8 | # Name of the rule 9 | 10 | Brief description and criteria for meeting the rule. 11 | 12 | * first criterion 13 | * second criterion 14 | * third criterion 15 | 16 | Brief description of why the rule should be met. 17 | 18 | ### Verification 19 | 20 | Pseudocode or actual code that can be used to automatically verify the rule and/or assign appropriate points. 21 | 22 | it 'meets the rule' do 23 | expect(something).to be true 24 | end 25 | -------------------------------------------------------------------------------- /quality-metrics/qm-006-supports.md: -------------------------------------------------------------------------------- 1 | --- 2 | SMQM: 006 3 | Author: John Bellone 4 | Status: In Progress - Admin Only 5 | License: Apache-2.0 6 | --- 7 | 8 | # Cookbook lists at least one platform it supports 9 | 10 | The cookbook has a list of the platforms that it `supports` specified as 11 | part of its metadata. 12 | 13 | Cookbook consumers should have confidence in knowing that a cookbook 14 | is intended to work on a platform in the supports list. 15 | 16 | ### Verification 17 | 18 | Pseudocode or actual code that can be used to automatically verify the rule and/or assign appropriate points. 19 | 20 | ```ruby 21 | it 'has a supports' do 22 | expect(cookbook.supports).to_not be_nil 23 | end 24 | ``` 25 | -------------------------------------------------------------------------------- /quality-metrics/qm-010-multi_platform.md: -------------------------------------------------------------------------------- 1 | --- 2 | SMQM: 010 3 | Author: Tim Smith 4 | Status: Accepted 5 | License: Apache-2.0 6 | --- 7 | 8 | # Cookbook Supports Multiple Platforms 9 | 10 | The cookbook author has indicated support for multiple platforms by including more than 1 supports attribute in the metadata.rb 11 | 12 | Well written cookbooks include support for a diverse set of platforms. It's key that those platforms are noted in the metadata.rb so that they will be listed on Supermarket. 13 | 14 | ### Verification 15 | 16 | Pseudocode or actual code that can be used to automatically verify the rule. 17 | 18 | it 'supports multiple platforms' do 19 | expect(metadata.rb).to contain >1 'supports' 20 | end 21 | -------------------------------------------------------------------------------- /quality-metrics/qm-011-contributing_doc.md: -------------------------------------------------------------------------------- 1 | --- 2 | SMQM: 011 3 | Author: Tim Smith 4 | Status: In Progress - Admin Only 5 | License: Apache-2.0 6 | --- 7 | 8 | # Cookbook Includes CONTRIBUTING.MD file 9 | 10 | The cookbook includes a `CONTRIBUTING.md` file in the root of the cookbook directory with instructions for contributing to the cookbook. 11 | 12 | Proper contributing documentation is an important aide to ease the barrier to cookbook contribution and therefore should be included in all published cookbooks. 13 | 14 | ### Verification 15 | 16 | Pseudocode or actual code that can be used to automatically verify the rule. 17 | 18 | it 'has a contributing doc' do 19 | expect(CONTRIBUTING.md).to exist 20 | end 21 | -------------------------------------------------------------------------------- /quality-metrics/qm-013-testing_doc.md: -------------------------------------------------------------------------------- 1 | --- 2 | SMQM: 013 3 | Author: Tim Smith 4 | Status: In Progress - Admin Only 5 | License: Apache-2.0 6 | --- 7 | 8 | # Cookbook Includes TESTING.MD File 9 | 10 | The cookbook includes a `TESTING.md` file in the root of the cookbook directory with instructions for testing the cookbook. 11 | 12 | Proper testing documentation is an important aide to ease the barrier to cookbook contribution and improve cookbook quality. Therefor a testing document should be included in all published cookbooks. 13 | 14 | ### Verification 15 | 16 | Pseudocode or actual code that can be used to automatically verify the rule. 17 | 18 | it 'has a testing doc' do 19 | expect(TESTING.md).to exist 20 | end 21 | -------------------------------------------------------------------------------- /quality-metrics/qm-012-binaries.md: -------------------------------------------------------------------------------- 1 | --- 2 | SMQM: 012 3 | Author: Clinton Wolfe 4 | Status: In Progress - Admin Only 5 | License: Apache-2.0 6 | --- 7 | 8 | # Cookbook does not contain binaries 9 | 10 | The cookbook should not contain any binary files. 11 | 12 | Cookbook consumers should be able to examine the contents of the 13 | cookbook's source code in order to evaluate its actions and security 14 | implications. 15 | 16 | Community cookbooks that need to install binaries should obtain them 17 | from signed artifact repositories. 18 | 19 | ### Verification 20 | 21 | Pseudocode: 22 | 23 | it 'does not contain binaries' do 24 | list_all_files(cookbook).each do |file| 25 | expect(file).to_not be binary 26 | end 27 | end 28 | -------------------------------------------------------------------------------- /quality-metrics/qm-007-integration-tests.md: -------------------------------------------------------------------------------- 1 | --- 2 | SMQM: 007 3 | Author: JJ Asghar 4 | Status: On Hold 5 | License: Apache-2.0 6 | --- 7 | 8 | # Cookbook includes integration tests 9 | 10 | There should be at least one of the following directories with test files: 11 | 12 | * serverspec 13 | * bats 14 | * TBD 15 | 16 | Integration testing allows you to assert the state of the converged node to ensure it is in the desired state. 17 | 18 | Integration testing is pinnacle to velocity and verifying community cookbooks do what we expect. 19 | 20 | ### Verification 21 | 22 | Pseudocode or actual code that can be used to automatically verify the rule and/or assign appropriate points. 23 | 24 | ```bash 25 | [ $(find test/integration/ -name '*_spec.rb' | wc -l) -ge $(find recipes/ -name '*.rb' | wc -l ) ] 26 | ``` 27 | -------------------------------------------------------------------------------- /quality-metrics/qm-004-issues-url.md: -------------------------------------------------------------------------------- 1 | --- 2 | SMQM: 004 3 | Author: John Bellone 4 | Status: Closed 5 | License: Apache-2.0 6 | --- 7 | 8 | # Cookbook lists address for issues 9 | 10 | The cookbook has a `issues_url` specified as part of its metadata. 11 | 12 | Cookbook consumers should know where to create issues - including 13 | bugs, feature requests, etc - from the Supermarket page. 14 | 15 | ### Verification 16 | 17 | Pseudocode or actual code that can be used to automatically verify the rule and/or assign appropriate points. 18 | 19 | ```ruby 20 | it 'has a issues_url' do 21 | expect(cookbook.issues_url).to_not be_nil 22 | end 23 | ``` 24 | 25 | # Status 26 | 27 | This quality metric has been closed and is currently implemented as part of the [Cookbook Passes Foodcritic](qm-009-foodcritic.md). [FC0064 - Ensure issues_url is set in metadata](http://www.foodcritic.io/#FC064). 28 | -------------------------------------------------------------------------------- /quality-metrics/qm-005-source-url.md: -------------------------------------------------------------------------------- 1 | --- 2 | SMQM: 005 3 | Author: John Bellone 4 | Status: Closed 5 | License: Apache-2.0 6 | --- 7 | 8 | # Cookbook lists a source address for repository 9 | 10 | The cookbook has a `source_url` specified as part of its metadata. 11 | 12 | Cookbook consumers should know where to look for the source code to 13 | take a look at previous releases, pull-requests or unreleased changes. 14 | 15 | ### Verification 16 | 17 | Pseudocode or actual code that can be used to automatically verify the rule and/or assign appropriate points. 18 | 19 | ```ruby 20 | it 'has a source_url' do 21 | expect(cookbook.source_url).to_not be_nil 22 | end 23 | ``` 24 | 25 | # Status 26 | 27 | This quality metric has been closed and is currently implemented as part of the [Cookbook Passes Foodcritic](qm-009-foodcritic.md). [FC0065 - Ensure source_url is set in metadata](http://www.foodcritic.io/#FC065). 28 | -------------------------------------------------------------------------------- /quality-metrics/qm-003-license.md: -------------------------------------------------------------------------------- 1 | --- 2 | SMQM: 003 3 | Author: Nathen Harvey 4 | Status: In Progress - Admin Only 5 | License: Apache-2.0 6 | --- 7 | 8 | # Cookbook has an open source license 9 | 10 | Cookbook is licensed using an open source license. 11 | 12 | Acceptable licenses are any OSI-approved open source license. The [Software 13 | Package Data Exchange® (SPDX®) specification](https://spdx.org/licenses/) will 14 | be used to identify licenses and their approval status with the OSI. 15 | 16 | Cookbooks without open source licenses may not be modifiable and may not even be legally used by consumers. 17 | 18 | ### Verification 19 | 20 | Pseudocode or actual code that can be used to automatically verify the rule and/or assign appropriate points. 21 | 22 | it 'has an open source license' do 23 | expect(collection_of_osi_approved_license_strings) 24 | .to include(cookbook_version.license) 25 | end 26 | -------------------------------------------------------------------------------- /EDITORS.md: -------------------------------------------------------------------------------- 1 | # Editors 2 | 3 | The Editors manage the cookbook quality metrics repository and assist with the 4 | workflow. They are not responsible for managing the content of RFCs, only 5 | formatting and procedure. To volunteer as an editor please create a pull request 6 | against this document that adds your name to the list. The existing editors will 7 | vote on the request, with the Lieutenant retaining veto power. 8 | 9 | When a metric has been approved, an editor will: 10 | 11 | * Assign a number to the quality metric by updating the name of the file and 12 | the `SMQM` field 13 | * Move the file from `quality-metrics/new` to `quality-metrics/` 14 | * Update the status to `Accepted` 15 | 16 | The current editors are: 17 | 18 | * [Nathen Harvey](https://github.com/nathenharvey) 19 | * Slack - nathenharvey 20 | * [@nathenharvey](https://twitter.com/nathenharvey) 21 | * [Tim Smith](https://github.com/tas50) 22 | * [Sean O'Meara](https://github.com/someara) 23 | * [Nell Shamrell](https://github.com/nellshamrell) 24 | * [Robb Kidd](https://github.com/robbkidd) 25 | -------------------------------------------------------------------------------- /quality-metrics/new/qm-00-attribute-count.md: -------------------------------------------------------------------------------- 1 | --- 2 | SMQM: UNASSIGNED 3 | Author: David Aronsohn 4 | Status: Draft 5 | License: Apache-2.0 6 | --- 7 | 8 | # Score penalty on Attribute cound 9 | 10 | A score modifier based on the count of attributes in files. 11 | 12 | * Greater than 20 attributes should be a penalty 13 | * Greater than 50 should be another pentalty 14 | * Greater than 100 should be another pentalty 15 | 16 | A quality community cookbook should have as few attributes as possible by default for tuning. 17 | 18 | ### Verification 19 | 20 | Pseudocode or actual code that can be used to automatically verify the rule and/or assign appropriate points. 21 | 22 | it 'has less than 21 attributes' do 23 | expect(attributes declared in atributes/*.rb).to be lt 21 24 | end 25 | it 'has less than 51 attributes' do 26 | expect(attributes declared in atributes/*.rb).to be lt 51 27 | end 28 | it 'has less than 100 attributes' do 29 | expect(attributes declared in atributes/*.rb).to be lt 100 30 | end 31 | -------------------------------------------------------------------------------- /quality-metrics/qm-014-version_tags.md: -------------------------------------------------------------------------------- 1 | --- 2 | SMQM: 014 3 | Author: Robb Kidd 4 | Status: In Progress - Admin Only 5 | License: Apache-2.0 6 | --- 7 | 8 | # Cookbook source has a tag matching cookbook version 9 | 10 | The source repository for a cookbook version has a tag matching the version of 11 | the cookbook declared in metadata. 12 | 13 | * Depends on QM-005, metadata contains a `source_url` 14 | * Initially limited to cookbook projects whose source is available in GitHub. 15 | 16 | To facilitate troubleshooting, consumers of a cookbook should be able to trace 17 | the version they are using to the state of the cookbook's source at the time the 18 | cookbook build artifact was produced. 19 | 20 | ### Verification 21 | 22 | Pseudocode: 23 | 24 | it 'has a tag in source that matches the cookbook version' do 25 | cookbook_version = cookbook_version_from_metadata 26 | source_repo_response = http_get "#{source_url}/tree/#{cookbook_version}" 27 | expect(source_repo_response).to be 200 28 | end 29 | 30 | ... or ... 31 | 32 | it 'has a tag in source that matches the cookbook version' do 33 | # require 'octokit' 34 | source_tags = octokit_tags_for_repo cookbook_source_url_from_metadata 35 | expect(source_tags).to include cookbook_version_from_metadata 36 | end 37 | -------------------------------------------------------------------------------- /SMQM.md: -------------------------------------------------------------------------------- 1 | # Supermarket Quality Metrics 2 | 3 | This file lists each quality metric. 4 | 5 | SMQM | Name | Status 6 | ---- | --------------------------------------------------- | ----------- 7 | 001 | Published to the Supermarket | In Progress - Admin Only 8 | 002 | Cookbook has collaborators on the Supermarket | Implemented 9 | 003 | Cookbook has an open source license | In Progress - Admin Only 10 | 004 | Cookbook lists address for issues | Closed 11 | 005 | Cookbook lists a source address for repository | Closed 12 | 006 | Cookbook lists at least one platform it supports | In Progress - Admin Only 13 | 007 | Cookbook includes integration tests | Accepted 14 | 008 | Cookbook Passes Cookstyle | Accepted 15 | 009 | Cookbook Passes Foodcritic | Implemented 16 | 010 | Cookbook Supports Multiple Platforms | Accepted 17 | 011 | Cookbook Includes CONTRIBUTING.MD file | In Progress - Admin Only 18 | 012 | Cookbook does not contain binaries | In Progress - Admin Only 19 | 013 | Cookbook Includes TESTING.MD File | In Progress - Admin Only 20 | 014 | Cookbook source has a tag matching cookbook version | In Progress - Admin Only 21 | 22 | See the [README](README.md) for more information on the various states of a quality metric. 23 | -------------------------------------------------------------------------------- /MAINTAINERS.md: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | # Maintainers 6 | 7 | This file lists how the Cookbook Quality Metrics project is maintained. When making 8 | changes to the system, this file tells you who needs to review your patch - you need at 9 | least two maintainers to provide a :+1: on your pull request. Additionally, you need 10 | to not receive a veto from a Lieutenant or the Project Lead. 11 | Check out [How Chef is Maintained](https://github.com/chef/chef-rfc/blob/master/rfc030-maintenance-policy.md#how-the-project-is-maintained) 12 | for details on the process, how to become a maintainer, lieutenant, or the project lead. 13 | 14 | ## Components 15 | 16 | ## Cookbook Quality Metrics 17 | 18 | Handles the [Cookbook Quality Metrics](https://github.com/chef-cookbooks/cookbook-quality-metrics) 19 | component which is used by the [Supermarket](https://github.com/chef/supermarket). 20 | 21 | To mention the team, use @chef-cookbooks/quality-metrics-maintainers 22 | 23 | ### Lieutenant 24 | 25 | * [Robb Kidd](https://github.com/robbkidd) 26 | * Slack - robbkidd 27 | * [@robbkidd](https://twitter.com/robbkidd) 28 | 29 | ### Maintainers 30 | 31 | * [Tim Smith](https://github.com/tas50) 32 | * [Joshua Timberman](https://github.com/jtimberman) 33 | * [Steven Murawski](https://github.com/smurawski) 34 | * [Thom May](https://github.com/thommay) 35 | * Slack - thom 36 | * [@thommay](https://twitter.com/thommay) 37 | * [Peter Burkholder](https://github.com/pburkholder) 38 | * [Adam Leff](https://github.com/adamleff) 39 | * Slack - adamleff 40 | * [@adamleff](https://twitter.com/adamleff) 41 | * [Jennifer Davis](https://github.com/iennae) 42 | * [Carl Perry](https://github.com/edolnx) 43 | * [Seth Thomas](https://github.com/cheeseplus) 44 | * [Andre Elizondo](https://github.com/andrewelizondo) 45 | * [JJ Asghar](https://github.com/jjasghar) 46 | * Slack - j^2 47 | * [@jjasghar](https://twitter.com/jjasghar) 48 | * [Matt Ray](https://github.com/mattray) 49 | * [Matt Wrock](https://github.com/mwrock) 50 | * [Nell Shamrell](https://github.com/nellshamrell) 51 | * [Ryan Cragun](https://github.com/ryancragun) 52 | * [Stephen Lauck](https://github.com/stephenlauck) 53 | * [Nathen Harvey](https://github.com/nathenharvey) 54 | * Slack - nathenharvey 55 | * [@nathenharvey](https://twitter.com/nathenharvey) 56 | 57 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Cookbook Quality Metrics 2 | 3 | This is a system used to provide an objective measure of a number of cookbook 4 | quality metrics. The system takes inspiration from [Foodcritic](http://www.foodcritic.io/). 5 | 6 | **Umbrella Project**: [Supermarket](https://github.com/chef/chef-oss-practices/blob/master/projects/supermarket.md) 7 | 8 | * **[Project State](https://github.com/chef/chef-oss-practices/blob/master/repo-management/repo-states.md):** Maintained 9 | * **Issues [Response Time Maximum](https://github.com/chef/chef-oss-practices/blob/master/repo-management/repo-states.md):** 14 days 10 | * **Pull Request [Response Time Maximum](https://github.com/chef/chef-oss-practices/blob/master/repo-management/repo-states.md):** 14 days 11 | 12 | This repository is a place where the community can start collaborating on 13 | metrics. 14 | 15 | The lifecycle of a metric will be: 16 | 17 | * *Draft* - this is a proposed metric, ready for community discussion and 18 | approval. 19 | * *Accepted* - this metric has been accepted and merged into the master branch. 20 | * *In Progress* - data for this metric is being gathered and is visible by some 21 | mechanism but not displayed on the Supermarket by default. 22 | * *Implemented* - this metric has been implemented and is shown on the Supermarket. 23 | * *Closed* - this metric has not been accepted or has been removed from the 24 | system. 25 | 26 | ## Proposing New Metrics 27 | 28 | * create a new branch in this repository 29 | * copy `quality-metrics/template.md` to `quality-metrics/new/qm-000-SHORT_NAME.md` 30 | * Add new metric to your new file 31 | * Submit a pull request against this repository 32 | 33 | ## Approving New Metrics 34 | 35 | * New metrics will be reviewed by the people listed in `MAINTAINERS.md` 36 | * At least two MAINTAINERS must approve each new metric with a :+1: 37 | * Any maintainer may :-1: a proposed metric. This will increase the 38 | requirement for a metric to be merged to an absolute majority of maintainers. 39 | * An EDITOR (someone listed in `EDITORS.md`) will: 40 | * Assign a number to the quality metric by updating the name of the file and 41 | the `SMQM` field 42 | * Move the file from `quality-metrics/new` to `quality-metrics/` 43 | * Update the status to `Accepted` 44 | * Add the metric to `SMQM.md` 45 | 46 | The metric's status should be changed to `In Progress` once data is being gathered for that metric. 47 | 48 | The metric's status should be changed to `Implemented` when the metric is being measured and is visible on the [public Supermarket](https://supermarket.chef.io). 49 | 50 | ## Viewing Cookbook Metrics 51 | 52 | The [Supermarket](https://supermarket.chef.io) will show the metrics for each cookbook. We are currently concerned with coming up with a good list of cookbook quality metrics outside of the technical implementation. Once this moves into the implementation phase, there may be metrics here which require adjustments to Chef, Supermarket, or other tools. 53 | 54 | ## Maintenance Policy 55 | 56 | This project generally follows [Chef's Maintenance Policy](https://github.com/chef/chef-rfc/blob/master/rfc030-maintenance-policy.md). 57 | 58 | * The **Project Lead** for this is project is the current project lead for [Chef](https://github.com/chef/chef) as specified in the [MAINTAINERS file](https://github.com/chef/chef/blob/master/MAINTAINERS.md). 59 | 60 | * The **Lieutenant** and **Maintainers** for this project are listed in `MAINTAINERS.md`. 61 | -------------------------------------------------------------------------------- /MAINTAINERS.toml: -------------------------------------------------------------------------------- 1 | [Preamble] 2 | title = "Maintainers" 3 | text = """ 4 | This file lists how the Cookbook Quality Metrics project is maintained. When making 5 | changes to the system, this file tells you who needs to review your patch - you need at 6 | least two maintainers to provide a :+1: on your pull request. Additionally, you need 7 | to not receive a veto from a Lieutenant or the Project Lead. 8 | Check out [How Chef is Maintained](https://github.com/chef/chef-rfc/blob/master/rfc030-maintenance-policy.md#how-the-project-is-maintained) 9 | for details on the process, how to become a maintainer, lieutenant, or the project lead. 10 | """ 11 | 12 | [Org] 13 | [Org.Components] 14 | title = "Components" 15 | 16 | [Org.Components.CookbookQuality] 17 | title = "Cookbook Quality Metrics" 18 | team = "quality-metrics-maintainers" 19 | text = """ 20 | Handles the [Cookbook Quality Metrics](https://github.com/chef-cookbooks/cookbook-quality-metrics) 21 | component which is used by the [Supermarket](https://github.com/chef/supermarket). 22 | """ 23 | 24 | lieutenant = "robbkidd" 25 | 26 | maintainers = [ 27 | "tas50", 28 | "jtimberman", 29 | "smurawski", 30 | "thommay", 31 | "pburkholder", 32 | "adamleff", 33 | "iennae", 34 | "edolnx", 35 | "cheeseplus", 36 | "andrewelizondo", 37 | "jjasghar", 38 | "mattray", 39 | "mwrock", 40 | "nellshamrell", 41 | "ryancragun", 42 | "stephenlauck", 43 | "nathenharvey" 44 | ] 45 | 46 | [people] 47 | [people.nathenharvey] 48 | Name = "Nathen Harvey" 49 | GitHub = "nathenharvey" 50 | Slack = "nathenharvey" 51 | Twitter = "nathenharvey" 52 | 53 | 54 | [people.tas50] 55 | Name = "Tim Smith" 56 | GitHub = "tas50" 57 | 58 | [people.jtimberman] 59 | Name = "Joshua Timberman" 60 | GitHub = "jtimberman" 61 | 62 | [people.smurawski] 63 | Name = "Steven Murawski" 64 | GitHub = "smurawski" 65 | 66 | [people.thommay] 67 | Name = "Thom May" 68 | GitHub = "thommay" 69 | Slack = "thom" 70 | Twitter = "thommay" 71 | 72 | [people.pburkholder] 73 | Name = "Peter Burkholder" 74 | GitHub = "pburkholder" 75 | 76 | [people.adamleff] 77 | Name = "Adam Leff" 78 | GitHub = "adamleff" 79 | Slack = "adamleff" 80 | Twitter = "adamleff" 81 | 82 | [people.iennae] 83 | Name = "Jennifer Davis" 84 | GitHub = "iennae" 85 | 86 | [people.edolnx] 87 | Name = "Carl Perry" 88 | GitHub = "edolnx" 89 | 90 | [people.cheeseplus] 91 | Name = "Seth Thomas" 92 | GitHub = "cheeseplus" 93 | 94 | [people.andrewelizondo] 95 | Name = "Andre Elizondo" 96 | GitHub = "andrewelizondo" 97 | 98 | [people.jjasghar] 99 | Name = "JJ Asghar" 100 | GitHub = "jjasghar" 101 | Slack = "j^2" 102 | Twitter = "jjasghar" 103 | 104 | [people.mattray] 105 | Name = "Matt Ray" 106 | GitHub = "mattray" 107 | 108 | [people.mwrock] 109 | Name = "Matt Wrock" 110 | GitHub = "mwrock" 111 | 112 | [people.nellshamrell] 113 | Name = "Nell Shamrell" 114 | GitHub = "nellshamrell" 115 | 116 | [people.ryancragun] 117 | Name = "Ryan Cragun" 118 | GitHub = "ryancragun" 119 | 120 | [people.stephenlauck] 121 | Name = "Stephen Lauck" 122 | GitHub = "stephenlauck" 123 | 124 | [people.robbkidd] 125 | Name = "Robb Kidd" 126 | GitHub = "robbkidd" 127 | Twitter = "robbkidd" 128 | Slack = "robbkidd" 129 | -------------------------------------------------------------------------------- /Rakefile: -------------------------------------------------------------------------------- 1 | # 2 | # Copyright:: 2015-2019, Chef Software, Inc. 3 | # License:: Apache License, Version 2.0 4 | # 5 | # Licensed under the Apache License, Version 2.0 (the "License"); 6 | # you may not use this file except in compliance with the License. 7 | # You may obtain a copy of the License at 8 | # 9 | # http://www.apache.org/licenses/LICENSE-2.0 10 | # 11 | # Unless required by applicable law or agreed to in writing, software 12 | # distributed under the License is distributed on an "AS IS" BASIS, 13 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 14 | # See the License for the specific language governing permissions and 15 | # limitations under the License. 16 | # 17 | 18 | require 'rake' 19 | 20 | SOURCE = File.join(File.dirname(__FILE__), 'MAINTAINERS.toml') 21 | TARGET = File.join(File.dirname(__FILE__), 'MAINTAINERS.md') 22 | 23 | # The list of repositories that teams should own 24 | REPOSITORIES = ['chef-cookbooks/cookbook-quality-metrics'].freeze 25 | 26 | begin 27 | require 'tomlrb' 28 | require 'octokit' 29 | require 'pp' 30 | task default: 'maintainers:generate' 31 | 32 | namespace :maintainers do 33 | desc 'Generate MarkDown version of MAINTAINERS file' 34 | task :generate do 35 | maintainers = Tomlrb.load_file SOURCE 36 | out = "\n\n" 37 | out << "\n\n" 38 | out << '# ' + maintainers['Preamble']['title'] + "\n\n" 39 | out << maintainers['Preamble']['text'] + "\n" 40 | out << components(maintainers['people'], maintainers['Org']['Components']) 41 | File.open(TARGET, 'w') do |fn| 42 | fn.write out 43 | end 44 | end 45 | 46 | desc 'Synchronize GitHub teams' 47 | task :synchronize do 48 | Octokit.auto_paginate = true 49 | get_github_teams 50 | prepare_teams(source['Org']['Components'].dup) 51 | sync_teams! 52 | end 53 | end 54 | 55 | def github 56 | @github ||= Octokit::Client.new(netrc: true) 57 | end 58 | 59 | def source 60 | @source ||= Tomlrb.load_file SOURCE 61 | end 62 | 63 | def teams 64 | @teams ||= { 'quality-metrics-maintainers' => { 'title' => 'Cookbook Quality Metrics' } } 65 | end 66 | 67 | def add_members(team, name) 68 | teams['quality-metrics-maintainers']['members'] ||= [] 69 | teams['quality-metrics-maintainers']['members'] << name 70 | teams[team] ||= {} 71 | teams[team]['members'] ||= [] 72 | teams[team]['members'] << name 73 | end 74 | 75 | def set_team_title(team, title) 76 | teams[team] ||= {} 77 | teams[team]['title'] = title 78 | end 79 | 80 | def gh_teams 81 | @gh_teams ||= {} 82 | end 83 | 84 | # we have to resolve team names to ids. While we're at it, we can get the privacy 85 | # setting, so we know whether we need to update it 86 | def get_github_teams 87 | github.org_teams('chef-cookbooks').each do |team| 88 | gh_teams[team[:slug]] = { 'id' => team[:id], 'privacy' => team[:privacy] } 89 | end 90 | end 91 | 92 | def get_github_team(team) 93 | github.team_members(gh_teams[team]['id']).map do |member| 94 | member[:login] 95 | end.sort.uniq.map(&:downcase) 96 | rescue 97 | [] 98 | end 99 | 100 | def create_team(team) 101 | puts "creating new github team: #{team} with title: #{teams[team]['title']} " 102 | t = github.create_team('chef-cookbooks', name: team, description: teams[team]['title'], 103 | privacy: 'closed', repo_names: REPOSITORIES, 104 | accept: 'application/vnd.github.ironman-preview+json') 105 | gh_teams[team] = { 'id' => t[:id], 'privacy' => t[:privacy] } 106 | end 107 | 108 | def compare_teams(current, desired) 109 | # additions are the subtraction of the current state from the desired state 110 | # deletions are the subtraction of the desired state from the current state 111 | [desired - current, current - desired] 112 | end 113 | 114 | def prepare_teams(cmp) 115 | %w(text paths).each { |k| cmp.delete(k) } 116 | if cmp.key?('team') 117 | team = cmp.delete('team') 118 | add_members(team, cmp.delete('lieutenant')) if cmp.key?('lieutenant') 119 | add_members(team, cmp.delete('maintainers')) if cmp.key?('maintainers') 120 | set_team_title(team, cmp.delete('title')) 121 | else 122 | %w(maintainers lieutenant title).each { |k| cmp.delete(k) } 123 | end 124 | cmp.each { |_k, v| prepare_teams(v) } 125 | end 126 | 127 | def update_team(team, additions, deletions) 128 | create_team(team) unless gh_teams.key?(team) 129 | update_team_privacy(team) 130 | add_team_members(team, additions) 131 | remove_team_members(team, deletions) 132 | rescue 133 | puts "failed for #{team}" 134 | end 135 | 136 | def update_team_privacy(team) 137 | return 138 | return if gh_teams[team]['privacy'] == 'closed' 139 | puts "Setting #{team} privacy to closed from #{gh_teams[team]['privacy']}" 140 | github.update_team(gh_teams[team]['id'], privacy: 'closed', 141 | accept: 'application/vnd.github.ironman-preview+json') 142 | end 143 | 144 | def add_team_members(team, additions) 145 | additions.each do |member| 146 | puts "Adding #{member} to #{team}" 147 | github.add_team_membership(gh_teams[team]['id'], member, role: 'member', 148 | accept: 'application/vnd.github.ironman-preview+json') 149 | end 150 | end 151 | 152 | def remove_team_members(team, deletions) 153 | deletions.each do |member| 154 | puts "Removing #{member} from #{team}" 155 | github.remove_team_membership(gh_teams[team]['id'], member, 156 | accept: 'application/vnd.github.ironman-preview+json') 157 | end 158 | end 159 | 160 | def sync_teams! 161 | teams.each do |name, details| 162 | current = get_github_team(name) 163 | desired = details['members'].flatten.sort.uniq.map(&:downcase) 164 | additions, deletions = compare_teams(current, desired) 165 | update_team(name, additions, deletions) 166 | end 167 | end 168 | 169 | def get_person(person) 170 | source['people'][person] 171 | end 172 | 173 | def components(list, cmp) 174 | out = '## ' + cmp.delete('title') + "\n\n" 175 | out << cmp.delete('text') + "\n" if cmp.key?('text') 176 | out << "To mention the team, use @chef-cookbooks/#{cmp.delete('team')}\n\n" if cmp.key?('team') 177 | if cmp.key?('lieutenant') 178 | out << "### Lieutenant\n\n" 179 | out << person(list, cmp.delete('lieutenant')) + "\n\n" 180 | end 181 | out << maintainers(list, cmp.delete('maintainers')) + "\n" if cmp.key?('maintainers') 182 | cmp.delete('paths') 183 | cmp.each { |_k, v| out << components(list, v) } 184 | out 185 | end 186 | 187 | def maintainers(list, people) 188 | o = "### Maintainers\n\n" 189 | people.each do |p| 190 | o << person(list, p) + "\n" 191 | end 192 | o 193 | end 194 | 195 | def person(list, person) 196 | out = if list[person].key?('GitHub') 197 | "* [#{list[person]['Name']}](https://github.com/#{list[person]['GitHub']})" 198 | else 199 | "* #{list[person]['Name']}" 200 | end 201 | out << "\n * Slack - #{list[person]['Slack']}" if list[person].key?('Slack') 202 | out << "\n * [@#{list[person]['Twitter']}](https://twitter.com/#{list[person]['Twitter']})" if list[person].key?('Twitter') 203 | out << "\n * [#{list[person]['email']}](mailto:#{list[person]['email']})" if list[person].key?('email') 204 | out << "\n * #{list[person]['phone']}" if list[person].key?('phone') 205 | out << "\n * [ServerFault](#{list[person]['ServerFault']})" if list[person].key?('ServerFault') 206 | out 207 | end 208 | rescue LoadError 209 | STDERR.puts "\n*** Gems missing. Please run bundle install.\n\n" 210 | end 211 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "{}" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright {yyyy} {name of copyright owner} 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | 203 | --------------------------------------------------------------------------------