├── assets ├── KPI_config.png ├── Sample_PBI.png ├── Sample_KPI_PBI.png ├── Advanced_editor.png ├── Modify_JSON_source.png ├── Sample_Metrics_PBI.png ├── Advanced_editor_query.png └── Advanced_editor_metrics_query.png ├── samples ├── GitHub Copilot - Telemetry Sample (Usage).pbix ├── GitHub Copilot - Telemetry Sample (Metrics with KPI).pbix └── copilot_metrics_response_sample_small.json ├── .github ├── CODEOWNERS ├── SUPPORT.md ├── SECURITY.md ├── CONTRIBUTING.md └── CODE_OF_CONDUCT.md ├── queries ├── organization_metrics.pq ├── enterprise_metrics.pq ├── organization_usage.pq ├── teams_metrics.pq ├── enterprise_usage.pq └── enterprise_team_metrics.pq ├── LICENSE.txt ├── publish ├── azure-pipelines.yml └── README.md ├── USAGE_API.md └── README.md /assets/KPI_config.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/github-copilot-resources/copilot-metrics-viewer-power-bi/HEAD/assets/KPI_config.png -------------------------------------------------------------------------------- /assets/Sample_PBI.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/github-copilot-resources/copilot-metrics-viewer-power-bi/HEAD/assets/Sample_PBI.png -------------------------------------------------------------------------------- /assets/Sample_KPI_PBI.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/github-copilot-resources/copilot-metrics-viewer-power-bi/HEAD/assets/Sample_KPI_PBI.png -------------------------------------------------------------------------------- /assets/Advanced_editor.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/github-copilot-resources/copilot-metrics-viewer-power-bi/HEAD/assets/Advanced_editor.png -------------------------------------------------------------------------------- /assets/Modify_JSON_source.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/github-copilot-resources/copilot-metrics-viewer-power-bi/HEAD/assets/Modify_JSON_source.png -------------------------------------------------------------------------------- /assets/Sample_Metrics_PBI.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/github-copilot-resources/copilot-metrics-viewer-power-bi/HEAD/assets/Sample_Metrics_PBI.png -------------------------------------------------------------------------------- /assets/Advanced_editor_query.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/github-copilot-resources/copilot-metrics-viewer-power-bi/HEAD/assets/Advanced_editor_query.png -------------------------------------------------------------------------------- /assets/Advanced_editor_metrics_query.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/github-copilot-resources/copilot-metrics-viewer-power-bi/HEAD/assets/Advanced_editor_metrics_query.png -------------------------------------------------------------------------------- /samples/GitHub Copilot - Telemetry Sample (Usage).pbix: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/github-copilot-resources/copilot-metrics-viewer-power-bi/HEAD/samples/GitHub Copilot - Telemetry Sample (Usage).pbix -------------------------------------------------------------------------------- /.github/CODEOWNERS: -------------------------------------------------------------------------------- 1 | # This is a comment 2 | # Each line is a file pattern followed by one or more owners. 3 | 4 | # The '*' pattern refers to all files in the repository. 5 | * @jasonmoodie @Eldrick19 6 | 7 | -------------------------------------------------------------------------------- /samples/GitHub Copilot - Telemetry Sample (Metrics with KPI).pbix: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/github-copilot-resources/copilot-metrics-viewer-power-bi/HEAD/samples/GitHub Copilot - Telemetry Sample (Metrics with KPI).pbix -------------------------------------------------------------------------------- /queries/organization_metrics.pq: -------------------------------------------------------------------------------- 1 | let 2 | // Replace and with your actual token and org name. 3 | url = "https://api.github.com/orgs//copilot/metrics", 4 | headers = [ 5 | #"Accept" = "application/vnd.github+json", 6 | #"Authorization" = "Bearer ", 7 | #"X-GitHub-Api-Version" = "2022-11-28" 8 | ], 9 | Source = Json.Document(Web.Contents(url, [Headers=headers])), -------------------------------------------------------------------------------- /queries/enterprise_metrics.pq: -------------------------------------------------------------------------------- 1 | let 2 | // Replace and with your actual token and enterprise name. 3 | url = "https://api.github.com/enterprises//copilot/metrics", 4 | headers = [ 5 | #"Accept" = "application/vnd.github+json", 6 | #"Authorization" = "Bearer ", 7 | #"X-GitHub-Api-Version" = "2022-11-28" 8 | ], 9 | Source = Json.Document(Web.Contents(url, [Headers=headers])), -------------------------------------------------------------------------------- /queries/organization_usage.pq: -------------------------------------------------------------------------------- 1 | let 2 | // Replace and with your actual token and org name. 3 | url = "https://api.github.com/orgs//copilot/usage", 4 | headers = [ 5 | #"Accept" = "application/vnd.github+json", 6 | #"Authorization" = "Bearer ", 7 | #"X-GitHub-Api-Version" = "2022-11-28" 8 | ], 9 | Source = Json.Document(Web.Contents(url, [Headers=headers])) 10 | in 11 | Source -------------------------------------------------------------------------------- /queries/teams_metrics.pq: -------------------------------------------------------------------------------- 1 | let 2 | // Replace , and with your actual token and enterprise name. 3 | url = "https://api.github.com/orgs//team//copilot/metrics", 4 | headers = [ 5 | #"Accept" = "application/vnd.github+json", 6 | #"Authorization" = "Bearer ", 7 | #"X-GitHub-Api-Version" = "2022-11-28" 8 | ], 9 | Source = Json.Document(Web.Contents(url, [Headers=headers])), -------------------------------------------------------------------------------- /queries/enterprise_usage.pq: -------------------------------------------------------------------------------- 1 | let 2 | // Replace and with your actual token and enterprise name. 3 | url = "https://api.github.com/enterprises//copilot/usage", 4 | headers = [ 5 | #"Accept" = "application/vnd.github+json", 6 | #"Authorization" = "Bearer ", 7 | #"X-GitHub-Api-Version" = "2022-11-28" 8 | ], 9 | Source = Json.Document(Web.Contents(url, [Headers=headers])) 10 | in 11 | Source -------------------------------------------------------------------------------- /queries/enterprise_team_metrics.pq: -------------------------------------------------------------------------------- 1 | let 2 | // Replace , and with your actual token and enterprise name. 3 | url = "https://api.github.com/enterprises//team//copilot/metrics", 4 | headers = [ 5 | #"Accept" = "application/vnd.github+json", 6 | #"Authorization" = "Bearer ", 7 | #"X-GitHub-Api-Version" = "2022-11-28" 8 | ], 9 | Source = Json.Document(Web.Contents(url, [Headers=headers])), -------------------------------------------------------------------------------- /.github/SUPPORT.md: -------------------------------------------------------------------------------- 1 | 2 | # Support 3 | 4 | ## How to file issues and get help 5 | 6 | This project uses GitHub issues to track bugs and feature requests. Please search the existing issues before filing new issues to avoid duplicates. For new issues, file your bug or feature request as a new issue. 7 | 8 | The project is maintained by GitHub staff and the community. We will do our best to respond to support and community questions in a timely manner. 9 | 10 | ## GitHub Support Policy 11 | 12 | Support for this project is limited to the resources listed above. -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright GitHub 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /publish/azure-pipelines.yml: -------------------------------------------------------------------------------- 1 | trigger: 2 | - main 3 | 4 | pool: 5 | vmImage: 'ubuntu-latest' 6 | 7 | steps: 8 | # Install Power BI Actions 9 | - task: PowerShell@2 10 | inputs: 11 | targetType: 'inline' 12 | script: | 13 | Install-Module -Name MicrosoftPowerBIMgmt -Scope CurrentUser -Force 14 | Install-Module -Name MicrosoftPowerBIMgmt.Profile -Scope CurrentUser -Force 15 | Install-Module -Name MicrosoftPowerBIMgmt.Reports -Scope CurrentUser -Force 16 | displayName: 'Install Power BI Actions' 17 | 18 | # Deploy PBIX file using PowerShell with Service Principal Authentication 19 | - task: PowerShell@2 20 | inputs: 21 | targetType: 'inline' 22 | script: | 23 | # Authenticate with Power BI Service using Service Principal 24 | $tenantId = "$(TenantId)" 25 | $appId = "$(AppId)" 26 | $appSecret = "$(AppSecret)" 27 | $secureAppSecret = ConvertTo-SecureString -String $appSecret -AsPlainText -Force 28 | $credential = New-Object System.Management.Automation.PSCredential($appId, $secureAppSecret) 29 | Connect-PowerBIServiceAccount -ServicePrincipal -Credential $credential -Tenant $tenantId 30 | 31 | # Ensure the MicrosoftPowerBIMgmt.Reports module is imported 32 | Import-Module MicrosoftPowerBIMgmt.Reports 33 | 34 | # Publish PBIX file 35 | New-PowerBIReport -Path '../samples/GitHubCopilotTelemetrySample.pbix' -WorkspaceId "$(WorkspaceId)" -Name 'copilot_metrics_usage_sample' 36 | 37 | # Note: Ensure that the necessary modules and credentials are correctly set up for this script to work. 38 | displayName: 'Deploy PBIX file to Power BI Workspace using PowerShell' -------------------------------------------------------------------------------- /.github/SECURITY.md: -------------------------------------------------------------------------------- 1 | Thanks for helping make GitHub safe for everyone. 2 | 3 | # Security 4 | 5 | GitHub takes the security of our software products and services seriously, including all of the open source code repositories managed through our GitHub organizations, such as [GitHub](https://github.com/GitHub). 6 | 7 | Even though [open source repositories are outside of the scope of our bug bounty program](https://bounty.github.com/index.html#scope) and therefore not eligible for bounty rewards, we will ensure that your finding gets passed along to the appropriate maintainers for remediation. 8 | 9 | ## Reporting Security Issues 10 | 11 | If you believe you have found a security vulnerability in any GitHub-owned repository, please report it to us through coordinated disclosure. 12 | 13 | **Please do not report security vulnerabilities through public GitHub issues, discussions, or pull requests.** 14 | 15 | Instead, please send an email to opensource-security[@]github.com. 16 | 17 | Please include as much of the information listed below as you can to help us better understand and resolve the issue: 18 | 19 | * The type of issue (e.g., buffer overflow, SQL injection, or cross-site scripting) 20 | * Full paths of source file(s) related to the manifestation of the issue 21 | * The location of the affected source code (tag/branch/commit or direct URL) 22 | * Any special configuration required to reproduce the issue 23 | * Step-by-step instructions to reproduce the issue 24 | * Proof-of-concept or exploit code (if possible) 25 | * Impact of the issue, including how an attacker might exploit the issue 26 | 27 | This information will help us triage your report more quickly. 28 | 29 | ## Policy 30 | 31 | See [GitHub's Safe Harbor Policy](https://docs.github.com/en/site-policy/security-policies/github-bug-bounty-program-legal-safe-harbor#1-safe-harbor-terms) -------------------------------------------------------------------------------- /.github/CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | ## Contributing 2 | 3 | Hi there! We're thrilled that you'd like to contribute to this project. Your help is essential for keeping it great. 4 | 5 | Contributions to this project are [released](https://docs.github.com/en/site-policy/github-terms/github-terms-of-service#6-contributions-under-repository-license) to the public under the [project's open source license](LICENSE.txt). 6 | 7 | Please note that this project is released with a [Contributor Code of Conduct](CODE_OF_CONDUCT.md). By participating in this project you agree to abide by its terms. 8 | 9 | 10 | ## Pre-requirements 11 | 12 | Before you start contributing to this project, please ensure that you have the following: 13 | 14 | - [PowerBI Report Builder](https://www.microsoft.com/en-us/power-platform/products/power-bi/downloads) installed on your machine. 15 | - Access to the [GitHub Copilot Metrics API](https://docs.github.com/en/rest/copilot?apiVersion=2022-11-28) within your organization. 16 | 17 | ## Submitting a pull request 18 | 19 | 1. [Fork][fork] and clone the repository 20 | 1. Create a new branch: `git checkout -b my-branch-name` 21 | 1. Make your change, add tests, and make sure the tests still pass 22 | 1. Push to your fork and [submit a pull request][pr] 23 | 1. Pat yourself on the back and wait for your pull request to be reviewed and merged. 24 | 25 | Here are a few things you can do that will increase the likelihood of your pull request being accepted: 26 | 27 | - Follow the [style guide][style]. 28 | - Write tests. 29 | - Keep your change as focused as possible. If there are multiple changes you would like to make that are not dependent upon each other, consider submitting them as separate pull requests. 30 | - Write a [good commit message](http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html). 31 | 32 | ## Resources 33 | 34 | - [How to Contribute to Open Source](https://opensource.guide/how-to-contribute/) 35 | - [Using Pull Requests](https://help.github.com/articles/about-pull-requests/) 36 | - [GitHub Help](https://help.github.com) -------------------------------------------------------------------------------- /.github/CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as 6 | contributors and maintainers pledge to making participation in our project and 7 | our community a harassment-free experience for everyone, regardless of age, body 8 | size, disability, ethnicity, gender identity and expression, level of experience, 9 | nationality, personal appearance, race, religion, or sexual identity and 10 | orientation. 11 | 12 | ## Our Standards 13 | 14 | Examples of behavior that contributes to creating a positive environment 15 | include: 16 | 17 | * Using welcoming and inclusive language 18 | * Being respectful of differing viewpoints and experiences 19 | * Gracefully accepting constructive criticism 20 | * Focusing on what is best for the community 21 | * Showing empathy towards other community members 22 | 23 | Examples of unacceptable behavior by participants include: 24 | 25 | * The use of sexualized language or imagery and unwelcome sexual attention or 26 | advances 27 | * Trolling, insulting/derogatory comments, and personal or political attacks 28 | * Public or private harassment 29 | * Publishing others' private information, such as a physical or electronic 30 | address, without explicit permission 31 | * Other conduct which could reasonably be considered inappropriate in a 32 | professional setting 33 | 34 | ## Our Responsibilities 35 | 36 | Project maintainers are responsible for clarifying the standards of acceptable 37 | behavior and are expected to take appropriate and fair corrective action in 38 | response to any instances of unacceptable behavior. 39 | 40 | Project maintainers have the right and responsibility to remove, edit, or 41 | reject comments, commits, code, wiki edits, issues, and other contributions 42 | that are not aligned to this Code of Conduct, or to ban temporarily or 43 | permanently any contributor for other behaviors that they deem inappropriate, 44 | threatening, offensive, or harmful. 45 | 46 | ## Scope 47 | 48 | This Code of Conduct applies both within project spaces and in public spaces 49 | when an individual is representing the project or its community. Examples of 50 | representing a project or community include using an official project e-mail 51 | address, posting via an official social media account, or acting as an appointed 52 | representative at an online or offline event. Representation of a project may be 53 | further defined and clarified by project maintainers. 54 | 55 | ## Enforcement 56 | 57 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 58 | reported by contacting the project team at . All 59 | complaints will be reviewed and investigated and will result in a response that 60 | is deemed necessary and appropriate to the circumstances. The project team is 61 | obligated to maintain confidentiality with regard to the reporter of an incident. 62 | Further details of specific enforcement policies may be posted separately. 63 | 64 | Project maintainers who do not follow or enforce the Code of Conduct in good 65 | faith may face temporary or permanent repercussions as determined by other 66 | members of the project's leadership. 67 | 68 | ## Attribution 69 | 70 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, 71 | available at [http://contributor-covenant.org/version/1/4][version] 72 | 73 | [homepage]: http://contributor-covenant.org 74 | [version]: http://contributor-covenant.org/version/1/4/ -------------------------------------------------------------------------------- /USAGE_API.md: -------------------------------------------------------------------------------- 1 | # GitHub Copilot Metrics Viewer for Power BI 2 | 3 | > **Note:** The legacy GitHub Copilot Usage API is deprecated. Please refer to the new [GitHub Copilot Metrics API](https://github.com/orgs/community/discussions/141071). 4 | 5 | Located in the `./samples` directory you'll find sample JSON and PBIX files used to create the dashboard below. 6 | 7 | ![Image of a Power BI dashboard with GitHuub Copilot Metrics API data displayed.](./assets/Sample_PBI.png) 8 | 9 | ## Setup 10 | ### Connect to Metrics API 11 | > Notes: The REST API provides metrics for the previous 28 days and is refreshed daily with data from the previous day. This is currently in beta, so please ensure you are using the latest version of the [REST API](https://docs.github.com/en/rest/copilot/copilot-usage). 12 | 13 | In order to connect we'll need to generate a token and link to your metrics data: 14 | 1. Download and open the sample `GitHub Copilot - Telemetry Sample (DM).pbix` file. 15 | 2. Determine if you'll be using the `Enterprise` or `Organization` URL. 16 | 3. Follow the instructions below to generate a token with permissions to access the API: 17 | [REST API endpoints for GitHub Copilot usage metrics - GitHub Docs](https://docs.github.com/en/rest/copilot/copilot-usage) 18 | >**IMPORTANT: Do not share this token and ensure you follow you organizations security policies.** 19 | 4. Open the **Power Query Editor** by right clicking the `GH Copilot - Details` and selecting **Edit query**. 20 | 5. Select **Advanced editor**. 21 | ![Image of Power Query Advanced Editor.](./assets/Advanced_editor.png) 22 | 6. Replace the first 2 lines with following, ensure to replace and with the values from step 1 and 2. 23 | 24 | **Organization** 25 | ```powerquery 26 | let 27 | // Replace and with your actual token and org name. 28 | url = "https://api.github.com/orgs//copilot/usage", 29 | headers = [ 30 | #"Accept" = "application/vnd.github+json", 31 | #"Authorization" = "Bearer ", 32 | #"X-GitHub-Api-Version" = "2022-11-28" 33 | ], 34 | Source = Json.Document(Web.Contents(url, [Headers=headers])), 35 | ``` 36 | **Enterprise** 37 | ```powerquery 38 | let 39 | // Replace and with your actual token and enterprise name. 40 | url = "https://api.github.com/enterprises//copilot/usage", 41 | headers = [ 42 | #"Accept" = "application/vnd.github+json", 43 | #"Authorization" = "Bearer ", 44 | #"X-GitHub-Api-Version" = "2022-11-28" 45 | ], 46 | Source = Json.Document(Web.Contents(url, [Headers=headers])), 47 | ``` 48 | 7. Your Power Query will look something like this: 49 | ![Image of Power Query Advanced Editor.](./assets/Advanced_editor_query.png) 50 | 8. Click **OK** to close the editor and select `Anonymous` authentication if prompted. 51 | 9. Repeat steps 4 and 8 for the `GH Copilot - Summary` **Source**. 52 | 10. Click **Close and Apply** in the top-left of the **Power Query Editor**. 53 | 11. On the **Report View** page click **Refresh** to load the new data into your dashboard. 54 | 55 | ### Modify to use a local JSON file 56 | > Note: This example provided a proof of concept for loading metrics data and requires an exported JSON file. If you have access to the REST API you can configure the **Source** accordingly. 57 | 58 | 1. Download and open the sample `GitHub Copilot - Telemetry Sample (Usage).pbix` file. 59 | 2. The file contains three data sources on the right hand side. 60 | 61 | | Name | Description | 62 | | :-------------------- | :----------------------------------------------------- | 63 | | GH Copilot - Details | Detailed breakdown of metrics data by language and editor. | 64 | | GH Copilot - Summary | Daily summary of metrics data. | 65 | | Last Refresh | Used to display the data time of data refresh on the top-right corner of the dashboard. | 66 | 67 | 3. Open the **Power Query Editor** by right clicking the `GH Copilot - Details` and selecting **Edit query**. 68 | 4. Modify the **Source** step by clicking the settings icon, selecting your JSON file and clicking **OK**. 69 | ![Image of a data source selector in Power Query Editor.](./assets/Modify_JSON_source.png) 70 | 6. Repeat steps 3 and 4 for the `GH Copilot - Summary` **Source**. 71 | 7. Click **Close and Apply** in the top-left of the **Power Query Editor**. 72 | 8. On the **Report View** page click **Refresh** to load the new data into your dashboard. 73 | 9. **Happy Customizing!** 74 | 75 | ## Publishing 76 | If you need help deploying or publishing this script, please see: [Publish README](/publish/README.md) 77 | 78 | ## Maintainers 79 | 80 | @jasonmoodie, @Eldrick19 81 | 82 | ## Support 83 | 84 | These are just files for you to download and use as you see fit. If you have questions about how to use them, please reach out to the maintainers, but we cannot guarantee a response with SLAs. -------------------------------------------------------------------------------- /samples/copilot_metrics_response_sample_small.json: -------------------------------------------------------------------------------- 1 | [ 2 | { 3 | "date": "2024-06-24", 4 | "total_active_users": 24, 5 | "total_engaged_users": 20, 6 | "copilot_ide_code_completions": { 7 | "total_engaged_users": 20, 8 | "languages": [ 9 | { 10 | "name": "python", 11 | "total_engaged_users": 10 12 | }, 13 | { 14 | "name": "ruby", 15 | "total_engaged_users": 10 16 | } 17 | ], 18 | "editors": [ 19 | { 20 | "name": "vscode", 21 | "total_engaged_users": 13, 22 | "models": [ 23 | { 24 | "name": "default", 25 | "is_custom_model": false, 26 | "custom_model_training_date": null, 27 | "total_engaged_users": 13, 28 | "languages": [ 29 | { 30 | "name": "python", 31 | "total_engaged_users": 6, 32 | "total_code_suggestions": 249, 33 | "total_code_acceptances": 123, 34 | "total_code_lines_suggested": 225, 35 | "total_code_lines_accepted": 135 36 | }, 37 | { 38 | "name": "ruby", 39 | "total_engaged_users": 7, 40 | "total_code_suggestions": 496, 41 | "total_code_acceptances": 253, 42 | "total_code_lines_suggested": 520, 43 | "total_code_lines_accepted": 270 44 | } 45 | ] 46 | } 47 | ] 48 | }, 49 | { 50 | "name": "neovim", 51 | "total_engaged_users": 7, 52 | "models": [ 53 | { 54 | "name": "a-custom-model", 55 | "is_custom_model": true, 56 | "custom_model_training_date": "2024-02-01", 57 | "languages": [ 58 | { 59 | "name": "typescript", 60 | "total_engaged_users": 3, 61 | "total_code_suggestions": 112, 62 | "total_code_acceptances": 56, 63 | "total_code_lines_suggested": 143, 64 | "total_code_lines_accepted": 61 65 | }, 66 | { 67 | "name": "go", 68 | "total_engaged_users": 4, 69 | "total_code_suggestions": 132, 70 | "total_code_acceptances": 67, 71 | "total_code_lines_suggested": 154, 72 | "total_code_lines_accepted": 72 73 | } 74 | ] 75 | } 76 | ] 77 | } 78 | ] 79 | }, 80 | "copilot_ide_chat": { 81 | "total_engaged_users": 13, 82 | "editors": [ 83 | { 84 | "name": "vscode", 85 | "total_engaged_users": 13, 86 | "models": [ 87 | { 88 | "name": "default", 89 | "is_custom_model": false, 90 | "custom_model_training_date": null, 91 | "total_engaged_users": 12, 92 | "total_chats": 45, 93 | "total_chat_insertion_events": 12, 94 | "total_chat_copy_events": 16 95 | }, 96 | { 97 | "name": "a-custom-model", 98 | "is_custom_model": true, 99 | "custom_model_training_date": "2024-02-01", 100 | "total_engaged_users": 1, 101 | "total_chats": 10, 102 | "total_chat_insertion_events": 11, 103 | "total_chat_copy_events": 3 104 | } 105 | ] 106 | } 107 | ] 108 | }, 109 | "copilot_dotcom_chat": { 110 | "total_engaged_users": 14, 111 | "models": [ 112 | { 113 | "name": "default", 114 | "is_custom_model": false, 115 | "custom_model_training_date": null, 116 | "total_engaged_users": 14, 117 | "total_chats": 38 118 | } 119 | ] 120 | }, 121 | "copilot_dotcom_pull_requests": { 122 | "total_engaged_users": 12, 123 | "repositories": [ 124 | { 125 | "name": "demo/repo1", 126 | "total_engaged_users": 8, 127 | "models": [ 128 | { 129 | "name": "default", 130 | "is_custom_model": false, 131 | "custom_model_training_date": null, 132 | "total_pr_summaries_created": 6, 133 | "total_engaged_users": 8 134 | } 135 | ] 136 | }, 137 | { 138 | "name": "demo/repo2", 139 | "total_engaged_users": 4, 140 | "models": [ 141 | { 142 | "name": "a-custom-model", 143 | "is_custom_model": true, 144 | "custom_model_training_date": "2024-02-01", 145 | "total_pr_summaries_created": 10, 146 | "total_engaged_users": 4 147 | } 148 | ] 149 | } 150 | ] 151 | } 152 | } 153 | ] -------------------------------------------------------------------------------- /publish/README.md: -------------------------------------------------------------------------------- 1 | # Publishing your PBI Report 2 | 3 | The report can be published many different ways. Publishing from PowerBI Desktop is likely the fastest. For a more automated version which would include report version, you should use a repository. Below are the instructions on how to deploy your report in Azure DevOps repos, into your Power BI Workspace using a Azure DevOps Pipeline. 4 | 5 | ### Installing the ADO Extension 6 | 7 | Install the extension 8 | These instructions will show you how to install the Power BI Action extension that we are going to use for this tutorial, 9 | 10 | 1. Sign in to your Azure DevOps organization. 11 | 1. Go to Organization Settings. 12 | 1. Select Extensions. 13 | 1. Click on Browse Marketplace at the top right. 14 | 1. Search for and install `Power BI Actions`. [Power BI Actions Extension!](https://marketplace.visualstudio.com/items?itemName=maikvandergaag.maikvandergaag-power-bi-actions) 15 | Screenshot 2024-07-17 at 8 19 15 AM 16 | 17 | 18 | ### Get the PBI Workspace ID. Open Powershell in Administrative Mode and run: 19 | 1. Install Power BI Management module 20 | `Install-Module -Name MicrosoftPowerBIMgmt -Scope CurrentUser -Force` 21 | 22 | 1. Sign in to Power BI 23 | `Login-PowerBI` 24 | 25 | 3. List all workspaces 26 | `Get-PowerBIWorkspace -Scope Organization` 27 | 28 | Write down the Workpace ID and save it for later 29 | 30 | ### Create the App Registration 31 | 1. Go to Entra AD 32 | Screenshot 2024-07-17 at 8 04 48 AM 33 | 34 | 1. Set `API Permissions -> Add a permission` 35 | 1. Power BI Service 36 | 1. Chose Delegated permissions 37 | 1. Chose: 38 | - Tenant.Read.All 39 | - Tenant.ReadWrite.All 40 | - Report.ReadWrite.All 41 | Screenshot 2024-07-17 at 8 06 50 AM 42 | 43 | ### Power BI 44 | 45 | Setting up the tenant 46 | The next step is to configure your Power BI, 47 | 48 | 1. Sign in to the Power BI portal. 49 | 1. Click the gear icon on the top right and select Admin portal. 50 | 1. Select the Tenant settings and scroll down to Developer Settings and allow service principals to use Power BI APIs 51 | Screenshot 2024-07-17 at 7 53 01 AM 52 | 53 | ### Configure a workspace 54 | Now create the workspace where your pipeline will publish the reports and grant permission to your Azure AD App to do this, 55 | 56 | 1. Select Workspaces tab. 57 | 1. Click the three vertical dots on the right of your new workspace. 58 | 1. Click Manage access. 59 | Screenshot 2024-07-17 at 8 02 30 AM 60 | 61 | 4. Search the app that you have previously registered in Azure AD and grant it the permission of **Admin**. 62 | Screenshot 2024-07-17 at 7 58 15 AM 63 | 64 | ### Create a new service connection 65 | 1. Go to Project Settings. 66 | 1. Select Service connections. 67 | 1. Click the New service connection button and select Power BI Service Connection. 68 | Screenshot 2024-07-17 at 8 11 27 AM 69 | 70 | 4. Fill in the parameters for the service connection and allow all pipelines to use this connection option. Be sure to pick the type of _Service Principal_ 71 | 1. Click OK to create the connection. 72 | Screenshot 2024-07-17 at 8 14 41 AM 73 | 74 | 75 | ### Creating the Build Pipeline 76 | It is time to create your build pipeline. 77 | 78 | 1. From the dashboard, select Pipelines. 79 | 1. Click the New pipeline button. 80 | 1. Select Azure Git Repos as a source that will trigger our pipeline. 81 | 1. Azure DevOps will suggest several templates, select the Starter Pipeline, to begin with a bare bone pipeline. 82 | 1. Create 4 variables for: 83 | - AppId 84 | - AppSecret 85 | - TenantId 86 | - WorkspaceId 87 | Screenshot 2024-07-17 at 8 16 38 AM 88 | 89 | 90 | 91 | 1. Add the following YAML snippet to your pipeline 92 | ``` yaml 93 | trigger: 94 | - main 95 | 96 | pool: 97 | vmImage: 'ubuntu-latest' 98 | 99 | steps: 100 | Install Power BI Actions 101 | - task: PowerShell@2 102 | inputs: 103 | targetType: 'inline' 104 | script: | 105 | Install-Module -Name MicrosoftPowerBIMgmt -Scope CurrentUser -Force 106 | Install-Module -Name MicrosoftPowerBIMgmt.Profile -Scope CurrentUser -Force 107 | Install-Module -Name MicrosoftPowerBIMgmt.Reports -Scope CurrentUser -Force 108 | displayName: 'Install Power BI Actions' 109 | 110 | Deploy PBIX file using PowerShell with Service Principal Authentication 111 | - task: PowerShell@2 112 | inputs: 113 | targetType: 'inline' 114 | script: | 115 | # Authenticate with Power BI Service using Service Principal 116 | $tenantId = "$(TenantId)" 117 | $appId = "$(AppId)" 118 | $appSecret = "$(AppSecret)" 119 | $secureAppSecret = ConvertTo-SecureString -String $appSecret -AsPlainText -Force 120 | $credential = New-Object System.Management.Automation.PSCredential($appId, $secureAppSecret) 121 | Connect-PowerBIServiceAccount -ServicePrincipal -Credential $credential -Tenant $tenantId 122 | 123 | # Ensure the MicrosoftPowerBIMgmt.Reports module is imported 124 | Import-Module MicrosoftPowerBIMgmt.Reports 125 | 126 | # Publish PBIX file 127 | New-PowerBIReport -Path './samples/GitHubCopilotTelemetrySample.pbix' -WorkspaceId "$(WorkspaceId)" -Name 'copilot_metrics_usage_sample' 128 | 129 | # Note: Ensure that the necessary modules and credentials are correctly set up for this script to work. 130 | displayName: 'Deploy PBIX file to Power BI Workspace using PowerShell' 131 | ``` 132 | The remaining step is to save, check-in, and then watch it automatically deploy as the trigger is set to monitor the main branch 133 | 134 | 135 | 136 | 137 | 138 | 139 | 140 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # GitHub Copilot Metrics Viewer for Power BI 2 | 3 | With the release of the latest [GitHub Copilot Metrics API](https://github.com/orgs/community/discussions/141071) many teams are looking to leverage this data to help monitor usage against their KPIs. For some, the Copilot Metrics Viewer ([github-copilot-resources/copilot-metrics-viewer](https://github.com/github-copilot-resources/copilot-metrics-viewer)) might be a great option. 4 | 5 | However, many organizations that we work with already have established Power BI teams. If your organization is **already using Power BI, please read on!** 6 | 7 | ## Table of Contents 8 | - [New Metrics API Features](#new-features) 9 | - [Metrics Dashboard Setup](#metrics-dashboard-setup) 10 | - [Connect to Metrics API](#connect-to-metrics-api) 11 | - [Connect to local JSON data source](#connect-to-local-json-data-source) 12 | - [Collect Feedback on GitHub Copilot Usage](#collect-feedback-on-github-copilot-usage) 13 | - [KPI - Savings Dashboard](#kpi---savings-dashboard) 14 | - [Publishing](#publishing) 15 | - [Maintainers](#maintainers) 16 | - [Support](#support) 17 | 18 | ## New Metrics API Features 19 | - New metrics for Pull Request summaries and Copilot Chat on GitHub.com. 20 | - Clarity on code completions and Copilot Chat usage in IDEs. 21 | - User engagement summaries updated daily. 22 | - Custom model slicing support, coming soon! 23 | - Data can be aggregated by enterprise, organization or team. 24 | 25 | Located in the `./samples` directory you'll find sample JSON and PBIX files used to create the dashboard below. 26 | > Note: The legacy GitHub Copilot Usage API is still available and can be used with the `GitHub Copilot - Telemetry Sample (Usage).pbix` file. For instructions, see [Usage_API.md](USAGE_API.md). 27 | 28 | ![Image of the Power BI dashboard with sample GitHub Copilot Metrics API data displayed.](./assets/Sample_Metrics_PBI.png) 29 | 30 | ## Metrics Dashboard Setup 31 | 32 | ### Connect to Metrics API 33 | > Notes: The REST API provides metrics for the previous 28 days and is refreshed daily with data from the previous day. Please ensure you are using the latest version of the [REST API](https://docs.github.com/en/enterprise-cloud@latest/rest/copilot/copilot-metrics). 34 | 35 | In order to connect we'll need to generate a token and link to your metrics data: 36 | 1. Download and open the sample `GitHub Copilot - Telemetry Sample (Metrics with KPI).pbix` file. 37 | 2. Determine if you'll be using the `Enterprise`, `Organization` or `Team` URL. 38 | 3. Follow the instructions below to generate a token with permissions to access the API: 39 | [REST API endpoints for Copilot metrics - GitHub Enterprise Cloud Docs](https://docs.github.com/en/enterprise-cloud@latest/rest/copilot/copilot-metrics) 40 | >**IMPORTANT: Do not share this token and ensure you follow you organizations security policies.** 41 | 4. Ensure Copilot Metrics API access is enabled for your [organization](https://docs.github.com/en/enterprise-cloud@latest/copilot/managing-copilot/managing-github-copilot-in-your-organization/managing-policies-for-copilot-in-your-organization) or [enterprise](https://docs.github.com/en/enterprise-cloud@latest/copilot/managing-copilot/managing-copilot-for-your-enterprise/managing-policies-and-features-for-copilot-in-your-enterprise#configuring-policies-for-github-copilot). 42 | 5. The file contains the following data sources, descriptions are inlcuded below. 43 | 44 | | Name | Description | 45 | | :--------------------------------- | :------------------------------------------------------------ | 46 | | config | Configuration used to display date of refresh and KPI dashboard | 47 | | source | Base source used to connect to API or local JSON files. | 48 | | GH Copilot - dotcom chat | Detailed metrics of code completions using chat on the web. | 49 | | GH Copilot - ide chat | Detailed metrics of code completions using chat within the IDE. | 50 | | GH Copilot - ide code completions editors | Detailed metrics of code completions within the IDE. | 51 | | GH Copilot - ide code completions languages | Detailed metrics of engaged users by language within the IDE. | 52 | | GH Copilot - pull requests | Detailed metrics of pull requests. | 53 | | GH Copilot - summary | Daily summary of active and engaged users. | 54 | 5. Open the **Power Query Editor** by clicking **Transform data** in the top-menu and selecting **Tranform data**. 55 | 6. Click on **source** in the left-menu. 56 | 7. Select **Advanced editor**. 57 | ![Image of Power Query Advanced Editor.](./assets/Advanced_editor.png) 58 | 8. Replace the first 2 lines with following, ensure to replace `` along with the relevant ``, `` and `` with the values from step 2. 59 | 60 | **Enterprise** 61 | ```powerquery 62 | let 63 | // Replace and with your actual token and enterprise name. 64 | url = "https://api.github.com/enterprises//copilot/metrics", 65 | headers = [ 66 | #"Accept" = "application/vnd.github+json", 67 | #"Authorization" = "Bearer ", 68 | #"X-GitHub-Api-Version" = "2022-11-28" 69 | ], 70 | Source = Json.Document(Web.Contents(url, [Headers=headers])), 71 | ``` 72 | 73 | **Enterprise Team** 74 | ```powerquery 75 | let 76 | // Replace , and with your actual token and enterprise name. 77 | url = "https://api.github.com/enterprises//team//copilot/metrics", 78 | headers = [ 79 | #"Accept" = "application/vnd.github+json", 80 | #"Authorization" = "Bearer ", 81 | #"X-GitHub-Api-Version" = "2022-11-28" 82 | ], 83 | Source = Json.Document(Web.Contents(url, [Headers=headers])), 84 | ``` 85 | 86 | **Organization** 87 | ```powerquery 88 | let 89 | // Replace and with your actual token and org name. 90 | url = "https://api.github.com/orgs//copilot/metrics", 91 | headers = [ 92 | #"Accept" = "application/vnd.github+json", 93 | #"Authorization" = "Bearer ", 94 | #"X-GitHub-Api-Version" = "2022-11-28" 95 | ], 96 | Source = Json.Document(Web.Contents(url, [Headers=headers])), 97 | ``` 98 | 99 | **Team** 100 | ```powerquery 101 | let 102 | // Replace , and with your actual token and enterprise name. 103 | url = "https://api.github.com/orgs//team//copilot/metrics", 104 | headers = [ 105 | #"Accept" = "application/vnd.github+json", 106 | #"Authorization" = "Bearer ", 107 | #"X-GitHub-Api-Version" = "2022-11-28" 108 | ], 109 | Source = Json.Document(Web.Contents(url, [Headers=headers])), 110 | ``` 111 | 8. Your Power Query will look something like this: 112 | ![Image of Power Query Advanced Editor.](./assets/Advanced_editor_metrics_query.png) 113 | 9. Click **OK** to close the editor and select `Anonymous` authentication if prompted. 114 | 10. Click **Close and Apply** in the top-left of the **Power Query Editor**. 115 | 11. On the **Report View** page click **Refresh** to load the new data into your dashboard. 116 | 117 | ### Connect to local JSON data source 118 | > Note: This example provided a proof of concept for loading metrics data and requires an exported JSON file. If you have access to the REST API you can configure the **Source** accordingly. 119 | 120 | 1. Download and open the sample `GitHub Copilot - Telemetry Sample (Metrics with KPIs).pbix` file. 121 | 2. Open the **Power Query Editor** by clicking **Transform data** in the top-menu and selecting **Tranform data**. 122 | 3. Click on **source** query in the left-menu 123 | 4. In the right-menu under **APPLIED STEPS** click the gear (settings) icon, selecting your JSON file and clicking **OK**. 124 | ![Image of a data source selector in Power Query Editor.](./assets/Modify_JSON_source.png) 125 | 5. Click **Close and Apply** in the top-left of the **Power Query Editor**. 126 | 6. On the **Report View** page click **Refresh** to load the new data into your dashboard. 127 | 7. **Happy Customizing!** 128 | 129 | ## Collect Feedback on GitHub Copilot Usage 130 | 131 | To better understand how GitHub Copilot is being used in your organization and to help determine accurate time and cost savings, we recommend collecting direct feedback from your developers. This information can be used to refine the values in the KPI dashboard and ensure your savings estimates reflect real-world usage. 132 | 133 | You can use the following **Microsoft Forms** template to gather feedback. Simply duplicate the form and customize it for your organization: 134 | 135 | [GitHub Copilot Usage Feedback – Microsoft Forms Template](https://forms.office.com/Pages/ShareFormPage.aspx?id=v4j5cvGGr0GRqy180BHbR6zql0pB1xhIi5wwWWSq6RVUQ0JQSkZOMElYOFdHWUFWWVhPRllTQ1ZRUi4u&sharetoken=Gb49retb5qghvCQiQILO) 136 | 137 | Gathering this feedback will help you: 138 | - Validate or adjust the average weekly hour savings. 139 | - Understand adoption and satisfaction. 140 | - Support your ROI calculations with real user data. 141 | 142 | Feel free to adapt the form to include any additional questions relevant to your team. 143 | 144 | ## KPI - Savings Dashboard 145 | A new KPI tab has been added to the dashboard to help you estimate savings. The KPI tab is configured to display the potential time and cost savings. You can configure the KPI tab to display these details by modifying the following fields in the `config` data source from the **Table view**: 146 | 147 | | Name | Description | 148 | | :------------------------- | :----------------------------------------------- | 149 | | total_devs | Total number of developers at your organization. | 150 | | avg_hourly_salary | Average hourly salary of developers. | 151 | | annual_work_weeks | Total number of work weeks in a year. | 152 | | average_weekly_hour_savings| Average number of hours developers saved per week. The default is 3.5 hours and assumed a 10% time saving, but this can be updated based on customer survey data or other measurements. | 153 | 154 | These values can be modified in the `config` data source below: 155 | ![Image of the KPI config table in the Power BI.](./assets/KPI_config.png) 156 | 157 | Once configured, the KPI dashboard will display this potential savings against current usage pulled from the Metrics API: 158 | ![Image of the KPI tab in the Power BI.](./assets/Sample_KPI_PBI.png) 159 | 160 | 161 | ## Publishing 162 | If you need help deploying or publishing this script, please see: [Publish README](/publish/README.md) 163 | 164 | ## Maintainers 165 | 166 | @jasonmoodie, @Eldrick19 167 | 168 | ## Support 169 | 170 | These are just files for you to download and use as you see fit. If you have questions about how to use them, please reach out to the maintainers, but we cannot guarantee a response with SLAs. --------------------------------------------------------------------------------