├── .github ├── ISSUE_TEMPLATE │ ├── bug-report---.md │ ├── custom-issue---.md │ └── feature-request---.md └── PULL_REQUEST_TEMPLATE.md ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── Dockerfile ├── LICENSE ├── README.md ├── action.yml ├── docs ├── ARG_SPEC.md ├── SERVICE_ACCOUNT.md ├── SIMPLE_USAGE.md └── assets │ └── preview.png └── entrypoint.sh /.github/ISSUE_TEMPLATE/bug-report---.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: "Bug report \U0001F41B" 3 | about: Create a report to help us improve 4 | title: '' 5 | labels: bug 6 | assignees: asadmansr 7 | 8 | --- 9 | 10 | **Describe the bug** 11 | A clear and concise description of what the bug is. 12 | 13 | **To Reproduce** 14 | Steps to reproduce the behavior (example below): 15 | 1. Go to '...' 16 | 2. Click on '....' 17 | 3. Scroll down to '....' 18 | 4. See error 19 | 20 | **Expected behavior** 21 | A clear and concise description of what you expected to happen. 22 | 23 | **Screenshots** 24 | If applicable, add screenshots to help explain your problem. 25 | 26 | **Access to the following pieces of information (if you can):** 27 | - Log output 28 | - GitHub Workflow yml file 29 | 30 | **Additional context** 31 | Add any other context about the problem here. 32 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/custom-issue---.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: "Custom issue \U0001F914" 3 | about: Questions or concerns 4 | title: '' 5 | labels: '' 6 | assignees: '' 7 | 8 | --- 9 | 10 | If you would like to reach the maintainers of this project, feel free to open this issue and let us know how we can help. 11 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature-request---.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: "Feature request \U0001F680" 3 | about: Suggest an idea for this project 4 | title: '' 5 | labels: enhancement 6 | assignees: asadmansr 7 | 8 | --- 9 | 10 | **Is your feature request related to a problem? Please describe.** 11 | A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] 12 | 13 | **Describe the solution you'd like** 14 | A clear and concise description of what you want to happen. 15 | 16 | **Describe alternatives you've considered** 17 | A clear and concise description of any alternative solutions or features you've considered. 18 | 19 | **Additional context** 20 | Add any other context or screenshots about the feature request here. 21 | -------------------------------------------------------------------------------- /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | ### Pull Request Checklist 2 | - [ ] I have added proper commit messages 3 | - [ ] I have updated tests and documentation 4 | - [ ] I have assigned the required reviewers and labels 5 | 6 | ### Issue Number 7 | 8 | ``` 9 | 10 | ``` 11 | 12 | ### Short Description of Change 13 | 14 | ``` 15 | 16 | ``` 17 | 18 | ### Motivation and Context 19 | 20 | ``` 21 | 22 | ``` 23 | 24 | ### Testing Steps 25 | 26 | ``` 27 | 28 | ``` 29 | 30 | ### Risks Associated 31 | 32 | ``` 33 | 34 | ``` 35 | 36 | ### Additional Information 37 | 38 | ``` 39 | 40 | ``` -------------------------------------------------------------------------------- /CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as 6 | contributors and maintainers pledge to making participation in our project and 7 | our community a harassment-free experience for everyone, regardless of age, body 8 | size, disability, ethnicity, sex characteristics, gender identity and expression, 9 | level of experience, education, socio-economic status, nationality, personal 10 | appearance, race, religion, or sexual identity and orientation. 11 | 12 | ## Our Standards 13 | 14 | Examples of behavior that contributes to creating a positive environment 15 | include: 16 | 17 | * Using welcoming and inclusive language 18 | * Being respectful of differing viewpoints and experiences 19 | * Gracefully accepting constructive criticism 20 | * Focusing on what is best for the community 21 | * Showing empathy towards other community members 22 | 23 | Examples of unacceptable behavior by participants include: 24 | 25 | * The use of sexualized language or imagery and unwelcome sexual attention or 26 | advances 27 | * Trolling, insulting/derogatory comments, and personal or political attacks 28 | * Public or private harassment 29 | * Publishing others' private information, such as a physical or electronic 30 | address, without explicit permission 31 | * Other conduct which could reasonably be considered inappropriate in a 32 | professional setting 33 | 34 | ## Our Responsibilities 35 | 36 | Project maintainers are responsible for clarifying the standards of acceptable 37 | behavior and are expected to take appropriate and fair corrective action in 38 | response to any instances of unacceptable behavior. 39 | 40 | Project maintainers have the right and responsibility to remove, edit, or 41 | reject comments, commits, code, wiki edits, issues, and other contributions 42 | that are not aligned to this Code of Conduct, or to ban temporarily or 43 | permanently any contributor for other behaviors that they deem inappropriate, 44 | threatening, offensive, or harmful. 45 | 46 | ## Scope 47 | 48 | This Code of Conduct applies both within project spaces and in public spaces 49 | when an individual is representing the project or its community. Examples of 50 | representing a project or community include using an official project e-mail 51 | address, posting via an official social media account, or acting as an appointed 52 | representative at an online or offline event. Representation of a project may be 53 | further defined and clarified by project maintainers. 54 | 55 | ## Enforcement 56 | 57 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 58 | reported by contacting the project team at asadmansr@gmail.com. All 59 | complaints will be reviewed and investigated and will result in a response that 60 | is deemed necessary and appropriate to the circumstances. The project team is 61 | obligated to maintain confidentiality with regard to the reporter of an incident. 62 | Further details of specific enforcement policies may be posted separately. 63 | 64 | Project maintainers who do not follow or enforce the Code of Conduct in good 65 | faith may face temporary or permanent repercussions as determined by other 66 | members of the project's leadership. 67 | 68 | ## Attribution 69 | 70 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, 71 | available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html 72 | 73 | [homepage]: https://www.contributor-covenant.org 74 | 75 | For answers to common questions about this code of conduct, see 76 | https://www.contributor-covenant.org/faq 77 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributing 2 | 3 | Thank you for your interest to contribute! In short, feel free to open any issues or pull requests. I would love to add you as contributors to this repository. 4 | 5 |
6 | 7 | #### **Did you find a bug?** 8 | 9 | * Open a new issue 10 | * Be sure to follow the template provided when creating an issue 11 | 12 |
13 | 14 | #### **Would you like to fix a bug or add new features?** 15 | 16 | * Open a new issue (if issue is not already created) 17 | * Fork the repository and submit a new pull request. 18 | * Be sure to follow the template provided when creating an issue or pull request 19 | 20 |
21 | 22 | #### **Do you have any questions?** 23 | 24 | * If something does not make sense or you have a question, feel free to reach out to us by opening an issue. 25 | 26 |
27 | 28 | Thank you! -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM google/cloud-sdk:283.0.0-alpine 2 | 3 | RUN apk update && apk add --no-cache jq 4 | 5 | COPY entrypoint.sh /entrypoint.sh 6 | 7 | ENTRYPOINT ["/entrypoint.sh"] -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 Asad Mansoor 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Firebase Test Lab GitHub Action 2 | 3 | [![Release](https://img.shields.io/github/release/asadmansr/Firebase-Test-Lab-Action.svg)](https://github.com/asadmansr/Firebase-Test-Lab-Action/releases) 4 | [![Marketplace](https://img.shields.io/badge/GitHub-Marketplace-blue.svg)](https://github.com/marketplace/actions/firebase-test-lab-action) 5 | 6 | A GitHub Action to test mobile applications (Android, iOS) using Firebase Test Lab. 7 | 8 |
9 | 10 | ![](docs/assets/preview.png) 11 | 12 |
13 | 14 | Inspired by the 2020 [GitHub Actions Hackathon.](https://github.community/t5/Events/Featured-Event-GitHub-Actions-Hackathon/td-p/48206) 15 | 16 |
17 | 18 | ## Introduction 19 | 20 | Testing mobile applications can be a challenge. With Firebase Test Lab, testing becomes much easier whether it's validating new changes on a continuous integration (CI) pipeline or tracking down bugs on specific devices. This GitHub Action automates the setup of the gcloud command line tool and provides an easy interface to start testing quickly. 21 | 22 |
23 | 24 | ## Pre-requisites 25 | 26 | 1. `Service Account`: A service account is a special kind of account with specific permissions to authenticate with the Cloud Platform when used on a virtual machine for continuous integration. 27 | 28 | 2. `ARG SPEC File`: A YAML argument file that lists out all of the configurations for Firebase Test Lab. In this file, you can specify the test APK, filter the tests, select virtual or physical devices and indicate the type of test to perform. 29 | 30 | 31 |
32 | 33 | ## Usage 34 | workflows/main.yml: 35 | ``` 36 | name: Android CI 37 | on: [push] 38 | 39 | jobs: 40 | test: 41 | runs-on: ubuntu-latest 42 | steps: 43 | # Check out the repository 44 | - uses: actions/checkout@v2 45 | 46 | # Run the Firebase Test Lab Action 47 | - name: Run tests on Firebase Test Lab 48 | uses: asadmansr/Firebase-Test-Lab-Action@v1.0 49 | with: 50 | arg-spec: 'tests.yml:android-pixel-4' 51 | env: 52 | SERVICE_ACCOUNT: ${{ secrets.SERVICE_ACCOUNT }} 53 | ``` 54 | 55 | tests.yml: 56 | ``` 57 | android-pixel-4: 58 | type: instrumentation 59 | app: app-debug.apk 60 | test: app-debug-test.apk 61 | device: 62 | - model: flame 63 | version: 29 64 | locale: 'en' 65 | orientation: portrait 66 | ``` 67 | 68 |
69 | 70 | The following usage comes with additional instructions regarding the input and environment variables that can be found in the [Simple Usage Documentation](/docs/SIMPLE_USAGE.md). 71 | 72 | Currently, this GitHub Action only runs Android tests. Support for iOS coming soon. 73 | 74 |
75 | 76 | ## Inputs 77 | 78 | #### `arg-spec` 79 | 80 | YAML file that contains configuration for Firebase Test Lab. Format must be ARG_FILE:ARG_GROUP_NAME. **Required** 81 | 82 | #### `SERVICE_ACCOUNT` 83 | 84 | Copy-paste the content of the JSON-formatted service account file in GitHub's secret variables in settings. **Required** 85 | 86 |
87 | 88 | ## Contributing 89 | 90 | Are you facing an issue? Have some questions? Would like to implement a new feature? Learn more about our [contributing guidelines](CONTRIBUTING.md). 91 | 92 |
93 | 94 | ## Licence 95 | 96 | The project is released under the [MIT License](LICENSE). -------------------------------------------------------------------------------- /action.yml: -------------------------------------------------------------------------------- 1 | name: 'Firebase Test Lab Action' 2 | description: 'A GitHub Action to test mobile applications (Android, iOS) using Firebase Test Lab.' 3 | author: 'Asad Mansoor' 4 | inputs: 5 | arg-spec: 6 | description: 'a YAML formatted file that includes arguments for gcloud firebase test run command.' 7 | required: true 8 | runs: 9 | using: 'docker' 10 | image: 'Dockerfile' 11 | args: 12 | - ${{ inputs.arg-spec }} 13 | branding: 14 | icon: 'check-square' 15 | color: 'blue' -------------------------------------------------------------------------------- /docs/ARG_SPEC.md: -------------------------------------------------------------------------------- 1 | # ARG SPEC File 2 | 3 | To run tests on Firebase Test Lab, the `gcloud firebase test android run` command requires arguments to specify what and how to run the following tests. For example, to run instrumentation tests, the following command will need to be executed: 4 | 5 | ``` 6 | gcloud firebase test android run \ 7 | --type instrumentation \ 8 | --app app-debug-unaligned.apk \ 9 | --test app-debug-test-unaligned.apk \ 10 | --device model=Nexus6,version=21,locale=en,orientation=portrait \ 11 | --device model=Nexus7,version=19,locale=fr,orientation=landscape 12 | ``` 13 | 14 | The configuration of these tests can get long and complex. Instead, the GitHub Action will instead the ARG SPEC file as an input which has the predefined configurations of the corresponding tests. Developers will need to create a YAML file specify how they would like to run the tests and make a reference to that file during the execution of the GitHub Action. 15 | 16 |
17 | 18 | ## Example ARG SPEC file 19 | As an example, the configuration for the test are saved in the tests.yml file. A high-level arg-group is specified as `android-pixel-4` to specify that this is an Android test running on Pixel 4 device. However, the file name and arg-group can be named anything. 20 | 21 | ``` 22 | android-pixel-4: 23 | type: instrumentation 24 | app: app-debug.apk 25 | test: app-debug-test.apk 26 | device: 27 | - model: flame 28 | version: 29 29 | locale: 'en' 30 | orientation: portrait 31 | ``` 32 | 33 | In the GitHub Actions workflow, you must specify the `arg-spec` containing the path of this file along with the arg group. 34 | 35 |
36 | 37 | ## Arguments Reference 38 | The following arguments used above are defined here. To see the full list of arguments, please check out the reference links below. 39 | ``` 40 | : 41 | type: 42 | app: 43 | test: 44 | device: 45 | - model: 46 | version: 47 | locale: 48 | orientation: 49 | ``` 50 | 51 |
52 | 53 | ## Reference 54 | - [Firebase Test Lab](https://firebase.google.com/docs/test-lab/android/command-line) 55 | - [gcloud topic arg-file](https://cloud.google.com/sdk/gcloud/reference/topic/arg-files) 56 | - [gcloud firebase test android run](https://cloud.google.com/sdk/gcloud/reference/firebase/test/android/run) 57 | -------------------------------------------------------------------------------- /docs/SERVICE_ACCOUNT.md: -------------------------------------------------------------------------------- 1 | # Service Accounts 2 | 3 | Service accounts are special accounts that can be used in continuous integration pipelines to authenticate with cloud platforms. With service accounts, it is easier to revoke credentials as well as limit permissions to a certain service. It is always best practice to provide the least amount of permissions required to a service account. 4 | 5 | This GitHub Action requires a service account to authenticate with Firebase Test Lab. 6 | 7 |
8 | 9 | ## Generating Service Account 10 | Service account is just a JSON key-value pair that contain information on how to authenticate with the cloud platform. To generate a service account, follow the following documentation provided by Google. 11 | 12 | Visit: https://firebase.google.com/docs/test-lab/android/continuous 13 | 14 | In the requirements section, follow 15 | - step 2: create a service account. Remember to download the JSON-formatted key file. 16 | - step 3: enable required APIs 17 | 18 |
19 | 20 | ## Storing Service Account on GitHub Secrets 21 | Now it is time to store the service account on GitHub secrets, so GitHub Actions has access to it. Open your GitHub repository and follow the following steps: 22 | - Click on settings and select Secrets 23 | - Click on Add a new secret 24 | - The name of the secret **must** be SERVICE_ACCOUNT 25 | - Copy-paste the content the JSON-formatted key file into the value 26 | - Finally, click on Add secret 27 | 28 | -------------------------------------------------------------------------------- /docs/SIMPLE_USAGE.md: -------------------------------------------------------------------------------- 1 | # Simple Usage 2 | The following documents shows a simple tutorial on how to start using this GitHub Action. 3 | 4 |
5 | 6 | ## Step 1 - Define a ARG SPEC 7 | The ARG SPEC file defines what and how to run your tests. In this step, create a YAML file with the required arguments for Firebase Test Lab. To learn more about creating a ARG SPEC file, follow these [instructions.](ARG_SPEC.md) 8 | 9 |
10 | 11 | ## Step 2 - Service Account Authentication 12 | GitHub Actions will require authentication to communicate with Firebase Test Lab. In this step, you will need to create a service account and store it in the GitHub Secrets. To learn more about service account, follow these [instructions.](SERVICE_ACCOUNT.md) 13 | 14 |
15 | 16 | ## Step 3 - Workflow YML 17 | Lastly, integrate the Firebase Test Lab Action into your workflow. Note that, you must already have your build artifacts generated prior to calling the action. In the job, specify the `arg-spec` and `SERVICE_ACCOUNT` properties for the action to use. An example of the workflow can be found below. 18 | ``` 19 | name: Android CI 20 | on: [push] 21 | 22 | jobs: 23 | test: 24 | runs-on: ubuntu-latest 25 | steps: 26 | # Check out the repository 27 | - uses: actions/checkout@v2 28 | 29 | # Run the Firebase Test Lab Action 30 | - name: Run tests on Firebase Test Lab 31 | uses: asadmansr/Firebase-Test-Lab-Action@v1.0 32 | with: 33 | arg-spec: 'tests.yml:android-pixel-4' 34 | env: 35 | SERVICE_ACCOUNT: ${{ secrets.SERVICE_ACCOUNT }} 36 | ``` -------------------------------------------------------------------------------- /docs/assets/preview.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/asadmansr/Firebase-Test-Lab-Action/6155fb29b4b7c8025024aeda7e4c4573106aa5ef/docs/assets/preview.png -------------------------------------------------------------------------------- /entrypoint.sh: -------------------------------------------------------------------------------- 1 | #! /bin/bash 2 | 3 | set -e 4 | 5 | if [ -z "$SERVICE_ACCOUNT" ]; then 6 | echo "Service account is required to authorize gcloud to access the Cloud Platform." 7 | exit 2 8 | fi 9 | 10 | status=0 11 | arg_spec=$1 12 | service_account_file=/opt/service_account.json 13 | 14 | echo "$SERVICE_ACCOUNT" > $service_account_file 15 | 16 | project_id=$(cat $service_account_file | jq -r ".project_id") 17 | 18 | gcloud auth activate-service-account --key-file=$service_account_file 19 | gcloud config set project $project_id 20 | 21 | if gcloud firebase test android run $arg_spec 22 | then 23 | echo "Test matrix successfully finished" 24 | else 25 | status=$? 26 | echo "Test matrix exited abnormally with non-zero exit code: " $status 27 | fi 28 | 29 | rm $service_account_file 30 | 31 | exit $status --------------------------------------------------------------------------------