├── .github ├── CONTRIBUTING.md ├── ISSUE_TEMPLATE.md ├── PULL_REQUEST_TEMPLATE.md └── workflows │ └── check-links.yml ├── .travis.yml ├── LICENSE ├── README.md ├── arduino-ci-script.sh └── etc ├── astyle-configurations ├── 1.conf ├── 2.conf └── 3.conf ├── autoformat.sh └── codespell-ignore-words-list.txt /.github/CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contribution Rules 2 | Thanks for your interest in contributing to this free open source project! Please take the time to read and follow these rules before submitting an issue report or pull request. 3 | 4 | ## Issues 5 | - Do you need help using this project? Support requests should be made to the appropriate section of the [Arduino forum](http://forum.arduino.cc) rather than an issue report. Feel free to [send me a PM](http://forum.arduino.cc/index.php?action=pm;sa=send;u=224903) with a link to your forum thread. **Support will not be provided via PM**, I prefer to help you publicly so that others with the same question may benefit from the information. **Issue reports are to be used to report bugs or make feature requests only.** 6 | - Before submitting a bug report test using the [latest version of the project](https://github.com/per1234/arduino-ci-script/archive/master.zip) to be sure it hasn't already been fixed. **Don't report issues that only occur with old versions of the project.** 7 | - Search [existing pull requests and issues](https://github.com/per1234/arduino-ci-script/issues?q=) to be sure it hasn't already been reported. **Do not submit duplicate issue reports.** If you have additional information to provide about the issue then please comment on that issue. 8 | - Open an issue at https://github.com/per1234/arduino-ci-script/issues/new. 9 | - Describe the issue and what behavior you were expecting. Post complete error messages using [markdown code fencing](https://guides.github.com/features/mastering-markdown/#examples). 10 | - Provide a full set of steps necessary to reproduce the issue. Demonstration code should be complete, correct and simplified to the minimum amount of code necessary to reproduce the issue. Please use [markdown code fencing](https://guides.github.com/features/mastering-markdown/#examples) when posting code. 11 | - Be responsive. I may need you to provide more information, please respond as soon as possible. 12 | - If you find a solution to your problem update your issue report with an explanation of how you were able to fix it and close the issue. 13 | 14 | ## Pull Requests 15 | - Search [existing pull requests and issues](https://github.com/per1234/arduino-ci-script/pulls?q=) to make sure the change hasn't already been proposed. 16 | - Comment your code. The focus of Arduino is learning so it's best to be a bit more thorough about documenting code. 17 | - Follow the formatting conventions used throughout the rest of the project. Remove all trailing whitespace. 18 | - If appropriate, add or update tests in the [.travis.yml file](https://github.com/per1234/arduino-ci-script/blob/master/.travis.yml). 19 | - Update the [documentation](https://github.com/per1234/arduino-ci-script/blob/master/README.md) if your changes require it. This should be done in the same commit as the change. 20 | - **All commits must be atomic**. This means that the commit completely accomplishes a single task. Each commit should result in fully functional code. Multiple tasks should not be combined in a single commit. For more information please read http://www.freshconsulting.com/atomic-commits. 21 | - Commit messages: Use the [imperative mood](http://chris.beams.io/posts/git-commit/#imperative) in the commit title. Completely explain the purpose of the commit. Please read http://chris.beams.io/posts/git-commit for more tips on writing good commit messages. 22 | - Each pull request should address a single bug fix or enhancement, this may consist of multiple commits. If you have multiple, unrelated fixes or enhancements to contribute, then do each in a separate pull request. 23 | - Open a pull request at https://github.com/per1234/arduino-ci-script/compare. 24 | - If your pull request fixes an issue in the issue tracker, use the [closes/fixes/resolves syntax](https://help.github.com/articles/closing-issues-via-commit-messages) in the body to denote this. 25 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | Please read the Issues section of the Contributing Rules at the "contributing" link to the right before submitting an issue report. 2 | -------------------------------------------------------------------------------- /.github/PULL_REQUEST_TEMPLATE.md: -------------------------------------------------------------------------------- 1 | Please read the Pull Requests section of the Contributing Rules at the "contributing" link to the right before submitting an issue report. 2 | -------------------------------------------------------------------------------- /.github/workflows/check-links.yml: -------------------------------------------------------------------------------- 1 | name: Check Links 2 | 3 | # See: https://docs.github.com/en/free-pro-team@latest/actions/reference/events-that-trigger-workflows 4 | on: 5 | push: 6 | paths: 7 | - ".github/workflows/check-links.yml" 8 | - "**.md" 9 | pull_request: 10 | paths: 11 | - ".github/workflows/check-links.yml" 12 | - "**.md" 13 | schedule: 14 | # Run every Tuesday at 3 AM UTC to catch breakage caused by changes to the linked sites. 15 | - cron: "0 3 * * TUE" 16 | workflow_dispatch: 17 | repository_dispatch: 18 | 19 | jobs: 20 | check: 21 | runs-on: ubuntu-latest 22 | 23 | steps: 24 | - name: Checkout 25 | uses: actions/checkout@v2 26 | 27 | - name: Check links 28 | uses: gaurav-nelson/github-action-markdown-link-check@v1 29 | with: 30 | use-quiet-mode: yes 31 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | # This file is used to test the script with Travis CI 2 | 3 | env: 4 | global: 5 | # The Arduino IDE will be installed at APPLICATION_FOLDER/arduino 6 | - APPLICATION_FOLDER="${HOME}/arduino-ide" 7 | - SKETCHBOOK_FOLDER="${HOME}/arduino-sketchbook" 8 | matrix: 9 | allow_failures: 10 | # define the key used to determine whether job failures are allowed 11 | # The expected behavior of this job is failure because 1.6.3 doesn't support boards manager installation. 12 | - env: INSTALL_IDE_START_VERSION="1.6.3" VERBOSITY_LEVEL=2 VERBOSE_COMPILATION="false" 13 | include: 14 | 15 | 16 | # Test install_ide with no argument (using full version list). Causes the job to exceed Travis CI's maximum duration and thus can not be run. 17 | #- env: INSTALL_IDE_START_VERSION="" 18 | #language: generic 19 | 20 | 21 | # Test install_ide using full version list. Causes the job to exceed Travis CI's maximum duration and thus can not be run. 22 | #- env: INSTALL_IDE_START_VERSION="all" VERBOSITY_LEVEL=0 VERBOSE_COMPILATION="false" 23 | #language: generic 24 | 25 | 26 | # Test install_ide using custom version list 27 | # Test the use of the special version names "oldest", "newest", and "hourly" in a version list. 28 | # Test duplicates in version list 29 | - env: INSTALL_IDE_START_VERSION='("oldest" "1.8.3" "1.8.4" "1.8.4" "newest" "hourly")' VERBOSITY_LEVEL=1 VERBOSE_COMPILATION="true" 30 | language: generic 31 | 32 | 33 | # Test install_ide using version range. 34 | # Test the use of the special version name "newest" in a version range. 35 | # Test installing the full sane range of IDE versions 36 | - env: INSTALL_IDE_START_VERSION="1.6.5-r5" INSTALL_IDE_END_VERSION="newest" VERBOSITY_LEVEL=0 VERBOSE_COMPILATION="false" 37 | language: generic 38 | 39 | 40 | # Test the use of the special version name "hourly" in a version range. 41 | - env: INSTALL_IDE_START_VERSION="1.8.3" INSTALL_IDE_END_VERSION="hourly" VERBOSITY_LEVEL=0 VERBOSE_COMPILATION="false" 42 | language: generic 43 | 44 | 45 | # Allowed to fail 46 | # Test install_ide using single version 47 | # Test the failure behavior of install_package when a Boards Manager installation is attempted using an IDE version that doesn't support it. 48 | - env: INSTALL_IDE_START_VERSION="1.6.3" VERBOSITY_LEVEL=2 VERBOSE_COMPILATION="false" 49 | language: generic 50 | 51 | 52 | # Check for common issues with scripts 53 | - name: 'ShellCheck' 54 | language: minimal 55 | # Must define an empty install phase so that the default one won't be used 56 | install: true 57 | script: 58 | - shellcheck arduino-ci-script.sh 59 | 60 | 61 | # Check for inconsistent script formatting 62 | - name: 'Script Formatting Check' 63 | language: minimal 64 | install: 65 | - docker run --volume "$TRAVIS_BUILD_DIR":/mnt --workdir /mnt mvdan/shfmt:latest -i 2 -w . 66 | script: 67 | - git diff --color --exit-code 68 | 69 | 70 | - name: 'Unit Tests' 71 | language: minimal 72 | install: 73 | - git clone https://github.com/bats-core/bats-core.git 74 | - cd bats-core 75 | # Get new tags from the remote 76 | - git fetch --tags 77 | # Checkout the latest tag 78 | - git checkout $(git describe --tags `git rev-list --tags --max-count=1`) 79 | - sudo ./install.sh /usr/local 80 | - cd .. 81 | - rm --recursive --force bats-core 82 | - cd .. 83 | - git clone https://github.com/per1234/arduino-ci-script-tests 84 | - cd arduino-ci-script-tests 85 | - git checkout d9c33763ccde4467f43e1a5ddb8632e0575e6c3e 86 | script: 87 | - bats check_keywords_txt.bats 88 | - bats check_library_manager_compliance.bats 89 | - bats check_library_properties.bats 90 | - bats check_library_structure.bats 91 | - bats check_sketch_structure.bats 92 | - bats miscellaneous.bats 93 | 94 | 95 | # Check the files in the repository for consistent formatting 96 | - name: "File Formatting Checks" 97 | language: minimal 98 | # Must define an empty install phase so that the default one won't be used 99 | install: true 100 | script: 101 | # Check for files starting with a blank line 102 | - find . -path ./.git -prune -o -type f -print0 | xargs -0 -L1 bash -c 'head -1 "$0" | grep --binary-files=without-match --regexp="^$"; if [[ "$?" == "0" ]]; then echo "Blank line found at start of $0."; false; fi' 103 | # Check for tabs 104 | - find . -path './.git' -prune -or -path './tests/check_keywords_txt' -prune -or -path './tests/check_library_manager_compliance' -prune -or -path './tests/check_library_properties' -prune -or -path './tests/check_library_structure' -prune -or -not -path './tests/check_library_properties.bats' -or -type f -exec grep --with-filename --line-number --binary-files=without-match --regexp=$'\t' '{}' \; -exec echo 'Tab found.' \; -exec false '{}' + 105 | # Check for trailing whitespace 106 | - find . -path './.git' -prune -or -path './tests/check_library_properties' -prune -or -type f -exec grep --with-filename --line-number --binary-files=without-match --regexp='[[:blank:]]$' '{}' \; -exec echo 'Trailing whitespace found.' \; -exec false '{}' + 107 | # Check for non-Unix line endings 108 | - find . -path './.git' -prune -or -path './tests/check_keywords_txt' -prune -or -path './tests/check_library_manager_compliance' -prune -or -path './tests/check_library_properties' -prune -or -path './tests/check_library_structure' -prune -or -type f -exec grep --files-with-matches --regexp=$'\r$' '{}' \; -exec echo 'Non-Unix EOL detected.' \; -exec false '{}' + 109 | # Check for blank lines at end of files 110 | - find . -path ./.git -prune -o -type f -print0 | xargs -0 -L1 bash -c 'tail -1 "$0" | grep --binary-files=without-match --regexp="^$"; if [[ "$?" == "0" ]]; then echo "Blank line found at end of $0."; false; fi' 111 | # Check for files that don't end in a newline (https://stackoverflow.com/a/25686825) 112 | - find . -path ./.git -prune -or -path './tests/check_keywords_txt' -prune -or -path './tests/check_library_manager_compliance' -prune -or -path './tests/check_library_properties' -prune -or -path './tests/check_library_structure' -prune -or -type f -print0 | xargs -0 -L1 bash -c 'if test "$(grep --files-with-matches --binary-files=without-match --max-count=1 --regexp='.*' "$0")" && test "$(tail --bytes=1 "$0")"; then echo "No new line at end of $0."; false; fi' 113 | 114 | 115 | # Check for commonly misspelled words 116 | - name: "Spell Check" 117 | language: python 118 | python: 3.6 119 | install: 120 | # https://github.com/codespell-project/codespell 121 | - pip install codespell 122 | script: 123 | - codespell --skip="${TRAVIS_BUILD_DIR}/.git" --ignore-words="${TRAVIS_BUILD_DIR}/etc/codespell-ignore-words-list.txt" "${TRAVIS_BUILD_DIR}" 124 | 125 | 126 | # Default phases shared by all jobs that don't define their own 127 | install: 128 | - source "${TRAVIS_BUILD_DIR}/arduino-ci-script.sh" 129 | 130 | - set_script_verbosity "$VERBOSITY_LEVEL" 131 | 132 | - set_application_folder "$APPLICATION_FOLDER" 133 | - set_sketchbook_folder "$SKETCHBOOK_FOLDER" 134 | 135 | # Check for board definition errors that don't affect compilation 136 | - set_board_testing "true" 137 | 138 | # Check for library issues that don't affect compilation 139 | - set_library_testing "true" 140 | 141 | - install_ide "$INSTALL_IDE_START_VERSION" "$INSTALL_IDE_END_VERSION" 142 | 143 | # Install hardware packages 144 | # Test package install from this repository (can't do this because the repository isn't a hardware package) 145 | # - install_package 146 | # Test manual package install from compressed file download 147 | - install_package "https://github.com/SpenceKonde/ATTinyCore/archive/master.zip" 148 | # Test manual package install from Git repository clone 149 | - install_package "https://github.com/MCUdude/MightyCore.git" 150 | # Test manual package install from Git repository clone 151 | - install_package "https://github.com/JChristensen/mighty-1284p.git" "v1.6.3" 152 | 153 | # Test library installation from repository (can't do this because there is no library in this repository) 154 | # - install_library 155 | # Test library install from .zip file. A non-GitHub library download must be used because GitHub appends -{branch name} or -{release version} to the .zip downloads and having a library folder installed whose name contains "-" causes arduino 1.5.6 or older to hang. 156 | - install_library "https://bitbucket.org/teckel12/arduino-new-ping/downloads/NewPing_v1.8.zip" 157 | # Test library install from .zip file/folder that has a dot in the name. It must be renamed so that it won't have a hyphen in the folder name, which causes verification to hang with Arduino IDE 1.5.2 - 1.5.6-r2. 158 | # Test library install from .zip file with rename. 159 | - install_library "https://github.com/arduino-libraries/CapacitiveSensor/archive/0.5.1.zip" "CapacitiveSensor" 160 | # Test library install from git repo 161 | - install_library "https://github.com/sfrwmaker/WirelessOregonV2.git" 162 | # Test library install from git repo with branch 163 | - install_library "https://github.com/sde1000/NanodeUNIO.git" "master" 164 | # Test library install from git repo with rename 165 | - install_library "https://github.com/mikaelpatel/Arduino-Shell.git" "" "ArduinoShell" 166 | # Test library install from git repo with branch and rename 167 | - install_library "https://github.com/Avamander/max7456.git" "master" "max_7456" 168 | # Test set_verbose_output_during_compilation. 169 | - set_verbose_output_during_compilation "$VERBOSE_COMPILATION" 170 | 171 | # Boards Manager and Library Manager tests are done as late as possible to allow job 3 to do more thorough testing of script verbosity level 2 before the job fails 172 | 173 | # Test Boards Manager package install without URL. Test error handling of attempting to do a Boards Manager installation when the newest installed IDE version doesn't support it (should print a helpful error message and fail instead of hanging). 174 | - install_package "arduino:sam" 175 | # Test Boards Manager package install with URL 176 | - install_package "MiniCore:avr" "https://mcudude.github.io/MiniCore/package_MCUdude_MiniCore_index.json" 177 | 178 | # Test library install from Library Manager 179 | - install_library "Pushetta:1.0.1" 180 | 181 | 182 | script: 183 | - check_sketch_structure "${SKETCHBOOK_FOLDER}/libraries/NewPing/examples" 184 | 185 | - check_library_structure "${SKETCHBOOK_FOLDER}/libraries/CapacitiveSensor" 186 | 187 | - check_library_properties "${SKETCHBOOK_FOLDER}/libraries/NewPing" 188 | 189 | - check_keywords_txt "${SKETCHBOOK_FOLDER}/libraries/NewPing" 190 | 191 | - check_library_manager_compliance "${SKETCHBOOK_FOLDER}/libraries/NewPing" 192 | 193 | - check_code_formatting "1" "" "${APPLICATION_FOLDER}/arduino/examples/01.Basics/BareMinimum" 194 | 195 | # Test library installed from .zip with rename 196 | # Test library installed from .zip with dot in the folder name 197 | # Test board from hardware package manually installed from compressed file download 198 | # Test build_sketch with specific IDE version 199 | # Test build_sketch without absolute path 200 | - cd "${SKETCHBOOK_FOLDER}/libraries/CapacitiveSensor/examples/CapacitiveSensorSketch/" 201 | - build_sketch "CapacitiveSensorSketch.pde" "ATTinyCore-master:avr:attinyx5:LTO=disable,TimerClockSource=default,chip=85,clock=8internal,eesave=aenable,bod=disable,millis=enabled" "false" "1.8.4" 202 | 203 | # Test library installed from .zip 204 | # Test board from hardware package installed via Boards Manager without URL 205 | # Test build_sketch with "newest" special version name 206 | - build_sketch "${SKETCHBOOK_FOLDER}/libraries/NewPing/examples/NewPingExample/NewPingExample.pde" "arduino:sam:arduino_due_x_dbg" "false" "newest" 207 | 208 | # Test library installed from .git 209 | # Test board from hardware package installed with Boards Manager URL 210 | # Test build_sketch with an IDE version list 211 | - build_sketch "${SKETCHBOOK_FOLDER}/libraries/WirelessOregonV2/examples/OregonReceiver/OregonReceiver.ino" "MiniCore:avr:328:variant=modelP,BOD=2v7,LTO=Os,clock=16MHz_external" "false" '("1.8.3" "1.8.4")' 212 | 213 | # Test library installed from .git with branch 214 | # Test board from hardware package manually installed by cloning Git repository 215 | # Test build_sketch with an IDE version list using the "newest" special version name 216 | - build_sketch "${SKETCHBOOK_FOLDER}/libraries/NanodeUNIO/examples/NanodeUNIO_test/NanodeUNIO_test.pde" "MightyCore:avr:1284:pinout=standard,variant=modelP,BOD=2v7,LTO=Os,clock=16MHz_external" "false" '("1.8.4" "newest")' 217 | 218 | # Test library installed from .git with rename 219 | # Test board from hardware package manually installed by cloning Git repository with non-default branch 220 | # Test build_sketch with an IDE version range 221 | - build_sketch "${SKETCHBOOK_FOLDER}/libraries/ArduinoShell/examples/ShellBlink/ShellBlink.ino" "mighty-1284p:avr:avr_developers" "false" "1.8.3" "1.8.4" 222 | 223 | # Test library installed from .git with branch and rename 224 | # Test build_sketch with an IDE version range using the "newest" special version name 225 | - build_sketch "${SKETCHBOOK_FOLDER}/libraries/max_7456/examples/HelloWorld/HelloWorld.ino" "arduino:avr:uno" "false" "1.8.4" "newest" 226 | 227 | # Test build_sketch with no IDE version argument (should use all installed IDE versions) 228 | - build_sketch "${APPLICATION_FOLDER}/arduino/examples/01.Basics/BareMinimum/BareMinimum.ino" "arduino:avr:uno" "false" 229 | 230 | # Test build_sketch with "all" IDE version name 231 | - build_sketch "${APPLICATION_FOLDER}/arduino/examples/01.Basics/BareMinimum/BareMinimum.ino" "arduino:avr:uno" "false" "all" 232 | 233 | # Test library installed from Library Manager 234 | - build_sketch "${SKETCHBOOK_FOLDER}/libraries/Pushetta/examples/simple_notification/simple_notification.ino" "arduino:avr:uno" "false" "newest" 235 | 236 | # Test build_sketch allowed to fail (this will fail because WirelessOregonV2 is AVR specific) 237 | - build_sketch "${SKETCHBOOK_FOLDER}/libraries/WirelessOregonV2/examples/OregonReceiver/OregonReceiver.ino" "arduino:sam:arduino_due_x_dbg" "true" "newest" 238 | 239 | # build_sketch with folder argument tests: 240 | 241 | # Test build_sketch with folder argument with specific IDE version 242 | - build_sketch "${APPLICATION_FOLDER}/arduino/examples/01.Basics" "arduino:avr:uno" "false" "1.8.4" 243 | 244 | # Test build_sketch with folder argument with an IDE version list 245 | # Test build_sketch with an IDE version list using the "oldest" and "newest" special version names 246 | - build_sketch "${APPLICATION_FOLDER}/arduino/examples/01.Basics" "arduino:avr:uno" "false" '("oldest" "newest")' 247 | 248 | # Test build_sketch with folder argument with an IDE version range 249 | # Test build_sketch with an IDE version range using the "oldest" and "newest" special version names 250 | - build_sketch "${APPLICATION_FOLDER}/arduino/examples/01.Basics" "arduino:avr:uno" "false" "oldest" "newest" 251 | 252 | # Test build_sketch with folder argument with no IDE version specified (should use all installed IDE versions) 253 | - build_sketch "${APPLICATION_FOLDER}/arduino/examples/01.Basics" "arduino:avr:uno" "false" 254 | 255 | # Test build_sketch with folder argument with "all" IDE version name 256 | - build_sketch "${APPLICATION_FOLDER}/arduino/examples/01.Basics" "arduino:avr:uno" "false" "all" 257 | 258 | # Test build_sketch with folder argument required to fail (this will fail because WirelessOregonV2 is AVR specific) 259 | - build_sketch "${SKETCHBOOK_FOLDER}/libraries/WirelessOregonV2/examples" "arduino:sam:arduino_due_x_dbg" "require" "newest" 260 | 261 | - publish_report_to_gist "$REPORT_GITHUB_TOKEN" "$REPORT_GIST_URL" "true" 262 | 263 | - USER_NAME="$(echo "$TRAVIS_REPO_SLUG" | cut -d'/' -f 1)" 264 | - REPOSITORY_NAME="$(echo "$TRAVIS_REPO_SLUG" | cut -d'/' -f 2)" 265 | - publish_report_to_repository "$REPORT_GITHUB_TOKEN" "https://github.com/${USER_NAME}/CI-reports.git" "$REPOSITORY_NAME" "build_$(printf "%05d\n" "${TRAVIS_BUILD_NUMBER}")" "true" 266 | 267 | - display_report 268 | 269 | 270 | notifications: 271 | email: 272 | on_success: always 273 | on_failure: always 274 | webhooks: 275 | urls: 276 | # Use TravisBuddy to automatically comment on any pull request that results in a failed CI build 277 | - https://www.travisbuddy.com/ 278 | on_success: never 279 | on_failure: always 280 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2020 per1234, Vitalii Tereshchuk 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | arduino-ci-script 2 | ========== 3 | 4 | Bash script for continuous integration of [Arduino](http://www.arduino.cc/) projects. This is currently targeted for use with [Travis CI](http://travis-ci.org/) but it could be easily adapted to other purposes. 5 | 6 | [![Build Status](https://travis-ci.org/per1234/arduino-ci-script.svg?branch=master)](https://travis-ci.org/per1234/arduino-ci-script) 7 | 8 | 9 | ### Table of contents 10 | - [Installation](#installation) 11 | - [Usage](#usage) 12 | - [Publishing job reports](#publishing-job-reports) 13 | - [Troubleshooting](#troubleshooting) 14 | - [Contributing](#contributing) 15 | 16 | 17 | ### Installation 18 | The script can be used in multiple ways: 19 | 20 | #### Clone the latest release 21 | Include the latest release of the script in your project by adding the following lines to your build configuration file: 22 | ```yaml 23 | # Clone the script repository 24 | - git clone --depth 1 https://github.com/per1234/arduino-ci-script.git "${HOME}/scripts" 25 | - cd "${HOME}/scripts" 26 | # Get new tags from the remote 27 | - git fetch --tags 28 | # Checkout the latest tag 29 | - git checkout $(git describe --tags `git rev-list --tags --max-count=1`) 30 | - source "${HOME}/scripts/arduino-ci-script.sh" 31 | ``` 32 | 33 | #### Local copy 34 | If you're passing a token to the script's publish report functions then best security practices would be to use a static copy of the script so you can be sure of the commands the token is used with: 35 | - Download the latest version of the script from https://github.com/per1234/arduino-ci-script/releases by clicking one of the **Source code** links. 36 | - Unzip the downloaded file. 37 | - Copy arduino-ci-script.sh to a convenient location. 38 | - Include the script in your project by adding the following line to your build configuration file: 39 | ```yaml 40 | - source arduino-ci-script.sh 41 | ``` 42 | Be sure to check for new releases of the script so that you can benefit from the ongoing development work. You can receive notifications of releases by [watching the repository](https://github.com/per1234/arduino-ci-script/subscription). 43 | 44 | 45 | ### Usage 46 | See https://github.com/per1234/WatchdogLog/blob/master/.travis.yml for an example of the script in use. 47 | 48 | Please configure your continuous integration system to make the minimum number of downloads and sketch verifications necessary to effectively test your code. This will prevent wasting Arduino and Travis CI's bandwidth and keep the build durations short. 49 | 50 | ##### `set_script_verbosity SCRIPT_VERBOSITY_LEVEL` 51 | Control the level of verbosity of the script's output in the Travis CI log. Verbose output can be helpful for debugging but in normal usage it makes the log hard to read and may cause the log to exceed Travis CI's maximum log size of 4 MB, which causes the job to be terminated. The default verbosity level is `0`. 52 | - Parameter: **SCRIPT_VERBOSITY_LEVEL** - `0`, `1` or `2` (least to most verbosity). 53 | 54 | ##### `set_application_folder APPLICATION_FOLDER` 55 | - Parameter: **APPLICATION_FOLDER** - The folder to install the Arduino IDE (and Artistic Style if you use `check_code_formatting`) to. This should be set to `/usr/local/share` or a subfolder of that location. The folder will be created if it doesn't already exist. The Arduino IDE will be installed in the `arduino` subfolder. 56 | 57 | ##### `set_sketchbook_folder SKETCHBOOK_FOLDER` 58 | - Parameter: **SKETCHBOOK_FOLDER** - The folder to be set as the Arduino IDE's sketchbook folder. The folder will be created if it doesn't already exist. Libraries installed via `install_library` will be installed to the `libraries` subfolder. Non-Boards Manager hardware packages installed via `install_package` will be installed to the `hardware` subfolder. This setting is only supported by Arduino IDE 1.5.6 and newer. 59 | 60 | ##### `set_board_testing BOARD_TESTING` 61 | Turn on/off checking for problems with the board definition that generate a warning message during sketch verification but don't ordinarily cause it to fail, such as missing bootloader file. If this is turned on and a problem is detected the `build_sketch` command will return a non-zero exit status. This feature is off by default. 62 | - Parameter: **BOARD_TESTING** - `true`/`false` 63 | 64 | ##### `set_library_testing LIBRARY_TESTING` 65 | Turn on/off checking for problems with libraries that generate a warning message during sketch verification but don't ordinarily cause it to fail, such as missing or invalid items in the library.properties file. If this is turned on and a problem is detected the `build_sketch` command will return a non-zero exit status. This feature is off by default. 66 | - Parameter: **LIBRARY_TESTING** - `true`/`false` 67 | 68 | ##### Special version names: 69 | - `all`: Refers to all versions of the Arduino IDE (including the hourly build). In the context of `install_ide` this means all IDE versions compatible with the script (those that support the command line interface, 1.5.2 and newer). In the context of all other functions this means all IDE versions that were installed via `install_ide`. 70 | - `oldest`: The oldest release version of the Arduino IDE. In the context of `install_ide` this is the oldest of the IDE versions compatible with the script (1.5.2, the first version to have a command line interface). In the context of build_sketch this means the oldest IDE version that was installed via `install_ide`. 71 | - `newest`: In the context of `install_ide` this means the newest IDE release version. In the context of all other functions this means the newest IDE release version that was installed via `install_ide`. 'newest' will only match to the hourly build if that is the only version available. 72 | - `hourly`: The hourly build of the Arduino IDE. Note that this IDE version is intended for beta testing only. 73 | 74 | ##### `install_ide [IDEversionList]` 75 | Install a list of Arduino IDE version(s). 76 | - Parameter(optional): **IDEversionList** - A list of the versions of the Arduino IDE you want installed, in order from oldest to newest. e.g., `'("1.6.5-r5" "1.6.9" "1.8.2")'`. If no arguments are supplied all IDE versions will be installed. The script allows you to install all IDE versions with a command line interface (1.5.2 and newer) for the sake of being complete but I don't see a good reason for testing with the 1.5.x versions of the Arduino IDE. Please only install the IDE versions you actually need for your test to avoid wasting Arduino's bandwidth. This will also result in a shorter build duration. Installation of the IDE will be skipped if it's found to already be installed in the folder specified via the `set_application_folder` function so `install_ide` can also be used simply to inform the script which IDE versions are available. 77 | 78 | ##### `install_ide startIDEversion [endIDEversion]` 79 | Install a range of version(s) of the Arduino IDE. 80 | - Parameter: **startIDEversion** - The oldest version of the Arduino IDE to install. 81 | - Parameter(optional): **endIDEversion** - The newest version of the Arduino IDE to install. If this argument is omitted then only startIDEversion will be installed. 82 | 83 | ##### `install_package` 84 | "Manually" install the hardware package from the current repository. Packages are installed to `$SKETCHBOOK_FOLDER/hardware`. Assumes the hardware package is located in the root of the download or repository and has the correct folder structure. 85 | 86 | ##### `install_package packageURL` 87 | "Manually" install a hardware package downloaded as a compressed file. Packages are installed to `$SKETCHBOOK_FOLDER/hardware`. Assumes the hardware package is located in the root of the file and has the correct folder structure. 88 | - Parameter: **packageURL** - The URL of the hardware package download. The scheme component of the URL (e.g., `http://`, `https://`) is required. 89 | 90 | ##### `install_package packageURL [branchName]` 91 | "Manually" install a hardware package by cloning from a Git repository. Packages are installed to `$SKETCHBOOK_FOLDER/hardware`. Assumes the hardware package is located in the root of the repository and has the correct folder structure. 92 | - Parameter: **packageURL** - The URL of the Git repository. The scheme component of the URL (e.g., `http://`, `https://`) is required. The URL must end in `.git`. 93 | - Parameter(optional): **branchName** - Branch of the repository to install. If this argument is not specified or is left blank the default branch will be used. 94 | 95 | ##### `install_package packageID [packageURL]` 96 | Install a hardware package using the Arduino IDE (Boards Manager). Only the **Arduino AVR Boards** package is included with the Arduino IDE installation. Packages are installed to `$HOME/.arduino15/packages`. You must call `install_ide` before this function. This feature is only available with Arduino IDE 1.6.4 and newer. 97 | - Parameter: **packageID** - `package name:platform architecture[:version]`. If `version` is omitted the most recent version will be installed. e.g., `arduino:samd` will install the most recent version of **Arduino SAMD Boards**. 98 | - Parameter(optional): **packageURL** - The URL of the Boards Manager JSON file for 3rd party hardware packages. This can be omitted for hardware packages that are included in the official Arduino JSON file (e.g., Arduino SAM Boards, Arduino SAMD Boards, Intel Curie Boards). 99 | 100 | ##### `install_library` 101 | Install the library from the current repository. Assumes the library is in the root of the repository. The library is installed to the `libraries` subfolder of the sketchbook folder. 102 | 103 | ##### `install_library libraryName` 104 | Install a library that is listed in the Arduino Library Manager index. The library is installed to the `libraries` subfolder of the sketchbook folder. You must call `install_ide` before this function. This feature is only available with Arduino IDE 1.6.4 and newer installed. 105 | - Parameter: **libraryName** - The name of the library to install. You can specify a version separated from the name by a colon, e.g., "LiquidCrystal I2C:1.1.2". If no version is specified the most recent version will be installed. You can also specify comma-separated lists of library names. 106 | 107 | ##### `install_library libraryURL [newFolderName]` 108 | Download a library in a compressed file from a URL. The library is installed to the `libraries` subfolder of the sketchbook folder. 109 | - Parameter: **libraryURL** - The URL of the library download or library name in the Arduino Library Manager. The scheme component of the URL (e.g., `http://`, `https://`) is required. The download file can be in any compressed file format. Assumes the library is located in the root of the file. 110 | - Parameter(optional): **newFolderName** - Folder name to rename the installed library folder to. This can be useful if the default folder name of the downloaded file is problematic. The Arduino IDE gives include file preference when the filename matches the library folder name. GitHub's "Download ZIP" file is given the folder name `{repository name}-{branch name}`. Library folder names that contain `-` or `.` are not compatible with Arduino IDE 1.5.6 and older, arduino will hang if it's started with a library using an invalid folder name installed. 111 | 112 | ##### `install_library libraryURL [branchName [newFolderName]]` 113 | Install a library by cloning a Git repository. The library is installed to the `libraries` subfolder of the sketchbook folder. Assumes the library is located in the root of the repository. 114 | - Parameter: **libraryURL** - The URL of the library download or library name in the Arduino Library Manager. The scheme component of the URL (e.g., `http://`, `https://`) is required. The URL must end in `.git`. 115 | - Parameter(optional): **branchName** - Branch of the repository to install. If this argument is not specified or is left blank the default branch will be used. 116 | - Parameter(optional): **newFolderName** - Folder name to rename the installed library folder to. This can be useful if the default folder name of the downloaded file is problematic. The Arduino IDE gives include file preference when the filename matches the library folder name. Library folder names that contain `-` or `.` are not compatible with Arduino IDE 1.5.6 and older, arduino will hang if it's started with a library using an invalid folder name installed. If the `newFolderName` argument is specified the `branchName` argument must also be specified. If you don't want to specify a branch then use `""` for the `branchName` argument. 117 | 118 | ##### `set_verbose_output_during_compilation verboseOutputDuringCompilation` 119 | Turn on/off `arduino` verbose output during compilation (same as the IDE's **File > Preferences > Show verbose output during: > compilation**). This will show all the commands `arduino` runs during the process rather than just the compiler output. This is usually not very useful output and only clutters up the log. This feature is off by default. 120 | - Parameter: **verboseOutputDuringCompilation** - `true`/`false` 121 | 122 | ##### `check_sketch_structure searchPath` 123 | Check sketches to ensure they have the correct structure. 124 | - Parameter: **searchPath** - Path containing sketches. The path will be searched recursively and all sketches found under it will be checked. 125 | 126 | ##### `check_library_structure basePath [depth]` 127 | Check libraries to ensure they have the correct structure. This will also run `check_sketch_structure` on all sketches bundled with the library. 128 | - Parameter: **basePath** - Path containing a library. 129 | - Parameter(optional): **depth** - Folder depth relative to `basePath` where the libraries are located. A depth of 0 will check the library located at `basePath`. The default value is 0. 130 | 131 | ##### `check_library_properties searchPath [maximumSearchDepth]` 132 | Check [library.properties](https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification#libraryproperties-file-format) library metadata files for errors. 133 | - Parameter: **searchPath** - Path containing library.properties. 134 | - Parameter(optional): **maximumSearchDepth** - The recursive search depth. A depth of 0 will only search `searchPath` and no subfolders. The default value is 0. 135 | 136 | ##### `check_keywords_txt searchPath [maximumSearchDepth]` 137 | Check [keywords.txt](https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification#keywords) files for errors. 138 | - Parameter: **searchPath** - Path containing keywords.txt files. 139 | - Parameter(optional): **maximumSearchDepth** - The recursive search depth. A depth of 0 will only search `searchPath` and no subfolders. The default value is 0. 140 | 141 | ##### `check_library_manager_compliance libraryPath` 142 | Make some additional checks for compliance with the requirements for adding a library to the [Library Manager index](https://github.com/arduino/Arduino/wiki/Library-Manager-FAQ). This function should be used in combination with `check_library_structure` and `check_library_properties` to ensure full compliance with the requirements. 143 | - Parameter: **libraryPath** - Path of the library to check. 144 | 145 | ##### `check_code_formatting strictness excludedPathList targetPath` 146 | Check code formatting for compliance with the Arduino code style. The [Artistic Style](http://astyle.sourceforge.net) formatter tool is used for this check. If it's not already installed, it will be installed to the `astyle` subfolder of the folder specified to `set_application_folder`. Note that in the Travis CI job logs, the `check_code_formatting` output is "folded" to make it easier to browse. You can click the triangle in the left margin of the command to unfold the output. 147 | - Parameter: **strictness** - Determines how strict to be about code formatting compliance: 1 (least strict) - 3 (most strict). Each strictness level is based on the previous one, but with additional requirements. 148 | - `1`: The Arduino IDE's auto format configuration. 149 | - `2`: The configuration Arduino uses to format their example sketches. 150 | - `3`: A custom configuration based on a study of the prevailing styles used in official Arduino code. 151 | - Parameter: **excludedPathList** - A comma-separated list of paths to exclude from the check. 152 | - Parameter: **targetPath** - The path to run the check on. All code files will be checked recursively. 153 | 154 | ##### `build_sketch sketchPath boardID allowFail IDEversion` 155 | ##### `build_sketch sketchPath boardID allowFail [IDEversionList]` 156 | ##### `build_sketch sketchPath boardID allowFail startIDEversion endIDEversion` 157 | Verify/compile sketch(es). `build_sketch` will echo the `arduino` exit status to the log, which is documented at https://github.com/arduino/Arduino/blob/master/build/shared/manpage.adoc#exit-status. Note that in the Travis CI job logs, the compilation output is "folded" to make it easier to browse. You can click the triangles in the left margin to unfold. 158 | - Parameter: **sketchPath** - Path to a sketch or folder containing sketches. If a folder is specified it will be recursively searched and all sketches will be verified. 159 | - Parameter: **boardID** - `package:arch:board[:parameters]` ID of the board to be compiled for. e.g., `arduino:avr:uno`. Board-specific parameters are only supported by Arduino IDE 1.5.5 and newer. 160 | - Parameter: **allowFail** - `true`, `require`, or `false`. Allow the verification to fail without causing the CI build to fail. `require` will cause the build to fail if the sketch verification doesn't fail. 161 | - Parameter: **IDEversion** - A single version of the Arduino IDE to use to verify the sketch. 162 | - Parameter(optional): **IDEversionList** - A list of versions of the Arduino IDE to use to verify the sketch. e.g., `'("1.6.5-r5" "1.6.9" "1.8.2")'`. If no version list is provided all installed IDE versions will be used. 163 | - Parameter: **startIDEversion** - The start (inclusive) of a range of versions of the Arduino IDE to use to verify the sketch. 164 | - Parameter: **endIDEversion** - The end (inclusive) of a range of versions of the Arduino IDE to use to verify the sketch. 165 | 166 | ##### `display_report` 167 | Echo a tab separated report of all verification results to the log. The report is located in the `${HOME}/arduino-ci-script_report` folder and will be named according to the build number and job number. Note that Travis CI runs each build of the job in a separate virtual machine so if you have multiple jobs you will have multiple reports. The only way I have found to generate a single report for all tests is to run them as a single job. This means not setting multiple matrix environment variables in .travis.yml's `env` array. See: https://docs.travis-ci.com/user/environment-variables. The report consists of one line per verification: 168 | - Build timestamp - Timestamp of the sketch verification in UTC. 169 | - Build - The Travis CI build number. 170 | - Job - Travis CI job number. 171 | - Job URL - The URL of the Travis CI job log. 172 | - Build Trigger - The cause of this Travis CI build. Possible values are `push`, `pull_request`, `api`, `cron`. 173 | - Allow Job Failure - Whether the Travis CI configuration was set to allow the failure of this job without failing the build. See: https://docs.travis-ci.com/user/customizing-the-build/#Rows-that-are-Allowed-to-Fail. 174 | - PR# - Pull request number (if build was triggered by a pull request). 175 | - Branch - The branch of the repository that was built. 176 | - Commit - Commit hash of the build. 177 | - Commit range - The range of commits that were included in the push or pull request. 178 | - Commit Message - First line of the commit message. 179 | - Sketch filename 180 | - Board ID 181 | - IDE version 182 | - Program Storage (bytes) - Program storage usage of the compiled sketch. 183 | - Dynamic Memory (bytes) - Dynamic memory usage by global variables in the compiled sketch (not available for some boards). 184 | - \# Warnings - Number of warnings reported by the compiler during the sketch compilation. 185 | - Allow Failure - Whether the sketch verification was allowed to fail (set by the `allowFail` argument of `build_sketch`). 186 | - Exit Status - Exit status returned by arduino after the sketch verification. 187 | - \# Board Issues - The number of board issues detected. 188 | - Board Issue - Short description of the last board issue detected. 189 | - \# Library Issues - The number of library issues detected. Library issues are things that cause warnings in the sketch verification output that come from the IDE, rather than the compiler. 190 | - Library Issue - Short description of the last library issue detected. 191 | 192 | ##### `publish_report_to_repository REPORT_GITHUB_TOKEN repositoryURL reportBranch reportFolder doLinkComment` 193 | Add the report to a repository. See the [instructions for publishing job reports](#publishing-job-reports) for details. 194 | - Parameter: **REPORT_GITHUB_TOKEN** - The hidden or encrypted environment variable containing the GitHub personal access token. 195 | - Parameter: **repositoryURL** - The .git URL of the repository to publish the report to. This URL can be found by clicking the "Clone or download" button on the home page of the repository. The repository must already exist. 196 | - Parameter: **reportBranch** - The branch to publish the report to. The branch must already exist. 197 | - Parameter: **reportFolder** - The folder to publish the report to. The folder will be created if it doesn't exist. 198 | - Parameter: **doLinkComment** - `true` or `false` Whether to comment on the GitHub thread of the commit that triggered the build with a link to the report. 199 | 200 | ##### `publish_report_to_gist REPORT_GITHUB_TOKEN REPORT_GIST_URL doLinkComment` 201 | Add the report to the report gist. See the [instructions for publishing job reports](#publishing-job-reports) for details. 202 | - Parameter: **REPORT_GITHUB_TOKEN** - The hidden or encrypted environment variable containing the GitHub personal access token. 203 | - Parameter: **REPORT_GIST_URL** - The URL of the report gist. 204 | - Parameter: **doLinkComment** - `true` or `false` Whether to comment on the GitHub thread of the commit that triggered the build with a link to the report. 205 | 206 | 207 | ### Publishing job reports 208 | The script offers the option of publishing the job result reports to a repository or GitHub [gist](https://gist.github.com/) by using the `publish_report_to_repository` or `publish_report_to_gist` functions. This makes it easier to view the reports or to import them into a spreadsheet program. You also have the option of having the link to the reports automatically added in a comment to the commit that triggered the build. This requires some configuration, which is described in the instructions below. 209 | 210 | NOTE: For security reasons, reports for builds of pull requests from a fork of the repository can not be published. If the owner of that fork wants to publish reports they can create a GitHub token (and gist if using `publish_report_to_gist`) and configure the Travis CI settings for their fork of the repository following these instructions. 211 | 212 | #### Creating a GitHub personal access token 213 | This is required for either publishing option. 214 | 1. Sign in to your GitHub account. 215 | 2. Click your avatar at the top right corner of GitHub > **Settings** > **Developer settings** > **Personal access tokens** > **Generate new token**. 216 | 3. Check the appropriate permissions for the token: 217 | 1. If using `publish_report_to_gist` check **gist**. 218 | 2. If using `publish_report_to_repository` or setting the `doLinkComment` argument of `publish_report_to_gist` check **public_repo** (for public repositories only) or **repo** (for private and public repositories). 219 | 5. Click the **Generate token** button. 220 | 6. When the generated token is displayed click the clipboard icon next to it to copy the token. 221 | 7. Open the settings page for your repository on the Travis CI website. 222 | 8. In the **Environment Variables** section enter `REPORT_GITHUB_TOKEN` in the **Name** field and the token in the **Value** field. Make sure the **Display value in build log** switch is in the off position. 223 | 9. Click the **Add** button. 224 | 225 | An alternative to using a Travis CI hidden environment variable as described above is to define the GitHub personal access token as an encrypted environment variable: https://docs.travis-ci.com/user/environment-variables/#Encrypting-environment-variables. 226 | 227 | #### Creating a gist 228 | This is required for use of the `publish_report_to_gist` function. 229 | 1. Open https://gist.github.com/ 230 | 2. Sign in to your GitHub account. 231 | 3. Type an appropriate name in the **Filename including extension...** field. Gists sort files alphabetically so the filename should be something that will sort before the report filenames, which start at travis_ci_job_report_00001.001.tsv. 232 | 4. Add some text to the file contents box. 233 | 5. Click **Create secret gist** if you don't want the gist to be discoverable (it can still be read by anyone who knows the URL), or **Create public gist** to make it discoverable. 234 | 6. Copy the URL of the gist. 235 | 7. Open the settings page for your repository on the Travis CI website. 236 | 8. In the **Environment Variables** section enter `REPORT_GIST_URL` in the **Name** field and the URL of the gist in the **Value** field. You can turn the **Display value in build log** switch to the on position. The gist URL is not secret and this will provide more information in the log. 237 | 9. Click the **Add** button. 238 | 239 | 240 | ### Troubleshooting 241 | ##### Script hangs after an arduino command 242 | The Arduino IDE will usually try to start the GUI whenever there is an error in the command. Since the Travis CI build environment does not support this it will just hang for ten minutes until Travis CI automatically cancels the job. This means you get no useful information on the cause of the problem. 243 | 244 | ##### Verbose output 245 | Verbose output results in a harder to read log so you should leave it off or minimized when possible but it can be useful for troubleshooting. Note that turning on verbose output for a large build may cause the log to exceed 4 MB, which causes Travis CI to terminate the job. 246 | - Verbose script output - See [`set_script_verbosity` documentation](#set_script_verbosity-script_verbosity_level) in the Usage section. 247 | - Verbose output during compilation - See [`set_verbose_output_during_compilation` documentation](#set_verbose_output_during_compilation-verboseoutputduringcompilation) in the Usage section. 248 | - Verbose output for Travis CI - Add one or both of the following lines to your `.travis.yml` file to get more details of the Travis CI build process. 249 | - Print shell input lines as they are read: 250 | - `- set -o verbose` 251 | - Print a trace of simple commands, for commands, case commands, select commands, and arithmetic for commands and their arguments or associated word lists after they are expanded and before they are executed. The value of the PS4 variable is expanded and the resultant value is printed before the command and its expanded arguments. 252 | - `- set -o xtrace` 253 | 254 | ##### Problematic IDE versions 255 | Some older versions of the Arduino IDE have bugs or limitations that may cause problems if used with this script: 256 | - 1.5.1 and older - The command line interface was added in 1.5.2, thus no version older than that can be used. 257 | - 1.5.4 and older - Do not support board-specific parameters, set by custom **Tools** menu items. 258 | - 1.5.5 and older - Do not support setting preferences (`--pref`), thus `set_sketchbook_folder` can not be used if no newer IDE version has been installed. 259 | - 1.5.5-r2 and older - Don't recognize libraries that have a library.properties` file that doesn't define a `core-dependencies` property. The file include is successful but compilation of sketches that use the library functions will fail. 260 | - 1.5.6 and older - `-` or `.` are not allowed in sketch or library folder names. If any are present the Arduino IDE will hang indefinitely when it's executed. 261 | - 1.6.2 - Moves its hardware packages to the .arduino15 folder, causing all other IDE versions to use those cores, some of which are not compatible. For this reason 1.6.2 is not installed when a version range containing, but not starting or ending in, 1.6.2 is passed to `install_ide`. 1.6.2 is installed if it is explicitly specified in a version list. 262 | - 1.6.3 and older - Do not support installing boards (`--install-boards`), thus `install_package` can't be used if no newer IDE version has been installed. 263 | 264 | 265 | ### Contributing 266 | Pull requests or issue reports are welcome! Please see the [contribution rules](https://github.com/per1234/arduino-ci-script/blob/master/.github/CONTRIBUTING.md) for instructions. 267 | -------------------------------------------------------------------------------- /arduino-ci-script.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # This script is used to automate continuous integration tasks for Arduino projects 3 | # https://github.com/per1234/arduino-ci-script 4 | 5 | # Based on https://github.com/adafruit/travis-ci-arduino/blob/eeaeaf8fa253465d18785c2bb589e14ea9893f9f/install.sh#L11 6 | # It seems that arrays can't been seen in other functions. So instead I'm setting $IDE_VERSIONS to a string that is the command to create the array 7 | readonly ARDUINO_CI_SCRIPT_IDE_VERSION_LIST_ARRAY_DECLARATION="declare -a -r IDEversionListArray=" 8 | 9 | readonly ARDUINO_CI_SCRIPT_TEMPORARY_FOLDER="${HOME}/temporary/arduino-ci-script" 10 | readonly ARDUINO_CI_SCRIPT_IDE_INSTALLATION_FOLDER="arduino" 11 | readonly ARDUINO_CI_SCRIPT_VERIFICATION_OUTPUT_FILENAME="${ARDUINO_CI_SCRIPT_TEMPORARY_FOLDER}/verification_output.txt" 12 | readonly ARDUINO_CI_SCRIPT_REPORT_FILENAME="travis_ci_job_report_$(printf '%05d\n' "${TRAVIS_BUILD_NUMBER}").$(printf '%03d\n' "$(echo "$TRAVIS_JOB_NUMBER" | cut -d'.' -f 2)").tsv" 13 | readonly ARDUINO_CI_SCRIPT_REPORT_FOLDER="${HOME}/arduino-ci-script_report" 14 | readonly ARDUINO_CI_SCRIPT_REPORT_FILE_PATH="${ARDUINO_CI_SCRIPT_REPORT_FOLDER}/${ARDUINO_CI_SCRIPT_REPORT_FILENAME}" 15 | # The arduino manpage(https://github.com/arduino/Arduino/blob/master/build/shared/manpage.adoc#exit-status) documents a range of exit statuses. These exit statuses indicate success, invalid arduino command, or compilation failed due to legitimate code errors. arduino sometimes returns other exit statuses that may indicate problems that may go away after a retry. 16 | readonly ARDUINO_CI_SCRIPT_HIGHEST_ACCEPTABLE_ARDUINO_EXIT_STATUS=4 17 | readonly ARDUINO_CI_SCRIPT_SKETCH_VERIFY_RETRIES=3 18 | readonly ARDUINO_CI_SCRIPT_REPORT_PUSH_RETRIES=10 19 | 20 | readonly ARDUINO_CI_SCRIPT_SUCCESS_EXIT_STATUS=0 21 | readonly ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS=1 22 | 23 | # Arduino IDE 1.8.2 and newer generates a ton of garbage output (appears to be something related to jmdns) that must be filtered for the log to be readable and to avoid exceeding the maximum log length 24 | readonly ARDUINO_CI_SCRIPT_ARDUINO_OUTPUT_FILTER_REGEX='(^\[SocketListener\(travis-job-*|^ *[0-9][0-9]*: [0-9a-g][0-9a-g]*|^dns\[query,[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*:[0-9][0-9]*, length=[0-9][0-9]*, id=|^dns\[response,[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*\.[0-9][0-9]*:[0-9][0-9]*, length=[0-9][0-9]*, id=|^questions:$|\[DNSQuestion@|type: TYPE_IGNORE|^\.\]$|^\.\]\]$|^.\.\]$|^.\.\]\]$)' 25 | 26 | # Default value 27 | ARDUINO_CI_SCRIPT_TOTAL_SKETCH_BUILD_FAILURE_COUNT=0 28 | 29 | # Set the arduino command name according to OS (on Windows arduino_debug should be used) 30 | if [[ "$OSTYPE" == "cygwin" ]] || [[ "$OSTYPE" == "msys" ]]; then 31 | ARDUINO_CI_SCRIPT_ARDUINO_COMMAND="arduino_debug" 32 | else 33 | ARDUINO_CI_SCRIPT_ARDUINO_COMMAND="arduino" 34 | fi 35 | 36 | # Create the folder if it doesn't exist 37 | function create_folder() { 38 | local -r folderName="$1" 39 | if ! [[ -d "$folderName" ]]; then 40 | # shellcheck disable=SC2086 41 | mkdir --parents $ARDUINO_CI_SCRIPT_VERBOSITY_OPTION "$folderName" 42 | fi 43 | } 44 | 45 | function set_script_verbosity() { 46 | ARDUINO_CI_SCRIPT_VERBOSITY_LEVEL="$1" 47 | 48 | if [[ "$ARDUINO_CI_SCRIPT_VERBOSITY_LEVEL" == "true" ]]; then 49 | ARDUINO_CI_SCRIPT_VERBOSITY_LEVEL=1 50 | fi 51 | 52 | if [[ "$ARDUINO_CI_SCRIPT_VERBOSITY_LEVEL" -eq 1 ]]; then 53 | ARDUINO_CI_SCRIPT_VERBOSITY_OPTION="--verbose" 54 | ARDUINO_CI_SCRIPT_QUIET_OPTION="" 55 | # Show stderr only 56 | ARDUINO_CI_SCRIPT_VERBOSITY_REDIRECT="1>/dev/null" 57 | elif [[ "$ARDUINO_CI_SCRIPT_VERBOSITY_LEVEL" -eq 2 ]]; then 58 | ARDUINO_CI_SCRIPT_VERBOSITY_OPTION="--verbose" 59 | ARDUINO_CI_SCRIPT_QUIET_OPTION="" 60 | # Show stdout and stderr 61 | ARDUINO_CI_SCRIPT_VERBOSITY_REDIRECT="" 62 | else 63 | ARDUINO_CI_SCRIPT_VERBOSITY_LEVEL=0 64 | ARDUINO_CI_SCRIPT_VERBOSITY_OPTION="" 65 | # cabextract only takes the short option name so this is more universally useful than --quiet 66 | ARDUINO_CI_SCRIPT_QUIET_OPTION="-q" 67 | # Don't show stderr or stdout 68 | ARDUINO_CI_SCRIPT_VERBOSITY_REDIRECT="&>/dev/null" 69 | fi 70 | 71 | } 72 | 73 | # Deprecated, use set_script_verbosity 74 | function set_verbose_script_output() { 75 | set_script_verbosity 1 76 | } 77 | 78 | # Deprecated, use set_script_verbosity 79 | function set_more_verbose_script_output() { 80 | set_script_verbosity 2 81 | } 82 | 83 | # Turn on verbosity based on the preferences set by set_script_verbosity 84 | function enable_verbosity() { 85 | # Store previous verbosity settings so they can be set back to their original values at the end of the function 86 | shopt -q -o verbose 87 | ARDUINO_CI_SCRIPT_PREVIOUS_VERBOSE_SETTING="$?" 88 | 89 | shopt -q -o xtrace 90 | ARDUINO_CI_SCRIPT_PREVIOUS_XTRACE_SETTING="$?" 91 | 92 | if [[ "$ARDUINO_CI_SCRIPT_VERBOSITY_LEVEL" -gt 0 ]]; then 93 | # "Print shell input lines as they are read." 94 | # https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html 95 | set -o verbose 96 | fi 97 | if [[ "$ARDUINO_CI_SCRIPT_VERBOSITY_LEVEL" -gt 1 ]]; then 98 | # "Print a trace of simple commands, for commands, case commands, select commands, and arithmetic for commands and their arguments or associated word lists after they are expanded and before they are executed. The value of the PS4 variable is expanded and the resultant value is printed before the command and its expanded arguments." 99 | # https://www.gnu.org/software/bash/manual/html_node/The-Set-Builtin.html 100 | set -o xtrace 101 | fi 102 | } 103 | 104 | # Return verbosity settings to their previous values 105 | function disable_verbosity() { 106 | if [[ "$ARDUINO_CI_SCRIPT_PREVIOUS_VERBOSE_SETTING" == "0" ]]; then 107 | set -o verbose 108 | else 109 | set +o verbose 110 | fi 111 | 112 | if [[ "$ARDUINO_CI_SCRIPT_PREVIOUS_XTRACE_SETTING" == "0" ]]; then 113 | set -o xtrace 114 | else 115 | set +o xtrace 116 | fi 117 | } 118 | 119 | # Verbosity and, in some cases, errexit must be disabled before an early return from a public function, this allows it to be done in a single line instead of two 120 | function return_handler() { 121 | local -r exitStatus="$1" 122 | 123 | # If exit status is success and errexit is enabled then it must be disabled before exiting the script because errexit must be disabled by default and only enabled in the functions that specifically require it. 124 | # If exit status is not success then errexit should not be disabled, otherwise Travis CI won't fail the build even though the exit status was failure. 125 | if [[ "$exitStatus" == "$ARDUINO_CI_SCRIPT_SUCCESS_EXIT_STATUS" ]] && shopt -q -o errexit; then 126 | set +o errexit 127 | fi 128 | 129 | disable_verbosity 130 | 131 | return "$exitStatus" 132 | } 133 | 134 | function set_application_folder() { 135 | enable_verbosity 136 | 137 | ARDUINO_CI_SCRIPT_APPLICATION_FOLDER="$1" 138 | 139 | disable_verbosity 140 | } 141 | 142 | function set_sketchbook_folder() { 143 | enable_verbosity 144 | 145 | ARDUINO_CI_SCRIPT_SKETCHBOOK_FOLDER="$1" 146 | 147 | # Create the sketchbook folder if it doesn't already exist 148 | create_folder "$ARDUINO_CI_SCRIPT_SKETCHBOOK_FOLDER" 149 | 150 | # Set sketchbook location preference if the IDE is already installed 151 | if [[ "$INSTALLED_IDE_VERSION_LIST_ARRAY" != "" ]]; then 152 | set_ide_preference "sketchbook.path=$ARDUINO_CI_SCRIPT_SKETCHBOOK_FOLDER" 153 | fi 154 | 155 | disable_verbosity 156 | } 157 | 158 | # Deprecated 159 | function set_parameters() { 160 | set_application_folder "$1" 161 | set_sketchbook_folder "$2" 162 | } 163 | 164 | # Check for errors with the board definition that don't affect sketch verification 165 | function set_board_testing() { 166 | enable_verbosity 167 | 168 | ARDUINO_CI_SCRIPT_TEST_BOARD="$1" 169 | 170 | disable_verbosity 171 | } 172 | 173 | # Check for errors with libraries that don't affect sketch verification 174 | function set_library_testing() { 175 | enable_verbosity 176 | 177 | ARDUINO_CI_SCRIPT_TEST_LIBRARY="$1" 178 | 179 | disable_verbosity 180 | } 181 | 182 | # Install all specified versions of the Arduino IDE 183 | function install_ide() { 184 | enable_verbosity 185 | 186 | local -r startIDEversion="$1" 187 | local -r endIDEversion="$2" 188 | 189 | # https://docs.travis-ci.com/user/customizing-the-build/#Implementing-Complex-Build-Steps 190 | # set -o errexit will cause the script to exit as soon as any command returns a non-zero exit status. Without this the success of the function call is determined by the exit status of the last command in the function 191 | set -o errexit 192 | 193 | if [[ "$ARDUINO_CI_SCRIPT_APPLICATION_FOLDER" == "" ]]; then 194 | echo "ERROR: Application folder was not set. Please use the set_application_folder function to define the location of the application folder." 195 | return_handler "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 196 | fi 197 | 198 | # Generate an array declaration string containing a list all available Arduino IDE versions which support CLI 199 | # Save the current folder 200 | local -r previousFolder="$PWD" 201 | cd "$ARDUINO_CI_SCRIPT_TEMPORARY_FOLDER" 202 | # Create empty local repo for the purpose of getting a list of tags in the arduino/Arduino repository 203 | git init --quiet Arduino 204 | cd Arduino 205 | git remote add origin https://github.com/arduino/Arduino.git 206 | if [[ "$startIDEversion" != "1.6.2" ]] && [[ "$startIDEversion" != "1.6.2" ]]; then 207 | # See "Arduino IDE version blocklist" documentation below 208 | local -r IDEversion162regex=--regex='refs/tags/1\.6\.2' 209 | if [[ "$ARDUINO_CI_SCRIPT_VERBOSITY_LEVEL" -gt 0 ]]; then 210 | echo "NOTE: Due to not playing nicely with other versions, Arduino IDE 1.6.2 will not be installed unless explicitly specified in the version arguments." 211 | fi 212 | fi 213 | 214 | # Arduino IDE tag blocklist: 215 | # <1.5.2: no CLI (https://github.com/arduino/Arduino/blob/master/build/shared/manpage.adoc#history) 216 | # 1.5.4-r2: Not available for download 217 | # 1.5.5-r2: Not available for download 218 | # 1.5.7-macosx-java7: Not available for download 219 | # 1.5.8-macosx-java7: Not available for download 220 | # 1.6.2: has the nasty behavior of moving the included hardware cores to the .arduino15 folder, causing those versions to be used for all builds after Arduino IDE 1.6.2 is used. For that reason, 1.6.2 will only be installed if explicitly specified in the install_ide version arguments 221 | # 1.6.5-r2: Not available for download 222 | # 1.6.5-r3: Not available for download 223 | # 1.6.5-r2: Not available for download 224 | # 1.6.5-r3: Not available for download 225 | # 1.8.11-ms-store-1: Not available for download 226 | # 1.8.13-ms-store-1: Not available for download 227 | local -r ARDUINO_CI_SCRIPT_FULL_IDE_VERSION_LIST_ARRAY="${ARDUINO_CI_SCRIPT_IDE_VERSION_LIST_ARRAY_DECLARATION}'(\"$(git ls-remote --quiet --tags --refs | grep --invert-match --regexp='refs/tags/1\.0' --regexp='refs/tags/1\.5$' --regexp='refs/tags/1\.5\.1$' --regexp='refs/tags/1\.5\.4-r2$' --regexp='refs/tags/1\.5\.5-r2$' --regexp='refs/tags/1\.5\.7-macosx-java7$' --regexp='refs/tags/1\.5\.8-macosx-java7$' ${IDEversion162regex} --regexp='refs/tags/1\.6\.5-r2$' --regexp='refs/tags/1\.6\.5-r3$' --regexp='refs/tags/.*-ms-store.*$' | grep --regexp='refs/tags/[0-9]\+\.[0-9]\+\.[0-9]\+\(\(-.*$\)\|$\)' | cut --delimiter='/' --fields=3 | sort --version-sort | sed ':a;N;$!ba;s/\n/\" \"/g')\")'" 228 | cd .. 229 | # Remove the temporary repo 230 | rm Arduino --recursive --force 231 | # Go back to the previous folder location 232 | cd "$previousFolder" 233 | 234 | # Determine list of IDE versions to install 235 | generate_ide_version_list_array "$ARDUINO_CI_SCRIPT_FULL_IDE_VERSION_LIST_ARRAY" "$startIDEversion" "$endIDEversion" 236 | INSTALLED_IDE_VERSION_LIST_ARRAY="$ARDUINO_CI_SCRIPT_GENERATED_IDE_VERSION_LIST_ARRAY" 237 | 238 | # Set "$NEWEST_INSTALLED_IDE_VERSION" 239 | determine_ide_version_extremes "$INSTALLED_IDE_VERSION_LIST_ARRAY" 240 | NEWEST_INSTALLED_IDE_VERSION="$ARDUINO_CI_SCRIPT_DETERMINED_NEWEST_IDE_VERSION" 241 | 242 | create_folder "$ARDUINO_CI_SCRIPT_APPLICATION_FOLDER" 243 | 244 | # This runs the command contained in the $INSTALLED_IDE_VERSION_LIST_ARRAY string, thus declaring the array locally as $IDEversionListArray. This must be done in any function that uses the array 245 | # Dummy declaration to fix the "referenced but not assigned" warning. 246 | local IDEversionListArray 247 | eval "$INSTALLED_IDE_VERSION_LIST_ARRAY" 248 | 249 | # Determine whether any of the IDE versions to be installed require the creation of a virtual framebuffer (https://github.com/arduino/Arduino/blob/54264124b72eec40aaa22e327c16760f5e806c2a/build/shared/manpage.adoc#bugs) 250 | # This is necessary in Arduino IDE 1.6.13 and older (https://github.com/arduino/Arduino/pull/5578) when running on a headless system 251 | if [ -e /usr/bin/Xvfb ]; then 252 | local -r virtualFramebufferRequiredRegex='^1\.[56]\.' 253 | local IDEversion 254 | for IDEversion in "${IDEversionListArray[@]}"; do 255 | if [[ "$IDEversion" =~ $virtualFramebufferRequiredRegex ]]; then 256 | # based on https://learn.adafruit.com/continuous-integration-arduino-and-you/testing-your-project 257 | /sbin/start-stop-daemon --start $ARDUINO_CI_SCRIPT_QUIET_OPTION $ARDUINO_CI_SCRIPT_VERBOSITY_OPTION --pidfile /tmp/custom_xvfb_1.pid --make-pidfile --background --exec /usr/bin/Xvfb -- :1 -ac -screen 0 1280x1024x16 258 | sleep 3 259 | export DISPLAY=:1.0 260 | break 261 | fi 262 | done 263 | fi 264 | 265 | local IDEversion 266 | for IDEversion in "${IDEversionListArray[@]}"; do 267 | local IDEinstallFolder="$ARDUINO_CI_SCRIPT_APPLICATION_FOLDER/arduino-${IDEversion}" 268 | 269 | # Don't unnecessarily install the IDE 270 | if ! [[ -d "$IDEinstallFolder" ]]; then 271 | if [[ "$ARDUINO_CI_SCRIPT_VERBOSITY_LEVEL" -eq 0 ]]; then 272 | # If the download/installation process is going slowly when installing a lot of IDE versions this function may cause the build to fail due to exceeding Travis CI's 10 minutes without log output timeout so it's necessary to periodically print something. 273 | echo "Installing: $IDEversion" 274 | fi 275 | 276 | # Determine download file extension 277 | local tgzExtensionVersionsRegex="^1\.5\.[0-9]$" 278 | if [[ "$IDEversion" =~ $tgzExtensionVersionsRegex ]]; then 279 | # The download file extension prior to 1.6.0 is .tgz 280 | local downloadFileExtension="tgz" 281 | else 282 | local downloadFileExtension="tar.xz" 283 | fi 284 | 285 | if [[ "$IDEversion" == "hourly" ]]; then 286 | # Deal with the inaccurate name given to the hourly build download 287 | local downloadVersion="nightly" 288 | else 289 | local downloadVersion="$IDEversion" 290 | fi 291 | 292 | wget --no-verbose $ARDUINO_CI_SCRIPT_QUIET_OPTION --directory-prefix="${ARDUINO_CI_SCRIPT_TEMPORARY_FOLDER}/" "http://downloads.arduino.cc/arduino-${downloadVersion}-linux64.${downloadFileExtension}" 293 | tar --extract --directory="$ARDUINO_CI_SCRIPT_TEMPORARY_FOLDER" --file="${ARDUINO_CI_SCRIPT_TEMPORARY_FOLDER}/arduino-${downloadVersion}-linux64.${downloadFileExtension}" 294 | rm $ARDUINO_CI_SCRIPT_VERBOSITY_OPTION "${ARDUINO_CI_SCRIPT_TEMPORARY_FOLDER}/arduino-${downloadVersion}-linux64.${downloadFileExtension}" 295 | mv $ARDUINO_CI_SCRIPT_VERBOSITY_OPTION "${ARDUINO_CI_SCRIPT_TEMPORARY_FOLDER}/arduino-${downloadVersion}" "$IDEinstallFolder" 296 | fi 297 | done 298 | 299 | set_ide_preference "compiler.warning_level=all" 300 | 301 | # If a sketchbook location has been defined then set the location in the Arduino IDE preferences 302 | if [[ -d "$ARDUINO_CI_SCRIPT_SKETCHBOOK_FOLDER" ]]; then 303 | set_ide_preference "sketchbook.path=$ARDUINO_CI_SCRIPT_SKETCHBOOK_FOLDER" 304 | fi 305 | 306 | # Return errexit to the default state 307 | set +o errexit 308 | 309 | disable_verbosity 310 | } 311 | 312 | # Generate an array of Arduino IDE versions as a subset of the list provided in the base array defined by the start and end versions 313 | # This function allows the same code to be shared by install_ide and build_sketch. The generated array is "returned" as a global named "$ARDUINO_CI_SCRIPT_GENERATED_IDE_VERSION_LIST_ARRAY" 314 | function generate_ide_version_list_array() { 315 | local -r baseIDEversionArray="$1" 316 | local startIDEversion="$2" 317 | local endIDEversion="$3" 318 | 319 | # Convert "oldest" or "newest" to actual version numbers 320 | determine_ide_version_extremes "$baseIDEversionArray" 321 | if [[ "$startIDEversion" == "oldest" ]]; then 322 | startIDEversion="$ARDUINO_CI_SCRIPT_DETERMINED_OLDEST_IDE_VERSION" 323 | elif [[ "$startIDEversion" == "newest" ]]; then 324 | startIDEversion="$ARDUINO_CI_SCRIPT_DETERMINED_NEWEST_IDE_VERSION" 325 | fi 326 | 327 | if [[ "$endIDEversion" == "oldest" ]]; then 328 | endIDEversion="$ARDUINO_CI_SCRIPT_DETERMINED_OLDEST_IDE_VERSION" 329 | elif [[ "$endIDEversion" == "newest" ]]; then 330 | endIDEversion="$ARDUINO_CI_SCRIPT_DETERMINED_NEWEST_IDE_VERSION" 331 | fi 332 | 333 | if [[ "$startIDEversion" == "" || "$startIDEversion" == "all" ]]; then 334 | # Use the full base array 335 | ARDUINO_CI_SCRIPT_GENERATED_IDE_VERSION_LIST_ARRAY="$baseIDEversionArray" 336 | 337 | else 338 | local rawIDElist 339 | local -r IDEversionListRegex='\(' 340 | if [[ "$startIDEversion" =~ $IDEversionListRegex ]]; then 341 | # IDE versions list was supplied 342 | # Convert it to a temporary array 343 | local -r suppliedIDEversionListArray="${ARDUINO_CI_SCRIPT_IDE_VERSION_LIST_ARRAY_DECLARATION}${startIDEversion}" 344 | eval "$suppliedIDEversionListArray" 345 | local IDEversion 346 | for IDEversion in "${IDEversionListArray[@]}"; do 347 | # Convert any use of "oldest" or "newest" special version names to the actual version number 348 | if [[ "$IDEversion" == "oldest" ]]; then 349 | IDEversion="$ARDUINO_CI_SCRIPT_DETERMINED_OLDEST_IDE_VERSION" 350 | elif [[ "$IDEversion" == "newest" ]]; then 351 | IDEversion="$ARDUINO_CI_SCRIPT_DETERMINED_NEWEST_IDE_VERSION" 352 | fi 353 | # Add the version to the array 354 | rawIDElist="${rawIDElist} "'"'"$IDEversion"'"' 355 | done 356 | 357 | elif [[ "$endIDEversion" == "" ]]; then 358 | # Only a single version was specified 359 | rawIDElist="$rawIDElist"'"'"$startIDEversion"'"' 360 | 361 | else 362 | # A version range was specified 363 | eval "$baseIDEversionArray" 364 | local IDEversion 365 | for IDEversion in "${IDEversionListArray[@]}"; do 366 | if [[ "$IDEversion" == "$startIDEversion" ]]; then 367 | # Start of the list reached, set a flag 368 | local -r listIsStarted="true" 369 | fi 370 | 371 | if [[ "$listIsStarted" == "true" ]]; then 372 | # Add the version to the list 373 | rawIDElist="${rawIDElist} "'"'"$IDEversion"'"' 374 | fi 375 | 376 | if [[ "$IDEversion" == "$endIDEversion" ]]; then 377 | # End of the list was reached, exit the loop 378 | break 379 | fi 380 | done 381 | fi 382 | 383 | # Turn the raw IDE version list into an array 384 | declare -a -r rawIDElistArray="(${rawIDElist})" 385 | 386 | # Remove duplicates from list https://stackoverflow.com/a/13648438 387 | # shellcheck disable=SC2207 388 | readonly local uniqueIDElistArray=($(echo "${rawIDElistArray[@]}" | tr ' ' '\n' | sort --unique --version-sort | tr '\n' ' ')) 389 | 390 | # Generate ARDUINO_CI_SCRIPT_GENERATED_IDE_VERSION_LIST_ARRAY 391 | ARDUINO_CI_SCRIPT_GENERATED_IDE_VERSION_LIST_ARRAY="$ARDUINO_CI_SCRIPT_IDE_VERSION_LIST_ARRAY_DECLARATION"'(' 392 | for uniqueIDElistArrayIndex in "${!uniqueIDElistArray[@]}"; do 393 | ARDUINO_CI_SCRIPT_GENERATED_IDE_VERSION_LIST_ARRAY="${ARDUINO_CI_SCRIPT_GENERATED_IDE_VERSION_LIST_ARRAY} "'"'"${uniqueIDElistArray[$uniqueIDElistArrayIndex]}"'"' 394 | done 395 | ARDUINO_CI_SCRIPT_GENERATED_IDE_VERSION_LIST_ARRAY="$ARDUINO_CI_SCRIPT_GENERATED_IDE_VERSION_LIST_ARRAY"')' 396 | fi 397 | } 398 | 399 | # Determine the oldest and newest (non-hourly unless hourly is the only version on the list) IDE version in the provided array 400 | # The determined versions are "returned" by setting the global variables "$ARDUINO_CI_SCRIPT_DETERMINED_OLDEST_IDE_VERSION" and "$ARDUINO_CI_SCRIPT_DETERMINED_NEWEST_IDE_VERSION" 401 | function determine_ide_version_extremes() { 402 | local -r baseIDEversionArray="$1" 403 | 404 | # Reset the variables from any value they were assigned the last time the function was ran 405 | ARDUINO_CI_SCRIPT_DETERMINED_OLDEST_IDE_VERSION="" 406 | ARDUINO_CI_SCRIPT_DETERMINED_NEWEST_IDE_VERSION="" 407 | 408 | # Determine the oldest and newest (non-hourly) IDE version in the base array 409 | eval "$baseIDEversionArray" 410 | local IDEversion 411 | for IDEversion in "${IDEversionListArray[@]}"; do 412 | if [[ "$ARDUINO_CI_SCRIPT_DETERMINED_OLDEST_IDE_VERSION" == "" ]]; then 413 | ARDUINO_CI_SCRIPT_DETERMINED_OLDEST_IDE_VERSION="$IDEversion" 414 | fi 415 | if [[ "$ARDUINO_CI_SCRIPT_DETERMINED_NEWEST_IDE_VERSION" == "" || "$IDEversion" != "hourly" ]]; then 416 | ARDUINO_CI_SCRIPT_DETERMINED_NEWEST_IDE_VERSION="$IDEversion" 417 | fi 418 | done 419 | } 420 | 421 | function set_ide_preference() { 422 | local -r preferenceString="$1" 423 | 424 | # --pref option is only supported by Arduino IDE 1.5.6 and newer 425 | local -r unsupportedPrefOptionVersionsRegex="^1\.5\.[0-5]$" 426 | if ! [[ "$NEWEST_INSTALLED_IDE_VERSION" =~ $unsupportedPrefOptionVersionsRegex ]]; then 427 | install_ide_version "$NEWEST_INSTALLED_IDE_VERSION" 428 | 429 | # --save-prefs was added in Arduino IDE 1.5.8 430 | local -r unsupportedSavePrefsOptionVersionsRegex="^1\.5\.[6-7]$" 431 | if ! [[ "$NEWEST_INSTALLED_IDE_VERSION" =~ $unsupportedSavePrefsOptionVersionsRegex ]]; then 432 | # shellcheck disable=SC2086 433 | eval \"${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}/${ARDUINO_CI_SCRIPT_IDE_INSTALLATION_FOLDER}/${ARDUINO_CI_SCRIPT_ARDUINO_COMMAND}\" --pref "$preferenceString" --save-prefs "$ARDUINO_CI_SCRIPT_VERBOSITY_REDIRECT" 434 | else 435 | # Arduino IDE 1.5.6 - 1.5.7 load the GUI if you only set preferences without doing a verify. So I am doing an unnecessary verification just to set the preferences in those versions. Definitely a hack but I prefer to keep the preferences setting code all here instead of cluttering build_sketch and this will pretty much never be used. 436 | # shellcheck disable=SC2086 437 | eval \"${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}/${ARDUINO_CI_SCRIPT_IDE_INSTALLATION_FOLDER}/${ARDUINO_CI_SCRIPT_ARDUINO_COMMAND}\" --pref "$preferenceString" --verify "${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}/arduino/examples/01.Basics/BareMinimum/BareMinimum.ino" "$ARDUINO_CI_SCRIPT_VERBOSITY_REDIRECT" 438 | fi 439 | fi 440 | } 441 | 442 | function install_ide_version() { 443 | local -r IDEversion="$1" 444 | 445 | # Create a symbolic link so that the Arduino IDE can always be referenced by the user from the same path no matter which version is being used. 446 | if [[ "$OSTYPE" == "cygwin" ]] || [[ "$OSTYPE" == "msys" ]]; then 447 | # git-bash's ln just does a copy instead of making a symlink, which takes forever and fails when the target folder exists (despite --force), which takes forever. 448 | # Therefore, use the native Windows command mklink to create a directory junction instead. 449 | # Using a directory junction instead of symlink because supposedly a symlink requires admin privileges. 450 | 451 | # Windows doesn't seem to provide any way to overwrite directory junctions 452 | if [[ -d "${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}/${ARDUINO_CI_SCRIPT_IDE_INSTALLATION_FOLDER}" ]]; then 453 | rm --recursive --force "${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}/${ARDUINO_CI_SCRIPT_IDE_INSTALLATION_FOLDER:?}" 454 | fi 455 | # https://stackoverflow.com/a/25394801 456 | cmd <<<"mklink /J \"${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}\\${ARDUINO_CI_SCRIPT_IDE_INSTALLATION_FOLDER}\" \"${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER//\//\\}\\arduino-${IDEversion}\"" >/dev/null 457 | else 458 | ln --symbolic --force --no-dereference $ARDUINO_CI_SCRIPT_VERBOSITY_OPTION "${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}/arduino-${IDEversion}" "${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}/${ARDUINO_CI_SCRIPT_IDE_INSTALLATION_FOLDER}" 459 | fi 460 | } 461 | 462 | # Install hardware packages 463 | function install_package() { 464 | enable_verbosity 465 | 466 | set -o errexit 467 | 468 | local -r URLregex="://" 469 | if [[ "$1" =~ $URLregex ]]; then 470 | # First argument is a URL, do a manual hardware package installation 471 | # Note: Assumes the package is in the root of the download and has the correct folder structure (e.g. architecture folder added in Arduino IDE 1.5+) 472 | 473 | local -r packageURL="$1" 474 | 475 | # Create the hardware folder if it doesn't exist 476 | create_folder "${ARDUINO_CI_SCRIPT_SKETCHBOOK_FOLDER}/hardware" 477 | 478 | if [[ "$packageURL" =~ \.git$ ]]; then 479 | # Clone the repository 480 | local -r branchName="$2" 481 | 482 | local -r previousFolder="$PWD" 483 | cd "${ARDUINO_CI_SCRIPT_SKETCHBOOK_FOLDER}/hardware" 484 | 485 | if [[ "$branchName" == "" ]]; then 486 | git clone --quiet "$packageURL" 487 | else 488 | git clone --quiet --branch "$branchName" "$packageURL" 489 | fi 490 | cd "$previousFolder" 491 | else 492 | local -r previousFolder="$PWD" 493 | cd "$ARDUINO_CI_SCRIPT_TEMPORARY_FOLDER" 494 | 495 | # Delete everything from the temporary folder 496 | find ./ -mindepth 1 -delete 497 | 498 | # Download the package 499 | wget --no-verbose $ARDUINO_CI_SCRIPT_QUIET_OPTION "$packageURL" 500 | 501 | # Uncompress the package 502 | extract ./*.* 503 | 504 | # Delete all files from the temporary folder 505 | find ./ -maxdepth 1 -type f -delete 506 | 507 | # Install the package 508 | mv $ARDUINO_CI_SCRIPT_VERBOSITY_OPTION ./* "${ARDUINO_CI_SCRIPT_SKETCHBOOK_FOLDER}/hardware/" 509 | cd "$previousFolder" 510 | fi 511 | 512 | elif [[ "$1" == "" ]]; then 513 | # Install hardware package from this repository 514 | # https://docs.travis-ci.com/user/environment-variables#Global-Variables 515 | local packageName 516 | packageName="$(echo "$TRAVIS_REPO_SLUG" | cut -d'/' -f 2)" 517 | mkdir --parents $ARDUINO_CI_SCRIPT_VERBOSITY_OPTION "${ARDUINO_CI_SCRIPT_SKETCHBOOK_FOLDER}/hardware/$packageName" 518 | local -r previousFolder="$PWD" 519 | cd "$TRAVIS_BUILD_DIR" 520 | cp --recursive $ARDUINO_CI_SCRIPT_VERBOSITY_OPTION ./* "${ARDUINO_CI_SCRIPT_SKETCHBOOK_FOLDER}/hardware/${packageName}" 521 | # * doesn't copy .travis.yml but that file will be present in the user's installation so it should be there for the tests too 522 | cp $ARDUINO_CI_SCRIPT_VERBOSITY_OPTION "${TRAVIS_BUILD_DIR}/.travis.yml" "${ARDUINO_CI_SCRIPT_SKETCHBOOK_FOLDER}/hardware/${packageName}" 523 | cd "$previousFolder" 524 | 525 | else 526 | # Install package via Boards Manager 527 | 528 | local -r packageID="$1" 529 | local -r packageURL="$2" 530 | 531 | # Check if Arduino IDE is installed 532 | if [[ "$INSTALLED_IDE_VERSION_LIST_ARRAY" == "" ]]; then 533 | echo "ERROR: Installing a hardware package via Boards Manager requires the Arduino IDE to be installed. Please call install_ide before this command." 534 | return_handler "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 535 | fi 536 | 537 | # Check if the newest installed IDE version supports --install-boards 538 | local -r unsupportedInstallBoardsOptionVersionsRange1regex="^1\.5\.[0-9]$" 539 | local -r unsupportedInstallBoardsOptionVersionsRange2regex="^1\.6\.[0-3]$" 540 | if [[ "$NEWEST_INSTALLED_IDE_VERSION" =~ $unsupportedInstallBoardsOptionVersionsRange1regex ]] || [[ "$NEWEST_INSTALLED_IDE_VERSION" =~ $unsupportedInstallBoardsOptionVersionsRange2regex ]]; then 541 | echo "ERROR: --install-boards option is not supported by the newest version of the Arduino IDE you have installed. You must have Arduino IDE 1.6.4 or newer installed to use this function." 542 | return_handler "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 543 | else 544 | # Temporarily install the latest IDE version to use for the package installation 545 | install_ide_version "$NEWEST_INSTALLED_IDE_VERSION" 546 | 547 | # If defined add the boards manager URL to preferences 548 | if [[ "$packageURL" != "" ]]; then 549 | # Get the current Additional Boards Manager URLs preference value so it won't be overwritten when the new URL is added 550 | local priorBoardsmanagerAdditionalURLs 551 | local getPrefExitStatus 552 | # arduino --get-pref returns 4 when the preference does not exist, which is an acceptable circumstance. So it's necessary to unset errexit 553 | set +o errexit 554 | if [[ "$ARDUINO_CI_SCRIPT_VERBOSITY_LEVEL" -eq 0 ]]; then 555 | priorBoardsmanagerAdditionalURLs=$( 556 | "${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}/${ARDUINO_CI_SCRIPT_IDE_INSTALLATION_FOLDER}/${ARDUINO_CI_SCRIPT_ARDUINO_COMMAND}" --get-pref boardsmanager.additional.urls 2>/dev/null | tail --lines=1 557 | exit "${PIPESTATUS[0]}" 558 | ) 559 | getPrefExitStatus="$?" 560 | elif [[ "$ARDUINO_CI_SCRIPT_VERBOSITY_LEVEL" -eq 1 ]]; then 561 | priorBoardsmanagerAdditionalURLs=$( 562 | "${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}/${ARDUINO_CI_SCRIPT_IDE_INSTALLATION_FOLDER}/${ARDUINO_CI_SCRIPT_ARDUINO_COMMAND}" --get-pref boardsmanager.additional.urls | tail --lines=1 563 | exit "${PIPESTATUS[0]}" 564 | ) 565 | getPrefExitStatus="$?" 566 | else 567 | priorBoardsmanagerAdditionalURLs=$( 568 | "${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}/${ARDUINO_CI_SCRIPT_IDE_INSTALLATION_FOLDER}/${ARDUINO_CI_SCRIPT_ARDUINO_COMMAND}" --get-pref boardsmanager.additional.urls | tee /dev/tty | tail --lines=1 569 | exit "${PIPESTATUS[0]}" 570 | ) 571 | getPrefExitStatus="$?" 572 | fi 573 | set -o errexit 574 | 575 | if [[ "$getPrefExitStatus" == "4" ]]; then 576 | # No boardsmanager.additional.urls preference was set. This causes priorBoardsmanagerAdditionalURLs to have a garbage value with Arduino IDE 1.8.10 and newer. 577 | priorBoardsmanagerAdditionalURLs="" 578 | fi 579 | 580 | local -r blankregex="^[ ]*$" 581 | if [[ "$priorBoardsmanagerAdditionalURLs" =~ $blankregex ]]; then 582 | # There is no previous Additional Boards Manager URLs preference value 583 | local boardsmanagerAdditionalURLs="$packageURL" 584 | else 585 | # There is a previous Additional Boards Manager URLs preference value so append the new one to the end of it 586 | local boardsmanagerAdditionalURLs="${priorBoardsmanagerAdditionalURLs},${packageURL}" 587 | fi 588 | 589 | # grep returns 1 when a line matches the regular expression so it's necessary to unset errexit 590 | set +o errexit 591 | # shellcheck disable=SC2086 592 | eval \"${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}/${ARDUINO_CI_SCRIPT_IDE_INSTALLATION_FOLDER}/${ARDUINO_CI_SCRIPT_ARDUINO_COMMAND}\" --pref boardsmanager.additional.urls="$boardsmanagerAdditionalURLs" --save-prefs "$ARDUINO_CI_SCRIPT_VERBOSITY_REDIRECT" | tr --complement --delete '[:print:]\n\t' | tr --squeeze-repeats '\n' | grep --extended-regexp --invert-match "$ARDUINO_CI_SCRIPT_ARDUINO_OUTPUT_FILTER_REGEX" 593 | local -r arduinoPreferenceSettingExitStatus="${PIPESTATUS[0]}" 594 | set -o errexit 595 | # this is required because otherwise the exit status of arduino is ignored 596 | if [[ "$arduinoPreferenceSettingExitStatus" != "$ARDUINO_CI_SCRIPT_SUCCESS_EXIT_STATUS" ]]; then 597 | return_handler "$arduinoPreferenceSettingExitStatus" 598 | fi 599 | fi 600 | 601 | # Install the package 602 | # grep returns 1 when a line matches the regular expression so it's necessary to unset errexit 603 | set +o errexit 604 | # shellcheck disable=SC2086 605 | eval \"${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}/${ARDUINO_CI_SCRIPT_IDE_INSTALLATION_FOLDER}/${ARDUINO_CI_SCRIPT_ARDUINO_COMMAND}\" --install-boards "$packageID" "$ARDUINO_CI_SCRIPT_VERBOSITY_REDIRECT" | tr --complement --delete '[:print:]\n\t' | tr --squeeze-repeats '\n' | grep --extended-regexp --invert-match "$ARDUINO_CI_SCRIPT_ARDUINO_OUTPUT_FILTER_REGEX" 606 | local -r arduinoInstallPackageExitStatus="${PIPESTATUS[0]}" 607 | set -o errexit 608 | # this is required because otherwise the exit status of arduino is ignored 609 | if [[ "$arduinoInstallPackageExitStatus" != "$ARDUINO_CI_SCRIPT_SUCCESS_EXIT_STATUS" ]]; then 610 | return_handler "$arduinoPreferenceSettingExitStatus" 611 | fi 612 | 613 | fi 614 | fi 615 | 616 | set +o errexit 617 | 618 | disable_verbosity 619 | } 620 | 621 | function install_library() { 622 | enable_verbosity 623 | 624 | set -o errexit 625 | 626 | local -r libraryIdentifier="$1" 627 | 628 | # Create the libraries folder if it doesn't already exist 629 | create_folder "${ARDUINO_CI_SCRIPT_SKETCHBOOK_FOLDER}/libraries" 630 | 631 | local -r URLregex="://" 632 | if [[ "$libraryIdentifier" =~ $URLregex ]]; then 633 | # The argument is a URL 634 | # Note: this assumes the library is in the root of the file 635 | if [[ "$libraryIdentifier" =~ \.git$ ]]; then 636 | # Clone the repository 637 | local -r branchName="$2" 638 | local -r newFolderName="$3" 639 | 640 | local -r previousFolder="$PWD" 641 | cd "${ARDUINO_CI_SCRIPT_SKETCHBOOK_FOLDER}/libraries" 642 | 643 | if [[ "$branchName" == "" && "$newFolderName" == "" ]]; then 644 | git clone --quiet "$libraryIdentifier" 645 | elif [[ "$branchName" == "" ]]; then 646 | git clone --quiet "$libraryIdentifier" "$newFolderName" 647 | elif [[ "$newFolderName" == "" ]]; then 648 | git clone --quiet --branch "$branchName" "$libraryIdentifier" 649 | else 650 | git clone --quiet --branch "$branchName" "$libraryIdentifier" "$newFolderName" 651 | fi 652 | cd "$previousFolder" 653 | else 654 | # Assume it's a compressed file 655 | local -r newFolderName="$2" 656 | # Download the file to the temporary folder 657 | local -r previousFolder="$PWD" 658 | cd "$ARDUINO_CI_SCRIPT_TEMPORARY_FOLDER" 659 | 660 | # Delete everything from the temporary folder 661 | find ./ -mindepth 1 -delete 662 | 663 | wget --no-verbose $ARDUINO_CI_SCRIPT_QUIET_OPTION "$libraryIdentifier" 664 | 665 | extract ./*.* 666 | 667 | # Delete all files from the temporary folder 668 | find ./ -maxdepth 1 -type f -delete 669 | 670 | # Install the library 671 | mv $ARDUINO_CI_SCRIPT_VERBOSITY_OPTION ./* "${ARDUINO_CI_SCRIPT_SKETCHBOOK_FOLDER}/libraries/${newFolderName}" 672 | cd "$previousFolder" 673 | fi 674 | 675 | elif [[ "$libraryIdentifier" == "" ]]; then 676 | # Install library from the repository 677 | # https://docs.travis-ci.com/user/environment-variables#Global-Variables 678 | local libraryName 679 | libraryName="$(echo "$TRAVIS_REPO_SLUG" | cut -d'/' -f 2)" 680 | mkdir --parents $ARDUINO_CI_SCRIPT_VERBOSITY_OPTION "${ARDUINO_CI_SCRIPT_SKETCHBOOK_FOLDER}/libraries/$libraryName" 681 | local -r previousFolder="$PWD" 682 | cd "$TRAVIS_BUILD_DIR" 683 | cp --recursive $ARDUINO_CI_SCRIPT_VERBOSITY_OPTION ./* "${ARDUINO_CI_SCRIPT_SKETCHBOOK_FOLDER}/libraries/${libraryName}" 684 | # * doesn't copy .travis.yml but that file will be present in the user's installation so it should be there for the tests too 685 | cp $ARDUINO_CI_SCRIPT_VERBOSITY_OPTION "${TRAVIS_BUILD_DIR}/.travis.yml" "${ARDUINO_CI_SCRIPT_SKETCHBOOK_FOLDER}/libraries/${libraryName}" 686 | cd "$previousFolder" 687 | 688 | else 689 | # Install a library that is part of the Library Manager index 690 | 691 | # Check if Arduino IDE is installed 692 | if [[ "$INSTALLED_IDE_VERSION_LIST_ARRAY" == "" ]]; then 693 | echo "ERROR: Installing a library via Library Manager requires the Arduino IDE to be installed. Please call install_ide before this command." 694 | return_handler "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 695 | fi 696 | 697 | # Check if the newest installed IDE version supports --install-library 698 | local -r unsupportedInstallLibraryOptionVersionsRange1regex="^1\.5\.[0-9]$" 699 | local -r unsupportedInstallLibraryOptionVersionsRange2regex="^1\.6\.[0-3]$" 700 | if [[ "$NEWEST_INSTALLED_IDE_VERSION" =~ $unsupportedInstallLibraryOptionVersionsRange1regex ]] || [[ "$NEWEST_INSTALLED_IDE_VERSION" =~ $unsupportedInstallLibraryOptionVersionsRange2regex ]]; then 701 | echo "ERROR: --install-library option is not supported by the newest version of the Arduino IDE you have installed. You must have Arduino IDE 1.6.4 or newer installed to use this function." 702 | return_handler "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 703 | else 704 | local -r libraryName="$1" 705 | 706 | # Temporarily install the latest IDE version to use for the library installation 707 | install_ide_version "$NEWEST_INSTALLED_IDE_VERSION" 708 | 709 | # Install the library 710 | # shellcheck disable=SC2086 711 | eval \"${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}/${ARDUINO_CI_SCRIPT_IDE_INSTALLATION_FOLDER}/${ARDUINO_CI_SCRIPT_ARDUINO_COMMAND}\" --install-library "$libraryName" "$ARDUINO_CI_SCRIPT_VERBOSITY_REDIRECT" 712 | 713 | fi 714 | fi 715 | 716 | set +o errexit 717 | 718 | disable_verbosity 719 | } 720 | 721 | # Extract common file formats 722 | # https://github.com/xvoland/Extract 723 | function extract() { 724 | if [ -z "$1" ]; then 725 | # display usage if no parameters given 726 | echo "Usage: extract ." 727 | echo " extract [path/file_name_2.ext] [path/file_name_3.ext]" 728 | return "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 729 | else 730 | local filename 731 | for filename in "$@"; do 732 | if [ -f "$filename" ]; then 733 | case "${filename%,}" in 734 | *.tar.bz2 | *.tar.gz | *.tar.xz | *.tbz2 | *.tgz | *.txz | *.tar) 735 | tar --extract --file="$filename" 736 | ;; 737 | *.lzma) 738 | unlzma $ARDUINO_CI_SCRIPT_QUIET_OPTION ./"$filename" 739 | ;; 740 | *.bz2) 741 | bunzip2 $ARDUINO_CI_SCRIPT_QUIET_OPTION ./"$filename" 742 | ;; 743 | *.rar) 744 | eval unrar x -ad ./"$filename" "$ARDUINO_CI_SCRIPT_VERBOSITY_REDIRECT" 745 | ;; 746 | *.gz) 747 | gunzip ./"$filename" 748 | ;; 749 | *.zip) 750 | unzip -qq ./"$filename" 751 | ;; 752 | *.z) 753 | eval uncompress ./"$filename" "$ARDUINO_CI_SCRIPT_VERBOSITY_REDIRECT" 754 | ;; 755 | *.7z | *.arj | *.cab | *.chm | *.deb | *.dmg | *.iso | *.lzh | *.msi | *.rpm | *.udf | *.wim | *.xar) 756 | 7z x ./"$filename" 757 | ;; 758 | *.xz) 759 | unxz $ARDUINO_CI_SCRIPT_QUIET_OPTION ./"$filename" 760 | ;; 761 | *.exe) 762 | cabextract $ARDUINO_CI_SCRIPT_QUIET_OPTION ./"$filename" 763 | ;; 764 | *) 765 | echo "extract: '$filename' - unknown archive method" 766 | return "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 767 | ;; 768 | esac 769 | else 770 | echo "extract: '$filename' - file does not exist" 771 | return "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 772 | fi 773 | done 774 | fi 775 | } 776 | 777 | function set_verbose_output_during_compilation() { 778 | enable_verbosity 779 | 780 | local -r verboseOutputDuringCompilation="$1" 781 | if [[ "$verboseOutputDuringCompilation" == "true" ]]; then 782 | ARDUINO_CI_SCRIPT_DETERMINED_VERBOSE_BUILD="--verbose" 783 | else 784 | ARDUINO_CI_SCRIPT_DETERMINED_VERBOSE_BUILD="" 785 | fi 786 | 787 | disable_verbosity 788 | } 789 | 790 | # Verify the sketch 791 | function build_sketch() { 792 | enable_verbosity 793 | 794 | local -r sketchPath="$1" 795 | local -r boardID="$2" 796 | local -r allowFail="$3" 797 | local -r startIDEversion="$4" 798 | local -r endIDEversion="$5" 799 | 800 | # Set default value for buildSketchExitStatus 801 | local buildSketchExitStatus="$ARDUINO_CI_SCRIPT_SUCCESS_EXIT_STATUS" 802 | 803 | generate_ide_version_list_array "$INSTALLED_IDE_VERSION_LIST_ARRAY" "$startIDEversion" "$endIDEversion" 804 | 805 | if [[ "$ARDUINO_CI_SCRIPT_GENERATED_IDE_VERSION_LIST_ARRAY" == "$ARDUINO_CI_SCRIPT_IDE_VERSION_LIST_ARRAY_DECLARATION"'()' ]]; then 806 | echo "ERROR: The IDE version(s) specified are not installed" 807 | buildSketchExitStatus="$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 808 | else 809 | eval "$ARDUINO_CI_SCRIPT_GENERATED_IDE_VERSION_LIST_ARRAY" 810 | local IDEversion 811 | for IDEversion in "${IDEversionListArray[@]}"; do 812 | # Install the IDE 813 | # This must be done before searching for sketches in case the path specified is in the Arduino IDE installation folder 814 | install_ide_version "$IDEversion" 815 | 816 | # The package_index files installed by some versions of the IDE (1.6.5, 1.6.5) can cause compilation to fail for other versions (1.6.5-r4, 1.6.5-r5). Attempting to install a dummy package ensures that the correct version of those files will be installed before the sketch verification. 817 | # Check if the newest installed IDE version supports --install-boards 818 | local unsupportedInstallBoardsOptionVersionsRange1regex="^1\.5\.[0-9]$" 819 | local unsupportedInstallBoardsOptionVersionsRange2regex="^1\.6\.[0-3]$" 820 | if ! [[ "$IDEversion" =~ $unsupportedInstallBoardsOptionVersionsRange1regex ]] && ! [[ "$IDEversion" =~ $unsupportedInstallBoardsOptionVersionsRange2regex ]]; then 821 | # shellcheck disable=SC2086 822 | eval \"${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}/${ARDUINO_CI_SCRIPT_IDE_INSTALLATION_FOLDER}/${ARDUINO_CI_SCRIPT_ARDUINO_COMMAND}\" --install-boards arduino:dummy "$ARDUINO_CI_SCRIPT_VERBOSITY_REDIRECT" 823 | if [[ "$ARDUINO_CI_SCRIPT_VERBOSITY_LEVEL" -gt 1 ]]; then 824 | # The warning is printed to stdout 825 | echo "NOTE: The warning above \"Selected board is not available\" is caused intentionally and does not indicate a problem." 826 | fi 827 | fi 828 | 829 | if [[ "$sketchPath" =~ \.ino$ ]] || [[ "$sketchPath" =~ \.pde$ ]]; then 830 | # A sketch was specified 831 | if ! [[ -f "$sketchPath" ]]; then 832 | echo "ERROR: Specified sketch: $sketchPath doesn't exist" 833 | buildSketchExitStatus="$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 834 | elif ! build_this_sketch "$sketchPath" "$boardID" "$IDEversion" "$allowFail"; then 835 | # build_this_sketch returned a non-zero exit status 836 | buildSketchExitStatus="$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 837 | fi 838 | else 839 | # Search for all sketches in the path and put them in an array 840 | local sketchFound="false" 841 | # https://github.com/koalaman/shellcheck/wiki/SC2207 842 | declare -a sketches 843 | mapfile -t sketches < <(find "$sketchPath" -name "*.pde" -o -name "*.ino") 844 | local sketchName 845 | for sketchName in "${sketches[@]}"; do 846 | # Only verify the sketch that matches the name of the sketch folder, otherwise it will cause redundant verifications for sketches that have multiple .ino files 847 | local sketchFolder 848 | sketchFolder="$(echo "$sketchName" | rev | cut -d'/' -f 2 | rev)" 849 | local sketchNameWithoutPathWithExtension 850 | sketchNameWithoutPathWithExtension="$(echo "$sketchName" | rev | cut -d'/' -f 1 | rev)" 851 | local sketchNameWithoutPathWithoutExtension 852 | sketchNameWithoutPathWithoutExtension="${sketchNameWithoutPathWithExtension%.*}" 853 | if [[ "$sketchFolder" == "$sketchNameWithoutPathWithoutExtension" ]]; then 854 | sketchFound="true" 855 | if ! build_this_sketch "$sketchName" "$boardID" "$IDEversion" "$allowFail"; then 856 | # build_this_sketch returned a non-zero exit status 857 | buildSketchExitStatus="$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 858 | fi 859 | fi 860 | done 861 | 862 | if [[ "$sketchFound" == "false" ]]; then 863 | echo "ERROR: No valid sketches were found in the specified path" 864 | buildSketchExitStatus="$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 865 | fi 866 | fi 867 | done 868 | fi 869 | 870 | disable_verbosity 871 | 872 | return $buildSketchExitStatus 873 | } 874 | 875 | function build_this_sketch() { 876 | # Fold this section of output in the Travis CI build log to make it easier to read 877 | echo -e "travis_fold:start:build_sketch" 878 | 879 | local -r sketchName="$1" 880 | local -r boardID="$2" 881 | local -r IDEversion="$3" 882 | local -r allowFail="$4" 883 | 884 | # Produce a useful label for the fold in the Travis log for this function call 885 | echo "build_sketch $sketchName $boardID $allowFail $IDEversion" 886 | 887 | # Arduino IDE 1.8.0 and 1.8.1 fail to verify a sketch if the absolute path to it is not specified 888 | # http://stackoverflow.com/a/3915420/7059512 889 | local absoluteSketchName 890 | absoluteSketchName="$( 891 | cd "$(dirname "$1")" 892 | pwd 893 | )/$(basename "$1")" 894 | 895 | # Define a dummy value for arduinoExitStatus so that the while loop will run at least once 896 | local arduinoExitStatus=255 897 | # Retry the verification if arduino returns an exit status that indicates there may have been a temporary error not caused by a bug in the sketch or the arduino command 898 | while [[ $arduinoExitStatus -gt $ARDUINO_CI_SCRIPT_HIGHEST_ACCEPTABLE_ARDUINO_EXIT_STATUS && $verifyCount -le $ARDUINO_CI_SCRIPT_SKETCH_VERIFY_RETRIES ]]; do 899 | # Verify the sketch 900 | # shellcheck disable=SC2086 901 | "${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}/${ARDUINO_CI_SCRIPT_IDE_INSTALLATION_FOLDER}/${ARDUINO_CI_SCRIPT_ARDUINO_COMMAND}" $ARDUINO_CI_SCRIPT_DETERMINED_VERBOSE_BUILD --verify "$absoluteSketchName" --board "$boardID" 2>&1 | tr --complement --delete '[:print:]\n\t' | tr --squeeze-repeats '\n' | grep --extended-regexp --invert-match "$ARDUINO_CI_SCRIPT_ARDUINO_OUTPUT_FILTER_REGEX" | tee "$ARDUINO_CI_SCRIPT_VERIFICATION_OUTPUT_FILENAME" 902 | local arduinoExitStatus="${PIPESTATUS[0]}" 903 | local verifyCount=$((verifyCount + 1)) 904 | done 905 | 906 | if [[ "$arduinoExitStatus" != "$ARDUINO_CI_SCRIPT_SUCCESS_EXIT_STATUS" ]]; then 907 | # Sketch verification failed 908 | local buildThisSketchExitStatus="$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 909 | else 910 | # Sketch verification succeeded 911 | local buildThisSketchExitStatus="$ARDUINO_CI_SCRIPT_SUCCESS_EXIT_STATUS" 912 | 913 | # Parse through the output from the sketch verification to count warnings and determine the compile size 914 | local warningCount=0 915 | local boardIssueCount=0 916 | local libraryIssueCount=0 917 | while read -r outputFileLine; do 918 | # Determine program storage memory usage 919 | local programStorageRegex="Sketch uses ([0-9,]+) *" 920 | if [[ "$outputFileLine" =~ $programStorageRegex ]] >/dev/null; then 921 | local -r programStorageWithComma=${BASH_REMATCH[1]} 922 | fi 923 | 924 | # Determine dynamic memory usage 925 | local dynamicMemoryRegex="Global variables use ([0-9,]+) *" 926 | if [[ "$outputFileLine" =~ $dynamicMemoryRegex ]] >/dev/null; then 927 | local -r dynamicMemoryWithComma=${BASH_REMATCH[1]} 928 | fi 929 | 930 | # Increment warning count 931 | local warningRegex="warning: " 932 | if [[ "$outputFileLine" =~ $warningRegex ]] >/dev/null; then 933 | warningCount=$((warningCount + 1)) 934 | fi 935 | 936 | # Check for board issues 937 | local bootloaderMissingRegex="Bootloader file specified but missing: " 938 | if [[ "$outputFileLine" =~ $bootloaderMissingRegex ]] >/dev/null; then 939 | local boardIssue="missing bootloader" 940 | boardIssueCount=$((boardIssueCount + 1)) 941 | fi 942 | 943 | local boardsDotTxtMissingRegex="Could not find boards.txt" 944 | if [[ "$outputFileLine" =~ $boardsDotTxtMissingRegex ]] >/dev/null; then 945 | local boardIssue="Could not find boards.txt" 946 | boardIssueCount=$((boardIssueCount + 1)) 947 | fi 948 | 949 | local buildDotBoardNotDefinedRegex="doesn't define a 'build.board' preference" 950 | if [[ "$outputFileLine" =~ $buildDotBoardNotDefinedRegex ]] >/dev/null; then 951 | local boardIssue="doesn't define a 'build.board' preference" 952 | boardIssueCount=$((boardIssueCount + 1)) 953 | fi 954 | 955 | # Check for library issues 956 | # This is the generic "invalid library" warning that doesn't specify the reason 957 | local invalidLibrarRegex1="Invalid library found in" 958 | local invalidLibrarRegex2="from library$" 959 | if [[ "$outputFileLine" =~ $invalidLibrarRegex1 ]] && ! [[ "$outputFileLine" =~ $invalidLibrarRegex2 ]] >/dev/null; then 960 | local libraryIssue="Invalid library" 961 | libraryIssueCount=$((libraryIssueCount + 1)) 962 | fi 963 | 964 | local missingNameRegex="Invalid library found in .* Missing 'name' from library" 965 | if [[ "$outputFileLine" =~ $missingNameRegex ]] >/dev/null; then 966 | local libraryIssue="Missing 'name' from library" 967 | libraryIssueCount=$((libraryIssueCount + 1)) 968 | fi 969 | 970 | local missingVersionRegex="Invalid library found in .* Missing 'version' from library" 971 | if [[ "$outputFileLine" =~ $missingVersionRegex ]] >/dev/null; then 972 | local libraryIssue="Missing 'version' from library" 973 | libraryIssueCount=$((libraryIssueCount + 1)) 974 | fi 975 | 976 | local missingAuthorRegex="Invalid library found in .* Missing 'author' from library" 977 | if [[ "$outputFileLine" =~ $missingAuthorRegex ]] >/dev/null; then 978 | local libraryIssue="Missing 'author' from library" 979 | libraryIssueCount=$((libraryIssueCount + 1)) 980 | fi 981 | 982 | local missingMaintainerRegex="Invalid library found in .* Missing 'maintainer' from library" 983 | if [[ "$outputFileLine" =~ $missingMaintainerRegex ]] >/dev/null; then 984 | local libraryIssue="Missing 'maintainer' from library" 985 | libraryIssueCount=$((libraryIssueCount + 1)) 986 | fi 987 | 988 | local missingSentenceRegex="Invalid library found in .* Missing 'sentence' from library" 989 | if [[ "$outputFileLine" =~ $missingSentenceRegex ]] >/dev/null; then 990 | local libraryIssue="Missing 'sentence' from library" 991 | libraryIssueCount=$((libraryIssueCount + 1)) 992 | fi 993 | 994 | local missingParagraphRegex="Invalid library found in .* Missing 'paragraph' from library" 995 | if [[ "$outputFileLine" =~ $missingParagraphRegex ]] >/dev/null; then 996 | local libraryIssue="Missing 'paragraph' from library" 997 | libraryIssueCount=$((libraryIssueCount + 1)) 998 | fi 999 | 1000 | local missingURLregex="Invalid library found in .* Missing 'url' from library" 1001 | if [[ "$outputFileLine" =~ $missingURLregex ]] >/dev/null; then 1002 | local libraryIssue="Missing 'url' from library" 1003 | libraryIssueCount=$((libraryIssueCount + 1)) 1004 | fi 1005 | 1006 | local invalidVersionRegex="Invalid version found:" 1007 | if [[ "$outputFileLine" =~ $invalidVersionRegex ]] >/dev/null; then 1008 | local libraryIssue="Invalid version found:" 1009 | libraryIssueCount=$((libraryIssueCount + 1)) 1010 | fi 1011 | 1012 | local invalidCategoryRegex="is not valid. Setting to 'Uncategorized'" 1013 | if [[ "$outputFileLine" =~ $invalidCategoryRegex ]] >/dev/null; then 1014 | local libraryIssue="Invalid category" 1015 | libraryIssueCount=$((libraryIssueCount + 1)) 1016 | fi 1017 | 1018 | local spuriousFolderRegex="WARNING: Spurious" 1019 | if [[ "$outputFileLine" =~ $spuriousFolderRegex ]] >/dev/null; then 1020 | local libraryIssue="Spurious folder" 1021 | libraryIssueCount=$((libraryIssueCount + 1)) 1022 | fi 1023 | 1024 | done <"$ARDUINO_CI_SCRIPT_VERIFICATION_OUTPUT_FILENAME" 1025 | 1026 | rm $ARDUINO_CI_SCRIPT_VERBOSITY_OPTION "$ARDUINO_CI_SCRIPT_VERIFICATION_OUTPUT_FILENAME" 1027 | 1028 | # Remove the stupid comma from the memory values if present 1029 | local -r programStorage=${programStorageWithComma//,/} 1030 | local -r dynamicMemory=${dynamicMemoryWithComma//,/} 1031 | 1032 | if [[ "$boardIssue" != "" && "$ARDUINO_CI_SCRIPT_TEST_BOARD" == "true" ]]; then 1033 | # There was a board issue and board testing is enabled so fail the build 1034 | local buildThisSketchExitStatus="$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 1035 | fi 1036 | 1037 | if [[ "$libraryIssue" != "" && "$ARDUINO_CI_SCRIPT_TEST_LIBRARY" == "true" ]]; then 1038 | # There was a library issue and library testing is enabled so fail the build 1039 | local buildThisSketchExitStatus="$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 1040 | fi 1041 | fi 1042 | 1043 | # Add the build data to the report file 1044 | echo "$(date -u "+%Y-%m-%d %H:%M:%S")"$'\t'"$TRAVIS_BUILD_NUMBER"$'\t'"$TRAVIS_JOB_NUMBER"$'\t'"https://travis-ci.org/${TRAVIS_REPO_SLUG}/jobs/${TRAVIS_JOB_ID}"$'\t'"$TRAVIS_EVENT_TYPE"$'\t'"$TRAVIS_ALLOW_FAILURE"$'\t'"$TRAVIS_PULL_REQUEST"$'\t'"$TRAVIS_BRANCH"$'\t'"$TRAVIS_COMMIT"$'\t'"$TRAVIS_COMMIT_RANGE"$'\t'"${TRAVIS_COMMIT_MESSAGE%%$'\n'*}"$'\t'"$sketchName"$'\t'"$boardID"$'\t'"$IDEversion"$'\t'"$programStorage"$'\t'"$dynamicMemory"$'\t'"$warningCount"$'\t'"$allowFail"$'\t'"$arduinoExitStatus"$'\t'"$boardIssueCount"$'\t'"$boardIssue"$'\t'"$libraryIssueCount"$'\t'"$libraryIssue"$'\r' >>"$ARDUINO_CI_SCRIPT_REPORT_FILE_PATH" 1045 | 1046 | # Adjust the exit status according to the allowFail setting 1047 | if [[ "$buildThisSketchExitStatus" == "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" && ("$allowFail" == "true" || "$allowFail" == "require") ]]; then 1048 | buildThisSketchExitStatus="$ARDUINO_CI_SCRIPT_SUCCESS_EXIT_STATUS" 1049 | elif [[ "$buildThisSketchExitStatus" == "$ARDUINO_CI_SCRIPT_SUCCESS_EXIT_STATUS" && "$allowFail" == "require" ]]; then 1050 | buildThisSketchExitStatus="$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 1051 | fi 1052 | 1053 | if [[ "$buildThisSketchExitStatus" != "$ARDUINO_CI_SCRIPT_SUCCESS_EXIT_STATUS" ]]; then 1054 | ARDUINO_CI_SCRIPT_TOTAL_SKETCH_BUILD_FAILURE_COUNT=$((ARDUINO_CI_SCRIPT_TOTAL_SKETCH_BUILD_FAILURE_COUNT + 1)) 1055 | fi 1056 | ARDUINO_CI_SCRIPT_TOTAL_WARNING_COUNT=$((ARDUINO_CI_SCRIPT_TOTAL_WARNING_COUNT + warningCount + 0)) 1057 | ARDUINO_CI_SCRIPT_TOTAL_BOARD_ISSUE_COUNT=$((ARDUINO_CI_SCRIPT_TOTAL_BOARD_ISSUE_COUNT + boardIssueCount + 0)) 1058 | ARDUINO_CI_SCRIPT_TOTAL_LIBRARY_ISSUE_COUNT=$((ARDUINO_CI_SCRIPT_TOTAL_LIBRARY_ISSUE_COUNT + libraryIssueCount + 0)) 1059 | 1060 | # End the folded section of the Travis CI build log 1061 | echo -e "travis_fold:end:build_sketch" 1062 | # Add a useful message to the Travis CI build log 1063 | 1064 | echo "arduino Exit Status: ${arduinoExitStatus}, Allow Failure: ${allowFail}, # Warnings: ${warningCount}, # Board Issues: ${boardIssueCount}, # Library Issues: ${libraryIssueCount}" 1065 | 1066 | return $buildThisSketchExitStatus 1067 | } 1068 | 1069 | # Print the contents of the report file 1070 | function display_report() { 1071 | enable_verbosity 1072 | 1073 | if [ -e "$ARDUINO_CI_SCRIPT_REPORT_FILE_PATH" ]; then 1074 | echo -e '\n\n\n**************Begin Report**************\n\n\n' 1075 | cat "$ARDUINO_CI_SCRIPT_REPORT_FILE_PATH" 1076 | echo -e '\n\n' 1077 | echo "Total failed sketch builds: $ARDUINO_CI_SCRIPT_TOTAL_SKETCH_BUILD_FAILURE_COUNT" 1078 | echo "Total warnings: $ARDUINO_CI_SCRIPT_TOTAL_WARNING_COUNT" 1079 | echo "Total board issues: $ARDUINO_CI_SCRIPT_TOTAL_BOARD_ISSUE_COUNT" 1080 | echo "Total library issues: $ARDUINO_CI_SCRIPT_TOTAL_LIBRARY_ISSUE_COUNT" 1081 | echo -e '\n\n' 1082 | else 1083 | echo "No report file available for this job" 1084 | fi 1085 | 1086 | disable_verbosity 1087 | } 1088 | 1089 | # Add the report file to a Git repository 1090 | function publish_report_to_repository() { 1091 | enable_verbosity 1092 | 1093 | local -r token="$1" 1094 | local -r repositoryURL="$2" 1095 | local -r reportBranch="$3" 1096 | local -r reportFolder="$4" 1097 | local -r doLinkComment="$5" 1098 | 1099 | if [[ "$token" != "" ]] && [[ "$repositoryURL" != "" ]] && [[ "$reportBranch" != "" ]]; then 1100 | if [ -e "$ARDUINO_CI_SCRIPT_REPORT_FILE_PATH" ]; then 1101 | # Location is a repository 1102 | if git clone --quiet --branch "$reportBranch" "$repositoryURL" "${HOME}/report-repository"; then 1103 | # Clone was successful 1104 | create_folder "${HOME}/report-repository/${reportFolder}" 1105 | cp $ARDUINO_CI_SCRIPT_VERBOSITY_OPTION "$ARDUINO_CI_SCRIPT_REPORT_FILE_PATH" "${HOME}/report-repository/${reportFolder}" 1106 | local -r previousFolder="$PWD" 1107 | cd "${HOME}/report-repository" 1108 | git add $ARDUINO_CI_SCRIPT_VERBOSITY_OPTION "${HOME}/report-repository/${reportFolder}/${ARDUINO_CI_SCRIPT_REPORT_FILENAME}" 1109 | git config user.email "arduino-ci-script@nospam.me" 1110 | git config user.name "arduino-ci-script-bot" 1111 | # Only pushes the current branch to the corresponding remote branch that 'git pull' uses to update the current branch. 1112 | git config push.default simple 1113 | if [[ "$TRAVIS_TEST_RESULT" != "0" ]]; then 1114 | local -r jobSuccessMessage="FAILED" 1115 | else 1116 | local -r jobSuccessMessage="SUCCESSFUL" 1117 | fi 1118 | # Do a pull now in case another job has finished about the same time and pushed a report after the clone happened, which would otherwise cause the push to fail. This is the last chance to pull without having to deal with a merge or rebase. 1119 | git pull $ARDUINO_CI_SCRIPT_QUIET_OPTION 1120 | git commit $ARDUINO_CI_SCRIPT_QUIET_OPTION $ARDUINO_CI_SCRIPT_VERBOSITY_OPTION --message="Add Travis CI job ${TRAVIS_JOB_NUMBER} report (${jobSuccessMessage})" --message="Total failed sketch builds: $ARDUINO_CI_SCRIPT_TOTAL_SKETCH_BUILD_FAILURE_COUNT" --message="Total warnings: $ARDUINO_CI_SCRIPT_TOTAL_WARNING_COUNT" --message="Total board issues: $ARDUINO_CI_SCRIPT_TOTAL_BOARD_ISSUE_COUNT" --message="Total library issues: $ARDUINO_CI_SCRIPT_TOTAL_LIBRARY_ISSUE_COUNT" --message="Job log: https://travis-ci.org/${TRAVIS_REPO_SLUG}/jobs/${TRAVIS_JOB_ID}" --message="Commit: https://github.com/${TRAVIS_REPO_SLUG}/commit/${TRAVIS_COMMIT}" --message="$TRAVIS_COMMIT_MESSAGE" --message="[skip ci]" 1121 | local gitPushExitStatus="1" 1122 | local pushCount=0 1123 | while [[ "$gitPushExitStatus" != "$ARDUINO_CI_SCRIPT_SUCCESS_EXIT_STATUS" && $pushCount -le $ARDUINO_CI_SCRIPT_REPORT_PUSH_RETRIES ]]; do 1124 | pushCount=$((pushCount + 1)) 1125 | # Do a pull now in case another job has finished about the same time and pushed a report since the last pull. This would require a merge or rebase. Rebase should be safe since the commits will be separate files. 1126 | git pull $ARDUINO_CI_SCRIPT_QUIET_OPTION --rebase 1127 | git push $ARDUINO_CI_SCRIPT_QUIET_OPTION $ARDUINO_CI_SCRIPT_VERBOSITY_OPTION "https://${token}@${repositoryURL#*//}" 1128 | gitPushExitStatus="$?" 1129 | done 1130 | cd "$previousFolder" 1131 | rm --recursive --force "${HOME}/report-repository" 1132 | if [[ "$gitPushExitStatus" == "$ARDUINO_CI_SCRIPT_SUCCESS_EXIT_STATUS" ]]; then 1133 | if [[ "$doLinkComment" == "true" ]]; then 1134 | # Only comment if it's job 1 1135 | local -r firstJobRegex='\.1$' 1136 | if [[ "$TRAVIS_JOB_NUMBER" =~ $firstJobRegex ]]; then 1137 | local reportURL 1138 | reportURL="${repositoryURL%.*}/tree/${reportBranch}/${reportFolder}" 1139 | comment_report_link "$token" "$reportURL" 1140 | fi 1141 | fi 1142 | else 1143 | echo "ERROR: Failed to push to remote branch." 1144 | return_handler "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 1145 | fi 1146 | else 1147 | echo "ERROR: Failed to clone branch ${reportBranch} of repository URL ${repositoryURL}. Do they exist?" 1148 | return_handler "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 1149 | fi 1150 | else 1151 | echo "No report file available for this job" 1152 | fi 1153 | else 1154 | if [[ "$token" == "" ]]; then 1155 | echo "ERROR: GitHub token not specified. Failed to publish build report. See https://github.com/per1234/arduino-ci-script#publishing-job-reports for instructions." 1156 | fi 1157 | if [[ "$repositoryURL" == "" ]]; then 1158 | echo "ERROR: Repository URL not specified. Failed to publish build report." 1159 | fi 1160 | if [[ "$reportBranch" == "" ]]; then 1161 | echo "ERROR: Repository branch not specified. Failed to publish build report." 1162 | fi 1163 | return_handler "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 1164 | fi 1165 | 1166 | disable_verbosity 1167 | } 1168 | 1169 | # Add the report file to a gist 1170 | function publish_report_to_gist() { 1171 | enable_verbosity 1172 | 1173 | local -r token="$1" 1174 | local -r gistURL="$2" 1175 | local -r doLinkComment="$3" 1176 | 1177 | if [[ "$token" != "" ]] && [[ "$gistURL" != "" ]]; then 1178 | if [ -e "$ARDUINO_CI_SCRIPT_REPORT_FILE_PATH" ]; then 1179 | # Get the gist ID from the gist URL 1180 | local gistID 1181 | gistID="$(echo "$gistURL" | rev | cut -d'/' -f 1 | rev)" 1182 | 1183 | # http://stackoverflow.com/a/33354920/7059512 1184 | # Sanitize the report file content so it can be sent via a POST request without breaking the JSON 1185 | # Remove \r (from Windows end-of-lines), replace tabs by \t, replace " by \", replace EOL by \n 1186 | local reportContent 1187 | reportContent=$(sed -e 's/\r//' -e's/\t/\\t/g' -e 's/"/\\"/g' "$ARDUINO_CI_SCRIPT_REPORT_FILE_PATH" | awk '{ printf($0 "\\n") }') 1188 | 1189 | # Upload the report to the Gist. I have to use the here document to avoid the "Argument list too long" error from curl with long reports. Redirect output to dev/null because it dumps the whole gist to the log 1190 | eval curl --header "\"Authorization: token ${token}\"" --data @- "\"https://api.github.com/gists/${gistID}\"" "$ARDUINO_CI_SCRIPT_VERBOSITY_REDIRECT" < Examples > INCOMPATIBLE." 1915 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_ARCHITECTURES_EMPTY_EXIT_STATUS) 1916 | else 1917 | # Check for invalid architectures 1918 | local validArchitecturesRegex='^((\*)|(ameba)|(arm)|(arc32)|(avr)|(efm32)|(esp32)|(esp8266)|(FP51)|(i586)|(i686)|(iot2000)|(mbed)|(megaavr)|(mraa)|(msp430)|(navspark)|(nRF5)|(nRF51822)|(nrf52)|(nRF52832)|(particle-photon)|(particle-electron)|(particle-core)|(pic)|(pic32)|(RFduino)|(sam)|(samd)|(samd_beta)|(Seeed_STM32F4)|(Simblee)|(solox)|(stm)|(STM32)|(stm32)|(STM32F1)|(STM32F2)|(STM32F3)|(STM32F4)|(stm32f4)|(STM32L1)|(STM32L4)|(teensy)|(win10)|(x86))$' 1919 | # Split string on , 1920 | IFS=',' 1921 | local validArchitectureFound=false 1922 | # Disable globbing, otherwise it fails when one of the architecture values is * 1923 | set -o noglob 1924 | # Check for * architecture. If this is found then the other architecture values don't matter 1925 | local wildcardArchitectureFound=false 1926 | for rawArchitecture in $architecturesValue; do 1927 | # The Arduino IDE ignores leading or trailing whitespace on architectures 1928 | # Strip leading whitespace 1929 | local architecture="${rawArchitecture#"${rawArchitecture%%[![:space:]]*}"}" 1930 | # Strip trailing whitespace 1931 | architecture="${architecture%"${architecture##*[![:space:]]}"}" 1932 | 1933 | if [[ "$architecture" == "*" ]]; then 1934 | wildcardArchitectureFound=true 1935 | validArchitectureFound=true 1936 | break 1937 | fi 1938 | done 1939 | if [[ "$wildcardArchitectureFound" == false ]]; then 1940 | for rawArchitecture in $architecturesValue; do 1941 | # The Arduino IDE ignores leading or trailing whitespace on architectures 1942 | # Strip leading whitespace 1943 | local architecture="${rawArchitecture#"${rawArchitecture%%[![:space:]]*}"}" 1944 | # Strip trailing whitespace 1945 | architecture="${architecture%"${architecture##*[![:space:]]}"}" 1946 | 1947 | if [[ "$architecture" =~ $validArchitecturesRegex ]]; then 1948 | validArchitectureFound=true 1949 | else 1950 | local aliasCheckPassed=true 1951 | # If an architecture alias is used then the correct architecture must also be present 1952 | check_architecture_alias "$architecture" '^((Avr)|(AVR)|([aA][tT][mM][eE][lL].?[aA][vV][rR]))$' "$architecturesValue" 'avr' "$normalizedLibraryPropertiesPath" 1953 | if [[ "$?" == "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" ]]; then 1954 | aliasCheckPassed=false 1955 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_INVALID_ARCHITECTURE_EXIT_STATUS) 1956 | fi 1957 | check_architecture_alias "$architecture" '^((Sam)|(SAM))$' "$architecturesValue" 'sam' "$normalizedLibraryPropertiesPath" 1958 | if [[ "$?" == "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" ]]; then 1959 | aliasCheckPassed=false 1960 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_INVALID_ARCHITECTURE_EXIT_STATUS) 1961 | fi 1962 | check_architecture_alias "$architecture" '^((Samd)|(SAMD)|(SamD)|((samD)|([aA][tT][mM][eE][lL].?[sS][aA][mM]))$' "$architecturesValue" 'samd' "$normalizedLibraryPropertiesPath" 1963 | if [[ "$?" == "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" ]]; then 1964 | aliasCheckPassed=false 1965 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_INVALID_ARCHITECTURE_EXIT_STATUS) 1966 | fi 1967 | check_architecture_alias "$architecture" '^((Arc32)|(ARC32)|([aA][rR][cC].32)|([iI][nN][tT][eE][lL].?[aA][rR][cC]32))$' "$architecturesValue" 'arc32' "$normalizedLibraryPropertiesPath" 1968 | if [[ "$?" == "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" ]]; then 1969 | aliasCheckPassed=false 1970 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_INVALID_ARCHITECTURE_EXIT_STATUS) 1971 | fi 1972 | check_architecture_alias "$architecture" '^((Esp8266)|(ESP8266)|([eE][sS][pP].8266)|(8266)|([eE][sS][pP])|([eE][sS][pP][rR][eE][sS][sS][iI][fF].?(8266)?))$' "$architecturesValue" 'esp8266' "$normalizedLibraryPropertiesPath" 1973 | if [[ "$?" == "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" ]]; then 1974 | aliasCheckPassed=false 1975 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_INVALID_ARCHITECTURE_EXIT_STATUS) 1976 | fi 1977 | check_architecture_alias "$architecture" '^((Esp32)|(ESP32)|([eE][sS][pP].32)|(arduino-esp32)|([eE][sS][pP][rR][eE][sS][sS][iI][fF].?32))$' "$architecturesValue" 'esp32' "$normalizedLibraryPropertiesPath" 1978 | if [[ "$?" == "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" ]]; then 1979 | aliasCheckPassed=false 1980 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_INVALID_ARCHITECTURE_EXIT_STATUS) 1981 | fi 1982 | check_architecture_alias "$architecture" '^((Teensy)|(TEENSY))$' "$architecturesValue" 'teensy' "$normalizedLibraryPropertiesPath" 1983 | if [[ "$?" == "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" ]]; then 1984 | aliasCheckPassed=false 1985 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_INVALID_ARCHITECTURE_EXIT_STATUS) 1986 | fi 1987 | 1988 | if [[ "$aliasCheckPassed" == true ]]; then 1989 | echo "WARNING: ${normalizedLibraryPropertiesPath}/library.properties: architectures field contains an unknown architecture: ${architecture}. Note: architecture values are case-sensitive." 1990 | fi 1991 | fi 1992 | done 1993 | fi 1994 | # Re-enable globbing 1995 | set +o noglob 1996 | # Set IFS back to default 1997 | unset IFS 1998 | 1999 | # At least one known architecture must be present 2000 | if [[ "$validArchitectureFound" == false ]]; then 2001 | echo "ERROR: ${normalizedLibraryPropertiesPath}/library.properties: architectures field (${architecturesValue}) doesn't contain any known architecture values." 2002 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_INVALID_ARCHITECTURE_EXIT_STATUS) 2003 | fi 2004 | fi 2005 | fi 2006 | 2007 | # Check for invalid lines (anything other than property, comment, or blank line) 2008 | if grep --quiet --invert-match --extended-regexp --regexp='=' --regexp='^[[:space:]]*(#|$)' <<<"$libraryProperties"; then 2009 | echo "ERROR: ${normalizedLibraryPropertiesPath}/library.properties: Invalid line found. Installation of a library with invalid line will cause all compilations to fail. library.properties must only consist of property definitions, blank lines, and comments (#)." 2010 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_INVALID_LINE_EXIT_STATUS) 2011 | fi 2012 | 2013 | # Check for incorrect includes field name case 2014 | if ! check_field_name_case "$libraryProperties" 'includes' "$normalizedLibraryPropertiesPath"; then 2015 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_INCLUDES_MISSPELLED_EXIT_STATUS) 2016 | fi 2017 | 2018 | # Check for misspelled includes field name 2019 | if grep --quiet --ignore-case --regexp='^[[:space:]]*include[[:space:]]*=' <<<"$libraryProperties"; then 2020 | echo "ERROR: ${normalizedLibraryPropertiesPath}/library.properties: Misspelled includes field name. It must be spelled exactly \"includes\". See https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification#libraryproperties-file-format" 2021 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_INCLUDES_MISSPELLED_EXIT_STATUS) 2022 | fi 2023 | 2024 | # Check for empty includes value 2025 | if grep --quiet --regexp='^[[:space:]]*includes[[:space:]]*=[[:space:]]*$' <<<"$libraryProperties"; then 2026 | echo "ERROR: ${normalizedLibraryPropertiesPath}/library.properties: Undefined includes field. Either define the field or remove it. See https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification#libraryproperties-file-format" 2027 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_EMPTY_INCLUDES_EXIT_STATUS) 2028 | fi 2029 | 2030 | # Check for incorrect dot_a_linkage field name case 2031 | if ! check_field_name_case "$libraryProperties" 'dot_a_linkage' "$normalizedLibraryPropertiesPath"; then 2032 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_DOT_A_LINKAGE_MISSPELLED_EXIT_STATUS) 2033 | fi 2034 | 2035 | # Check for misspelled dot_a_linkage field name 2036 | if grep --quiet --ignore-case --extended-regexp --regexp='^[[:space:]]*((dot_a_linkages)|(dot-?a-?linkages?))[[:space:]]*=' <<<"$libraryProperties"; then 2037 | echo "ERROR: ${normalizedLibraryPropertiesPath}/library.properties: Misspelled dot_a_linkage field name. It must be spelled exactly \"dot_a_linkage\". See https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification#libraryproperties-file-format" 2038 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_DOT_A_LINKAGE_MISSPELLED_EXIT_STATUS) 2039 | fi 2040 | 2041 | # Check for incorrect precompiled field name case 2042 | if ! check_field_name_case "$libraryProperties" 'precompiled' "$normalizedLibraryPropertiesPath"; then 2043 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_PRECOMPILED_MISSPELLED_EXIT_STATUS) 2044 | fi 2045 | 2046 | # Check for misspelled precompiled field name 2047 | if grep --quiet --ignore-case --extended-regexp --regexp='^[[:space:]]*((precompile)|(pre[-_]compiled?))[[:space:]]*=' <<<"$libraryProperties"; then 2048 | echo "ERROR: ${normalizedLibraryPropertiesPath}/library.properties: Misspelled precompiled field name. It must be spelled exactly \"precompiled\". See https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification#libraryproperties-file-format" 2049 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_PRECOMPILED_MISSPELLED_EXIT_STATUS) 2050 | fi 2051 | 2052 | # Check for incorrect ldflags field name case 2053 | if ! check_field_name_case "$libraryProperties" 'ldflags' "$normalizedLibraryPropertiesPath"; then 2054 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_LDFLAGS_MISSPELLED_EXIT_STATUS) 2055 | fi 2056 | 2057 | # Check for misspelled ldflags field name 2058 | if grep --quiet --ignore-case --extended-regexp --regexp='^[[:space:]]*((ldflag)|(ld[-_]flags?))[[:space:]]*=' <<<"$libraryProperties"; then 2059 | echo "ERROR: ${normalizedLibraryPropertiesPath}/library.properties: Misspelled ldflags field name. It must be spelled exactly \"ldflags\". See https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification#libraryproperties-file-format" 2060 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_LDFLAGS_MISSPELLED_EXIT_STATUS) 2061 | fi 2062 | 2063 | done <<<"$(find "$normalizedLibraryPropertiesSearchPath" -maxdepth "$maximumSearchDepth" -type d | sort --dictionary-order)" 2064 | 2065 | return "$exitStatus" 2066 | } 2067 | 2068 | # https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification#keywords 2069 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=1 2070 | readonly ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_INCONSEQUENTIAL_MULTIPLE_TABS_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2071 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=$((ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER + 1)) 2072 | readonly ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_INVALID_LINE_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2073 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=$((ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER + 1)) 2074 | readonly ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_INCONSEQUENTIAL_LEADING_SPACE_ON_KEYWORD_TOKENTYPE_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2075 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=$((ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER + 1)) 2076 | readonly ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_MULTIPLE_TABS_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2077 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=$((ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER + 1)) 2078 | readonly ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_LEADING_SPACE_ON_KEYWORD_TOKENTYPE_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2079 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=$((ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER + 1)) 2080 | readonly ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_FOLDER_DOESNT_EXIST_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2081 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=$((ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER + 1)) 2082 | readonly ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_MISSPELLED_FILENAME_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2083 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=$((ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER + 1)) 2084 | readonly ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_INCORRECT_FILENAME_CASE_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2085 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=$((ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER + 1)) 2086 | readonly ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_INVALID_FIELD_SEPARATOR_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2087 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=$((ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER + 1)) 2088 | readonly ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_BOM_CORRUPTED_KEYWORD_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2089 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=$((ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER + 1)) 2090 | readonly ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_INVALID_KEYWORD_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2091 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=$((ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER + 1)) 2092 | readonly ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_INVALID_KEYWORD_TOKENTYPE_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2093 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=$((ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER + 1)) 2094 | readonly ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_LEADING_SPACE_ON_RSYNTAXTEXTAREA_TOKENTYPE_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2095 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=$((ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER + 1)) 2096 | readonly ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_INVALID_RSYNTAXTEXTAREA_TOKENTYPE_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2097 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=$((ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER + 1)) 2098 | readonly ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_INVALID_REFERENCE_LINK_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2099 | function check_keywords_txt() { 2100 | local -r keywordsTxtSearchPath="$1" 2101 | local maximumSearchDepth="$2" 2102 | if [[ "$maximumSearchDepth" == "" ]]; then 2103 | # Set default search depth 2104 | maximumSearchDepth=0 2105 | fi 2106 | 2107 | local exitStatus=$ARDUINO_CI_SCRIPT_SUCCESS_EXIT_STATUS 2108 | 2109 | # Replace backslashes with slashes 2110 | local -r keywordsTxtSearchPathWithSlashes="${keywordsTxtSearchPath//\\//}" 2111 | local -r keywordsTxtRegex='[kK][eE][yY][wW][oO][rR][dD][sS]\.[tT][xX][tT]$' 2112 | if [[ ! -d "$keywordsTxtSearchPathWithSlashes" && "$keywordsTxtSearchPathWithSlashes" =~ $keywordsTxtRegex ]]; then 2113 | # Path contains the filename but we only want the folder 2114 | local -r keywordsTxtSearchPathWithoutFile="${keywordsTxtSearchPathWithSlashes::-12}" 2115 | else 2116 | local -r keywordsTxtSearchPathWithoutFile="$keywordsTxtSearchPathWithSlashes" 2117 | fi 2118 | # Remove trailing slash 2119 | local -r normalizedKeywordsTxtSearchPath="${keywordsTxtSearchPathWithoutFile%/}" 2120 | 2121 | # Check whether folder exists 2122 | if [[ ! -d "$normalizedKeywordsTxtSearchPath" ]]; then 2123 | echo "ERROR: ${normalizedKeywordsTxtSearchPath}: Folder doesn't exist." 2124 | return $ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_FOLDER_DOESNT_EXIST_EXIT_STATUS 2125 | fi 2126 | 2127 | while read -r normalizedKeywordsTxtPath; do 2128 | # Check for misspelled keywords.txt filename 2129 | if [[ "$(find "$normalizedKeywordsTxtPath" -type f -iname 'keyword.txt')" || "$(find "$normalizedKeywordsTxtPath" -type f -iregex '.*/keywords?\.text')" || "$(find "$normalizedKeywordsTxtPath" -type f -iregex '.*/keywords?\.txt\.txt')" ]]; then 2130 | echo "ERROR: ${normalizedKeywordsTxtPath}: Incorrectly spelled keywords.txt file. It must be spelled exactly \"keywords.txt\"." 2131 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_MISSPELLED_FILENAME_EXIT_STATUS) 2132 | fi 2133 | 2134 | # Check for incorrect filename case 2135 | local keywordsTxtFound=false 2136 | while read -r foundKeywordsTxtPath; do 2137 | # The while loop always runs once, even if no file was found 2138 | if [[ "$foundKeywordsTxtPath" == "" ]]; then 2139 | continue 2140 | fi 2141 | 2142 | if [[ "${foundKeywordsTxtPath: -12}" == 'keywords.txt' ]]; then 2143 | keywordsTxtFound=true 2144 | else 2145 | echo "ERROR: ${foundKeywordsTxtPath}: Incorrect filename case, which causes it to not be recognized on a filename case-sensitive OS such as Linux. It must be exactly \"keywords.txt\"." 2146 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_INCORRECT_FILENAME_CASE_EXIT_STATUS) 2147 | fi 2148 | done <<<"$(find "$normalizedKeywordsTxtPath" -maxdepth 1 -type f -iname 'keywords.txt' | sort --dictionary-order)" 2149 | 2150 | # Check whether the folder contains a keywords.txt file 2151 | if [[ "$keywordsTxtFound" == false ]]; then 2152 | # no point in doing any more checks on this folder 2153 | continue 2154 | fi 2155 | 2156 | # Read the keywords.txt file line by line 2157 | # Split into lines by CR 2158 | while IFS='' read -d $'\r' -r keywordsTxtCRline || [[ -n "$keywordsTxtCRline" ]]; do 2159 | # Split into lines by LN 2160 | while IFS='' read -r keywordsTxtLine || [[ -n "$keywordsTxtLine" ]]; do 2161 | # Skip blank lines and comments 2162 | local blankLineRegex='^[[:space:]]*$' 2163 | local commentRegex='^[[:space:]]*#' 2164 | if [[ "$keywordsTxtLine" =~ $blankLineRegex ]] || [[ "$keywordsTxtLine" =~ $commentRegex ]]; then 2165 | continue 2166 | fi 2167 | 2168 | # Skip BOM corrupted blank lines and comments 2169 | if grep --quiet $'\xEF\xBB\xBF' <<<"$keywordsTxtLine"; then 2170 | local BOMcorruptedCommentRegex='^.[[:space:]]*#' 2171 | local BOMcorruptedBlankLineRegex='^.[[:space:]]*$' 2172 | if [[ "$keywordsTxtLine" =~ $BOMcorruptedCommentRegex ]] || [[ "$keywordsTxtLine" =~ $BOMcorruptedBlankLineRegex ]]; then 2173 | echo "WARNING: ${normalizedKeywordsTxtPath}/keywords.txt: BOM found. In this case it does not cause an issue but it's recommended to use UTF-8 encoding for keywords.txt." 2174 | continue 2175 | fi 2176 | fi 2177 | 2178 | local spacesSeparatorRegex='^[[:space:]]*[^[:space:]]+ +[^[:space:]]+' 2179 | # Check for invalid separator 2180 | if [[ "$keywordsTxtLine" =~ $spacesSeparatorRegex ]]; then 2181 | echo "ERROR: ${normalizedKeywordsTxtPath}/keywords.txt: Space(s) used as a field separator. Fields must be separated by a single true tab." 2182 | echo -e "\t$keywordsTxtLine" 2183 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_INVALID_FIELD_SEPARATOR_EXIT_STATUS) 2184 | # The rest of the checks will be borked by messed up field separators so continue to the next line 2185 | continue 2186 | fi 2187 | 2188 | # Check for multiple tabs used as separator where this causes unintended results 2189 | local consequentialMultipleSeparatorRegex='^[[:space:]]*[^[:space:]]+[[:space:]]*'$'\t''+[[:space:]]*'$'\t''+[[:space:]]*((KEYWORD1)|(LITERAL1))' 2190 | if [[ "$keywordsTxtLine" =~ $consequentialMultipleSeparatorRegex ]]; then 2191 | echo "ERROR: ${normalizedKeywordsTxtPath}/keywords.txt: Multiple tabs used as field separator. It must be a single tab. This causes the default keyword highlighting (as used by KEYWORD2, KEYWORD3, LITERAL2) to be used rather than the intended highlighting." 2192 | echo -e "\t$keywordsTxtLine" 2193 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_MULTIPLE_TABS_EXIT_STATUS) 2194 | # The rest of the checks will be borked by messed up field separators so continue to the next line 2195 | continue 2196 | fi 2197 | 2198 | # Check for multiple tabs used as separator where this causes no unintended results 2199 | local inconsequentialMultipleSeparatorRegex='^[[:space:]]*[^[:space:]]+[[:space:]]*'$'\t''+[[:space:]]*'$'\t''+[[:space:]]*((KEYWORD2)|(KEYWORD3)|(LITERAL2))' 2200 | if [[ "$keywordsTxtLine" =~ $inconsequentialMultipleSeparatorRegex ]]; then 2201 | echo "ERROR: ${normalizedKeywordsTxtPath}/keywords.txt: Multiple tabs used as field separator. It must be a single tab. This causes the default keyword highlighting (as used by KEYWORD2, KEYWORD3, LITERAL2). In this case that doesn't cause the keywords to be colored other than intended but it's recommended to fully comply with the Arduino library specification." 2202 | echo -e "\t$keywordsTxtLine" 2203 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_INCONSEQUENTIAL_MULTIPLE_TABS_EXIT_STATUS) 2204 | # The rest of the checks will be borked by messed up field separators so continue to the next line 2205 | continue 2206 | fi 2207 | 2208 | # Check for invalid line 2209 | local invalidLineRegex='^[[:space:]]*[^[:space:]]+[[:space:]]*$' 2210 | if [[ "$keywordsTxtLine" =~ $invalidLineRegex ]]; then 2211 | echo "ERROR: ${normalizedKeywordsTxtPath}/keywords.txt: Invalid line. If this was intended as a comment, it should use the correct # syntax." 2212 | echo -e "\t$keywordsTxtLine" 2213 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_INVALID_LINE_EXIT_STATUS) 2214 | # The rest of the checks are pointless so continue to the next line of keywords.txt 2215 | continue 2216 | fi 2217 | 2218 | # Get the field values 2219 | # Use a unique, non-whitespace field separator character 2220 | fieldSeparator=$'\a' 2221 | IFS=$fieldSeparator 2222 | # Strip leading whitespace. This is ignored by the Arduino IDE (even tabs) 2223 | local keywordsTxtLineFrontStripped="${keywordsTxtLine#"${keywordsTxtLine%%[![:space:]]*}"}" 2224 | # Change tabs to the field separator character for line splitting 2225 | # shellcheck disable=SC2206 2226 | local keywordsTxtLineSwappedTabs=(${keywordsTxtLineFrontStripped//$'\t'/$fieldSeparator}) 2227 | 2228 | # KEYWORD is the 1st field 2229 | local keywordRaw=${keywordsTxtLineSwappedTabs[0]} 2230 | # Strip trailing spaces 2231 | local keyword="${keywordRaw%"${keywordRaw##*[! ]}"}" 2232 | 2233 | # KEYWORD_TOKENTYPE is the 2nd field 2234 | local keywordTokentypeRaw=${keywordsTxtLineSwappedTabs[1]} 2235 | # The Arduino IDE strips trailing spaces from KEYWORD_TOKENTYPE 2236 | # Strip trailing spaces 2237 | local keywordTokentype="${keywordTokentypeRaw%"${keywordTokentypeRaw##*[! ]}"}" 2238 | 2239 | # REFERENCE_LINK is the 3rd field 2240 | local referenceLinkRaw=${keywordsTxtLineSwappedTabs[2]} 2241 | # The Arduino IDE strips leading and trailing whitespace from REFERENCE_LINK 2242 | # Strip leading spaces 2243 | local referenceLinkFrontStripped="${referenceLinkRaw#"${referenceLinkRaw%%[! ]*}"}" 2244 | # Strip trailing spaces 2245 | local referenceLink="${referenceLinkFrontStripped%"${referenceLinkFrontStripped##*[! ]}"}" 2246 | 2247 | # RSYNTAXTEXTAREA_TOKENTYPE is the 4th field 2248 | local rsyntaxtextareaTokentypeRaw=${keywordsTxtLineSwappedTabs[3]} 2249 | # The Arduino IDE strips trailing spaces from RSYNTAXTEXTAREA_TOKENTYPE 2250 | # Strip trailing spaces 2251 | local rsyntaxtextareaTokentype="${rsyntaxtextareaTokentypeRaw%"${rsyntaxtextareaTokentypeRaw##*[! ]}"}" 2252 | 2253 | # Reset IFS to default 2254 | unset IFS 2255 | 2256 | allowedKeywordCharactersRegex='^[a-zA-Z0-9_]+$' 2257 | # Check for corruption of KEYWORD field caused by UTF-8 BOM file encoding 2258 | if grep --quiet $'\xEF\xBB\xBF' <<<"$keyword"; then 2259 | echo "ERROR: ${normalizedKeywordsTxtPath}/keywords.txt: UTF-8 BOM file encoding has corrupted the first keyword definition. Please change the file encoding to standard UTF-8." 2260 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_BOM_CORRUPTED_KEYWORD_EXIT_STATUS) 2261 | elif ! [[ "$keyword" =~ $allowedKeywordCharactersRegex ]]; then 2262 | # Check for invalid characters in KEYWORD 2263 | # The Arduino IDE does recognize keywords that start with a number, even though these are not valid identifiers. 2264 | echo "ERROR: ${normalizedKeywordsTxtPath}/keywords.txt: Keyword: $keyword contains invalid character(s), which causes it to not be recognized by the Arduino IDE. Keywords may only contain the characters a-z, A-Z, 0-9, and _." 2265 | echo -e "\t$keywordsTxtLine" 2266 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_INVALID_KEYWORD_EXIT_STATUS) 2267 | fi 2268 | 2269 | # Check for invalid KEYWORD_TOKENTYPE 2270 | local validKeywordTokentypeRegex='^((KEYWORD1)|(KEYWORD2)|(KEYWORD3)|(LITERAL1)|(LITERAL2))$' 2271 | local validRsyntaxtextareaTokentypeRegex='^((RESERVED_WORD)|(RESERVED_WORD_2)|(DATA_TYPE)|(PREPROCESSOR)|(LITERAL_BOOLEAN))$' 2272 | if ! [[ "$keywordTokentype" =~ $validKeywordTokentypeRegex ]]; then 2273 | # Check if it's invalid only because of leading space 2274 | local keywordTokentypeWithoutLeadingSpace="${keywordTokentype#"${keywordTokentype%%[![:space:]]*}"}" 2275 | if [[ "$keywordTokentypeWithoutLeadingSpace" =~ $validKeywordTokentypeRegex ]]; then 2276 | # Check if the issue doesn't cause any change from the intended highlighting 2277 | local inconsequentialTokentypeRegex='((KEYWORD2)|(KEYWORD3)|(LITERAL2))' 2278 | if [[ "$keywordTokentypeWithoutLeadingSpace" =~ $inconsequentialTokentypeRegex ]]; then 2279 | echo "ERROR: ${normalizedKeywordsTxtPath}/keywords.txt: Leading space on the KEYWORD_TOKENTYPE field causes it to not be recognized, so the default keyword highlighting is used. In this case that doesn't cause the keywords to be colored other than intended but it's recommended to fully comply with the Arduino library specification." 2280 | echo -e "\t$keywordsTxtLine" 2281 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_INCONSEQUENTIAL_LEADING_SPACE_ON_KEYWORD_TOKENTYPE_EXIT_STATUS) 2282 | else 2283 | echo "ERROR: ${normalizedKeywordsTxtPath}/keywords.txt: Leading space on the KEYWORD_TOKENTYPE field causes it to not be recognized, so the default keyword highlighting is used." 2284 | echo -e "\t$keywordsTxtLine" 2285 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_LEADING_SPACE_ON_KEYWORD_TOKENTYPE_EXIT_STATUS) 2286 | fi 2287 | elif ! [[ "$keywordTokentypeWithoutLeadingSpace" == "" && "$rsyntaxtextareaTokentype" =~ $validRsyntaxtextareaTokentypeRegex ]]; then 2288 | # It's reasonable to leave KEYWORD_TOKENTYPE blank if RSYNTAXTEXTAREA_TOKENTYPE is defined and valid. This will not be compatible with 1.6.4 and older but that's really no big deal. 2289 | echo "ERROR: ${normalizedKeywordsTxtPath}/keywords.txt: Invalid KEYWORD_TOKENTYPE: $keywordTokentype causes the default keyword highlighting to be used. See: https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification#keyword_tokentype" 2290 | echo -e "\t$keywordsTxtLine" 2291 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_INVALID_KEYWORD_TOKENTYPE_EXIT_STATUS) 2292 | fi 2293 | fi 2294 | 2295 | # Check for invalid REFERENCE_LINK 2296 | if [[ "$referenceLink" != "" ]]; then 2297 | # The Arduino IDE must be installed to check if the reference page exists 2298 | if [[ "$NEWEST_INSTALLED_IDE_VERSION" == "" ]]; then 2299 | echo "WARNING: Arduino IDE is not installed so unable to check for invalid reference links. Please call install_ide before running check_keywords_txt." 2300 | else 2301 | install_ide_version "$NEWEST_INSTALLED_IDE_VERSION" 2302 | if [[ ! $(find "${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}/${ARDUINO_CI_SCRIPT_IDE_INSTALLATION_FOLDER}/reference/www.arduino.cc/en/Reference/" -type f -name "${referenceLink}.html") ]]; then 2303 | echo "ERROR: ${normalizedKeywordsTxtPath}/keywords.txt: REFERENCE_LINK value: $referenceLink is not a valid Arduino Language Reference page. See: https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification#reference_link" 2304 | echo -e "\t$keywordsTxtLine" 2305 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_INVALID_REFERENCE_LINK_EXIT_STATUS) 2306 | fi 2307 | fi 2308 | fi 2309 | 2310 | # Check for invalid RSYNTAXTEXTAREA_TOKENTYPE 2311 | if [[ "$rsyntaxtextareaTokentype" != "" ]]; then 2312 | if ! [[ "$rsyntaxtextareaTokentype" =~ $validRsyntaxtextareaTokentypeRegex ]]; then 2313 | # Check if it's invalid only because of leading space 2314 | local rsyntaxtextareaTokentypeWithoutLeadingSpace="${rsyntaxtextareaTokentype#"${rsyntaxtextareaTokentype%%[![:space:]]*}"}" 2315 | if [[ "$rsyntaxtextareaTokentypeWithoutLeadingSpace" =~ $validRsyntaxtextareaTokentypeRegex ]]; then 2316 | echo "ERROR: ${normalizedKeywordsTxtPath}/keywords.txt: Leading space on the RSYNTAXTEXTAREA_TOKENTYPE field causes it to not be recognized, so the default keyword highlighting is used." 2317 | echo -e "\t$keywordsTxtLine" 2318 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_LEADING_SPACE_ON_RSYNTAXTEXTAREA_TOKENTYPE_EXIT_STATUS) 2319 | else 2320 | echo "ERROR: ${normalizedKeywordsTxtPath}/keywords.txt: Invalid RSYNTAXTEXTAREA_TOKENTYPE: $rsyntaxtextareaTokentype causes the default keyword highlighting to be used. See: https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification#rsyntaxtextarea_tokentype" 2321 | echo -e "\t$keywordsTxtLine" 2322 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_KEYWORDS_TXT_INVALID_RSYNTAXTEXTAREA_TOKENTYPE_EXIT_STATUS) 2323 | fi 2324 | fi 2325 | fi 2326 | done <<<"$keywordsTxtCRline" 2327 | done <"${normalizedKeywordsTxtPath}/keywords.txt" 2328 | 2329 | done <<<"$(find "$normalizedKeywordsTxtSearchPath" -maxdepth "$maximumSearchDepth" -type d | sort --dictionary-order)" 2330 | 2331 | return "$exitStatus" 2332 | } 2333 | 2334 | # https://github.com/arduino/Arduino/wiki/Library-Manager-FAQ#how-is-the-library-list-generated 2335 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=1 2336 | readonly ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_FOLDER_DOESNT_EXIST_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2337 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=$((ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER + 1)) 2338 | readonly ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_EXE_FOUND_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2339 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=$((ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER + 1)) 2340 | readonly ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_DOT_DEVELOPMENT_FOUND_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2341 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=$((ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER + 1)) 2342 | readonly ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_SYMLINK_FOUND_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2343 | readonly ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_CHECK_FOLDER_NAME_OFFSET=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2344 | readonly ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_NAME_HAS_INVALID_FIRST_CHARACTER_EXIT_STATUS=$((ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_CHECK_FOLDER_NAME_OFFSET + ARDUINO_CI_SCRIPT_CHECK_FOLDER_NAME_INVALID_FIRST_CHARACTER_EXIT_STATUS)) 2345 | readonly ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_NAME_HAS_INVALID_CHARACTER_EXIT_STATUS=$((ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_CHECK_FOLDER_NAME_OFFSET + ARDUINO_CI_SCRIPT_CHECK_FOLDER_NAME_INVALID_CHARACTER_EXIT_STATUS)) 2346 | readonly ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_NAME_TOO_LONG_EXIT_STATUS=$((ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_CHECK_FOLDER_NAME_OFFSET + ARDUINO_CI_SCRIPT_CHECK_FOLDER_NAME_TOO_LONG_EXIT_STATUS)) 2347 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=$((ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_NAME_TOO_LONG_EXIT_STATUS + 1)) 2348 | readonly ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_BLANK_URL_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2349 | ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER=$((ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER + 1)) 2350 | readonly ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_NAME_IS_RESERVED_EXIT_STATUS=$ARDUINO_CI_SCRIPT_EXIT_STATUS_COUNTER 2351 | function check_library_manager_compliance() { 2352 | local -r libraryPath="$1" 2353 | # Replace backslashes with slashes 2354 | local -r libraryPathWithSlashes="${libraryPath//\\//}" 2355 | # Remove trailing slash 2356 | local -r normalizedLibraryPath="${libraryPathWithSlashes%/}" 2357 | 2358 | local exitStatus=$ARDUINO_CI_SCRIPT_SUCCESS_EXIT_STATUS 2359 | 2360 | # Check whether folder exists 2361 | if [[ ! -d "$normalizedLibraryPath" ]]; then 2362 | echo "ERROR: ${libraryPath}: Folder doesn't exist." 2363 | return $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_FOLDER_DOESNT_EXIST_EXIT_STATUS 2364 | fi 2365 | 2366 | # Check for .exe files 2367 | local -r exePath=$(find "$normalizedLibraryPath" -type f -name '*.exe') 2368 | if [[ "$exePath" != "" ]]; then 2369 | echo "ERROR: ${exePath}: .exe file found." 2370 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_EXE_FOUND_EXIT_STATUS) 2371 | fi 2372 | 2373 | # Check for .development file 2374 | local -r dotDevelopmentPath=$(find "$normalizedLibraryPath" -maxdepth 1 -type f -name '.development') 2375 | if [[ "$dotDevelopmentPath" != "" ]]; then 2376 | echo "ERROR: ${dotDevelopmentPath}: .development file found." 2377 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_DOT_DEVELOPMENT_FOUND_EXIT_STATUS) 2378 | fi 2379 | 2380 | # Check for symlink 2381 | local -r symlinkPath=$(find "$normalizedLibraryPath" -type l) 2382 | if [[ "$symlinkPath" != "" ]]; then 2383 | echo "ERROR: ${symlinkPath}: Symlink found." 2384 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_SYMLINK_FOUND_EXIT_STATUS) 2385 | fi 2386 | 2387 | # Check for problems with library.properties 2388 | if [[ -f "$normalizedLibraryPath/library.properties" ]]; then 2389 | # Get rid of the CRs 2390 | local libraryProperties 2391 | libraryProperties=$(tr "\r" "\n" <"$normalizedLibraryPath/library.properties") 2392 | local nameValue 2393 | nameValue="$(get_library_properties_field_value "$libraryProperties" 'name')" 2394 | 2395 | # Check if the library.properties name value meets the requirements of the Library Manager indexer 2396 | check_library_properties_name "$nameValue" 2397 | local -r checkLibraryPropertiesNameExitStatus=$? 2398 | if [[ $checkLibraryPropertiesNameExitStatus -ne $ARDUINO_CI_SCRIPT_SUCCESS_EXIT_STATUS ]]; then 2399 | if [[ "$checkLibraryPropertiesNameExitStatus" == "$ARDUINO_CI_SCRIPT_CHECK_LIBRARY_PROPERTIES_NAME_RESERVED_NAME_EXIT_STATUS" ]]; then 2400 | echo "ERROR: ${normalizedLibraryPath}/library.properties: name value: $nameValue starts with \"arduino\". These names are reserved for official Arduino libraries." 2401 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_NAME_IS_RESERVED_EXIT_STATUS) 2402 | else 2403 | echo "ERROR: ${normalizedLibraryPath}/library.properties: name value: $nameValue does not meet the requirements of the Arduino Library Manager indexer. See: https://github.com/arduino/Arduino/wiki/Arduino-IDE-1.5:-Library-specification#libraryproperties-file-format" 2404 | exitStatus=$(set_exit_status "$exitStatus" $((ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_CHECK_FOLDER_NAME_OFFSET + checkLibraryPropertiesNameExitStatus))) 2405 | fi 2406 | fi 2407 | 2408 | local urlValue 2409 | urlValue="$(get_library_properties_field_value "$libraryProperties" 'url')" 2410 | if [[ "$urlValue" == "" ]]; then 2411 | echo "ERROR: ${normalizedLibraryPath}/library.properties: Blank url value: You must define a URL." 2412 | exitStatus=$(set_exit_status "$exitStatus" $ARDUINO_CI_SCRIPT_CHECK_LIBRARY_MANAGER_COMPLIANCE_BLANK_URL_EXIT_STATUS) 2413 | fi 2414 | fi 2415 | 2416 | return "$exitStatus" 2417 | } 2418 | 2419 | function check_code_formatting() { 2420 | local -r strictness="$1" 2421 | local -r excludedPathList="$2" 2422 | local -r targetPath="$3" 2423 | 2424 | local -r astyleConfigurationFolder="etc/astyle-configurations" 2425 | local -r astyleConfigurationExtension=".conf" 2426 | 2427 | # Fold the output in the Travis CI log 2428 | echo -e -n 'travis_fold:start:check_code_formatting\r' 2429 | 2430 | if [[ $strictness -lt 1 ]] || [[ $strictness -gt 3 ]]; then 2431 | echo "ERROR: Invalid strictness parameter value. Valid values are 1 (least strict) - 3 (most strict)" 2432 | return "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 2433 | fi 2434 | 2435 | if ! [[ -d "$targetPath" ]]; then 2436 | echo "ERROR: targetPath doesn't exist" 2437 | return "$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 2438 | fi 2439 | 2440 | local scriptFolder 2441 | # https://stackoverflow.com/a/246128 2442 | local scriptSource="${BASH_SOURCE[0]}" 2443 | while [ -h "$scriptSource" ]; do # Resolve $scriptSource until the file is no longer a symlink 2444 | scriptFolder="$(cd -P "$(dirname "$scriptSource")" >/dev/null 2>&1 && pwd)" 2445 | scriptSource="$(readlink "$scriptSource")" 2446 | [[ $scriptSource != /* ]] && scriptSource="$scriptFolder/$scriptSource" # If $scriptSource was a relative symlink, we need to resolve it relative to the path where the symlink file was located 2447 | done 2448 | scriptFolder="$(cd -P "$(dirname "$scriptSource")" >/dev/null 2>&1 && pwd)" 2449 | 2450 | # Assemble the find options for the excluded paths from the list 2451 | for excludedPath in ${excludedPathList//,/ }; do 2452 | excludeOptions="$excludeOptions -path $excludedPath -prune -or" 2453 | done 2454 | 2455 | local astylePath 2456 | astylePath=$(command -v astyle) 2457 | if [[ ! -e "$astylePath" ]]; then 2458 | # Install astyle 2459 | # Save the current folder 2460 | local -r previousFolder="$PWD" 2461 | mkdir --parents "${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}/astyle" 2462 | wget --no-verbose $ARDUINO_CI_SCRIPT_QUIET_OPTION --output-document="${ARDUINO_CI_SCRIPT_TEMPORARY_FOLDER}/astyle.tar.gz" "https://iweb.dl.sourceforge.net/project/astyle/astyle/astyle%203.1/astyle_3.1_linux.tar.gz" 2463 | tar --extract --file="${ARDUINO_CI_SCRIPT_TEMPORARY_FOLDER}/astyle.tar.gz" --directory="${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}" 2464 | cd "${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}/astyle/build/gcc" 2465 | eval "make $ARDUINO_CI_SCRIPT_VERBOSITY_REDIRECT" 2466 | astylePath="${ARDUINO_CI_SCRIPT_APPLICATION_FOLDER}/astyle/build/gcc/bin/astyle" 2467 | # Return to the previous folder 2468 | cd "$previousFolder" 2469 | fi 2470 | 2471 | # Set default exit status 2472 | exitStatus="$ARDUINO_CI_SCRIPT_SUCCESS_EXIT_STATUS" 2473 | 2474 | while read -r filename; do 2475 | # Check if it's a file (find matches on pruned folders) 2476 | if [[ -f "$filename" ]]; then 2477 | if ! diff --strip-trailing-cr "$filename" <("${astylePath}" --options="${scriptFolder}/${astyleConfigurationFolder}/${strictness}${astyleConfigurationExtension}" --dry-run <"$filename"); then 2478 | echo "ERROR: Non-compliant code formatting in $filename" 2479 | # Make the function fail 2480 | exitStatus="$ARDUINO_CI_SCRIPT_FAILURE_EXIT_STATUS" 2481 | fi 2482 | fi 2483 | done <<<"$(eval "find $targetPath -regextype posix-extended $excludeOptions \( -iregex '.*\.((ino)|(h)|(hpp)|(hh)|(hxx)|(h\+\+)|(cpp)|(cc)|(cxx)|(c\+\+)|(cp)|(c)|(ipp)|(ii)|(ixx)|(inl)|(tpp)|(txx)|(tpl))$' -and -type f \)")" 2484 | 2485 | echo -e -n 'travis_fold:end:check_code_formatting\r' 2486 | return "$exitStatus" 2487 | } 2488 | 2489 | # Set default verbosity (must be called after the function definitions 2490 | set_script_verbosity 0 2491 | 2492 | # Create the temporary folder 2493 | rm "$ARDUINO_CI_SCRIPT_TEMPORARY_FOLDER" --recursive --force 2494 | create_folder "$ARDUINO_CI_SCRIPT_TEMPORARY_FOLDER" 2495 | 2496 | # Create the report folder 2497 | create_folder "$ARDUINO_CI_SCRIPT_REPORT_FOLDER" 2498 | 2499 | # Add column names to report 2500 | echo "Build Timestamp (UTC)"$'\t'"Build"$'\t'"Job"$'\t'"Job URL"$'\t'"Build Trigger"$'\t'"Allow Job Failure"$'\t'"PR#"$'\t'"Branch"$'\t'"Commit"$'\t'"Commit Range"$'\t'"Commit Message"$'\t'"Sketch Filename"$'\t'"Board ID"$'\t'"IDE Version"$'\t'"Program Storage (bytes)"$'\t'"Dynamic Memory (bytes)"$'\t'"# Warnings"$'\t'"Allow Failure"$'\t'"Exit Status"$'\t'"# Board Issues"$'\t'"Board Issue"$'\t'"# Library Issues"$'\t'"Library Issue"$'\r' >"$ARDUINO_CI_SCRIPT_REPORT_FILE_PATH" 2501 | -------------------------------------------------------------------------------- /etc/astyle-configurations/1.conf: -------------------------------------------------------------------------------- 1 | # source: https://raw.githubusercontent.com/arduino/Arduino/1.8.9/build/shared/lib/formatter.conf 2 | 3 | # This configuration file contains a selection of the available options provided by the formatting tool "Artistic Style" 4 | # http://astyle.sourceforge.net/astyle.html 5 | # 6 | # If you wish to change them, don't edit this file. 7 | # Instead, copy it in the same folder of file "preferences.txt" and modify the copy. This way, you won't lose your custom formatter settings when upgrading the IDE 8 | # If you don't know where file preferences.txt is stored, open the IDE, File -> Preferences and you'll find a link 9 | 10 | mode=c 11 | 12 | # 2 spaces indentation 13 | indent=spaces=2 14 | 15 | # also indent macros 16 | indent-preprocessor 17 | 18 | # indent classes, switches (and cases), comments starting at column 1 19 | indent-classes 20 | indent-switches 21 | indent-cases 22 | indent-col1-comments 23 | 24 | # put a space around operators 25 | pad-oper 26 | 27 | # put a space after if/for/while 28 | pad-header 29 | 30 | # if you like one-liners, keep them 31 | keep-one-line-statements 32 | 33 | remove-comment-prefix 34 | -------------------------------------------------------------------------------- /etc/astyle-configurations/2.conf: -------------------------------------------------------------------------------- 1 | # source: https://raw.githubusercontent.com/arduino/Arduino/1.8.9/build/shared/examples_formatter.conf 2 | 3 | # This configuration file contains a selection of the available options provided by the formatting tool "Artistic Style" 4 | # http://astyle.sourceforge.net/astyle.html 5 | # 6 | # If you wish to change them, don't edit this file. 7 | # Instead, copy it in the same folder of file "preferences.txt" and modify the copy. This way, you won't lose your custom formatter settings when upgrading the IDE 8 | # If you don't know where file preferences.txt is stored, open the IDE, File -> Preferences and you'll find a link 9 | 10 | mode=c 11 | 12 | # 2 spaces indentation 13 | indent=spaces=2 14 | 15 | # also indent macros 16 | indent-preprocessor 17 | 18 | # indent classes, switches (and cases), comments starting at column 1 19 | indent-classes 20 | indent-switches 21 | indent-cases 22 | indent-col1-comments 23 | 24 | # put a space around operators 25 | pad-oper 26 | 27 | # put a space after if/for/while 28 | pad-header 29 | 30 | # if you like one-liners, keep them 31 | keep-one-line-statements 32 | 33 | style=java 34 | attach-namespaces 35 | attach-classes 36 | attach-inlines 37 | attach-extern-c 38 | indent-modifiers 39 | indent-namespaces 40 | indent-labels 41 | indent-preproc-block 42 | indent-preproc-define 43 | indent-preproc-cond 44 | unpad-paren 45 | add-brackets 46 | remove-comment-prefix 47 | -------------------------------------------------------------------------------- /etc/astyle-configurations/3.conf: -------------------------------------------------------------------------------- 1 | # source: https://github.com/arduino/ArduinoCore-avr/issues/71#issuecomment-466763471 2019-06-09 2 | 3 | # formatter.conf, examples_formatter.conf 4 | mode=c 5 | 6 | 7 | # examples_formatter.conf 8 | # http://astyle.sourceforge.net/astyle.html#_style=java 9 | # Considering changing this to the synonym "style=attach", which seems more descriptive 10 | style=java 11 | 12 | 13 | # examples_formatter.conf 14 | attach-namespaces 15 | 16 | # examples_formatter.conf 17 | attach-classes 18 | 19 | # examples_formatter.conf 20 | attach-inlines 21 | 22 | # examples_formatter.conf 23 | attach-extern-c 24 | 25 | 26 | # formatter.conf, examples_formatter.conf 27 | indent=spaces=2 28 | 29 | # formatter.conf, examples_formatter.conf 30 | indent-classes 31 | 32 | # formatter.conf, examples_formatter.conf 33 | indent-switches 34 | 35 | # formatter.conf, examples_formatter.conf 36 | indent-cases 37 | 38 | # formatter.conf, examples_formatter.conf 39 | indent-col1-comments 40 | 41 | # examples_formatter.conf 42 | indent-modifiers 43 | 44 | # examples_formatter.conf 45 | indent-namespaces 46 | 47 | # examples_formatter.conf 48 | indent-labels 49 | 50 | # examples_formatter.conf 51 | indent-preproc-define 52 | 53 | 54 | # formatter.conf, examples_formatter.conf 55 | pad-header 56 | 57 | # formatter.conf, examples_formatter.conf 58 | pad-oper 59 | 60 | # examples_formatter.conf 61 | unpad-paren 62 | 63 | 64 | # formatter.conf, examples_formatter.conf 65 | remove-comment-prefix 66 | 67 | # formatter.conf, examples_formatter.conf 68 | # http://astyle.sourceforge.net/astyle.html#_keep-one-line-statements 69 | # "Don't break complex statements and multiple statements residing on a single line." 70 | # I don't like one line complex statements, but I guess since it's in formatter.conf it must stay. 71 | keep-one-line-statements 72 | 73 | 74 | 75 | # Options from examples_formatter.conf that I think should be removed: 76 | 77 | # http://astyle.sourceforge.net/astyle.html#_indent-preproc-block 78 | # "Indent preprocessor blocks at brace level zero and immediately within a namespace. There are restrictions on what will be indented. Blocks within methods, classes, arrays, etc., will not be indented. Blocks containing braces or multi-line define statements will not be indented. Without this option the preprocessor block is not indented." 79 | # This does indent for #ifdef, but not for #ifndef, so it's quite inconsistent 80 | # Indentation of preprocessor directives as done by this option is not very common in Arduino AVR Boards core, and where it is used, it's typically done inconsistently throughout the file 81 | indent-preproc-block 82 | 83 | # http://astyle.sourceforge.net/astyle.html#_indent-preproc-cond 84 | # "Indent preprocessor conditional statements to the same level as the source code." 85 | # Indentation of preprocessor directives as done by this option is very rare in Arduino AVR Boards core 86 | indent-preproc-cond 87 | 88 | 89 | 90 | # Options I have not implemented from formatter.conf or examples_formatter.conf: 91 | 92 | # examples_formatter.conf 93 | # Not a valid option in the latest version of AStyle. I think the correct option name is "add-braces", which I do use in my configuration 94 | # add-brackets 95 | 96 | # formatter.conf, examples_formatter.conf 97 | # Not a valid option in the latest version of AStyle. 98 | # indent-preprocessor 99 | 100 | 101 | 102 | # Options I have added: 103 | 104 | # http://astyle.sourceforge.net/astyle.html#_add-braces 105 | # "I believe this is the correct option name to use instead the "add-brackets" option used in examples_formatter.conf. "add-brackets" is not a valid option in the latest version of AStyle" 106 | add-braces 107 | 108 | # http://astyle.sourceforge.net/astyle.html#_convert-tabs 109 | # "Converts tabs into spaces in the non-indentation part of the line." 110 | # AStyle is already configured to use spaces for indentation by indent=spaces=2. The Arduino IDE uses spaces instead of tabs by default. 111 | convert-tabs 112 | 113 | # http://astyle.sourceforge.net/astyle.html#_attach-return-type 114 | # "Attach the return type to the function name in function definitions." 115 | attach-return-type 116 | 117 | # http://astyle.sourceforge.net/astyle.html#_attach-return-type 118 | # "Attach the return type to the function name in function declarations." 119 | attach-return-type-decl 120 | 121 | # http://astyle.sourceforge.net/astyle.html#_align-pointer 122 | # "Attach a pointer or reference operator (*, &, or ^) to either the variable type (left) or variable name (right), or place it between the type and name (middle)." 123 | # In Arduino AVR Boards core, name alignment of pointers is somewhat more common, though all possible styles are used 124 | # I don't care which style is chosen (type, middle, name), but I do think one should be chosen and used. 125 | align-pointer=name 126 | -------------------------------------------------------------------------------- /etc/autoformat.sh: -------------------------------------------------------------------------------- 1 | shfmt -i 2 -w ../arduino-ci-script.sh 2 | -------------------------------------------------------------------------------- /etc/codespell-ignore-words-list.txt: -------------------------------------------------------------------------------- 1 | ba 2 | propert 3 | te 4 | exampels 5 | --------------------------------------------------------------------------------