├── .gitignore ├── CHANGELOG.md ├── CONTRIBUTING.md ├── LICENSE ├── README.md ├── __phutil_library_init__.php ├── __phutil_library_map__.php ├── engine └── XcodeUnitTestEngine.php └── parser └── XcodeTestResultParser.php /.gitignore: -------------------------------------------------------------------------------- 1 | .phutil_module_cache 2 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # 3.1.0 2 | 3 | * Removed Xcode 8 betas 1-5 workaround. Beta 6 resolved the bug; please upgrade to Xcode 8 beta 6 before upgrading to 3.1.0 of this library. 4 | 5 | # 3.0.7 6 | 7 | * Resolved bug in which a failed unit test on Xcode 8 betas would cause arc unit to hang. 8 | 9 | # 3.0.6 10 | 11 | * Workaround for bug in Xcode 8 betas 1 and 2 that cause `arc unit` to hang. 12 | 13 | # 3.0.5 14 | 15 | * Resolved bug where coverage information was being lost for the last file in a coverage report. 16 | 17 | # 3.0.4 18 | 19 | * Resolved [issue #3](https://github.com/google/arc-xcode-test-engine/issues/3): Using test engine with Quick testing library gets "Unable to parse line" error. Thanks to [KevM](https://github.com/KevM) for fixing. 20 | 21 | # 3.0.3 22 | 23 | * Resolved coverage failure with Xcode 7.3 and newer. Thanks to [aelam](https://github.com/aelam) for filing [#2](https://github.com/google/arc-xcode-test-engine/issues/2). 24 | 25 | # 3.0.2 26 | 27 | * Resolves crash caused by referencing the wrong stderr variable. The error was 28 | `Attempt to read from undeclared property XcodeUnitTestEngine::stderr`. 29 | 30 | # 3.0.1 31 | 32 | * If tests fail to build we no longer generate a 33 | `Unhandled Exception ("PhutilTypeMissingParametersException")` exception on Phabricator. We were 34 | not previously providing a name to the ArcanistUnitTestResult instance. 35 | 36 | # 3.0.0 37 | 38 | * Commands now always run from the root directory of the project rather than from the shell's 39 | current working directory. This may be a breaking change. 40 | 41 | # 2.1.1 42 | 43 | * Improved overall test parsing time by approximately 50%. 44 | 45 | # 2.1.0 46 | 47 | * Added a `pre-build` configuration option. 48 | 49 | # 2.0.2 50 | 51 | * Error line detection is now more flexible. Any output line with "error:" will be detected and 52 | bubble up as an error. 53 | 54 | # 2.0.1 55 | 56 | * Better handling of non-unit-test failures, such as CocoaPods getting out of sync. 57 | 58 | # 2.0.0 59 | 60 | * Coverage is now enabled and **reported** unless explicitly disabled with --no-coverage. 61 | * xcodebuild or llvm-cov failures now properly throw exceptions rather than silently continuing. 62 | 63 | # 1.1.0 64 | 65 | * Coverage is now enabled unless explicitly disabled with --no-coverage. 66 | 67 | # 1.0.1 68 | 69 | * Don't run unit tests if no files were provided to the engine and we're not being asked 70 | to run all tests. 71 | 72 | # 1.0.0 73 | 74 | * Initial release. 75 | * Provides `arc unit` and `arc unit --coverage` support for a single xcode project/target 76 | per-project. 77 | -------------------------------------------------------------------------------- /CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | Want to contribute? Great! First, read this page (including the small print at the end). 2 | 3 | ## Pull requests 4 | 5 | Pull requests can be hard to review if they try to tackle too many things 6 | at once. Phabricator's "[Writing Reviewable Code](https://secure.phabricator.com/book/phabflavor/article/writing_reviewable_code/)" 7 | provides a set of guidelines that help increase the likelihood of your 8 | pull request getting merged. 9 | 10 | In short (slightly modified from the original article): 11 | 12 | - A pull request should be as small as possible, but no smaller. 13 | - The smallest a pull request can be is a single cohesive idea: don't 14 | make pull requests so small that they are meaningless on their own. 15 | - Turn large pull requests into small pull requests by dividing large 16 | problems into smaller problems and solving the small problems one at 17 | a time. 18 | - Write sensible pull request descriptions. 19 | 20 | ### Code review 21 | 22 | All submissions, including submissions by project members, require review. We 23 | use GitHub pull requests for this purpose. 24 | 25 | ### Before you contribute 26 | 27 | Before we can use your code, you must sign the 28 | [Google Individual Contributor License Agreement](https://developers.google.com/open-source/cla/individual?csw=1) 29 | (CLA), which you can do online. The CLA is necessary mainly because you own the 30 | copyright to your changes, even after your contribution becomes part of our 31 | codebase, so we need your permission to use and distribute your code. We also 32 | need to be sure of various other things—for instance that you'll tell us if you 33 | know that your code infringes on other people's patents. You don't have to sign 34 | the CLA until after you've submitted your code for review and a member has 35 | approved it, but you must do it before we can put your code into our codebase. 36 | Before you start working on a larger contribution, you should get in touch with 37 | us first through the issue tracker with your idea so that we can help out and 38 | possibly guide you. Coordinating up front makes it much easier to avoid 39 | frustration later on. 40 | 41 | ### The small print 42 | 43 | Contributions made by corporations are covered by a different agreement than 44 | the one above, the [Software Grant and Corporate Contributor License Agreement](https://cla.developers.google.com/about/google-corporate). 45 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | 2 | Apache License 3 | Version 2.0, January 2004 4 | http://www.apache.org/licenses/ 5 | 6 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 7 | 8 | 1. Definitions. 9 | 10 | "License" shall mean the terms and conditions for use, reproduction, 11 | and distribution as defined by Sections 1 through 9 of this document. 12 | 13 | "Licensor" shall mean the copyright owner or entity authorized by 14 | the copyright owner that is granting the License. 15 | 16 | "Legal Entity" shall mean the union of the acting entity and all 17 | other entities that control, are controlled by, or are under common 18 | control with that entity. For the purposes of this definition, 19 | "control" means (i) the power, direct or indirect, to cause the 20 | direction or management of such entity, whether by contract or 21 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 22 | outstanding shares, or (iii) beneficial ownership of such entity. 23 | 24 | "You" (or "Your") shall mean an individual or Legal Entity 25 | exercising permissions granted by this License. 26 | 27 | "Source" form shall mean the preferred form for making modifications, 28 | including but not limited to software source code, documentation 29 | source, and configuration files. 30 | 31 | "Object" form shall mean any form resulting from mechanical 32 | transformation or translation of a Source form, including but 33 | not limited to compiled object code, generated documentation, 34 | and conversions to other media types. 35 | 36 | "Work" shall mean the work of authorship, whether in Source or 37 | Object form, made available under the License, as indicated by a 38 | copyright notice that is included in or attached to the work 39 | (an example is provided in the Appendix below). 40 | 41 | "Derivative Works" shall mean any work, whether in Source or Object 42 | form, that is based on (or derived from) the Work and for which the 43 | editorial revisions, annotations, elaborations, or other modifications 44 | represent, as a whole, an original work of authorship. For the purposes 45 | of this License, Derivative Works shall not include works that remain 46 | separable from, or merely link (or bind by name) to the interfaces of, 47 | the Work and Derivative Works thereof. 48 | 49 | "Contribution" shall mean any work of authorship, including 50 | the original version of the Work and any modifications or additions 51 | to that Work or Derivative Works thereof, that is intentionally 52 | submitted to Licensor for inclusion in the Work by the copyright owner 53 | or by an individual or Legal Entity authorized to submit on behalf of 54 | the copyright owner. For the purposes of this definition, "submitted" 55 | means any form of electronic, verbal, or written communication sent 56 | to the Licensor or its representatives, including but not limited to 57 | communication on electronic mailing lists, source code control systems, 58 | and issue tracking systems that are managed by, or on behalf of, the 59 | Licensor for the purpose of discussing and improving the Work, but 60 | excluding communication that is conspicuously marked or otherwise 61 | designated in writing by the copyright owner as "Not a Contribution." 62 | 63 | "Contributor" shall mean Licensor and any individual or Legal Entity 64 | on behalf of whom a Contribution has been received by Licensor and 65 | subsequently incorporated within the Work. 66 | 67 | 2. Grant of Copyright License. Subject to the terms and conditions of 68 | this License, each Contributor hereby grants to You a perpetual, 69 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 70 | copyright license to reproduce, prepare Derivative Works of, 71 | publicly display, publicly perform, sublicense, and distribute the 72 | Work and such Derivative Works in Source or Object form. 73 | 74 | 3. Grant of Patent License. Subject to the terms and conditions of 75 | this License, each Contributor hereby grants to You a perpetual, 76 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 77 | (except as stated in this section) patent license to make, have made, 78 | use, offer to sell, sell, import, and otherwise transfer the Work, 79 | where such license applies only to those patent claims licensable 80 | by such Contributor that are necessarily infringed by their 81 | Contribution(s) alone or by combination of their Contribution(s) 82 | with the Work to which such Contribution(s) was submitted. If You 83 | institute patent litigation against any entity (including a 84 | cross-claim or counterclaim in a lawsuit) alleging that the Work 85 | or a Contribution incorporated within the Work constitutes direct 86 | or contributory patent infringement, then any patent licenses 87 | granted to You under this License for that Work shall terminate 88 | as of the date such litigation is filed. 89 | 90 | 4. Redistribution. You may reproduce and distribute copies of the 91 | Work or Derivative Works thereof in any medium, with or without 92 | modifications, and in Source or Object form, provided that You 93 | meet the following conditions: 94 | 95 | (a) You must give any other recipients of the Work or 96 | Derivative Works a copy of this License; and 97 | 98 | (b) You must cause any modified files to carry prominent notices 99 | stating that You changed the files; and 100 | 101 | (c) You must retain, in the Source form of any Derivative Works 102 | that You distribute, all copyright, patent, trademark, and 103 | attribution notices from the Source form of the Work, 104 | excluding those notices that do not pertain to any part of 105 | the Derivative Works; and 106 | 107 | (d) If the Work includes a "NOTICE" text file as part of its 108 | distribution, then any Derivative Works that You distribute must 109 | include a readable copy of the attribution notices contained 110 | within such NOTICE file, excluding those notices that do not 111 | pertain to any part of the Derivative Works, in at least one 112 | of the following places: within a NOTICE text file distributed 113 | as part of the Derivative Works; within the Source form or 114 | documentation, if provided along with the Derivative Works; or, 115 | within a display generated by the Derivative Works, if and 116 | wherever such third-party notices normally appear. The contents 117 | of the NOTICE file are for informational purposes only and 118 | do not modify the License. You may add Your own attribution 119 | notices within Derivative Works that You distribute, alongside 120 | or as an addendum to the NOTICE text from the Work, provided 121 | that such additional attribution notices cannot be construed 122 | as modifying the License. 123 | 124 | You may add Your own copyright statement to Your modifications and 125 | may provide additional or different license terms and conditions 126 | for use, reproduction, or distribution of Your modifications, or 127 | for any such Derivative Works as a whole, provided Your use, 128 | reproduction, and distribution of the Work otherwise complies with 129 | the conditions stated in this License. 130 | 131 | 5. Submission of Contributions. Unless You explicitly state otherwise, 132 | any Contribution intentionally submitted for inclusion in the Work 133 | by You to the Licensor shall be under the terms and conditions of 134 | this License, without any additional terms or conditions. 135 | Notwithstanding the above, nothing herein shall supersede or modify 136 | the terms of any separate license agreement you may have executed 137 | with Licensor regarding such Contributions. 138 | 139 | 6. Trademarks. This License does not grant permission to use the trade 140 | names, trademarks, service marks, or product names of the Licensor, 141 | except as required for reasonable and customary use in describing the 142 | origin of the Work and reproducing the content of the NOTICE file. 143 | 144 | 7. Disclaimer of Warranty. Unless required by applicable law or 145 | agreed to in writing, Licensor provides the Work (and each 146 | Contributor provides its Contributions) on an "AS IS" BASIS, 147 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 148 | implied, including, without limitation, any warranties or conditions 149 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 150 | PARTICULAR PURPOSE. You are solely responsible for determining the 151 | appropriateness of using or redistributing the Work and assume any 152 | risks associated with Your exercise of permissions under this License. 153 | 154 | 8. Limitation of Liability. In no event and under no legal theory, 155 | whether in tort (including negligence), contract, or otherwise, 156 | unless required by applicable law (such as deliberate and grossly 157 | negligent acts) or agreed to in writing, shall any Contributor be 158 | liable to You for damages, including any direct, indirect, special, 159 | incidental, or consequential damages of any character arising as a 160 | result of this License or out of the use or inability to use the 161 | Work (including but not limited to damages for loss of goodwill, 162 | work stoppage, computer failure or malfunction, or any and all 163 | other commercial damages or losses), even if such Contributor 164 | has been advised of the possibility of such damages. 165 | 166 | 9. Accepting Warranty or Additional Liability. While redistributing 167 | the Work or Derivative Works thereof, You may choose to offer, 168 | and charge a fee for, acceptance of support, warranty, indemnity, 169 | or other liability obligations and/or rights consistent with this 170 | License. However, in accepting such obligations, You may act only 171 | on Your own behalf and on Your sole responsibility, not on behalf 172 | of any other Contributor, and only if You agree to indemnify, 173 | defend, and hold each Contributor harmless for any liability 174 | incurred by, or claims asserted against, such Contributor by reason 175 | of your accepting any such warranty or additional liability. 176 | 177 | END OF TERMS AND CONDITIONS 178 | 179 | APPENDIX: How to apply the Apache License to your work. 180 | 181 | To apply the Apache License to your work, attach the following 182 | boilerplate notice, with the fields enclosed by brackets "[]" 183 | replaced with your own identifying information. (Don't include 184 | the brackets!) The text should be enclosed in the appropriate 185 | comment syntax for the file format. We also recommend that a 186 | file or class name and description of purpose be included on the 187 | same "printed page" as the copyright notice for easier 188 | identification within third-party archives. 189 | 190 | Copyright [yyyy] [name of copyright owner] 191 | 192 | Licensed under the Apache License, Version 2.0 (the "License"); 193 | you may not use this file except in compliance with the License. 194 | You may obtain a copy of the License at 195 | 196 | http://www.apache.org/licenses/LICENSE-2.0 197 | 198 | Unless required by applicable law or agreed to in writing, software 199 | distributed under the License is distributed on an "AS IS" BASIS, 200 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 201 | See the License for the specific language governing permissions and 202 | limitations under the License. 203 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # xcode-test-engine for arc 2 | 3 | xcode-test-engine is a test engine for use with [Phabricator](http://phabricator.org)'s `arc` 4 | command line tool. 5 | 6 | ## Features 7 | 8 | xcode-test-engine presently works well with projects that have a 9 | single root workspace and a single unit test target. 10 | 11 | Supports `arc unit`: 12 | 13 | ~ $ arc unit 14 | PASS 42ms★ MyTests::Test1 15 | PASS 3ms★ MyTests::Test2 16 | PASS 2ms★ MyTests::Test3 17 | PASS 2ms★ MyTests::Test4 18 | PASS 1ms★ MyTests::Test5 19 | 20 | And `arc unit --coverage`: 21 | 22 | ~ $ arc unit --coverage 23 | COVERAGE REPORT 24 | 5% src/Widget.m 25 | 18% src/SomeDirectory/AnotherWidget.m 26 | 69% src/SomeDirectory/WidgetBuilder.m 27 | 80% src/Controller/WidgetViewController.m 28 | 93% src/Controller/WidgetTableViewController.m 29 | 30 | ## Installation 31 | 32 | ### Project-specific 33 | 34 | Add this repository as a git submodule. 35 | 36 | git submodule init 37 | git submodule add 38 | 39 | Your `.arcconfig` should list `xcode-test-engine` in the `load` 40 | configuration: 41 | 42 | { 43 | "load": [ 44 | "path/to/arc-xcode-test-engine" 45 | ] 46 | } 47 | 48 | ### Global 49 | 50 | Clone this repository to the same directory where `arcanist` and 51 | `libphutil` are globally located. Your directory structure will 52 | look like so: 53 | 54 | arcanist/ 55 | libphutil/ 56 | arc-xcode-test-engine/ 57 | 58 | Your `.arcconfig` should list `arc-xcode-test-engine` in the `load` 59 | configuration (without a path): 60 | 61 | { 62 | "load": [ 63 | "arc-xcode-test-engine" 64 | ] 65 | } 66 | 67 | ## Usage 68 | 69 | Create a `.arcunit` file in the root of your project and add the following content: 70 | 71 | { 72 | "engines": { 73 | "xcode": { 74 | "type": "xcode-test-engine", 75 | "include": "(\\.(m|h|mm|swift)$)", 76 | "exclude": "(/Pods/)" 77 | } 78 | } 79 | } 80 | 81 | Feel free to change the include/exclude regexes to suit your project's needs. 82 | 83 | Now modify your `.arcconfig` file by adding the following configuration: 84 | 85 | { 86 | "unit.xcode": { 87 | "build": { 88 | "workspace": "path/to/Workspace.xcworkspace", 89 | "scheme": "UnitTestsScheme", 90 | "configuration": "Debug", 91 | "destination": "platform=iOS Simulator,name=iPhone 6S" 92 | }, 93 | "coverage": { 94 | "product": "SomeFramework.framework/SomeFramework" 95 | } 96 | } 97 | } 98 | 99 | ### Configuration options 100 | 101 | #### Build configuration options 102 | 103 | Any value provided to "build" will be passed along to xcodebuild as a flag. 104 | 105 | "unit.xcode": { 106 | "build": { ... } 107 | } 108 | 109 | #### Picking the coverage product 110 | 111 | Provide the path to the product for which coverage should be calculated. If building a 112 | library/framework this might be the framework binary product. 113 | 114 | "unit.xcode": { 115 | "coverage": { 116 | "product": " ... " 117 | } 118 | } 119 | 120 | #### Pre-build commands 121 | 122 | Some projects require a pre-build script that runs before the xcodebuild command. 123 | 124 | "unit.xcode": { 125 | "pre-build": "execute this command before the build step" 126 | } 127 | 128 | For example, CocoaPods projects may need to be re-generated. 129 | 130 | "pre-build": "pod install --project-directory=path/ --no-repo-update" 131 | 132 | ### Supporting multiple test engines 133 | 134 | We support Arcanist's *multi test engine* extension. Visit the [extension]((https://github.com/tagview/arcanist-extensions#multi_test_engine) page to learn more about it. 135 | 136 | Your `.arcconfig` file will have something similar to the following configuration: 137 | 138 | ``` 139 | { 140 | "unit.engine": "MultiTestEngine", 141 | "unit.engine.multi-test.engines": [ 142 | { 143 | "engine": "XcodeUnitTestEngine", 144 | "unit.engine.xcode.config": { 145 | "build": { 146 | "workspace": "YourWorkSpace.xcworkspace", 147 | "scheme": "Scheme1", 148 | "configuration": "Debug", 149 | "destination": "platform=iOS Simulator,name=iPhone SE" 150 | }, 151 | "coverage": { 152 | "product": "SomeFramework.framework/SomeFramework" 153 | } 154 | } 155 | }, 156 | { 157 | "engine": "XcodeUnitTestEngine", 158 | "unit.engine.xcode.config": { 159 | "build": { 160 | "workspace": "YourWorkSpace.xcworkspace", 161 | "scheme": "AnotherScheme", 162 | "configuration": "Debug" 163 | } 164 | } 165 | } 166 | ] 167 | } 168 | ``` 169 | 170 | ### Viewing coverage results in Phabricator 171 | 172 | Coverage will appear for affected source files in Side-by-Side mode as a colored bar. 173 | 174 | ## License 175 | 176 | Licensed under the Apache 2.0 license. See LICENSE for details. 177 | -------------------------------------------------------------------------------- /__phutil_library_init__.php: -------------------------------------------------------------------------------- 1 | 2, 11 | 'class' => array( 12 | 'XcodeTestResultParser' => 'parser/XcodeTestResultParser.php', 13 | 'XcodeUnitTestEngine' => 'engine/XcodeUnitTestEngine.php', 14 | ), 15 | 'function' => array(), 16 | 'xmap' => array( 17 | 'XcodeTestResultParser' => 'ArcanistTestResultParser', 18 | 'XcodeUnitTestEngine' => 'ArcanistUnitTestEngine', 19 | ), 20 | )); 21 | -------------------------------------------------------------------------------- /engine/XcodeUnitTestEngine.php: -------------------------------------------------------------------------------- 1 | getEnableCoverage(); 55 | 56 | return ($arcCoverageFlag !== false) && $this->hasCoverageKey; 57 | } 58 | 59 | private function checkArcConfigFile() { 60 | // Throws exception if the config file `.arcconfig` is missing or it's in invalid JSON format. 61 | 62 | $this->projectRoot = $this->getWorkingCopy()->getProjectRoot(); 63 | $config_path = $this->getWorkingCopy()->getProjectPath('.arcconfig'); 64 | 65 | if (!Filesystem::pathExists($config_path)) { 66 | throw new ArcanistUsageException( 67 | pht( 68 | "Unable to find '%s' file to configure xcode-test engine. Create an ". 69 | "'%s' file in the root directory of the working copy.", 70 | '.arcconfig', 71 | '.arcconfig')); 72 | } 73 | 74 | $data = Filesystem::readFile($config_path); 75 | try { 76 | phutil_json_decode($data); 77 | } catch (PhutilJSONParserException $ex) { 78 | throw new PhutilProxyException( 79 | pht( 80 | "Expected '%s' file to be a valid JSON file, but ". 81 | "failed to decode '%s'.", 82 | '.arcconfig', 83 | $config_path), 84 | $ex); 85 | } 86 | } 87 | 88 | protected function loadEnvironment() { 89 | 90 | $this->checkArcConfigFile(); 91 | 92 | $config_manager = $this->getConfigurationManager(); 93 | if ($config_manager == null) { 94 | throw new ArcanistUsageException( 95 | pht( 96 | "Unable to setup configuration manager. Make sure that ". 97 | "unit.engine is set properly in '%s'", 98 | '.arcconfig')); 99 | } 100 | $unit_config = $config_manager->getConfigFromAnySource('unit.xcode'); 101 | 102 | if ($unit_config == null) { 103 | throw new ArcanistUsageException( 104 | pht( 105 | "Unable to find '%s' keys in .arcconfig.", 106 | 'unit.xcode')); 107 | } 108 | 109 | $this->xcodebuild = $unit_config['build']; 110 | 111 | $this->hasCoverageKey = array_key_exists('coverage', $unit_config); 112 | 113 | if ($this->shouldGenerateCoverage()) { 114 | $this->xcodebuild["enableCodeCoverage"] = "YES"; 115 | $this->coverage = $unit_config['coverage']; 116 | } else { 117 | $this->xcodebuild["enableCodeCoverage"] = "NO"; 118 | } 119 | 120 | if (array_key_exists('pre-build', $unit_config)) { 121 | $this->preBuildCommand = $unit_config['pre-build']; 122 | } 123 | } 124 | 125 | public function run() { 126 | $this->loadEnvironment(); 127 | 128 | if (!$this->getRunAllTests()) { 129 | $paths = $this->getPaths(); 130 | if (empty($paths)) { 131 | return array(); 132 | } 133 | } 134 | 135 | $xcodeargs = array(); 136 | foreach ($this->xcodebuild as $key => $value) { 137 | if ($key == "workspace") { 138 | $value = $this->projectRoot."/".$value; 139 | } 140 | $xcodeargs []= "-$key \"$value\""; 141 | } 142 | 143 | if (!empty($this->preBuildCommand)) { 144 | $future = new ExecFuture($this->preBuildCommand); 145 | $future->setCWD(Filesystem::resolvePath($this->getWorkingCopy()->getProjectRoot())); 146 | $future->resolvex(); 147 | } 148 | 149 | // Build and run unit tests 150 | $future = new ExecFuture('%C %C test', 151 | $this->xcodebuildBinary, implode(' ', $xcodeargs)); 152 | 153 | list($builderror, $xcbuild_stdout, $xcbuild_stderr) = $future->resolve(); 154 | 155 | // Error-code 65 is thrown for build/unit test failures. 156 | if ($builderror !== 0 && $builderror !== 65) { 157 | return array(id(new ArcanistUnitTestResult()) 158 | ->setName("Xcode test engine") 159 | ->setUserData($xcbuild_stderr) 160 | ->setResult(ArcanistUnitTestResult::RESULT_BROKEN)); 161 | } 162 | 163 | // Extract coverage information 164 | $coverage = null; 165 | if ($builderror === 0 && $this->shouldGenerateCoverage()) { 166 | // Get the OBJROOT 167 | $future = new ExecFuture('%C %C -showBuildSettings test', 168 | $this->xcodebuildBinary, implode(' ', $xcodeargs)); 169 | $future->setCWD(Filesystem::resolvePath($this->getWorkingCopy()->getProjectRoot())); 170 | list(, $settings_stdout, ) = $future->resolve(); 171 | if (!preg_match('/OBJROOT = (.+)/', $settings_stdout, $matches)) { 172 | throw new Exception('Unable to find OBJROOT configuration.'); 173 | } 174 | $objroot = $matches[1]; 175 | 176 | $future = new ExecFuture("find %C -name Coverage.profdata", $objroot); 177 | list(, $coverage_stdout, ) = $future->resolve(); 178 | $profdata_path = explode("\n", $coverage_stdout)[0]; 179 | 180 | $future = new ExecFuture("find %C | grep %C", $objroot, $this->coverage['product']); 181 | list(, $product_stdout, ) = $future->resolve(); 182 | $product_path = explode("\n", $product_stdout)[0]; 183 | 184 | $future = new ExecFuture('%C show -use-color=false -instr-profile "%C" "%C"', 185 | $this->covBinary, $profdata_path, $product_path); 186 | $future->setCWD(Filesystem::resolvePath($this->getWorkingCopy()->getProjectRoot())); 187 | 188 | try { 189 | list($coverage, $coverage_error) = $future->resolvex(); 190 | } catch (CommandException $exc) { 191 | if ($exc->getError() != 0) { 192 | throw $exc; 193 | } 194 | } 195 | } 196 | 197 | // TODO(featherless): If we publicized the parseCoverageResults method on 198 | // XcodeTestResultParser we could parseTestResults, then call parseCoverageResults, 199 | // and the logic here would map the coverage results to the test results. This 200 | // might be a cleaner approach. 201 | 202 | return id(new XcodeTestResultParser()) 203 | ->setEnableCoverage($this->shouldGenerateCoverage()) 204 | ->setCoverageFile($coverage) 205 | ->setProjectRoot($this->projectRoot) 206 | ->setXcodeArgs($xcodeargs) 207 | ->setStderr($xcbuild_stderr) 208 | ->parseTestResults(null, $xcbuild_stdout); 209 | } 210 | } 211 | -------------------------------------------------------------------------------- /parser/XcodeTestResultParser.php: -------------------------------------------------------------------------------- 1 | xcodeargs = $args; 30 | return $this; 31 | } 32 | 33 | /** 34 | * Parses stdout of `xcodebuild` as provided via $test_results and returns an 35 | * array of ArcanistUnitTestResult instances. 36 | */ 37 | public function parseTestResults($path, $test_results) { 38 | if (!$test_results) { 39 | $result = id(new ArcanistUnitTestResult()) 40 | ->setName("Xcode test engine") 41 | ->setUserData($this->stderr) 42 | ->setResult(ArcanistUnitTestResult::RESULT_BROKEN); 43 | return array($result); 44 | } 45 | 46 | $arccovmap = null; 47 | if ($this->enableCoverage) { 48 | $arccovmap = $this->parseCoverageResults(); 49 | } 50 | 51 | $results = array(); 52 | $accumulator = array(); 53 | $suite = null; 54 | 55 | $currentTestCase = null; 56 | // Break the test result stdout into lines. 57 | foreach(preg_split("/((\r?\n)|(\r\n?))/", $test_results) as $line) { 58 | if ($this->startsWith($line, 'Test Suite')) { 59 | if (preg_match('/Test Suite \'(.+?)\' started/', $line, $matches)) { 60 | $suite = $matches[1]; 61 | } else { 62 | $suite = null; 63 | } 64 | continue; 65 | } 66 | // Get the current test case name 67 | $testCase = $this->testCaseNameFromLine($line); 68 | if ($testCase != null) { 69 | $currentTestCase = $testCase; 70 | } 71 | // Check if the 'error:' exits and and this error is from a test case and NOT a log. 72 | // The test failure contains the test case name and we check that. 73 | if (strpos($line, 'error:') !== false && $currentTestCase != null && strpos($line, $currentTestCase) !== false) { 74 | $result = new ArcanistUnitTestResult(); 75 | $result->setName('xcode-unit-engine'); 76 | foreach ($this->xcodeargs as $arg) { 77 | if (preg_match('/-scheme (.+)/', $arg, $matches)) { 78 | $result->setName(trim($matches[1], "'\"")); 79 | break; 80 | } 81 | } 82 | $result->setResult(ArcanistUnitTestResult::RESULT_BROKEN); 83 | $result->setUserData($line); 84 | $results []= $result; 85 | continue; 86 | } 87 | // Test has started? 88 | if ($this->endsWith($line, 'started.')) { 89 | $accumulator = array(); 90 | continue; 91 | } 92 | // Within a test, informational line. 93 | if (!$this->startsWith($line, 'Test Case')) { 94 | // Treat these lines as information for the test case. 95 | $accumulator []= $line; 96 | continue; 97 | } 98 | 99 | // End of a test. 100 | if (!preg_match('/\'-\[\S*\s(?:test)*(.+?)\]\' (.+?) \((.+?) seconds\).$/', $line, $matches)) { 101 | echo "Unable to parse line:\n$line\n"; 102 | continue; 103 | } 104 | 105 | $userdata = implode("\n", $accumulator); 106 | $userdata = str_replace($this->projectRoot.'/', '', $userdata); 107 | 108 | $result = new ArcanistUnitTestResult(); 109 | $result->setName($matches[1]); 110 | if ($suite) { 111 | $result->setNamespace($suite); 112 | } 113 | switch ($matches[2]) { 114 | case 'passed': 115 | $result->setResult(ArcanistUnitTestResult::RESULT_PASS); 116 | break; 117 | case 'failed': 118 | $result->setResult(ArcanistUnitTestResult::RESULT_FAIL); 119 | break; 120 | } 121 | $result->setDuration(floatval($matches[3])); 122 | $result->setUserData($userdata); 123 | if ($arccovmap) { 124 | // TODO(featherless): Rather than provide the same coverage to every 125 | // test object, identify whether we can provide the coverage once 126 | // somewhere else. 127 | $result->setCoverage($arccovmap); 128 | } 129 | $results []= $result; 130 | 131 | $accumulator = array(); 132 | } 133 | return $results; 134 | } 135 | 136 | private function testCaseNameFromLine($line) { 137 | if ($this->startsWith($line, 'Test Case')) { 138 | $components = explode("'", $line); 139 | if (sizeof($components) >= 2) { 140 | return $components[1]; 141 | } 142 | } 143 | return null; 144 | } 145 | 146 | /** 147 | * Return a map of filename (relative to $this->projectRoot) to 148 | * https://secure.phabricator.com/book/phabricator/article/arcanist_coverage/ 149 | */ 150 | private function parseCoverageResults() { 151 | $filemap = array(); 152 | $filename = null; 153 | $file = array(); 154 | foreach(preg_split("/((\r?\n)|(\r\n?))/", $this->coverageFile) as $line) { 155 | if (substr($line, 0, 1) === '/') { 156 | if ($filename && $this->startsWith($filename, $this->projectRoot)) { 157 | $filemap[$filename] = $file; 158 | } 159 | $file = array(); 160 | $filename = $line; 161 | continue; 162 | } 163 | $file []= $line; 164 | } 165 | if ($file && $filename) { 166 | // Commit the hanging file. 167 | $filemap[$filename] = $file; 168 | } 169 | 170 | $arccovmap = array(); 171 | foreach ($filemap as $filename => $coverage) { 172 | $cov = ''; 173 | foreach ($coverage as $line) { 174 | if (preg_match('/^\s+\d+\|\s+(\d+)\|/', $line, $matches) 175 | && count($matches) > 1) { 176 | if (intval($matches[1]) > 0) { 177 | $cov .= 'C'; // Covered 178 | } else { 179 | $cov .= 'U'; // Uncovered 180 | } 181 | continue; 182 | } 183 | // Couldn't find a number 184 | $cov .= 'N'; // Not executable 185 | } 186 | $arcname = str_replace($this->projectRoot.'/', '', $filename); 187 | $arcname = str_replace(':', '', $arcname); 188 | $arccovmap[$arcname] = $cov; 189 | } 190 | return $arccovmap; 191 | } 192 | 193 | // http://stackoverflow.com/questions/834303/startswith-and-endswith-functions-in-php 194 | private function startsWith($haystack, $needle) { 195 | return $needle === "" || strrpos($haystack, $needle, -strlen($haystack)) !== false; 196 | } 197 | 198 | // http://stackoverflow.com/questions/834303/startswith-and-endswith-functions-in-php 199 | function endsWith($haystack, $needle) { 200 | return $needle === "" || (($temp = strlen($haystack) - strlen($needle)) >= 0 && strpos($haystack, $needle, $temp) !== false); 201 | } 202 | } 203 | --------------------------------------------------------------------------------