├── CHANGELOG.rst ├── CMakeLists.txt ├── README.md ├── cmake ├── Modules │ └── CodeCoverage.cmake ├── code_coverage-extras.cmake.develspace.in └── code_coverage-extras.cmake.installspace.in └── package.xml /CHANGELOG.rst: -------------------------------------------------------------------------------- 1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2 | Changelog for package code_coverage 3 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 4 | 5 | 0.4.4 (2020-10-29) 6 | ------------------ 7 | * Added python3-coverage support and error if python*-coverage is missing. (`#27 `_) 8 | * Contributors: Stefan Fabian 9 | 10 | 0.4.3 (2020-07-30) 11 | ------------------ 12 | * Use multiple python coverage files (`#23 `_) 13 | Co-authored-by: hslusarek 14 | * add note that robot_calibration uses this package 15 | * fix `#25 `_ 16 | * Add new report formats (`#24 `_) 17 | * Add html report format 18 | * Add console report format 19 | * Add '--include' and '--omit' flag to python-coverage commands 20 | * Contributors: Alexander Gutenkunst, Michael Ferguson, hslusarek 21 | 22 | 0.4.2 (2020-05-05) 23 | ------------------ 24 | * Add option for specifying extra flags to genhtml (`#20 `_) 25 | This modification allows you to add flags to the genhtml step so that you can do things like output the lcov report with demangled C++ function names, e.g.: 26 | catkin_make -DGENHTML_EXTRA_FLAGS="--demangle-cpp" -DENABLE_COVERAGE_TESTING=ON -DCMAKE_BUILD_TYPE=Debug test1_coverage_report 27 | * Add catkin_make build step in usage example (`#19 `_) 28 | * bump cmake version for noetic 29 | * Contributors: Immanuel Martini, Michael Ferguson, mschickler 30 | 31 | 0.4.1 (2020-04-05) 32 | ------------------ 33 | * update package.xml for noetic 34 | * Contributors: Michael Ferguson 35 | 36 | 0.4.0 (2020-04-02) 37 | ------------------ 38 | * add support for python-based coverage (`#18 `_) 39 | * Contributors: Michael Ferguson 40 | 41 | 0.3.0 (2019-11-18) 42 | ------------------ 43 | * update target name in readme 44 | * Merge pull request `#15 `_ from rhaschke/master 45 | Simplify + clarify usage 46 | * add link to original source 47 | * simplify usage 48 | - automatically include(CodeCoverage) 49 | - clarify that APPEND_COVERAGE_COMPILER_FLAGS() needs to be called before defining any target 50 | * Merge pull request `#13 `_ from leunMar/fix/usage_docu 51 | Fix example of add_code_coverage() in README.md 52 | * Fix example of add_code_coverage() in README.md 53 | * Contributors: Immanuel Martini, Michael Ferguson, Robert Haschke 54 | 55 | 0.2.4 (2019-07-18) 56 | ------------------ 57 | * Merge pull request `#11 `_ from agutenkunst/master 58 | Keep *.info.cleaned. Closes `#10 `_ 59 | * Keep *.info.cleaned. Closes `#10 `_ 60 | * Merge pull request `#8 `_ from jschleicher/documentation 61 | documentation: degrade to test_depend 62 | * documentation degrade to test_depend 63 | and note that build type should be set to Debug 64 | * Contributors: Alexander Gutenkunst, Joachim Schleicher, Michael Ferguson 65 | 66 | 0.2.3 (2018-08-24) 67 | ------------------ 68 | * Merge pull request `#7 `_ from jschleicher/fix/installed_package 69 | fix search path for installed package 70 | * fix search path for installed package 71 | Closes `#6 `_ 72 | * Contributors: Joachim Schleicher, Michael Ferguson 73 | 74 | 0.2.2 (2018-08-21) 75 | ------------------ 76 | * fix name of installspace file 77 | * Contributors: Michael Ferguson 78 | 79 | 0.2.1 (2018-08-13) 80 | ------------------ 81 | * Merge pull request `#3 `_ from mikeferguson/catkin_build_fix 82 | Add support for catkin_tools (fixes `#2 `_) 83 | * add information on how to run with catkin_tools 84 | * minor escaping patch to work with catkin_tools 85 | * Contributors: Michael Ferguson 86 | 87 | 0.2.0 (2018-08-01) 88 | ------------------ 89 | * First release 90 | * Contributors: Michael Ferguson 91 | -------------------------------------------------------------------------------- /CMakeLists.txt: -------------------------------------------------------------------------------- 1 | cmake_minimum_required(VERSION 3.0.2) 2 | project(code_coverage) 3 | 4 | find_package(catkin REQUIRED) 5 | 6 | catkin_package(CFG_EXTRAS code_coverage-extras.cmake) 7 | 8 | ## Install all cmake files 9 | install(DIRECTORY cmake/Modules 10 | DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION}/cmake) 11 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # code_coverage 2 | 3 | ROS package to run coverage testing 4 | 5 | ## Examples 6 | 7 | * The [robot_calibration](https://github.com/mikeferguson/robot_calibration) package uses this to generate coverage for C++ code using Travis-CI and Codecov.io 8 | 9 | ## Usage 10 | To use this with your ROS package: 11 | 12 | * Add code_coverage as a test depend in your package.xml 13 | * Update your CMakeLists.txt, in the testing section add the following. **NOTE** the order of test targets and coverage macros: 14 | ```cmake 15 | if(CATKIN_ENABLE_TESTING AND ENABLE_COVERAGE_TESTING) 16 | find_package(code_coverage REQUIRED) 17 | # Add compiler flags for coverage instrumentation before defining any targets 18 | APPEND_COVERAGE_COMPILER_FLAGS() 19 | endif() 20 | 21 | # Add your targets here 22 | 23 | if (CATKIN_ENABLE_TESTING) 24 | # Add your tests here 25 | 26 | # Create a target ${PROJECT_NAME}_coverage_report 27 | if(ENABLE_COVERAGE_TESTING) 28 | set(COVERAGE_EXCLUDES "*/${PROJECT_NAME}/test*" "*/${PROJECT_NAME}/other_dir_i_dont_care_about*") 29 | add_code_coverage( 30 | NAME ${PROJECT_NAME}_coverage_report 31 | DEPENDENCIES _run_tests_${PROJECT_NAME} 32 | ) 33 | endif() 34 | endif() 35 | ``` 36 | * **Note**: The variable `COVERAGE_EXCLUDES` must have some content! 37 | * Now you can build and run the tests (you need a debug build to get reasonable coverage numbers): 38 | 39 | - if using CATKIN_MAKE: 40 | ``` 41 | catkin_make -DENABLE_COVERAGE_TESTING=ON -DCMAKE_BUILD_TYPE=Debug 42 | catkin_make -DENABLE_COVERAGE_TESTING=ON -DCMAKE_BUILD_TYPE=Debug PACKAGE_NAME_coverage_report 43 | ``` 44 | - if using CATKIN_TOOLS: 45 | ``` 46 | catkin config --cmake-args -DENABLE_COVERAGE_TESTING=ON -DCMAKE_BUILD_TYPE=Debug 47 | catkin build 48 | catkin build PACKAGE_NAME -v --no-deps --catkin-make-args PACKAGE_NAME_coverage_report 49 | ``` 50 | 51 | * The output will print where the coverage report is located 52 | 53 | ## Python rostest Support 54 | 55 | While the C++ interface and Python-based unit tests require no 56 | modification to get coverage information, Python-based nodes 57 | run from rostest launch files need a bit of additional 58 | instrumentation turned on: 59 | 60 | ```xml 61 | 62 | 63 | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | 72 | 73 | 75 | 76 | 77 | 79 | 80 | 81 | ``` 82 | 83 | In the CMakeLists, you will need to pass this argument: 84 | 85 | ```cmake 86 | add_rostest(example_rostest.test ARGS coverage:=ENABLE_COVERAGE_TESTING) 87 | ``` 88 | -------------------------------------------------------------------------------- /cmake/Modules/CodeCoverage.cmake: -------------------------------------------------------------------------------- 1 | # Copyright (c) 2012 - 2017, Lars Bilke 2 | # All rights reserved. 3 | # From: https://github.com/bilke/cmake-modules/blob/master/CodeCoverage.cmake 4 | # 5 | # Redistribution and use in source and binary forms, with or without modification, 6 | # are permitted provided that the following conditions are met: 7 | # 8 | # 1. Redistributions of source code must retain the above copyright notice, this 9 | # list of conditions and the following disclaimer. 10 | # 11 | # 2. Redistributions in binary form must reproduce the above copyright notice, 12 | # this list of conditions and the following disclaimer in the documentation 13 | # and/or other materials provided with the distribution. 14 | # 15 | # 3. Neither the name of the copyright holder nor the names of its contributors 16 | # may be used to endorse or promote products derived from this software without 17 | # specific prior written permission. 18 | # 19 | # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND 20 | # ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED 21 | # WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 22 | # DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR 23 | # ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES 24 | # (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 25 | # LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON 26 | # ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 27 | # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 28 | # SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 29 | # 30 | # CHANGES: 31 | # 32 | # 2012-01-31, Lars Bilke 33 | # - Enable Code Coverage 34 | # 35 | # 2013-09-17, Joakim Söderberg 36 | # - Added support for Clang. 37 | # - Some additional usage instructions. 38 | # 39 | # 2016-02-03, Lars Bilke 40 | # - Refactored functions to use named parameters 41 | # 42 | # 2017-06-02, Lars Bilke 43 | # - Merged with modified version from github.com/ufz/ogs 44 | # 45 | # 2018-06-12, Michael Ferguson 46 | # - Forked for ROS support 47 | # 48 | # 2020-10-28, Stefan Fabian 49 | # - Added python3-coverage support 50 | # 51 | 52 | include(CMakeParseArguments) 53 | 54 | # Check prereqs 55 | find_program( GCOV_PATH gcov ) 56 | find_program( LCOV_PATH NAMES lcov lcov.bat lcov.exe lcov.perl) 57 | find_program( GENHTML_PATH NAMES genhtml genhtml.perl genhtml.bat ) 58 | find_program( GCOVR_PATH gcovr PATHS ${CMAKE_SOURCE_DIR}/scripts/test) 59 | find_program( SIMPLE_PYTHON_EXECUTABLE python ) 60 | find_program( PYTHON_COVERAGE_PATH python-coverage ) 61 | 62 | if (NOT PYTHON_COVERAGE_PATH) 63 | find_program( PYTHON_COVERAGE_PATH python3-coverage ) 64 | endif() 65 | 66 | if(NOT GCOV_PATH) 67 | message(FATAL_ERROR "gcov not found! Aborting...") 68 | endif() # NOT GCOV_PATH 69 | 70 | if (NOT PYTHON_COVERAGE_PATH) 71 | message(FATAL_ERROR "Neither python3-coverage nor python-coverage not found! Aborting...") 72 | endif() 73 | 74 | if("${CMAKE_CXX_COMPILER_ID}" MATCHES "(Apple)?[Cc]lang") 75 | if("${CMAKE_CXX_COMPILER_VERSION}" VERSION_LESS 3) 76 | message(FATAL_ERROR "Clang version must be 3.0.0 or greater! Aborting...") 77 | endif() 78 | elseif(NOT CMAKE_COMPILER_IS_GNUCXX) 79 | message(FATAL_ERROR "Compiler is not GNU gcc! Aborting...") 80 | endif() 81 | 82 | set(COVERAGE_COMPILER_FLAGS "-g -O0 --coverage -fprofile-arcs -ftest-coverage" 83 | CACHE INTERNAL "") 84 | 85 | set(CMAKE_CXX_FLAGS_COVERAGE 86 | ${COVERAGE_COMPILER_FLAGS} 87 | CACHE STRING "Flags used by the C++ compiler during coverage builds." 88 | FORCE ) 89 | set(CMAKE_C_FLAGS_COVERAGE 90 | ${COVERAGE_COMPILER_FLAGS} 91 | CACHE STRING "Flags used by the C compiler during coverage builds." 92 | FORCE ) 93 | set(CMAKE_EXE_LINKER_FLAGS_COVERAGE 94 | "" 95 | CACHE STRING "Flags used for linking binaries during coverage builds." 96 | FORCE ) 97 | set(CMAKE_SHARED_LINKER_FLAGS_COVERAGE 98 | "" 99 | CACHE STRING "Flags used by the shared libraries linker during coverage builds." 100 | FORCE ) 101 | mark_as_advanced( 102 | CMAKE_CXX_FLAGS_COVERAGE 103 | CMAKE_C_FLAGS_COVERAGE 104 | CMAKE_EXE_LINKER_FLAGS_COVERAGE 105 | CMAKE_SHARED_LINKER_FLAGS_COVERAGE ) 106 | 107 | if(NOT CMAKE_BUILD_TYPE STREQUAL "Debug") 108 | message(WARNING "Code coverage results with an optimised (non-Debug) build may be misleading") 109 | endif() # NOT CMAKE_BUILD_TYPE STREQUAL "Debug" 110 | 111 | if(CMAKE_C_COMPILER_ID STREQUAL "GNU") 112 | link_libraries(gcov) 113 | else() 114 | set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} --coverage") 115 | endif() 116 | 117 | # Defines a target for running and collection code coverage information 118 | # Builds dependencies, runs all the ROS tests and outputs reports. 119 | # NOTE! The executable should always have a ZERO as exit code otherwise 120 | # the coverage generation will not complete. 121 | # 122 | # ADD_CODE_COVERAGE( 123 | # NAME testrunner_coverage # New target name 124 | # DEPENDENCIES testrunner # Dependencies to build first 125 | # ) 126 | function(ADD_CODE_COVERAGE) 127 | 128 | set(options NONE) 129 | set(oneValueArgs NAME) 130 | set(multiValueArgs DEPENDENCIES) 131 | cmake_parse_arguments(Coverage "${options}" "${oneValueArgs}" "${multiValueArgs}" ${ARGN}) 132 | 133 | if(NOT LCOV_PATH) 134 | message(FATAL_ERROR "lcov not found! Aborting...") 135 | endif() # NOT LCOV_PATH 136 | 137 | if(NOT GENHTML_PATH) 138 | message(FATAL_ERROR "genhtml not found! Aborting...") 139 | endif() # NOT GENHTML_PATH 140 | 141 | # Determine directory to store python coverage files 142 | set(COVERAGE_DIR $ENV{HOME}/.ros) 143 | if(DEFINED ENV{ROS_HOME}) 144 | set(COVERAGE_DIR $ENV{ROS_HOME}) 145 | endif() 146 | 147 | # Cleanup C++ counters 148 | add_custom_target(${Coverage_NAME}_cleanup_cpp 149 | # Cleanup lcov 150 | COMMAND ${LCOV_PATH} --directory . --zerocounters 151 | WORKING_DIRECTORY ${PROJECT_BINARY_DIR} 152 | DEPENDS ${Coverage_DEPENDENCIES} 153 | COMMENT "Resetting CPP code coverage counters to zero." 154 | ) 155 | 156 | # Cleanup python counters 157 | add_custom_target(${Coverage_NAME}_cleanup_py 158 | COMMAND ${PYTHON_COVERAGE_PATH} erase 159 | WORKING_DIRECTORY ${COVERAGE_DIR} 160 | COMMENT "Resetting PYTHON code coverage counters to zero." 161 | ) 162 | 163 | # Cleanup before we run tests (attach to the tests target, which is completed before any test is run) 164 | add_dependencies(tests ${Coverage_NAME}_cleanup_cpp) 165 | add_dependencies(tests ${Coverage_NAME}_cleanup_py) 166 | 167 | # Create C++ coverage report 168 | add_custom_target(${Coverage_NAME}_cpp 169 | COMMAND export PYTHONIOENCODING=UTF-8 170 | # Create baseline to make sure untouched files show up in the report 171 | COMMAND ${LCOV_PATH} -c -i -d . -o ${PROJECT_BINARY_DIR}/${Coverage_NAME}.base 172 | # Capturing lcov counters and generating report 173 | COMMAND ${LCOV_PATH} --directory . --capture --output-file ${PROJECT_BINARY_DIR}/${Coverage_NAME}.info 174 | # add baseline counters 175 | COMMAND ${LCOV_PATH} -a ${PROJECT_BINARY_DIR}/${Coverage_NAME}.base -a ${PROJECT_BINARY_DIR}/${Coverage_NAME}.info --output-file ${PROJECT_BINARY_DIR}/${Coverage_NAME}.total || (exit 0) 176 | COMMAND ${LCOV_PATH} --remove ${PROJECT_BINARY_DIR}/${Coverage_NAME}.total ${COVERAGE_EXCLUDES} --output-file ${PROJECT_BINARY_DIR}/${Coverage_NAME}.info.removed || (exit 0) 177 | COMMAND ${LCOV_PATH} --extract ${PROJECT_BINARY_DIR}/${Coverage_NAME}.info.removed "'*/${PROJECT_NAME}/*'" --output-file ${PROJECT_BINARY_DIR}/${Coverage_NAME}.info.cleaned || (exit 0) 178 | COMMAND ${GENHTML_PATH} ${GENHTML_EXTRA_FLAGS} -o ${Coverage_NAME} ${PROJECT_BINARY_DIR}/${Coverage_NAME}.info.cleaned || (exit 0) 179 | COMMAND ${CMAKE_COMMAND} -E remove ${PROJECT_BINARY_DIR}/${Coverage_NAME}.base ${PROJECT_BINARY_DIR}/${Coverage_NAME}.total || (exit 0) 180 | WORKING_DIRECTORY ${PROJECT_BINARY_DIR} 181 | DEPENDS _run_tests_${PROJECT_NAME} 182 | ) 183 | 184 | # Ensure that the include component uses the correct path for symlinked source directories 185 | get_filename_component(REAL_SOURCE_DIR ${PROJECT_SOURCE_DIR} REALPATH) 186 | 187 | # Create Python coverage report 188 | add_custom_target(${Coverage_NAME}_py 189 | # Rename .coverage file generated by nosetests to avoid overwriting during combine step 190 | COMMAND if [ -f ${PROJECT_BINARY_DIR}/.coverage ]\; then mv ${PROJECT_BINARY_DIR}/.coverage ${PROJECT_BINARY_DIR}/.coverage.nosetests\; fi 191 | COMMAND cp ${PROJECT_BINARY_DIR}/.coverage* ${COVERAGE_DIR}/ || echo "WARNING: No python coverage!" 192 | COMMAND ${PYTHON_COVERAGE_PATH} combine || echo "WARNING: No python coverage to combine!" 193 | COMMAND ${PYTHON_COVERAGE_PATH} report --include "*${REAL_SOURCE_DIR}*" --omit ${COVERAGE_EXCLUDES} || echo "WARNING: no python report to output" 194 | COMMAND ${PYTHON_COVERAGE_PATH} xml --include "*${REAL_SOURCE_DIR}*" --omit ${COVERAGE_EXCLUDES} || echo "WARNING: No python xml to output" 195 | COMMAND ${PYTHON_COVERAGE_PATH} html --include "*${REAL_SOURCE_DIR}*" --omit ${COVERAGE_EXCLUDES} || echo "WARNING: No python html to output" 196 | WORKING_DIRECTORY ${COVERAGE_DIR} 197 | DEPENDS _run_tests_${PROJECT_NAME} 198 | ) 199 | 200 | add_custom_target(${Coverage_NAME} 201 | DEPENDS ${Coverage_NAME}_cpp 202 | DEPENDS ${Coverage_NAME}_py 203 | COMMENT "Processing code coverage counters and generating report." 204 | ) 205 | 206 | # Show where to find the lcov info report 207 | add_custom_command(TARGET ${Coverage_NAME} POST_BUILD 208 | COMMAND ; 209 | COMMENT "Lcov code coverage info report saved in ${PROJECT_BINARY_DIR}/${Coverage_NAME}.info." 210 | ) 211 | 212 | # Show info where to find the C++ report 213 | add_custom_command(TARGET ${Coverage_NAME} POST_BUILD 214 | COMMAND ; 215 | COMMENT "Open ${PROJECT_BINARY_DIR}/${Coverage_NAME}/index.html in your browser to view the coverage report." 216 | ) 217 | 218 | # Show info where to find the Python report 219 | add_custom_command(TARGET ${Coverage_NAME} POST_BUILD 220 | COMMAND ; 221 | COMMENT "Python code coverage info saved in ${COVERAGE_DIR} directory." 222 | COMMENT "Python code coverage html-format: ${COVERAGE_DIR}/htmlcov/index.html." 223 | ) 224 | 225 | endfunction() # SETUP_TARGET_FOR_COVERAGE 226 | 227 | function(APPEND_COVERAGE_COMPILER_FLAGS) 228 | # Set flags for all C++ builds 229 | set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${COVERAGE_COMPILER_FLAGS}" PARENT_SCOPE) 230 | set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${COVERAGE_COMPILER_FLAGS}" PARENT_SCOPE) 231 | # Turn on coverage in python nosetests (see README for requirements on rostests) 232 | set(ENV{CATKIN_TEST_COVERAGE} "1") 233 | message(STATUS "Appending code coverage compiler flags: ${COVERAGE_COMPILER_FLAGS}") 234 | endfunction() # APPEND_COVERAGE_COMPILER_FLAGS 235 | 236 | option(ENABLE_COVERAGE_TESTING "Turn on coverage testing" OFF) 237 | -------------------------------------------------------------------------------- /cmake/code_coverage-extras.cmake.develspace.in: -------------------------------------------------------------------------------- 1 | # Append cmake modules from source directory to the cmake module path 2 | list(APPEND CMAKE_MODULE_PATH "@CMAKE_CURRENT_SOURCE_DIR@/cmake/Modules") 3 | include(CodeCoverage) 4 | -------------------------------------------------------------------------------- /cmake/code_coverage-extras.cmake.installspace.in: -------------------------------------------------------------------------------- 1 | # Append the installed cmake modules to the cmake module path 2 | list(APPEND CMAKE_MODULE_PATH "${code_coverage_DIR}/../../../@CATKIN_PACKAGE_SHARE_DESTINATION@/cmake/Modules") 3 | include(CodeCoverage) 4 | -------------------------------------------------------------------------------- /package.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | code_coverage 4 | 0.4.4 5 | CMake configuration to run coverage 6 | Michael Ferguson 7 | Michael Ferguson 8 | 9 | BSD 10 | 11 | https://github.com/mikeferguson/code_coverage 12 | https://github.com/mikeferguson/code_coverage/issues 13 | 14 | catkin 15 | 16 | lcov 17 | python-coverage 18 | python3-coverage 19 | 20 | --------------------------------------------------------------------------------