├── CMakeLists.txt ├── LICENSE ├── README.md ├── jetson_csi_cam.launch └── package.xml /CMakeLists.txt: -------------------------------------------------------------------------------- 1 | cmake_minimum_required(VERSION 2.8.3) 2 | project(jetson_csi_cam) 3 | 4 | ## Compile as C++11, supported in ROS Kinetic and newer 5 | # add_compile_options(-std=c++11) 6 | 7 | ## Find catkin macros and libraries 8 | ## if COMPONENTS list like find_package(catkin REQUIRED COMPONENTS xyz) 9 | ## is used, also find other catkin packages 10 | find_package(catkin REQUIRED COMPONENTS 11 | gscam 12 | ) 13 | 14 | ## System dependencies are found with CMake's conventions 15 | # find_package(Boost REQUIRED COMPONENTS system) 16 | 17 | 18 | ## Uncomment this if the package has a setup.py. This macro ensures 19 | ## modules and global scripts declared therein get installed 20 | ## See http://ros.org/doc/api/catkin/html/user_guide/setup_dot_py.html 21 | # catkin_python_setup() 22 | 23 | ################################################ 24 | ## Declare ROS messages, services and actions ## 25 | ################################################ 26 | 27 | ## To declare and build messages, services or actions from within this 28 | ## package, follow these steps: 29 | ## * Let MSG_DEP_SET be the set of packages whose message types you use in 30 | ## your messages/services/actions (e.g. std_msgs, actionlib_msgs, ...). 31 | ## * In the file package.xml: 32 | ## * add a build_depend tag for "message_generation" 33 | ## * add a build_depend and a run_depend tag for each package in MSG_DEP_SET 34 | ## * If MSG_DEP_SET isn't empty the following dependency has been pulled in 35 | ## but can be declared for certainty nonetheless: 36 | ## * add a run_depend tag for "message_runtime" 37 | ## * In this file (CMakeLists.txt): 38 | ## * add "message_generation" and every package in MSG_DEP_SET to 39 | ## find_package(catkin REQUIRED COMPONENTS ...) 40 | ## * add "message_runtime" and every package in MSG_DEP_SET to 41 | ## catkin_package(CATKIN_DEPENDS ...) 42 | ## * uncomment the add_*_files sections below as needed 43 | ## and list every .msg/.srv/.action file to be processed 44 | ## * uncomment the generate_messages entry below 45 | ## * add every package in MSG_DEP_SET to generate_messages(DEPENDENCIES ...) 46 | 47 | ## Generate messages in the 'msg' folder 48 | # add_message_files( 49 | # FILES 50 | # Message1.msg 51 | # Message2.msg 52 | # ) 53 | 54 | ## Generate services in the 'srv' folder 55 | # add_service_files( 56 | # FILES 57 | # Service1.srv 58 | # Service2.srv 59 | # ) 60 | 61 | ## Generate actions in the 'action' folder 62 | # add_action_files( 63 | # FILES 64 | # Action1.action 65 | # Action2.action 66 | # ) 67 | 68 | ## Generate added messages and services with any dependencies listed here 69 | # generate_messages( 70 | # DEPENDENCIES 71 | # std_msgs # Or other packages containing msgs 72 | # ) 73 | 74 | ################################################ 75 | ## Declare ROS dynamic reconfigure parameters ## 76 | ################################################ 77 | 78 | ## To declare and build dynamic reconfigure parameters within this 79 | ## package, follow these steps: 80 | ## * In the file package.xml: 81 | ## * add a build_depend and a run_depend tag for "dynamic_reconfigure" 82 | ## * In this file (CMakeLists.txt): 83 | ## * add "dynamic_reconfigure" to 84 | ## find_package(catkin REQUIRED COMPONENTS ...) 85 | ## * uncomment the "generate_dynamic_reconfigure_options" section below 86 | ## and list every .cfg file to be processed 87 | 88 | ## Generate dynamic reconfigure parameters in the 'cfg' folder 89 | # generate_dynamic_reconfigure_options( 90 | # cfg/DynReconf1.cfg 91 | # cfg/DynReconf2.cfg 92 | # ) 93 | 94 | ################################### 95 | ## catkin specific configuration ## 96 | ################################### 97 | ## The catkin_package macro generates cmake config files for your package 98 | ## Declare things to be passed to dependent projects 99 | ## INCLUDE_DIRS: uncomment this if you package contains header files 100 | ## LIBRARIES: libraries you create in this project that dependent projects also need 101 | ## CATKIN_DEPENDS: catkin_packages dependent projects also need 102 | ## DEPENDS: system dependencies of this project that dependent projects also need 103 | catkin_package( 104 | # INCLUDE_DIRS include 105 | # LIBRARIES jetson_csi_cam 106 | # CATKIN_DEPENDS gscam 107 | # DEPENDS system_lib 108 | ) 109 | 110 | ########### 111 | ## Build ## 112 | ########### 113 | 114 | ## Specify additional locations of header files 115 | ## Your package locations should be listed before other locations 116 | include_directories( 117 | # include 118 | ${catkin_INCLUDE_DIRS} 119 | ) 120 | 121 | ## Declare a C++ library 122 | # add_library(${PROJECT_NAME} 123 | # src/${PROJECT_NAME}/jetson_csi_cam.cpp 124 | # ) 125 | 126 | ## Add cmake target dependencies of the library 127 | ## as an example, code may need to be generated before libraries 128 | ## either from message generation or dynamic reconfigure 129 | # add_dependencies(${PROJECT_NAME} ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS}) 130 | 131 | ## Declare a C++ executable 132 | ## With catkin_make all packages are built within a single CMake context 133 | ## The recommended prefix ensures that target names across packages don't collide 134 | # add_executable(${PROJECT_NAME}_node src/jetson_csi_cam_node.cpp) 135 | 136 | ## Rename C++ executable without prefix 137 | ## The above recommended prefix causes long target names, the following renames the 138 | ## target back to the shorter version for ease of user use 139 | ## e.g. "rosrun someones_pkg node" instead of "rosrun someones_pkg someones_pkg_node" 140 | # set_target_properties(${PROJECT_NAME}_node PROPERTIES OUTPUT_NAME node PREFIX "") 141 | 142 | ## Add cmake target dependencies of the executable 143 | ## same as for the library above 144 | # add_dependencies(${PROJECT_NAME}_node ${${PROJECT_NAME}_EXPORTED_TARGETS} ${catkin_EXPORTED_TARGETS}) 145 | 146 | ## Specify libraries to link a library or executable target against 147 | # target_link_libraries(${PROJECT_NAME}_node 148 | # ${catkin_LIBRARIES} 149 | # ) 150 | 151 | ############# 152 | ## Install ## 153 | ############# 154 | 155 | # all install targets should use catkin DESTINATION variables 156 | # See http://ros.org/doc/api/catkin/html/adv_user_guide/variables.html 157 | 158 | ## Mark executable scripts (Python etc.) for installation 159 | ## in contrast to setup.py, you can choose the destination 160 | # install(PROGRAMS 161 | # scripts/my_python_script 162 | # DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION} 163 | # ) 164 | 165 | ## Mark executables and/or libraries for installation 166 | # install(TARGETS ${PROJECT_NAME} ${PROJECT_NAME}_node 167 | # ARCHIVE DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION} 168 | # LIBRARY DESTINATION ${CATKIN_PACKAGE_LIB_DESTINATION} 169 | # RUNTIME DESTINATION ${CATKIN_PACKAGE_BIN_DESTINATION} 170 | # ) 171 | 172 | ## Mark cpp header files for installation 173 | # install(DIRECTORY include/${PROJECT_NAME}/ 174 | # DESTINATION ${CATKIN_PACKAGE_INCLUDE_DESTINATION} 175 | # FILES_MATCHING PATTERN "*.h" 176 | # PATTERN ".svn" EXCLUDE 177 | # ) 178 | 179 | ## Mark other files for installation (e.g. launch and bag files, etc.) 180 | # install(FILES 181 | # # myfile1 182 | # # myfile2 183 | # DESTINATION ${CATKIN_PACKAGE_SHARE_DESTINATION} 184 | # ) 185 | 186 | ############# 187 | ## Testing ## 188 | ############# 189 | 190 | ## Add gtest based cpp test target and link libraries 191 | # catkin_add_gtest(${PROJECT_NAME}-test test/test_jetson_csi_cam.cpp) 192 | # if(TARGET ${PROJECT_NAME}-test) 193 | # target_link_libraries(${PROJECT_NAME}-test ${PROJECT_NAME}) 194 | # endif() 195 | 196 | ## Add folders to be run by python nosetests 197 | # catkin_add_nosetests(test) 198 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Peter Moran 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Nvidia Jetson CSI camera launcher for ROS 2 | 3 |

4 | 5 | This ROS package makes it simple to use CSI cameras on the Nvidia Jetson TK1, TX1, or TX2 with ROS via gstreamer and the Nvidia multimedia API. This is done by properly configuring [`gscam`](http://wiki.ros.org/gscam) to work with the Nvidia hardware. 6 | 7 | **Features** 8 | 9 | * Control resolution and framerate. 10 | * Camera calibration support. 11 | * Works efficiently, allowing for high resolution and/or high fps video by taking advantage of gstreamer and the Nvidia multimedia API. 12 | * Multi camera support. 13 | 14 | --- 15 | 16 | # Installation 17 | 18 | In order to get started, you will need to download the `jetson_csi_cam` repository as well as its dependencies. Just follow the steps below and you should be good to go. 19 | 20 | If you'd like to learn more about `gscam`, check out their [ROS wiki page](http://wiki.ros.org/gscam) or their [Github repository](https://github.com/ros-drivers/gscam). 21 | 22 | **Note:** This package was tested on a Nvidia Jetson TX2 with L4T R27.1, ROS Kinetic, and the [Leopard Imaging IMX377CS](https://www.leopardimaging.com/LI-JETSON-KIT-IMX377CS-X.html) CSI camera. 23 | 24 | ## Dependencies 25 | 26 | For the purpose of this guide, we will assume you already have: 27 | 28 | * Gstreamer-1.0 and the Nvidia multimedia API (typically installed by Jetpack) 29 | * ROS Kinetic 30 | * Older versions of ROS may work, provided that a version of `gscam` that supports gstreamer-1.0 is available for that ROS version, but this is untested. 31 | * `gscam` with gstreamer-1.0 support. 32 | * The following steps will show how to build `gscam` from source to support this, so don't worry about it yet. 33 | 34 | With these dependencies accounted for, lets get everything installed. 35 | 36 | ## 1. Download `jetson_csi_cam` 37 | 38 | Clone this repository into you `catkin_workspace`. 39 | 40 | ``` 41 | cd ~/catkin_workspace/src 42 | git clone https://github.com/peter-moran/jetson_csi_cam.git 43 | ``` 44 | 45 | ## 2. Install `gscam` with gstreamer-1.0 support 46 | 47 | Clone `gscam` into your `catkin_workspace`. 48 | 49 | ``` 50 | cd ~/catkin_workspace/src 51 | git clone https://github.com/ros-drivers/gscam.git 52 | ``` 53 | 54 | Then edit `./gscam/Makefile` and add the CMake flag `-DGSTREAMER_VERSION_1_x=On` to the first line of the file, so that it reads: 55 | 56 | EXTRA_CMAKE_FLAGS = -DUSE_ROSBUILD:BOOL=1 -DGSTREAMER_VERSION_1_x=On 57 | 58 | While this flag is only necessary if you have both `gstreamer-0.1` and `gstreamer-1.0` installed simultaneously, it is good practice to include. 59 | 60 | ## 3. Build everything 61 | 62 | Now we build and register `gscam` and `jetson_csi_cam` in ROS. 63 | 64 | ``` 65 | cd ~/catkin_workspace 66 | catkin_make 67 | source ~/.bashrc 68 | ``` 69 | 70 | At this point everything should be ready to go. 71 | 72 | --- 73 | 74 | # Usage 75 | 76 | > **TL;DR:** To publish the camera stream to the ROS topic `/csi_cam_0/image_raw`, use this command in the terminal: 77 | > 78 | > ``` 79 | > roslaunch jetson_csi_cam jetson_csi_cam.launch width:= height:= fps:= 80 | > ``` 81 | > If you have another camera on your Jetson TX2, to publish the other camera stream to the ROS topic `/csi_cam_1/image_raw`, use this command in the terminal: 82 | > 83 | > ``` 84 | > roslaunch jetson_csi_cam jetson_csi_cam.launch sensor_id:=1 width:= height:= fps:= 85 | > ``` 86 | > If you would like to learn more of the details, read ahead. 87 | 88 | ## Capturing Video 89 | 90 | ### Turning on the video stream 91 | 92 | To publish your camera's video to ROS (using the default settings) execute the following: 93 | 94 | ``` 95 | roslaunch jetson_csi_cam jetson_csi_cam.launch 96 | ``` 97 | 98 | > **Wait, where is the video?** This launch file only *publishes* the video to ROS, making it available for other programs to use. This is because we don't want to view the video every time we use the camera (eg the computer may be processing it first). Thus we use separate programs to view it. I'll discuss this in a later section, but if you can't wait, run `rqt_image_view` in a new terminal to see the video. 99 | 100 | You can confirm the video is running by entering `rostopic list` in the terminal. You should be able to see the `/csi_cam/image_raw` topic (aka your video) along with a bunch of other topics with similar names -- unless you changed the `camera_name` argument from the default. 101 | 102 | ### Setting video options 103 | 104 | Most of the time we'll want to use settings other than the defaults. We can easily change these by passing command line arguments to `roslaunch`. For example, if I want the camera to run at 4k resolution at 15 fps, I would use the following: 105 | 106 | ``` 107 | roslaunch jetson_csi_cam jetson_csi_cam.launch width:=3840 height:=2160 fps:=15 108 | ``` 109 | 110 | In other words, to set any of the arguments use the `:=` options for `roslaunch`. 111 | 112 | #### Accepted arguments for `jetson_csi_cam.launch` 113 | 114 | * `sensor_id` -- The sensor id of each camera 115 | * `width` -- Image Width 116 | * `height` -- Image Height 117 | * `fps` -- Desired framerate. True framerate may not reach this if set too high. 118 | * `cam_name` -- The name of the camera (corrsponding to the camera info). 119 | * `frame_id` -- The TF frame ID for the camera. 120 | * `sync_sink` -- Whether to synchronize the app sink. Setting this to false may resolve problems with sub-par framerates. 121 | 122 | ## Testing your video stream 123 | 124 | ### View the video 125 | 126 | To view the video, simply run `rqt_image_view` in a new terminal. A window will pop up and the video should be inside. You may need to go to the pulldown in the top left to choose your camera's video topic, by default you want to use `/csi_cam/image_raw`. 127 | 128 | ### Calculate true framerate 129 | 130 | To check the true framerate of your video you can use the `rostopic hz` tool, which shows the frequency any topic is published at. 131 | 132 | ``` 133 | rostopic hz /csi_cam/image_raw 134 | ``` 135 | 136 | When the true framerate is below the framerate you asked for, it is either because the system cannot keep up or the camera does not have that setting. If you are using the Nvidia Jetson TX2, you can get higher resolution video at higher true FPS by switching to a [higher power mode](http://www.jetsonhacks.com/2017/03/25/nvpmodel-nvidia-jetson-tx2-development-kit/). It can also be good to dial back your requested FPS to be close or slightly above to your true FPS. 137 | 138 | ## Camera Calibration 139 | 140 | The `jetson_csi_cam` package is set up to make camera calibration very simple. To calibrate your camera, all you need to do is follow the [monocular camera calibration guide](http://wiki.ros.org/camera_calibration/Tutorials/MonocularCalibration) on the ROS wiki, with the following notes: 141 | 142 | * As the guide states, you'll need a printout of a chessboard. If you want something quick, use this [chessboard for 8.5"x11" paper](http://www.vision.caltech.edu/bouguetj/calib_doc/htmls/pattern.pdf) with an 8x6 grid. Please note: while there are nine by seven *squares*, we use a size of 8x6 because we are counting the *internal vertices*. You will need to measure the square size yourself since printing will slightly distort the chessboard size, but the squares should be 30mm on each side. 143 | 144 | * In Step 2, make sure to start the camera via `roslaunch` as discussed above. 145 | 146 | * In Step 3, make sure to set your `image` and `camera` arguments correctly as below, also *make sure that the chessboard size and square size are set correctly* for your chessboard. 147 | 148 | ``` 149 | rosrun camera_calibration cameracalibrator.py --size 8x6 --square image:=/csi_cam/image_raw camera:=/csi_cam 150 | ``` 151 | 152 | After following this guide and clicking **COMMIT** (as it tells you to do), your calibration data should be automatically published with your video under the topic `/csi_cam/camera_info`. Most other ROS packages needing this information will automatically recognize it. 153 | -------------------------------------------------------------------------------- /jetson_csi_cam.launch: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | -------------------------------------------------------------------------------- /package.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | jetson_csi_cam 4 | 1.0.0 5 | A package for easily connecting to CSI cameras from the Jetson TX1/TX2 via GStreamer 6 | 7 | 8 | Peter Moran 9 | 10 | 11 | 12 | MIT 13 | 14 | 15 | 16 | http://petermoran.org/csi-cameras-on-tx2/ 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | catkin 37 | gscam 38 | gscam 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | --------------------------------------------------------------------------------