In this article, we cover ROS installation of the Intel RealSense Camera on the NVIDIA Jetson TK1. Note: This article was updated 9-9-16. Looky here:
Background
Note: This article is intended for intermediate to advanced users who are familiar with ROS.
One of the intended uses of Intel RealSense Cameras is robotics. The premiere operating system for robots is Robot Operating System (ROS). A great platform for running ROS and RealSense Cameras? Jetson TK1! Let’s get the shotgun out and have wedding!
Using the information in this article, you will:
- Build a new kernel for the Jetson TK1
- Build the RealSense Camera driver library
- Install ROS on the Jetson TK1
- Install the RealSense Camera ROS Node
- Be fulfilled as a human being
Let’s get started. The first step is to read through this article to get familiar with the steps involved. Then start installation.
Installation
In order to get started, there are three prerequisites required. These instructions are based on a freshly flashed Jetson TK1, the one in the video was done using JetPack 2.2, which installed L4T 21.4. You should also have Git installed:
$ sudo apt-get install git
Note: The installation in the video was done on a fresh Linux for Tegra (L4T) JetPack installation with the standard JetsonHacks postFlash done afterwards. This sets up the USB ports appropriately, and eliminates the need to run the setupTK1 script mentioned later.
First, librealsense needs to be installed on the Jetson TK1. This involves building a new kernel to support the video modes of the RealSense camera in the UVC module, and building the librealsense library. Here is an article on how to do that, librealsense installation on Jetson TK1. Make sure that you remember to run the setupTK1 script to assign the USB port to USB 3.0 speeds and turn USB auto-suspend off. At the end of the RealSense camera installation, you should run some of the examples provided to make sure that the camera is installed and works properly.
A good practice is to make a clone of the image after building a new kernel, so that if something goes wrong later (that could never happen, could it?) you don’t have to start again from scratch.
Second, we need to have ROS installed on the Jetson. Once the RealSense camera is working, from a new Terminal install ROS:
$ git clone https://github.com/jetsonhacks/installROS.git
$ cd installROS
$ ./installROS.h
$ cd ..
This will install ROS Indigo ros-base, rosdep, and rosinstall.
Note: In an earlier article article on installing ROS on the Jetson TK1, we used the Grinch kernel. The Grinch kernel provides access to a large number of peripherals on the TK1. Because we modified the stock kernel for the RealSense camera in our first step, the Grinch kernel is not used here. If you need the Grinch kernel, you will have to recompile the Grinch kernel with the RealSense camera changes. That is an excercise left to the reader.
Third, we download the realsense_camera package installer:
Open a new Terminal, which will source the new environment setup by ROS, and:
$ git clone https://github.com/jetsonhacks/installRealSenseCameraROS.git
$ cd installRealSenseCameraROS
We need is a Catkin Workspace for our base of operations. There is a convenience script to create a new Catkin Workspace.
$ ./setupCatkinWorkspace [workspace name]
In the video above, jetsonros is the workspace name. This script creates an initialized Catkin Workspace in the ~/ directory.
With the prerequisites installed, we’re ready to install the realsense_camera package:
$ ./installRealSense.sh [workspace name]
where [workspace name] is the name of the Catkin Workspace where you want the realsense_camera package installed. In the video, the workspace name used is jetsonros.
If you do not have a swap file enabled on your Jetson, there may be issues compiling the package because the TK1 does not have enough memory to compile this in one pass. The installation script has been changed since the video was filmed to compile using only one core to relieve memory pressure, i.e.
$ catkin_make -j1″
If this doesn’t fix the problem, refer to the video for a workaround.
Note: As of this writing, the ROS package in the debian repository cv_bridge is hard linked against an OpenCV package which is not installed on the Jetson (2.4.8). There are several ways to get around this, discussed on the ROS Answers forum. For this installation, installing cv_bridge from source is chosen.
At this point, you are ready to launch the node.
Launch RealSense Camera Node
There are several launch files included in the realsense_camera package. These are covered in the README.md file in real_sense camera directory. In order to launch the camera on the Jetson:
$ cd [catkin workspace] , e.g. cd ~/jetsonros
$ source devel/setup.bash
$ roslaunch realsense_camera realsense_r200_nodelet_standalone_preset.launch
Visualzation Workstation
On your visualization workstation, you can view the camera configuration:
$ rosrun rqt_reconfigure rqt_reconfigure
If you intend to view a point cloud, you must setup a frame of reference, i.e.
$ rosrun tf static_transform_publisher 0.0 0.0 0.0 0.0 0.0 0.0 map camera_depth_optical_frame 100
You can also open RVIZ and load the provided RVIZ configuration file: realsenseRvizConfiguration1.rviz.
$ roscd realsense_camera
$ rosrun rviz rviz -d rviz/realsenseRvizConfiguration1.rviz
Please read the README.md file for more information.
Note: You can install RViz on the Jetson TK1 using these instructions.
Conclusion
RealSense camera support under ROS is still relatively new. Since this article was initially published, there have been many changes to the ROS node package. The above scripts clamp the software to the publish date, but if you wish to use this package it will be worth investing some time in updating to later releases. Things are shaping up quite nicely for this new entry in the RGBD camera space.
23 Responses
great work. Might want to include a
source devel/setup.bash
in the catkinsetup script
I’ve been trying to follow these instructions and have repeatedly failed to get ROS working with the R200 Realsense camera. I am flashing my TK1 with JetPack 2.2, then installing librealsense following your installLibrealsense repository. At this point the camera works properly with the Intel executables in librealsense/bin. If I try to follow this by using installROS and installRealsenseCameraROS repository scripts, then when I launch the Realsense camera node, the TK1 locks up and crashes. Alternatively, if I follow the installation of librealsense by following your postFlash and then installGrinch repositories, the Realsense camera no longer works properly with the Intel executables (it loses depth sensing, acting as a simple color camera). Can you clarify what order to apply the repositories you mentioned in this post: postFlash, installGrinch, installLibrealsense, installROS, installRealsenseCameraROS in order to get the the ROS camera node working?
Hi Chris,
A point of clarification. The install librealsense scripts build a new kernel for the Jetson. If you install the Grinch kernel, then naturally the version that you created for running librealsense is replaced by the Grinch kernel, and the the RealSense camera probably will not work as expected. If you need the R200 to work with the Grinch kernel, you will have to change the Grinch kernel to reflect the changes needed.
The postFlash script sets the TK1 to use USB 3.0, which is needed by the R200. You could also do this using the setupTK1.sh in the installLibrealsense repository, it does much the same thing. Then installLibrealsense. The camera should be working at that point.
Next installROS, as shown in the video, and then installRealsenseCameraROS.
My guess is that if you are having issues (without the Grinch kernel installed), it has to do with the installRealsenseCameraROS. Unfortunately when I wrote this article they had not finalized a version for use; it has probably changed over time. You’ll have to look through the scripts and try to figure out what the issues are. If you can determine where the program actually crashes, that would be a pretty good clue as to what is wrong. You can try making sure that roscore is running before launching your node to maybe get further along in the process.
I can confirm that roscore is running prior to launching the realsense camera node. When I launch the camera node (I’ve tried multiple launch files), the last line posted in the terminal states “camera started”. If a more complete report of what the camera node echos would be of interest to you, I can provide it. I’ll try to sort out what might be amiss with RealsenseCameraROS, but I am not a professional developer. My goal at this time is only to get the camera node working. I don’t need the Grinch kernel at this time. I was only using it as a guess that it might help with ROS compatibility with the camera, but it sounds like the this is not the case.
Perhaps this discussion thread has some clues to the underlying issue. It also mentions your website!
https://github.com/IntelRealSense/librealsense/issues/73
That issue is for the Jetson TX1. This article is about the Jetson TK1.
I spent quite some time working with this. I replicated the issue that you have, but the obvious fixes that I tried did not solve the problem. I basically rolled back all the software in the repository to what it should have been when I filmed this episode.
Unfortunately that means there will be no quick fix, I’ll have to spend some time figuring out what is wrong. My guess at this point is that there was a change to one of the base ROS packages that causes some sort of conflict, but it’s going to take me awhile to sort it out. That the machine hard freezes makes it a little more challenging.
Jim, thanks for your continued help. I’m glad to know you have replicated the problem. I tried building several different realsense_camera package releases for ROS (some releases around the time you published your video, as well as some of the most recent releases). All releases I tried exhibit the same problem.
Hi Chris,
I believe I’ve found the issue. Could you please go through the instructions in the article again and see if it works for you? Thank you for your patience.
Jim, I can confirm that your fix worked for me!!! Thanks so much for your effort to fix this. I doubt I could have figured it out myself… Was the essential fix to install intel-ros/realsense object 52e2c87b72c537f3e998dacc7c20a753afbeb9f6? Were there any other important fixes?
The checkout rollback just did some cosmetic changes, the newer versions kinda sorta work but by the time I had found the real issue and flashed the Jetson a dozen times I’d lost patience with it.
The real issue, which was causing the whole system to crash when the camera was asked to start streaming, was in the installROS scripts:
I separated that out in case people want to use it still. The postFlash script does:
So it must something ugly in the restricted repository that mucks things up.
That makes me believe that it’s at a low level in the system. I could not figure out how to easily debug it with ROS running (and it’s in the particularly nasty bit to begin with), so I’ll take this as a minor victory for now.
It’s the nature of ROS, and distributed systems in general, I’m afraid. There’s so many moving parts that it’s difficult to pinpoint problems when they occur and chase them down.
We’ve installed the Grinch kernel, and are trying to run r200_nodelet_default.launch. But the last message we get is “/camera – Starting camera” … then it freezes. Any idea why this might be?
Did you recompile the Grinch kernel with the necessary UVCVideo patches?
Great work as usual. Thanks Jim!
You are welcome, and thanks for reading!
first of all thank you for sharing all this stuff 🙂
I have a Jetson TK1, R200 and asus xtion pro.
I would like to run ROS mapping and localization but i’m not sure which path to follow.
1) R200 or Xtion ?
2) Does tk1 handles the payload ?
Can you share your thoughts ?
I don’t really have any recommendations. If you have the ASUS working on ROS and are only operating indoors, it’s probably the easiest to get working. The TK1 should be able to handle the payload. Thanks for reading!
Hello, sorry for poor English. Im trying to make the camera work but it dont. Could you try on your machine?
Many thanks, love your work
Iwato
Hi Iwato,
Did you make a new kernel with the RealSense changes for the TK1? Which version of L4T is running on your TK1?
Dear Jim,
THank you for your quick response. I rebuilt the Kernel like in the librealsense video, and it worked ok. Also, the L4T version is 21.4 on the TK1 Jetson. It seems there is an error of missing files on the side of openCV, where it has a error when building that.
Many thanks
Iwato
Dear Jim,
Have you tried the script yet? I am terribly sorry for bothering you. I have attempted several package fixes but none of the ones have worked. Currently I am avoiding openCV, and resorted to installing just the realsense package, yet I am having a problem with that as well. Nevertheless, I am working fervently. I noticed it helps to upgrade gcc to version above 5.3, as in the issues sections it was recommended.
This camera’s ROS node is the last requierment for my project. I hope we could work on it together for a fix!!!
Iwato
Hi Iwato,
Unfortunately I do not have any more information to share about this subject. I hope you can find the source of your problem.
Dear Jim,
It seems as though as every day passes there is less support for the TK1. It’s a shame but it is the truth. I enjoyed working with the Jetson while I had it, but its time has arrived and I think I must move forward. I placed an order for an UP board, wonder how that is going to go!
Thanks, excellent content!
Iwato