JetsonHacks

Developing on NVIDIA® Jetson™ for AI on the Edge

OpenCV, Python, Onboard Camera – NVIDIA Jetson TX2

In this article, we build a simple demonstration of a Canny Edge Detector using OpenCV, Python, and the onboard camera of the NVIDIA Jetson TX2 Development Kit. Looky here:

Background

Back in 1986, John F. Canny developed the Canny Edge detector. The Canny Edge is one of the image processing milestones which is still in use today.
You can read some more about the Canny Edge Detector and the technical details here: OpenCV.org Canny Edge Detector and here: Wikipedia – Canny edge detector

In this article we will use a simple Python script which will use the OpenCV library implementation of the Canny Edge Detector to read the onboard camera and run the frames through the filter. Earlier we went over on how to build the OpenCV library for the Jetson. There is a repository on the JetsonHacks Github account which contains a build script to help in the process. You will need to enable GStreamer support. As of this writing the current script enables GStreamer support (OpenCV 3.3), while earlier versions did not. GStreamer must be enabled to support the onboard camera. Note: The standard OpenCV4Tegra installed by JetPack does not have GStreamer support enabled, so the onboard camera will not work with it.

The Codez

The file cannyDetection.py is available in the Examples folder of the buildOpenCVTX2 repository on the JetsonHacks Github account. The script is also available as a Github Gist. The Gist is seen in its entirety further down below.

GStreamer Camera Pipeline

The first task is to open the onboard camera. Camera access is through a GStreamer pipeline:

cap = cv2.VideoCapture("nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,format=(string)I420, framerate=(fraction)30/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink")

Canny Edge Detector

The main part of the filter processing reads a frame from the camera, converts it to gray scale, runs a gaussian blur on the gray scale image, and then runs the Canny Edge Detector on that result:

ret_val, frame = cap.read();
hsv=cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
blur=cv2.GaussianBlur(hsv,(7,7),1.5)
edges=cv2.Canny(blur,0,edgeThreshold)

Not surprisingly, it is more concise to just write the code. In the code, both the GaussianBlur and Canny functions have parameters to fine tune the results.

At this point, we could simply display it on a window on the screen:

cv2.imshow('Canny Edge Detector', edges)

In the script, we add a little user interface sugar which allows us to optionally display each step. The only interesting part is that the image for each step is composited into one larger frame. This requires that the images be converted to the same color space before compositing. In the video, the Jetson TX2 is set to run at maximum performance (See note below). Here’s the full script:

Conclusion

Many people use OpenCV for everyday vision processing tasks, and the Canny Edge Detector is a valuable tool. The purpose of this article is to show how to access the onboard camera using GStreamer in a Python script with OpenCV. More importantly, I played guitar in the video.

Notes

  • In the video, the Jetson TX2 is running L4T 28.1, OpenCV 3.3 with GStreamer support enabled
  • In the video, the Jetson TX2 is running ‘$ sudo nvpmodel -m 0’
Facebook
Twitter
LinkedIn
Reddit
Email
Print

10 Responses

  1. Hi Jim thanks you for these great tutorials. I would like save a video with fourcc = cv2.VideoWriter_fourcc(*’XVID’)out = cv2.VideoWriter(‘output.avi’,fourcc, 20.0, (640,480)) However i am getting an error when trying play video “Could not demultiplex stream” in loading Video recorded

    Thank you again

      1. Thank you jim. I’m getting the same error with all codecs installed. I’m going crazy :(.

        Could you update this great post with your python code to record the screen or record/save to a file, the camera videostreaming? It would be a great help.

        Thanks a lot again!

  2. Thank you so much for your videos. Like Hyperrjas, I’m also having issues with recording the video. I’m very new to programming in general so I’m fumbling my way through all of this.

    Any chance you could throw in an update to show how you would record the video.

    1. hey I was able to figured it out. I made a chance in the fourcc call and used ‘M’,’J’,’P’,’G’ for my supplied argument. this canceled any errors I was getting and allowed me to write to file.

      Hyperrjas try this if your still having issues.

      fHeight = int(cap.get(3))
      fWidth = int(cap.get(4))
      fourcc = cv2.VideoWriter_fourcc(‘M’,’J’,’P’,’G’)
      out = cv2.VideoWriter(‘output.avi’, fourcc, 30, (fWidth, fHeight)
      out.write(frame) #inside while loop

  3. I’m still having the same error.

    Could you paste/share your code like gist recipe? I would like take a look to know where is my code problem.

    I would like to record canny edge option.

    Thank you so much again :).

    Regards

  4. Hello,
    thank you for the code, it’s helpful as I wanted to access the camera on board. I having one issue regarding memory, after sometime the kernel kills the application as memory consumption is too much. Is there any way by which i can use the camera continuously without killing the application.

    thanks

    1. What version of L4T are you using?
      What version of OpenCV are you using?
      Did you modify the Python code?
      What did you try to do to fix the issue?

  5. thank you for t reply 🙂

    L4t Version: R28.2
    OpenCv: 4.0.0-pre
    No I didn’t modify the code, before using the application for my purpose i was just checking the code.

    1. How long does it run before it runs out of memory? Are you running any other programs? If you open the System Monitor and monitor memory usage, does the memory gradually grow every few minutes? I ran the program overnight and did not experience any issues.

Leave a Reply

Your email address will not be published. Required fields are marked *

Disclaimer

Some links here are affiliate links. If you purchase through these links I will receive a small commission at no additional cost to you. As an Amazon Associate, I earn from qualifying purchases.

Books, Ideas & Other Curiosities