JetsonHacks

Developing on NVIDIA® Jetson™ for AI on the Edge

NVIDIA Jetson AGX Orin Developer Kit

The NVIDIA Jetson AGX Orin Developer Kit is now available! The new Jetson flagship has up to 8X the raw AI compute performance and up to 2X the energy efficiency of Jetson AGX Xavier. Looky here:

Introduction

The Jetson AGX Orin is the next evolution of the the Jetson product line. Here, we’ll cover some of the highlights, just to wet our appetite for upcoming development.

We’ll cover mostly the AGX Orin hardware in this article. The software side is another major upgrade which we will soon be covering. The software upgrades will run on both the Orin series and the Xavier series of Jetsons. Some highlights for the system software:

  • Latest compute stack with latest versions of CUDA 11 and TensorRT 8
  • Linux Kernel 5.10
  • Ubuntu 20.04 based root file system
  • UEFI for CPU bootloader

There is also a group of frameworks which leverage these new under pinnings we will talk about in upcoming articles.

Specs

Before we get much further, the price at introduction is $1999 USD.

Jetson AGX Orin Module

You can think of the Jetson AGX Orin Developer Kit as two parts. The first is the Jetson AGX Orin Module, which is a compute module. The second part is a carrier board which provides input/output, memory and electrical connections to the module. Here’s the NVIDIA provided specs:

For the Jetson AGX Orin Module:

The AGX Orin Module consists of three main parts. The first is the Jetson Orin System On a Chip (SoC). The SoC is the brains of the Jetson and contains all the compute elements. The SoC attaches to a PCB which hold the main memory and eMMC, along with support circuitry and a 699 pin interface connector. This assembly is in a metal case which also thermally connects to the SoC:

Image attribution (????: NVIDIA)

The module is 100mm x 100m x 16mm. On the Developer Kit, attached to the Jetson AGX Orin module is a heat sink with integrated fan. The heat sink/fan is referred to as a thermal solution. For many developers, they will replace this particular thermal solution to meet requirements in their own designs.

Jetson AGX Orin Developer Kit Carrier Board

The second part of the Developer Kit is the carrier board. NVIDIA makes the design for the Jetson AGX Developer Kit Carrier Board freely available. Here are the specs:

The carrier board provides access to the input and output capabilities of the kit, along with supplying the module with power. People connect their peripherals to connectors on the sides of the Developer Kit, which connect to the carrier board. Here are the four different different sides, which hold the headers and jacks mentioned above:

One side holds a secret! From the factory, the Jetson AGX Orin is beautiful. I modified mine to fix that by adding googly eyes. This marks a cover held on to the Developer Kit with magnets. Removing the cover exposes a PCIe connector. The cover also holds WiFi antennas. Be careful when removing the cover as the antennas are connected to a WiFi card on the carrier board:

Let’s take a look at what’s inside after removing the case cover. There are 4 screws that hold the Developer Kit together. The screws are easily accessible on the underside of the Developer Kit. Removing the screws, we can take off the case cover to expose the Jetson AGX Orin module. You can see the stack top to bottom, with the carrier board (on a plastic stand), Jetson AGX Orin module, and heat sink with fan:

The Jetson AGX Module connects to the carrier board with a 699 pin connector. By gently rocking the Jetson Module, we can detach the module from the carrier board. Be careful, as the module is attached to the carrier board with a small wire. The wire controls the fan on the heat sink:

Benchmarks

NVIDIA provides Deep Learning benchmark tests for comparing performance of different devices. Here’s a comparison of the Jetson AGX Xavier with the Jetson AGX Orin:

Performance on Vision AI and Conversational AI Models

Courtesy NVIDIA

Jetson AGX Orin delivers a 3-6X performance gain over Jetson AGX Xavier. Of course, all of this is highly variable based on your application. However, you can expect a 1.5-2X boost for the CPU. For the GPU, the AGX Orin has 4X the number of cores as the AGX Xavier, so it’s reasonable to expect a 3-10X bump depending. The Deep Learning Accelerator (DLA) is faster than that on the Xaviers.

Availability

The Developer Kit is available now through participating outlets. This includes Arrow, Seeed Studio and Silicon Highway.

Documentation

NVIDIA provides several thousand pages of documentation for the Jetson AGX Orin and the developer kit. Also, the mechanical and support files are freely available. The documents are available from the Jetson Download Center. Access may require registration to access some of the documentation, but don’t worry. Registration is free!

Conclusion

When doing the video, I was gobsmacked by the natural language translation while running another serious deep learning model at the same time. The AGX Orin is a serious machine. If you get the chance, take it for a spin!

Facebook
Twitter
LinkedIn
Reddit
Email
Print

14 Responses

  1. I’ll be interested when the $400 version comes out, if it has more capability than it’s original announcement.

  2. Just checked out mail and noticed they increased price to 2,350$. I guess that must be due to Shangai lockdowns. In previous post I said they will milk this out for years and they did in in a week! Nice Nvidia.

        1. I got a hold of NVIDIA and gave them a good talking to and that you had sussed out their little plan. Of course, once they knew the jig was up, they were forced to lower the price back down to $1999 at Arrow. I am glad you were able to figure that out for us!

          Either that or someone realized that there was a pricing mix up and changed the price back. Hard to tell.

  3. I agree with you about the RIVA natural language translation examples.
    They are quite interesting. Also to be able to run deepstream models at the same time is going to make for a potent combination.
    I think I have a new obsession coming on.

    1. Certainly for robotics it becomes interesting because you can run multiple models at once. Vision recognition, object segmentation, path planning and so on. Another really interesting area is how it enables human interaction. Now it can “understand” what you are saying, with text to speech it will be able to talk to you, and see what you are doing (such as gestures). All without round tripping back to the cloud. To have this all available on device means a whole lot of different applications and interactions open up.
      Try not let your new obsession take away your humanity 🙂

  4. ROS on the AGX Orin.
    I would like to try out some ROS demos on the AGX Orin that pertain to vision.
    What is your recommendations on a Stereo camera?
    Ease of integration to the Jetson ecosphere and python examples are a plus.

    1. Hi Joe, been enjoying your YT videos!
      The manufacturer that pays the most attention to the Jetson market is Stereolabs, who makes the ZED cameras. ZEDs run a DNN on the Jetson to calculate their depth maps. ZEDs have been supporting the Jetson since day one. They have one of the original Jetson Champs, Walter, working in R&D.

      I would think that the Intel RealSense cameras fit pretty will into the Jetson ecosystem. Their are others, of course, but those two are closest out of the box from what I know.

      Each of the cameras have their advantages/draw backs, so it depends on the use case.

    1. It depends on your application. Are you putting it on a robot, a wall, or ?
      You can use longer screws on the 4 mounting holes on the bottom. You can replace the bottom plastic stand with standoffs, and so on.
      Good luck on your project!

Leave a Reply

Your email address will not be published. Required fields are marked *

Disclaimer

Some links here are affiliate links. If you purchase through these links I will receive a small commission at no additional cost to you. As an Amazon Associate, I earn from qualifying purchases.

Books, Ideas & Other Curiosities