JetsonHacks

Developing on NVIDIA® Jetson™ for AI on the Edge

Build TX1 Kernel and Modules – NVIDIA Jetson TX1

Note: This article is for an older version of L4T 24.2.1. A newer version of this article is now available. Please see: Build Kernel for Jetson TX1 .

In this article, we cover building a kernel onboard the NVIDIA Jetson TX1. Looky here:

Background and Motivation

Note: This article is for intermediate users, and the methods within are somewhat experimental. You should be familiar with the purpose of the kernel. You should be able to read shell scripts to understand the steps described.

Note: The kernel source must match the version of L4T that has been flashed onto the Jetson. For example, here we use the kernel source for L4T 24.2.1 with L4T 24.2.1. Kernel versions are not compatible across releases.

When the Jetson TX1 was first shipped the operating system, L4T 23.1, was a hybrid 64-bit kernel, 32-bit user space affair. The only practical route to rebuild the kernel was to use a host computer because two different development toolchains are needed.

With the introduction of the L4T 24.X releases (currently L4T 24.2), both the kernel and the user space are now 64-bit. NVIDIA gives detailed instructions on how to build the system using a host computer. There are other good sets of instructions around, including “Compiling Tegra X1 source code” over at RidgeRun.

If you are building systems which require generating the entirety of Jetson TX1 system, those are good options. For a person like me, it’s a little overkill. Most of the time I just want to compile an extra driver or two as modules to support some extra hardware with the TX1. What to do, what to do …

As it turns out the 24.X kernels are mostly 64-bit, but there’s a sprinkle of 32-bit object files here and there. So here’s an idea: why not just grab the compiled 32-bit sprinkles from a host build, and put them into the kernel build process? That way the kernel and modules can be built on the Jetson TX1 itself without the need for a host development environment.

Now normally I am against developing on host machines when I can develop on a target machine. The TX1 is certainly competent enough for development. But in this case, it seemed worth it to build the needed 32-bit object files on a development host. Once the 32-bit files are built, it is a pretty straightforward task to build the rest of the kernel (with the associated modules) entirely on the TX1 itself.

Installation

The script files to build the kernel on the Jetson TX1 are available on the JetsonHacks Github account in the buildJetsonTX1 repository.

$ git clone https://github.com/jetsonhacks/buildJetsonTX1Kernel.git
$ cd buildJetsonTX1Kernel

There are three main scripts. The first script, getKernelSources.sh gets the kernel sources from the NVIDIA developer website, then unpacks the sources into /usr/src/kernel.

$ ./getKernelSources.sh

After the sources are installed, the script opens an editor on the kernel configuration file. In the video, the local version of the kernel is set. The stock kernel uses -tegra as its local version identifier. Make sure to save the configuration file when done editing.

The second script, patchAndBuildKernel.sh, patches one of the sources files to more easily compile the kernel.

$ ./patchAndBuildKernel.sh

Then the script copies over the 32-bit object files, and proceeds to build the kernel and modules using make. The modules are then installed in /lib/modules/

The third script, copyImage.sh, copies over the newly built Image and zImage files into the /boot directory.

$ ./copyImage.sh

Once the images have been copied over to the /boot directory, the machine must be restarted for the new kernel to take effect.

Spaces!

The kernel and module sources, along with the compressed versions of the source, are located in /usr/src

After building the kernel, you may want to save the sources off-board to save some space (they take up about 3GB) You can also save the boot images and modules for later use, and to flash other Jetsons from the PC host.

Conclusion

For a lot of use cases, it makes sense to be able to compile the kernel and add modules from the device itself. This particular technique is the first attempt at that. Note that it is new, and not thoroughly tested at this point. Use it at your own risk.

Note

The video above was made directly after flashing the Jetson TX1 with L4T 24.2 using JetPack 2.3.

Facebook
Twitter
LinkedIn
Reddit
Email
Print

22 Responses

  1. Can only the modules be compiled? I don’t want the whole kernel. I incidentally need the CH341 module you showed in the video. 🙂 Tried compiling the “regular” way, but that didn’t work – the module was not in the correct format (had the “Exec format error” – I guess it was 32bit instead of 64bit). So I am willing to try your scripts, but as I only need one module I thought it might be an overkill.

    1. The short answer is that it’s easiest just to build everything the first time just to figure out what is needed. The first thing is that you have to configure the system correctly (setting the local version to be the same as the current version), download all the source code, etc. The next part is that you have to enable the module that you want to build, in which it uses the configuration information to tell the module which version of the OS it is being built for.

      Then there’s the bit about ‘preparing the modules’ and then actually building the modules. Yes, once you know exactly what needs to be built and where it belongs you can build a module or two; unfortunately it’s a pretty steep learning curve and I’ve found it just easier to build everything in an automated fashion rather than fiddle with the bits and pieces. I know that’s not the answer you were looking for, and I’m sure there are a lot of people much more experienced who have words of advice that could walk you through it, but that’s what I usually end up doing.

      1. I thought it might be possible with your script. I think this is what you basically did in the video anyway. Downloaded the kernel source, configured to enable CH341 and then compiled. You even enabled CH341 as a module, so it was compiled as a module. Just instead of copyImage I will copyModule. As the compiled .ko should be in the directories anyway, then I should be able to find it and install it. So I will try your script.

        Can you conform the CH341 module actually loads and works? Can you modprobe CH341? I couldn’t, which is why I’m willing to try your way.

        Thanks for help!

    1. I’m new to Kernel stuff and I want to thank you for this. I needed the CH341 module to run cheap Chinese boards over USB and after hours of tentative this worked immediately.
      However, could I add the CH341 module directly when I flash everything from host PC using Jetpack?

      1. Thank you for the kind words. Unfortunately there’s really no way to configure JetPack to add custom modules. Usually what people end up doing is installing from JetPack initially, configuring a machine with everything they need and then clone the image back to the host machine. Then, if need be, you can flash the full image back to the Jetson without having to build everything from scratch. Thanks for reading!

  2. Thanks for this, worked first time and saved a lot of annoyance – as you say, cross-compiling is overkill to add a single kernel module!

  3. Do you have any other scripts that clean off the Jetson of extraneous stuff? Jetpack 3.0 in particular seems to fill it right up… I am only using remote login and CLUI so I suspect I could wipe all GUI… that should reclaim some room.

  4. Hi.
    I just follow your command with script.
    after reboot, I checked the kernel version on my PC.
    the kernel version did not change new kernel.
    the kernel vesion(3.10.96-tegra) was still use.
    How can I change the kernel version?
    If I miss something doing it please let me know.

    thank.

      1. Hi,
        same pb with me. I check the localversion string is “-jetsonbot-v0.1”. Also the kernel has been build in usr/modules. The thing is that I also followed your tutorial to install a ssd as the main storage, could it be the reason why the new kernel is not loaded ?
        PS : thx for your great work !

        1. Ok I don’t have the whole answer but if you put your Image & zImage inside the “/boot” directory of the mounted linux file sys of the SSD you have the right answer for “uname -r”

  5. Thank you for the kind words. As you noted, that’s the issue you are probably experiencing.
    In the SSD setup, the machine actually boots of the Image file that’s located on the eMMC, and then sets the root to point at the SSD. Therefore if you compile the kernel on the SSD itself, you need to also mirror it on the eMMC.
    You can just copy over the Image file, but if you choose that route then you have a new kernel image with the old modules. If you are trying to be able to switch between different media at boot time using the serial cable, it’s better to have everything booting the same OS and module structure, so copy everything over to the eMMC. Thanks for reading!

  6. Hi Jim,

    I ran into issues with the sprinkle of 32-bit object files ‘arch/arm64/kernel/vdso32/vgettimeofday.o’

    I like to keep L4T 24.2.1 and build the needed 32-bit object files on a development host. So I followed your link to github and downloaded the files.
    I noticed that scripts were to download L4T 28.1 (wget http://developer.download.nvidia.com/embedded/L4T/r28_Release_v1.0/BSP/source_release.tbz2) and patchAndBuildKernel.sh was missing.

    Are you still supporting L4T 24.2.1 custom kernel or should we just plant a New Operating System L4T 24.2.1 on the TX1?

    Thanks
    Steve

Leave a Reply

Your email address will not be published. Required fields are marked *

Disclaimer

Some links here are affiliate links. If you purchase through these links I will receive a small commission at no additional cost to you. As an Amazon Associate, I earn from qualifying purchases.

Books, Ideas & Other Curiosities