JetsonHacks

Developing on NVIDIA® Jetson™ for AI on the Edge

Jetson Orin Nano – Flashing QSPI Firmware for More Memory

Early production units of the NVIDIA Jetson Orin Nano Developer Kits can flash their QSPI firmware to gain one more gigabyte of system memory. Looky here:

Introduction

When you first boot up your Jetson Orin Nano Developer Kit, take a look at the available system memory. On early production units, we notice that even though there are 8GB of physical memory on the module, there are only 6.3GB available from the system. Normally we expect the system to reserve ~ 10% of memory on a system this size. That means that we have about 1GB less memory than we expect.

The firmware includes the bootloader(s), device configuration files and hardware configuration information.

Before the units go into production, there is typically “functional” firmware which may include extra memory carveouts for debugging and development. These extra carveouts are usually eliminated when the device enters production.

However, the early production units still have these memory carveouts in the firmware. Fortunately there is a firmware update which you can perform that will add back 1GB of system memory.

Jetson Orin Nano QSPI

The Jetson Orin Nano and Orin NX have a slightly different memory architecture from previous Jetson modules of the same form factor.

The previous Jetson Nano and Jetson Xavier NX are available in two flavors. The modules on the NVIDIA Jetson developer kits have a SD card. The firmware for these modules resides in a QSPI-NOR type of flash memory. The QSPI-NOR is a small chip on the Jetson module, which is commonly referred to as QSPI. NOR is a type of flash memory, QSPI is the serial interface which connects the flash memory to the Jetson Tegra chip. QSPI stands for Quad Serial Peripheral Interface.

On the other hand, the Jetson Nano and the Xavier NX production modules use a different type of flash memory called eMMC. This type of memory is similar to the flash memory in a USB thumb drive. The production modules do not have QSPI memory.

The bootloader firmware for the Jetson is kept on the eMMC. This is why the eMMC needs to be flashed before a system can boot from an external drive like a NVMe SSD. It is on the eMMC that the boot instructions reside.

There are advantages in the eMMC approach. One is parts count, simply using a eMMC chip of an appropriate size eliminates the need for all the support circuitry for an external drive. Reducing the parts count reduces the cost and complexity. If you can fit your application and data gathering requirements onto the eMMC, you can use a simpler carrier board.

However the major drawback is expandability. The eMMC for these modules can hold between 16 and 32GB. In the brave new world of machine learning, people simply didn’t have enough space to run their applications. The main advantage of having everything on the module disappears.

Orin Nano and Orin NX – No eMMC

There are probably a couple of reasons that there is no eMMC on the Orin Nano and Orin NX production modules. First is the reason above, plus bigger eMMC chips are expensive. Second has to do with parts availability. Having eMMC on board means another chip on board, with the usual parts availability issue.

In response NVIDIA simply adds a QSPI chip to the Orin small factor production modules. The QSPI chip holds the bootloader and hardware configuration files. The bootloader then loads Jetson Linux from a NVMe SSD attached to the carrier board.

Flashing the Orin Nano

There are a couple of ways to flash the new firmware on the Jetson Orin Nano Developer Kit. Remember that the developer kit is compatible with both the Orin Nano and the Orin NX modules. In order to flash the firmware, you will need an x86 PC, which we call the host. You will also need a data capable USB-C cable to connect the Orin Nano Dev Kit to the PC. Make sure that the cable is not only for charging, it must have data handling capabilities. You can use a GUI application that NVIDIA provides, called the SDK Manager. There is video on the JetsonHacks YouTube channel that provides a walk through of the process. If you are new to the flashing process, this is the NVIDIA preferred way to start.

Using the SDK Manager, you flash Jetson Linux to a new SD card and flash the QSPI firmware on the Jetson at the same time. An alternative method is to use the SDK Manager to flash Jetson Linux to a PCIe SSD and flash the QSPI firmware at the same time.

Flash the Orin Nano from the Command Line

The method here flashes only the QSPI. This is most useful for people that want to use the standard system SD card image that NVIDIA provides. Updating the QSPI firmware will gain the extra system memory, and any recent updates to the bootloader firmware. Note: If you are flashing the Orin Nano to boot from a SSD, then the QSPI firmware is also updated. You may not want to do the following procedure.

If you are a professional developer, or simply more comfortable using the Linux command line, you can flash the Orin Nano QSPI in the following manner. The entire process takes ~ 20 minutes, depending on the speed of your host machine and Internet connection.

Here is a link to the official NVIDIA documentation for flashing. The scripts here are convenience wrappers for those instructions.

$ git clone https://github.com/jetsonhacks/bootFromExternalStorage.git

$ cd bootFromExternalStorage

$ ./get_jetson_files.sh

The get_jetson_files.sh script will download the Board Support Package (BSP) and sample root file system (rootfs) for the Jetson Orin and Xaviers. The script will then decompress the files and copy NVIDIA user space libraries into the rootfs (apply_binaries.sh). Last, the script will install the prerequisites libraries on the host to flash the Jetson.

Then, set the Jetson into force recovery mode. One way to do this is to remove the power, jumper pins 9 and 10 together on the J14 button header, and apply power to the Jetson. Connect the Jetson via the USB-C cable to the host machine. Running lsusb in a Terminal on the host should then show the Orin. The Orin Nano will show as 0955:7523 NVIDIA Corp. APX in force recovery mode.

You’re now ready to flash the QSPI. Switch over to the top level board support package directory, Linux_for_Tegra. For release 35.3.1, this will be:

$ cd R35.3.1/Linux_for_Tegra

Then flash:

$ sudo ./flash.sh -c ./bootloader/t186ref/cfg/flash_t234_qspi.xml jetson-orin-nano-devkit mmcblk0p1

The flash part will take ~ 15 minutes or so. After the flash is complete, you are ready to reboot your Jetson again. Make sure that you remove the jumper.

You should have ~ 7.2GB available on your system after this update.

Conclusion

The firmware update is a little inconvenient if you only use the SD card method of development. It’s an added bit of work to get your Jetson working to its desired potential. But with a little work, it’s certainly nice to have the extra system memory for running those memory intensive applications.

Facebook
Twitter
LinkedIn
Reddit
Email
Print

6 Responses

    1. The newer Apples with M1 and M2 chips do not. Otherwise most every other notebook does. Anything that runs Windows is more than likely to be x86. Typically these are denoted as having Intel or AMD chips. Thanks for reading!

  1. Hi, I ran into an error with this. Any idea what this might be due to?

    [ 0.0795 ] MB1-BCT version: 0.10
    [ 0.0796 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_UNUSED5/ is not supported
    [ 0.0799 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_UNUSED5/ is not supported
    [ 0.0802 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_UNUSED5/ is not supported
    [ 0.0804 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_MCE_COVERAGE/ is not supported
    [ 0.0806 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_MCE_COVERAGE/ is not supported
    [ 0.0807 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_MCE_COVERAGE/ is not supported
    [ 0.0807 ] ERROR: /misc/tsc_controls/tsc_locking_config is not supported
    [ 0.0807 ] ERROR: /misc/tsc_controls/tsc_locking_diff_configuration is not supported
    [ 0.0807 ] ERROR: /misc/tsc_controls/tsc_locking_ref_frequency_configuration is not supported
    [ 0.0807 ] ERROR: /misc/tsc_controls/tsc_locking_control is not supported
    [ 0.0807 ] ERROR: /misc/tsc_controls/tsc_locking_adjust_configuration is not supported
    [ 0.0807 ] ERROR: /misc/tsc_controls/tsc_locking_fast_adjust_configuration is not supported
    [ 0.0807 ] ERROR: /misc/tsc_controls/tsc_locking_adjust_delta_control is not supported
    [ 0.0807 ] ERROR: /misc/tsc_controls/tsc_capture_control_ptx is not supported
    [ 0.0807 ] ERROR: /misc/tsc_controls/tsc_capture_config_ptx is not supported
    [ 0.0807 ] ERROR: /misc/tsc_controls/tsc_stscrsr is not supported
    [ 0.0807 ] ERROR: /misc/tsc_controls/tsc_locking_adjust_num_control is not supported
    [ 0.0807 ]
    [ 0.0807 ] Parsing config file :tegra234-mb1-bct-device-p3767-0000_cpp.dtb
    [ 0.0807 ] Added Platform Config 9 data with size :- 100
    [ 0.0807 ]
    [ 0.0807 ] Updating mb1-bct with firmware information
    [ 0.0812 ] tegrabct_v2 –chip 0x23 0 –mb1bct mb1_cold_boot_bct_MB1.bct –updatefwinfo readinfo_t234_min_prod.xml.bin
    [ 0.0829 ] tegrahost_v2 –chip 0x23 0 –align mb1_cold_boot_bct_MB1_aligned.bct
    [ 0.0839 ] tegrahost_v2 –chip 0x23 0 –magicid MBCT –appendsigheader mb1_cold_boot_bct_MB1_aligned.bct zerosbk
    [ 0.0844 ] adding BCH for mb1_cold_boot_bct_MB1_aligned.bct
    [ 0.0854 ] tegrasign_v3.py –key None –list mb1_cold_boot_bct_MB1_aligned_sigheader.bct_list.xml –pubkeyhash pub_key.key –sha sha512
    [ 0.0855 ] Assuming zero filled SBK key
    [ 0.0864 ] Warning: pub_key.key is not found
    [ 0.0865 ] tegrahost_v2 –chip 0x23 0 –updatesigheader mb1_cold_boot_bct_MB1_aligned_sigheader.bct.encrypt mb1_cold_boot_bct_MB1_aligned_sigheader.bct.hash zerosbk
    [ 0.0873 ] Generating recovery mb1-bct
    [ 0.0878 ] tegrabct_v2 –chip 0x23 0 –mb1bct mb1_bct.cfg –misc tegra234-mb1-bct-misc-p3767-0000_cpp.dtb –device tegra234-mb1-bct-device-p3767-0000_cpp.dtb
    [ 0.0883 ] MB1-BCT version: 0.10
    [ 0.0884 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_UNUSED5/ is not supported
    [ 0.0887 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_UNUSED5/ is not supported
    [ 0.0889 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_UNUSED5/ is not supported
    [ 0.0891 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_MCE_COVERAGE/ is not supported
    [ 0.0893 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_MCE_COVERAGE/ is not supported
    [ 0.0894 ] ERROR: carveout /misc/carveout/aux_info@CARVEOUT_MCE_COVERAGE/ is not supported
    [ 0.0894 ] ERROR: /misc/tsc_controls/tsc_locking_config is not supported
    [ 0.0894 ] ERROR: /misc/tsc_controls/tsc_locking_diff_configuration is not supported
    [ 0.0894 ] ERROR: /misc/tsc_controls/tsc_locking_ref_frequency_configuration is not supported
    [ 0.0894 ] ERROR: /misc/tsc_controls/tsc_locking_control is not supported
    [ 0.0894 ] ERROR: /misc/tsc_controls/tsc_locking_adjust_configuration is not supported
    [ 0.0894 ] ERROR: /misc/tsc_controls/tsc_locking_fast_adjust_configuration is not supported
    [ 0.0894 ] ERROR: /misc/tsc_controls/tsc_locking_adjust_delta_control is not supported
    [ 0.0894 ] ERROR: /misc/tsc_controls/tsc_capture_control_ptx is not supported
    [ 0.0894 ] ERROR: /misc/tsc_controls/tsc_capture_config_ptx is not supported
    [ 0.0894 ] ERROR: /misc/tsc_controls/tsc_stscrsr is not supported
    [ 0.0894 ] ERROR: /misc/tsc_controls/tsc_locking_adjust_num_control is not supported
    [ 0.0894 ]
    [ 0.0894 ] Parsing config file :tegra234-mb1-bct-device-p3767-0000_cpp.dtb
    [ 0.0894 ] Added Platform Config 9 data with size :- 100
    [ 0.0894 ]
    [ 0.0895 ] Updating mb1-bct with firmware information
    [ 0.0899 ] tegrabct_v2 –chip 0x23 0 –mb1bct mb1_bct_MB1.bct –recov –updatefwinfo readinfo_t234_min_prod.xml.bin
    [ 0.0912 ] tegrahost_v2 –chip 0x23 0 –align mb1_bct_MB1_aligned.bct
    [ 0.0923 ] tegrahost_v2 –chip 0x23 0 –magicid MBCT –appendsigheader mb1_bct_MB1_aligned.bct zerosbk
    [ 0.0928 ] adding BCH for mb1_bct_MB1_aligned.bct
    [ 0.0938 ] tegrasign_v3.py –key None –list mb1_bct_MB1_aligned_sigheader.bct_list.xml –pubkeyhash pub_key.key –sha sha512
    [ 0.0939 ] Assuming zero filled SBK key
    [ 0.0948 ] Warning: pub_key.key is not found
    [ 0.0948 ] tegrahost_v2 –chip 0x23 0 –updatesigheader mb1_bct_MB1_aligned_sigheader.bct.encrypt mb1_bct_MB1_aligned_sigheader.bct.hash zerosbk
    [ 0.0957 ] Error: Skip generating mem_bct because sdram_config is not defined
    [ 0.0957 ] Error: Skip generating mem_bct because sdram_config is not defined
    [ 0.0957 ] Copying signatures
    [ 0.0962 ] tegrahost_v2 –chip 0x23 0 –partitionlayout readinfo_t234_min_prod.xml.bin –updatesig images_list_signed.xml
    [ 0.1003 ] mb1_t234_prod_aligned_sigheader.bin.encrypt filename is from images_list
    [ 0.1005 ] psc_bl1_t234_prod_aligned_sigheader.bin.encrypt filename is from images_list
    [ 0.1005 ] Boot Rom communication
    [ 0.1010 ] tegrarcm_v2 –new_session –chip 0x23 0 –uid –download bct_br br_bct_BR.bct –download mb1 mb1_t234_prod_aligned_sigheader.bin.encrypt –download psc_bl1 psc_bl1_t234_prod_aligned_sigheader.bin.encrypt –download bct_mb1 mb1_bct_MB1_sigheader.bct.encrypt
    [ 0.1015 ] BR_CID: 0x80012344705DD5032800000012020240
    [ 0.1297 ] Sending bct_br
    [ 0.1733 ] ERROR: might be timeout in USB write.
    Error: Return value 3
    Command tegrarcm_v2 –new_session –chip 0x23 0 –uid –download bct_br br_bct_BR.bct –download mb1 mb1_t234_prod_aligned_sigheader.bin.encrypt –download psc_bl1 psc_bl1_t234_prod_aligned_sigheader.bin.encrypt –download bct_mb1 mb1_bct_MB1_sigheader.bct.encrypt
    Reading board information failed.

    1. It’s hard to tell. What host machine are you running this on? Which operating system?
      What does lsusb report for the Jetson in recovery mode?

  2. Do you know the config or xml file to change to make the root fs >16gig My NVMe is 128GIG.
    I would think it would be a small change in a config file somewhere.

Leave a Reply

Your email address will not be published. Required fields are marked *

Disclaimer

Some links here are affiliate links. If you purchase through these links I will receive a small commission at no additional cost to you. As an Amazon Associate, I earn from qualifying purchases.

Books, Ideas & Other Curiosities