JetsonHacks

Developing on NVIDIA® Jetson™ for AI on the Edge

Run Jetson TX1 from SD Card

This is a short article on copying the contents of the internal flash memory of a NVIDIA Jetson TX1 over to a SD card, and then running the Jetson from the SD card itself. Looky here:

Background

There may be times when you want to run your Jetson TX1 from a SD card. For example, you may have to share your Jetson with other people or you may need more disk space to develop applications. This is pretty straightforward. There are a few up front warnings.

First, not all SD cards are created equal. The Jetson TX1 supports UHS-I, speed class up to 10 and capacity to 2 TB. You want to get a card with a speed class of 10, but be aware that the quality of some SD cards can vary even between the same brand and model. People have reported issues with the same model of card from the same manufacturer, one works another may not. Make sure to test out your card after installation.

The SD card used in the video is a PNY Elite Performance 64GB Flash Memory High Speed SDXC Class 10 UHS-I. I have had good luck with these.

Second, note that this is a copy of the the internal flash memory of the Jetson TX1. There are a couple other ways to do the same procedure, for example you can ‘flash’ the SD card from a PC. However, one of the issues is that it is difficult to load CUDA and other packages on to the SD card in as simple a manner as JetPack allows you to do with the internal flash memory. The easiest way to get a complete working system on the SD card is to use JetPack to do a normal install which sets up everything to internal flash, and then copy over the root system of the internal flash to the SD card.

Third, this is a mostly GUI solution, there are command line equivalents that others may use and are more sophisticated. Just be forewarned that if you ask for help, others may speak in ‘CLI‘ language.

Installation

Note: The installation on the video was done on a Jetson TX1 running L4T 24.2.1, which was flashed from JetPack 2.3.1.

Because the installation demonstration is using mostly GUI tools, please refer to the video for the walk through. Here are the basic steps:

First, format the SD card with a ext4 format. You will need at least 1 partition. The partition size should be at least 16GB. There are a couple of ways of doing this, an easy way is to use the Disks application which provides a GUI for formatting disks.

Second, mount the SD card. Double clicking the SD card icon will mount the SD card and open a file browser.

Third, copy the contents of the root directory of the internal flash memory to the SD card. In the video example:

$ sudo cp -ax / ‘/media/ubuntu/SD Root’

copies the root directory to the mounted SD card named ‘SD Root’. For a nearly full internal flash memory, this will take about 20 minutes.

The last setup step is to modify the file extlinux.conf file on the SD card. Make sure that you switch over to the SD card, e.g.

$ cd ‘/media/ubuntu/SD Root’

make a backup of the extlinux.conf file, and then edit it:

$ cd /boot/extlinux
$ sudo cp extlinux.conf extlinux.conf.original
$ sudo gedit /boot/extlinux/extlinux.conf

An editor will open on the configuration file. Duplicate the eMMC entry (usually named primary), and change the entries as shown in the video. One of the major changes indicates the device and partition, which is:

Internal eMMC

root=/dev/mmcblk0p1

SD Card:

root=/dev/mmcblk1p1

Sample extlinux.conf File

The completed example file shown in the video is:

TIMEOUT 30
DEFAULT sdcard

MENU TITLE p2371-2180 eMMC boot options

LABEL sdcard
MENU LABEL SD Card
LINUX /boot/Image
INITRD /boot/initrd
FDT /boot/tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb
APPEND fbcon=map:0 console=tty0 console=ttyS0,115200n8 androidboot.modem=none androidboot.serialno=P2180A00P00940c003fd androidboot.security=non-secure tegraid=21.1.2.0.0 ddr_die=2048M@2048M ddr_die=2048M@4096M section=256M memtype=0 vpr_resize usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 debug_uartport=lsport,0 earlyprintk=uart8250-32bit,0x70006000 maxcpus=4 usbcore.old_scheme_first=1 lp0_vec=${lp0_vec} nvdumper_reserved=${nvdumper_reserved} core_edp_mv=1125 core_edp_ma=4000 gpt android.kerneltype=normal androidboot.touch_vendor_id=0 androidboot.touch_panel_id=63 androidboot.touch_feature=0 androidboot.bootreason=pmc:software_reset,pmic:0x0 net.ifnames=0 root=/dev/mmcblk1p1 rw rootwait

LABEL internalemmc
MENU LABEL Internal EMMC
LINUX /boot/Image
INITRD /boot/initrd
FDT /boot/tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb
APPEND fbcon=map:0 console=tty0 console=ttyS0,115200n8 androidboot.modem=none androidboot.serialno=P2180A00P00940c003fd androidboot.security=non-secure tegraid=21.1.2.0.0 ddr_die=2048M@2048M ddr_die=2048M@4096M section=256M memtype=0 vpr_resize usb_port_owner_info=0 lane_owner_info=0 emc_max_dvfs=0 touch_id=0@63 video=tegrafb no_console_suspend=1 debug_uartport=lsport,0 earlyprintk=uart8250-32bit,0x70006000 maxcpus=4 usbcore.old_scheme_first=1 lp0_vec=${lp0_vec} nvdumper_reserved=${nvdumper_reserved} core_edp_mv=1125 core_edp_ma=4000 gpt android.kerneltype=normal androidboot.touch_vendor_id=0 androidboot.touch_panel_id=63 androidboot.touch_feature=0 androidboot.bootreason=pmc:software_reset,pmic:0x0 net.ifnames=0 root=/dev/mmcblk0p1 rw rootwait

Make sure to save the file.

Now restart the machine and it will boot from the SD card. The boot order on the Jetson TX1 is SD card and then eMMC, so if the SD card is present with a boot directory it will boot from there.

Note: If the machine does not boot, hook up a serial console and examine the boot sequence. The entries that you have changed should show up in the boot options.

Once the machine boots, you should run some typical programs or compiles to make sure everything works and is installed correctly. There are some things to watch out for, such as check to make sure that settings for swap files are correct. In general, it is a copy of the internal eMMC, so it should all pretty much behave itself.

Facebook
Twitter
LinkedIn
Reddit
Email
Print

39 Responses

  1. Hi, thank you for the instructions. I have two nodes, one with L4T 24.1 I installed with the older jetpack and one with 24.2 which I installed with a newer jetpack on their emmc. I followed your instructions and I have no problem when I’m using my SD card on the node that has the same version of L4T, it boots to SD card perfectly and I checked it boots up to the sd card. But when I try to boot up the node that has a different version of L4t (installed with different jetpack) in the internal emmc, it never boots: Here is the log I when I tried to put the sd card of 24.1 to the one that has 24.2 on emmc:

    1823 bytes read in 436 ms (3.9 KiB/s)
    p2371-2180 eMMC boot options
    1: SD Card
    2: internal EMMC
    Enter choice: 1: SD Card
    Retrieving file: /boot/Image
    19400088 bytes read in 923 ms (20 MiB/s)
    append: fbcon=map:0 console=tty0 console=ttyS0,115200n8 androidboot.modem=none androidboot.serialno=P2180A00P00940c003fd androidboot.security=non-secure tt
    Retrieving file: /boot/tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb
    304910 bytes read in 511 ms (582 KiB/s)
    ## Flattened Device Tree blob at 82000000
    Booting using the fdt blob at 0x82000000
    reserving fdt memory region: addr=80000000 size=20000
    Using Device Tree in place at 0000000082000000, end 000000008204d70d
    Can’t create DT prop nvidia,emc-save-restore-mod-regs to copy
    Can’t create DT node emc-table@102000 to copy
    Can’t create DT prop reg to copy
    ERROR: could not update sku property FDT_ERR_NOSPACE.
    ERROR: board-specific fdt fixup failed: FDT_ERR_NOSPACE
    – must RESET the board to recover.

    FDT creation failed! hanging…### ERROR ### Please RESET the board ###

    After some googling, it seems this error is due to the difference device tree of the internal emmc (needed before rootfs get mounted from sd card) and the sd card. Did I get it right? is there any easy solution? Does this mean I only can use my sd card on the same version of that jetpack installed on emmc?

    1. Using the method described in the article, the machine boots from the files in /boot/ on the eMMC, then sets the root directory to be the SD card. Therefore it expects the SD card to have the same kernel as the eMMC. That means you can’t mix and match.

      You can also create a system that boots from the SD card without going through the eMMC. Since the Jetson always looks at the SD card slot first to boot from, that means you can have different kernels on the eMMC and the SD card. Everything runs off the SD card. This latter method is described in the NVIDIA documentation.

  2. Can someone confirm if it’s indeed possible to boot completely from the SD card?

    So far all the instructions I’ve seen including this one are only letting you boot the root filesystem from the SD card.

    http://www.tsuremote.net/Karekinada/jetson/nvl4t_docs/nvl4t_docs/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/l4t_uboot_guide.html

    So you’re still reliant on the u-boot within the eMMC. This document says “you must flash U-Boot to internal eMMC only”.

    To me this means there is no way we can put a drive into the machine (SD, USB, etc) and have it boot completely from it. So you can’t have different drives/cards laying around with that have different jetpack versions on them.

    You also can’t stick an SD card into a brand new TK2 and I expect it to boot. But, you can on other ARM processors. I did this all the time on an OMAP3730.

    1. Previous to version L4T 28.1, this was possible. However, the boot process changed to a four step process in converting everything over to 64-bit.

      This question was answered in the NVIDIA Developers Forum. Here’s the answer:

      linuxdev writes: The answer to COMPLETELY booting from SD is no, it cannot. The boot loader and environment remain on eMMC regardless of which media the boot loader looks to for further configuration. Flashing to mmcblk1p1 changes extlinux.conf and whether it points to eMMC’s version of extlinux.conf or whether it points to the SD card’s extlinux.conf. Boot loader binary remains on eMMC. Only the root partition of eMMC can be deleted.

  3. Hi kangalow!
    When i follow your step,The user of Jetson TX1 always from ubuntu to Nvidia by itslef,When i boot my TX1,The user is NVIDIA,no ubuntu,Could you please tell me the reason?thank you so much!

  4. Hi there, has anyone tested on Jetpack 3.1? Just updated my TX1 and this is not longer working, there were issues copying to the SD and even tough I changed my extlinux.conf to:

    TIMEOUT 30
    DEFAULT sdcard

    MENU TITLE p2371-2180 eMMC boot options

    LABEL sdcard
    MENU LABEL primary kernel
    LINUX /boot/Image
    INITRD /boot/initrd
    APPEND ${cbootargs} root=/dev/mmcblk1p1 rw rootwait

    LABEL primary
    MENU LABEL primary kernel
    LINUX /boot/Image
    INITRD /boot/initrd
    APPEND ${cbootargs} root=/dev/mmcblk0p1 rw rootwait

    It is not working anymore. Any tips?

    1. Same problem here, on a TX2. Followed all these instructions, but the Jetson boots straight up from internal eMMC, and basically ignores extlinux.conf. Would love to know the answer to this.

  5. I am having the same experience as Ben P., these directions no longer seem to work with Jetpack 3.1.

    In fact, after doing these edits, I then edited extlinux.conf to remove the root=/dev/mmcblk0p1 completely, still booted from emmc. So, then I actually went and deleted all the files from /boot/extlinux diretory, and the thing still boots up, so with the Jetpack 3.1 release, unfortunately, it no longer seems to be using this .conf to determine the boot conf.

    I bought one of these Jetson TX1 to participate in the RocketChallenge contest, but even just getting to the point where the system is usable with enough disk space to install files and play around with it is turning out to be a major pain the rear. Any help you can provide to figuring out how to boot Jetpack 3.1 and use the SD card for the root filesystem would be grately appreciated

      1. Hello Kangalow,
        I am working on jetson-TX2 with ubuntu 18.04 and has a jetpack 4.2 installed and i have a samsung 128GB SD card.

        I followed all the steps as mentioned above, the extlinux.conf file of internal emmc is stored at /boot/extlinux and it looks like this
        TIMEOUT 30
        DEFAULT primary

        MENU TITLE L4T boot options

        LABEL primary
        MENU LABEL primary kernel
        LINUX /boot/Image
        INITRD /boot/initrd
        APPEND ${cbootargs} quiet

        While the extlinux.conf file is stored at media/device/device/boot/extlinux and that looks like this:
        TIMEOUT 30
        DEFAULT sdcard

        MENU TITLE L4T boot options

        LABEL sdcard
        MENU LABEL SD card
        LINUX /boot/Image
        INITRD /boot/initrd
        APPEND ${cbootargs} root = /dev/mmcblk2p1 rw rootwait

        LABEL primary
        MENU LABEL primary kernel
        LINUX /boot/Image
        INITRD /boot/initrd
        APPEND ${cbootargs} quiet

        Please tell me what am I doing wrong to not get the desired output?

  6. OK, I actually just figured this out, maybe it’s help BenP. I’m posting some instructions from the NVIDIA Tegra Linux Driver Package Development Guide 28.1 Release (I would post a direct link, but not sure it’s hosted anyway. Google for it and download and .unzip to see yourself).

    Boot Sequence and Sysboot Configuration Files
    The U-Boot functionality includes a default booting scan sequence. It scans bootable devices in the following order:
    • External SD Card
    • Internal eMMC
    • USB Device
    • NFS Device
    It looks for an extlinux.conf configuration file in the following directory of the bootable partition in the device:
    /boot/extlinux

    So, the problem is, on boot up, the system first scans the SD Card, then the EMMC for the extlinux.conf. I accidentally skipped an extremely important step in the video, the cd /media/ubuntu/sdcard, so I was actually editing the emmc version of the extlinux.conf, not the sdcard’s version (which was left exactly the same as the original, which tells the board to use the emmcs filesystem as root).

    Anyway, my bad, got be more careful. Thanks for making all the tutorials, they’ve been super helpful and hopefully now I can get down to the business of making something cool.

  7. Hi kangalow, I was actually able to resolve my problem, I was editing the wrong extlinux.conf, I was editing the one on the emmc, not the one on /media/nividia/ which reflected the original of course, so I shouldn’t expect to see anything different. Once I edited the correct extlinux.conf, everything is working as expected.

    Thanks for your site and videos, they have been very helpful.

    1. Thank you for the kind words. I’m glad you got it to work!
      The extlinux.conf is a little confusing, I’ve tried to make it as clear as I could but the naturally inclination is to switch to the /boot directory, which always lands you back on the eMMC in this case.
      You should remember to set your extlinux.conf on the eMMC to the original, in case you ever remove the SD card and try to boot the machine.

      1. Your instructions in the article actually say to “$ cd /boot/extlinux”, which will end up editing the extlinux.conf on the internal eMMC. But even if I edit BOTH the extlinux.conf on the internal eMMC and on the sd card, Jetson doesn’t seem to want to boot from the SD Card. (using a TX2 on Jetpack 3.2)

        1. Strange, after rebooting a couple of times, now it’w working perfectly! Thanks for a great article!

  8. Hi, good article. Just curious to know. Does reboot from sd or ssd slow the tx1/2. I am working on a SLAM project so speed maters and I am not really a hardware guy. Any comment is appreciate.

  9. Thx for a good article. I have a question to ask you. I installed jetpack on tx1 board internal flash memory.
    but Because of the lack of storage, I wanna try to move root to other sd card likely this article. and I wonder if there could be any problem by transfer jetpack to internal memory to external memory. Moving Jetpack from internal memory to external one won’t make any problem?

    1. I do not have an answer to your question. The article covers how to do what you describe, but “any problem” includes such a large number of things that it is impossible to prove. Thanks for reading!

  10. Can you post a video about how to clone a TX2 without the miniUSB port? I am working on a robot with TX2. Recently, the minUSB port of the TX2 broke. I have to buy a new TX2 and would like to clone my existing one to the new board.

    Thanks

  11. Hi there, Could you please outline for me how this procedure changes for a Jetson TX2 rather than a TX1?
    Many thanks

  12. I am using the image from JetPack 3.3, but my TX1 will not boot from the SD card. The extlinux.conf file on the card was what was modified. Is this something that the serial console could take care of?

  13. Thanks for the wonderful tutorial! Loved it.

    I followed every single step from the tutorial.
    I am using jetpack 4.6 and Ubuntu 18.04. Below is my configuration. I edited the extlinux.conf file on the jetson SD card. After the reboot however it is still running on emmc.

    Please help.

    TIMEOUT 30
    DEFAULT sdcard

    MENU TITLE L4T boot options

    LABEL sdcard
    MENU LABEL SD Card
    LINUX /boot/Image
    INITRD /boot/initrd
    APPEND ${cbootargs} quiet root=/dev/mmcblk1p1 rw rootwait rootfstype=ext4 console=ttyTCU0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 rootfstype=ext4

    LABEL internalemmc
    MENU LABEL Internal EMMC
    LINUX /boot/Image
    INITRD /boot/initrd
    APPEND ${cbootargs} quiet root=/dev/mmcblk0p1 rw rootwait rootfstype=ext4 console=ttyTCU0,115200n8 console=tty0 fbcon=map:0 net.ifnames=0 rootfstype=ext4

    # When testing a custom kernel, it is recommended that you create a backup of
    # the original kernel and add a new entry to this file so that the device can
    # fallback to the original kernel. To do this:
    #
    # 1, Make a backup of the original kernel
    # sudo cp /boot/Image /boot/Image.backup
    #
    # 2, Copy your custom kernel into /boot/Image
    #
    # 3, Uncomment below menu setting lines for the original kernel
    #
    # 4, Reboot

    # LABEL backup
    # MENU LABEL backup kernel
    # LINUX /boot/Image.backup
    # INITRD /boot/initrd
    # APPEND ${cbootargs}

Leave a Reply

Your email address will not be published. Required fields are marked *

Disclaimer

Some links here are affiliate links. If you purchase through these links I will receive a small commission at no additional cost to you. As an Amazon Associate, I earn from qualifying purchases.

Books, Ideas & Other Curiosities