THIS POST IS DEPRECATED. Please see: https://wp.me/p7ZgI9-3bE
Here’s a fast way to set up your Jetson Nano to run from a USB drive. Looky here:
Background
For external storage, the Jetson Nano uses a Micro SD Card. The SD card holds the operating system, applications and any data in use. In the overall scheme of things, this device provides relatively slow access. Also, even though Micro SD cards are much better now than they have been in the past, the cards have a reputation of low reliability in long term or heavy use.
Most desktop/laptop computers use a different type of external storage, such as a Hard Disk Drive (HDD) or Solid Sstate Disk (SSD). Typically these are built into the computer, though you can add an external one also.
In fact, we can add one of those types of drives to the Jetson Nano through the USB 3.0 port! We will cover setting up our USB drive so that it can be the “root file system” or rootfs for the Nano. The end result is that the system will be much snappier in response because the disk access is much faster.
Typically most larger computers boot directly to the disk drive. However, this is not possible using the current configuration of the boot loader on the Jetson Nano.
Remember that the term ‘boot’ is shorthand for the slang term ‘bootstrapping’, that is, pulling ones self up by their own boot straps. The Jetson uses a two step boot process. The basic idea is that the boot loader loads a memory image with minimal support for key attached peripherals, and then loads over itself with the Linux kernel specified. It’s really clever and quite tricky.
By default, the bootloader loads the Linux kernel from the SD card and then configures the rootfs to point to files on the SD card. Here we will load the Linux kernel from the SD card, but we will Pivot the Root to point the rootfs to files on the USB drive.
Why is this difficult?
During the boot process, the Jetson bootloaders use an initrd (initial ramdisk) to load a temporary file system into memory to load the Linux kernel. The initrd is a minimal file system, with just enough for loading the kernel. What we would like to do is load the kernel, and then switch over to the USB drive.
Here’s the problem. While the actual USB drivers are built in to the Linux kernel, the firmware for the USB devices themselves are not. By the time the root file system gets mounted in the boot process, the USB controller is not yet initialized. You can’t talk to the USB drive, so the system falls back to the SD card as the rootfs. The system then loads the USB firmware to make the USB drive available.
In a previous article, we recompiled the Linux kernel to include the USB firmware to circumvent this issue. That is a fairly lengthy process that takes ~45 minutes. However, one of our readers (a special thanks to George!) pointed out a much easier way in a post they wrote on the Jetson Nano Developer Forum!
The idea is to include the USB firmware in the initrd so that the USB drive is available early on in the boot cycle. It takes less than a minute to build a new initrd with the change, and we’re off and running.
Note: In the video, we refer to the initramfs. As in most things technical, the initrd and initramfs are not strictly equivalent, but for the purposes of this discussion we treat them the same.
Install
You should do this on a freshly flashed Micro SD card. You can use a 16GB card for this process. In the video, we use a Samsung T5 500 GB USB SSD. On the JetsonHacksNano account on Github, there is a repository named rootOnUSB. Clone the repository, and then switch to the repositories directory:
$ git clone https://github.com/JetsonHacksNano/rootOnUSB
$ cd rootOnUSB
This is a 4 step process.
Step 1
Build the initramfs with USB support, so that USB is available early in the boot process. A convenience script named addUSBToInitramfs.sh provides this functionality.
$ ./addUSBToInitramfs.sh
Step 2
Prepare a USB drive (preferably USB 3.0, SSD, HDD, or SATA->USB) by formatting the disk as ext4 with a partition. In the video, we use the ‘Disks’ application. It is easier if you only plug in one USB drive during this procedure. When finished, the disk should show as /dev/sda1 or similar. Note: Make sure that the partition is ext4, as NTSF will appear to copy correctly but cause issues later on. Typically it is easiest to set the volume label for later use during this process.
Step 3
Copy the application area of the micro SD card to the USB drive. copyRootToUSB.sh copies the contents of the entire system micro SD card to the USB drive. Naturally, the USB drive storage should be larger than the micro SD card. Note: Make sure that the USB drive is mounted before running the script. In order to copyRootToUSB:
usage: ./copyRootToUSB.sh [OPTIONS]
-d | --directory Directory path to parent of kernel
-v | --volume_label Label of Volume to lookup
-p | --path Device Path to USB drive (e.g. /dev/sda1)
-h | --help This message
In the video, we:
$ ./copyRootToUSB.sh -p /dev/sda1
Remember to mount the USB drive before attempting to copy.
Step 4
Modify the /boot/extlinux/extlinux.conf file. An entry should be added to point to the new rootfs (typically this is /dev/sda1). There is a sample configuration file: sample-extlinux.conf
You should make a backup of the original extlinux.conf file. Also, when you edit the file you should make a backup of the original configuration and relabel the backup. This will allow you to access an alternate boot method from the serial console in case something goes sideways.
Then you should changed the INITRD line to:
INITRD /boot/initrd-xusb.img
So that the system uses the initramfs that we built that includes the USB firmware. Then set the root to the USB drive.
Here are some examples. You can set the drive by the UUID of the disk drive, the volume label of the drive, or the device path:
APPEND ${cbootargs} root=UUID=0e437280-bea0-42a2-967f-a240dd3075eb rootwait rootfstype=ext4
APPEND ${cbootargs} root=LABEL=JetsonNanoSSD500 rootwait rootfstype=ext4
APPEND ${cbootargs} root=/dev/sda1 rootwait rootfstype=ext4
The first entry is most specific, the last most generic. Note that you are not guaranteed that a USB device is enumerated in a certain order and will always have the same device path. That is, if you leave another USB drive plugged in along with your root disk when you boot the Jetson, the root disk may have a different path than originally, such as /dev/sdb1.
Also, there is a convenience file: diskUUID.sh which will determine the UUID of a given device. This is useful for example to determine the UUID of the USB drive. Note: If the UUID returned is not similar in length to the above example, then it is likely that the device is not formatted as ext4.
$ ./diskUUID.sh
While this defaults to sda1 (/dev/sda1), you can also determine other drive UUIDs. The /dev/ is assumed, use the -d flag. For example:
$ ./diskUUID.sh -d sdb1
You may find this information useful for setting up the extlinux.conf file
Reboot and Enjoy!
After editing the /boot/extlinux/extlinux.conf file, you are ready to reboot the system. Remember that the system boots from the micro SD card, so it must be present. The extlinux.conf that the system reads during the boot procedure is on the SD card. Even though there is an extlinux.conf file on the USB drive, the one on the SD card is the one that is read during the boot process. Modifying the one on the USB drive will have not effect.
Benchmarks
If you use the Micro SD card, you typically get average read rates ~ 87 MB/s and access times of ~ 0.52 msec. With a SSD, you get average read rates of ~367 MB/s and access times of 0.29 msec. About a 4X speedup! A HDD drive performs about the same as the Micro SSD, but tends to be much more reliable over time.
Special Thanks
Thank you George for pointing this out!
Notes
- Even though the Jetson Nano has 4 USB 3.0 connectors, they all go through one USB hub. Therefore, you end up sharing the USB bandwidth with all the other USB peripherals. Bandwidth can fill up quick!
- Mostly recently tested on Jetson Nano, L4T 32.2.1 [JetPack 4.2.2]
- In the video, tested on Jetson Nano, L4T 32.2.1 [JetPack 4.2.2]
98 Responses
In attempting to use the addUSBToInitramfs.sh script, on a fresh Ubuntu 18.04 minimal install, I’m experiencing the following:
Adding USB to initramfs
Warning: couldn’t identify filesystem type for fsck hook, ignoring.
I: The initramfs will attempt to resume from /dev/zram3
I: (UUID=3bfa4a08-9aa0-4f47-9811-a18035593b13)
I: Set the RESUME variable to override this.
/sbin/ldconfig.real: Warning: ignoring configuration file that cannot be opened: /etc/ld.so.conf.d/aarch64-linux-gnu_EGL.conf: No such file or directory
/sbin/ldconfig.real: Warning: ignoring configuration file that cannot be opened: /etc/ld.so.conf.d/aarch64-linux-gnu_GL.conf: No such file or directory
Which is strange since both of the files exist, although they were initially:
/etc/ld.so.conf.d/aarch64-linux-gnu-[GL|EGL].conf
So both files existed, but the script used an underscore rather than a hypen, so I ‘ln -s’ the two files swapping the _ for a – and they’re still not being found. Strange.
Any thoughts?
Thanks – Jeff
No thoughts, these are the same warnings that occur during the stock setup.
Ah… thanks. My first mistake was not watching your video first.
I’ve remedied that this afternoon.
Great work! Thank….!
What happens if you run a apt-get update, apt-get upgrade or apt-get dist-upgrade after doing this USB mod? Does the Micro SD Card get updated, the USB drive, both? Thanks.
The drive that is the rootfs is updated. Typically people do not dist-upgrade on the Jetson, as L4T is specific to the machine, and the upgrade from the Ubuntu repository is generic.
How would I get the OS from the USB drive to use the GPU drivers with acceleration?
Ideally I would like to use a completely different distribution like Alpine Linux. I was able to boot with Alpine’s mini rootfs which is really amazing. It only uses 10 MB!
I tried Nvidia’s official sample rootfs and was not able to get acceleration working. How can I fix this? I believe if I get it working on Ubuntu I can use that to get it working on other systems like Arch or Alpine.
Please ask this question on the official NVIDIA Jetson forum where a large number of developers and NVIDIA engineers share their experience. The forum is here: https://devtalk.nvidia.com/default/board/371/jetson-nano/
Thank you for this info. It worked out perfectly fine for me 🙂
You are welcome. I am glad you got it to work. Thanks for reading!
Sorry to disturb but i would like to ask if i boot from a external USB device, would i still be able to have access to the Jetson Nano’s GPIO ports?
Yes you have access to the GPIO pins. Thanks for reading!
Wow, that’s great. Thank you for the prompt reply!
You are welcome.
Hi, may i know if the 18.04 that is running on the SSD is a stock Ubuntu or ARM ubuntu?
I don’t understand the question. The OS is L4T, an Ubuntu 18.04 variant that runs on ARM.
Hi, I am currently working on a project that requires ubuntu 16.04 to run a SDK. I was wondering if it is possible to boot ubuntu 16.04 on the jetsonano using this method.
Thanks with regards
I don’t know how you run Ubuntu 16 on the Jetson Nano. Only 18.04 is supported.
Noted. Thanks for the assist!!
You are welcome, and thanks for reading!
Hi, I followed your instruction and succeeded in booting from usb ssd. Thanks a lot.
Just one question, I noticed that the sd card is still needed for booting. Can I clone the sd card as a backup and use the clone for booting in case the original is corrupted?
Sure, assuming that you know how to clone the SD card. Thanks for reading!
Hi, actually I can prepare a fresh sd card and configure it by repeating the above steps except step 3. No need to clone the old sd card. This way the new sd card can go thru the first boot procedures.
Hi, Why allocate 10 GB of free space?
You do not have to, but I usually do in case I want to add an extra partition later. This can be for a swap partition. However generally I just keep one around in case I want to easily wipe a partition, but have some things saved elsewhere. Thanks for reading!
Hi Jim,
thanks for this tutorial. I successfully managed to switch root to an usb ssd. But what I have noticed is that on my jetson nano still the usb ssd is displayed as mounted in the ubuntu application launcher and in the files application. In your video first, before switching the root to usb ssd, also your usb ssd was displayed there. And after successfully switching only the sdcard is displayed there and usb ssd disappeared. At my jetson nano now both sdcard and usb ssd are displayed. Do you have a clue why? I have to admit that I’m kinda linux novice.
My guess is that you did not update the extlinux.conf file correctly. You must change the extlinux.conf file on the SD card, the one on the USB drive is ignored during the boot process.
Hi Jim,
thanks for your reply. I figured out that R32.2.1 and R32.3.1 behave differently. I only have the issue with R32.3.1. Doing the same steps with R32.2.1 (as you did) works fine.
In regard to your guess, I did change the extlinux.conf, my home folder is now 920GB large, so it worked, it would say. But the SSD is in the new release also visible in /media, what it wasn’t in the old release.
Is it possible to boot the Jetson Nano via a Network. So network boot 5 at a time?
I don’t have any experience with this. Please ask this question on the official NVIDIA Jetson forum where a large number of developers and NVIDIA engineers share their experience. The forum is here: https://devtalk.nvidia.com/default/board/371/jetson-nano/
Hi all, a couple of questions:
I followed these instructions and all went well, I now have my Nano booting from a USB connected SSD. The thing is that when I plug another device (specifically an rplidar) into another USB port, it is recognized as TTYUSB0 which is already mapped to sda1, the SSD. Of course it’s rejected by the OS. Have I done something wrong or is there some special medicine I need to apply to override the overlap?
Also, is it safe to remove the SD card after booting from the SSD?
thanks,
jeff
Do you mean that the USB is plugged in, the Nano is running, and then you plug in the RPLidar it gets assigned to ttyUSB0?
Not sure why you want to remove the SD card. Did you try it to see what happens?
Yes, here is what I get, first without the rplidar, then with it plugged in….
jeffib@jeff-nano01:~$ lsusb
Bus 002 Device 002: ID 0bda:0411 Realtek Semiconductor Corp.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 007: ID 2109:0715 VIA Labs, Inc.
Bus 001 Device 005: ID 1c4f:0002 SiGma Micro Keyboard TRACER Gamma Ivory
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 002: ID 0bda:5411 Realtek Semiconductor Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
jeffib@jeff-nano01:~$ ls -l /dev | grep ttyUSB
jeffib@jeff-nano01:~$ lsusb
Bus 002 Device 002: ID 0bda:0411 Realtek Semiconductor Corp.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 8087:0a2b Intel Corp.
Bus 001 Device 007: ID 2109:0715 VIA Labs, Inc.
Bus 001 Device 010: ID 10c4:ea60 Cygnal Integrated Products, Inc. CP210x UART Bridge / myAVR mySmartUSB light
Bus 001 Device 005: ID 1c4f:0002 SiGma Micro Keyboard TRACER Gamma Ivory
Bus 001 Device 004: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 002: ID 0bda:5411 Realtek Semiconductor Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
jeffib@jeff-nano01:~$ ls -l /dev | grep ttyUSB
lrwxrwxrwx 1 root root 7 Jan 15 08:20 gps0 -> ttyUSB0
crw-rw—- 1 root dialout 188, 0 Jan 15 08:20 ttyUSB0
jeffib@jeff-nano01:~$
*************
I have not tried pulling the sd card but I’ll give it a try….
Thanks,
Jeff
I do not know what you are expecting here.
You appear to have several different USB devices plugged in. I don’t see an external USB hub.
A USB drive mounts as /dev/sd* it does not mount as ttyUSB*. Unless you are using a powered USB hub, I would be suspicious of not having enough power to drive the peripherals.
You would be better off looking at the dmesg log to see what happens when you plug in the lidar. You can use $dmesg –follow to watch.
I got it to work by setting the udev to rplidar and rebooting.
Thanks for your input!
Jeff
I have a new Nano running 4.3. I’m trying to boot off of an NVME M.2 drive using the M.2 interface. It there on boot, I can copy the OS to it, but get a lot of errors on reboot. Have not tried serial trouble shooting yet not have I recompiled the OS, so using the ./addUSBToInitramfs.sh method above. Unfortunately the system fails on reboot with a lot of reoccurring errors related to PCIe issues (haven’t stopped the display to be able to read them). So mostly I’m wondering if I have all the needed drivers? Do I need something different or should I rebuild the OS?
I do not have anything to share with regards to what you are trying to accomplish. As explained in the article, this is about adding a USB based HD/SSD. The script addUSBToInitramfs makes the USB firmware available to the boot loader. Good luck on your project!
kangalow, thank you… It looks similar to the USB3 issue, but needs the nvme driver… I added a line to the /etc/modules
nvme
And at least I think I am getting a clean boot from the SD card now.
For those interested, I think I have the nvme SSD working as a boot drive. Have only done it once…
Details on the Nvidia Dev site, but add nvme line to /etc/modules, then the addTo and copy scripts from here. Make the same modes to extlinux.conf.
I don’t know if the same “stunt” will work for the USB3 drivers or not (add to /etc/modules).
Thanks for the great tip, however I was facing another issue with the my external SSD has power issues, the solution was to add a line to disable usb power management
https://hamwaves.com/usb.autosuspend/en/index.html
$ sudo $EDITOR /etc/udev/rules.d/usb-power.rules
Thanks for the installation instructions – it worked smoothly!
But it’s a bit slower than expected: Avg Read Rate 95 MB/s. Avg Access Time 19ms.
I have formatted a 2TB Seagate Ultra Slim with a nominal speed of 500 MB/s, the nanoSSD partition is 1.9 TB. The formatting is Linux ext4
Two USB ports are occupied by Mouse + Keyboard. Nothing more.
Can you explain why it’s slower than expected? Does the partition size matter?
When I run ./addUSBToInitramfs.sh I get this:
Warning: couldn’t identify filesystem type for fsck hook, ignoring.
I: The initramfs will attempt to resume from /dev/zram3
I: (UUID=93fe9be3-a4e1-4cd5-95d4-cb903e551ddf)
I: Set the RESUME variable to override this.
/sbin/ldconfig.real: Warning: ignoring configuration file that cannot be opened: /etc/ld.so.conf.d/aarch64-linux-gnu_EGL.conf: No such file or directory
/sbin/ldconfig.real: Warning: ignoring configuration file that cannot be opened: /etc/ld.so.conf.d/aarch64-linux-gnu_GL.conf: No such file or directory
I have not found these warnings to effect the installation. Thanks for reading!
I do not have any experience with that particular drive. You may want to check via lsusb how to see if it is using USB superspeed. You could also check to see if using a powered USB 3.0 hub helps. Thanks for reading!
Thanks for the reply! The disk is confirmed usb3 – though I don’t know if I have connected it to the right USB port on the nano.
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=tegra-xusb/4p, 5000M
|__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/4p, 5000M
|__ Port 4: Dev 3, If 0, Class=Mass Storage, Driver=uas, 5000M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=tegra-xusb/5p, 480M
|__ Port 2: Dev 2, If 0, Class=Hub, Driver=hub/4p, 480M
|__ Port 1: Dev 3, If 0, Class=Hub, Driver=hub/3p, 480M
|__ Port 2: Dev 5, If 0, Class=Human Interface Device, Driver=usbhid, 1.5M
|__ Port 2: Dev 5, If 1, Class=Human Interface Device, Driver=usbhid, 1.5M
|__ Port 3: Dev 4, If 1, Class=Human Interface Device, Driver=usbhid, 12M
|__ Port 3: Dev 4, If 0, Class=Human Interface Device, Driver=usbhid, 12M
Bus 002 Device 003: ID 0bc2:ab2d Seagate RSS LLC
Bus 002 Device 002: ID 0bda:0411 Realtek Semiconductor Corp.
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 004: ID 062a:5918 Creative Labs
Bus 001 Device 005: ID 05ac:021d Apple, Inc. Aluminum Mini Keyboard (ANSI)
Bus 001 Device 003: ID 05ac:1005 Apple, Inc.
Bus 001 Device 002: ID 0bda:5411 Realtek Semiconductor Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 002 Device 003: ID 0bc2:ab2d Seagate RSS LLC
Couldn’t open device, some information will be missing
Device Descriptor:
bLength 18
bDescriptorType 1
bcdUSB 3.00
bDeviceClass 0 (Defined at Interface level)
bDeviceSubClass 0
bDeviceProtocol 0
bMaxPacketSize0 9
idVendor 0x0bc2 Seagate RSS LLC
idProduct 0xab2d
bcdDevice 1.00
iManufacturer 2
iProduct 3
iSerial 1
bNumConfigurations 1
Configuration Descriptor:
bLength 9
bDescriptorType 2
wTotalLength 121
bNumInterfaces 1
bConfigurationValue 1
iConfiguration 0
bmAttributes 0x80
(Bus Powered)
MaxPower 36mA
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 0
bNumEndpoints 2
bInterfaceClass 8 Mass Storage
bInterfaceSubClass 6 SCSI
bInterfaceProtocol 80 Bulk-Only
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 15
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x02 EP 2 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 15
Interface Descriptor:
bLength 9
bDescriptorType 4
bInterfaceNumber 0
bAlternateSetting 1
bNumEndpoints 4
bInterfaceClass 8 Mass Storage
bInterfaceSubClass 6 SCSI
bInterfaceProtocol 98
iInterface 0
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x81 EP 1 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 15
MaxStreams 32
Data-in pipe (0x03)
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x02 EP 2 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 15
MaxStreams 32
Data-out pipe (0x04)
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x83 EP 3 IN
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 15
MaxStreams 32
Status pipe (0x02)
Endpoint Descriptor:
bLength 7
bDescriptorType 5
bEndpointAddress 0x04 EP 4 OUT
bmAttributes 2
Transfer Type Bulk
Synch Type None
Usage Type Data
wMaxPacketSize 0x0400 1x 1024 bytes
bInterval 0
bMaxBurst 0
Command pipe (0x01)
I have run through this process four times and get it to boot onto mu USB SSD fine. However after about 15 to 30 minutes it locks up and a reboot generates hundreds of errors and complains that it cannot mount /dev/sda1.
I can sometimes get it to reboot but it is very unstable. Any ideas?
I have a Noctua fan and it is cooling the SSD as well and neither report higher than 45 degrees C.
How are you powering your Jetson and the SSD?
It reports failed to mount /media/f9….. mu UUID
and then hundreds of sd 0:0:0:0 rejecting I/O to offline device
I am using the barrel jack with the recommended 4 amp 5 volt power pack supply. Of note is that the SSD is an NVMe drive in a USB3.1 carrier. I suspect that this takes a little longer to power up, maybe I need some form of longer timer to let it properly come on line before it can be available as the boot drive?
I keep seeing this in the boot up:
fsck: error 2 (No such file or directory) while executing fsck.ext4 for /dev/sda1
mounting /dev on /root/dev failed: No such file or directory
Hello jim,
I have used this repository before and never experienced any problems. However today i decided to upgrade my jetson nano’s jetpack version to 4.3.
I noticed that after reflashing and then mounting to a usb drive (following above procedure) i am no longer able to use cuda, cuBLAS on my Jetson Nano. Do you know why this may be occuring ? Does this repo work with jetpack 4.3 ?
Would it be wise to do complete wipe(“ERASE”) to my SSD instead of just formatting the drive to ext4 ? Is it possible that old ll-lib files are somehow still on the SSD from my previous jetpack ?
Hello again,
I know you said to do this on a freshly flashed board.. I did not heed your wisdom and i installed the nvidia deepstream sdk and then transferred the OS to the USB.. it seems that was the cause of the issue.
Moving the OS to the usb and then installing deepstream will prevent any cuda related errors from occurring.
Maybe you could enlighten me alittle bit and tell me why installing a program and then moving to USB causes problems ?
I’m glad you got it to work. It’s hard to tell specifically why copying it over doesn’t work. I believe that when the software is created, there are many small gremlins that are set loose to make it harder for completing tasks such as you describe. Or it might just be that there are hard links here and there that don’t translate when they are transferred to another drive. Either way, that’s why it’s recommended to start with a fresh system.
I have run into problems when copying because certain things are bound to a particular UUID – and this is particularly true when using something like a file system image flashed with Etcher – every system has the same UUID. This can cause two different problems:
1. If you have two different copies of the same image, (say one on a SD card of one size and another on a different, larger, SD card and you plug them both in at the same time), because the UUID’s on the two different cards are identical, you get a UUID collision and things go crazy.
2. If you have a working SD card and decide to *copy* everything to a new, larger, SD card, this won’t work because the new SD card has a different UUID.
The way to deal with this is to make sure that either the clone has the same UUID if you’re going to boot with it, or to change the UUID in the boot configuration files.
Jetson Xavier NX Module (in Jetson Nano Dev Kit Carrier Board) with JetPack4.4: RootOnUSB worked with just one modification, to change the version of usb firmware used from tegra21x_xusb_firmware to tegra19x_xusb_firmware in the file data/usb-firmware
Thanks for sharing the original.
Thanks for sharing this, and thanks for reading!
So far so good! :>) Running on Eoan Ermine with USB NVMe 256GB!
I am glad you were able to get it to work. Thanks for sharing!
Hi, thanks for such nice tutorial (and scripts!). Works amazingly well. I have a question: Jetson Nano doesn’t boot if I remove Sdcard after this. Is it possible to run Nano without it now that all datas are on SSD ?
Thank you for the kind words. The SD card is needed to boot the machine. The Nano does not have an option for booting directly from USB. Thanks for reading.
They would have to create a raspberry pi noobs type installer for the jetson nano, it would make things much easier for users.
The device tree overlay seems to be having issues playing nice with the now modified /boot/extlinux/extlinux.conf. If you use jetson-io, the guide illustrates this point:
Creates a new DTB or Device Tree overlay (DTBO) for the list of functions provided. If the command creates a new DTB file, the Linux boot configuration file (/boot/extlinux/extlinux.conf) is updated with a new option to boot using this DTB.
Any idea how to make them play nice with each other?
You haven’t your issue, or which extlinux.conf file you modified. How did you modify your extlinux.conf file to accommodate the new DTB?
Actually I managed to solve the issue by adding an FDT entry with the custom DTB located on the sdmmc (the sd card) instead of the usb bootfs.
More details here:
https://forums.developer.nvidia.com/t/updating-the-dtb-while-running-on-usb-boot/127108/12
I am glad you got it to work.
Dear all I’m trying to use a 1TB drive but not using an sd card fresh install.
I’m able to perform all the steps mentioned in the video but when I reboot the nano does not boot from the ssd drive.
But on Step 3, before the copying process, I get the following errors:
—————-
Preparing to unpack …/libavahi-glib1_0.7-3.1ubuntu1.2_arm64.deb …
Unpacking libavahi-glib1:arm64 (0.7-3.1ubuntu1.2) over (0.7-3.1ubuntu1.2) …
Setting up nvidia-l4t-xusb-firmware (32.4.3-20200625213809) …
Starting xusb firmware post-install procedure.
ERROR. Procedure for xusb firmware update FAILED.
Cannot install package. Exiting…
dpkg: error processing package nvidia-l4t-xusb-firmware (–configure):
installed nvidia-l4t-xusb-firmware package post-installation script subprocess returned error exit status 1
dpkg: dependency problems prevent configuration of nvidia-l4t-initrd:
nvidia-l4t-initrd depends on nvidia-l4t-xusb-firmware (= 32.4.3-20200625213809); however:
Package nvidia-l4t-xusb-firmware is not configured yet.
dpkg: error processing package nvidia-l4t-initrd (–configure):
dependency problems – leaving unconfigured
Setting up libavahi-glib1:arm64 (0.7-3.1ubuntu1.2) …
No apport report written because the error message indicates its a followup error from a previous failure.
Processing triggers for libc-bin (2.27-3ubuntu1.2) …
Errors were encountered while processing:
nvidia-l4t-xusb-firmware
nvidia-l4t-initrd
E: Sub-process /usr/bin/dpkg returned an error code (1)
—————-
Thank you for your help !
Hello,
I ended up to solve my problem using a fresh installation. I spent a couple of hours rebuilding and setting up the system again but it works like a charm.
Thank you for the nice tutorial !
Have installed it three times now. Twice on a fresh microSD which comes up with the nvidia logo which just freezes, once on my first uSD a few days old for which the monitor stays black. Using a 120gb ssd with usb3 adapter cable. usb kb and usb mouse. 5.2V 2A to get past undervoltage problem when I first tried the Jetson Nano.
2A is barely enough to run just the module. You should try either a powered USB hub for the drive, or use the power plug for the Jetson. Thanks for reading!
After multiple failures referring to the ssd as per diskUUID.sh, it finally worked by using /dev/sda1. I doubt I will have any other drives in the 4 USB ports.
It does show the underpowered notification. I will try the 5W mode for now until I can order a heftier power supply.
Having issues with rootOnUSB for Jetson Xavier NX with Ubuntu 20.04.1 code focus. Although, I’ve changed the version of usb firmware from tegra21x_xusb_firmware to tegra19x_xusb_firmware in the file data/usb-firmware. After all steps is successfully applied, after reboot, then blinking cursor, nothing else. Using T300 expansion board from Geekworm and Samsung 250GB SSD. I’ve successfully performed this with Jetson Nano Ubuntu 20.04.1 on similar T300 and Samsung 250GB SSD.
So I’ve tried this with latest image that includes JP 4.4. Firstly updated, then followed instructions and ended up problems with initrd, cannot get pass this. Noticed that in extlinux there were a bit more information compared to the one shown in video (I was doing everything through the XRDP, maybe that’s the problem).
Now again back to reflashing sd card and trying again, but in general everything should work as intended or there were some changes in JP4.4 Nano image?
Regards
I’ve just tested out with JetPack 4.4, works fine with the same steps taken. And yes, that in extlinux there were a bit more information compared to the one shown in video. Just copy and paste the whole section.
Hi,
it works OK for me on two Jetson Nano’s , however my WiFi connection has stopped working on the two Nano’s. Any suggestions on how I could fix this?
Both Nano’s have Intel Wireless 8265 / 8275.
Help very much appreciated!
Hi folks,
The problem was with my browsers, which appeared to have some form of conflict with my WiFi. I deleted one browser and kept my favorite. Problem solved, sorry if I have caused any confusion.
rootFromUSB ‘is doing exactly what it says on the tin’
I’m successfully running Jetson Nano using T130 mSATA SSD shield using 128Gb ssd chip. The problem is my combo wireless Logitech keyboard and mouse M170/K235 became sluggish, sometimes just keyboard or mouse freeze. After waiting for some time they work again and some random time suddenly sluggish or freeze again. I powering my Jetson using a 5v/5A power supply through jack DC. Is my power supply couldn’t give enough power?
Have you set the nvpmodel to MAXN?
It works very well also with the new jetson Nano pack!
Thank you very much
I am glad you found it useful, and thanks for reading!
I got Jetson nano 2GB edition, and have tried this method.
I could complete from step 1 to step 4, however, it hangs while boot up.
Maybe I should check something…
The USB is actually a USB-NVMe converter with 64GB SSD. If not connected it, Jetson will boot from the SD card, and then the 64GB media will be automatically mounted if connected.
By the way, I’ve felt some difficulty with Ubuntu, because it can not use any external media that the capacity greater than 128GB. I’ve tried 256GB SD, 256GB USB, 3TB USB, not only ARM64, but also x86_64. I wish if Jetson has a SATA interface.
I have created a new SD card, and before doing anything, I have followed the instruction on the GitHub. Then, I could switch the root directory on the USB media.
Jetson nano 2GB works fine to me. Thanx!
When I have created a new SD card, something goes different. The desktop is the ordinary Ubuntu desktop! I think Nvidia had changed the image.
While running the script, I must ignore the error messages. I guess I did something wrong against the error message on the failure try.
Hiya, I hope you can help. I’ve tried this and on Step 3 I get this error :
Building dependency tree
Reading state information… Done
rsync is already the newest version (3.1.2-2.1ubuntu1.1).
0 to upgrade, 0 to newly install, 0 to remove and 143 not to upgrade.
rsync: mkdir “/media\#012/media/matt/NanoSSD” failed: No such file or directory (2)
rsync error: error in file IO (code 11) at main.c(675) [Receiver=3.1.2]
I’ve reformatted the drive, using Disks. Its’s a USB HDD, not SSD, i’m sure that should not matter ?
Thanks…
Matt
The \#012 indicates that there is a bad character in your path description. It is unlikely that the path would be /media/media/matt … Hope this helps.
hey i had eject the ssd from ubuntu to make it work. I had the same error as you, and once i ejected it the copy started to work.
If you try to copy the root files using the “-p” option:
./copyRootToUSB.sh -p /dev/sda1
it will fail. rsync will not be able to make the directories and copy the files.
rsync: mkdir “/media\#012/media/tom/NanoSSD128GB” failed: No such file or directory (2)
rsync error: error in file IO (code 11) at main.c(675) [Receiver=3.1.2]
However using the “-v” option and declaring the volume partition you want to copy the files to does work! example:
./copyRootToUSB.sh -v NanoSSD128GB
Thank you for the info, and thanks for reading!
I think this method is broken with JetPack 4.5, I ended up with a boot loop. JetPack 4.5 now natively supports boot from USB but you first have to once install it from the SD card image and then you can move it over to the USB drive. However, if find the documentation on the Nvidia site quite confusing and I’m wondering if this method can updated to include support for JetPack 4.5?
Now I know why this didn’t work on JetPack 4.5 ! I will go find the Nvidia docs
We can have a tutorial clear as this one about boot from USB on JetPack 4.5?
The A02 board has an issue with the XHCI driver not detecting USB drives (SSD/HDD) at the first attempt in U-Boot. A restart of the USB driver in U-Boot solves the issue. NVIDIA is investigating this for JP45.
Hi I have been following this instruction four times but after reboot I got:
Gave up waiting for root file system device…
….
ALERT! UID=05c16421-b543…. does not exist. Dropping to shell!
In my initramfs I have specified ‘rootwait’ and copied the result from ./diskUUID.sh (exacltly like in the movie). I am using Jetson Nano Developer Kit 2 GB – not the 4GB version. Whole operation I did on fresh Ubuntu image created from NVIDIA site.
Don’t know how to progress and where is the problem.
Thanks in advance
Slawomir
What version of L4T/JetPack are you using? This is for JetPack 4.4/L4T 32.4 and will not work with the most recent JetPack 4.5/L4T 32.5 +.
Thanks for the reply.
I downloaded latest SD card image for A02 board (2GB JETSON dev kit). I have chcked on Nvidia site here and there is no JetPack 4.4 available for 2GB dev kit (A02).
https://developer.nvidia.com/jetpack-sdk-44-archive
I presume it is for 4GB board. In case of JetPack 4.5 it is available for both versions:
https://developer.nvidia.com/embedded/jetpack
Ah sorry. I have found JetPack 4.4.1 L4T for 2gb dev kit here
https://developer.nvidia.com/jetpack-sdk-441-archive
I will try tho one. Thanks for help
Anyone an idea as too how to upgrade to a newer kernel version? Do I need to boot into the sd-card, upgrade there and build
OK mine shows the NVIDIA logo then goes black then back to the NVIDIA logo..
How to make it stop going to the NVIDIA logo then back black and then back to the NVIDIA logo? I don’t understand how I can put back the original config file back if the NVIDIA is stuck in a boot loop?!
I don’t know enough about your system configuration to help. Which version of JetPack are you using? Without the serial console output it’s difficult to tell why it is in the boot loop. You are probably best off flashing a new SD card to run the system. On newer JetPacks, you can use the NVIDIA SDK Manager to setup the Jetson to run from the USB drive. This article was written before that capability was added.