The JetPack 4.4 Over The Air (OTA) updates issue that some folks encountered on on the first couple of days of the release have been fixed! Life be good again.
That gives us marching orders. The end goal will be to upgrade our systems to the new version. I don’t recall really having a good article about upgrading Jetson articles here on JetsonHacks, so I think it’s about time to make an effort to do so. We’ll share some tips and tricks from how some people work through this process.
There are several things that are apparent from watching peoples reactions to the recent update/upgrade issues. One can certainly tell the professional developers from the normal people.
Developers
Most professional developers are very wary of system updates/upgrades (there’s a certain amount of dread in updating a system). Part of that it is human nature. The trade off is that your are on a system which you know is flawed, but if you have been working on it a while you know where the flaws are hiding. An update/upgrade may fix the flaws, but may require a significant amount of work on the developers part to take advantage of the new fixes, features and so on.
On the other hand, the developer may have no interest in the flaws that are being addressed in the update. However, the updates and upgrade may affect parts of the system that will require work on the developers part for integration. For example, an update to a system library (like CUDA, let’s say) means that any programs that are compiled against an earlier version need to be rebuilt for the new version. New features are great and everything, but if the developer doesn’t use them in their project it’s just extra work.
Also, each library version update seems to have its own little niggles for it to build and work properly with other libraries. Niggling can be time consuming.
The experienced developer knows at any time things can break, and out of self defense tend to have strategies for minimizing the amount of down time in getting things back in running order.
Normal People
Most people have a different expectation of updates and upgrades than a developer. The basic idea of updating and upgrading is that the system will become better, increase performance and improve capabilities.
This is true as most updates don’t break the system, and the system is better afterwards. However, this is also a “Rainbows and Unicorn” view of the world. People can benefit from managing their expectations about what updates and upgrades can bring, and take some precautions to protect themselves in case things go south.
Backups
Backups are two things. Mind numbingly boring, and absolute sheer terror. Boring in that there is really no sense of pleasure when they are working correctly, and absolute terror when you have to actually depend on one to work.
Because developers are frequently dealing at a system level, there is always the possibility that they can break the system (perhaps irreparably). Therefore, they tend to have various backup strategies. This usually includes some sort of source version control for their code base, backups for their data, full system backups and so on.
Out of self defense, the developer generally has a method to regenerate a new system, typically with scripts to automate the build process. When a new major version of an OS comes out they will build a new system from scratch. They keep the previous version, along with their development environment, and build a new version. That way, if there are issues with the new version they can go back to the old version and check for differences. Extra dependencies always seem to sneak in but don’t get tracked correctly, forcing a rebuild from scratch exposes these.
Developers know that if you can’t build your system from scratch, you don’t have a system. Instead, you have a ball of pain that is guaranteed to bite you at the worst time.
Most normal people don’t think in these terms. However they do know from possibly unpleasant experiences that they need to keep a backup of some sort, or at the very least backup data that they feel is important.
Marching Orders – Backup & Upgrade
That sets up our next couple of articles. “Backup for Jetson” and “Upgrading to JetPack 4.4 via OTA updates”, or the same articles with even more clever names. Look for those, coming to a web page near you soon.
We can benefit from talking about how to go about backing up our system, and various ways of preparing for what we will call ‘something bad happening’.
Once we have our backups ready, then we will be able to upgrade our system.
Upgrading is straightforward, especially after all the fretting we’ve been doing about backups and doomsday scenarios. There is another case we will talk about in the upgrade.
In an earlier article, Jetson Xavier NX – Run from SSD , we setup our system to run on a NVMe SSD once the system boots. We will need to take an extra step when we upgrade our system to accommodate this.
Never ending fun!
6 Responses
Thanks for doing this,we need it. I, unfortunately jumped on the OTA wagon right away, and saw the incessant blinking cursor. I built the SD card and booted and all was good until i realized that my SSD was no longer working, but your previous instructions indicated future upgrades should run a script, which did the job brilliantly, and I was happily up and running right where i left off, but with the new Jetpack on board. Keep up the good work. I cant imagine how many cumulative man-hours of work it has saved us all.
Thank you for the kind words. The OTA wagon is very seductive, bright and shiny and new and all. Thanks for reading!
Rick,
What previous instructions/script did you follow to get your system back from the dead (or the 6 time Ctrl-Alt-Del to revive state).
Thanks!
I think my comment might have been a little misleading. Here is what i did step by step:
1. after the ota and reboot my system would only show the blinking cursor as jim described.
2. i just went ahead and imaged an sd card with this new release from the nvidia download area. This booted normally and my system was running the latest updates.
3. the script i mentioned is from Jims tutorial on booting from a Solid State Drive. Although my system was running, it wasnt booting from my SSD. I went back to the tutorial and saw a line that mentioned future updates would have to rerun a script that would copy the boot info from the SD card to the SSD so that they would be in sync again, and it also configured startup on the SD card to use the SSD as the boot drive. Once i did that, i had my upgraded system, and my SSD booted, and all of my data was back in sync.
As i understand it now, the OTA is fixed,but if you have the blinking cursor, you will have to image an sd and boot from it.
I upgraded the Jetson Nano to Jetpack 4.4 last night (7/15/2020) using the Nvidia SDK Manager version 1.2 from a VMware Ubuntu 18.04 installation on Windows 10 connected to the Jetson via micro usb. It completed successfully without problems, though it took awhile. The instructions in the Nvidia SDK Manager user guide were straight forward. Only problem for me was in finding the “force recovery mode” pins to jump on the B02 developer kit. They are on the backside of the board by the heatsink and labeled on the bottom of the board for anyone else who didn’t know.
After reading all of the comments on this subject, I am going to stay with my old, dependable 4.3 until further notice. However, if you do need any additional testing (NX) I would be glad to give it a go. Thanks for all the help.