I feel like we made some good progress last week. The jtop: The Ultimate Tool for Monitoring NVIDIA Jetson Devices article and video is the first “how to” article of the year. Baby steps to get back in the game!
The Idea Shelf is a place to keep some of the great ideas that I’ve run across on the web. The first entry is Bret Victor – WorryDream is a website which feels like trying to drink water out of a fire hose. There are some great ideas there! Definitely worth taking the time to explore.
The website adds a place to house the older newsletters that we have sent out. This area will usually have all of the newsletters except for the latest one. To get the latest one directly into your inbox, subscribe to the Jetson Hacks Newsletter. Newsletters are sent out once or twice a month. I wrote a new one, and put it on the shelf just this time. There are now a couple newsletters in the archive:
Two Steps Forward, One Back
We have been making good progress the last couple of weeks. Articles, blog posts, ideas, videos. Cleaning up the website means it’s much easier to organize. Everything in it’s place, as it were. Website traffic up 10%. Life be good.
I think a good next step is to fix up some of the videos that have gotten long in the tooth. One subject is running a Jetson from a SSD. On the surface, that would seem easy …
As a “Be bold” strategy, the plan is to upgrade a Jetson Xavier NX to JetsonPack 5.1 and boot it from a NVMe. Article and Video. Since the new Jetson Orin NX and Nano support only this method, me figures that’s the way to go.
I fire up the laptop, and place the Xavier NX into recovery mode. The NVIDIA SDK Manager tells me that I need to upgrade. That seems about right. I go through the upgrade process of version 1.8.4 to 1.9.1 (let’s say I’m not a fan), and eagerly await the brave, new world. Here it is:
No SDKs for you!
Let’s say I have a love/hate relationship with the SDK Manager. I love to hate it. OK, start again.
First apt purge the sdkmanager. Second, remove the hidden folder ~.nvsdkm. Let’s see if 1.9.0 works. Install that from the .deb file. Nope. OK. Look through the logs. There seems to be trouble with downloading files. Each file prefix starts with https:/ instead of https:// in the log. That has to be a logging error, right? The prefix error is part of the logging, I’m guessing. If it was the actual file name, then nothing would ever work. That would be an obvious issue. Certainly I would not be the first to find it.
Trying the Tricks
There’s always lots of tricks to try. Some are superstitions. Rebooting between installs seems to help a good amount of the time. Getting rid of caches, going through log files. Checking the /etc/hosts file. All sorts of tricks. In this case, I couldn’t get it to work.
A little more back story. I remember now that on the original 1.8.4 install SDK Manager indicated that a newer version is available. When I tried to update, it said it could not find it. So I installed 1.9.1 manually. Ain’t my first rodeo with SDK Manager/Jet Pack Installer.
No Bueno
Here in Southern California, there’s the slang term “No Bueno” meaning no good. Realizing that my new dream of upgrading to 1.9.1 was at stake, I reinstalled 1.8.4.
Works perfectly, of course. The world isn’t stopping completely. The 1.8.4 SDK Manager tells me that there is a new version available. 1.9.1. That seem like a good idea. I hit the upgrade button, and this time everything goes smoothly. 1.9.1 installs itself. I’m pretty happy at this point, everything is as it should be. The SDK Manager relaunches:
Maybe It’s Me
I’m telling myself that February 2nd is Groundhog Day, if you have seen the movie. But I appear to be stuck in some sort of horrible upgrade loop here. While this little story only takes a few minutes to read, rest assured that this has been going on for several hours. I may have been swearing a little. It ain’t over yet.
Here’s the thing. Everyday I read about these great break through in computer hardware and software. These great machines that will change our everyday lives, and maybe even replace us! I will say that they have changed our lives, we spend more time doing updates than anything else. Plus, the updates don’t seem to have any benefit.
Sure, most machines work great for a little while. Then there’s this software “update” cycle which is the death spiral of any given machine. I have a Sonos speaker which bricked itself during an update, never to be heard from again.
Another thing to note is that this is typical in any development cycle. My experience is that people use 20 percent of development time fixing things that have no right to be broken. If you don’t plan this into your project, then it’s not realistic.
Sigh.
4 Responses
Sorry to hear of your pain dealing with the SDK, etc. At least I feel that I am not alone. Im dissapointed in Nvidia’s support. So very little of the Nvidia tech works the very first time (or mulitiple times thereafter) you try it.
I hope you dont give up like I have. Looking forward to your eventual solution.
Thanks
My experience with these types of things is that it’s par for the course. Just have to work through it. When you have a relatively small group trying to support a multitude of products, it’s difficult to test against. Usually it’s something small and silly. Just takes a while to find it. In part that’s why it’s useful for people to share their experiences. With that said, the SDK Manager/JetPack installer has always had issues. The security breeches a few months ago made thing more difficult, and meant that less time went to product development than normal. Thanks for reading!
I hate the SDK manager to the bottom of my heart. It is an opaque piece of bloatware that makes life difficult but it is the only way I have found to “get things working” or update U-boot etc.
Nvidia is guilty of having c**p software and excellent hardware. I include the L4T/MMAPI sample software in this claim which is written as if by an 18 year old school student: “what’s object oriented programming?” etc etc. Also, I have spent so long on the forums investigating issues that apparently even the experts aren’t sure how to solve.
I had a similar experience working with TI SoCs. Raspberry Pi is the only hardware company that comes to mind that also has excellent software and support. Glad to hear I’m not the only one who suffers! Rant over.
I feel that there are two pulls here. First, NVIDIA wants to be “open source”, providing a Linux experience on their Jetsons. They want to be able to leverage that. Second, they have a huge codebase which is closed (CUDA and so on) which has so much baggage behind it that it’s not realistic to release it as open source. The CUDA software stack is across all NVIDIA products (graphics cards, Jetsons, etc), so they would have to rewrite the entire world to release it. A lot of the issues seem to fall “between the cracks”. Linux support is difficult at best, the fast pace of the NVIDIA software stack changes exacerbates the problem.
The software tools for managing Jetson images underneath the SDK Manager seems relatively OK. The GUI of SDK Manager attempts to tie them together. Unfortunately this is all tied into credentialing in the developer program. When they had security issues a few months ago, things really went sideways. Unfortunately SDK Manager is part of so many Tegra based products now, there’s no quick fix. Thanks for reading!