JetsonHacks

Developing on NVIDIA® Jetson™ for AI on the Edge

ChatGPT – 4 Useful Insights

There has been a lot of hype around ChatGPT over the last several months. Deservedly so. Here’s some thoughts on how to use it. Looky here:

Introduction

When something like ChatGPT explodes onto the scene, there’s a great divide between what people with a technical background see and everyone else. With ChatGPT, that’s quite a big chasm.

At its core, ChatGPT is a Large Language Model (LLM). It is a Generative, Pre-trained, Transformer (GPT). The OpenAI GPT is trained on a corpus which consists of most of the open Internet, circa 2021. The model is supplemented with Reinforcement Learning from Human Feedback (RHLF). The ‘Chat’ part of ChatGPT refers to an actual application which interacts with the model, a chatbot. A chatbot takes a prompt from the user, and in this case returns a response inferenced from GPT.

When you talk to most people, they believe that you ask ChatGPT a question and it responds with an answer. Typically they believe that there is some type of data store where answers are stored, and the machine is looking up the information.

However, it’s doing something different than that. GPT takes a prompt, breaks it down into tokens, and converts this to a vector of numbers. Based on all of the data that it has seen (remember, it’s the whole internet), it then tries to predict what the next tokens would be. It does this over and over using computational statistics to produce the response.

Check the Answer!

Once the model has been initially trained, people ask it questions and ‘grade’ the answers, RHLF. The trainers then adjust the parameters on the model to take this information into account. But here’s the thing.

By its nature, a GPT is building an answer that “looks” correct. If you request an answer with references, it will provide what looks like an answer with what appears to be references. It might cite an author prominent in the field, a reasonable title, and an associated journal. And it might even be right! However, it’s goal is to produce an answer that looks correct, not one that is correct.

There in lies the rub. People assume that computers are authoritarian in the sense that they will always give a correct answer. In contrast a GPT is non-deterministic. It can give different answers to the same input. Sure, there are some algorithms that are non-deterministic, such as Monte Carlo simulations. For the most part of the last 70 years, computers compute and give an answer based on the input deterministically. Same output on the same input. Not so with neural networks.

More Human than Human

In that way, a GPT is like a person. You get different answers from the same person all the time. You know that some people lie or have a different viewpoint which generates answers that you might consider wrong. With these early models, you have to kind of assume that you’re talking to a liar that lies.

With that said, that doesn’t mean a GPT isn’t useful. Quite the contrary. For the most part, it handles a lot of tasks that are worth the price of admission. Translation between languages, like English to French. Converting lists to tables, adding rows and columns to tables. Making outlines of tasks, the list goes on. Researchers have identified over 100 emergent behaviors that GPT exhibits which no one expected.

But the deal is that you have to check its work. You have to know the answer when you see it, and call it on nonsense. People do that all the time with other people of course. It’s just that we’ve seen the computer be an authoritative voice for so long, it will take a lot of time to work through the new world order.

Does this change everything?

With all the hype and noise around ChatGPT, it’s hard to get a reading on how important this technology is going to be in the near future. You see people using it to generate blog posts and video scripts, for example. There is going to be a lot of short term arbitrage opportunities for tasks like that. However, in the long run there’s no value add to it. Simply rewriting or regurgitating previous internet articles doesn’t provide any benefit. We’ll see people adjust to that.

At the same time, adding ChatGPT type capabilities to a browser breaks the whole search paradigm. Use the new ChatGPT extensions on the Microsoft Edge browser and see how many times you have to use a search engine after that. The thing that fuels search engines? Ad revenue.

With ad revenue drying up, content creators will be faced with having to monetize their content differently. It’s not clear what will be the winning mechanism, but some type of subscription or paywall mechanism seems likely. We’ve seen this with Substack, Locals and Medium already.

This is certainly technology worth checking out.

Facebook
Twitter
LinkedIn
Reddit
Email
Print

7 Responses

  1. What blows my mind is that those models can actually generate computer code that is correct and appropriate! Now THAT’s pure (artificial) magic!

    1. On a lot of common tasks, it can do some pretty amazing stuff. Once you get into more specialty code, where it doesn’t have as large of a corpus to train on, it seems to get confused pretty easily. Thanks for watching!

  2. Great article. The new thing being introduced by chatGPT is definitely nuanced. It’s true it gets a lot of things wrong (and will apologize immediately when called out). But I still think it has strong potential as a tutor. Being able to ask any question at any time has got to speed up the learning process. I’m using it that way for now and learning new things faster than I used to..

    I’m also exploring the API, which is a powerful tool! This is all great timing with the arrival of the Jetson Orin opening up a door to deeper explorations.

    1. Thank you for the kind words. To me, it’s obvious that this changes how we use computers, and the whole financial ecosystem around being online. Online advertising as we know it goes away for the most part. How do you monetize search engines, or are they even stand alone sites anymore? When that goes away, then how do people creating content get compensated? Either they have to get direct sponsorship, or go behind a paywall. The whole game changes. The ramifications are far reaching, not that it’s a bad thing per se.

      There’s a little bit of a quandary here. Right up front ChatGPT says that it’s a research preview. Many people want to use it to learn, but what are you learning if you don’t know if it’s true? Put aside the inherent bias from any training set for the moment. You kinda gotta know what the answer really is, or recognize it when you see it. It’s a really interesting problem. They’re trying to mitigate it by crowd sourcing ratings (the people who are currently using ChatGPT) to fine tune the parameters of the underlying model(s). How well that works remains to be seen, but it “seems” useful even at this early stage.

      With that said, the less amazing parts can provide a huge amount of work on their own. For example, let’s say you have a lot of Python code. It’s uncommented of course, like every good programmer creates. It may not have unit tests. By using the API, you can tell GPT to add docstrings to each function, and do a design document for each module. Build unit tests, then test them, then rewrite them until they pass. All rather painlessly, the programmer just needs to review it for the most part when using GPT 4.

      Over the coming months, I’ll be including this in the workflow of the videos and coding. This video and article are a bookmark to the starting point for when we start doing some interesting things that normally are just annoying or busywork.

    2. I believe that “apologizing” is a psychological trick that the system uses to buy more time to generate the next answer. Even if it generates the correct answer and you call it out, it “apologizes”. It takes some time to generate the tokens from the inference. You have the canned “oh, I’m so very, very sorry” which it spaces out in the pseudo typing of the response. It uses that extra time to attempt a better, deeper match. Of course, it can’t be the least bit sorry. The system implementors know that people will anthropomorphize the system to think it’s empathetic with them.

  3. It seems there is an emerging rivalry between chatGPT and Claude, a new AI assistant developed by Anthropic. The differences between these two models are interesting – see https://scale.com/blog/chatgpt-vs-claude for details. And you are correct that the financial implications for Google and Microsoft are significant, this is a sea-change moment in the evolution of tools.

    1. Claude looks interesting. As the money starts flowing into GPTs, we would expect to see a large number of competitors. OpenAI has early mover advantage.

      It’s the perfect storm for Microsoft. Bing has been lagging way behind. They invest in OpenAI, add ChatGPT to Bing, and they are not only back in the game, but causing competitors a major headache.

      Same perfect storm for Google, but the other way around.

Leave a Reply

Your email address will not be published. Required fields are marked *

Disclaimer

Some links here are affiliate links. If you purchase through these links I will receive a small commission at no additional cost to you. As an Amazon Associate, I earn from qualifying purchases.

Books, Ideas & Other Curiosities