Jon Stokes explores how developers will build apps on top of ChatGPT and similar models.

I often encounter confusion about the basic mechanics of how these models can be used in software, so this is my attempt to help clear this up for everyone who’s looking at this space and trying to understand how they can adapt to it.

Most people imagine, for instance, that in order to put GPT-4 to work on their own data, they’ll need to train a version of it on their proprietary data. So in their mind, the process will work something like this:

All the model’s training data + my proprietary data → training → inference → 🤑

You could do all of the above, but it would be extremely expensive. It’s also not even remotely necessary.

An alternate approach would be to fine-tune a pre-trained model so that it knows about a much smaller corpus. Recently, this fine-tuning approach was the best way to build apps like the above. OpenAI offers fine-tuning capabilities via their API, in fact. But of course, you need someone on your team who knows how to do this, and for other reasons, it’s not necessarily ideal for many use cases.

💬 But there’s another way that was sort of working with GPT-3.5, and as of today’s GPT-4 announcement should really work quite well. I’ve taken to calling it the CHAT stack:

Context
History
API
Token window

Read More at Jon Stokes

Read the rest at Jon Stokes