Ten plain answers about the AI behind tools like ChatGPT and Claude.
Â
1. What is a large language model, and what does GPT stand for?
A large language model is a program trained on a huge amount of text to predict what comes next. "Large" refers to its size, a vast number of internal settings tuned as it learned. GPT stands for Generative Pre-trained Transformer. It's generative because it writes new text rather than looking up stored answers. It's pre-trained because it first learned general language from vast reading, before it was trained to be a helpful assistant (question 8). And the transformer is the design it's built on, which weighs how the words relate to each other to work out which ones matter.
2. How does it come up with an answer?
It learned the patterns in a vast amount of text, broken into small pieces called tokens, which are often parts of words. It writes by predicting a likely next piece, again and again. Think of someone who has read a huge share of what we've written, but learned all of it from text, not from living in the world.
3. Why does AI sometimes make things up?
Because it's built to produce likely text, not checked facts. When it doesn't know something, the most likely-sounding answer can still be wrong, and it sounds just as sure either way. Check anything that matters.
4. Why do I get a different answer to the same question?
Most software is deterministic: the same input always gives the same output, like a calculator, where 2 + 2 is always 4. Generative AI isn't. It picks among likely next words with some randomness built in, so the same question can come back worded differently, or now and then with a different answer.
5. How do I ask for what I want?
Treat it like a capable rookie on day one: it has the skills but none of your context. Give the instructions first, then the context. Say what's in and out of scope, who it's for, and the format you want, and show an example. Vague requests get vague answers. Clear context in, useful work out. If it names a source, check that source yourself.
6. Does it remember what I tell it?
Within one chat, yes, up to a limit. The whole conversation sits in its working memory, called the context window, which is separate from what it learned in training. A very long chat can outgrow that space, and then it may lose track of early details, or you may need to start a new chat. It doesn't learn from your chat as you go. Some assistants, Claude among them, also have a memory feature that carries notes about you into later chats. You can see what it keeps, change it, or turn it off in settings.
7. How does AI learn in the first place?
It makes a prediction, scores how wrong it was, and adjusts itself to do better next time, over and over. In older machine learning, people chose which details mattered. In deep learning, the model finds them itself. If it learns its examples too exactly, it fails on anything new, like a student who memorized last year's test answers. A useful model learns the general pattern instead.
8. How was it taught to be helpful, not just to predict text?
First, people wrote example conversations, and the model trained on those. Next, people ranked several of its answers from best to worst, and those rankings trained a second program to score answers the way people would. Then the AI was tuned to earn better scores, and those last two steps were repeated. The aim is answers that are helpful, truthful and harmless.
9. Can AI understand pictures?
Yes. Vision models learn from huge sets of images, many paired with text that describes them. Much as our own vision does, they start with edges and shapes and build up to whole objects. They can say what's in a picture, find where something is, and follow it across video frames. Today's assistants combine pictures with language, so you can show one a photo and ask about it.
10. Can anyone explain why AI gave a particular answer?
Not fully. Even the builders can't always say what patterns a model found, or whether it has blind spots. That's the "black box". You learn its habits by testing it. Researchers have begun to find ideas inside large working models, and to show that changing them changes what the model does.
Want to go deeper?
These answers draw on my LinkedIn series, On the Origin of AI Markets:
Davinci, Dynamite, Generative AI & Prompt Engineering
AI Vision, Neural Networks & Multi-Modal Models
AI Availability (Pt. 1): Data Wrangling