AI is just guessing.
I got asked last week whether the AI was thinking. The person told me they use ChatGPT every single day, she is sharp, and she was a little embarrassed to ask. She shouldn’t have been. Here is the short version: AI like ChatGPT and Claude works by predicting the most likely next word, one word at a time, based on patterns it learned from an enormous amount of text, not by looking anything up or reasoning the way a person does. Most people who use these tools all day have no mental model of what is happening on the other side of that text box, and it costs them. It shows up as frustration when the output is wrong, and as a quiet fear that they are the only one who does not get it.
Here is the thing I keep telling people: you do not need to know how an engine is built to drive a car, but knowing a little bit changes how you drive. I manage an AI database product management team at Oracle. I sit in the rooms where these systems get evaluated for real enterprise use, so this is not a review from the sidelines. It is a field guide from inside the building. And once you understand what is happening under the hood, you stop being surprised when the tool makes something up, and you start writing better prompts without even trying.
So instead of spending the next two hours scrolling social media, give me fifteen minutes. Five questions, no engineering degree required.
- Q01: What is really happening when I type something and hit enter?
- Q02: If it is just guessing, why is it any good?
- Q03: What is a token, and why do I keep running out of room?
- Q04: Why does it sound so human one minute and make things up the next?
- Q05: If I remember one thing, what makes AI work in the real world?
Q01: What is really happening when I type something and hit enter?
Here is the honest answer, and it surprises people every time: the AI is predicting the next word. That is the whole engine. You type a message, and it looks at everything you wrote, then it guesses the most likely next word. Then it looks at your message plus that new word, and guesses the word after that. It does this over and over, one piece at a time, until it has a full response.
That is why the text streams onto your screen one chunk at a time instead of appearing all at once. You are watching it build the answer word by word, the same way you might if I stopped you mid-sentence and asked you to fill in what comes next.
The mental model that helped me most: it is the autocomplete on your phone, taken to a scale that is hard to picture. When you text “running a little” and your phone offers “late,” that is a tiny version of the same trick. Now imagine that trained on close to everything ever written on the internet, and you have the general idea. Nothing is being looked up in a database, and nothing is being reasoned through the way you would reason. It is a very, very good guess about what word should come next.
Sit with that for a second, because it explains almost everything else the tool does, including the parts that annoy you.
Q02: If it is just guessing, why is it any good?
Because of what it read before you ever showed up. Before any of us typed a word into ChatGPT or Claude, the model went through a phase the researchers call pre-training. In plain terms, it was fed a staggering amount of text, close to the whole public internet, and its only job was to get good at predicting the next word across all of it.
When you practice next-word prediction on that much writing, something interesting falls out of it. To guess the next word in a sentence about a lease agreement, the model has to pick up on how contracts tend to read. To finish a sentence about the mouse and the cheese, it has to absorb that mice eat cheese and not the other way around. It never got taught grammar or facts directly. It learned the patterns because guessing well required them. That is why the output is useful instead of random.
This is also the part that gives people a false sense of what they are dealing with. It feels like the tool knows things. What it has is an extraordinary sense of what is likely to be said. Most of the time those overlap, which is why it works. When they do not overlap is when you get burned, and that is Q04.
Q03: What is a token, and why do I keep running out of room?
You have probably hit a moment where the AI told you the conversation was too long, or it seemed to forget something you said earlier. Tokens are why. Before the model reads your text, it chops it into small pieces called tokens. A token is not a word. It is closer to three or four letters, so a word like “tokenizer” might get split into “token” and “izer,” and a short common word is often one token on its own.
Why break words apart like that? Two reasons, and they are practical. If the model only knew whole words, a single typo would hand it something it had never seen and it would freeze. And going the other direction, letter by letter, would make every sentence enormously long and slow. Tokens are the compromise in the middle. Common chunks get their own token, and everything else gets built from those pieces.
Here is why you care, in real terms. Every model has a limit on how many tokens it can hold in mind at once, and that is what “context limit” means. A long PDF, a giant email thread, a transcript, all of it eats tokens fast. You do not need to count them. You just need to know the word exists, so that when the tool starts forgetting the top of a long document, you understand it is not broken. It ran out of room. Start a fresh chat, or feed it the document in smaller pieces.
Q04: Why does it sound so human one minute and make things up the next?
This one confused me until I understood the two phases these models go through. Pre-training, from Q02, gives you a model that is brilliant at predicting internet text, but internet text is not an assistant. It is comment sections, arguments, half-finished thoughts, and the model will happily continue any of it. So there is a second phase, post-training, where the model gets shaped into something that answers your question, stays polite, and behaves like a helpful colleague. Post-training is the difference between the raw engine and the friendly thing you talk to. It is the reason the ChatGPT moment felt like magic when the rest of us had been ignoring language models for years.
Now the making-things-up part. Remember, at its core the tool is producing the most plausible next words, not looking up a verified fact. Most of the time plausible and true line up. But when you ask about something obscure, or something that happened after its training cutoff, or a specific number, it will still produce a confident, fluent, wrong answer, because a confident fluent answer is what plausible text looks like. It is not lying to you. It has no mechanism to know it does not know. That is why I treat every output like a first draft from a talented new hire: I read it critically, I check anything that matters, and I never paste a number into a deck without verifying it myself.
One more thing that belongs here, because it comes up on my team constantly. The consumer version of these tools does not know your company’s private data, and you should keep it that way. Do not paste confidential documents into a public chatbot. At Oracle I use an enterprise-licensed AI subscription for anything sensitive, which is a different thing entirely from the free consumer app. Know which one you are in before you hit paste.
Q05: If I remember one thing, what makes AI work in the real world?
Here is the juicy part, and it is the opposite of what most of the AI hype tells you. The public conversation is obsessed with the model. Which one is smartest, which one just launched, whose benchmark went up. Inside the companies building real systems, the model is often the least interesting variable. What decides whether an AI product works in production is three unglamorous things: the quality of the data you feed it, how carefully you evaluate the output, and the systems around it that make it reliable and safe (security is a BIG thing and will become bigger as things progress).
I can tell you from the Oracle side that this is where the real work lives. My team does not win or lose on having a cleverer model than the next company. We win on whether the data is clean, whether we have a rigorous way to measure if an answer is good, and whether the whole thing holds up when thousands of people hit it at once. The model is the engine. Clean data, real evaluation, and reliability are the rest of the car, the part that gets you somewhere without crashing. When everyone is running the same engine, the differentiator becomes your organization’s own data, which is why an AI database matters.
Why does that matter to you, sitting at your desk with ChatGPT or Claude open? Because the same principle scales all the way down. Your results depend far more on what you feed the tool and how you check its work than on which model you picked. A clear prompt with the right context, and a habit of verifying what comes back, will beat a fancier model used carelessly every single time. Stop chasing the newest model. Get good at feeding it well and checking its answers.
This week, do one thing. The next time you ask AI for help with a real task, don’t just fire off the question. Tell it who the answer is for, what you’re trying to accomplish, and what a good result looks like. Then fact-check one thing it tells you before you use it. That is Q05 in miniature, and it is the whole game.
VP @ Oracle | LinkedIn Top Voice for AI | LinkedIn Learning Instructor | Views are my own
AI For You is a weekly newsletter for corporate women (and yes, the guys reading this too, welcome) navigating careers in the AI era. Real tools, real context, no tech background required.