Why does an AI model give different answers every time? And honestly, why does it sometimes feel downright random?
In classic computing, everything was deterministic: same input, same output, every time. That’s how traditional information systems worked (and still work). But large language models operate in a probabilistic world. Instead of following a fixed path, they’re essentially rolling weighted dice.
That’s not a bug. It’s the magic.
Why the “randomness” helps
This probabilistic behavior lets models fill in the gaps, because even when we think we’re giving complete information, we really aren’t.
Humans do this all the time. If my wife asks me to go buy lettuce, she doesn’t need to tell me what car to drive, which grocery store we prefer, or even what type of lettuce we usually buy. I fill in the holes effortlessly.
LLMs do the same thing. If I ask for help designing an API and forget to describe a key piece, the model will often say, “I’ll assume your method signature looks like X…” and just like that, it fills in exactly the detail I didn’t know I forgot.
The downside: hallucinations
Of course, this is also one way we get hallucinations. When the holes are too big, the model invents an entire story to connect them.
So how do we get more predictable output?
That’s where the very un-fancy term prompt engineering comes in.
The more structure, clarity, and constraints you give—outlines, formats, priorities, exclusions—the more deterministic the contours become, while still letting the model stay flexible where it needs to.
Hopefully this wasn’t too dicey of a topic.