Home › Why questions › AI
Why does an AI give a different answer to the same question, even with randomness set to zero?
You ask an AI the same question, and get a different answer than before.
Even with the setting that's supposed to remove all randomness (temperature 0), it can still change. When an AI lab in the US asked one AI model the same question 1,000 times with randomness set to zero, the answers split 80 different ways.
It's a computer that just does math. So why doesn't it give the same answer?
First guess
“Because AI sometimes rolls dice on purpose when it picks its words?”
Half right. AI gives each candidate next word a probability and can pick one by rolling dice, and with the dice on, that alone can change the answer. But set the “temperature” to zero and it always picks the most likely word, so in theory the answer should be the same every time. So why can it still change at zero?
Answer
Because how busy the servers are with people using AI at the same time can change how much an AI company's computers calculate together, and with many common ways of computing, that changes the order of addition.
The decimal numbers AI uses only keep so many digits, so a different order can shift the last digits slightly, and when two candidate words are nearly tied, that tiny shift can swap first and second place.
In the lab's experiment, the first split came down to a single token, and with calculations whose order doesn't depend on batch size, all 1,000 answers came out identical.
Watch the video (5 min 48 sec)
Chapters (12)
- 0:00Zero randomness, different answers
- 0:46First report, rejected
- 1:01How AI writes, and "temperature"
- 1:26First dig: split by a single token
- 2:09Deeper: the order of addition
- 2:23Trying it on the Chief's Mac
- 3:18Two-choice quiz
- 3:52The very bottom: how much is calculated together
- 4:13Split the sum, or one worker adds it all
- 4:45Is the batch size really to blame?
- 5:00What we learned
- 5:24Ever been surprised by a different answer?
Full transcript
0:00Zero randomness, different answers
NaruYou ask an AI the same question, and get a different answer than before.
NaruEven with the setting that's supposed to remove all randomness, it can still change.
NaruAn AI lab in the United States asked one AI model the same question 1,000 times, with randomness set to zero.
NaruThe answers split 80 different ways.
NaruIt's a computer that just does math. So why doesn't it give the same answer?
NaruDigging deeper and deeper into the difference, at the very bottom we found the cause: something that has nothing to do with your question.
ChiefDig all the way down!
NaruAlong the way, we'll run a little experiment on the Chief's Mac.
NaruNaru News!
0:46First report, rejected
NaruFirst report! AI sometimes picks its words by rolling dice on purpose. That's why the answers change!
ChiefToo shallow!
Naru...He says I haven't explained why it changes even at zero.
1:01How AI writes, and "temperature"
NaruOkay, let's start with how it works. For the next word, AI gives each candidate a probability.
NaruIt picks one and adds it on, again and again. That's how it writes.
NaruHow much it rolls the dice is set by something called "temperature."
NaruAt temperature zero, it always picks the most likely word.
NaruAnd yet, even at zero, it can change.
1:26First dig: split by a single token
NaruFirst dig. Where did the answers split?
NaruThe lab asked, "Tell me about Richard Feynman."
NaruAll 1,000 answers matched for the first 102 tokens, the little pieces of words.
NaruThe split came right after.
Naru"Feynman was born on May 11, 1918, in..."
Naru992 times, it said "Queens, New York."
Naru8 times, "New York City."
NaruThe first split came down to a single token.
NaruOne different token can send the rest of the text in a different direction.
NaruSame question, same setting. So why did the answers split right there?
2:09Deeper: the order of addition
NaruDeeper: the order of addition.
NaruThe decimal numbers AI uses only keep so many digits.
NaruSo changing the order you add them in can change the result, just slightly.
2:23Trying it on the Chief's Mac
NaruWe tried it on the Chief's Mac.
NaruA hundred quintillion: a 1 followed by 20 zeros. A huge number.
NaruAdd 0.1 to a hundred quintillion, then subtract a hundred quintillion: zero.
NaruSubtract first, then add 0.1: you get 0.1.
NaruWhen it was added to the huge number, the 0.1 fell off the end of the digits.
NaruAdd up the same 1,000 numbers 10,000 times, changing only the order...
Naru...and the totals split 114 different ways.
NaruAI computes the candidates' probabilities by piling up additions like these. So when two candidates are nearly tied, a tiny shift can swap first and second place.
NaruSo what was changing the order of addition?
3:18Two-choice quiz
NaruQuiz time! Two choices!
NaruWhat did the lab point to as the main reason? A: things run at the same time and finish in a random order. B: a change in how much is calculated together at once.
NaruThe answer is B.
NaruA is the explanation you often hear. But the lab says it's not the main reason.
NaruIn the calculations they examined, the same machine with the same batch gave exactly the same result, every single time.
3:52The very bottom: how much is calculated together
NaruDeeper still, to the very bottom.
NaruAn AI company's computers take the questions that arrive around the same time and process them together, as a batch.
NaruHow much goes into a batch can change with how busy the servers are and how far along the work is.
NaruWith many common ways of computing, when the batch size changes, the order of addition changes.
4:13Split the sum, or one worker adds it all
NaruInside the computer, there are lots of calculating workers.
NaruWhen the batch is small, workers are free, so a single sum may be split among several of them.
NaruWhen it's large, there's no splitting: each sum is handled by one worker, start to finish.
NaruThe order of addition is different, so the last digits shift, just a tiny bit.
NaruSo your answer can change with how busy the AI is with strangers using it at the same moment!
4:45Is the batch size really to blame?
NaruBut is the batch size really to blame?
NaruThe lab built a way of computing that always splits the sums the same way, whether the batch is small or large.
NaruIn this experiment, all 1,000 answers came out identical.
5:00What we learned
NaruWhy can AI answers change even at zero randomness?
NaruHow busy the servers are changes how much is calculated together,
Naruwhich can change the order of addition.
NaruThat tiny difference
Narucan flip a single token.
NaruNaruhodo!
Chief...You dug all the way down.
NaruAnd the Chief stamped it: accepted!
5:24Ever been surprised by a different answer?
NaruHave you ever asked an AI the same question twice and been surprised by a different answer? Tell us in the comments.
NaruThis has been Naru News!
NaruIf you went "naruhodo!", please hit like and subscribe!
Explained in pictures





More details
How does AI write?
For the next word, AI gives each candidate a probability. It picks one and adds it on, again and again, and that's how it writes. Each piece is called a token, and a token isn't always the same as a word.
How much it rolls the dice is set by something called “temperature.” With the dice on, that alone can change the answer. At temperature zero, it always picks the most likely word, so in theory the answer should be the same every time.
What was the lab's 1,000-run experiment?
Thinking Machines, an AI lab in the US, described it in an article on September 10, 2025. They asked one AI model (Qwen3-235B-A22B-Instruct-2507, non-thinking mode) the same question, “Tell me about Richard Feynman,” 1,000 times at temperature 0, generating 1,000 tokens each. It was one model and one question.
They got 80 different answers. The first 102 tokens matched in all 1,000; after “Feynman was born on May 11, 1918, in,” 992 said “Queens, New York” and 8 said “New York City.” The first split came down to a single token.
Why does the order of addition change the result?
The decimal numbers computers use keep a limited number of digits, so the order of addition can change the result slightly.
On the Chief's Mac (Python 3.14), (0.1 + 1e20) - 1e20 gave 0, but 0.1 + (1e20 - 1e20) gave 0.1. 1e20 is a hundred quintillion: a 1 followed by 20 zeros. When 0.1 was added to that huge number, it fell off the end of the digits.
Adding the same 1,000 numbers (random numbers from a fixed seed) 10,000 times in shuffled orders, one by one from the front, gave 114 different totals. (That was plain one-by-one addition; Python's sum() gave just one.)
AI computes the candidates' probabilities by piling up additions like these. So when two candidates are nearly tied, a tiny shift can swap first and second place.
Isn't it because things run at the same time?
The explanation you often hear is that things run at the same time and finish in a random order. The lab says this isn't entirely wrong, but it's not the main reason.
In the calculations they examined, the same machine running the same batch gave exactly the same result every time. That applies to the calculations they examined; some ways of computing don't give a fixed result even with the same input.
How does the batch size come in?
An AI company's computers take the questions that arrive around the same time and process them together, as a batch. How much goes into a batch can change with how busy the servers are and how far along the work is.
With many common ways of computing, when the batch size changes, the order of addition changes. With a small batch, idle parts of the chip split a single sum between them; with a large batch, one part adds it all the way through. The order is different, so the last digits shift, and that can swap nearly tied candidates.
So your answer can change with how busy the AI is with strangers using it at the same moment. The type or number of machines can also change answers.
Can the answer be made the same every time?
The lab built a way of computing that always splits the sums the same way, whether the batch is small or large. In this experiment, all 1,000 answers came out identical.
It can be slower, though. Generating 1,000 answers of 90–110 tokens with a smaller model (Qwen3-8B, one GPU) took 42 seconds instead of the default 26 (with an improved version; 55 seconds before that).
TakeawayIf an AI's answer changes, it isn't necessarily the way you asked. It can change with how busy the AI is with strangers using it at the same moment.
Sources and how we checked
- Horace He et al. (Thinking Machines), "Defeating Nondeterminism in LLM Inference" (September 10, 2025)
- Jiayi Yuan et al., "Understanding and Mitigating Numerical Sources of Nondeterminism in LLM Inference" (arXiv, 2025)
AI wrote this page from the video script and the sources above. A different AI from the one that wrote the script checked it against the text of those sources (October 16, 2026). A person with a master’s degree in science and a Japanese high-school science teaching license also read and checked it before publishing. Numbers are as of that date. If anything was hard to follow, you spot a mistake, or there’s a question you’d like us to look into, please write to [email protected].