Naru News日本語

Home › Why questions › AI

Why does an AI give a different answer to the same question, even with randomness set to zero?

Video published November 18, 2026 (5 min 48 sec) · Checked October 16, 2026 · 6 min read

You ask an AI the same question, and get a different answer than before.

Even with the setting that's supposed to remove all randomness (temperature 0), it can still change. When an AI lab in the US asked one AI model the same question 1,000 times with randomness set to zero, the answers split 80 different ways.

It's a computer that just does math. So why doesn't it give the same answer?

First guess

“Because AI sometimes rolls dice on purpose when it picks its words?”

Half right. AI gives each candidate next word a probability and can pick one by rolling dice, and with the dice on, that alone can change the answer. But set the “temperature” to zero and it always picks the most likely word, so in theory the answer should be the same every time. So why can it still change at zero?

Answer

Because how busy the servers are with people using AI at the same time can change how much an AI company's computers calculate together, and with many common ways of computing, that changes the order of addition.

The decimal numbers AI uses only keep so many digits, so a different order can shift the last digits slightly, and when two candidate words are nearly tied, that tiny shift can swap first and second place.

In the lab's experiment, the first split came down to a single token, and with calculations whose order doesn't depend on batch size, all 1,000 answers came out identical.

Watch the video (5 min 48 sec)

Chapters (12)
  1. 0:00Zero randomness, different answers
  2. 0:46First report, rejected
  3. 1:01How AI writes, and "temperature"
  4. 1:26First dig: split by a single token
  5. 2:09Deeper: the order of addition
  6. 2:23Trying it on the Chief's Mac
  7. 3:18Two-choice quiz
  8. 3:52The very bottom: how much is calculated together
  9. 4:13Split the sum, or one worker adds it all
  10. 4:45Is the batch size really to blame?
  11. 5:00What we learned
  12. 5:24Ever been surprised by a different answer?
Full transcript

0:00Zero randomness, different answers

NaruYou ask an AI the same question, and get a different answer than before.

NaruEven with the setting that's supposed to remove all randomness, it can still change.

NaruAn AI lab in the United States asked one AI model the same question 1,000 times, with randomness set to zero.

NaruThe answers split 80 different ways.

NaruIt's a computer that just does math. So why doesn't it give the same answer?

NaruDigging deeper and deeper into the difference, at the very bottom we found the cause: something that has nothing to do with your question.

ChiefDig all the way down!

NaruAlong the way, we'll run a little experiment on the Chief's Mac.

NaruNaru News!

0:46First report, rejected

NaruFirst report! AI sometimes picks its words by rolling dice on purpose. That's why the answers change!

ChiefToo shallow!

Naru...He says I haven't explained why it changes even at zero.

1:01How AI writes, and "temperature"

NaruOkay, let's start with how it works. For the next word, AI gives each candidate a probability.

NaruIt picks one and adds it on, again and again. That's how it writes.

NaruHow much it rolls the dice is set by something called "temperature."

NaruAt temperature zero, it always picks the most likely word.

NaruAnd yet, even at zero, it can change.

1:26First dig: split by a single token

NaruFirst dig. Where did the answers split?

NaruThe lab asked, "Tell me about Richard Feynman."

NaruAll 1,000 answers matched for the first 102 tokens, the little pieces of words.

NaruThe split came right after.

Naru"Feynman was born on May 11, 1918, in..."

Naru992 times, it said "Queens, New York."

Naru8 times, "New York City."

NaruThe first split came down to a single token.

NaruOne different token can send the rest of the text in a different direction.

NaruSame question, same setting. So why did the answers split right there?

2:09Deeper: the order of addition

NaruDeeper: the order of addition.

NaruThe decimal numbers AI uses only keep so many digits.

NaruSo changing the order you add them in can change the result, just slightly.

2:23Trying it on the Chief's Mac

NaruWe tried it on the Chief's Mac.

NaruA hundred quintillion: a 1 followed by 20 zeros. A huge number.

NaruAdd 0.1 to a hundred quintillion, then subtract a hundred quintillion: zero.

NaruSubtract first, then add 0.1: you get 0.1.

NaruWhen it was added to the huge number, the 0.1 fell off the end of the digits.

NaruAdd up the same 1,000 numbers 10,000 times, changing only the order...

Naru...and the totals split 114 different ways.

NaruAI computes the candidates' probabilities by piling up additions like these. So when two candidates are nearly tied, a tiny shift can swap first and second place.

NaruSo what was changing the order of addition?

3:18Two-choice quiz

NaruQuiz time! Two choices!

NaruWhat did the lab point to as the main reason? A: things run at the same time and finish in a random order. B: a change in how much is calculated together at once.

NaruThe answer is B.

NaruA is the explanation you often hear. But the lab says it's not the main reason.

NaruIn the calculations they examined, the same machine with the same batch gave exactly the same result, every single time.

3:52The very bottom: how much is calculated together

NaruDeeper still, to the very bottom.

NaruAn AI company's computers take the questions that arrive around the same time and process them together, as a batch.

NaruHow much goes into a batch can change with how busy the servers are and how far along the work is.

NaruWith many common ways of computing, when the batch size changes, the order of addition changes.

4:13Split the sum, or one worker adds it all

NaruInside the computer, there are lots of calculating workers.

NaruWhen the batch is small, workers are free, so a single sum may be split among several of them.

NaruWhen it's large, there's no splitting: each sum is handled by one worker, start to finish.

NaruThe order of addition is different, so the last digits shift, just a tiny bit.

NaruSo your answer can change with how busy the AI is with strangers using it at the same moment!

4:45Is the batch size really to blame?

NaruBut is the batch size really to blame?

NaruThe lab built a way of computing that always splits the sums the same way, whether the batch is small or large.

NaruIn this experiment, all 1,000 answers came out identical.

5:00What we learned

NaruWhy can AI answers change even at zero randomness?

NaruHow busy the servers are changes how much is calculated together,

Naruwhich can change the order of addition.

NaruThat tiny difference

Narucan flip a single token.

NaruNaruhodo!

Chief...You dug all the way down.

NaruAnd the Chief stamped it: accepted!

5:24Ever been surprised by a different answer?

NaruHave you ever asked an AI the same question twice and been surprised by a different answer? Tell us in the comments.

NaruThis has been Naru News!

NaruIf you went "naruhodo!", please hit like and subscribe!

Explained in pictures

Bars for the next-word candidates (sunny, cloudy, rainy) and a temperature dial. At temperature 0, it always picks the most likely word, “sunny”
This picture shows how AI writes. The bars on the left are the candidates for the next word, with a picture of each one's probability. AI picks one, adds it on, and repeats, and that's how it writes. The “temperature” on the right sets how much it rolls the dice. At zero, it always picks the most likely word, so, as at the bottom, it says “sunny” every time, and in theory the answer should be the same every time.
A tree branching in two again and again from one point: one different token can change what follows
This picture shows where the answers split in the lab's 1,000-run experiment. All 1,000 answers matched for the first 102 tokens, the little pieces of words. After “Feynman was born on May 11, 1918, in,” 992 said “Queens, New York” and 8 said “New York City.” Like the branches spreading from the dot on the left, one different token can send the rest of the text in a different direction.
The same numbers added front to back give 0.6000000; back to front, 0.6000001. The last digit differs, just slightly
This picture shows how the order of addition can change the result (the numbers are illustrative). The decimal numbers AI uses only keep so many digits. So adding the same numbers front to back or back to front can change the last digit of the result (circled) just slightly. On the Chief's Mac, adding up the same 1,000 numbers 10,000 times in shuffled orders gave 114 different totals.
Split among workers: 0.7329154. One worker: 0.7329157. The same sum, with the last digit shifted
This picture shows calculating workers inside the computer, a stand-in for the parts that do the math (the numbers are illustrative). When the batch is small, workers are free, so a single sum may be split among several of them and the subtotals combined. When it's large, there's no splitting: one worker handles each sum start to finish. The order of addition is different, so the split sum (left) and the single worker's sum (right) differ in the last digit, just a tiny bit.
What the lab built: whether the batch is small or large, the sum is split the same way, and both give 0.7329157
This picture shows what the lab built (the numbers are illustrative). Even when the batch is small (left), it doesn't switch to splitting the sum; it adds the same way as with a large batch (right), so the results match (the = in the middle). With this, the lab asked the same question 1,000 times, and in this experiment all 1,000 answers came out identical, evidence that batch-dependent addition was the cause.

More details

How does AI write?

For the next word, AI gives each candidate a probability. It picks one and adds it on, again and again, and that's how it writes. Each piece is called a token, and a token isn't always the same as a word.

How much it rolls the dice is set by something called “temperature.” With the dice on, that alone can change the answer. At temperature zero, it always picks the most likely word, so in theory the answer should be the same every time.

What was the lab's 1,000-run experiment?

Thinking Machines, an AI lab in the US, described it in an article on September 10, 2025. They asked one AI model (Qwen3-235B-A22B-Instruct-2507, non-thinking mode) the same question, “Tell me about Richard Feynman,” 1,000 times at temperature 0, generating 1,000 tokens each. It was one model and one question.

They got 80 different answers. The first 102 tokens matched in all 1,000; after “Feynman was born on May 11, 1918, in,” 992 said “Queens, New York” and 8 said “New York City.” The first split came down to a single token.

Why does the order of addition change the result?

The decimal numbers computers use keep a limited number of digits, so the order of addition can change the result slightly.

On the Chief's Mac (Python 3.14), (0.1 + 1e20) - 1e20 gave 0, but 0.1 + (1e20 - 1e20) gave 0.1. 1e20 is a hundred quintillion: a 1 followed by 20 zeros. When 0.1 was added to that huge number, it fell off the end of the digits.

Adding the same 1,000 numbers (random numbers from a fixed seed) 10,000 times in shuffled orders, one by one from the front, gave 114 different totals. (That was plain one-by-one addition; Python's sum() gave just one.)

AI computes the candidates' probabilities by piling up additions like these. So when two candidates are nearly tied, a tiny shift can swap first and second place.

Isn't it because things run at the same time?

The explanation you often hear is that things run at the same time and finish in a random order. The lab says this isn't entirely wrong, but it's not the main reason.

In the calculations they examined, the same machine running the same batch gave exactly the same result every time. That applies to the calculations they examined; some ways of computing don't give a fixed result even with the same input.

How does the batch size come in?

An AI company's computers take the questions that arrive around the same time and process them together, as a batch. How much goes into a batch can change with how busy the servers are and how far along the work is.

With many common ways of computing, when the batch size changes, the order of addition changes. With a small batch, idle parts of the chip split a single sum between them; with a large batch, one part adds it all the way through. The order is different, so the last digits shift, and that can swap nearly tied candidates.

So your answer can change with how busy the AI is with strangers using it at the same moment. The type or number of machines can also change answers.

Can the answer be made the same every time?

The lab built a way of computing that always splits the sums the same way, whether the batch is small or large. In this experiment, all 1,000 answers came out identical.

It can be slower, though. Generating 1,000 answers of 90–110 tokens with a smaller model (Qwen3-8B, one GPU) took 42 seconds instead of the default 26 (with an improved version; 55 seconds before that).

Takeaway

If an AI's answer changes, it isn't necessarily the way you asked. It can change with how busy the AI is with strangers using it at the same moment.

Sources and how we checked

AI wrote this page from the video script and the sources above. A different AI from the one that wrote the script checked it against the text of those sources (October 16, 2026). A person with a master’s degree in science and a Japanese high-school science teaching license also read and checked it before publishing. Numbers are as of that date. If anything was hard to follow, you spot a mistake, or there’s a question you’d like us to look into, please write to [email protected].

Next questions