What is latent space in AI?
In one minute
Latent space is the unseen space an AI forms when it sums up photos, voices, text and so on as lists of far fewer numbers than they started with, treating each list as a “place.”
Inside a computer, a picture is a list of numbers too. A small black-and-white picture of a handwritten digit (28 by 28 pixels, the tiny dots a picture is made of) is 784 numbers, one for each pixel’s brightness. Bigger or color pictures take many more. The AI practices packing the 784 numbers into 30, then computing 784 brightness numbers back from those 30 to restore the picture. Since it learns to make the restored picture close to the original, the 30 numbers keep the features that matter.
Two numbers, say 3 and 5, mark one place on graph paper: 3 across and 5 up. In the same way, the list of 30 numbers can be seen as one point in latent space. Similar pictures gather at nearby points. Change the 30 numbers of a “1” point little by little toward those of a “7” point, restore a picture at each step, and it shifts little by little from “1” toward “7.”
Picture-making AIs such as Stable Diffusion also finish a point of noise (random numbers) bit by bit in this space, then turn that point into a picture at the end.
A common mix-up
“Each number in latent space has a meaning people chose, such as how big the smile is or whether there are glasses.”
Only partly right. Directions that match features of a picture can sometimes be found in latent space. Since a point is a list of numbers, you can subtract or add two points, number by number, pairing the numbers in the same position. In a 2015 study with an AI that makes pictures of faces, the point for a woman not smiling was subtracted from the point for a smiling woman. The difference left over is “the smile,” and in the space it is one direction. Adding it to the point for a man not smiling gave a point that turned into a picture of a smiling man. But nobody decided in advance what each number would handle. The numbers are set during training, and people search afterward for the direction of a feature by comparing points for pictures with and without it, such as smiling and unsmiling faces. A single number rarely has a clear meaning. In the same study, the calculation was unstable when it used one face picture at a time, such as a single smiling woman, and became stable only when the points for three faces of each kind were averaged.
Explained in pictures
You’re free to use these figures in lessons, slides and posts if you credit Naru News (naru.news). CC BY 4.0.
Think of it like this
Packing a dish into a short recipe, then cooking it again
Imagine you want to tell a friend far away about a delicious curry. Trying to describe the curry itself in detail would never end: the shapes of the vegetables, the color, the smell. So you pack only what matters for making it again into a recipe with a few numbers, such as “2 onions, 300 grams of meat, spiciness 3, simmer 30 minutes.” With that recipe, your friend can cook a similar curry. The few numbers that sum up the dish are like a point in latent space.
Similar dishes have similar recipes. The recipes for a hot curry and a mild curry differ in little more than the spiciness number. Change that number little by little from 3 to 5, and the curry gets a little hotter each time. It is the same as the picture changing little by little as you move from point to point in latent space.
But the comparison has limits. People choose the items in a recipe, and they have names such as “spiciness.” The numbers in latent space are set as the AI practices packing and restoring, and none of them has a name. Usually, even people can’t really tell what each number stands for. And latent space isn’t a real place, like a shelf of recipe books. It is a space in the mind’s eye: lists of numbers seen as “places.”
Key points
Why is it called “latent” and a “space”?
The English word latent comes from a Latin word meaning “lying hidden,” and it describes something that is present but hidden or not yet active, as in “latent ability” (power that hasn’t shown itself yet). The Japanese name, senzai kūkan, uses senzai, a word with the same meaning: existing hidden inside without showing on the surface.
The numbers in a photo stand for the brightness and color of each pixel, so you can see them directly. The numbers in latent space are not like that. The AI made them while learning; they don’t appear on the photo, yet they hold the photo’s important features hidden inside. That is why they are called latent.
It is called a space because the list of numbers is read as numbers that give a place. A place on graph paper is set by two numbers, across and up, and a place in a room by three: across, depth and height. In the same way, a list of 30 numbers can be seen as one place in a space with 30 directions. Seen as places, whether two lists are alike can be judged by how far apart the two points are.
But there is no real room-like place inside an AI. What is inside the AI is lists of numbers, and people picture them as a “space.” A space with four or more directions can’t be drawn by people.
Why squeeze things down?
To turn big data into a few numbers that keep only what matters. One neural network (a computing system modeled loosely on the brain’s nerve connections) built for this is the autoencoder. Its first half, the squeezer (encoder), packs the data into a few numbers, and its second half, the restorer (decoder), rebuilds the original data from those numbers. Both halves are trained together so that the gap between the original data and the rebuilt version gets smaller. A 2006 paper showed a way to train autoencoders well even when they stack many layers.
Because there are few numbers in the middle, the AI can’t keep everything. It has no choice but to pick and keep the features that matter for restoring. The 30 numbers it keeps are not picked from the original pixels; the AI makes them while learning. In the paper, small pictures of handwritten digits (28×28 pixels, 784 numbers) packed into 30 numbers could be restored far closer to the original than with a long-used method that also packed them into 30 numbers. This was checked on 10,000 new pictures, apart from the 60,000 used for training.
Pack into just two numbers, and you can draw the data on a flat chart. For handwritten digits, this gave a clearer picture of the data than packing into two numbers with the long-used method. The paper says packing data into a few numbers makes it easier to sort, show, send and store.
But what comes back is not exactly the same as the original. Fine details that don’t fit into a few numbers are lost.
How does the restoring part draw a picture from numbers?
It doesn’t draw lines one stroke at a time, the way people do with a brush. It computes the brightness of every pixel as a number. For handwritten digits, put in the 30 numbers of a point in latent space and out come 784 numbers. Set them out in 28 rows of 28, paint each pixel brighter the bigger its number, and you have a picture. For a color picture, it puts out three numbers per pixel: the strength of red, green and blue.
The computing is done by a neural network (a system that multiplies numbers by other numbers called “weights” and adds them up, in many stacked layers; more on the “Neural network” page). It multiplies the 30 numbers by weights and adds them up to make the numbers of the next layer, makes the layer after that from those, and so on, until the last layer gives 784 numbers.
People didn’t choose the weights. They were set, together with the squeezing part, by adjusting them little by little over many pictures so that the restored picture comes out close to the original. That is why putting in the point a learned picture was packed into gives back a picture very like it.
When making a new picture, the restoring part does the same thing. What differs is where the point comes from. A variational autoencoder (VAE) picks a point from around the middle of latent space (see “How do picture-making AIs use it?” below). Stable Diffusion starts from a point made of random numbers (noise) and, using the words of the request as a guide, guesses the noise mixed into the point and removes it bit by bit. The finished point is 4 sheets of 64×64 numbers (16,384 numbers). One pass through the restoring part turns it into a 512×512-pixel picture (three numbers per pixel, 786,432 numbers).
How is it different from an embedding?
The two words are very close and overlap a lot. An embedding is a list of numbers that an AI turns a word, sentence, picture or other item into, so that similar things come close together (more on the “Embedding” page). The space where those lists gather as points is called the embedding space. Some glossaries treat latent space as meaning the same as embedding space.
The difference is what each word points at. “Embedding” points at the list of numbers (the point) given to each item, and it comes up a lot when comparing words or finding text with a similar meaning. “Latent space” points at the space where the points gather. In research on picture-making AIs especially, the set of number lists that become pictures when passed through the restoring part has been called latent space. It is the word used when talking about moving around in that space or picking points to make new pictures.
It is used in research on language AIs too. A 2024 study had a language AI pass its in-between thinking on to its next step as lists of numbers inside the AI, without writing it out in words. The paper describes this as having the AI reason in a latent space instead of in human language (more on the “Neuralese” page).
What happens when you move between points?
Moving from point to point means changing the list of numbers of one point, little by little, into the list of the other. For example, the point exactly halfway between two points is the list you get by adding the numbers in the same position and dividing by 2. Passing the points along the way through the restoring part one by one lets you see the change as pictures.
In a well-learned latent space, moving little by little from point to point makes the picture smoothly change little by little too. A 2015 study checked this with an AI that makes pictures of bedrooms. The AI was a kind of GAN (a way to make pictures by pitting two AIs against each other), and it turns a point made of 100 numbers into a picture by passing it through its making part. The researchers picked 9 points at random and made pictures while moving little by little from one point to the next. The pictures changed smoothly, and every picture along the way looked like a bedroom. A room without a window slowly turned into a room with a giant window, and something that looked like a TV slowly turned into a window.
The researchers see this kind of change, where objects slowly appear or disappear, as a sign that the AI has learned a way of representing what is in the pictures.
Adding and subtracting worked too. In the same study, using the points of an AI that makes pictures of faces, the researchers averaged three points each for a smiling woman, a woman not smiling and a man not smiling, then calculated “smiling woman − woman not smiling + man not smiling.” Turned into a picture, that point gave the face of a smiling man. “Man with glasses − man without glasses + woman without glasses” gave the face of a woman with glasses (all of these are faces the AI made). Doing the same calculation directly on the brightness of the picture’s pixels gave messy, noisy pictures, with faces overlapping out of line.
In a latent space of just two numbers, if you set points evenly across it and turn each one back into a picture of a handwritten digit, the digit changes from place to place, and where neighbors meet you can see one digit smoothly turning into another. A 2013 paper also shows figures made this way.
How do picture-making AIs use it?
Pick a point in latent space and pass it through the restoring part, and you should get a new picture. The variational autoencoder (VAE), introduced in a 2013 paper, has a trick for this. It is trained so that the points it packs pictures into gather around the middle of the space and connect smoothly. Then you can make a new picture just by picking a point from around there and passing it through the restoring part.
Well-known picture-making AIs today also use latent space. A paper released in December 2021 ran a diffusion model (a method that makes a picture from pure noise by removing the noise a little at a time) not on a picture’s pixels but inside the latent space of an autoencoder trained beforehand (a latent diffusion model). Most of the information in a digital picture is fine detail people can barely see. The squeezing part drops that detail, and the model learns the content and layout of pictures in a small space. When making a picture, the point whose noise has been removed in latent space just needs one pass through the restoring part to become a picture. Once trained, the squeezing and restoring parts can be reused for several diffusion models.
Training the most powerful diffusion models that work on pixels could take hundreds of days of computing, counted as if done on a single GPU (a fast chip used for AI computing). Making 50,000 pictures took about 5 days on one GPU. In latent space there are fewer numbers to handle, so the computing is far lighter. But there is a sweet spot for how much to squeeze. In the paper, not squeezing at all, or shrinking the picture’s width and height only to 1/2, made training progress slowly, while shrinking them to 1/32 made picture quality stop improving early on. Shrinking the width and height to between 1/4 and 1/16 was the sweet spot between lightness and picture quality.
Stable Diffusion works this way too. Its squeezing part packs a picture into lists of numbers at 1/8 the picture’s width and height, and the noise work happens in that latent space. The words of the request are also turned into numbers and fed into the calculation that removes the noise. In training, the AI practiced guessing the noise mixed into the points for pictures, using the words that came with each picture as a guide, so the point is finished bit by bit into one that fits the words (more on the “Diffusion model” page).
What is still unclear, and what should you watch out for?
What each single number in latent space stands for is usually not clear to people. The numbers were set as the AI practiced packing and restoring; people didn’t name them. Sometimes their meaning can be guessed by looking into them, but not always (research that reads what goes on inside AIs is called interpretability).
Directions with a clear meaning, like a “smile direction,” can’t always be found neatly. In the 2015 study too, adding and subtracting with one face at a time gave unstable results, and became stable only when the points for three faces of each kind were averaged.
Similar things gathering close together is also a result of training; the AI didn’t arrange them by understanding their meaning the way people do. How close points are, and the directions in the space, don’t always match what people feel is “alike.”
Check yourself
Q1An AI called an autoencoder practices, over and over, packing a picture of a handwritten digit (784 numbers) into 30 numbers and then turning it back into the original picture. What does the AI learn from this practice?
Answer and explanation
Answer: C A way to make the 30 numbers for restoring while learning
With only 30 numbers in the middle, the AI can’t keep everything, so it comes to pick and keep the features that matter for restoring the picture. The 30 numbers are neither picked from the original pixels nor averages over fixed cells of the picture; the AI makes them while it learns to shrink the gap between the original and the restored picture. Fine details that don’t fit into a few numbers are lost, so the restored picture is not exactly the same as the original.
Q2In a 2015 study, researchers picked 9 points at random in the latent space of an AI that makes pictures of bedrooms, and made pictures while moving little by little from one point to the next. What were the pictures along the way like?
Answer and explanation
Answer: B All still looked like bedrooms, with windows and such changing
Every picture along the way looked like a bedroom, and they changed smoothly: for example, a room without a window slowly turned into a room with a large window. The researchers see this kind of change, where objects slowly appear or disappear, as a sign that the AI has learned a way of representing what is in the pictures. Along the way, they never faded into some other kind of room.
More on this → Key points: What happens when you move between points?
Q3Why do picture-making AIs such as Stable Diffusion make pictures inside a latent space where the picture is squeezed down, rather than working on the picture’s pixels (tiny dots)?
Answer and explanation
Answer: A Fewer numbers to handle, so the computing is far lighter
In a latent space where the picture is squeezed down, there are fewer numbers to handle, so the calculations for removing noise are far lighter. At the end, the finished point is passed once through the restoring part to become a full-size picture. Fine details people can barely see are actually dropped by the squeezing part. In the experiments of the latent diffusion paper, squeezing too much stopped picture quality from improving, and shrinking the width and height to between 1/4 and 1/16 was the sweet spot. And the latent space is not free of noise: the AI still starts from a point of random numbers (noise) and removes it bit by bit, using the words of the request as a guide.
More on this → Key points: How do picture-making AIs use it?
Related terms
- Embedding
- Autoencoder
- Variational autoencoder (VAE)
- Generative adversarial network (GAN)
- Diffusion model
- Generative AI
- Neural network
- Deep learning
- Machine learning
- Parameter
- Neuralese
- Interpretability
- Dimensionality reduction
Sources and how we checked
- Hinton and Salakhutdinov, “Reducing the Dimensionality of Data with Neural Networks” (University of Toronto; Science 313 (5786), pp. 504–507, July 28, 2006; read the abstract, the main text (encoder and decoder; the curves, handwritten digits, faces and documents experiments; conclusion) and the legends of Figures 2–4 in the PDF on the author’s page; read November 13, 2026)
- Kingma and Welling, “Auto-Encoding Variational Bayes” (the variational autoencoder; arXiv, version 1 December 20, 2013, version 11 December 2022; abstract, Sections 2.1, 2.3, 3 and 5, and the captions of Figures 4 and 5; read November 13, 2026)
- Radford, Metz and Chintala, “Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks” (DCGAN; indico Research and Facebook AI Research; arXiv, November 2015, version 2 January 2016, submitted to ICLR 2016; abstract, Section 6.1 “Walking in the latent space,” Section 6.3, and Figures 4, 7 (including the labels in the figure) and 8; read November 13, 2026)
- Rombach et al., “High-Resolution Image Synthesis with Latent Diffusion Models” (latent diffusion models; CVPR 2022; arXiv, December 2021, version 2 April 2022; abstract, Section 1, the caption of Figure 2, Section 3 (3.1 and 3.2), Section 4.1 and the caption of Figure 6; read November 13, 2026)
- CompVis, “Stable Diffusion” (GitHub README; the downsampling-factor 8 autoencoder, and pretraining on 256×256 images before fine-tuning on 512×512; read November 13, 2026)
- CompVis, “stable-diffusion-v1-4” model card (Hugging Face; the training procedure: the encoder that turns images into latent representations, mapping H×W×3 to H/8×W/8×4, and how text prompts are fed in; read November 13, 2026)
- Hao et al., “Training Large Language Models to Reason in a Continuous Latent Space” (Coconut; FAIR at Meta and UC San Diego; arXiv, December 2024, version 4 August 2026; abstract, Section 1 and Table 1; read November 13, 2026)
- Google, “Machine Learning Glossary” (entries for latent space (a synonym for embedding space), embedding space, autoencoder, encoder and decoder; read November 13, 2026)
- Google, “Embeddings: Embedding space and static embeddings” (Machine Learning Crash Course; closer points are more alike, and single dimensions are rarely easy to interpret; read November 13, 2026)
- TensorFlow, “Convolutional Variational Autoencoder” (TensorFlow tutorial; a VAE maps inputs to the parameters of a probability distribution, giving a continuous, structured latent space, and a 2D latent space where each digit morphs into another; read November 13, 2026)
- IBM, Japanese page “What is latent space?” (January 28, 2025; for the Japanese name “senzai kūkan” and the point that similar data points sit close together; read November 13, 2026)
- Weblio, Japanese dictionary entry for “senzai” (Digital Daijisen; existing hidden inside, without showing on the surface; read November 13, 2026)
- Wiktionary, “latent” (English edition; etymology from Latin latēns, “lying hidden,” and meaning; read November 13, 2026)
AI wrote this page from the sources above, and a different AI checked it against the text of those sources (November 13, 2026). A person with a master’s degree in science and a Japanese high-school science teaching license also read and checked it before publishing. Numbers are as of that date. If anything was hard to follow, you spot a mistake, or there’s a term you’d like explained, please write to [email protected].
AI terms
- AI agent
- AI alignment
- Alignment faking
- Artificial general intelligence (AGI)
- Artificial intelligence (AI)
- Autoformalization
- Backdoor
- Benchmark
- Chain of thought (CoT)
- Context window
- Data poisoning
- Deep learning
- Deepfake
- Diffusion model
- Embedding
- Emergent misalignment
- Evaluation awareness
- Fine-tuning
- Formal verification
- Generalization
- Generative adversarial network (GAN)
- Generative AI
- Gradual disempowerment
- Hallucination
- Hyperparameter
- Intelligence explosion
- Interpretability
- Jailbreaking
- Large language model (LLM)
- You are hereLatent space
- Lean
- Machine learning
- Mixture of experts (MoE)
- Neural network
- Neuralese
- Overfitting
- Parameter
- Persona vectors
- Pre-training
- Prompt
- Prompt injection
- Reasoning model
- Recursive language model (RLM)
- Recursive self-improvement (RSI)
- Red teaming
- Reinforcement learning (RL)
- Retrieval-augmented generation (RAG)
- Reward hacking
- Shutdown resistance
- Sleeper agent
- Subliminal learning
- Superintelligence (ASI)
- System prompt
- Technological singularity
- Time horizon
- Token
- Transfer learning
- Transformer