Naru News日本語

Home › Glossary › AI

What is latent space in AI?

Term explainer · Checked November 13, 2026 · 17 min read

In one minute

Latent space is the unseen space an AI forms when it sums up photos, voices, text and so on as lists of far fewer numbers than they started with, treating each list as a “place.”

Inside a computer, a picture is a list of numbers too. A small black-and-white picture of a handwritten digit (28 by 28 pixels, the tiny dots a picture is made of) is 784 numbers, one for each pixel’s brightness. Bigger or color pictures take many more. The AI practices packing the 784 numbers into 30, then computing 784 brightness numbers back from those 30 to restore the picture. Since it learns to make the restored picture close to the original, the 30 numbers keep the features that matter.

Two numbers, say 3 and 5, mark one place on graph paper: 3 across and 5 up. In the same way, the list of 30 numbers can be seen as one point in latent space. Similar pictures gather at nearby points. Change the 30 numbers of a “1” point little by little toward those of a “7” point, restore a picture at each step, and it shifts little by little from “1” toward “7.”

Picture-making AIs such as Stable Diffusion also finish a point of noise (random numbers) bit by bit in this space, then turn that point into a picture at the end.

A common mix-up

“Each number in latent space has a meaning people chose, such as how big the smile is or whether there are glasses.”

Only partly right. Directions that match features of a picture can sometimes be found in latent space. Since a point is a list of numbers, you can subtract or add two points, number by number, pairing the numbers in the same position. In a 2015 study with an AI that makes pictures of faces, the point for a woman not smiling was subtracted from the point for a smiling woman. The difference left over is “the smile,” and in the space it is one direction. Adding it to the point for a man not smiling gave a point that turned into a picture of a smiling man. But nobody decided in advance what each number would handle. The numbers are set during training, and people search afterward for the direction of a feature by comparing points for pictures with and without it, such as smiling and unsmiling faces. A single number rarely has a clear meaning. In the same study, the calculation was unstable when it used one face picture at a time, such as a single smiling woman, and became stable only when the points for three faces of each kind were averaged.

Explained in pictures

A sketch of a handwritten digit being squeezed into a few numbers and turned back into a picture (numbers from a 2006 study; drawings made for this page). Title: “Squeeze a picture into a few numbers, then back.” From the left: a navy “2” drawn on a grid, labeled “Original,” “28×28 pixels” and “784 numbers.” An arrow leads to a navy trapezoid, wide on the left and narrow on the right, reading “Squeeze,” labeled “Squeezer” and “(encoder)” below. An arrow leads to 30 cells in different shades inside an orange frame, with “One point in latent space” above and “30 numbers” below. An arrow leads to a navy trapezoid, narrow on the left and wide on the right, reading “Restore,” labeled “Restorer” and “(decoder)” below. An arrow leads to a slightly blurred “2,” labeled “Restored,” “Very close” and “784 numbers.” A dotted bracket under the original and the restored picture reads “Both parts learn together to shrink the gap between the two.” Band at the bottom: To restore it, keep only the key features in a few numbers.
This sketch shows, from left to right, a picture of a handwritten digit being packed into a few numbers and turned back into a picture. The picture on the left is 784 numbers, the brightness of 28×28 pixels (tiny dots). The squeezing AI (the left trapezoid) packs it into 30 numbers. That list of 30 numbers is one point in latent space. The restoring AI (the right trapezoid) rebuilds a 784-number picture from that point. The dotted bracket below is where the original and the restored picture are compared. Because both parts learn together to shrink the gap, the AI comes to keep in the 30 numbers only the features that matter for restoring the picture.
A sketch of the restoring AI (decoder) making a picture from the numbers of a point in latent space (numbers from a 2006 study; only some of the circles and lines are drawn). Title: “The restorer computes each pixel’s brightness.” From the left: 30 cells in different shades inside an orange frame, with “One point in latent space” above and “30 numbers” below. An arrow leads to a pale navy frame holding four columns of circles, 4, 5, 6 and 7 from left to right, with every circle joined by thin lines to every circle in the next column; the left column is orange and the right column navy. Below the frame: “Restorer (decoder),” “Multiply by weights” and “and add, layer by layer.” An arrow leads to a white box of numbers: “0 0 0 0,” “0 0.4 0.7 0.9,” “0.7 0.3 0.1 0,” “0 0.1 0.8 1” and a vertical ellipsis, labeled “784 numbers” and “brightness of each pixel.” An arrow leads to a 28×28 grid on navy, where light cells form the digit “2,” labeled “The picture,” “Set out in 28 rows of 28,” and “bigger numbers brighter.” Below that: “The weights were learned so restored pictures come out close to the originals.” Band at the bottom: No strokes one by one: every pixel’s brightness at once.
This sketch shows, from left to right, the restoring AI making a picture from one point in latent space. The orange frame on the left is the 30 numbers of the point. The circles and lines in the middle are the restoring part: it multiplies the numbers of one layer by weights set during learning, adds them up, and repeats this layer after layer. The 784 numbers that come out of the last layer are each the brightness of one pixel. Set out in 28 rows of 28, with pixels painted brighter the bigger their number, they become the “2” on the right. It doesn’t draw strokes one by one; it computes the brightness of every pixel at once.
A sketch of similar pictures gathering close together in latent space, and pictures changing bit by bit as you move from point to point (drawn flat with just two numbers; points and pictures made up for this page). Title: “Near means alike; walk and the picture morphs.” In a wide frame at the top, “Latent space,” are three groups of points: navy circles on the left, “pictures of 1,” orange squares on the right, “pictures of 7,” and white diamonds at the top middle, “pictures of 0.” A dotted orange arrow runs across from point A in the “1” group to point E in the “7” group, passing points B, C and D. Under the arrow: “Moving bit by bit.” Gray dotted lines run down from each point to five square pictures below, labeled on the left “Pictures from the points.” From left to right, the pictures show a straight “1,” one with a short bar on top, one with a longer bar and a slanted stroke, one closer to a “7,” and a clear “7.” Band at the bottom: Near points, similar pictures; in between, a smooth change.
This sketch draws latent space flat, like graph paper, using just two numbers. The marks in the top frame are points that pictures of handwritten digits were packed into: circles are pictures of “1,” squares of “7” and diamonds of “0.” Points for similar pictures gather close together. The dotted orange line is a path from point A, a “1,” to point E, a “7.” The five pictures below were made by passing points A to E on that path through the restoring AI, and the “1” at A gradually turns into a “7.” Most latent spaces have many more numbers (directions), so they can’t be drawn flat like this.
A sketch of how an AI that makes pictures using latent space (a latent diffusion model) makes a picture (picture drawn for this page). Title: “Image AIs finish a point in latent space.” Subtitle: A sketch of a latent diffusion model, such as Stable Diffusion. A large frame on the left, “Latent space,” with “Numbers at 1/8 of the picture’s width and height” below. At the top right of the frame, an orange bubble: “Prompt: mountains, sun.” At the bottom left, a cluster of scattered small gray specks: “Start from a noise point.” From there, five white circles joined by orange arrows lead up and to the right to an orange star, “Finished point.” Under the row of arrows: “Remove noise bit by bit” and “(over many steps).” A dotted arrow runs from the bubble to the row of arrows, labeled “Used as a guide.” From the star, an arrow leads right to a navy trapezoid, narrow on the left and wide on the right, “Restorer,” with “Just once, at the end” below. An arrow leads on to a picture of mountains and a sun, “The picture.” A dotted box at the bottom right: “In training, the squeezer turns pictures into points.” Band at the bottom: Working with few numbers keeps the computing light.
This sketch shows how a picture-making AI that uses latent space, such as Stable Diffusion, makes a picture. The large frame on the left is latent space. A picture is packed into lists of numbers at 1/8 its width and height, so there are far fewer numbers to handle. The gray speckles at the bottom left are a point made of random numbers (noise), where it starts. The row of orange arrows shows the point being moved while noise is removed bit by bit, using the words of the prompt (the request) as a guide. At the end, the finished point (the star) is passed through the restoring AI (the trapezoid) just once to become a full-size picture. Because the work happens in a space with few numbers, the computing is far lighter than working on the picture’s pixels.

You’re free to use these figures in lessons, slides and posts if you credit Naru News (naru.news). CC BY 4.0.

Think of it like this

Packing a dish into a short recipe, then cooking it again

Imagine you want to tell a friend far away about a delicious curry. Trying to describe the curry itself in detail would never end: the shapes of the vegetables, the color, the smell. So you pack only what matters for making it again into a recipe with a few numbers, such as “2 onions, 300 grams of meat, spiciness 3, simmer 30 minutes.” With that recipe, your friend can cook a similar curry. The few numbers that sum up the dish are like a point in latent space.

Similar dishes have similar recipes. The recipes for a hot curry and a mild curry differ in little more than the spiciness number. Change that number little by little from 3 to 5, and the curry gets a little hotter each time. It is the same as the picture changing little by little as you move from point to point in latent space.

But the comparison has limits. People choose the items in a recipe, and they have names such as “spiciness.” The numbers in latent space are set as the AI practices packing and restoring, and none of them has a name. Usually, even people can’t really tell what each number stands for. And latent space isn’t a real place, like a shelf of recipe books. It is a space in the mind’s eye: lists of numbers seen as “places.”

Key points

Why is it called “latent” and a “space”?

The English word latent comes from a Latin word meaning “lying hidden,” and it describes something that is present but hidden or not yet active, as in “latent ability” (power that hasn’t shown itself yet). The Japanese name, senzai kūkan, uses senzai, a word with the same meaning: existing hidden inside without showing on the surface.

The numbers in a photo stand for the brightness and color of each pixel, so you can see them directly. The numbers in latent space are not like that. The AI made them while learning; they don’t appear on the photo, yet they hold the photo’s important features hidden inside. That is why they are called latent.

It is called a space because the list of numbers is read as numbers that give a place. A place on graph paper is set by two numbers, across and up, and a place in a room by three: across, depth and height. In the same way, a list of 30 numbers can be seen as one place in a space with 30 directions. Seen as places, whether two lists are alike can be judged by how far apart the two points are.

But there is no real room-like place inside an AI. What is inside the AI is lists of numbers, and people picture them as a “space.” A space with four or more directions can’t be drawn by people.

Why squeeze things down?

To turn big data into a few numbers that keep only what matters. One neural network (a computing system modeled loosely on the brain’s nerve connections) built for this is the autoencoder. Its first half, the squeezer (encoder), packs the data into a few numbers, and its second half, the restorer (decoder), rebuilds the original data from those numbers. Both halves are trained together so that the gap between the original data and the rebuilt version gets smaller. A 2006 paper showed a way to train autoencoders well even when they stack many layers.

Because there are few numbers in the middle, the AI can’t keep everything. It has no choice but to pick and keep the features that matter for restoring. The 30 numbers it keeps are not picked from the original pixels; the AI makes them while learning. In the paper, small pictures of handwritten digits (28×28 pixels, 784 numbers) packed into 30 numbers could be restored far closer to the original than with a long-used method that also packed them into 30 numbers. This was checked on 10,000 new pictures, apart from the 60,000 used for training.

Pack into just two numbers, and you can draw the data on a flat chart. For handwritten digits, this gave a clearer picture of the data than packing into two numbers with the long-used method. The paper says packing data into a few numbers makes it easier to sort, show, send and store.

But what comes back is not exactly the same as the original. Fine details that don’t fit into a few numbers are lost.

How does the restoring part draw a picture from numbers?

It doesn’t draw lines one stroke at a time, the way people do with a brush. It computes the brightness of every pixel as a number. For handwritten digits, put in the 30 numbers of a point in latent space and out come 784 numbers. Set them out in 28 rows of 28, paint each pixel brighter the bigger its number, and you have a picture. For a color picture, it puts out three numbers per pixel: the strength of red, green and blue.

The computing is done by a neural network (a system that multiplies numbers by other numbers called “weights” and adds them up, in many stacked layers; more on the “Neural network” page). It multiplies the 30 numbers by weights and adds them up to make the numbers of the next layer, makes the layer after that from those, and so on, until the last layer gives 784 numbers.

People didn’t choose the weights. They were set, together with the squeezing part, by adjusting them little by little over many pictures so that the restored picture comes out close to the original. That is why putting in the point a learned picture was packed into gives back a picture very like it.

When making a new picture, the restoring part does the same thing. What differs is where the point comes from. A variational autoencoder (VAE) picks a point from around the middle of latent space (see “How do picture-making AIs use it?” below). Stable Diffusion starts from a point made of random numbers (noise) and, using the words of the request as a guide, guesses the noise mixed into the point and removes it bit by bit. The finished point is 4 sheets of 64×64 numbers (16,384 numbers). One pass through the restoring part turns it into a 512×512-pixel picture (three numbers per pixel, 786,432 numbers).

How is it different from an embedding?

The two words are very close and overlap a lot. An embedding is a list of numbers that an AI turns a word, sentence, picture or other item into, so that similar things come close together (more on the “Embedding” page). The space where those lists gather as points is called the embedding space. Some glossaries treat latent space as meaning the same as embedding space.

The difference is what each word points at. “Embedding” points at the list of numbers (the point) given to each item, and it comes up a lot when comparing words or finding text with a similar meaning. “Latent space” points at the space where the points gather. In research on picture-making AIs especially, the set of number lists that become pictures when passed through the restoring part has been called latent space. It is the word used when talking about moving around in that space or picking points to make new pictures.

It is used in research on language AIs too. A 2024 study had a language AI pass its in-between thinking on to its next step as lists of numbers inside the AI, without writing it out in words. The paper describes this as having the AI reason in a latent space instead of in human language (more on the “Neuralese” page).

What happens when you move between points?

Moving from point to point means changing the list of numbers of one point, little by little, into the list of the other. For example, the point exactly halfway between two points is the list you get by adding the numbers in the same position and dividing by 2. Passing the points along the way through the restoring part one by one lets you see the change as pictures.

In a well-learned latent space, moving little by little from point to point makes the picture smoothly change little by little too. A 2015 study checked this with an AI that makes pictures of bedrooms. The AI was a kind of GAN (a way to make pictures by pitting two AIs against each other), and it turns a point made of 100 numbers into a picture by passing it through its making part. The researchers picked 9 points at random and made pictures while moving little by little from one point to the next. The pictures changed smoothly, and every picture along the way looked like a bedroom. A room without a window slowly turned into a room with a giant window, and something that looked like a TV slowly turned into a window.

The researchers see this kind of change, where objects slowly appear or disappear, as a sign that the AI has learned a way of representing what is in the pictures.

Adding and subtracting worked too. In the same study, using the points of an AI that makes pictures of faces, the researchers averaged three points each for a smiling woman, a woman not smiling and a man not smiling, then calculated “smiling woman − woman not smiling + man not smiling.” Turned into a picture, that point gave the face of a smiling man. “Man with glasses − man without glasses + woman without glasses” gave the face of a woman with glasses (all of these are faces the AI made). Doing the same calculation directly on the brightness of the picture’s pixels gave messy, noisy pictures, with faces overlapping out of line.

In a latent space of just two numbers, if you set points evenly across it and turn each one back into a picture of a handwritten digit, the digit changes from place to place, and where neighbors meet you can see one digit smoothly turning into another. A 2013 paper also shows figures made this way.

How do picture-making AIs use it?

Pick a point in latent space and pass it through the restoring part, and you should get a new picture. The variational autoencoder (VAE), introduced in a 2013 paper, has a trick for this. It is trained so that the points it packs pictures into gather around the middle of the space and connect smoothly. Then you can make a new picture just by picking a point from around there and passing it through the restoring part.

Well-known picture-making AIs today also use latent space. A paper released in December 2021 ran a diffusion model (a method that makes a picture from pure noise by removing the noise a little at a time) not on a picture’s pixels but inside the latent space of an autoencoder trained beforehand (a latent diffusion model). Most of the information in a digital picture is fine detail people can barely see. The squeezing part drops that detail, and the model learns the content and layout of pictures in a small space. When making a picture, the point whose noise has been removed in latent space just needs one pass through the restoring part to become a picture. Once trained, the squeezing and restoring parts can be reused for several diffusion models.

Training the most powerful diffusion models that work on pixels could take hundreds of days of computing, counted as if done on a single GPU (a fast chip used for AI computing). Making 50,000 pictures took about 5 days on one GPU. In latent space there are fewer numbers to handle, so the computing is far lighter. But there is a sweet spot for how much to squeeze. In the paper, not squeezing at all, or shrinking the picture’s width and height only to 1/2, made training progress slowly, while shrinking them to 1/32 made picture quality stop improving early on. Shrinking the width and height to between 1/4 and 1/16 was the sweet spot between lightness and picture quality.

Stable Diffusion works this way too. Its squeezing part packs a picture into lists of numbers at 1/8 the picture’s width and height, and the noise work happens in that latent space. The words of the request are also turned into numbers and fed into the calculation that removes the noise. In training, the AI practiced guessing the noise mixed into the points for pictures, using the words that came with each picture as a guide, so the point is finished bit by bit into one that fits the words (more on the “Diffusion model” page).

What is still unclear, and what should you watch out for?

What each single number in latent space stands for is usually not clear to people. The numbers were set as the AI practiced packing and restoring; people didn’t name them. Sometimes their meaning can be guessed by looking into them, but not always (research that reads what goes on inside AIs is called interpretability).

Directions with a clear meaning, like a “smile direction,” can’t always be found neatly. In the 2015 study too, adding and subtracting with one face at a time gave unstable results, and became stable only when the points for three faces of each kind were averaged.

Similar things gathering close together is also a result of training; the AI didn’t arrange them by understanding their meaning the way people do. How close points are, and the directions in the space, don’t always match what people feel is “alike.”

Check yourself

  1. Q1An AI called an autoencoder practices, over and over, packing a picture of a handwritten digit (784 numbers) into 30 numbers and then turning it back into the original picture. What does the AI learn from this practice?

    Answer and explanation

    Answer: C A way to make the 30 numbers for restoring while learning

    With only 30 numbers in the middle, the AI can’t keep everything, so it comes to pick and keep the features that matter for restoring the picture. The 30 numbers are neither picked from the original pixels nor averages over fixed cells of the picture; the AI makes them while it learns to shrink the gap between the original and the restored picture. Fine details that don’t fit into a few numbers are lost, so the restored picture is not exactly the same as the original.

    More on this → Key points: Why squeeze things down?

  2. Q2In a 2015 study, researchers picked 9 points at random in the latent space of an AI that makes pictures of bedrooms, and made pictures while moving little by little from one point to the next. What were the pictures along the way like?

    Answer and explanation

    Answer: B All still looked like bedrooms, with windows and such changing

    Every picture along the way looked like a bedroom, and they changed smoothly: for example, a room without a window slowly turned into a room with a large window. The researchers see this kind of change, where objects slowly appear or disappear, as a sign that the AI has learned a way of representing what is in the pictures. Along the way, they never faded into some other kind of room.

    More on this → Key points: What happens when you move between points?

  3. Q3Why do picture-making AIs such as Stable Diffusion make pictures inside a latent space where the picture is squeezed down, rather than working on the picture’s pixels (tiny dots)?

    Answer and explanation

    Answer: A Fewer numbers to handle, so the computing is far lighter

    In a latent space where the picture is squeezed down, there are fewer numbers to handle, so the calculations for removing noise are far lighter. At the end, the finished point is passed once through the restoring part to become a full-size picture. Fine details people can barely see are actually dropped by the squeezing part. In the experiments of the latent diffusion paper, squeezing too much stopped picture quality from improving, and shrinking the width and height to between 1/4 and 1/16 was the sweet spot. And the latent space is not free of noise: the AI still starts from a point of random numbers (noise) and removes it bit by bit, using the words of the request as a guide.

    More on this → Key points: How do picture-making AIs use it?

Sources and how we checked

AI wrote this page from the sources above, and a different AI checked it against the text of those sources (November 13, 2026). A person with a master’s degree in science and a Japanese high-school science teaching license also read and checked it before publishing. Numbers are as of that date. If anything was hard to follow, you spot a mistake, or there’s a term you’d like explained, please write to [email protected].

AI terms

All 58 AI terms →