Naru News日本語

Home › Glossary › AI

What is an embedding in AI?

Also called: Embeddings, Vector embedding, Embedding vector, Embedding (vector), AI embedding

Term explainer · Checked November 12, 2026 · 12 min read

In one minute

An embedding in AI is a list of numbers that a word, sentence, picture or other item is turned into, so that similar things end up close together; the word also means turning them into such lists.

AI works by calculating with numbers. Text-writing AIs such as ChatGPT also cut text into small pieces (tokens) and replace each one with a list of hundreds to thousands of numbers before calculating. That list is the embedding. People don’t choose the numbers; they are set as the AI learns from lots of text.

Words used in similar ways get similar lists. Words used in similar ways usually have similar meanings, so words with similar meanings gather close together. But not always: opposites such as “strong” and “weak” also end up close.

A place on a map is fixed by two numbers, across and up–down. An embedding is like a map where each place is fixed by hundreds of numbers, a map no one can see. Using this closeness, you can find text with a similar meaning even when it shares no words.

A common mix-up

“An embedding is just a number an AI gives to each word.”

Only partly right. An AI cuts text into tokens (small pieces of words) and replaces each one with a number. Up to that point, it really is just numbering. But the numbers themselves carry no link in meaning: even if “cat” and “dog” had neighboring numbers, that wouldn’t mark them as similar. An embedding is a list of hundreds of numbers assigned to each of those token numbers. During training, the numbers in each list are adjusted little by little, and words used in similar ways come to have similar lists. The token number is a name tag that tells words apart; the embedding is a set of numbers that also shows how close words are.

Explained in pictures

A sketch of lists of numbers for words becoming places on a map (drawn with just two numbers per word; numbers and positions made up for this page; real embeddings use hundreds to thousands of numbers). Title: “Words used alike sit close together.” Subtitle: “Sketch: with just two numbers, each list is a spot on a flat map.” Left panel, headed “Token” and “Embedding”: arrows lead from the tags “cat,” “dog” and “apple” to two boxes of numbers each, under the column heads “1st” and “2nd” (cat −0.7, 0.7; dog −0.6, 0.5; apple 0.3, 0.6). Below: “Really hundreds to thousands,” “(a map no one can see)” and “Used alike, similar numbers.” An arrow leads to the map on the right. Map: an across axis, “Across: 1st number” (−1, 0, 1), and an up axis, “Up: 2nd number” (−1, 0, 1). Orange dotted lines run from the cat point to −0.7 on the across axis and 0.7 on the up axis. A dashed outline labeled “Animals” holds cat, dog and horse (navy dots), “Fruit” holds apple, lemon and grapes (white diamonds), and “Vehicles” holds train, bus and bicycle (orange squares). Every point has its name right next to it. Band at the bottom: Each word becomes a point on a map of numbers.
This sketch shows words becoming embeddings (lists of numbers) and lining up in a map-like space, drawn with just two numbers per word. On the left are the lists for each token (piece of text). Read the first number as across and the second as up–down, and each list becomes a point on the map at right (cat is −0.7 across and 0.7 up). Cat and dog are used in similar ways, so their numbers are similar and they sit close on the map too. Real embeddings fix a place with hundreds to thousands of numbers, so they make a map no one can see. A single number usually has no easy-to-understand meaning.
A sketch of finding the text closest to a question with embeddings (texts and positions made up for this page; the map is a 2D sketch). Title: “Find the text nearest the question.” Subtitle: How search works in RAG and more. Top left, “① Turn texts into numbers”: three cards, A “Foods unsafe for felines,” B “Naming your cat” and C “Mushrooms you can’t eat.” Bottom left, “② Same for the question”: an orange bubble, “What can’t my cat eat?” An arrow leads to the map on the right, “③ Pick the closest text”: right next to an orange star, “Question,” is navy dot A (labeled “Closest” above and “unsafe foods” below, circled with a dashed line), joined to the star by a solid orange line marked “near.” Far away are navy dots B (“naming”) and C (“mushrooms”), joined to the star by gray dotted lines marked “far.” Under the map: “The AI reads A with the question and answers.” Band at the bottom: No shared words needed: close meaning is enough.
This sketch shows how embeddings are used to find texts that fit a question. The star is the question and the dots A, B and C are texts, each placed where its list of numbers puts it. The texts are turned into lists ahead of time and stored; when a question comes in, it is turned into a list the same way, and the closest text is picked. B shares the word “cat” with the question and C shares “can’t eat,” yet the closest in meaning is A, which shares no words with it. In retrieval-augmented generation (RAG), the chosen text is given to the AI along with the question, and the AI writes the answer.
A sketch of the example “King − Man + Woman” from a 2013 paper (2D; positions made up for this page; real word lists have many more numbers). Title: “King − Man + Woman lands very close to Queen.” Key at the top: navy dot = where a word is, orange star = calculated point. On the left, a dotted orange arrow labeled “man→woman shift” runs from the point “Man” up and right to the point “Woman.” On the right, from the point “King,” a dotted orange arrow of the same direction and length, also labeled “man→woman shift,” ends at an orange star labeled “calculated.” Right next to the star is the point “Queen,” and a dashed circle around the star and Queen is labeled “very close.” Bottom left: “No word exactly there? Take the closest word.” Band at the bottom: The man–woman relation shows up as the same arrow.
This sketch shows an example of subtracting and adding with the number lists of words. The points are where words sit, and the orange arrows are the shift from Man to Woman. Very close to the spot reached by moving from King in the same direction and by the same length (the calculated point for King − Man + Woman) was Queen. During training, the difference between male and female had been picked up as a shift with a set direction. Because there may be no word exactly at the calculated point, the closest word is taken as the answer.

You’re free to use these figures in lessons, slides and posts if you credit Naru News (naru.news). CC BY 4.0.

Think of it like this

A supermarket where similar goods sit side by side

In a supermarket, similar goods are shelved near each other. Next to the milk you find yogurt and cheese, and near the apples you find oranges. So even in a store you’ve never been to, once you know where the milk is, you can guess roughly where the cheese is.

An embedding is like giving each word a spot in such a store, with its address written in numbers. Words used in similar ways are placed close together. To find text that fits a question, you just look at the shelves near the spot where the question lands.

But the comparison has limits. In a store, people decide the layout, and aisles have names such as “Dairy.” In an embedding, the layout is set by training, not by people, and each of the numbers that fix a spot usually has no name. A store has a flat floor, but an embedding’s spots lie in an invisible space defined by hundreds to thousands of numbers. And words with opposite meanings, such as “strong” and “weak,” tend to be placed close together, because they are used in similar sentences, like “Our team is strong today” and “Our team is weak today.”

Key points

Why is it called an “embedding”?

To embed something is to set it firmly inside something else. In AI, placing a word, sentence or picture as a single point inside an invisible space made of many numbers is called embedding it. That space is called the embedding space, and the list of numbers that gives the point’s location is the embedding, also called an embedding vector. A vector is a quantity with a direction and a size, and it can be written as a list of numbers. On a map, for example, an arrow that goes “3 east and 2 north” can be written as the list of two numbers 3 and 2.

How many numbers the list holds is called its number of dimensions. A place on a map is fixed by two numbers, east–west and north–south, so a map is two-dimensional. Word embeddings often use lists of 256, 512 or 1,024 numbers. People can’t draw spaces with four or more dimensions, so explanatory figures show only how close things are, on a two-dimensional map. Just as with two points on a map, how close two words are is measured by how far apart their lists of numbers are (there are several ways to measure this).

It has nothing to do with other kinds of “embedding.” Embedding a video or a map in a web page, or the computers built into appliances and cars (embedded systems), come from the same English word, embed, but they mean different things. On this page, an embedding is the list of numbers that stands for a word or picture inside an AI.

How is it different from a token?

A token is a small piece of text that an AI cuts sentences into when it reads (sometimes a whole word such as “hello,” sometimes only part of a word or character). The AI gives each token a number. The embedding is the list of numbers prepared for each of those token numbers. Inside the AI there is a large table with one row of numbers for each token number, and the AI looks up the row by the number. The numbers in the table are set by training.

In the 2017 paper in which Google researchers introduced the Transformer (the design that most of today’s text-writing AIs use), incoming tokens are turned into lists of 512 numbers with learned embeddings. A list of the same length that marks each token’s position is added to it. Adding two lists means adding the numbers in the same places: the first to the first, the second to the second, and so on. After that, the same kind of calculation is repeated many times.

At the entrance, each token has just one embedding. But as the calculation is repeated, the lists change to fit the surrounding words. The English word “orange,” for example, can mean a color or a fruit, and it ends up with a different list in each sentence. Embeddings like these, which change with the sentence, are called contextual embeddings.

How are the numbers decided?

People don’t set them one by one; training does. A table that looks up a list of numbers for each token number is placed at the AI’s entrance, and the AI is given lots of practice problems. So that it answers them correctly, the numbers in the table are adjusted little by little along with the rest of the AI.

A well-known example is word2vec, published in 2013 by Google researchers. Its practice problems use lots of text: guessing a word from the words around it, or guessing the surrounding words from the word in the middle. In the paper, it was trained on news articles with about 6 billion tokens.

With this practice, words that tend to appear surrounded by similar words end up with similar lists. In “They rode a burro down into the Grand Canyon” and “They rode a horse down into the canyon,” for example, burro and horse appear in similar sentences, so they end up close together. Nobody taught it meanings; the closeness grows out of words being used in similar ways.

Explanatory figures sometimes line words up on axes with names such as “how animal-like” or “how big.” But in real embeddings set by training, a single number rarely has such an easy-to-understand meaning. Sometimes you can guess what one stands for, but not always.

Can you really add and subtract words?

To some extent, yes. In a 2013 paper, with word lists learned in training, taking “King,” subtracting “Man” and adding “Woman” gave a list very close to that of “Queen.” The subtracting and adding are done on the numbers in the same places. The difference between male and female had been picked up automatically as a shift in a set direction. The system they trained had been taught nothing about grammar or meaning.

There may be no word exactly at the calculated spot, so the answer is the word closest to it. In another paper the same year, “Madrid − Spain + France” came closer to “Paris” than to any other word, and “Germany + capital” came close to “Berlin.”

But it doesn’t always work. In the paper with the King example, the system was given 8,000 questions about grammar relationships, such as “good is to better as rough is to what?”, and it answered close to 40% of them correctly.

What is it used for?

Besides its role inside text-writing AIs, its best-known use is searching by meaning. Ahead of time, documents are cut into short pieces, each piece is turned into a list of numbers, and the lists are stored. When a question comes in, it is turned into a list the same way, and the pieces with the closest lists are pulled out. Even if they don’t use the same words, pieces with a similar meaning can be found.

A 2020 paper used this to find passages in Wikipedia that contain the answer to a question. Wikipedia was split into about 21 million short passages, and questions and passages were each turned into lists of numbers. On most of the test sets, the share of questions with a passage containing the answer among the top 20 search results was 9 to 19 points higher than with a method that looks for exact word matches. The paper says, for example, that it can better match “bad guy” in a question with “villain” in a passage.

In the same year, the paper that named retrieval-augmented generation (RAG), where an AI looks up and reads documents before answering, also used an index of Wikipedia turned into lists of numbers in this way (more on the “Retrieval-augmented generation (RAG)” page).

Other uses include grouping similar things, picking recommendations, spotting things that are far from all the others, and sorting things into set categories. OpenAI’s AIs that make embeddings of text usually return a list of 1,536 or 3,072 numbers (the lists can also be made shorter).

What are its weak points?

Closeness is learned from how words are used, so it doesn’t always match the meaning people feel. For example, “strong” and “weak” have opposite meanings, but they are used in similar sentences, such as “Our team is strong today” and “Our team is weak today.” So embeddings learned from usage often fail to tell words with opposite meanings apart.

In embeddings that give each word only one list, a word with several meanings can only be shown as a single point. Contextual embeddings, whose lists change with the sentence, can show the same word with a different list in each sentence.

Leanings in the training text are copied too. A 2016 paper examined an embedding trained on Google News articles. Asked to complete “man is to computer programmer as woman is to x,” it answered “homemaker.” The paper warns that because such embeddings are so widely used, they tend not just to copy the gender stereotypes in text but to spread them further.

Search has weak points too. When an exact word matters, such as a rare name or a code number, searching for exact word matches can do better. So the two ways are sometimes combined. In the 2020 paper that searched Wikipedia for passages containing answers, combining them improved the results on some tasks.

Check yourself

  1. Q1An AI embedding places a word as a single point in a map-like space. Yet this space can’t be drawn the way an ordinary map can. Why?

    Answer and explanation

    Answer: B Each point’s place is set by hundreds of numbers

    In an embedding, each word’s place is set by a list of hundreds to thousands of numbers. A place on a map is set by two numbers and a place in a room by three, but people can’t draw a space where places are set by four or more numbers. So explanatory figures use just two numbers and show only how close things are on a flat map. Lots of points would not stop you from drawing a flat map, and the places are set by learned numbers, not by the shapes of the letters.

    More on this → Key points: Why is it called an “embedding”?

  2. Q2Retrieval-augmented generation (RAG) is a system where an AI looks up and reads documents before answering. When it searches with embeddings, how does it choose the documents that fit a question?

    Answer and explanation

    Answer: C It turns the question into an embedding too and picks the closest

    The documents are each turned into an embedding (a list of numbers) ahead of time and stored. When a question comes in, it is turned into a list the same way, and the documents with the closest lists are pulled out and given to the AI along with the question. Because the search goes by closeness in meaning, documents that don’t use the same words can still be found. The stored embeddings aren’t adjusted for each question. Picking the documents where the question’s words appear most is a different method that matches words directly, not one that uses embeddings.

    More on this → Key points: What is it used for?

  3. Q3“Strong” and “weak” have opposite meanings. In word embeddings learned from lots of text, where do these two words tend to end up?

    Answer and explanation

    Answer: A Close together, because they appear in similar sentences

    Words with opposite meanings are used in similar sentences, such as “Our team is strong today” and “Our team is weak today.” Embeddings learn from how words are used, so words used in similar ways end up close together, and opposites tend to end up close too. So such embeddings often fail to tell words with opposite meanings apart. Closeness in an embedding is not always the same as the closeness in meaning people feel.

    More on this → Key points: What are its weak points?

Sources and how we checked

AI wrote this page from the sources above, and a different AI checked it against the text of those sources (November 12, 2026). A person with a master’s degree in science and a Japanese high-school science teaching license also read and checked it before publishing. Numbers are as of that date. If anything was hard to follow, you spot a mistake, or there’s a term you’d like explained, please write to [email protected].

AI terms

All 58 AI terms →