Embeddings — semantic vectors

What an embedding is

An embedding is a representation of text (a word, sentence or document) as a vector of numbers — a list of hundreds or thousands of numbers that encode its meaning. The core idea: texts with similar meaning get vectors that are close in the space, and different texts get distant vectors.

For example, "dog" and "puppy" will be close; "dog" and "car" will be far apart. The trick: closeness is measured by meaning, not by identical words. So embeddings-based search finds relevant results even when you didn't use the exact same words — this is what's called semantic search.

How similarity is measured — cosine similarity

To know how close two vectors are, you usually use cosine similarity — a measure that examines the angle between the vectors (not the distance). The result ranges from -1 to 1:

• 1.0 — completely identical in meaning
• ~0.8 — very similar
• ~0 — unrelated

In practice, a semantic search engine computes the embedding of the query, and returns the passages with the highest cosine similarity. This computation is done quickly by a Vector Database even over millions of vectors.

Models & dimensions

You don't produce embeddings yourself — you use a dedicated embedding model. The choice affects quality, speed and cost:

• OpenAI — text-embedding-3-small (1536 dimensions, cheap and fast) and text-embedding-3-large (3072 dimensions, more accurate). A good default for most.
• Open source / local — families like BGE, E5 and nomic. Free and private, running locally or via Hugging Face.
• Multilingual — for Hebrew, make sure the model has good multilingual support. Modern models usually handle Hebrew reasonably.

The number of dimensions is the length of the vector. More dimensions = more "resolution" but more storage and compute. Important: you must never mix embeddings from different models in the same index — they aren't in the same space.

Main uses

1. RAG — the basis for retrieving information before the model answers. The most common use.
2. Semantic search — searching a site/product by meaning, not just keywords.
3. Classification — grouping texts into categories by similarity.
4. Clustering — discovering topics/groups within a collection of texts.
5. Recommendations — "similar articles," "related products."
6. Deduplication — detecting duplicate or very similar content.

Chunking — splitting documents correctly

You don't embed a whole document as one vector — you split it into chunks and embed each one. This is critical for RAG quality: a chunk that's too big dilutes the meaning; too small loses context.

• Typical size: 300–800 tokens per chunk, with an overlap of ~10–15% so you don't cut sentences in the middle.
• Smart splitting: prefer natural boundaries (paragraphs, headings) over arbitrary cuts.
• Metadata: store a source, title and date for each chunk — useful for filtering and citation.

Code example — basic semantic search

from openai import OpenAI
import numpy as np
client = OpenAI()

def embed(texts):
    r = client.embeddings.create(model="text-embedding-3-small", input=texts)
    return [d.embedding for d in r.data]

def cosine(a, b):
    a, b = np.array(a), np.array(b)
    return a @ b / (np.linalg.norm(a) * np.linalg.norm(b))

docs = ["how to reset a password", "support opening hours", "refund policy"]
doc_vecs = embed(docs)

query = "I forgot my password"
q_vec = embed([query])[0]

scores = [(cosine(q_vec, dv), d) for dv, d in zip(doc_vecs, docs)]
scores.sort(reverse=True)
print(scores[0])   # ('...0.86...', 'how to reset a password')

Note: the query didn't contain the word "reset," but the search found the right passage — because it understood the meaning. In production, this computation is done efficiently at scale by a Vector DB.

Common mistakes

• Mixing models. You must not search with one embedding model against an index built with another.
• Bad chunks. Most RAG problems are actually chunking problems, not model problems.
• Ignoring normalization. If the DB doesn't normalize, compare with cosine, not dot product.
• Not storing metadata. Without a source per chunk you can't cite or filter.
• Semantic search only. Combining with keyword search (hybrid) usually gives better results — see Advanced RAG.