Vector databases and embeddings, without the mystique

Embeddings and vector databases are the plumbing under RAG, semantic search, and most “AI that understands our content” features. They're less mysterious than the names suggest, and worth understanding because this is where a lot of quality is won or lost.

What an embedding is

An embedding turns a piece of text into a list of numbers that captures its meaning. Texts that mean similar things end up with similar numbers — close together in a high-dimensional space. That's the whole trick: meaning becomes geometry, and “similar” becomes “nearby”. It's why semantic search can match “how do I cancel” to a document titled “terminating your subscription”, even with no shared words.

What a vector database does

A vector database stores those embeddings and answers one question very fast: given this query, which stored items are nearest in meaning? Across millions of passages, doing that quickly and accurately is a genuine engineering problem, which is what these specialised databases exist to solve.

Where teams get it wrong

Three places, usually. They pick an embedding model without testing it on their actual content — a model tuned for general text can be mediocre on legal or medical language. They ignore chunking, so the wrong-sized passages get embedded and retrieval suffers. And they treat similarity as truth: the nearest passage isn't always the right one, which is why re-ranking and thresholds matter.

The practical takeaway

You don't need to build any of this from scratch — the tooling is mature. What you do need is to choose the embedding model deliberately, test retrieval on your real data, and resist the assumption that “closest” means “correct”. Get those right and the rest of your AI search quietly works.