How Data Is Stored and Retrieved from a Vector Database

The Ingestion Flow

Documents
   ↓
Split Into Chunks
   ↓
Embedding Model converts Chunk → Vector
   ↓
Store (Chunk Text + Vector + Metadata) in Vector DB

Each stored record contains three components:

Component	Description
Vector	Floating-point array (e.g., 1536 dimensions)
Chunk Text	The original text segment
Metadata	Source, author, date, category, access roles

The Query Flow

User Query
   ↓
Convert Query → Embedding (same model as ingestion)
   ↓
Similarity Search in Vector DB
   ↓
Retrieve Most Similar Chunks
   ↓
Pass Chunks to LLM as Context
   ↓
LLM Generates Final Answer

The Critical Misconception: Exact Match vs. Similarity

Vector databases do not search for exact embedding values. They perform nearest-neighbor similarity searches — finding vectors closest in semantic space.

Example

• Query Vector: [0.21, -0.44, 0.91, 0.03, ...]
• Stored Chunk Vector: [0.22, -0.41, 0.89, 0.04, ...]

These values differ, yet represent semantically similar content. The similarity score (cosine similarity or dot product) determines the match — not exact value equality.

Why This Matters for RAG Design

Because embeddings are continuous floating-point vectors:

1. Same meaning, different wording → high similarity score ✓
2. Same words, different context → lower similarity score
3. Exact keyword match irrelevant → semantic proximity is what counts

Similarity Metrics

Metric	Best For
Cosine Similarity	Normalized vectors, text embeddings
Dot Product	When magnitude matters
Euclidean Distance	Low-dimensional spaces

Practical Implication

When a user asks "How do I reset my password?" and your document says "Account recovery steps" — a vector search finds the match. A keyword search would not.