AI Solutions Architect Bootcamp — Day 1, Lesson 5: Embeddings and Vector Search
You've now mastered the foundations of LLM inference and production performance. Today we'll tackle the technology behind semantic search, intelligent memory retrieval, document discovery, and RAG.
Lesson 5 of 7 · 40–50 minutes · AWS Bedrock · AI Solutions Architect level
The central question: how can an AI application find information that is meaningfully related to a user's question, even when the words don't match?
1. Why traditional keyword search isn't enough
Imagine you're building an intelligent memory application. A user previously saved this note: "My automobile coverage expires on December 15." A month later, the user asks: "When does my car insurance end?"
| Stored information | User's question |
|---|---|
| automobile | car |
| coverage | insurance |
| expires | end |
A basic keyword search might miss the connection. A semantic search system can recognize that these expressions are related in meaning. That's where embeddings come in.
2. What is an embedding?
An embedding is a numerical vector that represents information in a learned mathematical space. A text embedding model converts text into a list of numbers — for example, "My car insurance expires soon" through Amazon Titan Text Embeddings might become [0.12, -0.45, 0.78, 0.03, ...] (illustrative values, not actual model output).
A vector is simply an ordered collection of numbers. A three-dimensional vector looks like v = [0.2, 0.7, -0.4]. Real embedding models commonly produce vectors with hundreds or thousands of dimensions. Each dimension is a learned numerical feature — you generally cannot assign a reliable human-readable meaning to one individual coordinate.
Two different uses of embeddings
Remember Lesson 1's internal token embeddings? There is an important distinction:
- Internal token embeddings help a language model process token IDs.
- Retrieval embeddings represent text, documents, or other content so a search system can compare them.
They're related mathematical ideas, but they're not interchangeable components.
3. How semantic similarity works
Imagine we embed three statements: (A) "My car insurance expires next month." (B) "My automobile policy ends soon." (C) "The restaurant serves excellent pizza." A well-trained embedding model should ideally place A and B closer together in semantic space than A and C.
Conceptual illustration. Actual text embeddings are high-dimensional, and a plotted projection does not fully preserve their geometry.
The search system doesn't need to understand the English meaning of each coordinate — it compares vectors mathematically. That brings us to cosine similarity.
4. Cosine similarity — the mathematics and an interactive explorer
Cosine similarity measures the angle between two nonzero vectors: cosine(a, b) = (a · b) / (∥a∥ × ∥b∥). There are three pieces:
- Dot product — multiply corresponding coordinates and add them.
- Magnitude — calculate the length of each vector.
- Normalize — divide the dot product by the product of their magnitudes.
For ordinary real-valued vectors, cosine similarity ranges from -1 to +1: +1 same direction, 0 perpendicular directions, -1 opposite directions. In semantic search, a higher cosine similarity often indicates greater relevance, but it is not a calibrated probability of correctness.
Interactive vector mathematics
Cosine Similarity Explorer
Adjust vectors a and b and watch the dot product, magnitudes, and cosine similarity update live.
— vector a — vector b
When the vectors point in the same direction, their cosine similarity is near +1. When they're perpendicular, it's 0. When they point in opposite directions, it's -1.
Worked example
Suppose a = [1, 2] and b = [2, 4]. Dot product: (1×2) + (2×4) = 10. Magnitudes: ∥a∥ = √5, ∥b∥ = √20. Therefore cosine(a, b) = 10 / (√5 × √20) = 1. The vectors point in the same direction, even though one is twice as long.
Architect insight: cosine similarity focuses on direction rather than magnitude. That's one reason it is widely used in embedding search. But choosing cosine similarity versus dot product or Euclidean distance should follow the embedding model's recommendations and your retrieval evaluation results.
5. What is a vector database?
A vector database stores embeddings and supports searches for nearby vectors. In a real application, each record should contain more than just a vector — for example:
{
"id": "memory_1042",
"user_id": "user_17",
"tenant_id": "tenant_8",
"text": "My automobile insurance expires December 15.",
"embedding": [0.12, -0.45, 0.78],
"source": "user_confirmed",
"updated_at": "2026-10-15"
}The vector shown is abbreviated and illustrative. When a user asks a question, the system can: (1) convert the question into an embedding, (2) search the index for similar vectors, (3) filter results using authorization and metadata, (4) retrieve the original text or structured records, (5) return relevant results to the application. This is vector retrieval, not yet a complete RAG system — we'll connect retrieval to answer generation in Lesson 6.
Exact search vs vector search
| Query | Better starting approach |
|---|---|
| Find policy ABC123 | Exact database lookup |
| Find documents discussing automobile coverage | Semantic vector search |
| Find policies that expire before December 31 | Structured database filter |
| Find complaints similar to this customer's description | Vector similarity search |
| Find documents about refunds that contain a specific product code | Hybrid keyword + vector search |
An enterprise architect should not replace every SQL query with vector search. Use the retrieval mechanism that matches the task.
6. Approximate Nearest Neighbor (ANN) search
Suppose you have 10 million stored embeddings. A naive approach would calculate similarity against every vector for each query — that's called exhaustive or exact nearest-neighbor search, and at scale, this can be expensive.
Vector search systems often use Approximate Nearest Neighbor (ANN) indexes to find highly similar vectors without scanning the entire dataset. One common indexing technique is HNSW (Hierarchical Navigable Small World) — think of it as building navigable connections between nearby vectors, allowing the search engine to explore promising regions efficiently.
The trade-off: exact search aims to find the true nearest neighbors, while ANN search often provides much faster retrieval, potentially missing some true nearest neighbors. Architects evaluate recall, latency, cost, index size, and update behavior when choosing an ANN configuration.
7. AWS vector database options
| AWS service | When I'd consider it |
|---|---|
| Amazon OpenSearch Serverless | Search-centric workloads requiring vector search, filtering, and hybrid retrieval |
| Amazon Aurora PostgreSQL with pgvector | Applications already using relational data, joins, and transactional records |
| Amazon S3 Vectors | Large, cost-sensitive vector collections with relatively infrequent queries |
| Amazon Bedrock Knowledge Bases | Managed ingestion and retrieval orchestration rather than building every pipeline component yourself |
Amazon Bedrock Knowledge Bases supports vector stores including OpenSearch Serverless, Aurora PostgreSQL, and S3 Vectors, with capabilities and limitations varying by backend. AWS's September 2026 guidance also emphasizes that the right backend depends on workload patterns, retrieval features, latency, and cost.
What would I choose for a memory-enabled application?
If your application already stores user memories in PostgreSQL, I would first evaluate PostgreSQL with pgvector rather than immediately introduce a separate vector database. Why? You can keep structured memory, ownership metadata, and embeddings in the same database, reducing synchronization complexity.
But I would benchmark this against your expected data volume and search concurrency. If search becomes the dominant workload, OpenSearch or another dedicated vector service may be a better fit.
8. Hands-on Python: generate embeddings with Amazon Bedrock
We'll use Amazon Titan Text Embeddings V2. AWS documents support for 256-, 512-, and 1,024-dimensional embeddings, with configurable normalization.
Step A — create an embedding
import boto3
import json
bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")
MODEL_ID = "amazon.titan-embed-text-v2:0"
def embed(text: str) -> list[float]:
response = bedrock.invoke_model(
modelId=MODEL_ID,
contentType="application/json",
accept="application/json",
body=json.dumps({
"inputText": text,
"dimensions": 512,
"normalize": True
})
)
data = json.loads(response["body"].read())
return data["embedding"]
vector = embed("My car insurance expires next month.")
print("Dimensions:", len(vector))
print("First five values:", vector[:5])This requires an AWS account, valid credentials, model availability, and the relevant Bedrock permissions. Invocations may incur charges.
Step B — compare three sentences
import numpy as np
sentences = [
"My car insurance expires next month.",
"My automobile policy ends soon.",
"The restaurant serves excellent pizza.",
]
vectors = [np.array(embed(text)) for text in sentences]
def cosine_similarity(a, b):
denominator = np.linalg.norm(a) * np.linalg.norm(b)
if denominator == 0:
raise ValueError("Cannot compare zero vectors")
return float(np.dot(a, b) / denominator)
query = vectors[0]
for text, vector in zip(sentences, vectors):
score = cosine_similarity(query, vector)
print(f"{score:.4f} {text}")What should you observe?
The first sentence compared with itself should produce a cosine similarity very close to 1. You would generally expect the second sentence to rank above the unrelated pizza sentence. But the actual scores and rankings are empirical — don't assume a particular numeric similarity threshold without evaluating the model on representative data.
Important engineering rule
Use the same embedding model and compatible embedding configuration for the stored documents and incoming queries. If you switch embedding models, you generally need to regenerate your stored vectors — different embedding spaces are not automatically comparable.
9. Build a tiny vector search engine without a vector database
Before using OpenSearch or pgvector, understand the retrieval algorithm. The following example is deliberately small and runs in memory.
import numpy as np
documents = [
"My car insurance expires on December 15.",
"My mortgage payment is due on the first.",
"The restaurant reservation is for Friday.",
"My automobile policy renewal is pending.",
]
# Uses the embed() function defined above.
document_vectors = np.array([embed(document) for document in documents])
query = "When does my vehicle coverage end?"
query_vector = np.array(embed(query))
# Vectors are normalized by the embedding model.
scores = document_vectors @ query_vector
top_indices = np.argsort(scores)[::-1][:2]
for index in top_indices:
print({
"score": round(float(scores[index]), 4),
"document": documents[index]
})This is an exact in-memory vector search. For four documents, that's fine. For millions of documents, you'd typically use an indexed vector search engine with filtering, persistence, and operational controls.
10. Enterprise reference architecture: semantic memory retrieval
Ingestion pipeline: capture (document / conversation) → validate + assign user/tenant ownership → chunk text when needed → Bedrock embedding model → store text + vector + metadata.
Search pipeline: authenticated user question → Bedrock query embedding → authorized vector search (tenant filter + similarity ranking) → relevant records + source metadata.
Five production rules
- Authorization before exposure — enforce tenant and user permissions in the retrieval path, not merely by asking the LLM to ignore unauthorized results.
- Metadata matters — store document IDs, source references, timestamps, and ownership alongside vectors.
- Handle edits and deletes — when an authoritative record changes, update or remove its derived embeddings and indexes.
- Evaluate retrieval quality — measure recall@k, precision@k, relevance, and the effect of metadata filters.
- Use hybrid retrieval when needed — exact identifiers and keyword matches can complement semantic similarity.
For an enterprise workflow built on a business rules or case management engine, the search result is supporting evidence. The authoritative business system still determines case status, eligibility, and approval rules.
Lesson 5 — Architect Knowledge Check
Lesson 5 Assessment
You need 4 out of 5 to progress to Lesson 6.
The core lesson: embeddings make semantic retrieval possible, but embeddings alone are not a knowledge system. Production quality comes from the combination of good representations, indexing, access control, metadata, retrieval evaluation, and authoritative data management.
Next in this series: Lesson 6 — Retrieval-Augmented Generation (RAG), where we turn this retrieval engine into a grounded AI assistant.
No comments:
Post a Comment