Chapter 11: Embeddings, Vector Databases & Semantic Search

Introduction

Welcome to Chapter 11! In the previous chapters, you’ve built a solid foundation in deep learning, neural networks, and training workflows. You’ve learned how models process data, but how do we make sense of unstructured data like text or images in a way that machines can truly “understand” their meaning and relationships? This is where embeddings come into play.

This chapter will introduce you to embeddings, which are numerical representations that capture the semantic meaning of data. We’ll then explore vector databases, specialized tools designed to store and efficiently query these embeddings. Finally, we’ll combine these concepts to build powerful semantic search capabilities, moving beyond simple keyword matching to understanding the intent behind a query. This knowledge is fundamental for building advanced AI applications, especially with Large Language Models (LLMs) and Retrieval Augmented Generation (RAG) systems.

To get the most out of this chapter, you should have a basic understanding of Python, some familiarity with neural networks, and perhaps a general idea of how Natural Language Processing (NLP) models work. Let’s dive in and unlock a new dimension of machine understanding!

Core Concepts: Understanding Semantic Meaning

Imagine you want a computer to understand that “car” and “automobile” are very similar, or that “king” is to “man” as “queen” is to “woman.” Traditional text processing struggles with this nuanced understanding. Embeddings provide a solution.

What are Embeddings?

At its heart, an embedding is a numerical representation (a vector of numbers) of an object, such as a word, sentence, image, or even a user. The magic lies in how these numbers are chosen: objects with similar meanings or characteristics are mapped to vectors that are “close” to each other in a high-dimensional space.

Think of it like this:

• Each word or piece of data is a point in a vast, invisible space.
• Words with similar meanings (like “cat” and “kitten”) are located very near each other.
• Words with opposite meanings (like “hot” and “cold”) might be far apart.
• Interestingly, relationships can also be encoded. The vector difference between “king” and “man” might be similar to the vector difference between “queen” and “woman.”

These embeddings are typically generated by sophisticated deep learning models (like BERT, Word2Vec, or Sentence-Transformers) that have been trained on massive datasets. The models learn to project complex data into this simplified, yet semantically rich, vector space.

Why are they important?

1. Semantic Understanding: They allow machines to grasp the meaning rather than just the literal words.
2. Feature Representation: They transform categorical or unstructured data into numerical features that machine learning models can easily process.
3. Similarity Search: They enable finding similar items quickly by comparing vector distances.

The Power of Vector Space

The beauty of embeddings lies in the mathematical properties of vector spaces. We can use distance metrics (like cosine similarity or Euclidean distance) to quantify how “similar” two embeddings are. A smaller distance or higher cosine similarity usually indicates greater semantic resemblance.

For example, if you embed a sentence like “I want to buy a new car” and another like “I’m looking for an automobile to purchase,” their embeddings would be very close. But a sentence like “I want to eat an apple” would have an embedding much further away.

Why Vector Databases?

You’ve learned about traditional relational databases (like SQL) and NoSQL databases (like MongoDB). These are excellent for structured data and keyword lookups. However, they are not designed for the unique challenge of efficiently storing and searching millions or billions of high-dimensional vectors based on similarity.

This is where vector databases step in. They are specialized databases optimized for storing, indexing, and querying vectors. Instead of exact matches, they perform Approximate Nearest Neighbor (ANN) searches, which quickly find vectors that are “close enough” to a query vector.

Key advantages of Vector Databases:

• Efficiency: Designed for fast similarity search using algorithms like HNSW (Hierarchical Navigable Small World) or IVF (Inverted File Index).
• Scalability: Can handle massive numbers of vectors and high query loads.
• Metadata Handling: Often allow associating metadata with vectors, enabling filtered searches.
• Real-time Applications: Power real-time recommendation engines, semantic search, and RAG systems.

Popular vector database options include Pinecone, Weaviate, Milvus, Qdrant, and ChromaDB. For our hands-on example, we’ll use ChromaDB due to its ease of use for local development and its growing popularity as of early 2026.

Semantic Search Explained

Semantic search is a search technique that goes beyond matching keywords. Instead, it aims to understand the meaning and context of a user’s query and retrieve results that are semantically relevant, even if they don’t contain the exact keywords.

The workflow for semantic search using embeddings and a vector database looks like this:

1. Data Ingestion:

   • Take your raw data (documents, product descriptions, images).
   • Use an embedding model to convert each piece of data into a vector embedding.
   • Store these embeddings (along with their original data or a reference to it) in a vector database.

2. Query Processing:

   • A user submits a query (e.g., “latest advancements in quantum computing”).
   • Use the same embedding model to convert the user’s query into a query vector.

3. Similarity Search:

   • Send the query vector to the vector database.
   • The vector database efficiently finds the top-K most similar vectors (and their associated data) to the query vector.

4. Result Retrieval:

   • Present the semantically relevant results to the user.

Here’s a visual representation of this process:

Figure 11.1: Semantic Search Architecture

This architecture is incredibly powerful and forms the backbone of many modern AI applications, including enhanced search engines, recommendation systems, and advanced LLM applications like RAG where an LLM can retrieve relevant context before generating a response.

Step-by-Step Implementation: Building a Simple Semantic Search

Let’s get hands-on and build a basic semantic search system using sentence-transformers for embeddings and ChromaDB as our vector database.

Prerequisites & Setup

First, ensure you have a Python environment (Python 3.10+ recommended as of early 2026).

We’ll need to install the necessary libraries:

• sentence-transformers: To generate text embeddings. (Current stable: ~2.2.2)
• chromadb: Our lightweight, easy-to-use vector database. (Current stable: ~0.4.24)

Open your terminal or command prompt and run:

pip install sentence-transformers~=2.2.0 chromadb~=0.4.20

Explanation:

• pip install: The standard Python package installer.
• sentence-transformers~=2.2.0: Installs sentence-transformers with a version compatible with 2.2.0 (e.g., 2.2.0, 2.2.1, 2.2.2). This ensures stability while allowing minor bug fixes.
• chromadb~=0.4.20: Installs chromadb with a version compatible with 0.4.20.

Step 1: Generating Embeddings

We’ll start by taking a few sentences and converting them into embeddings. We’ll use a pre-trained model from sentence-transformers. A good general-purpose model is all-MiniLM-L6-v2.

Create a new Python file named semantic_search_app.py.

# semantic_search_app.py

# 1. Import necessary libraries
from sentence_transformers import SentenceTransformer
import chromadb

print("Step 1: Generating Embeddings")

# 2. Define a list of documents (sentences in this case)
documents = [
    "The quick brown fox jumps over the lazy dog.",
    "A fast, agile fox leaps over a sleeping canine.",
    "Machine learning is a fascinating field.",
    "Artificial intelligence is rapidly advancing.",
    "Cats are known for their independent nature.",
    "Dogs are often considered loyal companions.",
    "The weather today is sunny and warm.",
    "It's a beautiful day with clear skies and high temperatures.",
]

# 3. Load a pre-trained Sentence-Transformer model
# This model converts sentences into 384-dimensional dense vector embeddings.
# It's optimized for semantic similarity.
print("Loading Sentence-Transformer model (all-MiniLM-L6-v2)...")
embedding_model = SentenceTransformer('all-MiniLM-L6-v2')
print("Model loaded successfully.")

# 4. Generate embeddings for our documents
print(f"Generating embeddings for {len(documents)} documents...")
document_embeddings = embedding_model.encode(documents)
print("Embeddings generated.")

# Let's inspect the first embedding and its dimension
print(f"\nFirst document: '{documents[0]}'")
print(f"Its embedding shape: {document_embeddings[0].shape}")
print(f"First 5 dimensions of the first embedding: {document_embeddings[0][:5]}")

# We'll continue with storing these in a vector database in the next step.

Explanation:

• from sentence_transformers import SentenceTransformer: Imports the class we need to load and use pre-trained embedding models.
• documents: A simple list of strings that represent our “knowledge base” for this example. In a real application, these could be paragraphs, articles, or product descriptions.
• SentenceTransformer('all-MiniLM-L6-v2'): This line downloads (if not already cached) and loads a specific pre-trained model. all-MiniLM-L6-v2 is a good balance of performance and speed, producing 384-dimensional vectors.
• embedding_model.encode(documents): This is the core step where the model processes each sentence in our documents list and returns a NumPy array where each row is the embedding vector for a corresponding sentence.
• document_embeddings[0].shape: Shows that each embedding is a 1D array of 384 numbers.
• document_embeddings[0][:5]: Prints the first five numbers of the first embedding to give you a sense of what these vectors look like (just floating-point numbers!).

Run this script to see the embeddings generated: python semantic_search_app.py

Step 2: Storing Embeddings in a Vector Database (ChromaDB)

Now that we have our embeddings, let’s store them in ChromaDB. ChromaDB is simple to use and can run entirely in memory or persist to disk. For this example, we’ll use an in-memory client.

Add the following code to semantic_search_app.py, right after the embedding generation section:

# ... (previous code for generating embeddings) ...

print("\nStep 2: Storing Embeddings in ChromaDB")

# 1. Initialize a ChromaDB client
# For a simple in-memory database, we just initialize the client without any path.
# For persistent storage, you'd provide a path: chromadb.PersistentClient(path="/path/to/db")
client = chromadb.Client()

# 2. Get or create a collection
# A collection is like a table in a relational database, holding our documents and embeddings.
# We specify the embedding function to ensure consistency.
collection_name = "my_semantic_documents"
try:
    collection = client.get_collection(name=collection_name)
    print(f"Collection '{collection_name}' already exists. Clearing it for a fresh start.")
    client.delete_collection(name=collection_name)
    collection = client.create_collection(name=collection_name)
except:
    print(f"Creating new collection '{collection_name}'.")
    collection = client.create_collection(name=collection_name)


# 3. Prepare data for ChromaDB
# ChromaDB requires IDs for each document, and optionally metadata.
ids = [f"doc_{i}" for i in range(len(documents))]
# Metadata can be any dictionary of key-value pairs associated with each document.
# This is useful for filtering results later.
metadatas = [{"source": "example_data", "index": i} for i in range(len(documents))]

# 4. Add documents and their embeddings to the collection
print(f"Adding {len(documents)} documents to ChromaDB collection '{collection_name}'...")
collection.add(
    embeddings=document_embeddings.tolist(), # ChromaDB expects a list of lists
    documents=documents,
    metadatas=metadatas,
    ids=ids
)
print("Documents added to ChromaDB.")

print(f"Number of items in collection: {collection.count()}")

# We'll perform searches in the next step.

Explanation:

• client = chromadb.Client(): Creates an in-memory ChromaDB client. If you wanted to save the database to disk, you’d use chromadb.PersistentClient(path="./my_vector_db").
• collection = client.create_collection(name="my_semantic_documents"): A collection is where your data lives within ChromaDB. We give it a name. The try-except block ensures we start fresh if the collection already exists from a previous run.
• ids = [f"doc_{i}" for i in range(len(documents))]: ChromaDB requires a unique ID for each item you add. We generate simple string IDs.
• metadatas = [{"source": "example_data", "index": i} for i in range(len(documents))]: Metadata allows you to store additional information alongside your embeddings, which can be useful for filtering or displaying results.
• collection.add(...): This is the core method for ingesting data.
  • embeddings=document_embeddings.tolist(): We pass our generated embeddings. Note that chromadb typically expects a list of lists for embeddings, so we convert the NumPy array.
  • documents=documents: The original text content associated with each embedding.
  • metadatas=metadatas: The optional metadata.
  • ids=ids: The unique identifiers for each document.
• collection.count(): Verifies that the documents were added correctly.

Run the script again: python semantic_search_app.py. You should see output confirming the documents were added.

Step 3: Performing Semantic Search

Now for the exciting part: querying our vector database semantically! We’ll take a user query, embed it, and ask ChromaDB to find the most similar documents.

Add the final section to semantic_search_app.py:

# ... (previous code for storing embeddings in ChromaDB) ...

print("\nStep 3: Performing Semantic Search")

# 1. Define a query
user_query = "Tell me about animals that are known for their loyalty."
print(f"User Query: '{user_query}'")

# 2. Embed the user query using the SAME model used for documents
print("Embedding user query...")
query_embedding = embedding_model.encode([user_query]).tolist() # Encode expects a list, returns a list of embeddings
print("Query embedded.")

# 3. Perform a similarity search in ChromaDB
# We ask for the top 3 most similar results.
print("Searching ChromaDB for similar documents...")
results = collection.query(
    query_embeddings=query_embedding,
    n_results=3, # How many top results we want
    include=['documents', 'distances', 'metadatas'] # What information to retrieve
)

print("\nSearch Results:")
# ChromaDB returns results as a dictionary with lists for each requested item
# (e.g., 'documents', 'distances', 'metadatas'). Each list corresponds to the n_results.
# We iterate through the first item in each list (since we only queried one embedding)
# and then through the individual results.
for i in range(len(results['documents'][0])):
    doc = results['documents'][0][i]
    dist = results['distances'][0][i]
    meta = results['metadatas'][0][i]
    print(f"  Result {i+1}:")
    print(f"    Document: '{doc}'")
    print(f"    Distance (Lower is better): {dist:.4f}")
    print(f"    Metadata: {meta}")
    print("-" * 20)

print("\nLet's try another query about the weather.")
user_query_2 = "What's the forecast for today?"
print(f"User Query: '{user_query_2}'")
query_embedding_2 = embedding_model.encode([user_query_2]).tolist()
results_2 = collection.query(
    query_embeddings=query_embedding_2,
    n_results=2,
    include=['documents', 'distances', 'metadatas']
)

print("\nSearch Results for second query:")
for i in range(len(results_2['documents'][0])):
    doc = results_2['documents'][0][i]
    dist = results_2['distances'][0][i]
    meta = results_2['metadatas'][0][i]
    print(f"  Result {i+1}:")
    print(f"    Document: '{doc}'")
    print(f"    Distance (Lower is better): {dist:.4f}")
    print(f"    Metadata: {meta}")
    print("-" * 20)

Explanation:

• user_query = "...": Our natural language question.
• query_embedding = embedding_model.encode([user_query]).tolist(): Crucially, we use the exact same embedding_model to embed the user’s query. This ensures that the query vector is in the same semantic space as our document vectors, making comparisons meaningful. We wrap the query in a list because encode expects a list of sentences.
• collection.query(...): This is where the vector database does its work.
  • query_embeddings=query_embedding: The embedded user query.
  • n_results=3: We ask for the top 3 most similar documents.
  • include=['documents', 'distances', 'metadatas']: Specifies what information we want back (the original text, the similarity distance, and any metadata).
• Result Parsing: ChromaDB returns results in a structured dictionary. We iterate through it to display the retrieved documents, their similarity distances (lower distance means more similar), and their associated metadata.

Run the complete script: python semantic_search_app.py.

Observe the results! For “animals that are known for their loyalty,” you should see results related to “Dogs.” For “What’s the forecast for today?”, you should see documents about “the weather” and “beautiful day.” This demonstrates semantic understanding far beyond simple keyword matching.

Mini-Challenge: Expanding Your Semantic Knowledge Base

Now it’s your turn! Let’s expand our small knowledge base.

Challenge:

1. Add at least two new sentences about “programming languages” to our documents list. For example:
   • “Python is a versatile programming language widely used in AI.”
   • “JavaScript is essential for web development.”
2. After adding them, run your script to re-ingest all documents into ChromaDB.
3. Perform a new semantic search using a query like: “Which languages are used for building websites?”
4. Verify that your new programming language documents are among the top results if they are relevant.

Hint:

• You’ll need to modify the documents list at the beginning of your script.
• The rest of your script (embedding and ChromaDB ingestion/query) should automatically adapt to the new list size.
• Remember the try-except block for collection creation will clear the old data, so your new documents will be the only ones.

What to observe/learn:

• How easily you can expand your knowledge base.
• How the semantic search system adapts to new information.
• The power of embeddings to connect queries to relevant concepts, even if the exact keywords aren’t present.

Common Pitfalls & Troubleshooting

1. Model Mismatch:

   • Pitfall: Using one SentenceTransformer model (e.g., all-MiniLM-L6-v2) to embed your documents and a different model (e.g., distilbert-base-nli-stsb-mean-tokens) to embed your queries.
   • Why it’s a problem: Each model creates its own unique vector space. Embeddings from different models are not directly comparable. It’s like comparing apples and oranges!
   • Solution: Always use the exact same embedding model for both ingesting your documents into the vector database and embedding your user queries.

2. Forgetting tolist() for ChromaDB:

   • Pitfall: Trying to pass a NumPy array directly to collection.add(embeddings=...) or collection.query(query_embeddings=...).
   • Why it’s a problem: While sentence-transformers returns NumPy arrays, many vector database clients (including ChromaDB in some contexts) expect standard Python lists of lists for embeddings.
   • Solution: Convert your NumPy arrays to Python lists using .tolist() before passing them to ChromaDB, as demonstrated in the examples.

3. Data Preprocessing (or lack thereof):

   • Pitfall: Feeding raw, noisy text (e.g., HTML tags, irrelevant symbols, inconsistent casing) directly to the embedding model.
   • Why it’s a problem: Embedding models are sensitive to input quality. Noisy data can lead to less accurate or less meaningful embeddings, reducing search relevance.
   • Solution: Implement robust text preprocessing steps (e.g., cleaning, normalization, lowercasing, removing stop words or special characters, stemming/lemmatization if appropriate) before generating embeddings. Consistency in preprocessing is key.

4. Misinterpreting Distance/Similarity Scores:

   • Pitfall: Assuming a specific distance value (e.g., 0.5) always means “similar.”
   • Why it’s a problem: The absolute values of distance or similarity metrics (like cosine similarity or Euclidean distance) are model and dataset-dependent. What’s “similar” in one context might not be in another.
   • Solution: Focus on the relative ranking of results. The top N results are generally the most relevant. Over time, you’ll develop an intuition for “good” scores within your specific application. Also, different models and distance metrics will yield different ranges.

Summary

Congratulations! You’ve successfully navigated the exciting world of embeddings, vector databases, and semantic search. Let’s recap the key takeaways from this chapter:

• Embeddings as Semantic Vectors: You learned that embeddings are numerical representations of data (like text) that capture its semantic meaning. Objects with similar meanings are represented by vectors that are close to each other in a high-dimensional space.
• Vector Databases for Efficient Search: Traditional databases struggle with similarity search. Vector databases (like ChromaDB) are purpose-built to store, index, and efficiently query these high-dimensional vectors using Approximate Nearest Neighbor (ANN) algorithms.
• Semantic Search Beyond Keywords: You implemented a semantic search system that understands the meaning of a query, rather than just matching keywords, by embedding both documents and queries into the same vector space and performing similarity lookups.
• Hands-on Application: You used sentence-transformers to generate embeddings and ChromaDB to store them and perform semantic queries, building a practical system from scratch.
• Critical Best Practices: You understood the importance of using the same embedding model for ingestion and querying, proper data preparation, and understanding the nuances of similarity scores.

This chapter equips you with a powerful set of tools fundamental to modern AI applications. The ability to represent and search data semantically opens doors to advanced information retrieval, recommendation systems, and the crucial Retrieval Augmented Generation (RAG) pattern often used with Large Language Models.

What’s Next? In the next chapter, we’ll build upon this foundation and explore Retrieval Augmented Generation (RAG), combining the power of semantic search with Large Language Models to create more informed and accurate AI assistants. Get ready to integrate these concepts into even more sophisticated systems!

References

1. Sentence-Transformers Documentation: The official documentation for the sentence-transformers library, providing details on models and usage.
   • https://www.sbert.net/
2. ChromaDB Documentation: The official guide for Chroma, a popular open-source vector database.
   • https://docs.trychroma.com/
3. Hugging Face Transformers Library: While we used sentence-transformers (which builds on Hugging Face), understanding the broader transformers ecosystem is beneficial.
   • https://huggingface.co/docs/transformers/index
4. OpenXcell Blog: 10 Best Embedding Models Powering AI Systems in 2026: Provides insights into various embedding models and their applications.
   • https://www.openxcell.com/blog/best-embedding-models/

This page is AI-assisted and reviewed. It references official documentation and recognized resources where relevant.