Vector databases are the backbone of modern AI applications that need semantic search, retrieval-augmented generation (RAG), or recommendation systems. Unlike traditional databases that match exact keywords, vector databases store and query high-dimensional embeddings, letting your app find conceptually similar content.

In this tutorial, you will set up both Pinecone (managed cloud) and Chroma (open-source, self-hosted), build a simple RAG pipeline with each, and learn when to pick one over the other.

Prerequisites

Before starting, make sure you have:

  • Python 3.9 or higher installed
  • An OpenAI API key (for generating embeddings)
  • A Pinecone account (free tier works)
  • Basic familiarity with pip and virtual environments

Step 1: Set Up Your Project Environment

Create a clean project directory and virtual environment:

mkdir vector-db-demo && cd vector-db-demo
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

Install the shared dependencies:

pip install openai tiktoken langchain langchain-openai

Create a .env file for your API keys:

OPENAI_API_KEY=sk-your-key-here
PINECONE_API_KEY=your-pinecone-key

Step 2: Set Up Chroma (Local Vector Database)

Chroma runs entirely on your machine with zero configuration. Install it:

pip install chromadb

Create a file called chroma_demo.py:

import chromadb
from chromadb.utils import embedding_functions
 
# Initialize persistent storage
client = chromadb.PersistentClient(path="./chroma_data")
 
# Use OpenAI embeddings
openai_ef = embedding_functions.OpenAIEmbeddingFunction(
    api_key="your-openai-key",
    model_name="text-embedding-3-small"
)
 
# Create a collection
collection = client.get_or_create_collection(
    name="documents",
    embedding_function=openai_ef
)
 
# Add documents
collection.add(
    documents=[
        "Vector databases store embeddings for semantic search.",
        "RAG combines retrieval with language model generation.",
        "Pinecone is a managed vector database service.",
        "Chroma is an open-source embedding database.",
    ],
    ids=["doc1", "doc2", "doc3", "doc4"]
)
 
# Query
results = collection.query(
    query_texts=["How do I search similar documents?"],
    n_results=2
)
 
print(results["documents"])

Run it:

python chroma_demo.py

You should see the two most semantically similar documents returned. Chroma handles embedding generation, storage, and querying in just a few lines.

Step 3: Set Up Pinecone (Managed Cloud Database)

Pinecone requires an account but handles scaling, replication, and infrastructure for you. Install it:

pip install pinecone-client openai

Create pinecone_demo.py:

from pinecone import Pinecone, ServerlessSpec
from openai import OpenAI
import os
 
# Initialize clients
pc = Pinecone(api_key=os.getenv("PINECONE_API_KEY"))
openai_client = OpenAI()
 
# Create an index (only needed once)
index_name = "documents"
 
if index_name not in pc.list_indexes().names():
    pc.create_index(
        name=index_name,
        dimension=1536,  # text-embedding-3-small dimension
        metric="cosine",
        spec=ServerlessSpec(cloud="aws", region="us-east-1")
    )
 
index = pc.Index(index_name)
 
# Prepare documents
documents = [
    "Vector databases store embeddings for semantic search.",
    "RAG combines retrieval with language model generation.",
    "Pinecone is a managed vector database service.",
    "Chroma is an open-source embedding database.",
]
 
# Generate embeddings
response = openai_client.embeddings.create(
    input=documents,
    model="text-embedding-3-small"
)
 
# Upsert vectors with metadata
vectors = [
    (f"doc{i+1}", emb.embedding, {"text": doc})
    for i, (doc, emb) in enumerate(zip(documents, response.data))
]
index.upsert(vectors=vectors)
 
# Query
query_embedding = openai_client.embeddings.create(
    input=["How do I search similar documents?"],
    model="text-embedding-3-small"
).data[0].embedding
 
results = index.query(vector=query_embedding, top_k=2, include_metadata=True)
 
for match in results.matches:
    print(f"Score: {match.score:.4f} | {match.metadata['text']}")

Step 4: Build a Simple RAG Pipeline

Now combine vector search with an LLM for question answering:

from openai import OpenAI
 
client = OpenAI()
 
def rag_answer(question, context_docs):
    context = "\n".join(context_docs)
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": f"Answer based on this context:\n{context}"},
            {"role": "user", "content": question}
        ]
    )
    return response.choices[0].message.content
 
# Use with either Chroma or Pinecone results
question = "What is the difference between Pinecone and Chroma?"
# retrieved_docs = your vector search results
answer = rag_answer(question, retrieved_docs)
print(answer)

Step 5: Choose the Right Database for Your Use Case

Practical Tips

Here are lessons learned from deploying vector databases in production:

1. Batch your upserts. Both Pinecone and Chroma perform significantly better when you insert documents in batches of 100-500 rather than one at a time.

2. Choose embedding dimensions wisely. OpenAI's text-embedding-3-small (1536 dims) offers a good balance of quality and cost. Use text-embedding-3-large only if retrieval accuracy is critical and cost is not a concern.

3. Store metadata generously. Always store the original text and relevant metadata (source URL, timestamp, category) alongside vectors. You will need them for filtering and display.

4. Use namespaces for multi-tenancy. In Pinecone, namespaces let you partition data per user or project without creating multiple indexes. In Chroma, use separate collections.

5. Monitor your recall. Set up evaluation with a small labeled dataset. Measure how often the correct document appears in your top-k results. Aim for 90%+ recall at k=5.

6. Pre-filter when possible. If you know the user only needs documents from a specific category, apply metadata filters before vector search. This reduces the search space and improves both speed and relevance.

Common Pitfalls to Avoid

  • Forgetting to wait for index readiness -- Pinecone indexes take 30-60 seconds to initialize. Always check describe_index before querying a new index.
  • Mixing embedding models -- If you embed documents with text-embedding-3-small, you must query with the same model. Mismatched dimensions or embedding spaces return garbage results.
  • Not handling duplicates -- Use deterministic IDs (e.g., hash of content) so re-indexing the same document overwrites rather than duplicates.

Next Steps

Once your basic pipeline works:

  1. Add chunking for long documents (aim for 200-500 tokens per chunk)
  2. Implement hybrid search (combine vector similarity with keyword BM25)
  3. Add a reranker (Cohere Rerank or a cross-encoder) for better precision
  4. Set up evaluation metrics to measure retrieval quality over time

Try pinecone

Try pinecone

Try chroma

Try chroma