Retrieval-Augmented Generation (RAG) lets you give AI models access to your own documents without fine-tuning. This tutorial walks you through building a production-ready RAG pipeline using LangChain.

What You'll Build

A system that can answer questions about your documents by:

  1. Splitting documents into chunks
  2. Embedding them into a vector store
  3. Retrieving relevant chunks for each query
  4. Generating answers grounded in your data

Prerequisites

  • Node.js 18+ or Python 3.10+
  • An API key from OpenAI or Anthropic
  • Some documents to index (PDFs, markdown, etc.)

Step 1: Project Setup

mkdir rag-pipeline && cd rag-pipeline
npm init -y
npm install langchain @langchain/openai @langchain/community chromadb

Step 2: Document Loading

import { DirectoryLoader } from "langchain/document_loaders/fs/directory";
import { PDFLoader } from "langchain/document_loaders/fs/pdf";
import { TextLoader } from "langchain/document_loaders/fs/text";
 
const loader = new DirectoryLoader("./docs", {
  ".pdf": (path) => new PDFLoader(path),
  ".txt": (path) => new TextLoader(path),
  ".md": (path) => new TextLoader(path),
});
 
const docs = await loader.load();

Step 3: Chunking

import { RecursiveCharacterTextSplitter } from "langchain/text_splitter";
 
const splitter = new RecursiveCharacterTextSplitter({
  chunkSize: 1000,
  chunkOverlap: 200,
});
 
const chunks = await splitter.splitDocuments(docs);

Step 4: Embedding & Storage

import { OpenAIEmbeddings } from "@langchain/openai";
import { Chroma } from "@langchain/community/vectorstores/chroma";
 
const vectorStore = await Chroma.fromDocuments(
  chunks,
  new OpenAIEmbeddings(),
  { collectionName: "my-docs" }
);

Step 5: Query Pipeline

import { ChatOpenAI } from "@langchain/openai";
import { RetrievalQAChain } from "langchain/chains";
 
const model = new ChatOpenAI({ modelName: "gpt-4o" });
const chain = RetrievalQAChain.fromLLM(model, vectorStore.asRetriever());
 
const result = await chain.call({
  query: "What is our refund policy?"
});

Production Tips

  • Chunk size matters: Start with 1000 characters, adjust based on your content type
  • Use hybrid search: Combine vector similarity with keyword matching
  • Cache embeddings: Don't re-embed unchanged documents
  • Monitor relevance: Log retrieval scores to catch quality degradation