A Practical Guide to Retrieval-Augmented Generation (RAG) for Business Data
RAG lets an LLM answer questions using your own private data instead of only its training data. Here's how the pipeline actually works.
Priya Nair
· 3 min read
Retrieval-Augmented Generation (RAG) is the pattern behind most useful
“chat with your data” products — customer support assistants that know
your help centre, internal tools that search your company’s documents,
and search experiences that understand meaning, not just keywords.
The core idea
A large language model only knows what it was trained on, and that
training data is frozen at a point in time. RAG solves this by retrieving
relevant information from your own data at the moment a question is asked,
and including that information in the prompt sent to the model. The model
then answers using both its general knowledge and the specific context
you’ve given it.
The pipeline, step by step
- Chunking: Break your source documents (help articles, product docs, internal wikis) into smaller passages — typically a few hundred words each, with some overlap between chunks to preserve context.
- Embedding: Convert each chunk into a vector — a list of numbers that represents its meaning — using an embedding model.
- Storage: Store those vectors in a vector-capable database. For teams
already running PostgreSQL, the
pgvectorextension means you don’t need a separate specialized database. - Retrieval: When a user asks a question, embed the question the same way, then find the stored chunks whose vectors are closest to it (using cosine similarity or a similar metric).
- Generation: Send the user’s question, along with the retrieved chunks, to the LLM with a prompt instructing it to answer using that context and to say when it doesn’t know.
Where teams get it wrong
- Chunking too coarsely or too finely. Whole documents as single chunks bury the relevant sentence in noise; single-sentence chunks lose context.
- Skipping evaluation. Without a test set of real questions and expected answers, you won’t know if retrieval quality is actually good until users complain.
- No fallback for “I don’t know.” A RAG system without an explicit instruction to acknowledge missing information will confidently hallucinate an answer instead.
- Ignoring access control. If different users should see different documents, retrieval needs to respect the same permissions as your application — not just search everything indiscriminately.
Getting started pragmatically
You don’t need a complex vector database cluster to start. A single
PostgreSQL database with pgvector, a modest embedding model, and a
handful of well-chunked documents is enough to validate whether RAG solves
a real problem for your users before investing in a more elaborate
pipeline.
Written by
Priya Nair
AI & Machine Learning Engineer
Priya builds production-grade AI features, from RAG pipelines to fine-tuned models, with an emphasis on evaluation and safety.
Related articles
How AI Works: From LLMs to Production AI Applications
A plain-English tour of how large language models actually work, and what separates a weekend AI demo from something you can safely put in front of paying customers.
How to Build an Agentic AI MVP
Agentic AI — models that plan, call tools, and take multi-step action — is genuinely useful, and genuinely easy to over-build. Here's how to scope a first version that ships.