RAG - Retrieval-Augmented Generation
A model only knows its training data. RAG gives it fresh or private knowledge at request time: find the relevant documents, put them into the prompt, and ask the model to answer from them.
The pipeline
text
Indexing (offline):
docs -> split into chunks -> embeddings -> vector store
Query (per request):
question -> embedding -> similarity search -> top-k chunks
-> prompt: "Answer using only these sources: ..." -> model -> answer + citationsAn embedding is a vector that represents meaning; similar texts have nearby vectors, so search works on meaning, not exact words. Most production systems combine it with keyword search (hybrid search) and a re-ranking step.
When you need it
| Situation | Better tool |
|---|---|
| Answers from your docs, tickets, knowledge base | RAG |
| Data changes often | RAG - reindex instead of retraining |
| Answers need sources the user can check | RAG |
| Small, stable knowledge that fits the context | just put it in the prompt |
| Change tone or output format | prompting or fine-tuning, not RAG |
| Live data (order status, balance) | tool calling to your API |
What the frontend does
- Citations - render source links next to claims; let the user open the original chunk. This is what makes RAG answers trustworthy.
- Retrieval states - "Searching docs..." before tokens start.
- Scope controls - filters like "only this project" or "docs after 2025" become metadata filters in the search.
- "No answer" state - when nothing relevant is found, a good system says so instead of guessing; design for it.
- Feedback - thumbs up/down per answer feeds evaluation of retrieval quality.
Most bad RAG answers are retrieval bugs, not model bugs: the right chunk was never found. Showing sources in the UI makes those bugs visible to users and to you.
updated
did you know it?