RAG Explained With a Real Example: The Phrase Bank Behind a Language App

My German learning material is spread across class notes, transcripts, meeting notes, worksheets and a journal. I wanted a practice app to suggest better expressions from that material, not from whatever a model happens to know. The answer was a small pipeline that turns those files into a searchable phrase bank, and then hands the most relevant phrases to the model at the moment it gives feedback.

That's RAG, explained with one sentence: look up the relevant pieces of your own material first, then give them to the model along with the question. Nothing is trained. The model never changes; the knowledge lives in your files. This article walks through the whole pipeline, with the decisions and numbers from the real build.

What is RAG, in plain terms?

A language model knows what it learned in training. It doesn't know your notes, your documents or last week's meeting. You could paste everything into every request, but context windows are finite, and recall gets worse as they fill up.¹ RAG solves this with a lookup step:

1. Retrieval: find the few pieces of your material that matter for this question.
2. Augmentation: put those pieces into the prompt.
3. Generation: the model answers, using them.

The retrieval usually works by meaning rather than keywords, which is what embeddings and vector search are for. Both are explained below, as they appear in the pipeline.

Building the bank: from files to phrase cards

The bank is built once, then updated only for new or changed files. Each step has a reason:

1. Cut the material into pages of 700 to 1,500 words. Small enough for one careful model call, large enough to keep context.
2. Ask a model to list the reusable phrases on each page, as JSON in a fixed shape: the phrase, its meaning, a plainer alternative, an example, a level and a topic. A local model does this on a laptop through Ollama, so the material never leaves the machine and the build costs nothing per call.
3. Check in code that each phrase is really on the page. Plain code compares the phrase's main word with the page's words, and drops phrases longer than ten words or containing a number or a name. This stops the model from inventing phrases that sound plausible. Engineers call it a grounding check.
4. Run a second pass that writes each phrase and example in correct German. If the corrected form differs, the corrected one is kept.
5. Store one card per line in a JSONL file, and give each card an embedding: a list of 1,024 numbers from the bge-m3 model that represents its meaning. bge-m3 supports more than 100 languages, so German phrases and English meanings land near each other.²

Every rejected phrase goes into a separate log with the reason it was dropped, along with every correction the second pass made. When something in the bank looks wrong, that log is the first place to look.

Using the bank: retrieval at question time

When a learner answers a practice question, two features use the bank:

• An upgrade for your answer. A model first names the plain expressions in the answer. Vector search then finds the cards closest in meaning, and the model picks one upgrade and rewrites the sentence with it.
• Words to try. Before answering, the learner can ask what a strong answer would say. The search finds candidate cards, and the model picks up to three.

Vector search compares the numbers of the learner's words with the numbers of every card; the closest ones win. That's the "retrieval" in retrieval-augmented generation.

Do you need a framework or a vector database?

Not necessarily, and here neither was needed at first.

No framework. Frameworks like LangChain and LangGraph help when a model decides its own next step: agents with loops, branches and tools. This is a fixed assembly line, the same steps for every page. About 1,500 lines of plain TypeScript do it with nothing hidden. A graph framework would make sense later for something genuinely agentic, like a model that plans a mock interview.

No vector database on the laptop. At a few thousand cards, comparing numbers in plain code takes milliseconds, so the laptop keeps its files.

A database online. The live app runs as short-lived serverless functions. Loading a 20 MB file of vectors into each one would be slow, and the file would have to live in git. So online, the cards live in Postgres (Supabase) with the pgvector extension, and the database does the comparing. pgvector performs exact nearest-neighbour search by default, "which provides perfect recall"; an index switches to approximate search, which "trades some recall for speed".³ An exact search over the 4,837 cards in the bank takes milliseconds, so no index is needed yet.

The rule that makes online search work: one embedding model

Embeddings from two different models can't be compared, any more than two maps drawn at different scales. So the online search uses the same bge-m3 model as the laptop, hosted by Cloudflare's Workers AI. Measured on a local copy of the bank, five test queries returned the same top five cards on both sides, in about 0.2 seconds.

If you ever change the embedding model, every card must be embedded again. The bank's change detection includes the model's name in each card's fingerprint, so a model change re-sends every card automatically.

How the quality was checked

The checks were cheap, and two of them changed the design:

• Probe before building. Three pages first, every card read by hand. The probe found the model copying the examples from its own instructions into the bank, and the grounding check was made stricter.
• Measure the alternatives. The second pass originally answered yes or no: "is this phrase correct?" A probe showed it let wrong phrases through and rejected right ones. Asking it to write the correct form instead fixed six of seven errors and broke none.
• Sample by hand. Forty random cards, each judged good, odd or wrong, repeated after every big change.

The general method is most of AI engineering: reproduce on one page, look at what was added and what was dropped, change one thing, run the same page again, and compare.

Lessons you can reuse in your own RAG

• Ask for structured output in a fixed shape, and check the shape in code. Ask once more if it's wrong.
• Never trust extraction blindly. A code check against the source catches invented content that reads perfectly well.
• Prefer "write the correct version" over "is this correct?" when you use a model as a checker.
• Keep a log of what was dropped and why. It shows you what the model is getting wrong.
• Fingerprint everything you process, files and pages, so updates only touch what changed.
• Use one embedding model everywhere, and re-embed everything when you change it.

FAQ

What's the difference between RAG and fine-tuning?

RAG changes what the model sees at question time; fine-tuning changes the model itself by training it further. For knowledge that changes, like notes, documents or prices, RAG is almost always the better first step.

Do I need LangChain to build RAG?

No. A fixed pipeline is a few functions: split, extract, check, embed, search. Frameworks earn their place when a model decides its own next steps.

How big should the chunks be?

Big enough to keep meaning together, small enough for careful processing. Here, pages of 700 to 1,500 words worked for extraction, and each final card is a single phrase with its example. Test a few sizes on your own material and read the results.

Which embedding model should I use?

One that covers your languages and that you can run in every place you need it. bge-m3 was chosen here because it handles German and English together and runs both locally and as a hosted service.²