Retrieval Augmented Generation (RAG) Development

What a RAG system does inside an app you already run

Retrieval augmented generation answers a question in two steps. First it searches your own content for the passages most likely to contain the answer, then it hands only those passages to a language model with instructions to answer from them and cite them. The model never needs to be retrained on your data, and when a document changes, the answers change with it.

Tepia usually adds RAG to software a client already operates. A building products distributor wants its dealer portal to answer installation questions from thousands of spec sheets and warranty PDFs. A healthcare staffing firm wants recruiters to ask about state credentialing rules without opening a binder. A manufacturer wants line technicians to search maintenance manuals by describing a fault code in plain words. In each case the existing app stays in place and gains a new API endpoint, a search or chat surface, and an ingestion service behind it.

The question Tepia asks first is not which model to use. It is which documents hold the truth, who is allowed to see each one, and how often they change. Those three answers decide most of the architecture.

The parts of a production RAG pipeline

A demo can be built in an afternoon. A pipeline that gives correct answers on your fiftieth thousand document takes deliberate choices at each stage. This is the stage map Tepia works through with every client:

Tepia keeps the index close to your existing stack where it can. If your app already runs on PostgreSQL, pgvector avoids adding a new system to operate. See our guide to building a RAG system on company data for the step by step version.

Permissions, freshness and citations

Most RAG failures in production are not model failures. They are an answer drawn from a document the user should not see, or from a version that was retired. Tepia stores the access list of every source document alongside its chunks and applies it as a filter inside the search query itself, not after the fact. If your app already has roles, the RAG layer reads them; if access lives in Microsoft Entra ID or Google Workspace groups, Tepia syncs those groups.

Freshness is handled by event driven sync where the source allows it. SharePoint and Google Drive change notifications, Zendesk webhooks and database triggers tell the ingestion service what changed, so only that file is parsed again. Deleted files are removed from the index the same day, which matters for contracts and HR policies.

Citations are the trust mechanism. Each answer shows the source title, section and a link that opens the exact passage. When retrieval returns nothing relevant, Tepia configures the assistant to say so and offer a search or a human contact instead of guessing. Users forgive a missing answer far faster than a wrong one.

Measuring whether answers are right

Tepia does not ship RAG on the strength of a good demo. Before launch we assemble an evaluation set of 150 to 300 real questions pulled from support tickets, search logs and interviews with the people who answer them today, each paired with the document that holds the correct answer.

• Retrieval recall: whether the right passage appears in the top results for each question.
• Groundedness: whether every claim in the answer is supported by a retrieved passage.
• Correctness: reviewed by your subject matter experts on a sample, scored against the expected answer.
• Refusal behavior: whether out of scope questions get a clear no rather than an invented reply.

The set runs automatically whenever Tepia changes a prompt, swaps a model, adjusts chunk sizes or adds a source, so a fix for one question cannot quietly break twenty others. In production, thumbs up and down feedback and logged questions with no good match feed the next round of content fixes. Our LLM evaluation and testing page covers the tooling.

How Tepia approaches RAG development

Tepia runs its six phase process on every RAG engagement, with a US based project manager accountable from the first workshop.

1. Discovery: an inventory of sources, owners, formats and access rules, plus interviews with the people who answer these questions today. Deliverables are an Investigation Summary, an Interview Summary and User Stories such as a dealer asking for the wind load rating of a specific window series.
2. Design: wireframes for the answer view, citation panel, no answer state and feedback controls, matched to your existing app’s style guide.
3. Development and Testing: connectors, index and retrieval service built first, then generation. The Alpha runs against the evaluation set; the Beta goes to a pilot group, with functional, user acceptance and non functional test plans covering latency and permission leaks.
4. Training: sessions for content owners on how document structure affects answers, and for support leads on reading the question logs.
5. Launch: full ingestion of the document library, staged rollout by team, and support reps watching early questions.
6. Support: monitoring of answer quality, new source connectors and model upgrades rerun against the evaluation set.

Typically Discovery takes 1 to 3 months and development and QA 2 to 6 months; a single source RAG feature added to an existing app sits at the short end. Read more about our process.

When you need custom RAG and when you do not

If your documents already live in Microsoft 365 and your staff only need answers inside Teams or Outlook, a packaged assistant with built in connectors may be enough, and Tepia will say so in Discovery. Custom RAG earns its place in different situations:

• The answers must appear inside your own product, for customers, dealers or patients, not only for employees.
• Your permission model is more specific than folder sharing, such as per account, per region or per contract.
• Answers need structured data from your database, like order history or equipment records, alongside documents.
• You need audit logs of every question, retrieved source and answer for compliance review.

Some teams ask whether fine tuning would be better. For questions about facts in changing documents, retrieval almost always wins; fine tuning suits style and format. Compare them in fine tuning vs RAG, see AI knowledge base search for internal teams, and talk to Tepia about your sources.

Frequently asked questions