Who Actually Builds RAG Systems? A Buyer's Guide to Evaluating a RAG Development Partner

What Does "RAG Development" Actually Involve?

Ask ten vendors what "RAG development" means and you'll get ten different scopes, and that's the first thing to sort out before evaluating anyone. Retrieval-augmented generation is not a single component you buy or a single skill you hire for. It's a pipeline: documents get ingested and split into chunks, chunks get embedded and stored in a retrievable index, a query pulls back the chunks most relevant to it, those chunks get reranked and assembled into context, and a language model generates an answer grounded in that context. Each stage is its own set of engineering decisions, and a team that's only good at one of them will present the whole pipeline as solved.

Chunking strategy alone has consequences most buyers never ask about: chunk too small and you lose the context a passage needs to make sense; chunk too large and you dilute the relevant signal with noise. The storage layer is its own decision, too, whether the index belongs in a dedicated vector database or can live in the relational store you already operate matters enough that we've written about it at length in our piece on vector databases versus Postgres with pgvector. And retrieval itself is rarely a single similarity search in production: systems that hold up under real queries typically combine lexical and vector search and rerank the combined results, because either approach alone misses cases the other catches.

Generation is the part that looks simplest and is easiest to get wrong quietly. A model can produce a fluent, confident answer from retrieved context that doesn't actually support the claim, and unless something in the pipeline checks for that, nobody notices until a customer does. That's why evaluation, some repeatable way of measuring whether retrieval found the right material and whether the answer is grounded in it, isn't a nice-to-have bolted on at the end. It's the part of the pipeline that tells you whether the rest of it is working at all.

What Questions Separate a Real RAG Team From a Prompt-Engineering Shop?

The fastest way to tell a real RAG team from a shop that's wrapped a chat interface around an API call is to ask a small number of specific questions and see whether the answers get more precise or more vague. Real practitioners answer with architecture. Everyone else answers with adjectives.

"Walk me through your retrieval strategy, not your model choice." The model is the least differentiated part of a RAG system in 2026; every credible team has access to the same handful of frontier models through an API. What separates outcomes is retrieval: hybrid search or dense vectors alone, how they handle documents that don't fit cleanly into a passage, how many chunks they pull back and how those get reranked. If the answer is "we send it to the model with your documents," that's not a retrieval strategy.

"How do you measure whether retrieval is actually working?" A team with production experience will describe some version of a held-out evaluation set: known questions with known correct source passages, checked automatically whenever the pipeline changes. A team without one will point at a demo and ask you to trust that it feels right. A demo run on ten curated documents tells you almost nothing about retrieval quality against your actual corpus.

"What happens when retrieval comes back empty or wrong?" Every production RAG system eventually gets a query it can't answer well, either because the information isn't in the corpus or because retrieval missed it. What the system does next, admit uncertainty, ask a clarifying question, escalate to a human, or confidently generate an answer anyway, is a design decision with real consequences. Teams that haven't thought about this in advance discover the failure mode in production, from a customer.

"Who is actually writing this, and have they operated one of these in production before?" This question most reliably separates capability from packaging. A team that has taken a RAG system through real load and a few painful lessons about chunking or reranking will talk about specific things that broke and what they changed. A team that hasn't will talk about the technology in general terms, because their experience is general.

Build vs. Buy vs. Partner: When Do You Need a Dedicated RAG Engineer Versus a Managed Product?

The build-versus-buy question for RAG follows the same logic as most infrastructure decisions, with one twist: the "buy" option is younger and less standardized than it is for, say, CRM or payments, so the ceiling shows up faster than people expect.

Managed RAG features are the right call when your need is generic and sits inside a platform you already pay for. A support tool with a built-in knowledge-base assistant, a documentation platform with semantic search, these solve a real problem without any custom engineering. If your requirement is "let customers ask questions against our help docs," building your own pipeline for that is usually a waste of engineering time. The ceiling appears when the platform's retrieval can't reach into your other systems, when you need control over chunking or reranking the vendor doesn't expose, or when quality plateaus below what your use case needs.

A dedicated in-house RAG engineer makes sense once retrieval quality against your specific, evolving corpus is core to your product rather than a support feature, and the work is continuous rather than a one-time build. If your knowledge base changes weekly, your retrieval logic needs regular tuning, and accuracy directly affects revenue or risk, that's ongoing engineering ownership, not a project with an end date.

A development partner is usually the right call in between: you need engineering depth across ingestion, retrieval, evaluation and generation, the work integrates with several existing systems, and you don't want to carry a full-time senior AI hire yet, either because the scope doesn't justify it or because you need the system stabilized before you know what ongoing ownership should look like.

What Does Production RAG Actually Cost to Run and Maintain?

Every RAG cost conversation StepTo has with a prospective client starts in the wrong place, on what the initial build costs, when the number that actually determines whether the project is worth doing is what it costs to run for the next two years.

Storage and hosting for the retrieval index scale with corpus size and, depending on architecture, with query volume: a knowledge base of a few thousand documents is inexpensive to serve, a corpus that grows continuously and gets queried at real product scale is a genuine infrastructure line item. Embedding costs follow a similar curve, every document ingested and every query issued against an embedding model adds up, and reranking, when the pipeline uses it, adds another call on top. None of this shows up in a demo built on fifty documents.

The less visible driver is the evaluation and monitoring layer: tooling that catches retrieval quality degrading as your corpus grows, as user queries drift from what the system was built against, and as the underlying models get updated by the vendor. Skipping this doesn't remove the cost, it defers it to the point where a customer notices the system is wrong, which is a more expensive way to find the same problem.

And every time source documents change meaningfully, some portion of the index needs reindexing, which is recurring work, not a one-time setup task. None of this needs an invented RAG-specific price tag to plan around. It needs a buyer who asks about these drivers before signing anything, and a vendor who can answer with specifics rather than a flat monthly number that doesn't map to any of them. If you want to see what StepTo charges for the engineering time behind this kind of work, our published rate card is public, there's no separate quote you have to request just to see the numbers.

How Does StepTo Approach RAG Engagements?

Our position on RAG is the one we'd want a client to hold us to if we were the buyer: build against real data volumes and real query patterns from the start, not a curated slice that looks good in a first demo, and staff the work with engineers senior enough to own retrieval, evaluation and generation as one connected system rather than three disconnected favors.

That's the approach behind our RAG-powered support automation case study, built for a Series B B2B SaaS platform, which cut first response time by 70% and auto-resolved 38% of tier-1 tickets without a human in the loop. Getting there wasn't a matter of connecting a support inbox to a chat model; it required building retrieval against the client's actual product documentation and support history, evaluating it against real historical tickets before it touched a live customer, and building escalation logic for cases retrieval genuinely can't answer well.

We work as a senior-led engineering team, not a rotating cast of contractors billing hours against a vague scope, and that matters more for RAG than for most software, because the failure modes are subtle enough that seniority is the difference between a system that's quietly wrong and one that's caught in evaluation before a customer ever sees it.