Case study: building a RAG-powered support automation system for a Series B B2B SaaS platform.
Key Results
StepTo built a retrieval-augmented generation (RAG) system for a Series B B2B SaaS platform (~80 employees) whose support team could no longer keep pace with ticket volume. The system ingests product docs, past resolved tickets, and internal runbooks into a pgvector store, with an LLM-backed triage layer that drafts first-response answers and routes edge cases to human agents — every AI-drafted response is reviewed by a support agent before it sends. The build took 4 months with a 3-person dedicated squad (1 ML/AI engineer, 1 backend engineer, 1 tech lead). Results measured over the following quarter: average first-response time dropped 70% (from ~5.5 hours to under 100 minutes), 38% of tier-1 tickets were fully auto-resolved with no human touch, and average handle time for tickets still requiring a human agent fell 45%.
A Series B B2B SaaS platform had scaled its customer base far faster than its support team. Tickets piled up, first responses routinely took hours, and support engineers kept re-answering the same questions because the real answers lived scattered across Slack threads and Notion pages — not in anything searchable at the point of need.
StepTo assembled a 3-person dedicated squad and delivered a production RAG system in 4 months, cutting first-response time by 70% and fully auto-resolving 38% of tier-1 ticket volume — with a human reviewing every AI-drafted response before it reaches a customer.
Measured over the quarter following full production rollout
The client, an ~80-person Series B B2B SaaS company, had grown ticket volume faster than headcount for three straight quarters. Every ticket was triaged manually, and average first-response time had crept up to roughly 5.5 hours — well outside what their enterprise SLAs allowed.
The deeper problem was institutional knowledge. Product docs existed but were incomplete, and the real answers to most recurring questions lived in past Slack threads, internal Notion runbooks, and the memory of a handful of senior support engineers — none of it searchable at the moment an agent actually needed it. New hires took months to ramp, and experienced agents spent a large share of their day re-answering questions that had already been answered dozens of times before.
The client wanted to use AI to close the gap, but they were wary of "AI support bots" that hallucinate — their product is technical, their customers are enterprise accounts, and a wrong answer sent automatically would cost more trust than the time it saved.
StepTo proposed a retrieval-augmented generation (RAG) system rather than a general-purpose chatbot: the LLM would only ever answer from content it retrieved from the client's own product docs, resolved tickets, and internal runbooks — never from open-ended generation.
Incoming tickets are classified and matched against a pgvector store holding embedded product documentation, 14 months of resolved tickets, and internal runbooks, refreshed nightly. A triage layer drafts a first-response answer with a confidence score; high-confidence drafts go into a human review queue built directly into the client's existing Zendesk workflow, while low-confidence or ambiguous tickets route straight to a human agent with no draft at all. No AI-generated response reaches a customer without a support agent approving, editing, or rejecting it first.
Audited the existing docs, Notion pages, Slack threads, and 14 months of resolved tickets to establish ground truth for what the AI would be trained to retrieve.
Built the ingestion pipeline — chunking, embedding, and metadata tagging — and stood up a pgvector store synced nightly with product docs, runbooks, and resolved tickets.
Built the LLM-backed triage layer that classifies incoming tickets, retrieves relevant context, and drafts a first-response answer with a confidence score.
Integrated a review queue directly into Zendesk so every AI-drafted response is approved, edited, or rejected by a support agent before it ever reaches a customer.
Piloted on 20% of ticket volume, tuned confidence thresholds against real agent overrides, then rolled out to the full support queue.
Every response the AI drafts is grounded in retrieved passages from the vector store, not free-form generation — it can only answer from what it actually found in your docs and past tickets. Every draft also carries a confidence score: high-confidence answers go to human review before sending, and low-confidence tickets skip the draft entirely and route straight to an agent. Nothing reaches a customer without a human approving it in the review queue.
For this client, the math was straightforward: a 45% reduction in average handle time across tickets still touched by a human, plus 38% of tier-1 volume needing no human time at all. Measured against the fully loaded cost of the support team's time, the system paid back its build cost within the first quarter after full rollout.
It almost always is — the discovery phase exists for exactly this reason. We audited what was accurate, flagged what was stale, and used 14 months of resolved tickets (not just the official docs) as a second source of ground truth, since the real answers often lived in past agent responses rather than the wiki. The system also flags recurring low-confidence topics post-launch, which becomes a live signal for where the documentation still has gaps.
No — it removes the repetitive tier-1 volume so the existing team can spend their time on the tickets that actually need a human: escalations, edge cases, and anything emotionally or technically complex. Headcount on the support team was unchanged; effective capacity went up.
Whether you need a full RAG system, an AI strategy review, or a dedicated team — let's talk. We'll have a scoped proposal to you within 48 hours.
Start the conversationTell us what you're building and we'll get back to you within one business day with a no-obligation assessment.
We'll reply within one business day.
Work with accountable, English-fluent professionals who communicate clearly, protect quality, and deliver with a steady operating rhythm. Cost efficiency matters, but performance is why clients stay with us.
