What does a RAG project typically cost? +
A first production RAG system lands between $30,000 and $150,000 USD depending on source count, document volume, evaluation complexity, and regulatory profile. Subsequent RAG workloads on the same retrieval platform are roughly 30% to 50% of that cost because the ingestion, retrieval, and observability layers are reused. Engagements typically start from $5,000 USD for a discovery and eval-set scoping phase that you keep regardless of whether you continue. We have a longer-form RAG cost guide on the blog covering the $15K to $400K+ range with example budgets.
Why do you build the eval set first instead of last? +
Because the eval set is the contract. Without it, "is this RAG system working?" is a vibes question, not an engineering question. With it, we have a number we can drive up week over week, a regression detector that catches when a chunking or model change breaks something, and a defensible answer for the day a senior stakeholder asks how we know the system is correct. Building the eval set first costs two weeks; not building it costs the project.
Can the system handle confidential or regulated data? +
Yes. We deploy into your cloud accounts (AWS, GCP, Azure), in the region you specify, with the encryption and access posture your compliance team requires. We work with SOC 2, HIPAA, and regional financial regulations as part of the architecture decision record. For the most sensitive workloads, we deploy with on-premise or VPC-only model serving so no document leaves your perimeter.
Which embedding and generation models do you use? +
We pick per workload. Closed-source frontier models (OpenAI, Anthropic, Google) when accuracy is paramount and the buyer is comfortable with the data-handling posture; open-source models (the Llama and Qwen families, Mixtral) when data sovereignty or unit economics demand it. Embedding models are picked the same way — measured against the eval set, not chosen by reputation. Every choice is documented and reviewed quarterly so the system rides the model-quality curve rather than aging into obsolescence.
How do you handle the corpus growing to millions of documents? +
Architecture is sized for one order of magnitude beyond current corpus size on day one. Past that, we partition the index, separate hot and cold storage, and add tiered retrieval (a fast first stage followed by a more expensive rerank stage on the top results). Cost per query stays predictable because we track it as a first-class metric with alerting tuned to it.
What happens when a model upgrade breaks our system? +
On every model upgrade, we re-run the full eval set on the new model. If accuracy drops, we tune prompts, retrieval, or thresholds until accuracy recovers — and only then cut traffic over. Buyers who skip this step are the ones surprised by silent regressions when a provider ships a new version. This is included in the monthly operate-with-you fee.
How do you compare RAG to fine-tuning? +
RAG is the right answer when the underlying knowledge changes (new documents arrive, policy updates, new product launches), when answer correctness must be auditable (citations matter), and when latency budget allows a retrieval round-trip. Fine-tuning is the right answer for style, format, and stable domain knowledge. Most production systems use both: RAG for the facts, fine-tuning or prompting for the voice. We have a longer-form comparison on the blog.
Can we run this entirely on-premise without external APIs? +
Yes. We have shipped fully self-hosted RAG systems on customer infrastructure with open-source embeddings and generation models, on-premise vector stores, and on-premise observability. The trade-off is unit economics and the speed of riding the frontier model improvement curve, both of which are explicit in the architecture decision record.
