How much does a RAG chatbot for company documents cost, and should you build or buy?

What is a RAG chatbot, in plain words?

RAG stands for retrieval-augmented generation. When someone asks a question, the system first searches your documents, then hands the best passages to a language model and asks it to answer from them, with links to the sources.

Two things follow. Nothing is retrained: your files sit in a search index outside the model. And the answer is only as good as the search, so duplicate, outdated or badly scanned files give fluent wrong answers.

How much does it cost to build a RAG chatbot?

Published 2026 price guides disagree by a factor of ten or more, because each prices a different project. None is a quote.

Read the VITON¹³ row as a narrower scope, not a discount: the scope caps documents, sources, user groups and test questions, and leaves scanning, integrations and provider fees outside the price.

Why do two quotes for the same chatbot differ so much?

• Data preparation: Hamza puts it at 30–40% of a build budget and gmware at 30–50%. In Hamza’s experience a folder of clean Markdown takes a day and years of scanned PDFs take weeks.
• Permissions: if staff may read different documents, the rules must travel from the source system into the search layer. gmware asks: does the sales team see HR’s documents?
• Evaluation: Hamza prices a 50–100 question test set at $1,000–$3,000 and calls it the best money in a build.
• Data location: OpenAI and Anthropic both add about 10% for regional data-residency endpoints; private hosting adds operations.
• Who builds it: Hamza says agencies quote two to three times a solo engineer for the same scope, and his own list is the low end of the table.

Can a ready-made tool replace a custom RAG build?

For a team that already lives in one suite, often yes. The table shows what each vendor prints on its own pricing page today.

Seats add up. Twenty people on ChatGPT Business cost $4,800 a year billed annually or $6,000 billed monthly; on the Copilot Business offer, $4,320 for the first year (our arithmetic). That is the range of a small custom build, so ask what a seat cannot do: run inside your website or Telegram bot, answer the public, or apply document-level rules.

What does a RAG chatbot cost to run each month?

Three meters run after launch: indexing your files, storing the index and paying tokens per question. Indexing is nearly free: 200 documents of 20 pages at roughly 600 tokens a page is 2.4 million tokens, and OpenAI’s text-embedding-3-small costs $0.02 per million, about five cents. Storage is small too: OpenAI’s file search includes 1 GB free.

Questions are the main meter. Assume 3,000 input tokens (instructions, five retrieved passages, the question) and 400 output tokens, the sizes Hamza uses, and 20 people asking five questions a day for 22 working days: 2,200 questions.

So the model bill for a small team is tens of dollars. Add $25–$150 a month of hosting, Hamza’s range. The cost that matters is a person who keeps documents current and reads the questions the assistant failed.

Where do RAG chatbots still give wrong answers?

• The wrong passage is found: a table or clause split across chunks is answered from half of it, and a scan without a text layer gives the search nothing.
• Old and duplicate files: gmware notes that retrieval returns the 2022 price sheet as readily as the current one. Give every document an owner and a date.
• Hallucination survives: Stanford researchers found RAG-based legal research tools from LexisNexis and Thomson Reuters hallucinated 17–33% of the time, less than a general chatbot but far from the elimination vendors had claimed.
• Access leaks: OWASP lists weak access control on embeddings, and shared vector stores that leak context between users, as an LLM top-10 weakness.
• Poisoned content: a document with hidden instructions can steer answers; OWASP advises validating sources and logging retrieval.

A company answers for what its assistant says. In Moffatt v. Air Canada (2024 BCCRT 149) a British Columbia tribunal rejected the airline’s argument that its chatbot was a separate entity and ordered it to pay CAD 812.02 in damages, interest and fees after wrong fare advice. That bot faced the public, but staff act on an internal assistant’s answers too.

What do GDPR and the CNIL say about a documents assistant?

If your documents hold personal data, a model provider that processes it is your processor. GDPR Article 28 then asks for a written contract binding it to your instructions, and Article 32 for security measures that fit the risk. France’s regulator, the CNIL, adds that a company connecting an assistant to its own knowledge base answers for that processing, and that for personal data or sensitive documents an on-premise deployment is generally the safer choice.

This is not legal advice. Ask your data protection officer or lawyer where the provider processes your data, what it retains and whether files are used for training.

Build, buy or skip it: which checklist fits your case?

• Skip RAG if your whole library is under about 100–200 pages and rarely changes. Hamza suggests a cached prompt, and the ChatGPT Business page lists a reasoning window of about 320 pages.
• Buy a seat plan if everyone works in Microsoft 365, Google Workspace or Notion, users number a few dozen and the suite’s own permissions suffice.
• Use a bot builder such as Chatbase if the audience is website visitors and the content is public.
• Build custom if different groups may read different documents, answers must link to the passage, sources sit in several systems, you need your own interface or a re-runnable test set, or data must stay in a chosen region.