Enterprise RAG · Knowledge AI · Security
A language model will answer even when it does not know. Retrieval-augmented generation, or RAG, is how enterprises stop that habit: find the passages that apply, give those passages to the model, and ask it to answer from them. The answer is only as safe as the search behind it. If search can see a folder the user cannot, the model can quote it.
When RAG is the right pattern
Use RAG when the answer depends on company knowledge that changes and that not everyone should see. Policies, contracts, product documentation, case history, engineering runbooks, and internal procedures fit. A fixed script or a form is still better when the task is a known transaction with no judgment in the wording.
RAG is not a copy of the intranet with a chat box on top. It is a permissioned lookup plus a writing step, with a way for the reader to open the source.
What usually goes wrong
Teams connect a model to “all the documents” and then discover the failure modes in production:
- Stale knowledge. The index still contains last year’s policy, a draft, and the final version. The model prefers whichever passage is worded more confidently.
- Wrong permissions. Search runs as a service account that can see every site. Users receive answers from files they could not open themselves.
- Untrusted text inside documents. A retrieved page can contain instructions aimed at the model. If the system obeys that text, a document becomes a way to change the assistant’s behavior.
- No citation. The answer sounds specific and cannot be checked. People either trust it blindly or stop using it.
- No evaluation. The demo used five friendly questions. Real users ask about exceptions, old product names, and topics the corpus does not cover.
Build the knowledge set on purpose
List the sources that are allowed to answer, who owns each source, and how often it changes. Leave out inboxes, personal drives, and working drafts unless a person has explicitly published them as guidance.
Chunk documents so a retrieved passage is understandable on its own, and keep the title, owner, date, and link with every chunk. When two versions exist, the index should prefer the one marked current. If currency cannot be determined, the assistant should say so instead of blending both.
Enforce the reader’s permissions
The retrieval query must run with the identity of the person asking, or with a filter that matches what that person is allowed to open. A shared index is acceptable. A shared view of every document is not.
Test this with two accounts before launch. The same question should return different sources, or a refusal, when the second account cannot see the file. If both accounts get the same confidential paragraph, the system is not ready.
Treat retrieved text as data, not as instructions
The model should use retrieved passages as evidence for the answer. It should not follow orders embedded in those passages, such as requests to ignore policy, reveal other documents, or change its role. Separate the system instructions from the retrieved content, and keep tool use unavailable unless the workflow explicitly needs it.
This is a design constraint, not a claim that any filter is perfect. High-risk assistants should still cite the passage and leave the action to a person.
Show the source and know when to refuse
Every substantive answer should point at the document, section, and date. If retrieval finds nothing relevant, the assistant should say it does not have a source. A fluent guess is the failure users remember.
Build an evaluation set from real questions, including questions whose answer is not in the corpus and questions each test user is not allowed to ask. Run it when documents are reindexed and when the model or prompt changes.
Operate the index
Someone must own refresh, deletion, and access reviews. When a policy is retired, it has to leave the index, not only the website. When a person changes teams, their entitlements have to change the search results the same day. Log which documents were retrieved for a question so a bad answer can be traced.
How Neural Xpert helps
Neural Xpert builds enterprise RAG that stays inside real permissions: source selection, identity-aware retrieval, citations, evaluation, and the operating routine that keeps the knowledge current. The result is an assistant people can check, not a search box that invents a policy.