RAG Is Not a Mandatory Pipeline, but an On-Demand Evidence Capability

Many RAG tutorials begin with a short pipeline: split documents into chunks, compute Embeddings, and write them to a vector database; when a question arrives, retrieve the Top-K results and let a large language model organize the answer. This pipeline explains how external knowledge enters generation, but it does not answer the hardest question in real engineering work: does the matched material belong to the correct repository, the correct version, and the correct call relationship?

I built mt_rag not to make every Codex task pass through a retrieval pipeline first, but to provide a local evidence capability that can be invoked on demand, can return to the original material, and stops when evidence is insufficient whenever a task must cross code, documentation, work tracking, attachments, and approved configuration.

That determines its place in the overall development system: safety rules and the user's goal still come first; the current source code remains the preferred source of implementation facts; Local RAG narrows the cross-source search space and connects candidate material back to verifiable evidence.

Knowledge Map for This Series#

The complete architecture can be divided into ten steps with clear responsibilities. They are dependencies inside the data system, not a Workflow that every development task must run from beginning to end.

flowchart TB
  S["2 Enterprise Sources<br/>code / documentation / work tracking / attachments"]
  R["3 Raw Data Control Plane<br/>Discovery / Freshness / Fetch Decision"]
  C["4 Local Immutable CAS<br/>Raw / IR / CPG / Manifest"]
  I["5 Typed IR<br/>DocumentIR / Issue IR / Code CPG"]
  E["6 Canonical Resolution<br/>Entity / Relation / Evidence"]
  D["7 Incremental Projection<br/>Event / Delta / Cursor / Tombstone"]
  Q["8 Query Stores<br/>Vector / Keyword / Neo4j"]
  H["9 Hybrid Retrieval and Graph Expansion<br/>70/30 RRF / Parent / Verified Path"]
  L["10 Controlled LLM<br/>Rewrite / Controller / Rerank / Answer"]
  O["11 Developer Output<br/>Summary / Chain / Citation / Limitations"]
  S --> R --> C --> I --> E --> D --> Q --> H --> L --> O

These ten steps organize four kinds of knowledge:

Knowledge domain	Core question	Corresponding chapters
Source and identity	Where did the material come from, which version is it, and what can it prove?	Enterprise Sources, Raw Control Plane, CAS
Structure and relationships	How do documents, issues, and code preserve structure, and how are cross-source entities aligned?	Typed IR, Canonical Resolution
Freshness and queries	How do changes safely enter different stores, and how are semantics, exact terms, and relationships handled together?	Incremental Projection, Query Stores, Hybrid Retrieval
Generation and delivery	What may the model decide, and how do candidates become engineering conclusions?	Controlled LLM, Developer Output

Not “One Database Plus One Prompt”#

The current architecture has three planes.

The evidence plane preserves replayable facts. Raw Snapshots, DocumentIR, and the complete Code Property Graph enter a local SHA-256 CAS; immutable Manifest Generations then reference these objects together with their Hashes, Commits, Locators, and Parser Versions. This plane answers “what exactly was read at that time,” instead of mixing the Manifest itself and CAS Objects into the same storage contract.

The query projection plane serves different query modes. Qdrant stores Dense Vectors; SQLite FTS and Exact Alias handle keywords and Identifiers; Neo4j stores Canonical Entities, Evidence-backed Relations, and the Semantic CPG. None of them is the final source of truth. They are query views that can be rebuilt from the evidence plane.

The answer plane constrains retrieval, graph paths, and the LLM under the same evidence contract. After the first round of Dense/Keyword retrieval, it searches for Verified Paths only from the graph_ids of the highest-ranked hits; the Controller may choose only answer, rerank, retrieve_again, or insufficient, and the total number of retrieval rounds is at most two. Deterministic code then validates the final call chain, citations, and Confidence.

flowchart LR
  A["Evidence Plane<br/>CAS + Manifest"] --> B["Projection Plane<br/>Qdrant + FTS + Neo4j"]
  B --> C["Answer Plane<br/>Controller + Guard"]
  C --> D["Grounded Developer Answer"]
  D -. "citation / locator / commit" .-> A

This separation looks more complex than “write text into a vector database,” but it solves different problems: CAS guarantees replayability, vectors and keywords support discovery, the graph proves extracted relationships, the LLM performs semantic judgment, and deterministic Guards prevent conclusions from crossing the evidence boundary.

What Is Actually Implemented Today#

Here, “validated locally” must be distinguished from “switched to production.” The current runtime policy keeps System RAG in Shadow, requires separate approval for Production Cutover, and forbids automatic deletion of Legacy data. Shadow Readiness therefore proves that the new path is internally consistent; it does not mean that production callers have switched over.

The current verifiable state of the local Shadow comes from one recent snapshot of a complete Generation:

Projection	Current local evidence
Evidence corpus	A large local corpus with both Parent and Child chunks
Query projections	Vector, keyword, and graph views bound to the same generation
Readiness	Shadow validation passes with current vector and graph cursors

This is a point-in-time local validation state, not a permanent corpus size and not an accuracy conclusion. What matters is that these projections are bound to the Manifest Hash, Collection, Graph Projection, and Consumer Cursor; if any version is inconsistent, the query code reports stale instead of continuing to produce an apparently normal answer.

How an Engineering Question Passes Through the System#

Suppose the question is, “What is the call chain for a GraphQL field from the frontend to the Provider?”

1. the current System RAG Shadow Grounded Query passes the original question directly to Hybrid Retrieval as active_query; Rewrite in the legacy answer pipeline belongs to a different query entry point and is not the first step of this path;
2. the Dense Lane handles semantic similarity, while the Keyword Lane handles exact symbols such as Field, Resolver, and Class;
3. Weighted RRF merges Ranks instead of directly adding raw scores from different scales;
4. a Child hit traces back to its Parent Context to preserve readable context;
5. only the Canonical Entities from the most relevant hits are used as starting points for querying paths with Verified Evidence in Neo4j;
6. the Controller determines whether the evidence is sufficient and permits at most one targeted retrieval retry;
7. the Developer Answer outputs only the verified node order; if an edge is missing, it records the evidence gap in limitations instead of inventing parameters, return values, or deployment status.

If the question already provides the exact file and Symbol, reading the current workspace directly is usually faster. RAG is not a mandatory Runtime here, but a tool for “finding the entry point and supplementing cross-source context”; final code facts should still be verified against the current Commit.

Four Invariants I Insist On#

1. Source identity must not be lost during Embedding. Every Chunk must be traceable back to its source_record_id, Locator, Original Hash, and CAS URI.
2. Similarity must not be promoted into proof of a relationship. A Vector Hit is a candidate; every Verified Graph Edge must carry independent Evidence.
3. Freshness must not become part of content identity. Time may affect ranking, but it must not create a new Content Hash and Embedding Identity every day.
4. Readiness does not grant deployment authority. A passing Shadow, passing tests, and zero Cursor lag cannot replace explicit confirmation for Production Cutover.

Failure Modes and Trade-offs#

Failure mode	System behavior	Cost and trade-off
An old Confluence page ranks highly	Query-time Freshness lowers its weight while preserving its historical evidence identity	A still-valid old document may rank lower, so version changes override the age policy
A Code hit comes from an old Commit	currentness_valid=false, so it cannot be treated as the current implementation	Repo Pins and Commit Lineage must be maintained
Vectors succeed while Neo4j lags	the Graph Cursor/Manifest Gate reports stale and does not output a Verified Chain	Availability gives way to call-chain correctness
The LLM cannot return valid JSON	use a Deterministic Fallback; return insufficient when there is no evidence	The answer may be less fluent, but it will not elaborate on unknown facts
Writing to one Store fails	the failed Consumer retains its old Cursor; other Consumers commit independently or keep their own last committed state	Multiple Cursors add operational state, but provide failure isolation

Verifiable Project Evidence#

• README.md: overview of the current data scope, directories, commands, and Shadow state;
• Runtime policy: CAS, Embedding, 70/30 RRF, the two-round Controller, and the Shadow/Cutover boundary;
• src/mt_rag/system/pipeline.py: assembly entry point from Raw through IR/CPG, Chunk, and Manifest;
• src/mt_rag/retrieval/system_index.py: Qdrant, Keyword, Exact Candidate Completion, and Weighted RRF;
• src/mt_rag/system_graph/query.py: Projection/Cursor validation and Verified Path queries;
• src/mt_rag/query/runtime.py: the controlled Controller and Grounded Answer runtime;
• Local readiness artifacts: generation bindings, Hashes, Cursors, and Blockers without exposing source-specific content.

References#

• Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks;
• Datawhale All-in-RAG: supplements Chinese-language fundamentals and the chapter learning path; it is not evidence for the project implementation described here;
• Cormack, Clarke, and Büttcher, Reciprocal Rank Fusion;
• Qdrant official documentation on Hybrid Queries;
• Neo4j official documentation on Cypher Path Matching;
• Qwen3 Embedding Technical Report.

The detailed ten-step series is currently available in Chinese. It does not repeat one large diagram: each article explains why a particular boundary exists, how the current code implements it, where it stops on failure, and what complexity I accepted in exchange.