Documentation  /  RAG architecture

HiNet iCore — RAG Architecture (P2)

Status: v1 design (2026-06-25), from a 6-cluster survey of 2024-2026 RAG SOTA judged against HiNet's constraints (fully local/MLX on M4 Max, personal multi-source data, provenance-rich, privacy-local, nightly-incremental). Reconciled with existing objects: CanonicalItem/Origin (canonical.py), Track 2 RAGView/chunks (Track2 §4.4/§7), and local-vault / memory-index / MemoryRecord / InferenceResponse.sources (Technical Spec).

Update (2026-06-25, P2.1 — embedder pinned + validated): the embedder is intfloat/multilingual-e5-base via mlx-embeddings (MLX-native, XLM-RoBERTa), validated end-to-end including cross-lingual retrieval (English query → Hebrew note). The original pick bge-m3 ships no MLX-loadable safetensors, so mlx-embeddings can neither load nor convert it; Qwen3-Embedding-0.6B hit an mlx-embeddings runtime bug — e5 is the clean MLX-ready multilingual pin (bump to e5-large later). The cross-encoder reranker is deferred to P2.2: mlx-embeddings exposes no reranker/classification head, so a local cross-encoder needs a custom MLX head or a torch dep; hybrid (dense+BM25+RRF) + e5 is the validated v1. Pipeline fixes that mattered: dense embeds chunk text only (query↔passage symmetry), BM25 is stopword-filtered (a stopword match used to dominate RRF), non-MLX-ready HF embedders convert-on-demand into a local cache.

Governing finding: for a personal corpus the high-ROI wins are retrieval mechanics (provenance pre-filter → hybrid → rerank → parent-merge → recency) and a sleep-time consolidation loop — NOT the hyped index-time machinery (LLM-per-chunk contextualization, GraphRAG, proposition indexing, HyDE). Our advantage: the knowledge graph and the "context" are already given by the Canonical Record's provenance — we don't pay an LLM to re-derive them.

1. v1 stack (adopt-now)

1. One store — the SQLCipher vault is the index. Add sqlite-vec (vector ANN) + FTS5 (BM25) as indexes inside the same encrypted file, beside chunks and the CanonicalItem. Do not use faiss/Qdrant/LanceDB — they sit outside SQLCipher and would leak provenance + vectors to plaintext, breaking privacy-local. At personal scale use flat (brute-force) vector search, not HNSW/ANN — better recall, no filter-vs-graph pitfalls, fast on M4 Max. (LanceDB is the documented fallback only above ~1–2M chunks, with its dir encrypted.)
2. Embedder — intfloat/multilingual-e5-base (dense), via mlx-embeddings, version-pinned. MLX-native (XLM-RoBERTa), ~100 languages (mixed-language personal data), validated end-to-end incl. cross-lingual retrieval (English query → Hebrew note); e5-large is the later upgrade path. Small (~0.5 GB), resident. Pin multilingual-e5-base+version into MemoryRecord.embedding.model — a swap forces a full re-embed. No Matryoshka truncation.
3. Hybrid retrieval — dense + BM25 + RRF (k=60). Dense (sqlite-vec cosine) for semantics; FTS5 BM25 for the literal tokens that saturate personal corpora (names, emails, IDs, filenames, @handles, channels); fuse with Reciprocal Rank Fusion (parameter-free, ~20 lines).
4. Reranking — DEFERRED to P2.2 (no local reranker in the shipped build). A cross-encoder rerank over the top 30–50 RRF candidates is the second-highest ROI after hybrid (+10–25% precision) and would produce the clean top-1..5 for the LLM + citation — but mlx-embeddings exposes no reranker/classification head, so a local cross-encoder needs a custom MLX head or a torch dep; deferred. No Cohere (cloud). v1 ships hybrid (dense + BM25 + RRF) + parent-merge.
5. Chunking — structural + parent-document + late chunking; NO LLM contextualization. - Boundaries from structure we already have: message/email-part/markdown-heading/file-section, recursive ~512-tok split inside large parts. Skip semantic-embedding chunking + proposition indexing. - Late chunking — the design target: embed a whole doc in one pass and mean-pool per chunk span → recovers cross-chunk context (pronouns, "the city"→"Berlin") at zero per-chunk LLM cost. (P2.1 reality: multilingual-e5-base has a 512-token window, so the shipped build embeds chunk text only with the provenance header below; true whole-doc late chunking is a later upgrade gated on a long-context MLX embedder — e.g. e5-large / bge-m3 once MLX-loadable.) - Provenance header prepended to each chunk before embed/BM25: [Slack › Acme › #eng › from Dana › 2026-03-14] built from Origin.display() + author + timestamp — captures most of "Contextual Retrieval" benefit for free. - Parent-document return: match on small child chunks, return the natural parent (thread / email chain / file) via thread.thread_id / conversation_id / record_id — coherent + citable.
6. Provenance pre-filter + citation — the backbone (most personal-data failures are wrong- source/wrong-time, not embedding misses). Surface every Origin/envelope field as an indexed column and PRE-filter (SQL WHERE → then vector+BM25 within the filtered set; never post-filter):

Citation is then native: each chunk → record_id → CanonicalItem → render Origin.display() + origin.url, emit into InferenceResponse.sources.memory_ids. Consent + retrieval are one WHERE clause.

2. Object reconciliation

• CanonicalItem = the episodic source record (what connectors emit; the citation target).
• chunk / MemoryRecord = an indexed sub-unit: embedding{vector_ref, model} + a record_id back to its CanonicalItem, carrying denormalized provenance facets for pre-filtering. RAGView is the corpus of these, refreshed each sleep cycle.
• InferenceResponse.sources.memory_ids → resolve to CanonicalItems → Origin citation. Retrieval may read derived semantic/graph layers, but citations always resolve to the episodic source.

3. Pipeline mapping

INGEST (connector.normalize → CanonicalItem) → vault (encrypted), dedupe by content_hash
SLEEP CYCLE (offline, all LLM work confined here):
   new/changed records → late-chunk + e5-base embed → upsert sqlite-vec + FTS5 (incremental)
   → extract/consolidate facts + conflict/invalidation pass → refresh provenance graph (SQL)
   → (later) RAPTOR subtrees, identity-resolution → eval gate (recall delta on held-out probes)
QUERY (via personalize() hook in hinetd):
   route → provenance pre-filter (WHERE incl. sensitivity_ceiling) → hybrid (flat dense + BM25 + RRF)
   → parent-merge + recency/importance → inject context → Qwen3-MoE → sources[]   (rerank deferred to P2.2)

personalize(messages, role) becomes: derive filters from the query/conversation, retrieve, and prepend the parent-merged, cited context (+ a persona system prompt). Single seam, already in place.

4. Personal-memory layer (beyond doc-RAG)

• Episodic = CanonicalItems (cite these). Semantic facts + a deterministic provenance graph = derived layers the sleep cycle maintains, each row carrying record_id back to the episodic source.
• Conflict = recency-wins + source-precedence + INVALIDATE (valid-from/to), never hard-delete — keep the AuditEvent hash-chain. Gives correct temporal answers ("where did I live in 2023").
• Cross-source identity (same human across email/messages/files) is the signature differentiator but an open 2026 problem — start with a per-entity index off participants[].person_id; owner- confirmed merges; do not gate v1 on solving it.

5. Phased path (add only on a measured trigger)

v1 (above) is near-ceiling for a personal corpus. Then, each gated by a real measured gap: 1. CRAG (corrective retrieval: grade hits, fall back) when retrieval quality is the bottleneck. 2. Gated agentic / ReAct retrieval + query rewrite/decomposition for multi-hop questions. 3. Self-Route (cheap-RAG vs stuff-long-context) — the real cost lever (generation dominates). 4. RAPTOR per-entity summary subtrees for "summarize my relationship with X over a year". 5. Graph upgrade — HippoRAG-2 (KG + Personalized PageRank; continual-learning fit, cheap builder) or LightRAG (true incremental delta-insert matching the sleep cycle) — provenance on every node, citations resolve to leaf records. Trigger: measured associative/multi-hop gap, after confirming local Qwen3 triple-extraction quality on the owner's real corpus.

6. On-device cost (M4 Max 128 GB)

Resident: Qwen3-MoE (dominant) + multilingual-e5-base (~0.5 GB), kept loaded. (No reranker resident — reranking is deferred to P2.2.) Query-time retrieval (flat search, personal scale): pre-filter ~1–10 ms · hybrid ~10–50 ms → < ~100 ms pre-LLM; generation dominates. All LLM-per-record work lives in the nightly sleep cycle (incremental embed of new records only, FTS5/sqlite-vec upserts, extract/consolidate) — query time is embed + flat + BM25 + one generation.

7. Open decisions (owner)

1. Embedder lock-in — RESOLVED (P2.1): multilingual-e5-base (pinned + validated; see top banner; e5-large the later upgrade). Decided before first full embed (a swap = full re-embed). Pinned in MemoryRecord.embedding.model.
2. Recency half-life — freshness curve for personal data (tune on real query mix).
3. Conflict precedence — per-source trust order (is a Gmail thread > a Slack msg for the same fact?).
4. Forgetting budget — invalidate-never-delete (default, full audit) vs evict stale at scale.
5. Identity-merge autonomy — auto-merge on high confidence vs owner-confirmed (recommended v1).
6. LLM-extracted graph — add HippoRAG-2/LightRAG only on a measured multi-hop gap; verify local triple-extraction quality first.
7. Eval, not vendor benchmarks — validate on the owner's own held-out probe set each sleep cycle (Track 2's "before/after recall delta") — distrust self-reported numbers (e.g. mem0 LOCOMO).

8. Related evidence & what we adopt

Retrieval diversity > redundancy — Ross, Koopman, van der Vegt & Zuccon, "How retriever redundancy and diversity impact RAG effectiveness" (arXiv:2608.13956, Aug 2026). In a controlled study (FictionalQA — synthetic, so the generator's prior can't answer) they find duplicate redundancy and LLM-paraphrase of the same content do not improve answer correctness, while a diverse retrieved set (different genres/forms) improves it by 17–47% — and the gain is driven by genre diversity itself, not by more copies of the answer being present.

What we adopt: - A diversity-aware selection step over the hybrid-retrieved candidates before the generator — spread across Origin genre/source/thread (mail vs file vs chat vs code; distinct senders/threads), MMR-style, instead of returning near-duplicate top-k. This is the cheap, principled alternative to the deferred cross-encoder reranker (§1.4): rather than re-scoring for relevance, we re-select for diversity, which the evidence says is what actually moves the generator. - content_hash dedup stays but is now explicitly necessary-not-sufficient — the study confirms exact copies add nothing, so: dedup, then diversify. - HiNet's vault is inherently multi-genre (mail / files / chat / code / calendar) and Origin already carries that signal, so diversity selection is near-zero extra cost. - Open question we'd raise with the authors (see community notes): does the diversity dividend hold for a single owner's personal cross-source corpus (genres = mail vs notes vs chat, answer recurring across them)? HiNet's provenance-rich personal RAG is a natural real-world testbed; the competence diversity index (effective-rank / distinct-source, Routing §2.5.1) is a candidate way to operationalize their "genre diversity."

Bottom line: v1 = provenance pre-filter + hybrid (multilingual-e5-base + FTS5 BM25 + RRF) + parent-merge + recency/invalidation + late chunking + provenance headers + episodic/semantic split + deterministic provenance graph + sleep-time consolidation — all inside the one SQLCipher vault, on MLX, injected through personalize(). (Cross-encoder reranking deferred to P2.2.) Everything heavier is enrichment on a measured trigger.

Next: E1 — holographic composition →  ·  All documentation →