HiNet.

Documentation  /  Technical spec

HiNet Technical Spec

Status: Draft

This technical spec translates the HiNet functional spec into an implementation-facing architecture. It intentionally echoes the functional vocabulary: iCore, qCore, aCore, GQ, local ownership, holographic inference, fractal training, thought convergence, and SSV-like verification.

The first build target is the basic user node: a local-first iCore runtime that can own data, manage consent, build memory, call models, run consolidation jobs, and prepare for future qCore and aCore protocols.

Findings & Status Update (2026-06-22)

Validated/decided since this draft (detail in the canonical docs): - Base: Qwen3-MoE family (Apache-2.0) is the production anchor; bf16 (not 4-bit) for the knowledge/deep training stage — int4 cuts knowledge capacity >2×. On-device runtime = MLX/mlx-lm (Foundational-Model-Plan §2, Node-Packaging-Addendum). - Personalization engine: depth spectrum LoRA → DoRA → full-FT, all-linear targeting; nightly sleep cycle with replay; facts→RAG, skills→Δ. Recipes + per-tier (pocket/laptop/cloud): Personalization-Engine. - Training CLI (2026-08-24): Soup (one-YAML PEFT/TRL + layer-streaming; not Wortsman Model Soups) was evaluated as a nightly-training candidate and rejected as the runtime. It is CUDA/PyTorch-first, 4-bit-default, and does not emit FlexOlmo-composable Δs against our frozen Qwen3-MoE anchor. Keep MLX on the ComputeBackend local path. Adopt only the ops discipline: multi-leg eval-gated promote/rollback, provenance-bound ship evidence, and tests that every training flag is actually read (silent no-ops are unacceptable on a sleep clock). Detail: Personalization-Engine §6a. - Composition: compose via task-vectors (τ = θ_ft − θ_base) on one byte-identical frozen base. Mergeability ∝ delta magnitude + conflict → merge complementary · route conflicting · retrieval-union for facts; DiLoCo-cadence re-basing; cluster+route at scale. - Seams (build now): ComputeBackend (local↔attested cloud sandbox, signed WorkOrder/verified receipt), VaultReplication (encrypt-local → ciphertext to owner-owned targets), model passport. The macOS node = Tauri shell + Rust security core + Python hinetd/MLX sidecar. - Working harness (reference impl of the above primitives): experiments/e1-holographic-composition/. - Track 2 implementation surface (connectors, agentic layer, app UX): HiNet-Track2-Functional-Spec.md.

The modules/contracts below remain the foundation; reconcile against the canonical docs where they differ.

Design Principles

  1. Local-first ownership: user data, identity material, memory, and learned artifacts start on the user's device.
  2. Consent before contribution: no network participation, model call, backup, training, or inference contribution runs without a policy grant.
  3. Atomic intelligence: an iCore must be useful alone before it can be useful in a qCore.
  4. Holographic composition: any valid qCore should be composed from meaningful iCore participants, not passive data shards.
  5. Verifiable execution path: agent and quorum outputs should carry enough evidence to audit who participated, what policy allowed it, and what was signed.
  6. Research separation: MVP mechanisms should be buildable now, while fractal training, qCore emergence, and aCore cryptography are specified as protocol surfaces that can mature.

Runtime / Packaging Stack

Runtime/packaging stack: see HiNet-Node-Packaging-Addendum.md.

Architecture Overview

flowchart TD
    Owner[Human Owner] --> NodeAPI[Local Node API]
    NodeAPI --> Consent[Consent Policy Engine]
    NodeAPI --> Vault[Encrypted Local Vault]
    NodeAPI --> Memory[Memory Index]
    NodeAPI --> ModelAdapter[Model Adapter]
    NodeAPI --> Audit[Audit Log]
    Consent --> Vault
    Vault --> Memory
    Memory --> ModelAdapter
    ModelAdapter --> LocalModel[Local Model]
    ModelAdapter --> RemoteModel[Approved Remote Model]
    NodeAPI --> Consolidation[Nightly Consolidation]
    Consolidation --> Vault
    Consolidation --> Memory
    NodeAPI --> PeerProtocol[Peer Protocol Stub]
    PeerProtocol --> qCore[qCore Session]
    qCore --> Other_iCores[Other iCores]
    NodeAPI --> aCore[aCore Execution]
    aCore --> ThoughtConvergence[Thought Convergence]
    ThoughtConvergence --> ThresholdOutput[Threshold Signed Output]

Core Modules

node-identity

Responsibilities:

  1. Create or import a persistent iCore identity.
  2. Store private key material locally.
  3. Derive public node identifiers and signing keys.
  4. Sign local audit events, peer announcements, consent grants, and protocol messages.
  5. Support future DID, libp2p PeerId, wallet, or on-chain identity bindings.

MVP behavior:

  1. Generate a local Ed25519 or secp256k1 identity.
  2. Store key material in an encrypted local keystore or OS keychain where possible.
  3. Export public identity as a node profile document.

Future behavior:

  1. Bind iCore identity to an owner wallet or DID.
  2. Support key rotation and recovery.
  3. Support threshold or social recovery for human-quorum-owned iCores.

Status (2026-08 — BUILT): MVP identity is implemented in hinetd/node_identity.py: an owner-held Ed25519 keypair (private key 0600 under ~/.hinet, never leaves the device except as VaultReplication ciphertext), the self-certifying node_id = "icore_" + crockford_base32( sha256(0x01 ‖ pubkey)[:20] ) (authoritative derivation), plus node name + sign() for the registration request. Full contracts — genesis/subkey split, recovery phrase, the central registry, and capability indexing — are specified in HiNet-Node-Identity-and-Registry.md.

local-vault

Responsibilities:

  1. Store raw imported data, extracted text, metadata, embeddings, and learned artifacts.
  2. Keep data local by default.
  3. Support encrypted backups and exports.
  4. Track retention and deletion policy.

MVP behavior:

  1. SQLite stores metadata, consent grants, memory records, and audit events.
  2. File storage holds imported artifacts and derived artifacts.
  3. Encryption at rest is required before multi-user or production use.

Responsibilities:

  1. Represent what the owner permits the node to do.
  2. Gate ingestion, model calls, training, qCore participation, aCore participation, backup, and export.
  3. Support revocation.
  4. Provide a policy decision for every sensitive operation.

MVP behavior:

  1. Consent grants are explicit records.
  2. Every operation references a consent grant.
  3. Revocation prevents future use.

Policy dimensions:

  1. Data source.
  2. Data sensitivity.
  3. Allowed operation.
  4. Allowed model target.
  5. Allowed network exposure.
  6. Allowed retention period.
  7. Compensation requirement.
  8. Human approval requirement.

memory-index

Responsibilities:

  1. Convert approved data into searchable memories.
  2. Maintain embeddings and summaries.
  3. Retrieve context for local inference.
  4. Preserve provenance from answer back to source data and consent grant.

MVP behavior:

  1. Document chunks are stored with source, owner, timestamp, and consent grant.
  2. Embeddings are created only through approved model adapters.
  3. Retrieval returns memory records plus provenance.

Future behavior:

  1. Memory consolidation into higher-level autobiographical summaries.
  2. Preference model and personal ontology.
  3. Conflict detection between stale and current memories.

Implemented (P2/P2.1, 2026-06-26): one encrypted sqlite vault — FTS5 (BM25) + float32-blob embeddings with flat cosine at personal scale; chunks carry denormalized Origin facets. Retrieval = consent/provenance pre-filter → hybrid (dense + BM25 + RRF) → parent-merge → cited records; embedder multilingual-e5-base on MLX (semantic + cross-lingual). Memory records are CanonicalItems (shared normalization). Code: app/osx/hinetd/hinetd/{vault,memory,embedder,chunking}.py; design: RAG Architecture + Canonical Record.

model-adapter

Responsibilities:

  1. Provide one interface for local and approved remote models.
  2. Enforce consent policy before prompt construction and execution.
  3. Log model calls and outputs.
  4. Support future adapter training and local fine-tuning.

MVP behavior:

  1. infer(request) receives context, prompt, policy, and model target.
  2. The adapter redacts or refuses data based on policy.
  3. Outputs include model metadata and audit IDs.

Future behavior:

  1. LoRA or adapter training.
  2. Prompt tuning.
  3. Petals-style distributed model calls.
  4. qCore routed inference.

node-api

Responsibilities:

  1. Expose local user and developer APIs.
  2. Provide endpoints for identity, consent, ingestion, memory search, inference, consolidation, export, and status.
  3. Keep dangerous operations behind local authentication.

MVP endpoints:

  1. GET /node/profile
  2. POST /consent/grants
  3. DELETE /consent/grants/{grant_id}
  4. POST /ingest/files
  5. POST /memory/search
  6. POST /inference/local
  7. POST /consolidation/run
  8. POST /node/export
  9. POST /node/kill-switch
  10. GET /audit/events

audit-log

Responsibilities:

  1. Record security-sensitive and intelligence-sensitive operations.
  2. Link operations to user identity, consent grant, source records, model target, and output.
  3. Support future settlement and dispute resolution.

MVP behavior:

  1. Append-only local table.
  2. Event hash chain to detect tampering.
  3. Exportable event log.

Future behavior:

  1. Signed event receipts.
  2. qCore and aCore execution transcripts.
  3. Selective disclosure proofs for disputes.

peer-protocol

Responsibilities:

  1. Prepare the node for network identity, discovery, qCore participation, and aCore execution.
  2. Define message schemas before production networking exists.
  3. Support local simulation for protocol tests.

MVP behavior:

  1. Peer messages are schema-only or local simulation.
  2. Node can produce a signed peer announcement.
  3. Node can evaluate a qCore participation request against consent policy.

Future behavior:

  1. libp2p PeerId and pubsub.
  2. DHT or delegated routing.
  3. Encrypted peer channels.
  4. Reputation, capability advertisements, and availability proofs.

Data Contracts

Contracts are shown in YAML-like form for readability. They are not final wire formats.

NodeProfile

node_id: "icore_..."
owner_label: "local user controlled label"
public_keys:
  identity: "..."
  signing: "..."
device:
  device_id: "..."
  capabilities:
    cpu: "..."
    gpu: "optional"
    memory_gb: 16
network:
  peer_enabled: false
  qcore_enabled: false
created_at: "2026-04-29T00:00:00Z"

ConsentGrant

grant_id: "grant_..."
owner_node_id: "icore_..."
scope:
  data_sources: ["local_file_import"]
  operations: ["ingest", "embed", "local_inference", "consolidate"]
  model_targets: ["local"]
  network_exposure: "none"
retention:
  raw_data: "until_revoked"
  derived_memory: "until_deleted"
requires_human_approval: false
created_at: "..."
revoked_at: null
signature: "..."

MemoryRecord

memory_id: "mem_..."
owner_node_id: "icore_..."
source:
  source_id: "src_..."
  source_type: "file"
  consent_grant_id: "grant_..."
content:
  text: "..."
  summary: "optional"
embedding:
  vector_ref: "vec_..."
  model: "..."
provenance:
  created_by: "ingestion_job"
  created_at: "..."
  source_hash: "..."

InferenceRequest

request_id: "infer_..."
owner_node_id: "icore_..."
prompt: "..."
context_policy:
  use_memory: true
  max_records: 12
  sensitivity_ceiling: "private_local"
model_target: "local"
consent_grant_id: "grant_..."

InferenceResponse

request_id: "infer_..."
response_id: "resp_..."
answer: "..."
model:
  target: "local"
  model_id: "..."
sources:
  memory_ids: ["mem_..."]
audit_event_id: "audit_..."
signature: "optional local signature"

PeerAnnouncement

node_id: "icore_..."
capabilities:
  local_inference: true
  qcore_participation: false
  acore_execution: false
availability:
  status: "local_only"
policy_summary:
  requires_explicit_approval: true
signed_at: "..."
signature: "..."

AuditEvent

event_id: "audit_..."
event_type: "inference.local.completed"
actor_node_id: "icore_..."
consent_grant_id: "grant_..."
input_hash: "..."
output_hash: "..."
previous_event_hash: "..."
created_at: "..."
signature: "..."

Protocol Sketches

iCoreTrainingRound

Purpose: local or federated learning step that evolves an iCore while preserving owner consent.

MVP local flow:

  1. Select approved data by consent grant.
  2. Build a training or consolidation dataset.
  3. Produce summaries, embeddings, preference updates, or adapter updates.
  4. Record artifact hashes and policy decisions.
  5. Let the user inspect and roll back outputs.

Future decentralized flow:

  1. qCore or network posts a training round manifest.
  2. iCore policy engine evaluates eligibility and consent.
  3. iCore trains locally on private data.
  4. iCore emits an update commitment, not raw data.
  5. Secure aggregation, differential privacy, or swarm learning protocol merges updates.
  6. Compensation event is recorded for accepted contribution.

Key open issue:

The network must define which updates are safe to share and how to prevent model inversion, membership inference, poisoning, and free-riding.

qCoreInferenceSession

Purpose: compose multiple iCores into a larger reasoning system.

Flow:

  1. Caller submits task, budget, sensitivity class, and desired capabilities.
  2. Router finds candidate iCores by capability, consent, reputation, availability, and expected contribution.
  3. Candidate iCores accept, reject, or request human approval.
  4. Session forms a qCore membership set.
  5. qCore runs one of several inference modes: - independent committee answers, - routed Mixture-of-Experts style expert calls, - Petals-like distributed model path, - hierarchical deliberation, - federated retrieval and synthesis.
  6. qCore produces a final output with participation receipt.
  7. Settlement records compensation obligations.

Research direction:

The qCore should eventually support holographic inference: each participant remains a usable local model or model committee, while the composition acts as a larger model.

QuorumMembership

Purpose: define who participates in qCore or aCore work.

Fields:

  1. Session ID.
  2. Participant node IDs.
  3. Selection rule.
  4. Consent grant IDs.
  5. Required threshold.
  6. Timeout.
  7. Compensation terms.
  8. Slashing or reputation terms.
  9. Transcript policy.

AgentExecutionDuty

Purpose: assign an aCore execution to a verifier/operator quorum.

Flow:

  1. Caller invokes aCore endpoint.
  2. Network selects operators based on stake, reputation, capability, diversity, and availability.
  3. Operators receive encrypted execution package or TEE/zk-compatible workload.
  4. Operators execute independently.
  5. Operators produce signed commitments to outputs and metadata.
  6. Thought convergence protocol determines final output object.
  7. Operators threshold-sign the final output object.
  8. Result is returned with proof and audit transcript.

ThoughtConvergenceRound

Purpose: bridge stochastic agent reasoning and deterministic signing.

Flow:

  1. Each operator proposes an output commitment.
  2. Operators reveal output summaries and validation metadata.
  3. Convergence engine clusters outputs by exact match, schema equality, embedding similarity, contradiction tests, or task validators.
  4. Candidate output must cross threshold alpha.
  5. For high-risk duties, candidate must remain stable for beta rounds.
  6. Operators sign the final canonical output, final structured output, or final answer bundle.

Important rule:

Threshold signatures should sign exact canonical bytes. Semantic similarity can help choose those bytes, but it is not itself a replacement for message equality in cryptographic signing.

ThresholdSignedOutput

output_id: "out_..."
acore_id: "acore_..."
session_id: "sess_..."
canonical_output:
  content_type: "application/json"
  bytes_hash: "..."
agreement:
  threshold: "2-of-3"
  mode: "semantic_consensus_with_canonicalization"
  alpha: 2
  beta: 2
participants:
  - node_id: "icore_..."
    commitment_hash: "..."
    signed: true
transcript_hash: "..."
signature:
  scheme: "BLS-threshold"
  value: "..."

Decentralized Network Layer (companion specs)

The node's network participation is specified in four companion docs; this section fixes the cross-cutting contracts they share (from the integration note) so implementations stay consistent.

Shared contracts (load-bearing — must not drift):

  1. One identity everywhere. One Ed25519 key per iCore; node_id self-certifying (authoritative derivation under node-identity). The same key signs the audit chain, registration, quorum receipts, and monorepo commits. Genesis k_id (fixes the id forever) vs rotatable k_sign (day-to-day signing).
  2. Registry = index, never authority. One hinet-registry on dedicated HiNet resources in the Quorumz GCP project; the owner-signed NodeRecord (node_id → pubkey → capabilities → endpoint → membrane → liveness) is the single shared record. Because records are signed + ids self-certify, the registry can withhold/reorder but cannot forge → DHT migration stays tractable. Canonical API surface = /v1/*. (Not hinetd/registry.py, which is the local MLX model registry.)
  3. Routing seam. Registry answers "who plausibly matches" (pgvector ANN over competence centroids + tag filter + membrane pre-filter); the router runs on the asker's node and makes the decision. declared competence is unweighted until challenged/usage-proven (sybil defense).
  4. One economy. Routing QueryReceipt and monorepo AttributionRecord are the same pay-per-use primitive: pool = price − platform_share, split by contribution weight, signed receipts appended to the audit chain; MVP computes + logs locally, settlement deferred. Membrane/lease rules identical in both.
  5. No-leak invariant. Only filtered text / Δ_public-derived artifacts leave; raw vault, Δ_private, weights never do; receipts carry hashes, not content. Competence signatures are computed from the Δ_public/DeltaView surface.

Phased build (integration note): P0 local (onboarding + identity + harness + dev-CLI — largely built) → P1 registry + ask-network (committee, signed receipts) → P2 signed-HTTP transport + monorepo MVP + auto routing → P3 libp2p/DHT + attested MoErging + settlement. Open decisions (P2P phasing, decentralized-FS, central-vs-DHT registry, aggregation default) are tracked in the integration note.

Holographic Fractal Training And Inference

The functional spec asks for a decentralized weight system that allows each node to function as a neural network and any collection of nodes to assemble into a larger neural network. Technically, this is the hardest research track in HiNet.

Candidate technical interpretations:

  1. iCore as personalized adapter: each user owns local memory, preferences, and adapters over a shared base model.
  2. iCore as expert module: qCore routes tasks to relevant iCores like a sparse MoE.
  3. iCore as federated learner: iCores perform local training and share privacy-preserving updates.
  4. iCore as retrieval expert: each node contributes private retrieval, summaries, or judgments without exposing raw data.
  5. iCore as model block host: advanced nodes host layers or modules in a Petals-like distributed path.
  6. qCore as deliberative committee: multiple iCores reason independently and converge through a protocol.

The MVP should implement options 1, 4, and 6 in local or simulated form. Options 2, 3, and 5 require deeper research and network infrastructure.

Research Questions

  1. Routing: how does the network select the right iCores for a task?
  2. Composition: are qCores committees, MoE routers, distributed model paths, or hybrids?
  3. Training: what updates can be shared safely?
  4. Attribution: how is contribution measured across retrieval, reasoning, validation, and model updates?
  5. Stability: how do iCores evolve without catastrophic forgetting or runaway drift?
  6. Fractality: what invariant ensures a subset of iCores remains a useful intelligence system?
  7. Holography: can a qCore be decomposed into smaller qCores without losing core function?

Near-Term Experiments

  1. Single iCore local memory plus model adapter.
  2. Simulated qCore with three local personas or model adapters.
  3. Semantic consensus over independent answers.
  4. Local adapter or preference update from nightly consolidation.
  5. Federated learning toy example with secure aggregation.
  6. MoE-style router over user-controlled local experts.

aCore Verification Research

SSV Analogy

SSV uses secret sharing, BFT consensus, and BLS threshold signatures for deterministic validator duties. Operators agree on the exact duty data, produce partial signatures, and aggregate them into a valid validator signature.

HiNet can borrow:

  1. Operator quorum assignment.
  2. Threshold key shares.
  3. Pre-consensus, consensus, and post-consensus phases.
  4. Partial signatures.
  5. Aggregated final signature.
  6. Fault tolerance assumptions such as n >= 3f + 1.

HiNet cannot directly copy:

  1. Deterministic duty data, because agent outputs are stochastic.
  2. Exact consensus assumptions, because useful answers may be semantically equivalent but byte-different.
  3. Slashing rules, because correctness of natural language or agentic action can be subjective.

Canonicalization Strategy

The protocol should converge before signing.

Recommended signing object:

  1. A canonical JSON output.
  2. A transcript hash.
  3. A list of participant commitments.
  4. A convergence mode.
  5. A policy and validator set.
  6. A dissent record if applicable.

This makes signatures verify exact bytes while preserving a record of semantic convergence.

Verification Tiers

Tier 0: Local audit only.

Tier 1: Multi-operator semantic consensus with signed transcript.

Tier 2: Threshold signature over canonical output.

Tier 3: TEE attestation for code/model environment.

Tier 4: Optimistic dispute and fraud proof.

Tier 5: zkML or zkAgent proof for bounded computations.

The MVP should implement Tier 0 in local form and specify Tier 1 and Tier 2 data structures.

Security And Privacy Requirements

Local Node

  1. Protect private keys.
  2. Encrypt sensitive data at rest before real users.
  3. Require local authentication for control APIs.
  4. Redact or block external model calls by policy.
  5. Maintain an append-only audit log.
  6. Provide export and delete paths.

Network Participation

  1. Authenticate peers.
  2. Encrypt peer messages.
  3. Prevent replay with nonces and timestamps.
  4. Rate-limit qCore and aCore requests.
  5. Validate consent before every outbound contribution.
  6. Isolate untrusted agent execution.

Model Safety

  1. Prevent accidental leakage of private data in prompts.
  2. Track derived memories and source provenance.
  3. Protect against prompt injection from imported content.
  4. Treat remote models as data processors requiring explicit consent.
  5. Evaluate poisoning risks before federated training.

Roadmap

Phase 0: Repo And Docs

  1. Commit concept docs.
  2. Expand functional spec.
  3. Add technical spec.
  4. Create implementation backlog from specs.

Phase 1: Local iCore MVP

  1. Local node identity.
  2. Local vault.
  3. Consent grants.
  4. File ingestion.
  5. Memory index.
  6. Local inference adapter.
  7. Audit log.
  8. Kill switch.

Phase 2: Personal Intelligence Loop

  1. Nightly consolidation.
  2. Memory summaries.
  3. Preference updates.
  4. Local adapter training experiment.
  5. Export and encrypted backup.

Phase 3: Simulated qCore

  1. Multi-iCore simulation on one machine.
  2. qCore routing and membership protocol.
  3. Committee inference.
  4. Semantic consensus.
  5. Contribution receipts.

Phase 4: Peer Network Prototype

  1. libp2p identity and discovery.
  2. Signed peer announcements.
  3. Encrypted qCore requests.
  4. Consent-gated participation.
  5. Basic reputation and availability.

Phase 5: aCore Prototype

  1. Addressable agent package.
  2. Agent execution duty schema.
  3. Multi-operator execution simulation.
  4. Thought convergence rounds.
  5. Threshold signature proof of canonical output.

Phase 6: Research Integrations

  1. Swarm learning experiment.
  2. Federated learning with secure aggregation.
  3. Petals-style distributed inference experiment.
  4. TEE attestation proof of concept.
  5. zkML or zkAgent feasibility study.

Reference Dependencies

Research and implementation references:

  1. Petals: collaborative inference and fine-tuning: https://arxiv.org/abs/2209.01188
  2. Hivemind decentralized deep learning: https://github.com/learning-at-home/hivemind
  3. Swarm Learning: https://www.nature.com/articles/s41586-021-03583-3
  4. Flower federated learning: https://flower.ai/
  5. Mixture-of-Experts survey: https://arxiv.org/abs/2407.06204
  6. Modular deep learning survey: https://arxiv.org/abs/2302.11529
  7. Quorum sensing wisdom of crowds: https://www.nature.com/articles/s41467-023-37950-7
  8. Neural-like microbial consortia: https://www.nature.com/articles/s41467-021-23336-0
  9. libp2p: https://libp2p.io/
  10. Helia/IPFS: https://github.com/ipfs/helia
  11. Automerge local-first sync: https://automerge.org/
  12. SSV protocol: https://github.com/ssvlabs/ssv-spec
  13. BLS signatures: https://www.ietf.org/archive/id/draft-irtf-cfrg-bls-signature-06.html
  14. Aegean multi-agent consensus: https://arxiv.org/html/2512.20184v1
  15. Threshold AI oracle direction: https://supra.com/documents/Threshold_AI_Oracles_Supra.pdf
  16. Private ML SDK and TEEs: https://github.com/nearai/private-ml-sdk
  17. zkAgent verifiable execution: https://eprint.iacr.org/2026/199

Next: Track 2 — personal data →  ·  All documentation →