TECHNICAL RESEARCH NOTE 02SYSTEM ARCHITECTURE

MMRY system specificationLayers, contracts, and reference sketches.

This page is for engineers and researchers. It documents the layered pipeline from multimodal capture through ingestion, intelligence, adaptive resolution, and hybrid recall — with illustrative code contracts drawn from observed implementations and open canon decisions (D0, D15–D18).

DIAGRAMEnd-to-end pipeline

Capture → ingestion → runtime → intelligence → lifecycle → recall

STORESKG + VDB + gist

Exact graph and vector index on one SQL adapter for talk; capsule index for vault

PAIR TESTCompress + selective retrieve

Index lookup must not require whole-file unfold (D15)

01 / SYSTEM DIAGRAM

Full-stack memory pipeline.

Six functional layers. Data flows downward through ingestion and intelligence; resolution bands govern what ordinary recall may see.

MMRY system architecture diagram showing Capture layer, Memory ingestion, Private runtime, MMRY Intelligence, Memory lifecycle resolution bands, and Recall and reasoning
MMRY layered architecture — capture through recall. Memory Cloud holds the vault; MEMO attaches feeling tracks; talk uses factual + affective traces with Imprint re-rank.
02 / LAYER REFERENCE

What each box does.

Functional responsibilities — not product names.

LAYER 01

Capture layer

Timestamped evidence enters with consent scope — not as asserted facts.

  • Voice & conversation
  • Photos, video & documents
  • Apps, messages & activity
  • Location, devices & sensors
LAYER 02

Memory ingestion

Segments episodes, resolves entities, writes linked representations with epistemic status.

  • Consent & privacy controls
  • Multimodal extraction
  • People, places & entity resolution
  • SensAffect imprints
LAYER 03

Private runtime

Edge-first processing with fail-closed writes and inspectable audit trails.

  • Provenance, access & audit
  • Pi 5 + Hailo-8 edge node
  • Local models & storage
  • Encrypted cloud bridge
LAYER 04

MMRY intelligence

Core engine: compress paired information + affect; index for selective retrieve.

  • Memory orchestrator
  • Semantic compression
  • Temporal & life timeline
  • Semantic vector memory
  • Knowledge–experience graph
LAYER 05

Memory lifecycle

Human-like view bands. Retrieval probability decreases; compressed core may persist.

  • VIVID → CLEAR → FADED → HAZY → FORGOTTEN
LAYER 06

Recall & reasoning

Assembles inspectable context for the next action without whole-file unfold.

  • Hybrid graph + vector retrieval
  • Context assembly
  • Personal memory twin
  • Assistant / agent interface
03 / REFERENCE IMPLEMENTATIONS

Code contracts and examples.

Illustrative sketches — not production drop-ins. They encode the pass/fail tests from the canon: paired store, selective retrieve, separated affect, protected decay.

TypeScript

Episode boundary — one interval is one memory

Memory event boundaries mark start/end on a timeline. Object, face, place, and event markers share the same interval identity (RX-27).

interface MemoryEventBoundary {
  eventId: string;
  startTimeMs: number;
  endTimeMs: number;
  markers: Array<{
    kind: "object" | "face" | "place" | "event";
    label: string;
    startTimeMs: number;
    endTimeMs: number;
  }>;
  tracks: {
    picture: string;   // compressed segment ref
    sound: string;
    feeling: string;   // MEMO / SensAffect imprint ref
    pose?: string;
    words?: string;
  };
}

// Ingest attaches modalities to the same episode — not separate drawers.
function bindToEpisode(
  boundary: MemoryEventBoundary,
  imprint: AffectiveImprint
): MemoryEventBoundary {
  return {
    ...boundary,
    tracks: { ...boundary.tracks, feeling: imprint.imprintId },
  };
}
04 / INTERFACE CONTRACTS

Pass/fail tests for implementations.

ContractPassFail
Compression + selective retrieve (D15)Index lookup returns one chunk/segment/trace without whole-file unfoldMarketing ratio with no rebuild; zip sold as memory
Complete core (D16)Forgotten band is a view; detail remains addressable in the coreIngest discards frames with no lineage to recover them
Information + affect paired (D0)Factual trace and affective trace (or picture + feeling track) stay linkedLog-only facts or mood-only scores presented as memory
KG + VDB talk store (D18)Exact graph and vector index both present on the conversational pathSimilarity-only search with no named edges for exact recall
One attachment law (D17)New modality uses episode interval, index, and cross-track joinsSeparate silo per sense with no relatable episode identity
READ NEXT

Research note 01 — narrative architecture.

The home page carries biological influence, SensAffect exposition, lineage, and the evidence ledger. This page is the engineer-facing complement.

← Return to research note 01