Architecture Overview

System Goals

• Local-first inference and control — no external API dependency for core operation
• Deterministic execution pathways for critical actions (HID injection, file writes)
• Service composition with explicit trust boundaries
• Operator authority at all layers — OtherU acts, humans approve or override

Major Planes

1) Perception

Captures relevant state from the environment:

• JetKVM HDMI capture — real-time desktop screenshots via hardware bridge
• Vision resize pipeline — raw capture letterboxed to model input resolution
• Optical flow cursor tracking — Lucas-Kanade tracker, confidence-scored position
• Memory retrieval — Hindsight (semantic + keyword + entity-graph recall) across Hermes Agent

2) Reasoning, Action and Memory

These three planes are all Hermes Agent: how it routes a request by complexity, the typed tools it can execute, and its three memory systems (Hindsight, session search, skills).

They are documented in full on the Hermes Agent page.

Reliability Model

• ember-server runs as a Docker container with restart on failure
• KV-cache on disk survives container restarts
• Hermes Agent handles channel disconnection and retry
• Cascade routing — if primary model is unavailable, Hermes falls through the configured model chain

Inference Stack

The reference deployment runs a single model via ember-server on AMD ROCm:

OS & Storage

Container Stack

The old multi-container OtherU Core stack (coordinator, orchestrator, responder, reasoner, coder, Fara, hardware-bridge, LightRAG, Redis, GSD) has been retired in favour of this simpler architecture. All agent roles are now handled by Hermes Agent's tool dispatch against the single ember-server model.