Hardware: Framework Desktop + jetKVM

Why Hardware Integration Matters

OtherU requires a trusted execution and interaction path. The Framework Desktop provides high-performance local inference while JetKVM creates a deterministic, out-of-band control channel for any connected computer — no software installation required on the target machine.

Framework Desktop

OtherU Core is designed around the Framework Desktop with an AMD Ryzen AI Max+ 395 (Strix Halo) processor.

Why unified memory matters: The iGPU accesses all 128GB of system RAM directly. Models that would require a high-end discrete GPU — including 46GB+ LLMs — run on this machine without any discrete GPU card.

JetKVM Role

JetKVM is a compact hardware device that provides:

• HDMI capture — real-time visual context at full resolution via WebRTC
• USB HID injection — deterministic mouse and keyboard control of any computer
• Out-of-band management — operates regardless of target OS state (BIOS, lock screen, crashed OS)

The target machine sees only a USB keyboard and mouse — no drivers or agents needed.

Integration Pattern

Coordinator
    │
    ▼  typed interaction intent (click x,y / type "text" / press Enter)
Hardware Bridge (port 8005)
    │
    ▼  coordinate validation + inverse transform (vision space → real screen)
JetKVM
    │
    ├── /screenshot  → WebRTC frame capture
    ├── /mouse/move  → USB HID mouse movement
    ├── /mouse/click → USB HID click
    └── /keyboard    → USB HID keypress / type

Screenshot Pipeline

1. Bridge captures raw frame from JetKVM WebRTC stream
2. Letterboxes the capture to the vision model's input resolution
3. Returns image_base64 + letterbox_scale + letterbox_padding + mouse_position
4. Fara-7B sees the letterboxed image and decides an action with coordinates in that space
5. Coordinator inverse-transforms coordinates back to real screen space before HID injection

Mouse Position Tracking

• Optical flow: Lucas-Kanade tracker initialized at screen center
• Confidence scoring: 0.0–1.0; falls back to dead-reckoning if tracking drift exceeds threshold
• Cursor hint: current cursor position injected into Fara's observation context so it doesn't need to locate the cursor visually

Display Configuration

Always use the display's native resolution EDID. Custom low-resolution EDIDs may lack fallback timing modes, causing the target GPU output to go dark. Software letterboxing in the bridge pipeline achieves the same result without touching the EDID.

# Reset to native EDID via bridge
curl -X POST http://127.0.0.1:8005/edid \
  -H 'Content-Type: application/json' \
  -d '{"edid_name": "T749-fHD720"}'

Operator Guidance

• Pin JetKVM firmware version before any production rollout
• Test capture latency and HID round-trip after bridge updates
• Hardware bridge source is baked into its Docker image — apply patches via docker cp then restart
• Set MAX_RECORD_DURATION appropriately for your use case (default is conservative)