Ember is OtherU's C/C++ inference engine for DeepSeek-V4-Flash on AMD Strix Halo (gfx1151). It serves applications and agents through a local HTTP API. It is designed for this model family and hardware platform; it is not a general-purpose GGUF runtime.
Source and releases
| Resource | Link |
|---|---|
| Source | otheru-ai/ember |
| Releases | Release history and downloads |
| License | MIT license |
| Performance | Measurements and methodology |
Ember is in active development. Use the documentation and configuration shipped with your selected release when reproducing a deployment.
Supported deployment
The standard deployment uses native Linux, approximately 128 GiB unified memory, and Docker Compose with access to the ROCm devices. The Compose service listens at http://127.0.0.1:8080 by default. OtherU's existing internal deployment uses port 8000.
Since release 2026.9.3, the default model is DeepSeek-V4-Flash-Vision-Exp. Vision requests require the matching projector. Image encodings and request limits are documented in the repository.
Follow Setup and Deployment to start the service. The release configuration handles the pinned image, model downloads, and artifact verification.
API and cache
Ember provides Chat Completions, Responses, Messages, and legacy Completions interfaces, plus health and model-information endpoints. See API and Integrations for requests and client configuration.
The persistent KV cache allows eligible prompt state to be reused across requests and restarts. Cache reuse depends on the model, prompt, and cache configuration. It is separate from the agent's long-term memory.
Ember has no built-in authentication. Keep it on loopback or use an authenticating gateway for remote clients.
Models and performance
Use artifacts published for Ember and keep their model, drafter, and projector versions compatible. Custom quantization formats require explicit runtime support; GGUF packaging alone does not establish compatibility.
Throughput varies with prompt length, output type, cache state, and speculative-decoding acceptance. Text-only benchmark results should not be read as vision performance. The performance dashboard records workload and release context so results can be compared meaningfully.
For the earlier text-only artifact and its custom format, see DeepSeek-V4 Quantization Pipeline.