Kyojin
Kyojin is Yamz Labs' open source inference engine for AMD Strix Halo (gfx1151), built on ExLlamaV3 with speculative decoding, serving their own EXL3 quantised builds of Qwen3.8-Flash-Next, GLM-5.3-Flash and MiMo-V2.6-Flash (95 to 106 GB) on one 128 GB machine through an OpenAI-compatible API. The script builds the latest tagged release with AMD's gfx1151 ROCm wheels, following Kyojin's own container recipe.
Install
bash -c "$(curl -fsSL https://raw.githubusercontent.com/Heretek-AI/ProxmoxVE/main/ct/kyojin.sh)"Run as root in the Proxmox VE shell. Pick Default for the sizing below, or Advanced to choose storage, network and resources yourself.
Notes
- Warning: Kyojin needs a gfx1151 GPU (Ryzen AI Max+ 395, 128 GB). The container needs /dev/kfd and /dev/dri passed through, which the script sets up when GPU passthrough is enabled.
- Warning: The 100 GB-class packs need almost all of a 128 GB machine, so the container gets a 120 GB RAM limit by default. Lower it only for a smaller context or pack.
- Warning: OpenAI-compatible API: http://<IP>:8731/v1. The server has no API key, so keep it on a trusted network.
- The install downloads about 14 GB of ROCm wheels and builds the engine, which takes 15 to 30 minutes.
- The first start downloads the Qwen3.8-Flash-Next pack (about 95 GB), then loads it. Follow it with: journalctl -fu kyojin
- Change the model (qwen, glm or mimo), context or port in /opt/kyojin/kyojin.env, then run: systemctl restart kyojin
- Check the GPU setup with: systemctl stop kyojin && kyojin-check
Script details
You might also need
Hermes Agent
LXCAI / Coding & Dev-Tools · debian 13
Hermes Agent is a self-improving AI agent built by Nous Research with persistent memory, autonomous skill creation, computer-use (CUA) support, and multi-platform gateway integration. This enhanced Heretek build includes pre-configured passwordless sudo for the hermes user, native X11/input shared libraries for cua-driver, and Docker Engine with Buildx and Compose v2.
GSQHalo.cpp
LXCAI / Coding & Dev-Tools · debian 13
GSQHalo.cpp is a llama.cpp fork tuned for ISTA-DASLab's GSQ-RCO quantizations of Qwen3.8-Flash-Next on AMD Strix Halo (gfx1151), with faster prefill kernels, MTP that fits in memory and a KV cache on SSD. The script installs AMD's ROCm 7.14.1 for gfx1151 in its own prefix, builds the fork from source and serves llama-server's OpenAI-compatible API.
Gufo
LXCAI / Coding & Dev-Tools · debian 13
Gufo is an open source local inference engine built and tuned for AMD Strix Halo (Ryzen AI MAX+ 395, Radeon 8060S gfx1151). It serves Qwen3.8, DeepSeek V4 Flash, speech, embedding, image and video models through an OpenAI-compatible API, with speculative decoding, concurrent requests and prompt caching. The script installs the official release build through Nix, using prebuilt ROCm from the Gufo binary cache.
Strata
LXCAI / Coding & Dev-Tools · debian 13
Strata is an open source inference engine for Qwen3.8-Flash-Next and other large MoE models, with a Strix Halo (gfx1151) HIP engine, MTP drafting and an OpenAI-compatible API. The script installs the Strata Terminal, which downloads the signed engine for the GPU with its own HIP runtime, and serves ISTA-DASLab's Qwen3.8-Flash-Next Coder by default.