inference engines · runtime matrix
Two validated routes. vLLM XPU serves five of six recipe families on GPTQ-INT4 artifacts; llama.cpp SYCL carries GGUF workflows like Muse-Glimmer. OpenVINO is under evaluation. SGLang is blocked.
validated research · gptq-int4
Primary engine for GPTQ-INT4 artifacts: five of six validated model families, MTP and DFlash speculation, fp8 KV, and OpenAI-compatible server APIs. Pinned container image digests with family-specific patches.
production · gguf workflows
The GGUF route — Muse-Glimmer-30B vision + DFlash at 128K context. Build with -DGGML_SYCL_F16=ON and run flash attention with a q8_0 key cache and q4_1 value cache.
runtime matrix
Primary engine for high-throughput GPTQ-INT4 serving
106.7 tok/s Qwen3.8 · 186.6 tok/s Nemotron · 170.9 tok/s MoE
Serves five of six recipe families. Features speculative decoding via MTP and DFlash drafts, prefix caching, and fp8 KV cache compression for deep 128K context windows.
vllm/vllm-openai-xpu@sha256:f01e24f6…ONEAPI_DEVICE_SELECTOR=level_zero:gpumtp_nightly + mtp_boundaryNative oneAPI DPC++ SYCL compilation for GGUF
42.8 tok/s decode · 3,240 tok/s pp512 · 128K context
High-speed GGUF inference. Requires flash attention with a q8_0 key cache and q4_1 value cache. Do not use q4_0 for the key cache.
-DGGML_SYCL_F16=ON -DGGML_SYCL_TARGET=INTEL-ctk q8_0 -ctv q4_1 -fa 1-ngl 99 offloads all layers to Arc GPUIntel official inference stack across client and server hardware
Broad format support · NNCF INT4/INT8 quantization
Supported Intel inference stack under active evaluation. A validated XeCores recipe still needs repeatable Battlemage measurements before publication.
High-throughput structured generation runtime
Pending upstream Intel GPU kernel support
Currently blocked on Intel XPU backend kernel changes. Will be reassessed as upstream XPU device support develops in SGLang releases.