intel arc pro b70 · two public cookbooks
Everything below was measured on this hardware: six LLM model families on llama.cpp SYCL and vLLM XPU, plus MiniMax H3 text-to-video on ComfyUI. Exact image digests, required patches, and the honest limits of every number.
inference cookbook · text & vision
Six model families, one page per family, each pinned to its own container digest and patch list: Qwen3.8-27B, Qwen3.6-35B-A3B, Qwen3.6-27B, Nemotron-3.5-Lightning, Ornith-1.5-35B-A3B, and Muse-Glimmer-30B. Power sweet spots, KV cache policy, and a stable cross-model benchmark format.
videogen cookbook · video generation
MiniMax H3 FL2VA text-to-video on ComfyUI 0.33.0 with PyTorch 2.13.0+xpu. Six weight files (~40 GiB), a two-card role split (UNet+VAE on GPU0, CLIP on GPU1), a single-card CPU-CLIP fallback, and a gallery where every clip lists its wall time.
text & vision inference
quantized on the B70 with gptqmodel 7.3.2
106.7 tok/s C1 (n=5, cache on) · C5 127.4 Σ-streams
The most-served model on this host. Draft-INT4 overlay lifts the cache-off cell to 112.7; concurrent serving via mixed-split v5 reaches C32 224.2 on short prompts. Also the only family with Windows 11 standalone kits.
vllm/vllm-openai-xpu@sha256:f01e24f6… · vLLM 0.27.2rc1.dev77 · kernels 0.1.12.3SergiioB/Qwen3.8-27B-GPTQ-Int4-sym-G128-MTP-BF16 rev 9d189a60 (~18.2 GiB, MTP tensors kept BF16)MTP-preserved community GPTQ, full 128K window
170.9 tok/s MTP4 (p512/g128, n=5) · 101.6 tok/s at exact 128K
MoE decode reads only the ~3 GB of active experts per token, so it self-limits to ~140 W: run it at a 150 W cap — same speed, 80 W less heat. MTP2 beats MTP4 deep in context (85.8% acceptance at 128K).
vllm/vllm-openai-xpu@sha256:2c427ef4… · vLLM 0.26.1rc1.dev457 · kernels 0.1.12llmfan46/Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-Preserved-GPTQ-Int4MTP-preserved community GPTQ, heretic-v2 base
69.3 tok/s MTP4 · 47.6 tok/s held at full 128K
The KV cache decides this recipe: FP16 KV needs ~9.5 GiB at 128K and does not fit; fp8 halves it. In a resident 32K document session, follow-up TTFT drops from 38.2 s cold to ~4.1–4.5 s on 91% prefix hits.
2c427ef4 digest (same as the MoE) · vLLM 0.26.1rc1.dev457llmfan46/Qwen3.6-27B-uncensored-heretic-v2-Native-MTP-Preserved-GPTQ-Int4GPTQ-INT4 G64 target + NVFP4→BF16 DFlash draft
186.6 tok/s C1 (p2048/g128, n=5, accept 56.5%) · 7,160 tok/s cold input
Native MTP accepts 0% on this stack; the converted DFlash draft is the working speculator — 1.81× over the matched no-spec control. Capacity run completed 119,904 tokens at a 120K limit. Two fixes went upstream (vllm#52159, vllm-xpu-kernels#524).
vllm/vllm-openai-xpu@sha256:1da0a954… · vLLM 0.26.1rc1.dev668 · kernels 0.1.12.3SergiioB/…-GPTQ-INT4-G64-sym (~18 GB) + draft …-DFlash-BF16 (~1.7 GB)qwen3_5_moe: 30 GDN + 10 full-attn layers, 256 experts × 8 active
106.6 tok/s C1 host-measured · LMX card 108.4 tok/s, 9,073 prefill (230 W)
There is no official GPTQ for this architecture, so the target is a local experts-only quant with the MTP layer left in BF16. MixedCal-v2 calibration cuts RTN fallback from 24.76% to 10.37% — a conversion-quality win at speed parity. Self-reported E2 with raw evidence.
f01e24f6 digest (same generation as Qwen3.8) · vLLM 0.27.2rc1.dev77 · WNA16 MoE backendSergiioB/Ornith-1.5-35B-A3B-GPTQ-Int4-sym-G128-MTP-BF16-MixedCal-v2 (~22.77 GiB, 6 shards)UD-Q4_K_XL + mmproj + dflash GGUF trio
26.8 t/s at 128K (n=5) · +52% DFlash gain at p32768
The llama.cpp path: reasoning channel, vision via mmproj, and block-diffusion drafting all verified. The build flag -DGGML_SYCL_F16=ON is mandatory — without it prefill loses 3.4× (pp4096 1301 → 293 t/s). The DFlash gain grows with context depth.
d2f83055d · IntelLLVM 2026.0 · GGML_SYCL_F16=ONpower sweet spots · measured, single-stream p2k/g128
diffusion video generation
GPU0 UNet+VAE (~26.9 GB) · GPU1 CLIP (~17.0 GB)
~50.3 s/it SDPA · 43.72 s/it OmniXPU ESIMD+FP8 · 5 s clip ≈ 11 min
Two cards, two roles — a role split without tensor parallelism. Native canvas 1344×768; 124 frames ≈ 5 s, 243 ≈ 10 s, longer pieces concatenate from acts. OmniXPU kernels buy 1.15× s/it (grainier); the turbo LoRA cuts sampling to 8 steps.
Scene is finished clip length; generate is ComfyUI wall from prompt to saved file. Single observations at 180 W, 8 steps, turbo 0.75 — planning numbers, published medians need repeats.
where to start