Hugging Face Integrates Nunchaku 4-bit Diffusion into Diffusers Library
SVDQuant-based quantization now runs natively in Diffusers, cutting VRAM requirements from 24GB to 12GB while accelerating inference.
#inference-optimization
ZeroGPU's Claude Code Plugin Routes Tasks to Smaller Models for Cost Reduction
A new plugin directs lightweight AI tasks to specialized smaller language models, targeting inference cost reduction through intelligent task routing.
Thaw adds Git-style branching to running LLMs, enabling mid-inference agent forks
A new open-source tool lets developers branch LLM inference mid-generation, skip redundant prefill computation, and merge agent outputs—addressing a core bottleneck in multi-agent reasoning systems.