Baseten is now a supported inference provider on Hugging Face Hub, enabling developers to run open-weights LLMs like DeepSeek V4 Flash and Kimi K3 directly from model pages.
Ukraine-based MacPaw partners with Liquid AI to build locally hosted AI inference, positioning SetApp as a platform for AI-native applications with credit-based pricing.
The AI infrastructure company deploys portable compute units across three continents, betting distributed inference closer to users will outpace centralized facilities.
OpenAI cut GPT-5.6 Luna prices 80% and introduced speed-tiered variants, signaling a shift from capacity competition to cost-per-outcome optimization.
OpenAI reduces GPT-5.6 Luna costs by 80% and Terra by 20%, while launching Fast mode for GPT-5.6 Sol to deliver 2.5× faster inference at doubled pricing.
Coinbase is reported to have migrated inference workloads to Zhipu AI's GLM and Moonshot's Kimi, reportedly achieving a 50% cost reduction, though the claim lacks independent verification.
The Harvard-founded chip maker closes a $300M Series C led by Sequoia, doubling its valuation in seven months on custom prefill and decode silicon.
Alphabet is developing a custom AI accelerator designed to reduce per-token power consumption and reduce dependence on Nvidia's dominance.
A tech investment firm finances AI infrastructure using inference-specific silicon as collateral, signaling a shift from training chips to efficient model-serving hardware.
IBM Research and Hugging Face show that intelligent model selection in agentic systems requires optimizing infrastructure and workload patterns, not just picking cheaper models for easy tasks.
A modular voice AI pipeline achieves sub-second latency by pairing Google DeepMind's Gemma 4 model with Cerebras inference acceleration, powering conversational robots and assistants.
The inference-focused chipmaker has raised $800M total and secured $1B in system orders as competition in custom AI silicon intensifies.
Wix-owned no-code platform Base44 rolls out proprietary Base1 model trained on tens of millions of real user interactions, betting on specialization over dependence on external LLMs.
Run a private, OpenAI-compatible LLM endpoint on HF infrastructure with one command—no Kubernetes, billed per-minute.
OpenAI unveiled Jalapeño, an ASIC chip co-developed with Broadcom to handle AI inference workloads and reduce dependence on Nvidia GPUs.
OpenAI unveiled Jalapeño, a custom inference processor built with Broadcom, marking the company's entry into purpose-built silicon and a bid to reduce reliance on Nvidia GPUs.
OpenAI and Broadcom announced Jalapeño, a purpose-built AI chip for language model inference, designed to improve efficiency and reduce costs at scale.
Hugging Face demonstrates real-time GitHub issue classification using local open-weights models on NVIDIA hardware, eliminating API dependency.
The inference infrastructure startup is raising at a $13B valuation just five months after a $300M Series E, but split pricing reveals how startups are gaming funding rounds.
As inference costs soar, enterprises are discovering that smaller models handle 80% of workloads just fine—and the economics could reshape OpenAI and Anthropic's path to IPO.
A GitHub research project claims to compile LLM computation graphs into single CUDA kernels with formal correctness guarantees, but lacks published benchmarks or third-party validation.
A Hugging Face hackathon project demonstrates that serving four different small models in a single agent economy is tractable when infrastructure abstracts tokenizer variance.
Organizations are exploring on-premise language models as pre-filters to reduce API spend on commercial LLMs, though cost savings remain context-dependent.
The AI chip startup is shifting focus to its inference-as-a-service platform following its $20B partial exit with Nvidia.
The AI chip startup is pivoting toward inference-as-a-service, backed by existing investors including Disruptive and Infinitium.
The Korean chip startup raises Series B at $570M valuation, targeting the data-movement bottleneck that GPUs can't solve alone.
A new multi-part guide demystifies torch.profiler traces, starting with matrix operations and scaling to large language model optimization.
A new inference cloud startup backed by FUSE VC is deploying specialized chips to undercut GPU-heavy competitors in the race for AI inference capacity.
A new TRL protocol reduces per-step model synchronization from terabytes to tens of megabytes by shipping only changed parameters across distributed training pipelines.
AI gateway OpenRouter raises $113M Series B from CapitalG, doubling its valuation in 12 months as enterprises increasingly avoid vendor lock-in.
NVIDIA releases diffusion language models at 3B, 8B, and 14B scales that generate and refine tokens in parallel, offering latency improvements for GPU-constrained inference workloads.
Cerebras Systems priced its IPO at $185/share and opened at $385, closing day one at $311 with a $66B market cap.
Hugging Face's engineering blog details how asynchronous continuous batching eliminates CPU-GPU idle gaps that waste nearly a quarter of LLM inference runtime.
Oracle has staked its enterprise future on a $300B compute deal with OpenAI, betting that AI's real profits live in inference — not model training.