inference

Crusoe's oversubscribed round, co-led by Atreides, Mubadala and Valor, funds modular Spark data centers and its Managed Inference platform, which claims to outpace vLLM.

AMD and Embedded LLM tested five speculative-decoding drafting methods in vLLM on AMD Instinct GPUs, with some configurations reaching up to 2.87x throughput over the non-speculative baseline.

NVIDIA's Groq-derived Groq 3 LPX inference accelerator is now shipping, promising ultrafast token generation for agentic coding workloads, with Nebius first to deploy it.

UK inference-chip startup Fractile is in talks to raise about $600M at a $6.5B pre-money valuation, driven by an initial $250M chip deal with Anthropic.

Etched unveiled working A0 silicon for Sohu, its transformer-only inference ASIC, disclosing $800 million raised at a $5 billion valuation and more than $1 billion in signed customer contracts.

AWS added Grok 4.3 to Bedrock on June 15, making xAI a model provider on the platform and debuting a separate Mantle inference path.

OpenAI's first custom silicon, the Broadcom-built Jalapeño inference accelerator, went from design to tape-out in nine months and targets deployment by the end of 2026.

The startup says its 3nm Napier inference chip turns matrix multipliers into adders, with commercial sales of a 72-chip system slated for the second half of 2027.

d-Matrix says its Corsair inference accelerator is in full production, pairing on-chip SRAM with GPUs to cut AI response times by more than 10x.

A new MIT CSAIL inference method has a model inspect its prompt in a Python REPL and recursively call itself over snippets, processing inputs beyond its context window.

London-based Fractile closes a $220M Series B led by Accel, Founders Fund, and Factorial Funds to tape out SRAM-based inference chips promising radically faster and cheaper AI output generation.

Cloudflare detailed the custom infrastructure behind its Workers AI expansion to host models like Kimi K2.5, including a Rust-based inference engine and a lossless weight-compression system.