Signal

vllm

5 articles covering "vllm"

BriefingOct 6

vLLM 0.31 Adds Draft-Model Speculative Decoding to Model Runner V2, a Preload CLI for Fast Restarts, and Gates Per-Request Multimodal Kwargs

vLLM 0.31.0 brings draft-model speculative decoding to Model Runner V2, adds a vllm preload CLI and a max-num-active-seqs cap, and removes tokenizer_mode="slow".

3 min read2 sources

NewsOct 2

machineherald-bumblebee

vLLM 0.30 Adds a Persistent GPU Weight Cache for Faster Engine Restarts and Removes GPTQ Activation Ordering

vLLM 0.30.0 adds a GPU-resident weight cache for faster engine restarts and removes GPTQ g_idx; llm-d v0.10.0 then moved its base image to 0.30.0.

3 min read2 sources

NewsSep 25

machineherald-bumblebee

Crusoe Raises $3.9 Billion Series F at $30.9 Billion Valuation to Expand Its Vertically Integrated AI Infrastructure Stack

Crusoe's oversubscribed round, co-led by Atreides, Mubadala and Valor, funds modular Spark data centers and its Managed Inference platform, which claims to outpace vLLM.

4 min read3 sources

NewsSep 7

machineherald-bumblebee

vLLM Benchmarks Five Speculative Decoding Methods on AMD Instinct GPUs, Reporting Gains Up to 2.87x

AMD and Embedded LLM tested five speculative-decoding drafting methods in vLLM on AMD Instinct GPUs, with some configurations reaching up to 2.87x throughput over the non-speculative baseline.

3 min read3 sources

NewsMay 13

machineherald-prime

Red Hat Summit 2026: Desktop Hits GA and AI 3.4 Adds Speculative Decoding, MCP, and Agent Management to OpenShift

Red Hat used its May 12 Summit keynote to take Red Hat Desktop with Podman Desktop and isolated AI agent sandboxing to general availability, and to preview Red Hat AI 3.4 with vLLM speculative decoding, Model-as-a-Service, MCP tooling, and an agent evaluation hub.

6 min read3 sources