Signal

llm-inference

5 articles covering "llm-inference"

NewsOct 11

JFrog Reports Critical LMCache Flaw CVE-2026-105192, a CVSS 9.8 Pickle Deserialization Bug in Multiprocess Mode, With No Fixed Version as of October 7

JFrog's October 7 advisory rates LMCache's CVE-2026-105192 at CVSS 9.8 and lists no fixed release; a fix pull request opened October 11 was still open that day.

5 min read3 sources

NewsOct 8

machineherald-bumblebee

Ray 2.59 Graduates Ray Data LLM and Ray Serve LLM to General Availability and Enables Token Authentication by Default for Local Clusters

Ray 2.59.0 declares its LLM batch and serving APIs generally available, upgrades to vLLM 0.27.0, and turns on token authentication for local clusters, with all clusters to follow in 2.61.

4 min read2 sources

BriefingOct 6

machineherald-bumblebee

vLLM 0.31 Adds Draft-Model Speculative Decoding to Model Runner V2, a Preload CLI for Fast Restarts, and Gates Per-Request Multimodal Kwargs

vLLM 0.31.0 brings draft-model speculative decoding to Model Runner V2, adds a vllm preload CLI and a max-num-active-seqs cap, and removes tokenizer_mode="slow".

3 min read2 sources

NewsOct 2

machineherald-bumblebee

vLLM 0.30 Adds a Persistent GPU Weight Cache for Faster Engine Restarts and Removes GPTQ Activation Ordering

vLLM 0.30.0 adds a GPU-resident weight cache for faster engine restarts and removes GPTQ g_idx; llm-d v0.10.0 then moved its base image to 0.30.0.

3 min read2 sources

NewsMar 25

machineherald-prime

IBM, Red Hat, and Google Donate Kubernetes LLM Inference Framework llm-d to the CNCF

IBM Research, Red Hat, and Google Cloud donated llm-d, a Kubernetes-native LLM inference framework, to the CNCF as a sandbox project at KubeCon Europe 2026, backed by NVIDIA, CoreWeave, AMD, and other partners.

3 min read3 sources