Infrastructure
47 articles RSS
PyTorch 2.14 Adds NVGEMM Kernels for Inductor, a Prototype torch.switch Operator and @dynamic_spec; 2.14.1 Fixes Silent Errors on MPS and CUDA 13.2
PyTorch 2.14.0 (September 2, 2026) brings CuTeDSL-based NVGEMM to Inductor, a prototype switch op and declarative dynamic shapes; 2.14.1 followed on September 30 with correctness fixes.
SGLang v0.5.21 Lets Prefill-Decode Instances Switch Roles Without Restart, Moves Its Prefix Cache to a Rust Core and Lists Ten Upgrade Notes
SGLang v0.5.21 lets PD instances switch between prefill and decode without a restart, makes a Rust prefix-cache core the default, and removes deprecated endpoints.
TensorRT-LLM v1.4.0rc0 Labels Two API Changes BREAKING, Including Removal of the KVCacheManagerV2 Python Backend, and Lists Five Known Issues
NVIDIA's TensorRT-LLM v1.4.0rc0, published October 14, 2026, labels two API changes (BREAKING) in its release notes and lists five known issues.
Ollama 0.40.2 Upgrades Downloaded Models in the Background and Keeps Backups, 0.40.3 Pauses It, and 0.40.4 Adds Decision Models to ollama run
Ollama's v0.40.2 release notes say older models are upgraded in the background with backups kept on disk; v0.40.3 pauses the upgrade, and v0.40.4 adds decision models to the CLI.
OpenAI Adds Ultrafast Mode for GPT-6.1 Sol in the API, Listing It at $12 Input and $60 Output per Million Tokens, Six Times Standard Rates
OpenAI's changelog says GPT-6.1 Sol gained an Ultrafast service tier on Oct 8; its pricing page lists $12 input and $60 output per million tokens, six times Standard.
TRL 1.15.0 Turns On a Triton Fused LM Head by Default, Deprecates use_liger_kernel in Four Trainers and Drops Python 3.10
TRL v1.15.0 scores SFT, DPO, KTO, GRPO and RLOO tokens with a Triton fused LM head by default, deprecates use_liger_kernel in four trainers and requires Python 3.11 or later.
OpenAI Simplifies API Usage Tiers From Five to Three, With Build, Launch and Grow Starting at $5, $100 and $500 in Credit Purchases
OpenAI's API changelog says it cut usage tiers from five to three on October 6. Its rate-limits guide lists monthly usage limits of $500, $5,000 and $200,000.
huggingface_hub v2.2.0 Makes hf jobs stats Print One Snapshot by Default and Fixes a Race That Could Let Concurrent Downloads Skip hf_xet
The October 8 release changes hf jobs stats to exit after one snapshot unless -f is passed, fixes an hf_xet version-lookup race, and repairs HfFileSystem cache bugs.
MLflow 3.17.0 Adds Per-Resource-Type Permissions and Opt-In Trace Rollups, and Requires a Coordinated Database Upgrade
MLflow 3.17.0, published October 7, adds sub-resource permission tiers and opt-in daily trace analytics rollups, and its notes require stopping all writers for SQL schema upgrades.
Transformers v5.19.0 Adds Google's EmbeddingGemma 2 Multimodal Embedding Model and Deprecates the paged Attention Prefix for Continuous Batching
Hugging Face Transformers v5.19.0 adds EmbeddingGemma 2, a Gemma 4-based embedding model for text, images, audio and video, and flags breaking changes.
Ray 2.59 Graduates Ray Data LLM and Ray Serve LLM to General Availability and Enables Token Authentication by Default for Local Clusters
Ray 2.59.0 declares its LLM batch and serving APIs generally available, upgrades to vLLM 0.27.0, and turns on token authentication for local clusters, with all clusters to follow in 2.61.
vLLM 0.31 Adds Draft-Model Speculative Decoding to Model Runner V2, a Preload CLI for Fast Restarts, and Gates Per-Request Multimodal Kwargs
vLLM 0.31.0 brings draft-model speculative decoding to Model Runner V2, adds a vllm preload CLI and a max-num-active-seqs cap, and removes tokenizer_mode="slow".