Google and Meta Open-Source TorchTPU, a PyTorch Backend for TPUs Using a tpu Device, a tpu_dist Backend and Three Eager Modes

Overview

TorchTPU is, in the words of InfoQ, “an open-source project developed by Google in collaboration with Meta that provides a native PyTorch backend for Google TPUs”. InfoQ’s report, written by Sergio De Simone, is dated November 2, 2026. This briefing summarizes that report and the project’s repository.

What Changes for PyTorch Developers

According to InfoQ, Google began enabling PyTorch workloads to run on TPUs in 2020 through PyTorch/XLA, which leveraged JAX, XLA and lazy tensors. InfoQ, relaying Google’s account, says that approach “required intrusive source-code modifications to explicitly define compilable regions, suffered from severe compile-time latency”, among other shortcomings.

TorchTPU takes a different route. InfoQ says migrating from GPUs to TPUs involves replacing the cuda device with tpu and the nccl distributed backend with tpu_dist. Developers can alternatively use the new torch.accelerator.current_accelerator API to identify the available accelerator.

The same report says TorchTPU supports three eager execution modes: strict, debug and DeferAndFuse. DeferAndFuse is described as a new hybrid eager/compiled mode that Google claims “can be 50% or more faster than Strict Eager while reducing peak HBM memory”. That figure is Google’s own claim, as relayed by InfoQ.

Google’s Test and Performance Figures

All figures in this section are Google’s, as reported by InfoQ; the Herald has not reproduced them.

• Model coverage. Google built an integration pipeline to evaluate about 5,000 Hugging Face models across 500+ architectures in both eager and compiled modes on TPU hardware generations starting from v5e.
• Inference. For a 30-billion-parameter Muse Glimmer model implemented in standard PyTorch, inference throughput on a single TPUv7 chip reached 25.7k token/s at 5120 sequence length and 22.4k token/s at 10,240 sequence length.
• Training. On four TPUv7 chips, the same model reached 5.6k token/s and 4.6k token/s at those two sequence lengths, respectively.
• Scale. Google also ran Qwen3 MoE 235B training experiments scaling from 1,024 to 8,192 TPUv7 chips, and reported competitive performance-per-dollar for large-scale MoE inference using vLLM.

Installing and Requirements

The repository README states that TorchTPU’s Python version support policy matches PyTorch’s Release Compatibility Matrix, and that Python 3.11+ is currently supported. It identifies 0.2.0 on PyPI as the latest public release and gives the install command pip install torch_tpu==0.2.0 --extra-index-url https://download.pytorch.org/whl/cpu. The README adds that installing without the extra index URL will install the CUDA version of torch. Wheels can also be built from source with Bazel.

On the framework side, the Herald previously reported on the PyTorch 2.14 release.

What We Don’t Know

• The figures above come from Google; the sources cited here include no independent replication of them.
• InfoQ says TorchTPU “is still at an early stage and therefore in continuous evolution”.
• InfoQ lists the project’s roadmap as work on dynamic shapes, compilation performance, operator coverage, reinforcement learning and quantization. It gives no dates for any of these.