Google Cloud has introduced co-operative time-slicing for reinforcement learning workloads through the llm-d project. The system is intended to increase accelerator use in large language model post-training.
At the centre of the launch is software that lets separate reinforcement learning jobs take turns on the same hardware when one pauses between sampling and training phases. Early benchmarks lifted aggregate accelerator duty cycles from about 40% to 70% without affecting model convergence or accuracy.
The release adds three components: a Snapshot Agent, an Accelerator Orchestrator, and Python client libraries. Together, they manage checkpointing and restoring device state so another job can use the same accelerator during otherwise idle periods.
Reinforcement learning post-training for large language models typically alternates between generating rollouts and running gradient updates. In many distributed setups, those steps happen one after the other, leaving trainer or sampler resources waiting for the other phase to finish.
Even asynchronous approaches do not eliminate the problem because training jobs still wait for fresh rollout data. As a result, accelerators can sit unused for 40% to 60% of a workload's lifecycle while still being reserved.
How it works
The new approach treats each phase of an RL workload as a schedulable unit. When one job reaches a pause point, the system checkpoints its CUDA context and memory state from accelerator memory into host DRAM, then restores another job's saved state onto the same hardware.
Only one job occupies the accelerator at a time, reducing the risk of memory conflicts while avoiding a full container restart or model reload. The paused workload remains resident in host memory until it regains access.
The architecture has three layers. At the workload layer, users wrap accelerator-intensive sections of code with acquire and yield calls through a client library. At the cluster layer, an orchestrator manages a lock queue for groups of jobs sharing the same hardware. At the node layer, a snapshot agent performs the hardware-level swap.
The system is designed to work with existing machine learning frameworks without changes to the underlying training or inference software. Google Cloud cited PyTorch FSDP and vLLM as examples of frameworks that can run unchanged around the time-slicing process.
Infrastructure focus
The launch reflects a broader push among AI infrastructure providers to extract more work from scarce accelerator fleets rather than relying only on adding more chips. As reinforcement learning becomes more central to tuning reasoning and coding models, labs face rising costs tied not just to GPU access but to how efficiently those resources are scheduled.
Google Cloud presented the release as part of the wider llm-d stack for inference, agent-based work, and reinforcement learning. That stack also includes llm-d-router for rollout generation throughput, an Agent Sandbox for tool use and isolated code execution, and work on a Weight Propagation Interface intended to improve weight transfer, observability, and reliability.
In a client-side example, a researcher can mark training and generation functions with decorators tied to the orchestrator while leaving the broader RL loop unchanged. The goal is to let researchers focus on modelling logic instead of building custom accelerator scheduling systems.
Open release
The time-slicing tools have been released with integration guides for RL workloads. Google Cloud also outlined work under way to reduce switching overhead, including faster checkpoint and restore methods and more selective memory snapshots, such as swapping specific model adapter regions instead of an entire device state.
It also plans to add automated scheduling to identify compatible workload patterns and place jobs dynamically. Another target is support beyond GPUs, including TPUs and other accelerator architectures.
The node-level snapshot agent is also designed to run outside Kubernetes in bare-metal and Slurm environments, widening its potential use in research and high-performance computing settings. The first backend implementation uses cuda-checkpoint.
The main commercial argument for the approach is that unused time on expensive accelerators compounds quickly in large RL clusters. Google Cloud said those gaps can amount to millions in wasted capital each year when training and sampling resources remain reserved but idle.