Add timeslice-verl package and fully-async two-RL-jobs guide

Adds the timeslice-verl integration package and a guide that time-slices two
verl fully_async_policy RL jobs' trainers on one shared 1-GPU node (each job
keeps a dedicated 1-GPU rollout node; 2-pod Ray cluster per job).

The package ships TimesliceFullyAsyncTrainer, a subclass of verl's
FullyAsyncTrainer that overrides the trainer's empty on_* lifecycle hooks
(same template-method convention as verl's v1 trainers). It registers under
the trainer name "timeslice" via verl's fully-async trainer registry — the
package's verl.plugins entry point makes `import verl` load the registering
module — and is selected with the hydra override
async_training.trainer_name=timeslice. No meta_path monkey-patching.
Placement-group pinning uses verl's per-pool extra bundle resources
(ray_pg_extra_resources hydra override) instead of patching
placement_group; the trainer refuses to train unpinned when
TIMESLICE_REQUIRE_PG_PINNING=1. Both verl features ship as format-patch
files in the guide's patches/ dir until they are available upstream
(default install path: the feat/fully-async-lifecycle-hooks fork branch).

New: pkg/integrations/verl/ (TimesliceFullyAsyncTrainer = TimesliceHooksMixin
over FullyAsyncTrainer, + PhaseLocks; 33 tests, pure-python: no verl/ray/
grpc/GPU needed)
New: guides/rl-frameworks/verl/ (README, job manifests for a math-RLVR and a
code-RLVR job, GPU monitor, platform values, verl feature patches)

Author: aishukamal

guides/rl-frameworks/verl/README.md

@@ -0,0 +1,22 @@
+# Time-Slicing Integration Guide for verl Workloads
+
+This guide covers integrating **verl** (Volcano Engine Reinforcement Learning framework) with the **llm-d-rl-time-slicing** platform using the pre-packaged `timeslice-verl` integration (`pkg/integrations/verl/`).
+
+### Motivation: Maximizing GPU Utilization
+
+In verl's **fully-async** recipe (`fully_async_policy`), rollout generation runs continuously while the trainer sits idle whenever it waits for the next batch of samples — in our reference runs that idle fraction was ~55–70% of wall-clock. Cooperative time-slicing backfills those valleys: multiple jobs' trainers share one physical GPU pool, and the platform checkpoints the idle trainer's GPU state to host RAM (`cuda-checkpoint`) at every handoff. Rollout engines keep dedicated GPUs and never stop generating.
+
+### How It Works
+
+The `timeslice-verl` package ships `TimesliceFullyAsyncTrainer`, a subclass of verl's `fully_async_policy` trainer that overrides the trainer's empty `on_*` lifecycle hooks to acquire/release the orchestrator lock at the trainer's natural wait points. It registers under the trainer name `timeslice` (verl's trainer registry; the package's `verl.plugins` entry point makes `import verl` load the registering module) and is selected with a single hydra override: `async_training.trainer_name=timeslice`. No monkey-patching and no verl source edits in your job — it requires a verl build with the fully-async lifecycle hooks + trainer registry and per-pool placement-group bundle resources (currently the `feat/fully-async-lifecycle-hooks` fork branch, until the commits land upstream).
+
+### Supported Modes
+
+| verl mode | Status | Notes |
+|---|---|---|
+| **Fully-async** (`fully_async_policy`) | **Supported** | Via the `TimesliceFullyAsyncTrainer` subclass + fork lifecycle hooks. See the runnable example below. |
+| v1 sync modes (e.g. `main_ppo` PPO/GRPO) | Not shipped in this PR | The v1 trainers expose the same `on_*` hook convention; integration is future work. |
+
+For a runnable example, see:
+
+* **[Fully-Async Example](examples/fully-async/README.md)** — two fully-async RL jobs (math RLVR + code RLVR) sharing one trainer GPU, each with a dedicated rollout GPU

guides/rl-frameworks/verl/examples/fully-async/README.md

@@ -0,0 +1,29 @@
+# Helm override values for the llm-d-rl-time-slicing umbrella chart (deploy/).
+# Applied on top of deploy/values-gke.yaml:
+#   helm install timeslice ./deploy -n timeslice-system --create-namespace \
+#     -f ./deploy/values-gke.yaml -f values-timeslice.yaml
+#
+# NOTE ON IMAGES: this guide needs platform images that include the IDLE
+# cold-start deadlock fix (jobs that deploy their pods before their first
+# Acquire register as IDLE; older orchestrators deadlock on that state).
+# Use the official images from a release that contains it, or override the
+# tags below with your own build.
+#
+# TODO: pin the tags below to the first tagged release that includes the
+# cold-start Acquire fix (#152); "latest" is used until then.
+
+timesliceorchestrator:
+  image:
+    repository: ghcr.io/llm-d-incubation/llm-d-rl-time-slicing/timesliceorchestrator
+    tag: latest
+    pullPolicy: Always
+
+snapshot-agent:
+  image:
+    repository: ghcr.io/llm-d-incubation/llm-d-rl-time-slicing/snapshot-agent
+    tag: latest
+    pullPolicy: Always
+
+# NOTE: no per-GPU device filter is needed in the 1-GPU-per-node topology —
+# each node has a single GPU and only the (labeled) head pods on the trainer
+# node are ever snapshot targets. This file is images-only.

guides/rl-frameworks/verl/examples/fully-async/job-a-math.yaml

@@ -0,0 +1,23 @@
+[build-system]
+requires = ["setuptools>=64"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "timeslice-verl"
+version = "0.1.0"
+description = "GPU time-slicing integration for verl fully-async RL training"
+requires-python = ">=3.10"
+dependencies = []
+
+# `import verl` auto-loads every module in the verl.plugins entry-point group;
+# importing timeslice_verl.trainer registers TimesliceFullyAsyncTrainer under
+# the trainer name "timeslice" (select it with the hydra override
+# async_training.trainer_name=timeslice; requires verl with fully-async
+# lifecycle hooks + trainer registry). The hooks are env-gated: inert unless
+# TIMESLICE_FULLY_ASYNC=1, so the same image works with and without the
+# time-slicing platform.
+[project.entry-points."verl.plugins"]
+timeslice = "timeslice_verl.trainer"
+
+[tool.setuptools]
+packages = ["timeslice_verl"]