[docs] Add slime quickstart guide by ChiragSW · Pull Request #221 · keras-team/kinetic

ChiragSW · 2026-05-05T09:17:44Z

The SLIME quickstart guide has been added. It has been ensured that steps work for kinetic.

gemini-code-assist

Code Review

This pull request introduces a new guide for LLM post-training using the SLIME framework on Kinetic, covering environment setup, Docker image building, and job submission. The review feedback suggests improving the efficiency of checkpoint uploads by using parallel transfers and ensuring script robustness in non-interactive environments by using full command names instead of shell aliases.

gemini-code-assist · 2026-05-05T09:18:49Z

+def _upload_directory_to_gcs(local_dir: str, gcs_dir: str) -> None:
+    from google.cloud import storage
+
+    if not gcs_dir.startswith("gs://"):
+        raise ValueError(f"Expected a gs:// output path, got {gcs_dir!r}")
+
+    bucket_name, _, prefix = gcs_dir[5:].partition("/")
+    client = storage.Client()
+    bucket = client.bucket(bucket_name)
+    local_root = Path(local_dir)
+
+    for path in local_root.rglob("*"):
+        if path.is_file():
+            rel = path.relative_to(local_root).as_posix()
+            blob = bucket.blob(f"{prefix.rstrip('/')}/{rel}")
+            blob.upload_from_filename(str(path))


The current implementation of _upload_directory_to_gcs is inefficient for LLM checkpoints as it uploads files sequentially. Additionally, it has a bug where it creates blobs with a leading slash if the GCS path is a root bucket (e.g., gs://my-bucket), and it relies on global imports which can be brittle during remote serialization.

I recommend using google.cloud.storage.transfer_manager for parallel uploads and making the function self-contained with its imports.

def _upload_directory_to_gcs(local_dir: str, gcs_dir: str) -> None: from pathlib import Path from google.cloud import storage from google.cloud.storage import transfer_manager if not gcs_dir.startswith("gs://"): raise ValueError(f"Expected a gs:// output path, got {gcs_dir!r}") bucket_name, _, prefix = gcs_dir[5:].partition("/") prefix = prefix.strip("/") client = storage.Client() bucket = client.bucket(bucket_name) local_root = Path(local_dir) # Collect all file paths relative to the local root files = [ str(p.relative_to(local_root)) for p in local_root.rglob("*") if p.is_file() ] transfer_manager.upload_many_from_filenames( bucket, files, source_directory=local_dir, blob_name_prefix=f"{prefix}/" if prefix else "", worker_type=transfer_manager.THREAD, )

You should use the data API for this if possible:
https://kinetic.readthedocs.io/en/latest/guides/data.html

gemini-code-assist · 2026-05-05T09:18:49Z

+hf download zai-org/GLM-Z1-9B-0414 \
+  --local-dir /root/GLM-Z1-9B-0414
+hf download --repo-type dataset zhuzilin/dapo-math-17k \
+  --local-dir /root/dapo-math-17k
+hf download --repo-type dataset zhuzilin/aime-2024 \
+  --local-dir /root/aime-2024


The command hf is typically an alias for huggingface-cli. In a non-interactive shell environment like a Kinetic pod, this alias may not be available. It is safer to use the full command name huggingface-cli to ensure the script runs correctly.

Suggested change

hf download zai-org/GLM-Z1-9B-0414 \

--local-dir /root/GLM-Z1-9B-0414

hf download --repo-type dataset zhuzilin/dapo-math-17k \

--local-dir /root/dapo-math-17k

hf download --repo-type dataset zhuzilin/aime-2024 \

--local-dir /root/aime-2024

huggingface-cli download zai-org/GLM-Z1-9B-0414 \

--local-dir /root/GLM-Z1-9B-0414

huggingface-cli download --repo-type dataset zhuzilin/dapo-math-17k \

--local-dir /root/dapo-math-17k

huggingface-cli download --repo-type dataset zhuzilin/aime-2024 \

--local-dir /root/aime-2024

codecov-commenter · 2026-05-05T09:51:06Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@983eb26). Learn more about missing BASE report.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #221   +/-   ##
=======================================
  Coverage        ?   90.52%           
=======================================
  Files           ?       67           
  Lines           ?    10153           
  Branches        ?        0           
=======================================
  Hits            ?     9191           
  Misses          ?      962           
  Partials        ?        0

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

jeffcarp

Thanks so much! Left some comments

jeffcarp · 2026-05-06T18:18:48Z

   guides/distributed_training
   advanced/batched_jobs
   guides/llm_finetuning
+   guides/llm_post_training


The doc structure was recombobulated in #210 - can you move this to docs/examples?

ChiragSW · 2026-05-06T18:42:28Z

Done! @jeffcarp

JyotinderSingh · 2026-05-08T05:14:27Z

+def _upload_directory_to_gcs(local_dir: str, gcs_dir: str) -> None:
+    from pathlib import Path
+    from google.cloud import storage
+    from google.cloud.storage import transfer_manager
+
+    if not gcs_dir.startswith("gs://"):
+        raise ValueError(f"Expected a gs:// output path, got {gcs_dir!r}")
+
+    bucket_name, _, prefix = gcs_dir[5:].partition("/")
+    prefix = prefix.strip("/")
+
+    client = storage.Client()
+    bucket = client.bucket(bucket_name)
+    local_root = Path(local_dir)
+
+    # Collect all file paths relative to the local root
+    files = [
+        str(p.relative_to(local_root))
+        for p in local_root.rglob("*") if p.is_file()
+    ]
+
+    transfer_manager.upload_many_from_filenames(
+        bucket,
+        files,
+        source_directory=local_dir,
+        blob_name_prefix=f"{prefix}/" if prefix else "",
+        worker_type=transfer_manager.THREAD,
+    )


Can we use the Data() API here? https://kinetic.readthedocs.io/en/latest/guides/data.html

Let's remove this function in that case and use the Data API instead.

JyotinderSingh · 2026-05-11T08:29:16Z

Please follow the repository conventions when writing example guides. The script should be present as a python file under the examples directory at the root of the repo and that file can directly be referenced here in the documentation using the following directive:

```{literalinclude} ../../examples/llm_post_training.py :language: python```

… helper function and used Data(...)

ChiragSW · 2026-05-13T10:31:31Z

@jeffcarp @JyotinderSingh the changes are done, please review

ChiragSW · 2026-05-13T16:53:56Z

@jeffcarp the renaming is done.

ChiragSW · 2026-06-21T13:00:51Z

@jeffcarp @JyotinderSingh any changes required?

ChiragSW requested review from JyotinderSingh, divyashreepathihalli and jeffcarp as code owners May 5, 2026 09:17

gemini-code-assist Bot reviewed May 5, 2026

View reviewed changes

jeffcarp requested changes May 6, 2026

View reviewed changes

ChiragSW added 2 commits May 7, 2026 00:10

[docs] Add slime quickstart guide

1a01f92

some minor changes

98dcb12

ChiragSW force-pushed the issue#217 branch from 782ba2f to 98dcb12 Compare May 6, 2026 18:41

ChiragSW requested a review from jeffcarp May 6, 2026 18:42

moved the file and updated index

016ee42

JyotinderSingh requested changes May 8, 2026

View reviewed changes

used Data(...) API

2353606

ChiragSW requested a review from JyotinderSingh May 8, 2026 15:40

JyotinderSingh reviewed May 11, 2026

View reviewed changes

moved the runable script to examples/llm_post_training.py and removed…

6eff7dd

… helper function and used Data(...)

ChiragSW requested a review from JyotinderSingh May 11, 2026 09:24

jeffcarp requested changes May 13, 2026

View reviewed changes

Comment thread docs/index.rst Outdated

changed the names to be slime specific

7cb4452

ChiragSW requested a review from jeffcarp June 21, 2026 13:00

Uh oh!

Conversation

ChiragSW commented May 5, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist Bot May 5, 2026

Choose a reason for hiding this comment

Uh oh!

jeffcarp May 5, 2026

Choose a reason for hiding this comment

Uh oh!

gemini-code-assist Bot May 5, 2026

Choose a reason for hiding this comment

Uh oh!

codecov-commenter commented May 5, 2026

Codecov Report

Uh oh!

jeffcarp left a comment

Choose a reason for hiding this comment

Uh oh!

jeffcarp May 6, 2026

Choose a reason for hiding this comment

Uh oh!

ChiragSW commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

JyotinderSingh May 8, 2026

Choose a reason for hiding this comment

Uh oh!

ChiragSW May 8, 2026

Choose a reason for hiding this comment

Uh oh!

JyotinderSingh May 11, 2026

Choose a reason for hiding this comment

Uh oh!

JyotinderSingh May 11, 2026

Choose a reason for hiding this comment

Uh oh!

ChiragSW commented May 13, 2026

Uh oh!

Uh oh!

ChiragSW commented May 13, 2026

Uh oh!

ChiragSW commented Jun 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ChiragSW commented May 6, 2026 •

edited

Loading