Skip to content

WINDOWS_SCAFFOLD.md

Latest commit

 

History

History
79 lines (56 loc) · 1.99 KB

WINDOWS_SCAFFOLD.md

File metadata and controls

79 lines (56 loc) · 1.99 KB
Raw

Windows scaffold — Goliath

Owned by diffusion-llm-mcp. Step-by-step for RTX 4090 / Windows 11.

Verified on Goliath (2026-06-17): RTX 4090 24 GB, 64 GB RAM, CUDA 13.1 driver, Python 3.13 via py.

Path A: llama.cpp + GGUF (recommended)

1. Dependencies

git --version
cmake --version
nvcc --version

CUDA Toolkit + VS 2022 Build Tools (C++) required if nvcc missing.

2. Diffusion branch

Set-Location C:\Users\sandr\dev
git clone https://github.com/ggml-org/llama.cpp.git llama.cpp-diffusion
Set-Location llama.cpp-diffusion
git fetch origin pull/24423/head:diffusion-support
git checkout diffusion-support

3. Build

cmake -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j
.\build\bin\Release\llama-diffusion-cli.exe --version

Use llama-diffusion-cli only — not llama-cli / llama-server.

4. Weights

New-Item -ItemType Directory -Path C:\Users\sandr\dev\models\diffusiongemma -Force
Set-Location C:\Users\sandr\dev\models\diffusiongemma
py -m pip install huggingface_hub
py -m huggingface_hub.cli download unsloth/diffusiongemma-26B-A4B-it-GGUF diffusiongemma-26B-A4B-it-Q4_K_M.gguf --local-dir .

5. Run

Set-Location C:\Users\sandr\dev\llama.cpp-diffusion
.\build\bin\Release\llama-diffusion-cli.exe `
  -m C:\Users\sandr\dev\models\diffusiongemma\diffusiongemma-26B-A4B-it-Q4_K_M.gguf `
  -ngl 99 -cnv -n 512

Optional: --diffusion-visual to watch canvas denoise.

Layout

C:\Users\sandr\dev\
├── llama.cpp-diffusion\
├── models\diffusiongemma\
└── repos\diffusion-llm-mcp\    ← this repo

Smoke test (Phase 0 exit criteria)

  • CLI --version OK
  • Load without OOM
  • Coherent 512-token block
  • Throughput note vs Qwen 32B AR baseline

Does not work yet

Ollama, LM Studio, NVFP4 on 4090, vLLM on Windows.

Full troubleshooting: mcp-central-docs/projects/diffusiongemma/WINDOWS_SCAFFOLD.md