Skip to content

MiniMax-M2.5.md

Latest commit

 

History

History
96 lines (75 loc) · 3.84 KB

MiniMax-M2.5.md

File metadata and controls

96 lines (75 loc) · 3.84 KB
Raw

MiniMax-M2.5 with ATOM vLLM Plugin Backend

This recipe shows how to run MiniMaxAI/MiniMax-M2.5 (HF architectures[0]: MiniMaxM2ForCausalLM, MoE + FP8 weights) with the ATOM vLLM plugin backend. For background on the plugin backend, see ATOM vLLM Plugin Backend.

The checkpoint uses custom modeling code; keep --trust-remote-code on the server command line.

Step 1: Pull the OOT Docker

docker pull rocm/atom-dev:vllm-latest

Step 2: Launch vLLM Server

The ATOM vLLM plugin backend keeps the standard vLLM CLI, server APIs, and general usage flow compatible with upstream vLLM. For general server options and API usage, refer to the official vLLM documentation.

The following matches the vLLM-ATOM benchmark entries in .github/benchmark/oot_benchmark_models.json. On multi-GPU hosts, the benchmark covers TP2, TP4, and TP8.

export AITER_QUICK_REDUCE_QUANTIZATION=INT4
export ATOM_ENABLE_QK_NORM_ROPE_CACHE_QUANT_FUSION=1
export ATOM_USE_GLUON_PA_DECODE=1
# cap the single-stage allreduce size (KB) to favor the faster path
export AITER_AR_1STAGE_MAX_KB=2048

vllm serve MiniMaxAI/MiniMax-M2.5 \
    --host localhost \
    --port 8000 \
    --async-scheduling \
    --load-format fastsafetensors \
    --tensor-parallel-size 2 \
    --trust-remote-code \
    --compilation-config '{"cudagraph_mode": "FULL_AND_PIECEWISE"}' \
    --kv-cache-dtype fp8 \
    --max-num-batched-tokens 16384 \
    --max-model-len 16384 \
    --gpu-memory-utilization 0.9 \
    --no-enable-prefix-caching

Caveat: the upstream config.json may advertise MTP-related fields; the current ATOM MiniMaxM2ForCausalLM path targets the main transformer. If you hit load or shape errors around MTP modules, compare with native ATOM server behavior and upstream vLLM release notes.

Step 3: Performance Benchmark

ISL=1000
OSL=100
CONC=4

vllm bench serve \
    --backend vllm \
    --base-url http://127.0.0.1:8000 \
    --endpoint /v1/completions \
    --model MiniMaxAI/MiniMax-M2.5 \
    --dataset-name random \
    --random-input-len "${ISL}" \
    --random-output-len "${OSL}" \
    --random-range-ratio 0.0 \
    --max-concurrency "${CONC}" \
    --num-prompts "$(( CONC * 8 ))" \
    --trust_remote_code \
    --num-warmups "${CONC}" \
    --request-rate inf \
    --ignore-eos \
    --disable-tqdm \
    --save-result \
    --percentile-metrics ttft,tpot,itl,e2el

Step 4: Accuracy Validation

Nightly OOT accuracy uses gsm8k with 3-shot in .github/scripts/atom_oot_test.sh (same as other full-validation models). For a local check:

lm_eval --model local-completions \
        --model_args model=MiniMaxAI/MiniMax-M2.5,base_url=http://localhost:8000/v1/completions,num_concurrent=64,max_retries=3,tokenized_requests=False,trust_remote_code=True \
        --tasks gsm8k \
        --num_fewshot 3 \
        --output_path ./lm_eval_minimax_m25_gsm8k

Reference metric (tracking baseline for this model family; replace with your run output and keep the raw JSON path next to the table):

  • Internal tracking: accuracy_baseline 0.9401 for MiniMaxAI/MiniMax-M2.5 in .github/benchmark/models_accuracy.json (see _baseline_note there for HF card context).
  • OOT gate: accuracy_test_threshold 0.92 on exact_match,flexible-extract (see atom-vllm-oot-test.yaml nightly matrix).

Example table shape after lm_eval (fill Value / Stderr from your console or ${output_path} JSON):

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     3|exact_match|↑  |0.9287|±  |0.0071|
|     |       |strict-match    |     3|exact_match|↑  |0.9272|±  |0.0072|

Raw results JSON: <path-to-lm_eval-output-*.json> (from --output_path above).