Skip to content

README.md

Latest commit

 

History

History
82 lines (56 loc) · 2.72 KB

README.md

File metadata and controls

82 lines (56 loc) · 2.72 KB
Raw

Fine-tune and Serve CodeLlama-syz

We have repleased a toy version of CodeLlama-syz at Huggingface, named CodeLlama-syz-toy. And we plan to build a comprehensive and high-quality dataset of Syz-program generation in Q&A format and release the formal version of CodeLlama-syz in the future.

1 Dataset Construction

We collect and sample the effective programs and their prompts generated by SyzGPT-generator during fuzzing. And use the following script to build our fine-tuning dataset. (Note: it is for reference only so far)

python build_dataset_from_fuzzing.py -h

2 Instruct Fine-Tune

2.1 Environments

  • Use Python 3.10 or newer.
  • Install the PyTorch 2.6.0 wheel that matches the target CUDA runtime (cu118, cu124, or cu126) from the official PyTorch wheel index.
  • Install the dependencies:
pip install -r requirements.txt

The fine-tuning script accepts local paths and Hugging Face model IDs, but it only loads trusted model implementations and safetensors weights. Convert legacy pickle-based .bin/.pth weights before using them.

  • Configure accelerator to use specific GPUs: Please refer to Internet for guidance.
  • Configure wandb according to your need: Please refer to Internet for guidance.
  • Download base LLM CodeLlama-7b-Instruct-hf.

2.2 Train

We have prepared a small dataset (8k train and 3k eval) at ./dataset.

You can also build your own dataset by transforming the contextually effective Syz-programs and the corresponding prompts in format of the shown jsonl.

Then we can luanch the traning by python or launch (if accelerator is configured) .

python/accelerate launch instruct_tuning.py \
    --model_path /path/to/CodeLlama \
    --max_step 16000 \
    --batch_size 1 \
    --seed N \
    --output_dir ./output/CodeLlama-7b-Instruct-syz-2epochs \
    # --enable_wandb
    # ... more params please refer to -h or src code

3 Serve Model

We recommend to use Ollama or FastChat to deploy the tuned model.

Take FastChat as an example:

  1. Install the dependencies according to Install FastChat.
  2. Launch the controller
python3 -m fastchat.serve.controller
  1. Load model
# recommend to use vllm optimization
CUDA_VISIBLE_DEVICES=X python3 -m fastchat.serve.vllm_worker --gpu_memory_utilization 0.6 --model-path output/CodeLlama-7b-Instruct-syz-2epochs
  1. Launch API
# support openai API
python3 -m fastchat.serve.openai_api_server --host 0.0.0.0 --port 38950

Make sure the machine serving this model is accessible to SyzGPT-fuzzer.