Skip to content
This repository was archived by the owner on Jun 10, 2026. It is now read-only.

Home

Jump to bottom
Noah Ziems edited this page Apr 21, 2025 · 6 revisions

API Documentation

This page documents the HTTP request and response formats for the Arbor API.

General Notes

  • Arbor currently supports inference and SFT of language models. Here is the current way endpoints work.
  • Arbor is also working to integrate DPO and GRPO. Those endpoints can be found below the SFT and inference.

Endpoint: Chat Inference

Description

Runs chat inference just like OpenAI api but with local models

URL

POST /v1/chat/completions

Request

Body

{
  "name": "string",
  "description": "string",
  "quantity": "integer"
}

Parameters

Field Type Required Description
model String Yes ID of the local model to use (e.g., gpt-3.5-turbo, gpt-4).
messages Array Yes List of message objects defining the conversation (role: system, user, or assistant).
temperature Float No Controls randomness (0.0 to 2.0). Lower values make output more focused (default: 1.0).
max_tokens Integer No Maximum number of tokens to generate in the response.
top_p Float No Controls diversity via nucleus sampling (0.0 to 1.0). Default: 1.0.
n Integer No Number of completions to generate (default: 1).

Response

Success (200 OK)

{
  "model": "gpt-3.5-turbo",
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant."
    },
    {
      "role": "user",
      "content": "What's the capital of France?"
    }
  ],
  "temperature": 0.7,
  "max_tokens": 100,
  "top_p": 1.0,
  "n": 1
}

Error (400 Bad Request)

TODO

Endpoint: Upload Training File

Description

Upload a file for training

URL

POST /v1/files

Request

Body

with open(file_path, 'rb') as file:
    files = {'file': file}
    response = requests.post(base_path + '/v1/files', files=files)

Parameters

Field Type Required Description
file File Yes A file containing uploaded training data in the proper format

Response

Success (200 OK)

{
  "id": "ef1b1eb5-be05-4783-a412-dc2e36afaba2",
  "object": "file",
  "bytes": 1227,
  "created_at": 1741705972,
  "filename": "training_data_sft.jsonl",
  "purpose": "training"
}

Endpoint: Submit SFT Job

Description

Submit a job for SFT to be done

URL

POST /v1/fine_tuning/jobs

Request

Body

{
  "model": "string",
  "training_file": "string",
  "suffix": "string"
}

Parameters

Field Type Required Description
model String Yes The huggingface id or local path of the model to be fine tuned (e.g., meta-llama/Llama-3.3-70B-Instruct).
training_file String Yes The ID of the previously uploaded training file
suffix String No A suffix that will be added to your fine-tuned model name

Response

Success (200 OK)

{
  "object": "fine_tuning.job",
  "id": "ftjob-a890525c-281e-4c64-b384-666ea0345722",
  "fine_tuned_model": null,
}

Error (400 Bad Request)

TODO

Endpoint: Get Fine- tuning Job

Description

Get a status update about the fine-tuning job. If the job has finished, fine_tuned_model is the path location of the finished model, which can be loaded into huggingface or used in inference.

URL

GET /v1/fine_tuning/jobs/{fine_tuning_job_id}

Response

Success (200 OK)

{
  "id": "bdcc5b2f-b46c-41be-a00c-3671793700f6",
  "status": "succeeded",
  "details": "",
  "fine_tuned_model": "/home/noah/Code/OSS/arbor/storage/models/ft:smollm2-135m-instruct:inhvr6:20250311_111457"
}

Error (400 Bad Request)

TODO

Endpoint: Update a model with GRPO

Description

Update a model using GRPO with a given batch

URL

POST /v1/fine_tuning/grpo

Request

Body

{
  "model": "string",
  "update_inference_model": "bool",
  "batch": [
    {
      "input": {
        "messages": [
          {
            "role": "user",
            "content": "What is the weather in San Francisco?"
          }
        ]
      },
      "completions": [
        {
          "role": "assistant",
          "content": "The weather in San Francisco is 70 degrees Fahrenheit.",
          "reward": 3
        },
        {
          "role": "assistant",
          "content": "The weather in San Francisco is 21 degrees Celsius.",
          "reward": 1
        }
      ]
    },
    ...
  ]
}

Parameters

Field Type Required Description
model String Yes The huggingface id or local path of the model to be fine tuned (e.g., meta-llama/Llama-3.3-70B-Instruct).
update_inference_model Bool No After taking a training step, update the model used for the inference endpoint
suffix String No A suffix that will be added to your fine-tuned model name

Response

Success (200 OK)

{
  "object": "fine_tuning.grpo",
  "id": "ftgrpo-a890525c-281e-4c64-b384-666ea0345722",
  "fine_tuned_model": "/home/noah/Code/OSS/arbor/storage/models/ft:smollm2-135m-instruct:inhvr6:20250311_111457",
}

Error (400 Bad Request)

TODO

GRPO Usage Example

GRPO is a work in progress and you are likely to run into issues as you use it

You first have to run the server:

git clone https://github.com/Ziems/arbor
cd arbor
uv pip install -e .
uv run arbor serve

Here is an example of GRPO being used to train Qwen2-0.5-Instruct to generate very short TLDR responses: GRPO Test

import requests
from openai import OpenAI
from datasets import load_dataset
client = OpenAI(
    base_url="http://127.0.0.1:8000/v1",  # Using Arbor server
    api_key="not-needed",  # If you're using a local server, you dont need an API key
)

def initialize_grpo(model, url='http://127.0.0.1:8000/v1/fine_tuning/grpo/initialize'):
    headers = {'Content-Type': 'application/json'}
    data = {
        'model': model,
        'suffix': 'test',
        'num_generations': 2
    }
    response = requests.post(url, headers=headers, json=data)
    return response


#"HuggingFaceTB/SmolLM2-135M-Instruct"
#"Qwen/Qwen2-0.5B-Instruct"
def run_grpo_step(model_name, batch, url='http://127.0.0.1:8000/v1/fine_tuning/grpo/step'):
    headers = {'Content-Type': 'application/json'}
    data = {
        'model': model_name,
        'update_inference_model': True,
        "batch": batch
    }
    response = requests.post(url, headers=headers, json=data)
    return response

def terminate_grpo(url='http://127.0.0.1:8000/v1/fine_tuning/grpo/terminate'):
    headers = {'Content-Type': 'application/json'}
    data = {
        'status': 'success'
    }
    response = requests.post(url, headers=headers, json=data)
    return response


def reward_func(prompts, completions):

    return [-abs(20 - len(completion)) if completion is not None else -300 for completion in completions]



dataset = load_dataset("trl-lib/tldr", split="train")
current_model = "Qwen/Qwen2-0.5B-Instruct"
initialize_response = initialize_grpo(model=current_model)

for i in range(len(dataset)):
    inputs = dataset[i]
    input_messages = [{"role": "user", "content": inputs["prompt"]}]
    response = client.chat.completions.create(
        model=current_model,
        messages=input_messages,
        temperature=0.7,
        n=2
    )
    completions = [{'content': choice.message.content, 'role': choice.message.role} for choice in response.choices]
    rewards = reward_func(inputs["prompt"], [c["content"] for c in completions])
    print(rewards)

    batch = []
    for completion, reward in zip(completions, rewards):
        batch.append({
            "messages": input_messages,
            "completion": completion,
            "reward": reward
        })
    step_response = run_grpo_step(model_name=current_model, batch=batch)
    current_model = step_response.json()["current_model"]

terminate_response = terminate_grpo()

To run this, just do:

uv run gpro_testing.py

Clone this wiki locally