Run your agent on local models. Free, private, and offline.
Magnitude is an open source inference server that runs the best local models for your hardware, plugged into the agent you already use. It profiles your machine, recommends the models that fit, then downloads, tunes, and runs them. Works with Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline, or use the built-in harness.
⭐ Help us reach more developers and grow the Magnitude community. Star this repo!
Send this to your agent to walk through models and setup:
Set up local models for me with the Magnitude CLI. Install it with `npm i -g @magnitudedev/cli` (or my package manager), then run `magnitude docs onboarding` and follow the instructions.
Your agent will profile your hardware, walk you through the best local models for it, download the ones you pick, and switch itself over to them.
Magnitude supports macOS and Linux. Windows is supported through WSL.
Want to browse the models directly?
npm i -g @magnitudedev/cli
magnitude setupThe interactive setup lets you browse the recommended models and choose one yourself.
- Free to run: no token costs, API keys, or rate limits
- Fully private and offline: models, prompts, and files stay on your machine
- Agent-first setup: one prompt and your agent walks you through the rest
- Knows your hardware: profiles your chip, memory, and bandwidth
- Recommends what fits: the best models for your machine, with estimated tok/s
- Tuned end to end: speculative decoding, concurrency, all set for your machine
- Models on demand: loaded on request, unloaded when idle or memory fills
- Open source: Apache 2.0, yours to modify
An open source inference server that runs the best local models for your hardware, plugged into the agent you already use. It profiles your machine, recommends the models that fit, then downloads, tunes, and runs them.
There's no fixed minimum. Magnitude profiles your hardware and recommends the best models for your machine. More memory lets you run larger models.
Your agent would be guessing. It doesn't know your hardware, which quant fits, or how fast it'll run. Magnitude gives it a catalog with recommendations computed for your machine, an onboarding flow that writes your harness config, and inference built for agent workloads. Models load just in time and unload when idle or memory gets tight.
Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline. During setup, your agent connects your harness to the model you pick. Or use Magnitude's built-in harness.
No. It runs in the background, loads models when your agent needs them, and unloads them when idle or memory gets tight. Your agent can install or switch models through the Magnitude CLI anytime.
No. Prompts, files, and models stay on your machine.
Yes. Once Magnitude and a model are downloaded, no internet connection needed.
Yes. You can download compatible GGUF models from Hugging Face and use them in Magnitude.
Magnitude is licensed under the Apache License 2.0.