Skip to main content

What is Magnitude?

An open source inference engine that optimizes each model for your exact hardware. It ships as a desktop app that runs open models and connects them to the agent you already use.

How is it faster than llama.cpp, Ollama, or LM Studio?

They ship one-size-fits-all kernels, because tuning each model for each chip by hand would take forever. Magnitude tunes them automatically. When you download a model, it spends up to a minute on your actual chip trying different ways to split up the math (how many GPU threads team up, how big a chunk each one takes, how far ahead to load data) and keeps the fastest combination that doesn’t change the model’s output. See the full benchmarks against Ollama.

What hardware do I need?

Any Apple Silicon, NVIDIA, AMD, or Intel GPU, or nothing but a CPU. There is no fixed minimum. Smaller machines run smaller models, and more memory lets you run larger ones.

What operating systems does it support?

macOS, Linux, and Windows.

Which models does it support?

See the full list at magnitude.dev/models. We write optimized kernels for the most popular open-weight families, which is how we beat generalist engines.

Which agents work with it?

One click connects Pi, OpenCode, Hermes, OpenClaw, Codex, Claude Code, Oh My Pi, and Cline. Anything else works through the OpenAI-compatible API.

Is it private?

Yes. Prompts, files, and models stay on your machine. No internet needed once a model is downloaded.

Can I use Magnitude from another computer, a container, or WSL?

Yes. Enable Network access for direct connections and use the server address and API key. SSH tunnels and mirrored WSL networking can use the local endpoint without enabling it. Only inference is available from other devices. See Network access and Windows and WSL.

Can I keep models on another drive?

Yes. Change Model storage in Settings, then move the existing models with the command it shows. See Settings.

Why is the first response slow and later ones fast?

Models load on demand and unload when idle, so the first request after a pause includes loading. Later turns reuse cached context, and supported models use speculative decoding. Click Load model in My Models to load ahead of time.