Why Magnitude?
- Knows your hardware: Profiles your chip, memory, and bandwidth.
- Recommends what fits: Ranks exact model, quantization, and context combinations for your machine, with estimated tok/s.
- No separate server: Includes its own inference engine, optimized for agent workloads.
- Manages models for you: Loads models when your agent needs them and unloads them when idle or when memory is needed.
- Tuned end to end: Configures acceleration, speculative decoding, context, and concurrency for your machine.
- Set up in one command: Pick a recommended model and Magnitude connects and launches your harness.
- Fully private and offline: Models, prompts, and files stay on your machine.
- Free to run: No token costs, API keys, or rate limits.
- Open source: Apache 2.0 licensed and yours to modify.