Skip to main content
Magnitude assesses your machine and recommends models that fit. Start with those recommendations rather than choosing a model by parameter count alone.

​
Supported configurations

See macOS, Windows, and Linux for OS requirements and installers.

​
Apple Silicon

Apple Silicon shares memory between the CPU and GPU. That gives the GPU access to a larger memory pool than a typical laptop’s dedicated graphics memory, but macOS and other apps use the same pool. For example, a Mac with 32 GB of unified memory does not have all 32 GB available for a model. Use Discover’s fit assessment and close memory-heavy apps if loading fails. Metal support is supplied by macOS; you do not need a separate GPU toolkit.

​
NVIDIA GPUs

On Linux and Windows, Magnitude can use its CUDA backend with a compatible NVIDIA GPU and driver. The current CUDA builds target Ampere-class and newer GPUs; older CUDA-capable cards are not automatically supported. Install your GPU’s supported NVIDIA driver. You can check that the driver sees it with:
Magnitude acquires compatible runtime components; you do not need the CUDA development toolkit. If acceleration is unavailable, check the hardware and compatibility information in the app before downloading a large model.

​
AMD and other Vulkan GPUs

Linux and Windows GPU support uses Vulkan for compatible devices. Magnitude does not currently ship a ROCm backend. Use your operating system’s supported graphics driver and Vulkan runtime. The backend requires Vulkan 1.1 or later, but the version alone does not guarantee that a particular GPU supports every model. Check the model’s compatibility in Catalog.

​
CPU-only machines

Magnitude can run models without a supported GPU, including on Intel Macs and Windows machines without a compatible GPU. Smaller models and lower-bit quantizations are usually more practical when response speed matters. CPU inference uses system RAM. Having enough memory to fit a large model does not mean it will generate quickly; use the speed estimates when choosing.

​
Memory and speed trade-offs

  • More memory lets you consider larger models, higher-fidelity quantizations, or larger contexts.
  • GPU memory matters on dedicated-GPU systems. A machine with 64 GB of system RAM and an 8 GB GPU does not have 72 GB of interchangeable GPU memory.
  • Longer context increases memory demand and can slow responses.
  • Other applications reduce available memory and can affect speed after assessment.
For a faster starting point, move Discover’s slider toward Faster. For a model that needs less memory, compare the Memory values and quantized variants. See Models.