magnitude

Open source inference engine for agents that optimizes itself for your exact hardware. Compiles and tunes its kernels on your device, so open models run up to 2x faster than llama.cpp. Works on Apple Silicon, NVIDIA, AMD, or just a CPU.

View on GitHub

Stars: 6,115
Forks: 408
+ today: +16
Created: 3mo

Ranking data as of October 1, 2026 (UTC).

Overview

The project is written in Rust and has 408 forks. It was created 3mo ago.

AI-generated summary of the repository README — may contain errors.

What it is

magnitudedev/magnitude is an open-source inference engine for running models used by agents. The project describes it as hardware-aware: it compiles and tunes kernels on the device, and says open models can run up to 2× faster than llama.cpp. Its README summary reports decode improvements of 92% on Metal and 19% on CUDA.

Capabilities

The project lists support for Apple Silicon, NVIDIA and AMD hardware, as well as CPUs. Its README preview describes downloading the app, choosing a recommended model in Discover, and connecting an agent through Connections. The project is written in Rust.

Getting started

The README preview says Magnitude is available for macOS, Windows and Linux. It directs users to download and install the app, open it, select a model in Discover, then connect an agent in Connections. For complete setup details, consult the repository README.

Installation

git clone https://github.com/magnitudedev/magnitude.git
cd magnitude
# see README for setup

Frequently Asked Questions

Which platforms are listed?

The README preview lists macOS, Windows and Linux.

What hardware does Magnitude describe supporting?

The project description lists Apple Silicon, NVIDIA, AMD and CPU hardware.

How does the README preview describe getting started?

Download and install the app, open it, choose a recommended model in Discover, then connect an agent in Connections. See the repository README for full setup details.